http_server.anubis 77.1 KB

Edit Raw Blame History


                                  The Anubis Project.
                                  A HTTP/HTTPS Server

                           Copyright (c) Alain Proute' 2003.
                                 All rights reserved.


   In this  file a  HTTP/HTTPS server  is defined. It  answers HTTP/HTTPS  requests, sends
   files  (images or  any other  kind of  file), constructs  HTML pages  on the  fly using
   informations received from the client (when  the URI ends by '.awp'), handles uploading
   of files, redirections and virtual hosts.  It is multitasking by itself, and can handle
   any number of clients simultaneously.


   * * * The '.awp' extension.

   '.awp'  (for 'Anubis Web  Page') is  a virtual  extension.  No  file actually  has this
   extension.  However,  when the  server receives a  request whose URI  (Uniform Resource
   Identifier) ends by '.awp',  it understands that, instead of sending a  file, it has to
   construct an HTML page on the fly. From this point of view, '.awp' is similar to '.php'
   or '.asp' in some other systems.

   However, your client never has to write this extension in his browser. Indeed, assuming
   that the Internet  domain name of your site is  'www.our-business.com', the client just
   has to enter this:

                               http://www.our-business.com

   in  the  address field  of  his  browser.  Because  you  will  have  set the  following
   redirection (see below for details):

                 redirect("/", "www.our-business.com", "/homepage.awp")

   the  server  will  understand  that   '/homepage.awp'  is  actually  requested.   As  a
   consequence, since the  extension is '.awp', the server will construct  an HTML page on
   the fly.

   Now, if your client clicks on one of  your hypertext links, he may again be directed to
   an '.awp' URI, simply because, you will have put the extension '.awp' explicitly in the
   link.

   All other extensions  ('.jpg', '.gif', '.pdf', etc...) are  treated as file extensions,
   and the files (if found) are sent 'as is' to the client. However, for security reasons,
   the file is sent only if the following conditions are satisfied:

     1. the file is  (on the server's disk)  within the subtree whose root  is the 'public
        directory' (see below).

     2. the MIME type associated to the extension can be recognized. This is the case only
        if this type is recorded in 'web/mime.anubis' (see and customize this file).


   * * * HTTP headers, web arguments and web pages.

   Each HTTP  request which arrives on  the server contains  a request line followed  by a
   series of HTTP headers. Each HTTP header  is a pair '(name,value)' assigning a value to
   a name. The type 'HTTP_header' is defined in 'web/common.anubis'.

   The  request may  also have  a 'body'.   The body  contains either  'web  arguments' or
   uploaded files  (or both).   The request  line itself may  also contain  web arguments.
   Like HTTP headers, 'web arguments' are pairs '(name,value)', but the difference is that
   these  pairs are  generated by  the page  within which  the client  clicks,  while HTTP
   headers  are  generated by  the  browser  itself.  The  type  'Web_arg'  is defined  in
   'web/common.anubis'.  It  has two alternatives,  one for ordinary web  arguments (pairs)
   and one for uploaded files.

   The  type 'Web_page'  describes pages  as  they may  appear in  the client's  browser's
   window. It is defined in 'web/html.anubis'.

read web/common.anubis
read web/html.anubis


   * * * Redirections and virtual hosts.

   The server can handle  virtual host, in other words, you may  have several sites on the
   same server, with  the same IP address  and same port number. This  is achieved through
   redirections,  using the  'Host'  HTTP header.  The  type 'Redirection'  is defined  in
   'web/common.anubis'.

   When you  start your server, you  provide a list  of redirections. You should  at least
   put something like:

                 redirect("/", "www.our-business.com", "/homepage.awp")

   in order to  have a convenient entry point  to your site. However, you  may also handle
   several sites with the same server, putting the following in your list:

                 redirect("/","www.our-business-1.com","/homepage_1.awp"),
                 redirect("/","www.our-business-2.com","/homepage_2.awp"),
                 redirect("/","www.our-business-3.com","/homepage_3.awp"),

   and you will certainly imagine other applications of this principle.


   * * * HTTP/HTTPS server description.

   Your HTTP server itself is described by the following type.

public type HTTP_ServerDescription:
   http_server_description(Int32                 ip_address,
                           Int32                 ip_port,
                           List(Redirection)     redirections,
                           String                server_directory,
                           List(String)          journal_extensions,
                           List(String)          journal_headers,
       (Int32,
        String,
        List(HTTP_header),
        List(Web_arg),
        Server)        -> (String,
                           String,
                           Web_page)             awp_handler,
                           (One) -> One          service).


   For an HTTPS server, there are some more components.

public type HTTPS_ServerDescription:
   https_server_description(Int32                 ip_address,
                            Int32                 ip_port,
                            String                server_common_name,
                            String                authorization_secret,
                            List(Redirection)     redirections,
                            String                server_directory,
                            List(String)          journal_extensions,
                            List(String)          journal_headers,
       (Int32   ip_address,
        String  uri,
        List(HTTP_header),
        List(Web_arg),
        Server)        -> (String,
                           String,
                           Web_page)              awp_handler,
                           (One) -> One           service).


   Explanations of the components of these types:


   'ip_address' is the TCP/IP address on which  the server will listen.  If you put 0, the
   server will listen  on all available TCP/IP interfaces. Otherwise, you  may want to use
   the  function 'ip_address' defined  in 'tools/basis.anubis',  for constructing  your IP
   address from  a quadruplet of  Word8. For example,  if you want  to start a  server only
   reachable from your local area network, put the local address of the server.


   'ip_port' is the port on which the server  listens. Normally, it is 80 for HTTP and 443
   for HTTPS, but any other value works.


   'server_common_name'  is  the  name of  the  server  as  declared  on your  SSL  server
   certificate. It is something like 'www.our-business.com', not a numerical address.


   'authorization_secret' is any string which should  be impossible to guess, and that you
   keep secret. It is used to forge authorizations for private file download.  You must be
   very careful that this  string (which should be long enough, and  choosen with the same
   care as  for a password) remains absolutely  secret.  Since you don't  have to remember
   it, you may just use at least  20 characters typed at random on the keyboard, including
   also decimal digits.  Do  not hesitate to choose a new string if  you have any doubt on
   its secrecy. Of course,  this string is not sent to the  outside, but used to construct
   SHA1 hashes.


   'redirections' is the list of redirections, already explained above.


   'server_directory'  is the  directory where  the server  finds and  puts its  data. The
   server creates the following subdirectories within the server directory:

      public/               containing the public files
      journal/              containing the journal files
      private_download/     containing temporary files to be privately downloaded
      upload_temporary/     containing temporary uploaded files

   You put  your public files  directly in  'public/'. The server  will never send  a file
   which  is  not  in  the  subtree  whose root  is  'public/'.   However,  the  directory
   'private_download/' may also contain files which are sent to clients, but they are sent
   only if the client is authorized to download the file. See the details below.

   The directory  'journal/' contains the journal  files constructed by  the server. Their
   names are of the form:

        yyyy_mm_dd_hh

   where 'yyyy' is the year, 'mm' the month,  'dd' the day, and 'hh' the hour to which the
   journal refers.  The next  components allow to  decide what  will actually go  into the
   journal files (and on the console of the server).

   The  directory 'upload_temporary/'  contains files  which have  just  been uploaded,but
   under  a  temporary  file name.  See  the  type  'Web_arg' in  'web/common.anubis'  for
   explanations.


   'journal_extensions' is the list of the extensions  of the URI you want to store in the
   journal. A possible choice is:

     [ ".awp", ".gif", ".jpg", ".png" ]

   At least, you should put '.awp'.  Whenever the  URI in the request ends by one of these
   extensions the request line  is stored in the journal (and shown  on the console of the
   server).


   'journal_headers' is  the list of HTTP  headers you want  to store in the  journal (and
   show on  the console of the  server) for requests which  are stored in  the journal.  A
   possible choice is:

      [ "user-agent", "host" ]


   'awp_handler' is the function which handles  the requests (only those whose URI ends by
   '.awp', because other requests are handled just by sending a file). Its type is:

        (Int32,String,List(HTTP_header),List(Web_arg)) -> (String,String,Web_page)

   So, it takes 4 arguments which are:

      Int32                ip_address,    // IP address of client
      String               uri,           // URI requested by the client
      List(HTTP_header)    headers,       // list of HTTP headers sent by the client
      List(Web_arg)        web_args,      // list of web arguments sent by the client

   and returns  a triplet of type  (String,String,Web_page). This function  is called each
   time a  new '.awp'  request arrives.   It receives the  IP address  of the  client, the
   (redirected) URI requested by the client, the  list of HTTP headers sent by the client,
   and the  list of  web arguments sent  by the  client. From these  data the  function is
   supposed to  construct the answer  to be sent  to the client.   This answer is  of type
   'Web_page'  (see 'web/html.anubis').  But  is also  returns two  strings which  are the
   so-called 'c_ticket' and 's_ticket'. If  you use 'web/kernel.anubis', the values of the
   two  tickets are produced  by the  function 'tickets_and_web_page'  to be  found there.
   Otherwise, just put empty strings "" for the two tickets.


   'service'  is a function  of type  (One) ->  One which  is executed  every second  in a
   separate virtual machine. This function may perform any repetitive task.


   * * * Starting your HTTP/HTTPS servers.

   Well, when your HTTP server descritions are ready, you can start your servers with:

public define One
   start_http_server
     (
       HTTP_ServerDescription description
     ).

public define One
   start_https_server
     (
       HTTPS_ServerDescription description
     ).

   These fonctions create  the subdirectories of the server directory  (but not the server
   directory itself, which must have been  created by hand), starts the server (in another
   virtual machine), and return immediatly.


   * * * Private download.

   It may happen that you want to propose private files for download. This means that such
   a file could be downloaded only by the authorized person. This feature can be used only
   under HTTPS, not under HTTP.

   The server creates the  subdirectory 'private_download/' within the 'server directory'.
   This  directory and its  subdirectories contain  private files  to be  downloaded.  The
   server never sends a file from this directory tree without checking the autorization.

   When you construct a page for an  identified client, you can insert a 'link for private
   download', like this:

            with filename = "/clients/smith/secrets.pdf",
              link(filename+"?auth="+make_authorization(authorisation_secret,filename),
                text(filename))

   This will  enable the client to download  the file 'secrets.pdf'. Notice  that the file
   name '/clients/smith/secrets.pdf'  is relative to  the subdirectory 'private_download/'
   of the server directory.

   The  value  of  the  web  argument  'auth'  is a  cryptographical  hash  which  is  the
   authorisation itself.  After the client clicks on the private download link, the server
   verifies that the authorization is valid and corresponds to the file which is requested
   ('/clients/smith/secrets.pdf'  in our example,  relative to  'private_download/').  The
   function 'make_authorisation' just constructs the authorisation, it does not manipulate
   files.  It is your responsability to put the file to be downloaded at the right place.


   --- That's all for the public part ! --------------------------------------------------


                                 --- Table of Contents ---


   *** (1) Types which are private to this file.

   *** (2) Tools.
      *** (2.1) Formating an error message.
      *** (2.2) Converting IP addresses.
      *** (2.3) Reading and unputting characters.
      *** (2.4) Reading and discarding characters.
      *** (2.5) Reading a character string.
      *** (2.6) Padding integers with zeros.
      *** (2.7) Converting web arguments to ASCII.
      *** (2.8) Server description.

   *** (3) Managing the journal.
      *** (3.1) Naming journal files.
      *** (3.2) Formating HTTP headers.
      *** (3.3) Formating web arguments.
      *** (3.4) Formating the whole request.
      *** (3.5) Putting it in the journal file (and on the console).

   *** (4) Reading the HTTP request.
      *** (4.1) Skipping leading blanks.
      *** (4.2) Reading a new line.
      *** (4.3) Reading a 'word'.
      *** (4.4) Separating the URI from the query string.
      *** (4.5) Reading the web arguments.
      *** (4.7) Reading the request line.
      *** (4.8) Reading the HTTP headers.
      *** (4.9) Getting the size of the request's body.
      *** (4.10) Reading the body of the request.

   *** (5) Making the HTTP answer.
      *** (5.1) Avoiding illegal URIs.
      *** (5.2) Managing authorizations for downloading private files.
      *** (5.3) Recognizing MIME types.
      *** (5.4) Formating HTTP headers.
      *** (5.5) Sending a file.
      *** (5.6) Answering a www-url encoded request.
      *** (5.7) Answering a multipart/form-data encoded request.
         *** (5.7.1) Finding the boundary.
         *** (5.7.2) Reading attributes from a multipart entity.
         *** (5.7.3) Creating a temporary filename for an uploaded file.
         *** (5.7.4) Saving an uploaded file under a temporary filename.
         *** (5.7.5) Removing the path from a file name.
         *** (5.7.6) Reading a multipart entity.
      *** (5.8) Handling redirections.
      *** (5.9) Answering both sorts of requests.

   *** (6) The HTTP/HTTPS server.
      *** (6.1) The HTTP request handler.
      *** (6.2) Starting the HTTP/HTTPS server.


read tools/basis.anubis
read tools/findstring.anubis
read tools/connections.anubis
read web/html.anubis
read web/mime.anubis


   *** (1) Types which are private to this file.

   We use the following self-explanatory types.

type Error:
   cannot_read_from_connection,
   not_get_or_post_request(String),
   end_of_line_expected,
   incorrect_content_length_value,
   colon_expected.

type HTTP_RequestType:
   get,
   post.

type HTTP_RequestLine:
   request_line (HTTP_RequestType type,
                 String uri,
                 List(Web_arg) query_string).

type EncodingType:
   www_url,
   multipart_form_data.


   *** (2) Tools.


      *** (2.1) Formating an error message.

   The next function formats an error message.

define String
   format
     (
       Error msg
     ) =
   if msg is
     {
       cannot_read_from_connection then
         "Cannot read from connection.",
       not_get_or_post_request(s) then
         "The request did not begin by 'GET' or 'POST': "+s+".",
       end_of_line_expected then
         "End of line expected.\n",
       incorrect_content_length_value then
         "Incorrect value for HTTP header 'Content-Length'.",
       colon_expected then
         "':' was expected.",
     }.


      *** (2.2) Converting IP addresses.

   We need two conversion functions for IP addresses:

      (Word8,Word8,Word8,Word8)   -->   Int32        ip_address
      Int32                   -->   String       ip_addr_to_string

   These conversions are defined in 'anubis/library/tools/basis.anubis'.


      *** (2.3) Reading and unputting characters.

   We need  a mecanism for  unputting several characters  (actually at least 3).   This is
   because  when  reading  the client  connection,  we  must  sometimes go  ahead  several
   characters,  and virtually  put them  back into  the connection,  so that  they  can be
   reread. Of  course, we do not send  them back to the  client.  We store them  in a list
   (hold by the  variable 'unput_chars'), and we manage this list,  so that characters may
   be virtually put back in the connection (this is called 'unputting').

variable List(Word8) unput_chars   = [].

   The most recently read one is the  head of list.  Fortunately, this variable is private
   to this virtual machine (hence to this client).


define One
   unput          // unputting a character (add it in front of the list)
     (
       Word8 character
     ) =
   unput_chars <- (List(Word8))[character . *unput_chars].


define Maybe(Word8)
   read_one_byte
     (
       Connection connection
     ) =
   if read(connection,1,60) is
     {
       error    then failure,
       timeout  then failure,
       ok(ba)   then nth(0,ba)
     }.


define Maybe(Word8)
   next_char        // reading a character (check the list first, and read on the connection
                    //   only when the list is empty).
     (
       Connection connection
     ) =
   if *unput_chars is
     {
       [ ] then read_one_byte(connection),

       [h . t] then
         unput_chars <- t;
         success(h)
     }.


      *** (2.4) Reading and discarding characters.

   The  next  function  reads  the  specified  number  of  bytes  (this  is  the  same  as
   'characters') from the connection and discards  them. This is used for discarding CR LF
   just before the body of a request.

define Result(Error,One)
   read_and_ignore
     (
       Connection connection,        // to client
       Int32 number_of_characters    // number of characters to read and ignore
     ) =
   if number_of_characters =< 0 then ok(unique) else
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then read_and_ignore(connection,number_of_characters-1)
     }.


      *** (2.5) Reading a character string.

   Sometimes values  of HTTP attributes or  web args are  presented in the form  of double
   quoted strings.  The next  function handles  the reading of  such things.   The leading
   double quote is already read in.  We must read subsequent characters until the next non
   backslashed double quote.

define Result(Error,String)
   read_string
     (
       Connection connection,       // connection with the client
       List(Word8) so_far              // characters read so far (in reverse order)
     ) =
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if c = '\\'
         then if next_char(connection) is
           {
             failure then error(cannot_read_from_connection),
             success(d) then
               if d = '\"'
               then read_string(connection,['\"' . so_far])
               else read_string(connection,[d, c . so_far])
           }
         else if c = '\"'
              then ok(implode(reverse(so_far)))
              else read_string(connection,[c . so_far])
     }.


      *** (2.6) Padding integers with zeros.

   'zero_pad_2' transforms  an integer (which is  assumed to be  between 0 and 99)  into a
   string with  exactly two digits.  This is used  for formating days, hours,  minutes and
   seconds.

define String
   zero_pad_2
     (
       Int32 n
     ) =
   with s = integer_to_string(n),
   if length(s) < 2
   then "0"+s
   else s.


      *** (2.7) Converting web arguments to ASCII.

   The function 'web_to_ascii' gets a character string and replaces web encoding by normal
   ASCII encoding. This amounts to replacing:

      +     by        blank
      %xx   by        the character whose ASCII code is xx in hexadecimal

   Note: We assume that '9' < 'A' (which is the case for ASCII code).


define Word8
   web_decode
     (
       Word8 x1,
       Word8 x2
     ) =
   with z1 = word8_to_int32(x1),
        n1 = if z1 =< '9' then (z1 - '0') else (z1 - 'A' + 10),
        z2 = word8_to_int32(x2),
        n2 = if z2 =< '9' then (z2 - '0') else (z2 - 'A' + 10),
         n = (n1 << 4) + n2,
     if n =< 127
     then truncate_to_word8(n)
     else if n = 224 then 'à' else
          if n = 226 then 'â' else
          if n = 231 then 'ç' else
          if n = 232 then 'è' else
          if n = 233 then 'é' else
          if n = 234 then 'ê' else
          if n = 235 then 'ë' else
          if n = 238 then 'î' else
          if n = 239 then 'ï' else
          if n = 244 then 'ô' else
          if n = 246 then 'ö' else
          if n = 249 then 'ù' else
          if n = 251 then 'û' else
          if n = 252 then 'ü' else
          truncate_to_word8(n).

define String
   web_to_ascii
     (
       String web_string,
       Int32 n,                       // current position in web_string
       List(Word8) so_far
     ) =
   if nth(n,web_string) is
     {
       failure then implode(reverse(so_far)),
       success(c) then
         if c = '+'
         then web_to_ascii(web_string,n+1,[' ' . so_far])
         else if c = '%'
              then if nth(n+1,web_string) is
                {
                  failure then implode(reverse(so_far)),
                  success(x1) then if nth(n+2,web_string) is
                    {
                      failure then implode(reverse(so_far)),
                      success(x2) then web_to_ascii(web_string,n+3,[web_decode(x1,x2) . so_far])
                    }
                }
              else web_to_ascii(web_string,n+1,[c . so_far])
     }.


      *** (2.8) Server description.

   We need a server description common for HTTP and HTTPS servers.

type ServerDescription:
   server_description(Int32                 ip_address,
                      Int32                 ip_port,
                      Maybe(String)         server_common_name,
                      Maybe(String)         authorization_secret,
                      List(Redirection)     redirections,
                      String                server_directory,
                      List(String)          journal_extensions,
                      List(String)          journal_headers,
       (Int32,
        String,
        List(HTTP_header),
        List(Web_arg),
        Server)        -> (String,
                           String,
                           Web_page)             awp_handler).


   We have  to construct this 'internal' server  description from the one  provided by the
   user of this program.

define ServerDescription
   internal_description
     (
       HTTP_ServerDescription d
     ) =
   if d is http_server_description(ip,port,redir,dir,exts,headers,handler,service) then
   server_description(ip,port,failure,failure,redir,dir,exts,headers,handler).

define ServerDescription
   internal_description
     (
       HTTPS_ServerDescription d
     ) =
   if d is https_server_description(ip,port,cn,as,redir,dir,exts,headers,handler,service) then
   server_description(ip,port,success(cn),success(as),redir,dir,exts,headers,handler).


   *** (3) Managing the journal.

   Concurrently  working machines  should not  try to  access the  same file  at  the same
   time. This problem may be solved by using the 'protect' mecanism.


      *** (3.1) Naming journal files.

   Since  journal messages  are rather  prolific, we  should have  at least  one  file per
   hour. Hence,  the name of  a journal  file must be  constructed from the  current year,
   month, day and hour. For example, it may be:

       2003_03_12_19

   (this is for the journal of 7 PM to 8 PM, 2003/mar/12).

define String
   make_current_journal_file_name
     =
   if convert_time(now) is date_and_time(y,m,d,h,_,_,_,_,_) then
   integer_to_string(y)+"_"+
   zero_pad_2(m)+"_"+
   zero_pad_2(d)+"_"+
   zero_pad_2(h).


      *** (3.2) Formating HTTP headers.

   HTTP headers may be shown on the  console or written in the journal. The function below
   formats a list of HTTP headers.

define String
   show_format
     (
       ServerDescription desc,
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then "",
       [h . t] then if h is http_header(name,value) then
         if member(journal_headers(desc),name)
         then "  | "+name+": "+value+"\n"+show_format(desc,t)
         else show_format(desc,t)
     }.


      *** (3.3) Formating web arguments.

   The same thing for web arguments.

define String
   show_format
     (
       List(Web_arg) lwa
     ) =
   if lwa is
     {
       [ ] then "",
       [h . t] then if h is
         {
           web_arg(n,v) then
             "  | "+n+"="+v+"\n"+show_format(t),
           upload(n,fn,tfn) then
             "  | "+n+"="+fn+" (uploaded as '"+tfn+"')\n"+show_format(t)
         }
     }.


      *** (3.4) Formating the whole request.

   It is cheap to transform month  numbers into abbreviated month names. This enhances the
   readability of the journal.

define String
   format_month
     (
       Int32 m
     ) =
   if m = 1  then "jan" else
   if m = 2  then "feb" else
   if m = 3  then "mar" else
   if m = 4  then "apr" else
   if m = 5  then "may" else
   if m = 6  then "jun" else
   if m = 7  then "jul" else
   if m = 8  then "aug" else
   if m = 9  then "sep" else
   if m = 10 then "oct" else
   if m = 11 then "nov" else
   if m = 12 then "dec" else
   "???".


   Below we format a whole HTTP request.   This may give this (actually, it depends on how
   you defined the values of 'journal_headers' and 'journal_extensions'):

   [3] 2003/mar/10 10:06:57 from 123.456.123.456: /homepage.awp
     | host: www.the-best-one.com
     | user-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0

   The leading number  between brackets is the number of the  virtual machine which served
   the URI.

define String
   format_request
     (
       ServerDescription desc,
       Connection client_connection,
       HTTP_RequestLine request_line,
       List(HTTP_header) headers,
       List(Web_arg) web_args
     ) =
   with dt = convert_time(now),
   if remote_IP_address_and_port(client_connection) is (addr,port) then
        integer_to_string(year(dt))+"/"+format_month(month(dt))+"/"+zero_pad_2(day(dt))+" "+
        zero_pad_2(hour(dt))+":"+zero_pad_2(minute(dt))+":"+zero_pad_2(second(dt))+
        " from "+ip_addr_to_string(addr)+
        ": "+uri(request_line)+"\n"+
        show_format(desc,headers)+
        show_format(web_args).


      *** (3.5) Putting it in the journal file (and on the console).

   We must not  forget to 'protect' this  operation, so that the messages  of two machines
   will not be mixed together.

define One
   log_journal_msg
     (
       ServerDescription desc,
       String msg,
     ) =
   with msg = to_byte_array("["+virtual_machine_id+"] "+msg+"\n"),
   protect
     (
       if file(server_directory(desc)+"/journal/"+make_current_journal_file_name,append) is
         {
           failure then unique,
           success(journal_file) then
             forget(reliable_write(file(journal_file),msg))
         };
       forget(reliable_write(file(stdout),msg))
     ).


   *** (4) Reading the HTTP request.


      *** (4.1) Skipping leading blanks.

   One of  the peculiarities of HTTP  is that the  characters 13 (carriage return)  and 10
   (line feed) followed by either a space (32)  or a tab (9), is considered as a blank not
   containing any new  line. 'skip_http_blanks' must skip all  blanks characters until the
   first non blank character, which should not be read in. Obviously, because of the above
   peculiarity, we need at least 3 characters of lookahead to do this.  In other words, we
   must be able to unput at least 3 characters (hopefully we are).

   Strictly blanks characters are 'space' and 'tab'.

define Bool
   is_strict_blank
     (
       Word8 c
     ) =
   if c = ' ' then true else c = '\t'.


   On the contrary, blanks include 13 and 10.

define Bool
   is_blank
     (
       Word8 c
     ) =
   if c = ' '    then true else
   if c = '\t'   then true else
   if c = 13     then true else
   c = 10.


   Skipping HTTP blanks.

define One
   skip_http_blanks
     (
       Connection connection
     ) =
   if next_char(connection) is
     {
       failure then unique,
       success(c) then
         if is_strict_blank(c)
         then skip_http_blanks(connection)
         else if c = 13
              then if next_char(connection) is
                {
                failure then (unput(c); unique),
                success(d) then
                  if d = 10
                  then if next_char(connection) is
                    {
                    failure then (unput(d); unput(c); unique),
                    success(e) then
                      if is_strict_blank(e)
                      then skip_http_blanks(connection)
                      else (unput(e); unput(d); unput(c); unique)
                    }
                  else (unput(d); unput(c); unique)
                }
              else (unput(c); unique)
     }.


      *** (4.2) Reading a new line.

   Normally in  HTTP a new  line is the  sequence 13 10  (carriage return line  feed), not
   followed by a space or tabulator. If it  is followed by a space or tabulator, the three
   characters are considered blanks, and no new  line has been read. Before trying to read
   a new line, we first skip leading spaces and  tabs.  Then we try to read 13 and 10, and
   we read another character. if this character  is space or tab, we consider we have read
   only blanks and we continue reading in order to find our new line.  Otherwise, we unput
   this character (which  may be for example the  first character of the name  of the next
   header), and answer that we have seen a new line.

   Warning: we  must not use this  function for reading  the last pair (13,10)  before the
   beginning of  the body, because  if the body  is empty, there  is no character  to read
   after  this pair,  so that  the server  could  wait for  a character  which will  never
   come.  This is  the reason  for 'read_and_ignore'  above, which  is used  precisely for
   reading that last (13,10) pair.

define Result(Error,One)
   read_new_line
     (
       Connection connection
     ) =
   skip_http_blanks(connection);
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if c = 13
         then if next_char(connection) is
           {
             failure then error(cannot_read_from_connection),
             success(d) then
               if d = 10
               then ok(unique)
               else (unput(d);
                     unput(c);
                     error(end_of_line_expected))
           }
         else (unput(c);
               error(end_of_line_expected))
     }.


      *** (4.3) Reading a 'word'.

   A 'word' is a sequence of characters which  begins either by a double quote or not by a
   double  quote.   (However,  any leading  blanks  are  read  in  and ignored.   This  is
   accomplished by 'skip_http_blanks'.) If it begins by  a double quote, it is read like a
   string, i.e.   it ends at  the next (non  backslashed) double quote.  Otherwise,  it is
   right delimited  by any character which  may be considered  as 'blank'. If the  word is
   double quoted, the closing double quote is read in. On the contrary, if the word is not
   double quoted, the right delimiting blank character  is not read in (it is 'unput' back
   into the connection), and may be read in again.  This is needed because carriage return
   or line feed which are 'blank', also have a meaning in HTTP.

define Result(Error,String)
   read_word_aux
     (
       Connection connection,
       List(Word8) so_far
     ) =
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if is_blank(c)
         then (unput(c);
               ok(implode(reverse(so_far))))
         else read_word_aux(connection,[c . so_far])
     }.

define Result(Error,String)
   read_word
     (
       Connection connection
     ) =
   skip_http_blanks(connection);
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if c = '\"'
         then read_string(connection,[])
         else read_word_aux(connection,[c])
     }.


      *** (4.4) Separating the URI from the query string.

   A 'query string' may be postfixed to  the URI, just after a question mark. For example,
   the client may send the following request:

      GET /catalog.awp?item=3&color=blue

   We separate this into an  URI: "/catalog.awp" and the string: "item=3&color=blue" which
   will be later transformed into the list:

          [web_arg("item","3"),web_arg("color","blue")]


define (String,String)
   separate_uri_from_query_string
     (
       String uri_and_query_string,
       Int32 n
     ) =
   if nth(n,uri_and_query_string) is
     {
       failure then (uri_and_query_string,""),
       success(c) then
         if c = '?'
         then (substr(uri_and_query_string,0,n),
               substr(uri_and_query_string,n+1,length(uri_and_query_string)-(n+1)))
         else separate_uri_from_query_string(uri_and_query_string,n+1)
     }.


      *** (4.5) Reading the web arguments.

   HTTP requests are sent in one of two formats:

     (1) www-url encoded
     (2) multipart/form-data encoded

   The first one  is the normal (historical)  way of encoding. The second  one is required
   for uploading files. A  server which is supposed to accept upload  of files must handle
   both formats.  The  first thing to do is  to decide the format of the  request. This is
   easily done by examining the HTTP headers. If we find the header:

     Content-Type: multipart/form-data

   the request  is multipart/form-data  encoded.  Otherwise, it  is 'www-url'  encoded. We
   first consider 'www-url' encoded requests.

   For a 'www-url' encoded request, the web  argument are either in the query string or in
   the body of the request, or both. The format is the same for both:

     name=value&name=value&...

   However, we may also have

     name
     name=
     name=&...
     name&...

   i.e. some parts may be missing. Hence, we must be careful.

   Furthermore, web arguments must be translated from web to ASCII when www-url encoded.

define Bool
   is_ampersand_or_equal
      (
       Word8 c
     ) =
   if c = '&' then true else c = '='.


   The function 'read_name_or_value'  reads the string 's' starting  at position 'n' until
   either the end of the string or the first '&' or '='.

define String
   read_name_or_value
     (
       String s,
       Int32 start,
       Int32 i
     ) =
   if nth(i,s) is
     {
       failure then substr(s,start,i - start),
       success(c) then
         if is_ampersand_or_equal(c)
         then substr(s,start,i-start)      // the separator is not included
         else read_name_or_value(s,start,i+1)
     }.


define List(Web_arg)
   read_www_url_encoded_web_args
     (
       String s,
       Int32 start,
     ) =
   with first = read_name_or_value(s,start,start),
     if first = ""
     then []
     else with i = start+length(first),
       if nth(i,s) is
         {
           failure then [web_arg(first,"")],
           success(c) then
             if c = '&'
             then [web_arg(first,"") . read_www_url_encoded_web_args(s,i+1)]
             else if c = '='
                  then with second1 = read_name_or_value(s,i+1,i+1),
                       // print("\""+second1+"\"\n");
                       with second = web_to_ascii(second1,0,[]),
                       [web_arg(first,second) . read_www_url_encoded_web_args(s,i+length(second1)+2)]
                  else alert
         }.


      *** (4.7) Reading the request line.

   'read_request_line' reads three  words and a new line from the  connection. It tries to
   recognize "GET" or "POST" in the first word, separates the URI from the query string in
   the second  word, transforms  the query string  into a  list of 'Web_arg',  and finally
   returns a datum of type 'HTTP_RequestLine' if no error arose.


define Result(Error,HTTP_RequestType)
   identify_get_or_post
     (
       String s
     ) =
   with s = to_lower(s),
   if s = "get"      then ok(get)    else
   if s = "post"     then ok(post)   else
   error(not_get_or_post_request(s)).

define Result(Error,HTTP_RequestLine)
   read_request_line
     (
       Connection connection
     ) =
   if read_word(connection) is
     {
     error(msg) then error(msg),
     ok(get_or_post) then if read_word(connection) is
       {
       error(msg) then error(msg),
       ok(uri_and_query_string) then if read_word(connection) is
         {
         error(msg) then error(msg),
         ok(http_version) then if read_new_line(connection) is
           {
           error(msg) then error(msg),
           ok(_) then if separate_uri_from_query_string(uri_and_query_string,0) is
             (uri,query_string) then if identify_get_or_post(get_or_post) is
             {
             error(msg) then error(msg),
             ok(request_type) then
               ok(request_line(request_type,uri,read_www_url_encoded_web_args(query_string,0)))
             }
           }
         }
       }
     }.


      *** (4.8) Reading the HTTP headers.

   Each header is made of a name  (containing only letters, the underscore, digits and the
   minus sign), a colon, a value, and a new line. The first empty line ends the headers.


   The next function tests characters acceptable in a header name.

define Bool
   is_header_name_char
     (
       Word8 c
     ) =
   with n = word8_to_int32(c),
   if ('a' =< n & n =< 'z') then true else
   if ('A' =< n & n =< 'Z') then true else
   if ('0' =< n & n =< '9') then true else
   if c = '-'               then true else
   c = '_'.

define Result(Error,String)
   read_header_name
     (
       Connection connection,
       List(Word8) so_far
     ) =
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if is_header_name_char(c)
         then read_header_name(connection,[to_lower(c) . so_far])
         else unput(c); ok(implode(reverse(so_far)))
     }.

define Result(Error,One)
   skip_colon
     (
       Connection connection
     ) =
   skip_http_blanks(connection);
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if c = ':'
         then ok(unique)
         else error(colon_expected)
     }.


define Result(Error,String)
   read_header_value
     (
       Connection connection,
       List(Word8) so_far
     ) =
   if next_char(connection) is
     {
       failure then error(cannot_read_from_connection),
       success(c) then
         if c = 13
         then if next_char(connection) is
           {
             failure then error(cannot_read_from_connection),
             success(d) then
               if d = 10
               then if next_char(connection) is
                 {
                   failure then error(cannot_read_from_connection),
                   success(e) then
                     if is_strict_blank(e)
                     then read_header_value(connection,[e . so_far])
                     else (unput(e); ok(implode(reverse(so_far))))
                 }
               else read_header_value(connection,[d, c . so_far])
           }
         else read_header_value(connection,[c . so_far])
     }.


   Reading a single header.

define Result(Error,Maybe(HTTP_header))
   read_header
     (
       Connection connection
     ) =
   if read_header_name(connection,[]) is
     {
       error(msg) then error(msg),
       ok(name) then
         if name = "" then
           if read_and_ignore(connection,2) /* 13 and 10 */ is
           {
             error(msg) then error(msg),
             ok(_) then     // this is the blank line
               ok(failure)  // end of headers
           }
         else if skip_colon(connection) is
             {
               error(msg) then error(msg),
               ok(_) then skip_http_blanks(connection);
                if read_header_value(connection,[]) is
                {
                  error(msg) then error(msg),
                  ok(value) then
                    ok(success(http_header(name,value)))
                }
             }
     }.


   Reading all the headers.

define Result(Error,List(HTTP_header))
   read_http_headers
     (
       Connection connection,
     ) =
   if read_header(connection) is
     {
       error(msg) then error(msg),
       ok(mbh) then if mbh is
         {
           failure then ok([ ]),
           success(header) then
             if read_http_headers(connection) is
               {
                 error(msg) then error(msg),
                 ok(others) then ok([header . others])
               }
         }
     }.


      *** (4.9) Getting the size of the request's body.

   The size of the body of the request is given under the 'Content-Length' header. If this
   header is not present, the size is assumed to be zero.

define Result(Error,Int32)
   get_body_size
     (
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then ok(0),
       [h . t] then if h is http_header(name,value) then
         if name = "content-length"
         then if string_to_integer(value) is
           {
             failure then error(incorrect_content_length_value),
             success(n) then ok(n)
           }
         else get_body_size(t)
     }.


      *** (4.10) Reading the body of the request.

   The body of the  request may be very big (it contains uploaded  files, if any). We read
   it using  the primitive 'read', which  returns the number  of bytes read, which  may be
   less than the number  of bytes we wanted to read. This is not  an error, but simply due
   to the  fact the  buffer associated with  the connection  in the Linux  (or MS-Windows)
   kernel has  a limited size.   Hence, we must  read bytes again  until we have  read the
   required number of bytes. However, if the  number of bytes read is zero, the connection
   may be broken. In that case, we must  not try to read indefinitely. On the contrary, we
   make at most 10 retries, with a small sleeping time between any two of them.

define Result(Error,ByteArray)
   read_http_body
     (
       Connection connection,
       Int32 body_size,
       ByteArray so_far,          // when calling this function, 'so_far' is the empty byte array
       Int32 retries              // this function is called with retries = 10
     ) =
   if body_size = 0 then ok(constant_byte_array(0,0)) else
   if retries =< 0 then error(cannot_read_from_connection) else
   if read(connection,body_size,60) is
     {
       error          then error(cannot_read_from_connection),
       timeout        then error(cannot_read_from_connection),
       ok(new_bytes)  then with
        ba        = so_far + new_bytes,                     // contains all the bytes read so far
        nr        = length(ba),                             // total read since the beginning
        nn        = length(new_bytes),                      // number of bytes just read
     if nr < body_size                                      // must read more bytes
     then if nn > 0                                         // if connection seems to work
          then read_http_body(connection,body_size,ba,1000) // continue reading
          else sleep(100);                                  // otherwise, sleep 1/10 of second
               read_http_body(connection,body_size,ba,      // and retry reading
                              retries-1)                    // but no more than 10 times
     else ok(ba)                                            // required number of bytes has been read
     }.


   Note: During sleeping,  'anbexec' runs other machines. Actually,  calling 'sleep', even
   for  one millisecond,  is  some way  of giving  up  explicitly, so  that other  virtual
   machines may work.


   *** (5) Making the HTTP answer.

   At  that  point we  have  read the  request  line,  the headers  and  the  body of  the
   request, and we must decide what to do.

   Actually, we can do one of the following:

     - send a file,
     - execute 'tickets_and_web_page' in case of an ".awp" URI.

   The uploaded file (which are in the body of the request) are saved into temporary files
   below.


      *** (5.1) Avoiding illegal URIs.

   For security reasons, we must avoid illegal  URIs, for example those which may climb up
   in the file hierarchy. First we accept only few characters in URIs.

define Bool
   is_legal_uri_char
     (
       Word8 c
     ) =
   with n = word8_to_int32(c),
   if ('a' =< n & n =< 'z') then true else        // accept 'a' to 'z'
   if ('A' =< n & n =< 'Z') then true else        // accept 'A' to 'Z'
   if ('0' =< n & n =< '9') then true else        // accept '0' to '9'
   if c = '.' then true else                      // accept '.' '-' '/' and '_'
   if c = '-' then true else
   if c = '/' then true else
   c = '_'.

   We do not accept ~ which is some  way of climbing. Of course, we cannot disallow single
   dots, which  are most often  present in legal  URIs, but we  must avoid double  dots ..
   which mean 'climb up'.

define Bool
   is_illegal_uri
     (
       String uri,
       Int32 n
     ) =
   if nth(n,uri) is
     {
       failure then false,
       success(c) then
         if c = '.'                               // first dot
         then if nth(n+1,uri) is
           {
             failure then false,
             success(d) then
               if d = '.'                         // second dot
               then true
               else is_illegal_uri(uri,n+1)
           }
         else is_illegal_uri(uri,n+1)
     }.


      *** (5.2) Managing authorizations for downloading private files.

   It may  be the case that  you provide to your  client a link for  downloading a private
   file, that is to  say a file which nobody else should be able  to download. Such a file
   should  not be  located  in the  public directory  tree,  but in  the private  download
   directory. This feature is available only under HTTPS for security reasons.

   It  works as  follows.  The  function 'make_authorization'  constructs  a cryptographic
   'authorization'. This authorization is  put in the query string of the  URI in the link
   for download.

   When the client subsequently clicks on the link, the server receives a 'GET' query with
   the path of the file as the URI, and the authorization as the value of the web argument
   'auth'. The server verifies that the autorisation  is valid for this file. If it is the
   case, it also  makes the current verifications (illegal URI, and  MIME type), and sends
   the file if everything is OK.


public define String
   make_authorization
     (
       String authorization_secret,
       String filename
     ) =
   to_ascii(sha1((authorization_secret,filename))).

   The function 'send_file' defined below handles the recognition of authorizations.


      *** (5.3) Recognizing MIME types.

   The extension of the (redirected) URI must be either ".awp" or recognized as associated
   to a MIME type. Otherwise, the server will not send the file. This is for security, but
   also because,  we must generate a 'Content-Type'  header in the answer,  with the right
   MIME type.

define String
   get_uri_extension_aux
     (
       String uri,
       Int32 n             // used for searching backwards
     ) =
   if nth(n,uri) is
     {
       failure then "",
       success(c) then
         if (c = '.' | c = '/')
         then substr(uri,n,length(uri)-n)
         else get_uri_extension_aux(uri,n-1)
     }.

define String
   get_uri_extension
     (
       String uri
     ) =
   get_uri_extension_aux(uri,
                         length(uri)-1).     // search starts at the right end


   For recognizing a MIME type we use 'known_mime_types' defined in 'web/mime.anubis'.

define Maybe(String)
   recognize_mime_type_from_ext
     (
       String ext,
       List(MIME) l
     ) =
   if l is
     {
       [ ] then failure,
       [h . t] then if h is mime(mime_type,extension) then
         if ext = extension
         then success(mime_type)
         else recognize_mime_type_from_ext(ext,t)
     }.

define Maybe(String)
   recognize_mime_type_from_uri
     (
       String uri
     ) =
   recognize_mime_type_from_ext(get_uri_extension(uri),known_mime_types).


      *** (5.4) Formating HTTP headers.

   This is the formating for sending to the client.

define Printable_tree
   format
     (
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then [crlf],
       [h . t] then if h is http_header(name,value) then
         [name,": ",value,crlf . format(t)]
     }.


      *** (5.5) Sending a file.

   We send 2 headers 'Content-Type' and 'Content-Length'.

define List(HTTP_header)
   headers_for_send_file
     (
       String mime_type,
       Int32 size,
     ) =
   [
     http_header("Content-Type",mime_type),
     http_header("Content-Length",integer_to_string(size)),
   ].


   Sending the body of the answer (i.e. the file itself).

define One
   send_file_body
     (
       ServerDescription desc,
       Connection connection,                     // connection with the client
       Connection file,                           // file to be sent already opened
       Int32 size,
       Int32 sent,
       String filename
     ) =
   if sent >= size then unique else
   if read(file,min(10000,size-sent),60) is
     {
       error    then log_journal_msg(desc,"Cannot read from file '"+filename+"'.\n"),
       timeout  then log_journal_msg(desc,"Cannot read from file timeout'"+filename+"'.\n"),
       ok(ba)   then
         with nr = length(ba),                          // get the number of bytes read
         if reliable_write(connection,ba) is
           {
             failure then log_journal_msg(desc,"Cannot write into connection.\n"),
             success(nw) then
               send_file_body(desc,connection,file,size,sent+nw,filename)
           }
     }.


   Sending the answer line, the headers and the body.

define One
   send_file
     (
       ServerDescription desc,
       Connection connection,
       List(HTTP_header) headers,
       Int32 size,
       Connection file,
       String filename
     ) =
   forget(reliable_write(connection,to_byte_array("HTTP/1.1 200 OK"+crlf)));
   forget(reliable_write(connection,format(headers)));
   send_file_body(desc,connection,file,size,0,filename).


   Checking if a connection is under SSL.

define Bool
   is_SSL
     (
       Connection c
     ) =
   if c is
     {
       file_r(_)   then false,
       file_w(_)   then false,
       file_rw(_)  then false,
       tcp(_)      then false,
       ssl(_)      then true
     }.


   Before opening and sending a file, we check the MIME type. It must be recognized.

define One
   send_file
     (
       ServerDescription desc,
       Connection connection,
       String uri,
       Maybe(String) mbauthorization
     ) =
   if recognize_mime_type_from_uri(uri) is
     {
       failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"),
       success(mime_type) then
         with directory = if mbauthorization is
           {
             failure                then server_directory(desc)+"/public",
             success(authorization) then
               log_journal_msg(desc,
                  "URI required for download: "+uri+" authorization: "+authorization+"\n");
               if make_authorization(if authorization_secret(desc) is
                                       {
                                         failure then "",
                                         success(s) then s
                                       },uri) = authorization
               then if is_SSL(connection)
                    then server_directory(desc)+"/private_download"
                    else log_journal_msg(desc,"Private download allowed only under SSL.\n");
                         server_directory(desc)+"/public"
               else log_journal_msg(desc,"Invalid authorization for download.\n");
                    server_directory(desc)+"/public"
           },
         if (Maybe(RStream))connect to file directory+uri is
           {
             failure then log_journal_msg(desc,"Cannot find file '"+directory+uri+"'.\n"),
             success(f) then with size = file_size(f),
               send_file(desc,
                         connection,
                         headers_for_send_file(mime_type,size),
                         size,
                         file(f),
                         uri)
           }
     }.


      *** (5.6) Answering a www-url encoded request.

   Standard headers are for answering ".awp" requests.

define List(HTTP_header)
   standard_headers
     (
       Int32 answer_body_size
     ) =
   [
     http_header("Content-Type","text/html"),
     http_header("Content-length",integer_to_string(answer_body_size))
   ].


define One
   www_url_answer
     (
       ServerDescription desc,
       Connection connection,          // connection with the client
       Int32 ip_addr,                  // IP address of the client
       HTTP_RequestLine request_line,  // request line sent by the client
       List(HTTP_header) headers,      // HTTP headers sent by the client
       ByteArray body,                 // body of client's request
       Server server
     ) =
   with all_web_args = query_string(request_line) +
                       read_www_url_encoded_web_args(to_string(body),0),
        uri = uri(request_line),
        ext = get_uri_extension(uri),
   (if member(journal_extensions(desc),ext)
    then log_journal_msg(desc,format_request(desc,connection,request_line,headers,all_web_args))
    else unique);
   if is_illegal_uri(uri,0)
   then log_journal_msg(desc,"Received illegal URI: "+uri+"\n")
   else (if ext = ".awp"
         then (if awp_handler(desc)
                     (ip_addr,uri,headers,all_web_args,server) is (c_ticket,s_ticket,wp) then
               forget(reliable_write(connection,
                      with answer_body = format(empty,c_ticket,s_ticket,wp),
                      [ "HTTP/1.1 200 OK", crlf,
                      format(standard_headers(length(answer_body))) .
                      answer_body])))
         else (send_file(desc,
                         connection,
                         uri,
                         if web_arg_value(all_web_args,"auth") is
                           {
                             not_found then failure,
                             found(v) then success(v)
                           }))).


      *** (5.7) Answering a multipart/form-data encoded request.

   In order to  support upload of files, we  must be able to read web  arguments which are
   encoded  in  a  multipart/form-data body.   The  first  thing  to  do  is to  find  the
   boundary. The  boundary is  a special string  which delimits  the various parts  of the
   'multipart' body.  It  is found within the value of the  'Content-Type' HTTP header, as
   the value of the 'boundary' attribute.


         *** (5.7.1) Finding the boundary.

   Hence,  we  just  have  to  find  the  string  'boundary='  within  the  value  of  the
   'Content-Type' header, and read the value of the boundary from there.

define Bool
   delimits_boundary
     (
       Word8 c
     ) =
   if c = ' '    then true else
   if c = 13     then true else
   if c = 10     then true else
   if c = 0      then true else
   if c = ','    then true else
   c = ';'.


define Maybe(String)
   get_boundary_value_3
     (
       String s,
       Int32 i,
       List(Word8) so_far
     ) =
   if nth(i,s) is
     {
       failure then success(implode(reverse(so_far))),
       success(c) then
         if delimits_boundary(c)
         then success(implode(reverse(so_far)))
         else get_boundary_value_3(s,i+1,[c . so_far])
     }.


define Maybe(String)
   get_boundary_value_2
     (
       String s,
       Int32 i,
     ) =
   if nth(i,s) is
     {
       failure then failure,
       success(c) then
         if is_blank(c)
         then get_boundary_value_2(s,i+1)
         else get_boundary_value_3(s,i+1,[c])
     }.

define Maybe(String)
   get_boundary_value_1
     (
       String s,           // string into which we must find '= ...'
       Int32 i             // position of start of search
     ) =
   if nth(i,s) is
     {
       failure then failure,
       success(c) then
         if is_blank(c)
         then get_boundary_value_1(s,i+1)
         else if c = '='
              then get_boundary_value_2(s,i+1)
              else failure
     }.


define Maybe(String)
   get_boundary
     (
       String content_type_header_value
     ) =
   if find("boundary",content_type_header_value,0) is
     {
       failure then failure,
       success(n) then // 'boundary' has been found at position n
         get_boundary_value_1(content_type_header_value,n+8)
     }.

define Maybe(String)
   get_boundary
     (
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then failure,
       [h . t] then if h is http_header(name,value) then
         if name = "content-type"
         then get_boundary(value)
         else get_boundary(t)
     }.


         *** (5.7.2) Reading attributes from a multipart entity.

   Entities in a multipart/form-data body are separated by instances of the string:

     --bbbbb

   where bbbbb is the boundary computed above. Actually, the body has the form:

   --bbbbb
   <entity 1>
   --bbbbb
   <entity 2>
   --bbbbb
   ...
   --bbbbb
   <last entity>
   --bbbbb


   We have  to extract an entity  which is in the  body between offsets  'start' and 'end'
   (computed when boundaries have been localized). The entity itself is made of two parts:
   headers and body.  The  body is separated from the headers by  a blank line. This blank
   line (a double crlf) marks the beginning of the body of the entity.  Within the headers
   of the entity, we look for a 'Content-Disposition' header, which should look like this:

      Content-Disposition: form-data; name="..."; filename="..." crlf

   We  are  just interested  in  the  name  and the  file  name.   Hence we  first  search
   'Content-Disposition', then we search 'name' and read the value, and we do the same for
   'filename'.

   If the 'filename' attribute is not present,  the web arg is an ordinary one, otherwise,
   it is an uploaded file.


   Below  is a  variant of  'find' (see  'tools/findstring.anubis'), with  an  extra 'end'
   argument.

define Maybe(Int32)
   find
     (
       String what,
       ByteArray where,
       Int32 start,
       Int32 end
     ) =
   if find(to_byte_array(what),where,start) is
     {
       failure then failure,
       success(n) then
         if n+length(what) >= end
         then failure
         else success(n)
     }.


define String
   read_attribute_value
     (
       ByteArray where,
       Int32 start,
       Int32 end,
       List(Word8) so_far
     ) =
   if start >= end then implode(reverse(so_far)) else
   if nth(start,where) is
     {
       failure then implode(reverse(so_far)),
       success(c) then
         if c = '\"'
         then implode(reverse(so_far))
         else read_attribute_value(where,start+1,end,[c . so_far])
     }.

define Maybe(String)
   find_attribute
     (
       String name,
       ByteArray where,
       Int32 start,
       Int32 end
     ) =
   with name = name+"=\"",
   if find(to_byte_array(name),where,start) is
     {
       failure then failure,
       success(n) then
         if n+length(name) >= end
         then failure
         else success(read_attribute_value(where,n+length(name),end,[]))
     }.


define Maybe((String,Maybe(String)))
   find_name_and_filename
     (
       ByteArray body,
       Int32 start,
       Int32 end
     ) =
   if find(to_byte_array("Content-Disposition"),body,start) is
     {
       failure then failure,
       success(n) then
         if find_attribute("name",body,n+19,end) is
         {
           failure then failure,
           success(name_value) then if find_attribute("filename",body,n+19,end) is
             {
               failure then success((name_value,failure)),
               success(filename_value) then success((name_value,success(filename_value)))
             }
         }
     }.


         *** (5.7.3) Creating a temporary filename for an uploaded file.

variable Int32 uploaded_file_count = 0.

   This variable is local  to the virtual machine.  Hence, its value is  0 each time a new
   requests arrives.  Temporary uploaded files  are stored in the directory represented by
   'upload_temporary_directory'. The filenames have the form:

      _m_n

   where  'm'  is the  number  of  the  virtual machine,  and  'n'  a number  obtained  by
   incrementing 'uploaded_file_count'. Notice that the program must do something with this
   file (move it  to some directory/name), otherwise, it will  probably be overwritten the
   next time the same machine works.


         *** (5.7.4) Saving an uploaded file under a temporary filename.

define Maybe(String)    // returns the temporary file name
   save_uploaded_file
     (
       ServerDescription desc,
       ByteArray body,
       Int32 start,
       Int32 end
     ) =
   uploaded_file_count <- 1 + *uploaded_file_count;
   with tfn = "_"+integer_to_string(virtual_machine_id)+"_"+integer_to_string(*uploaded_file_count),
   if (Maybe(WStream))connect to file server_directory(desc)+"/upload_temporary/"+tfn is
     {
       failure then failure,
       success(f) then
         if reliable_write(file(f),extract(body,start,end)) is
           {
             failure then failure,
             success(nw) then
               if nw = end - start
               then success(tfn)
               else failure
           }
     }.


         *** (5.7.5) Removing the path from a file name.

   When a file is uploaded, the browser sends  the complete path of the file on the client
   machine as the file name. Actually, this  is not quite normal. Nevertheless, we need to
   remove the path, and keep only the file name. This is achieved by 'remove_path' below.

define Int32
   file_name_begin
     (
       String full_name,
       Int32 i
     ) =
   if nth(i,full_name) is
     {
       failure then 0,
       success(c) then
         if c = '/'  then i+1 else
         if c = '\\' then i+1 else
         file_name_begin(full_name,i-1)
     }.

define String
   remove_path
     (
       String full_name
     ) =
   with l = length(full_name),
        b = file_name_begin(full_name,l-1),
   substr(full_name,b,l-b).


         *** (5.7.6) Reading a multipart entity.

define Maybe(Web_arg)
   get_multipart_entity
     (

       ServerDescription desc,
       ByteArray body,
       Int32 start,
       Int32 end
     ) =
   if find(to_byte_array(crlf+crlf),body,start) is
     {
       failure then failure,
       success(k) then
         if k >= end          // must be within this entity, not the next one
         then failure
         else if find_name_and_filename(body,start,k) is
           {
             failure then failure,
             success(n_mbfn) then if n_mbfn is (name,mbfn) then
               if mbfn is
                 {
                   failure then
                     success(web_arg(name,to_string(extract(body,k+4,end-2)))),
                        // we must substract 2 to end because of crlf just before the boundary

                   success(fn) then
                     if save_uploaded_file(desc,body,k+4,end-2) is
                       {
                         failure then failure,
                         success(tfn) then
                           success(upload(name,remove_path(fn),
                              server_directory(desc)+"/upload_temporary/"+tfn))

                       }
                 }
           }
     }.


define List(Web_arg)
   read_multipart_form_data_encoded_web_args
     (
       ServerDescription desc,
       ByteArray body,
       ByteArray __boundary,
       Int32 i,
     ) =
   if find(__boundary,body,i) is
     {
       failure then [ ],
       success(n) then
         if find(__boundary,body,n+length(__boundary)) is
           {
             failure then [ ],
             success(m) then
               if get_multipart_entity(desc,body,n+length(__boundary),m) is
               {
                 failure then [ ],
                 success(wa) then
                   [wa . read_multipart_form_data_encoded_web_args(desc,body,__boundary,m)]
               }
           }
     }.


define One
   multipart_form_data_answer
     (
       ServerDescription desc,
       Connection connection,
       Int32 ip_addr,
       HTTP_RequestLine request_line,
       List(HTTP_header) headers,
       ByteArray body,
       Server server
     ) =
   if get_boundary(headers) is
     {
       failure then unique,
       success(boundary) then
         with all_web_args = query_string(request_line) +
           read_multipart_form_data_encoded_web_args(desc,
                                                     body,
                                                     to_byte_array("--"+boundary),
                                                     0),
                       uri = uri(request_line),
                       ext = get_uri_extension(uri),
           log_journal_msg(desc,format_request(desc,connection,request_line,headers,all_web_args));
           if is_illegal_uri(uri,0)
           then log_journal_msg(desc,"Received illegal URI: "+uri+"\n")
           else
           if ext = ".awp" then
             (if awp_handler(desc)
                   (ip_addr,uri,headers,all_web_args,server) is (c_ticket,s_ticket,wp) then
              forget(reliable_write(connection,
                       with answer_body = format(empty,c_ticket,s_ticket,wp),
                       [ "HTTP/1.1 200 OK",crlf,
                        format(standard_headers(length(answer_body))) .
                        answer_body])))
           else unique
     }.


      *** (5.8) Handling redirections.

   'redirections' (of type 'List(Redirection)')  contains redirection directives. Each one
   has the form:

      redirect(required_uri,required_host,corresponding_uri).

   The  host required by  the client  may be  found in  the 'Host'  HTTP header.   The URI
   required by the client is given below as  'uri'. We just have to find the required host
   in the headers, and to find the corresponding redirection directive.


   In the next  fonction, the required host and  URI are known. We just have  to search in
   the 'redirections' list.

define String
   handle_redirection
     (
       String required_uri,
       String required_host,
       List(Redirection) redirections
     ) =
   if redirections is
     {
       [ ] then required_uri,
       [h . t] then if h is redirect(uri,host,target) then
         if host = required_host
         then if uri = required_uri
              then target
              else handle_redirection(required_uri,required_host,t)
         else handle_redirection(required_uri,required_host,t)
     }.


   Finding the 'Host' header. No redirection is performed if this header is not found.

define String
   handle_redirection                  // returns the redirected URI
     (
       ServerDescription desc,
       String uri,                     // original URI
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then uri,
       [h . t] then if h is http_header(name,value) then
         if name = "host"
         then handle_redirection(uri,value,redirections(desc))
         else handle_redirection(desc,uri,t)
     }.


      *** (5.9) Answering both sorts of requests.

   We must decide  if the request is www-url encoded  of multipart/form-data encoded. This
   is achieved through the header 'Content-Type'.

define EncodingType
   get_encoding_type
     (
       List(HTTP_header) headers
     ) =
   if headers is
     {
       [ ] then www_url,     // this is the default
       [h . t] then if h is http_header(name,value) then
         if name = "content-type"
         then if find("multipart/form-data",value,0) is
           {
             failure then www_url,
             success(_) then multipart_form_data
           }
         else get_encoding_type(t)
     }.


define One
   send_answer
     (
       ServerDescription desc,
       Connection connection,
       HTTP_RequestLine rqline,
       List(HTTP_header) headers,
       ByteArray body,
       Server server
     ) =
   if rqline is request_line(type,uri,qstring) then
   with rqline = request_line(type,handle_redirection(desc,uri,headers),qstring),
   if remote_IP_address_and_port(connection) is (ip_addr,_) then
   if get_encoding_type(headers) is
     {
       www_url then
         www_url_answer(desc,connection,ip_addr,rqline,headers,body,server),
       multipart_form_data then
         multipart_form_data_answer(desc,connection,ip_addr,rqline,headers,body,server)
     }.


   *** (6) The HTTP/HTTPS server.

   The command  'start_server' (declared in 'predefined.anubis') starts  a virtual machine
   which  opens  a  server TCP/IP  connection,  and  which  continuously listens  to  this
   connection. When a request arrives, this  machine delegates the work of deciphering and
   answering the request  to another virtual machine, and continues to  listen. The job of
   the delegated machine is defined by the HTTP request handler below.


      *** (6.1) The HTTP request handler.

   Here  is the HTTP  handler. It  is called  at each  new request  in a  separate virtual
   machine. It reads the headers of the HTTP request, determines body size, reads the body
   of the HTTP request, and answers the request.

define One
   handler
     (
       ServerDescription desc,
       Connection connection,        // connection with the client
       Server server
     ) =
   if remote_IP_address_and_port(connection) is (ip_addr,port) then
   if read_request_line(connection) is
     {
     error(msg) then log_journal_msg(desc,format(msg)),
     ok(request_line) then
       if read_http_headers(connection) is
       {
       error(msg) then log_journal_msg(desc,format(msg)),
       ok(headers) then if get_body_size(headers) is
         {
         error(msg) then log_journal_msg(desc,format(msg)),
         ok(body_size) then
           if read_http_body(connection,body_size,constant_byte_array(0,0),1000) is
           {
           error(msg) then log_journal_msg(desc,format(msg)),
           ok(body) then
             send_answer(desc,
                         connection,
                         request_line,
                         headers,
                         body,
                         server)
           }
         }
       }
     }.


define Server -> ((RWStream) -> One)
   http_handler
     (
       ServerDescription desc,
     ) =
   (Server server) |->
   (RWStream connection) |-> handler(desc,tcp(connection),server).

define Server -> ((SSL_Connection) -> One)
   https_handler
     (
       ServerDescription desc,
     ) =
   (Server server) |->
   (SSL_Connection connection) |-> handler(desc,ssl(connection),server).


      *** (6.2) Starting the HTTP/HTTPS server.


   The next function creates the directories (if they don't already exist).

define One
   create_directories
     (
       String server_dir
     ) =
   //forget(make_directory(server_dir+"/public",default_directory_mode));
   forget(make_directory(server_dir+"/upload_temporary",default_directory_mode));
   forget(make_directory(server_dir+"/private_download",default_directory_mode));
   forget(make_directory(server_dir+"/journal",default_directory_mode)).


   The 'notify' functions are used only when the server cannot accept a new connection.

define (One) -> One
   notify
     (
       ServerDescription desc,
     ) =
   (One u) |->
     log_journal_msg(desc,"HTTP server cannot accept new connection.\n").


define (One) -> One
   ssl_notify
     (
       ServerDescription desc,
     ) =
   (One u) |->
     log_journal_msg(desc,"HTTPS server cannot accept new connection.\n").


   Below is the command which starts an HTTP server.

public define One
   start_http_server
     (
       HTTP_ServerDescription desc
     ) =
   create_directories(server_directory(desc));
   with idesc = internal_description(desc),
   with ip_addr = ip_address(idesc),
        port = ip_port(idesc),
   if start_server(ip_addr,
                   port,
                   http_handler(idesc),
                   notify(idesc)) is
     {
       cannot_create_the_socket then print("Cannot create the listening socket.\n"),
       cannot_bind_to_port      then print("Cannot bind to port.\n"),
       cannot_listen_on_port    then print("Cannot listen on port.\n"),
       ok(server) then
         print("HTTP Server started on "+ip_addr_to_string(ip_addr)+
                  ":"+integer_to_string(port)+".\n");
         delegate (checking every 1000 milliseconds, wait for
                   (service(desc)(unique); false) then unique),
         unique
     }.


   Below is the command which starts an HTTPS server.

public define One
   start_https_server
     (
       HTTPS_ServerDescription desc
     ) =
   create_directories(server_directory(desc));
   with idesc = internal_description(desc),
   with ip_addr = ip_address(idesc),
        port = ip_port(idesc),
   if start_ssl_server(ip_addr,
                       port,
                       server_common_name(desc),
                       https_handler(idesc),
                       ssl_notify(idesc)) is
     {
       cannot_create_the_socket then print("Cannot create the listening socket.\n"),
       cannot_bind_to_port      then print("Cannot bind to port.\n"),
       cannot_listen_on_port    then print("Cannot listen on port.\n"),
       ok(server) then
         print("HTTPS Server started on "+ip_addr_to_string(ip_addr)+
                  ":"+integer_to_string(port)+".\n");
         delegate (checking every 1000 milliseconds, wait for
                   (service(desc)(unique); false) then unique),
         unique
     }.