diff --git a/calexium_lib/web/CXM_making_a_web_site.anubis b/calexium_lib/web/CXM_making_a_web_site.anubis index cf757f4..d4c7cdd 100644 --- a/calexium_lib/web/CXM_making_a_web_site.anubis +++ b/calexium_lib/web/CXM_making_a_web_site.anubis @@ -1,3719 +1,3722 @@ - - - *Project* Anubis - - *Title* Making interactive Web sites. - - *Copyright* Copyright (c) Alain Prouté 2004-2005. - - - *Author* Alain Prouté - - *Revised* January 2005. - - - *Overview* - - In this file we propose simple tools for making well structured interactive and secured - web sites. - - - ----------------------------------- Table of Contents --------------------------------- - - * (1) Structure of a web site. - ** (1.1) Three sorts of data. - ** (1.2) How requests are handled. - ** (1.3) What web pages are made of. - ** (1.4) Actions. - ** (1.5) States. - - * (2) Carrying on. - ** (2.1) Describing your web sites. - ** (2.2) Directories on the server's disk. - ** (2.3) Starting your web sites. - - * (3) The HTML interface. - ** (3.1) Types used by the HTML interface. - ** (3.2) ``in form'' versus ``off form''. - ** (3.3) Defining your own style. - ** (3.4) Actioners and forms. - ** (3.5) Local popup. - - --------------------------------------------------------------------------------------- - - -read tools/basis.anubis -read CXM_common.anubis -read CXM_multihost_http_server.anubis -read CXM_mime.anubis - - - - * (1) Structure of a web site. - - First of all we need to explain what a web site should be made of. Ideally, the - visitor (also called the 'client') should see the web site working as any other - interactive computer software. So, it should be clear that a 'session' (i.e. a visit - to the web site, including the consultation of several pages) is some kind of - conversation between the visitor and the web site, and that the web site should - maintain a 'current state' of this conversation. At each new request (click) from the - visitor, this state must be updated. This whole conversation is called a 'session' and - should not be confused with a single request. - - - - ** (1.1) Three sorts of data. - - All the data needed for putting a web site at work may be dispatched into three - categories: - - 1. Constant data (data that never change). These data may be hard coded into the - Anubis source files of the web site. - - 2. Permanent data (data which always exist independantly of the users connected to - the web site). These data are normally recorded into data bases. - - 3. Session data (data which depend on a particular visitor and which exist only - during the time he visits the web site). These data are stored into so-called - 'states'. - - - It is important to determine which data belongs to which category. This is part of your - design decisions. - - - - ** (1.2) How requests are handled. - - We want to separate the following two functionalities (which are used at each request - (click) during a single session): - - - computing the new state from the previous state and from the client request, and - updating the data base, - - - computing the page to be sent to the client from the new current state and from - the informations in the data base. - - - The next picture shows the structure we have in mind: - - - request +---------+ HTML page (with a hidden state name) - .-------------------| client |<--------------. - | .-----------------| | | - | | previous state +---------+ | - | | name (if any) | - | | | client side - ............................................................................ - | | | server side - | | | - | | .-------------------. | - | | | previous state | | - V V V | | - +---------------+ +---------------+ +--------------+ - | compute state | | server's disk | | compute page | - +---------------+ +---------------+ +--------------+ - ^ | | ^ ^ ^ ^ ^ - | | | | | | | | - | | `--------------------+--------------------' | | - | | new state | | | - read | `------------------------+--------------------' | - write | new state name | - update V | - +-----------+ | - | data base |--------------------------------------------' - +-----------+ read only - - - When the client begins a session, there is no previous state. In this case, a default - 'initial state' is used instead. - - The data base may be updated by 'compute state' box, but should not be update by the - 'compute page' box. The 'compute page' box should be allowed only to read the data - base. - - In this file, all the above stuff is defined, except the 'compute state' and 'compute - page' boxes. You just have to provide the function for computing a new state (compute - state) and the function for computing the page (compute page) from the new state. You - don't have to worry about state names, saving and retrieving states and the like. - - - - - - ** (1.3) What web pages are made of. - - What the client can see in his browser's window may be called a 'page'. Within a page, - we have several sorts of components: - - - 'local' components, i.e. all components which do not open a connection, like - texts, images, etc... possibly using JavaScript programmation, - - - 'actioners', which, when clicked upon, open a connection with our web site; they - may appear as links or buttons, etc... - - - 'foreign links', which when clicked upon, open a connection with another web site - (or ours eventually). - - Of course, what an actioner does is just ask our web site to perform an action. To that - end, the actioner essentially sends the name of the action to be performed. However, it - may be necessary to provide additional informations which may be seen as 'operands' of - the action. In order to attach operands to an action, HTML provides the notion of - 'form'. Indeed, a form contains essentially a set of input fields into which the client - may put values for the required operands of the action, and a submit button, which is - the actioner itself. Notice that a single form may contain several submit buttons, - which simply means that there are several distincts actions taking the same set of - operands. - - Restrictions must be put on the use of all theses gadgets. Indeed, for example, - putting a form within another form is officially meaningless in HTML, and the client's - browser may be seriously disturbed by this. In this file, we propose an interface to - the HTML language, which forbids such meaningless things, simply by imposing a strict - typing of HTML concepts. - - Each web site may be accessible through two communication channels: - - - a non secured channel (HTTP), - - a secured channel (HTTPS). - - Nevertheless, the whole thing should be considered as a single web site. For example, - you may have a secured page, obtained through HTTPS, containing public images obtained - through HTTP. An actioner in a non secured page may open a secured connection, and - conversely. - - Summarizing, a web page is made of local elements, foreign links and actioners. - Actioners receive operands from forms, and they also choose to communicate through the - non secured or through the secured channel. - - - - +-------------------+ - | page | - | | +---------------+ - | +--------------+ | | next page | - | | form | | | (non secured) | - | | +----------+ | | HTTP | | - | | | actioner |---------------------------->| | - | | +----------+ | | +---------------+ - | | | | - | | +----------+ | | +---------------+ - | | | actioner |---------------------------->| next page | - | | +----------+ | | HTTPS | (secured) | - | | | | | | - | +--------------+ | | | - | | +---------------+ - | | - +-------------------+ - - - Notice that actioners need no be necessarily put into forms. In that case, they work as - ordinary links, but they still may receive operands as we shall see. - - - - - ** (1.4) Actions. - - The client opens a new connection with our web site whenever he clicks on an - actioner. The result is that a request is sent, essentially made of a list of 'web - arguments'. Each web argument is a pair (name,value). One of these web arguments, the - 'action' web argument (whose name is "a"), determines the action to be performed. The - other web arguments (not including "s", used to identify the state) are the operands - for this action. - - Hence, the 'compute state' box in the picture above, splits naturally into as many - sub-boxes as there are actions. For this reason, we define the following type for - representing actions (where '$State' is the type representing session informations): - -public type Web_Action($SessionTicket, $State): - http_action (String name, // name of action - (Maybe($State)) -> Bool allow, // true if action allowed - (HTTP_Info http_info, - List(Web_arg) web_args, // actually only 'operands' web arguments - Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it), - https_action (String name, // name of action - (Maybe($State)) -> Bool allow, // true if action allowed - (HTTP_Info http_info, - List(Web_arg) web_args, // actually only 'operands' web arguments - Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it), - http_https_action (String name, - (Maybe($State)) -> Bool allow, // true if action allowed - (HTTP_Info http_info, - List(Web_arg) web_args, - Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it). - - 'http_action's are executed only under HTTP, and 'https_action's are executed only - under HTTPS. 'http_https_action's may be executed under both types of connections. - - Each action has a name, which is used to identify the action. Each action also has a - function 'allow' whose job is to verify that the action is allowed in the current - state, and a function 'do_it' for performing the action. The function 'do_it' receives - a lot of informations: - - - 'HTTP informations': - - the IP address of the client, - - the URI requested by the client (after redirection), - - the list of HTTP headers generated by the client's browser, - - the list of web arguments sent by the client (except "s" and "a"), - - the previous state (or the 'initial' or 'ticket expired' state if no previous - state can be found). - - In most cases, HTTP informations are not used. This is the reason why they are gathered - for simplicity into a unique datum of type 'HTTP_Info'. - -// For your convenience, we introduce the following simpler variants: -// -//public define Web_Action($State) -// http_action -// ( -// String name, -// $State -> Bool allow, -// (List(Web_arg),$State) -> $State do_it -// ) = -// http_action(name, -// (Maybe($State) ms) |-> if ms is -// { -// failure then true, -// success(s) then allow(s) -// }, -// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is -// { -// failure then (failure, []), -// success(s2) then (success(do_it(l,s2)), []) -// }). -// -//public define Web_Action($State) -// https_action -// ( -// String name, -// $State -> Bool allow, -// (List(Web_arg),$State) -> $State do_it -// ) = -// https_action(name, -// (Maybe($State) ms) |-> if ms is -// { -// failure then true, -// success(s) then allow(s) -// }, -// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is -// { -// failure then (failure, []), -// success(s2) then (success(do_it(l,s2)), []) -// }). -// -//public define Web_Action($State) -// http_https_action -// ( -// String name, -// $State -> Bool allow, -// (List(Web_arg),$State) -> $State do_it -// ) = -// http_https_action(name, -// (Maybe($State) ms) |-> if ms is -// { -// failure then true, -// success(s) then allow(s) -// }, -// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is -// { -// failure then (failure, []), -// success(s2) then (success(do_it(l,s2)), []) -// }). - - - - When you define your web site, you must provide the list of all the actions of the - site. When a new state has been computed, a graphical representation of this state - must be sent to the client. To that end, you must provide a function (named below - 'compute_page') of type: - - $State -> HTML_Page - - where the type 'HTML_Page' (defined below in this file) abstractly represents HTML - pages. - -public type HTML_Page:... - - It should be clear that states and pages are deeply linked together. Indeed, we really - understand the page shown to the client as a representation of the current state of the - conversation between the client and the web site, but also containing informations - taken from the data bases. - - - - - - ** (1.5) States. - - Now, we explain how you can define the type (say 'State') to be used as an instance of - the type parameter '$State'. The following is just a suggestion. - - Each state determines a page (since 'compute_page' computes a page from a - state). However, some components of the state may be independant of the page. It may be - the case for example for the indication of the natural language used by the - client. Hence, a state should be made of (at least) two parts: - - - informations which are the same for all pages, - - informations which are particular to each page. - - For example, you could define: - - type Page: // one alternative per page, with particular informations - login(...), // in the components - main_page(...), - ...etc... - - Now, the type 'State' could be defined as follows: - - type State: - state(Language, // informations valid for all pages - ..., - Page). // informations particular to a page - - However, if you are making a secured web site within which clients should be identified - (by id and password), it may be a good idea to have two sorts of states, one for non - identified clients and one for identified clients. In this case, define the type - 'State' as follows (this is just a suggestion): - - type State: - non_identified(Language), - identified(String id, - Language, - Page). - - When a request arrives, check if the previous state is 'identified(...)' or - 'non_identified(...)', and don't provide access to certain pages to non identified - clients. This is required for security. - - Some more words on security. If your site needs to identify clients, define the - initial state as 'non_identified(...)'. Construct a 'login' page, and check the id and - password of the client. If the id and password are correct, then change the state of - the client to 'identified(...)'. No other action should be able to do that. Now, be - confident that clients cannot forge states. The only information they have is the name - of a state, not the state itself which is never sent over the network, but only stored - on the server's disk. The name of the state is constructed using strong cryptographical - methods (sha1). If everything (since the 'login' page) is performed under HTTPS, even - state names cannot be seen by a third party. So, if the system retrieves a previous - state of the form 'identified(...)', you can be confident that your client is well - identified, and you can send him confidential informations. - - States have a limited life time. It may happen that a client clicks on a button at a - time its state is out of date. In this case, this system considers that the new state - is a special state named 'ticket expired'. You must provide a function producing this - state when you describe your web site. The page corresponding to this state must just - inform the client that he/she waited a too long time before clicking on a button, and - has to restart (a new conversation) from the begining. - - - - * (2) Carrying on. - - ** (2.1) Describing your web sites. - - Before you may start your web site, you must describe it, i.e. produce a datum of the - opaque type 'Web_Site'. - -public type Web_Site:... - - Producing such a datum may be performed by: - -public define Web_Site - make_web_site_description - ( - List(String) common_names, // for example: ["www.our-business.com", - // "192.168.0.1"] - // the second one is just for testing - String site_directory, // where 'public' and other directories are - // located (should NOT end with '/') - One -> One init, - (HTTP_Info) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) initial_state, - ($State expired, - HTTP_Info, - List(Web_arg), - Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_expired_state, - (HTTP_Info, - List(Web_arg), - Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_lost_state, - List(Web_Action($SessionTicket, $State)) actions, - (Maybe($SessionTicket), Maybe($State)) -> HTML_Page compute_page, - Int32 timeout, // seconds (todo: minutes) - List(Redirection) redirections, - String charset, - List(String) journal_extensions, - List(String) journal_headers, - String authorization_secret, - List(MIME) known_mime_types, - (String action_name, - List(Web_arg) args)-> One before_send_file - ). - - - Explanations: - - 'common_names' is the list of names of the site (the name the browser must send as the - value of the 'Host' HTTP header in order to access the site must be in that list). Such - a name generally looks like this: - - www.somewhere.com - - If you are using HTTPS, you also have an 'X.509 SSL server certificate'. The name of - the site must be exactly the same as the name on the certificate (which is precisely - called the 'common name' in the X.509 jargon). If the two names do not match, the site - will still work, but the transaction will not be transparent to the client. His browser - will complain that the name of the certificate does not match the name of the site, and - he will have to accept the certificate manually. - - 'site_directory' is the absolute path to the directory where the files needed by the - site are located. Usually this directory looks like: - - my_anubis/web_sites/www.somewhere.com - - However, this information is not computed from 'common_name', so that you can change - the common name (for example temporarily, for networking reasons) without loosing - access to the files. - - 'ticket_expired_state(expired_state,http_info,lwa,is_https)' must produce the state - whose graphical representation is a page explaining to the user that its 'ticket' (or - 'session information') has expired, and that he/she must close all popup windows and - start a new session. The arguments of the function contain the previous (expired) - state and all current informations concerning the user. This arguments may be useful - for example for producing the expiration message in the language chosen by the user. - You can also (and this may be much smarter) send a 'ticket prolongation page' - (including a new login for example), and resume the same conversation, since you have - all the pertinent informations at hand. In the case the ticket is definitely lost, the - second fonction 'ticket_lost_state' is used. - - Notice that despite the fact that the parameter $State is involved in the arguments of - the above function, the type 'Web_Site' does not depend on this parameter. This allows - to produce lists of web site descriptions, where each description may be constructed - with a different instance of $State. This is required because distinct sites must have - distinct types of session informations. This is made possible by the fact that the - type is obscure, and the constructor replaced by a function which assembles - 'ticket_expired_state', ticket_lost_state', 'actions' and 'compute_page' into a single - entity not depending on $State. You should have a look to the private part of this file - if you want more precisions about this programming technique. - - 'charset' is a string which will determine the character encoding to be used by the - browser. Typically, this string is one of: "UTF-8", "ISO-8859-1", "Windows-1252", - etc... - - 'before_send_file' is a function which is executed just before the HTTP server sends a - file. It gets an action name and the web arguments received with the request for that - file. Notice that this action name and these web arguments may be put into a - 'private_download' element, and will come back to the server at the time of the - download. - - - - ** (2.2) Directories on the server's disk. - - The description of you site contains the name of the directory within which the - required files are located. This may be for example: - - my_anubis/web_sites/www.our-business.com/ - - This is called the 'site directory' (for the given site). Within the site directory, - the following directories are created by this program: - - states - public - journal - private_download - upload_temporary - - The directory 'states' is used for storing states (session informations). Out of date - states are automatically removed after some time. - - The tree rooted at 'public' contains files that the server is allowed to send to the - clients. For security reasons, the server never sends a file which is not within the - tree whose root is this 'public' directory (except for the 'private download' mecanism; - see 'web/multihost_http_server.anubis'). Also, the MIME type (see 'web/mime.anubis') - must have been recognized before the file may be sent. - - The directory 'journal' contains the jounal files. The roles of the remaining - directories 'private_download' and 'upload_temporary' is explained in - 'multihost_http_server.anubis', where you will also find further informations on - 'public' and 'journal'. - - - - - ** (2.3) Web servers parameters. - - The web servers have several parameters useful for administration. They are described - as follows: - - public type WebServersParameters: - wsparms(Var(Bool) shutdown_required, - - - - - ** (2.4) Starting your web sites. - - When you have described all your web sites (you may want to have several web sites, and - they are distinguished by their 'common name'), you may start them all together using - 'start_web_sites' below. This function returns a result of the following type: - -public type Start_Web_Sites_Result: - cannot_bind_to_port(Int32), - cannot_bind_to_port(Int32,Int32), - ok(Server http_server, - Server https_server). - - Indeed, it may happen that the system cannot bind (begin to listen) to one of the two - ports (or to both). The main reason is that another server is already listening on that - port. Another reason may be that 'anbexec' has not been correctly installed, i.e. that - the 's' bit has not been set for 'user' and 'group' (there is not such problem under - Windows). Also notice that the Linux kernel may need a rather long time (up to several - minutes) before liberating a listening port. Now, if the system can bind to the two - ports, the pair of the two servers is returned. Two tools are useful for manipulating - servers: - - shutdown of type Server -> One - is_down of type Server -> Bool - - They are defined in 'predefined.anubis' (together with the type 'Server'). - - -public define Start_Web_Sites_Result - start_web_sites - ( - Int32 ip_address, // the IP address shared by the web sites - Int32 http_port, // usually: 80 - Int32 https_port, // usually: 443 - String ssl_certificate_common_name, - List(Web_Site) web_sites, // web sites to be started - Var(Bool) shutdown_required - ). - - 'ip_address' is the IP address on which the two servers listen. If you put 0, the - servers listen on all the IP addresses of the machine. This may be useful if the - machine has several network interfaces. - - 'ssl_certificate_common_name' is the common name of the SSL certificate that 'anbexec' - loads when it starts. One instance of 'anbexec' cannot handle more than one SSL server - certificate. This is due to a problem of conception of SSL itself. See the book 'SSL - and TLS' by Eric Rescorla (at Addison Wesley) for more explanations. - - Notice that the number of servers is always 2, regardless of the number of web sites - you are starting. - - The dynamic variable 'shutdown_required' may be used to control the shutdown of the two - servers from within the web site (typically the administration part). The servers will - shutdown as soon as this variable contains 'true'. So you must provide a variable - containing 'false' otherwise your servers will not run. You may also use the primitive - 'must_restart' (see 'predefined.anubis') to control the restarting of your servers. - - - - - - - * (3) The HTML interface. - - We propose an interface to dynamic HTML. Dynamic HTML includes HTML, and a combination - of CSS (Cascading Style Sheet) and JavaScript techniques for making HTML elements more - reactive and attractive on the client side. - - - ** (3.1) Types used by the HTML interface. - - For easy reference, we gather below the definitions of all the types used by the HTML - interface, and we comment them immediately. - - -public type HTML_Size: - absolute(Int32), // in pixels - percentage(Int32). - - -public type Text_Option: - size(Int32), // size of character font to use - font(String), // name of character font to use (such as "helvetica",...) - color(RGB), // color to be used for characters - italic, - oblique, - small_capitals, - bold, - underlined, - left_justified, - right_justified, - justified, // justified on both sides - line_through, - nowrap, - class(String). //CSS class - - A list of 'Text_Option' must be given with each text you want to put in your page. - - This indicate the way of reading text. -public type Reading_Way: - ltr, //the text is readable from "Left To Right" like english - rtl. //the text is readable from "Right To Left" like arabic - - - -public type CoreAttrs: - id (String), - class (String), - style (String), - title (String). - -public type I18n: - lang (String), - dir (Reading_Way). - -public type DIV_Option: - id (String), - class (String), - style (String), - title (String), - lang (String), - dir (Reading_Way). - - - A list of 'DIV_Option' must be given with each DIV you want to put in your page. - - -public type Table_Option: - background_color(RGB), // applied to all cells in the table - background_image(String url), - border(Int32 width_of_outer_edge, // if not present, all values are 0 - Int32 width_of_top_of_relief, - Int32 width_of_inner_edge, - RGB border_color), - width(Int32), // sets a minimal width for the table - percentage_width(Int32). - - -public define Table_Option nude = border(0,0,0,rgb(0,0,0)). - - - A list of 'Table_Option' must be given with each table. - - -public type BackgroundOption: - repeat, // repeat the background in both directions - repeat_horizontal, // repeat the background only horizontally - repeat_vertical, // repeat the background only verticall - no_repeat, // don't repeat the background - center. - - -public type Cell_Option: - left, // put the content of the cell on the left - h_center, // center the content of the cell horizontally - right, // put the content of the cell on the right - top, // put the content of the cell upwards - v_center, // center the content of tye cell vertically, - bottom, // put the content of the cell downwards - base_line, // align the content vertically according to base lines - background_color(RGB), - background_image(String url, BackgroundOption), - width(Int32), // sets a minimal width for the cell - percentage_width(Int32), - height(Int32), // sets a minimal height for the cell - columns(Int32), // lets the cell span over several columns - rows(Int32), // lets the cell span over several rows - nowrap. // do not allow text wrapping within the cell - - A list of 'Cell_Option' must be given with each cell and each row in a table. Options - given with a row apply to all the cells in the row, but are superseded by options given - with cells, which apply only to the cell they are given with. - - -public type HTML_Cell($T): - cell(List(Cell_Option) options, $T content). - - The parameter $T is later instantiated either to 'HTML_In_Form' or to 'HTML_Off_Form', - depending on where you put your table (within a form or not within a form). For your - convenience, we define the following particular case: - -public define HTML_Cell($T) - cell - ( - $T content - ) = - cell([],content). - - - -public type HTML_Row($T): - row(List(Cell_Option) options, List(HTML_Cell($T)) cells). - - Same remark as for 'HTML_Cell($T)'. We define several convenience functions: - -public define HTML_Row($T) - row - ( - List(HTML_Cell($T)) cells - ) = - row([],cells). - -public define HTML_Row($T) - row - ( - HTML_Cell($T) cell - ) = - row([],[cell]). - -public type Actioner_Connection: - same, // use same type of connection as current page - http, // use non secured connection - https. // use secured connection - -public type Other_Window_Option: - resizable, // the new window may be resized by the client - scrollbars, // the new window has scrollbars - width(Int32), // the new window has the specified width - height(Int32). // the new window has the specified height - -public type Actioner_Target: - same, - same (String label), - other(String window_name, List(Other_Window_Option)). - -public type Actioner_Aspect: - link (List(Text_Option),String text), // hypertext link - push_button (List(CoreAttrs),String text), - button (String url_off, String url_on), // rollover button - button (String url_off, String url_on, Int32 w, Int32 h), // idem with size - immediate_selector (String name, Int32 size, List(String) choices). - - -public type Actioner_Local_Action: - close_window. - - -public define Actioner_Aspect - link - ( - String text - ) = - link([],text). - - -public define Actioner_Aspect - link - ( - List(Text_Option) options, - Int32 i - ) = - link(options,integer_to_string(i)). - -public define Actioner_Aspect - link - ( - Int32 i - ) = - link([],i). - -public define Actioner_Aspect - button - ( - String url_img - ) = - button(url_img,url_img). - - - - Actioners are explained in details below. - - -public type TextAreaOption: - disabled, - read_only, - wrap_lines. - -public type HTML_In_Form: - literal_pt (Printable_tree), - literal (String), - sequence (List(HTML_In_Form) items), - text (List(Text_Option), String the_text), - preformated (List(Text_Option), String), - paragraph (List(Text_Option), String the_text), - image (String url), - image (String url, Int32 width, Int32 height), - table (List(Table_Option), List(HTML_Row(HTML_In_Form))), - center (HTML_In_Form), - mail_to (String email, HTML_In_Form element), - scroller (Int32 width, Int32 height, - Int32 content_width, Int32 content_height, - HTML_In_Form content), - actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, - String action_name, List((String,String)) extra_ops, - List(Actioner_Local_Action)), - foreign_link (List(Text_Option), String url, String name), - private_download (String abs_path, String name, String extra_ext, - Maybe((String,List((String,String)))) action), - text_input (String label_text, String label, String name, String init, Int32 width), - password_input (String label_text, String label, String name, Int32 width), - text_area (List(TextAreaOption), String name, String init, Int32 width, Int32 height), - file_upload (String name, Int32 width), - selector (String name, Int32 size, List(String) choices), - selector (String name, Int32 size, List(String) choices, String selected), - // List((String,String)) = List((code,name)) where : - // name appears in selector - // code is the web-arg value - selector_c (String name, Int32 size, List((String,String)) choices), - selector_c (String name, Int32 size, List((String,String)) choices, String selected), - radio_button (String label_text, String label, String name, String value, Bool checked), - check_box (String label_text, String label, String name, Bool checked), - div (List(DIV_Option), HTML_In_Form content), - div_empty (List(DIV_Option)), - hidden (String name, String value). - - - 'HTML_In_Form' defines all the elements you may put within a form. We define a - convenience function: - -public define HTML_In_Form literal(Printable_tree t) = literal_pt(t). - -public define HTML_In_Form - foreign_link - ( - Int32 tsize, - String url, - String name - ) = - foreign_link([size(tsize)],url,name). - -public define HTML_In_Form - actioner - ( - Actioner_Connection conn, - Actioner_Target targ, - Actioner_Aspect asp, - String action_name, - List((String,String)) extra_ops - ) = - actioner(conn,targ,asp,action_name,extra_ops,[]). - - - -public define HTML_In_Form - text_area - ( - String name, - String init, - Int32 width, - Int32 height - ) = - text_area([],name,init,width,height). - -public define HTML_In_Form - table - ( - List(HTML_Row(HTML_In_Form)) rows - ) = - table([],rows). - - -public define HTML_In_Form - private_download - ( - String abs_path, - String name, - String extra_ext - ) = - private_download(abs_path,name,extra_ext,failure). - -public define HTML_In_Form - private_download - ( - String abs_path, - String name, - String extra_ext, - String action_name, - List((String,String)) args - ) = - private_download(abs_path,name,extra_ext,success((action_name,args))). - -public define HTML_In_Form - text - ( - String s - ) = - text([],s). - - - - -public type HTML_Off_Form: - literal_pt (Printable_tree), - literal (String), - sequence (List(HTML_Off_Form) items), - text (List(Text_Option), String the_text), - preformated (List(Text_Option), String), - paragraph (List(Text_Option), String the_text), - image (String url), - image (String url, Int32 width, Int32 height), - table (List(Table_Option), List(HTML_Row(HTML_Off_Form))), - center (HTML_Off_Form), - mail_to (String email, HTML_Off_Form element), - scroller (Int32 width, Int32 height, - Int32 content_width, Int32 content_height, - HTML_Off_Form content), - fixed_size (HTML_Size width, HTML_Size height, HTML_Off_Form content), - fixed_size_2 (HTML_Size width, HTML_Size height, String name_of_HTML_file), - actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, - String action_name, List((String,String)) extra_ops, - List(Actioner_Local_Action)), - actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, - String action_name, List((String,String)) extra_ops, - List(Actioner_Local_Action), String form_name), - foreign_link (List(Text_Option), String url, String name), - private_download (String abs_path, String name, String extra_ext, - Maybe((String,List((String,String)))) action), - label (String name), - form (String form_name, List(CoreAttrs), HTML_In_Form content), - div (List(DIV_Option), HTML_Off_Form content), - div_empty (List(DIV_Option)). - - 'HTML_Off_Form' defines all the elements you may put outside any form. - - -public define HTML_Off_Form literal(Printable_tree t) = literal_pt(t). -public define HTML_Off_Form fixed_size(HTML_Size width, HTML_Size height, String name_of_HTML_file) - = fixed_size_2(width,height,name_of_HTML_file). - - -public define HTML_Off_Form - foreign_link - ( - Int32 tsize, - String url, - String name - ) = - foreign_link([size(tsize)],url,name). - - -public define HTML_Off_Form - actioner - ( - Actioner_Connection conn, - Actioner_Target targ, - Actioner_Aspect asp, - String action_name, - List((String,String)) extra_ops - ) = - actioner(conn,targ,asp,action_name,extra_ops,[]). - -public define HTML_Off_Form - table - ( - List(HTML_Row(HTML_Off_Form)) rows - ) = - table([],rows). - - - - We add two convenience functions for 'row'. The reason why we add two functions, one - for 'HTML_In_Form' and one for 'HTML_Off_Form', is that adding a schema with an - arbitrary '$T' creates too many ambiguities. This is due to the fact that, if we do so, - the arguments of the function do not refer to any of the types defined here. - -public define HTML_Row(HTML_In_Form) - row - ( - HTML_In_Form content - ) = - row([],[cell([],content)]). - -public define HTML_Row(HTML_Off_Form) - row - ( - HTML_Off_Form content - ) = - row([],[cell([],content)]). - - -public define HTML_Off_Form - private_download - ( - String abs_path, - String name, - String extra_ext - ) = - private_download(abs_path,name,extra_ext,failure). - - -public define HTML_Off_Form - private_download - ( - String abs_path, - String name, - String extra_ext, - String action_name, - List((String,String)) args - ) = - private_download(abs_path,name,extra_ext,success((action_name,args))). - -public define HTML_Off_Form - text - ( - List(Text_Option) lto, - Int32 i - ) = - text(lto,integer_to_string(i)). - - -public define HTML_Off_Form - text - ( - Int32 i - ) = - text([],i). - -public define HTML_Off_Form - text - ( - String s - ) = - text([],s). - - - Cell a gap between two other cells : - -public define HTML_Cell(HTML_Off_Form) - width_gap - ( - Int32 w - ) = - cell([width(w)],text([],"")). - -public define HTML_Cell(HTML_In_Form) - width_gap - ( - Int32 w - ) = - cell([width(w)],text([],"")). - - - - Row a gap between two other rows : - -public define HTML_Row(HTML_Off_Form) - height_gap - ( - Int32 h - ) = - row([],[cell([height(h)],text([],""))]). - -public define HTML_Row(HTML_In_Form) - height_gap - ( - Int32 h - ) = - row([],[cell([height(h)],text([],""))]). - - - Notice that the two types have alternatives in common (same name, same arguments types, - up to the value of the parameter $T), which correspond to elements which may be put - anywhere in the page. - - - -public type CSS_Style: - text_options(List(Text_Option)). - -public type CSS_File: - css_file(String file_name). - -define String - format - ( - List(Text_Option) l - ). - - - - -define Printable_tree - format_css_styles - ( - List(CSS_Style) l - ) = - if l is - { - [ ] then [ ], - [h . t] then - [ if h is - { - text_options(tos) then - [" body, span, p { ", format(tos), " }\n" ] - } - . format_css_styles(t)] - }. - - - -public type HTML_Meta: - keywords (List(String)), - refresh (Actioner_Connection connection, - Actioner_Target target, - String action_name, - Int32 delay), // in seconds - meta (String name, String content), - http_equiv (String name, String content), - generic_meta (List((String,String))), - literal (String). - - Meta tags are put in the 'head' of the HTML page. - - -public type Body_Option: - background_color (RGB), - background_image (String url), - background_image (String url, List(BackgroundOption)). - -public type HTML_Body: - body(List(Body_Option) options, HTML_Off_Form content). - - -public type HTML_Page: - html_page(String title, - List(HTML_Meta) meta_tags, - List(CSS_Style) styles, - List(CSS_File) css_files, - HTML_Body body). - -public define HTML_Page - html_page - ( - String title, - List(HTML_Meta) metas, - HTML_Body body - ) = - html_page(title,metas,[],[],body). - -public define HTML_Page - html_page - ( - String title, - List(HTML_Meta) metas, - List(CSS_Style) styles, - HTML_Body body - ) = - html_page(title, metas, styles, [], body). - - 'HTML_Page' represents the final product of the construction of a web page. - - - - - *** (3.2) ``in form'' versus ``off form''. - - There is a variety of HTML elements: texts, buttons, links, forms, inputs, etc... Some - of them may have a content, which is yet another HTML element (or several). Hence, it - is meaningful to say that an element is 'within' another one. Now, putting any element - within any other one may be meaningless. For example, an input element must be put - within a form (otherwise, it is useless), and a from within another form has no precise - meaning (and is forbidden by the HTML specification). - - Actually, the main criterium is ``within a form or not within a form''. So, HTML - elements in a given page are separated into two categories: those who are within a - form, and the others. Nevertheless, there are elements which may belong to both - categories, like images and texts. We want to make use of the strong typing mecanism of - Anubis in order to forbid non meaningful placement of elements. - - The type 'HTML_In_Form' defines elements to be put within forms. Similarly, - 'HTML_Off_Form' defines elements not to be put within forms. Both types are recursive, - and 'HTML_Off_Form' refers to 'HTML_In_Form' (via the 'form' alternative, of course), - but the two types are not cross recursive. This is the reason why it is impossible to - put a form within a form. In order to construct a web page, you essentially have to - produce a datum of type 'HTML_Off_Form' (maybe containing data of type 'HTML_In_Form'). - - In practice, you don't have to worry so much about these two types, because elements - which may be put anywhere are constructed for both types by functions with the same - name and the same arguments. Hence, for both types, you just write the same thing. You - are warned by the compiler only when you try to put an element at a place it is not - allowed. - - - - *** (3.3) Defining your own style. - - We provide generic tools for constructing HTML elements. However, your web site needs - to have a ``style''. - - To that end, you need to write down a set of ``styling functions'', using the tools - defined here. These styling functions allow the introduction of your colors and other - visual characteristics into the constructed elements once and for all. For example, you - may want all your texts to be rendered in the ``Helvetica'' font, in size 14 and using - some 'text_color'. You may write something like this: - - define RGB text_color = rgb(10,40,40). - - define HTML_Off_Form - text - ( - String the_text - ) = - text([font("helvetica"),size(14),color(text_color)], - the_text). - - (and the same one for type 'HTML_In_Form') so that in order to put a piece of text in a - page, you just write: - - text("... some text ...") - - and you don't have to provide the font, size and color for each text. If you want to - have several styles of text presentation, you just write several sets of such - convenience functions. This also suggests a trick. You may want for example different - colors for 'in form' texts and 'off form' texts. This may be achieved automatically by - defining two functions as above, with the same name and same argument type, but - returning either a 'HTML_In_Form' or a 'HTML_Off_Form'. - - If this preliminary work is well done, you will not waste your time later when you - concentrate on the actual informational content of your pages. - - This general principle should be applied to all sorts of elements. This is the best - thing to do in order to separate the functions defining the visual style from the - functions defining the informational content itself, so that changing the style without - changing the content becomes easy. This is also the best way for having a clean and - easily readable source for your web site. - - - - - - *** (3.4) Actioners and forms. - - We have gathered several notions from HTML into that of an 'actioner'. An actioner is - an HTML element which opens a connection to our server when clicked upon. Actioners may - have different visual aspects. They may look like hypertext links or like buttons - (rollovers), or even like selectors (with immediate action). In any case, their - behavior is the same: they open a connection to our server, and send a set of 'web - arguments', i.e. pairs 'name=value'. Among these web arguments, one of them denotes - the action to be performed, and the others should be considered as operands for this - action. Actually, the precise behavior of the actioner has several variants. - - The connection with the server may be secured (HTTPS) or non secured (HTTP). See the - type 'Actioner_Connection' above. - - You must also choose where the answer must be rendered. This may be in the same window - or in another window (or frame). If it is in another window, the name of that window - must be given. If the window does not exist, the browser will create it. Optionally, - you may give the dimensions of the new window and other characteristics. See the type - 'Actioner_Target' above. - - The actioner also has a visual aspect. See the type 'Actioner_Aspect' above. In the - case of a rollover button, you provide the URLs of two images (of the same size) - representing the button: - - url_off: to be used when the mouse is not over the button, - url_on: to be used when the mouse is over the button. - - You can also create rollover buttons without creating images. Just use the second - alternative named 'button'. The server creates the images automatically. - - The purpose of forms is just to give operands to actioners. If the actioner is placed - within a form, all the input elements which are within this form provide operands to - the actioner (except sometimes when they are not set by the client). If it is not put - within a form, the actioner gets no operand, except if the name of a form is explicitly - given, in which case the actioner gets all the inputs from that form as - operands. Furthermore, you may want to give extra operands to the actioner. This may be - useful for separating families of actioners with the same action name. Extra operands - 'name=value' must be given in the form of pairs '(name,value)'. - - Notice that the name of a form may be used by an actioner which is off the form, so as - to get the operands provided by this form. Also notice that several actioners may refer - to the same form, being either in the form, or referring to the form from the - outside. These actioners simply get the same set of operands, even if they correspond - to distinct actions. - - Input elements may be put only within a form. - - - - *** (3.5) Local popup. - - This element looks like a link or a rollover button. When this button is clicked upon, - a 'popup window' appears. Actually, this popup window is just a layer in the same HTML - page, which becomes suddenly visible. It is realized with a '
' HTML tag. In - particular, clicking on the button does not open any connection. This is why it is - called 'local'. The arguments have the following roles: - - Actioner_Aspect aspect of the button (same semantics as for actioners) - content content of the popup window - x, y, position of the popup window on the HTML page (not relative - to the button but to the page itself) - title title of the popup window - color color of the title bar and close button in the popup window. - A lightened version of this color is used for the background - of the popup window. - width width of the title bar - - - - - - --- That's all for the public part ! -------------------------------------------------- - - - - - - ----------------------------------- Table of Contents --------------------------------- - - *** [1] States. - *** [1.1] Saving and retrieving states. - *** [1.2] Deleting out of date states. - - *** [2] Tools. - *** [2.1] Directories. - *** [2.2] Secondary documents. - - *** [3] Managing web arguments. - *** [3.1] Prefixing web arguments names. - *** [3.2] Separating web arguments. - *** [3.3] Applying an action. - - *** [4] Web site descriptions and the 'awp handlers'. - *** [4.1] The type 'Web_Site'. - *** [4.2] Making a web site description. - *** [4.3] Starting the servers. - - *** [5] HTML Formating. - *** [5.1] The type 'HTML_Any($T)'. - *** [5.2] Formating a color. - *** [5.3] Creating buttons. - *** [5.4] Formating an actioner. - *** [5.5] Formating a private download link. - *** [5.6] Formating rows and cells in a table. - *** [5.7] Formating elements which may be put anywhere. - *** [5.8] Formating 'in form' elements. - *** [5.9] Formating 'off form' elements. - *** [5.10] Formating meta-tags. - - --------------------------------------------------------------------------------------- - - - - -public define String - doctype_w3c_header - = - "\n". - - - - - *** [1] States. - - We have to define functions for saving a state, retrieving a state, deleting out of - date states. We need one such function per web site. The types of the first two - functions depend on the parameter $State. This is not the case of the third one. The - fact that the instance of $State is variable from one web sites to the other implies - rather subtle manipulations using full functionality. - - - *** [1.1] Saving and retrieving states. - - Each state is saved into a file on the server's disk (in the directory represented by - the symbol 'state_directory', which is 'my_anubis/web_sites/common_name/states'). The - state is saved together with a time stamp whose value is obtained by adding the current - time to the given timeout for states. The state receives a name obtained by hashing - (using sha1) the content of the file itself, and then encoding the hash with - 'web_arg_encode'. The name of the file into which the state is saved is the - concatenation of "s" and the name of the state. - -read CXM_web_arg_encode.anubis - - The tool below constructs the function which is able to save a state on the server's - disk. - -define (Maybe($State) s) -> String // the function constructed returns the name of the state - make_save_state_function - ( - Int32 timeout, - String state_directory - ) = - (Maybe($State) mbs) |-> - if mbs is - { - failure then "", - success(s) then - with time_stamp = now+timeout, - to_be_saved = (time_stamp,s), - state_name = web_arg_encode(sha1(s)), - if save(to_be_saved,state_directory+"/s"+state_name) is ok - then state_name - else (print("Cannot create state file in '"+state_directory+"'.\n"); "") - }. - - - When a request arrives, we need to retrieve the previous state from the server's - disk. We receive the name of that state. If the state is out of date, the state file is - kept 3 days, and then deleted. - -type PreviousState($State): - not_found, // cannot retrieve the previous state - out_of_date($State), // the previous state is out of date - still_valid($State). // the previous state is still valid - -define (String state_name) -> PreviousState($State) - make_retrieve_state_function - ( - String state_directory - ) = - (String state_name) |-> - with file_path = state_directory+"/s"+state_name, - if (RetrieveResult((Int32,$State)))retrieve(file_path) is ok(d) - then ( - if d is (time_stamp,s) then - if time_stamp < now - then ( - forget(remove(file_path)); - out_of_date(s) - ) - else still_valid(s) // state has been successfully retrieved - ) - else not_found. - - - - *** [1.2] Deleting out of date states. - - We also need to delete states which are out of date and which will never be deleted by - the above method. This may be performed by a machine doing this periodically (say once - per states life time period). - -define (List(String) file_names) -> One - make_delete_out_of_date_states_function - ( - Maybe($State) dummy, - String state_directory - ) = - (List(String) file_names) |-df-> - if file_names is - { - [ ] then unique, - [h . t] then - with file_path = state_directory+"/"+h, - if (RetrieveResult((Int32,$State)))retrieve(file_path) is ok(d) - then ( - if d is (time_stamp,data) then - if time_stamp < now - then (forget(remove(file_path)); df(t)) - else df(t) - ) - else (forget(remove(file_path)); df(t)) - }. - - - The 'labelled arrow' |-df-> is documented in 'documentation/en/anubis_doc.txt'. - - Note: The argument 'dummy' (of type Maybe($State)) is not used in the body of the - function (hence its name). Nevertheless, it is required. Indeed, the Anubis compiler - does not accept a parameter in the body of a function (here the parameter is required - by the use of 'retrieve') if this parameter does not appear in the type of the - function. This is because this would create ambiguities that no explicit typing may - ever resolve. If you put a double slash in front of the declaration of 'dummy' above, - and if you compile this file, you will get a message like this one: - - Error in 'making_a_web_site.anubis', line 1300, column 7: - A definition may not contain parameters which are not present - in the declaration part (hidden parameters): - $State - - The type of the function constructed by 'make_delete_out_of_date_states_function' is - independant of the parameter $State. This is important because this allows to create - the list of such functions for all web sites. From this list, it is possible to call - the functions one after the other, so deleting out of date states for all web - sites. Actually, the next function receives a list of pairs (state_directory,function), - one for each web site. - -define One - delete_out_of_date_states // for all web sites - ( - List((String, List(String) -> One)) directories_and_functions - ) = - if directories_and_functions is - { - [ ] then unique, - [h . t] then if h is (state_directory,function) then - function(directory_list(state_directory,"s*")); - delete_out_of_date_states(t) - }. - - - The above function must be called periodically in a separate virtual machine. The - period we have choosen is (rather logically) the life time of states itself. This may - be achieved by an 'infinite' loop, using a 'sleep(timeout)'. However, the loop must not - be really infinite, because the servers may be shutdown. Hence, our loop must test - (rather frequently; say every second) if the servers are down. If they are, the loop - must be exited. - -define One - delete_states_loop - ( - List((String,List(String) -> One)) directories_and_functions, - Int32 timeout, - Int32 next_time, - Server http_server, - Server https_server, - Var(Bool) shutdown_required - ) = - if *shutdown_required - then (shutdown(http_server); shutdown(https_server)) - else unique; - if (is_down(http_server) & is_down(https_server)) - then unique - else if now > next_time - then - ( - delete_out_of_date_states(directories_and_functions); - delete_states_loop(directories_and_functions, - timeout, - now+timeout, - http_server, - https_server, - shutdown_required) - ) - else - ( - sleep(1000); // sleep just one second and try again - delete_states_loop(directories_and_functions, - timeout, - next_time, - http_server, - https_server, - shutdown_required) - ). - - The above loop must be run in a separate virtual machine. This will be done just after - the two servers are started. - - - - - - *** [2] Tools. - - *** [2.1] Directories. - - We need a tool for creating directories (if needed). - - (This tool has been moved to 'tools/basis.anubis'). - - - - - *** [2.2] Secondary documents. - - Some HTML elements (like '', '') cannot receive their content directly - from the current document, but only through an URL. For this reason, we implement a - mecanism for creating secondary documents on the fly. To that end we use the 'private - download' mecanism. - - A secondary document is formated by the same functions as the main document itself. The - next function takes an 'off form' element, creates the file containing the secondary - document in HTML format, and returns the URL at which the document will be available. - -define String - create_secondary_document - ( - String sd, // site directory - String as, // authorization_secret - String sn, // state name - $T -> Printable_tree format_element, - $T content, - HTML_Size width - ) = - with private_download_directory = sd+"/private_download", - hash = web_arg_encode(sha1(content)), - file_content = (Printable_tree) - [doctype_w3c_header, - "
", - format_element(content), - "
" - ], - file_name = "sd"+hash+".html", - file_path = private_download_directory+"/"+file_name, - if write_to_file(file_path,file_content) is - { - cannot_open_file then print("Cannot open file '"+file_path+"'.\n"); "", - write_error(n) then print("Error writing file '"+file_path+"'.\n"); "", - ok then file_name+"?zauth="+ - make_authorization(sd,as,private_download_directory+"/"+file_name) - }. - - - - - *** [2.3] Generating unique ids. - - In order to uniquely name object for JavaScript we generate unique ids from a counter. - -define Int32 - new_idnum - ( - Var(Int32) ic_v // 'idnum' counter variable - ) = - protect - with result = *ic_v+1, - ic_v <- result; - result. - - - - - - - *** [3] Managing web arguments. - - Web arguments are those pairs 'name=value' which are transmitted through the HTTP - protocol. We need precise naming conventions for these web arguments. - - - - *** [3.1] Prefixing web arguments names. - - We want to assign different roles to web arguments, and we also want to be able to - recognize its role directly from the name of a web argument. The name "s" is reserved - for the web argument whose value is the name of the current state. The name "a" is - reserved for the web argument whose value is the name of the action to be - performed. Other web arguments receive arbitrary names, and in order to avoid clashes, - these names are prefixed by: - - "p" for names of password inputs, - "o" for other web arguments - - The reason why password input names have a distinct prefix is that this allows the HTTP - server to hide the passwords on the console of the server and in the journal. - - - - - *** [3.2] Separating web arguments. - - When a new request arrives, we need to separate the web arguments, that is to say: - - - find the value of "s", and recover the corresponding state, - - find the value of "a", which is the name of the action to be performed, - - get the list of all the remaining web arguments (operands of the action). - - We must also determine if the previous state may be recovered. If it is not the case - (either because the previous state name is invalid, or the previous state is out of - date), we must check if there is an action name. Indeed, the presence of an action name - indicates that the user has clicked on one of our buttons or links. If on the contrary - there is no action name the user has just entered our address in his browser. In this - last case, we must send the first page of our site (maybe a 'login' page), but if there - is an action, we must send a page just saying that the session ticket has expired. If - the previous state is recovered and there is no action, the new state is the same as - the previous state. - - The result of the separation of the web arguments is of type: - -type Separated_Web_Args($State): - swa(Maybe(PreviousState($State)) previous_state, - Maybe(String) action_name, - List(Web_arg) operands). - - - - The next function constructs the function which separates the web arguments. - -define (List(Web_arg) lwa) -> Separated_Web_Args($State) - make_separate_web_args_function - ( - String state_directory, - String -> PreviousState($State) retrieve_state - ) = - (List(Web_arg) lwa) |-swaf-> - if lwa is - { - [ ] then - // - // no web arg found => no previous state and no action - // - swa(failure,failure,[]), - - [wa_1 . wa_others] then - // - // at least one web arg => - // separate other web args, and insert the first one as needed - // - if (Separated_Web_Args($State))swaf(wa_others) is - { - swa(ps1, // possible previous state - an1, // maybe an action name - op1) // operands so far - then - if wa_1 is - { - web_arg(n,v) then - with prefix = if substr(n,0,4) = "amp;" then substr(n,4,1) else substr(n,0,1), - name_start = (Int32)(if substr(n,0,4) = "amp;" then 5 else 1), - if prefix = "s" then - swa(success(retrieve_state(v)),an1,op1) else - if prefix = "a" then - swa(ps1,success(v),op1) else - if prefix = "t" then - swa(ps1,an1,[web_arg("target",v) . op1]) else - if prefix = "p" then - swa(ps1,an1,[web_arg(substr(n,name_start,length(n)-name_start),v) . op1]) else - if prefix = "o" then - swa(ps1,an1,[web_arg(substr(n,name_start,length(n)-name_start),v) . op1]) else - swa(ps1,an1,op1), - - upload(n,v,t) then - swa(ps1,an1,[upload(substr(n,1,length(n)-1),v,t) . op1]) - }} - }. - - - - - - - *** [3.3] Applying an action. - - When the web arguments are separated (and their names cleaned up from prefixes), we may - apply the action to the operands and the current state. We search for the action to be - applied in the list of actions. If no action is found, the new state is the same as - the previous state. Also, we deny the application of an HTTP action if the request - arrives through the HTTPS channel and conversely. - - -define (Maybe($State) previous, - String action_name, - HTTP_Info http_info, - List(Web_arg) lwa, - Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) - make_apply_action_function - ( - List(Web_Action($SessionTicket, $State)) actions - ) = - with f = - (Maybe($State) previous, - String action_name, - HTTP_Info http_info, - List(Web_arg) lwa, - Bool is_https, - List(Web_Action($SessionTicket, $State)) actions) |-f-> - if actions is - { - [ ] then (print("action '"+action_name+ - "' not found.\n"); (failure, previous, [])), - [ac1 . others] then if ac1 is - { - http_action(an,allow,do_it) then - if an = action_name - then if is_https - then (print("HTTP action '"+an+ - "' called through HTTPS (denied).\n"); - (failure, previous, [])) - else if allow(previous) - then do_it(http_info,lwa,previous) - else (failure, previous, []) - else f(previous,action_name,http_info,lwa,is_https,others), - - https_action(an,allow,do_it) then - if an = action_name - then if is_https - then if allow(previous) - then do_it(http_info,lwa,previous) - else (failure, previous, []) - else (print("HTTPS action '"+an+ - "' called through HTTP (denied).\n"); - (failure, previous, [])) - else f(previous,action_name,http_info,lwa,is_https,others), - - http_https_action(an,allow,do_it) then - if an = action_name - then if allow(previous) - then do_it(http_info,lwa,previous) - else (failure, previous, []) - else f(previous,action_name,http_info,lwa,is_https,others), - - } - }, - (Maybe($State) previous, - String action_name, - HTTP_Info http_info, - List(Web_arg) lwa, - Bool is_https) |-> - f(previous,action_name,http_info,lwa,is_https,actions). - - - - - - - - - *** [4] Web site descriptions and the 'awp handlers'. - - *** [4.1] The type 'Web_Site'. - - The type 'Web_Site_Description' is defined in 'web/multihost_http_server.anubis'. We - need another one, because, we have some extra informations to record for each site. - -public type Web_Site: - web_site((Int32,Int32) -> Web_Site_Description description, - List(String) -> One delete_out_of_date). - - - - - *** [4.2] Making a web site description. - - Below is the function which creates a web site description. It first creates (if - needed) the directories for the site, then constructs the tool functions for the site, - and the site handler. Finally, it constructs the web site description. - - We gather common (constant) informations in the following type: - -type CommonInfo: - info(String common_name, - Int32 http_port, - Int32 https_port, - String site_directory, - String authorization_secret - ). - - We need a forward declaration. - -public define Printable_tree - format - ( - CommonInfo cinfo, - String state_name, - HTML_Page page, - Bool is_https, - String charset - ). - - -define Printable_tree - format - ( - HTML_Size s - ) = - if s is - { - absolute(x) then ["\"",x,"\""], - percentage(x) then ["\"",x,"%\""] - }. - -define Printable_tree - top_redirection_page - ( - String common_name - ) = - [ doctype_w3c_header, - "", - "", - "" - ]. - -public define Web_Site - make_web_site_description - ( - List(String) common_names, // for example: ["www.our-business.com"] - String site_directory, - One -> One init, - (HTTP_Info) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) initial_state, - ($State expired, - HTTP_Info, - List(Web_arg), - Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_expired_state, - (HTTP_Info, - List(Web_arg), - Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_lost_state, - List(Web_Action($SessionTicket, $State)) actions, - (Maybe($SessionTicket), Maybe($State)) -> HTML_Page compute_page, - Int32 timeout, - List(Redirection) redirections, - String charset, - List(String) journal_extensions, - List(String) journal_headers, - String secret, - List(MIME) known_mime_types, - (String action_name, - List(Web_arg) args) -> One before_send_file - ) = - init(unique); - - - // - // make required directories (if needed) - // - with web_sites_directory = make_directory(my_anubis_directory+"/web_sites"), - base_directory = make_directory(site_directory), - state_directory = make_directory(site_directory+"/states"), - forget(make_directory(site_directory+"/public")); - // - // construct tool functions - // - with save_state = make_save_state_function(timeout,state_directory), - retrieve_state = make_retrieve_state_function(state_directory), - separate_web_args = make_separate_web_args_function(state_directory,retrieve_state), - apply_action = make_apply_action_function(actions), - // - // construct the site handler - // - site_handler = (Int32 http_port, Int32 https_port) |-> - ((String host_name, - HTTP_Info http_info, - List(Web_arg) lwa, - Bool is_https) |-> - ((List(HTTP_header),Printable_tree)) - if separate_web_args(lwa) is - { - swa(mb_previous_state,mb_action_name,operands) then - with state_and_headers = if mb_previous_state is - { - failure then - if mb_action_name is - { - failure then initial_state(http_info), - success(action_name) then - apply_action(failure,action_name,http_info,operands,is_https) - }, - success(previous_state) then if previous_state is - { - not_found then - if mb_action_name is - { - failure then initial_state(http_info), - success(_) then - ticket_lost_state(http_info,lwa,is_https) - }, - - out_of_date(state) then - ticket_expired_state(state,http_info,lwa,is_https), - - still_valid(state) then - if mb_action_name is - { - failure then (failure, success(state), []), - success(action_name) then - apply_action(success(state),action_name,http_info,operands,is_https) - } - } - }, - if state_and_headers is (session_ticket, mb_new_state, headers) then - with state_name = save_state(mb_new_state), - (headers, - format(info(host_name,http_port,https_port,site_directory,secret), - state_name,compute_page(session_ticket, mb_new_state),is_https,charset)) - }), - // - // make the delete_out_of_date function - // - delete_out_of_date = - make_delete_out_of_date_states_function((Maybe($State))failure, - site_directory+"/states"), - // - // construct the web site description - // - web_site((Int32 http_port, Int32 https_port) |-> - web_site_description(common_names, - site_directory, - redirections, - charset, - journal_extensions, - journal_headers, - secret, - known_mime_types, - site_handler(http_port,https_port), - (List(Web_arg) lwa) |-> if separate_web_args(lwa) is - swa(mb_previous_state,mb_action_name,operands) then - if mb_action_name is - { - failure then unique - success(an) then before_send_file(an,operands) - }), - delete_out_of_date). - - - - - *** [4.3] Starting the servers. - -public define Start_Web_Sites_Result - start_web_sites - ( - Int32 ip_address, // the IP address shared by the web sites - Int32 http_port, // usually: 80 - Int32 https_port, // usually: 443 - String ssl_certificate_common_name, - List(Web_Site) web_sites, // web sites to be started - Var(Bool) shutdown_required - ) = - with get_description = (Web_Site ws) |-> description(ws)(http_port,https_port), - with http_server_r = - start_http_server(ip_address,http_port, - map(get_description,web_sites), - load_denial_of_service_info), - with https_server_r = - start_https_server(ip_address,https_port, - ssl_certificate_common_name, - map(get_description,web_sites), - load_denial_of_service_info), - if http_server_r is ok(http_server) - then - ( - if https_server_r is ok(https_server) - then - ( - start_http_servers_tasks(map(get_description,web_sites), - [http_server,https_server], - 600); // period of 10 minutes - delegate - delete_states_loop( - map((Web_Site ws) |-> - (site_directory(description(ws)(http_port,https_port))+ - "/states",delete_out_of_date(ws)), - web_sites), - 3600*24*3, // keep out of date states 3 days - now, - http_server, - https_server, - shutdown_required), - ok(http_server,https_server) - ) - else cannot_bind_to_port(https_port) - ) - else - ( - if https_server_r is ok(https_server) - then cannot_bind_to_port(http_port) - else cannot_bind_to_port(http_port,https_port) - ). - - -public define One - start_web_sites - ( - Int32 ip_address, // the IP address shared by the web sites - Int32 http_port, // usually: 80 - Int32 https_port, // usually: 443 - String ssl_certificate_common_name, - List(Web_Site) web_sites, // web sites to be started - Var(Bool) shutdown_required - ) = - if (Start_Web_Sites_Result)start_web_sites(ip_address, - http_port, - https_port, - ssl_certificate_common_name, - web_sites, - shutdown_required) is - { - cannot_bind_to_port(n) then print("Cannot bind to port: "+n+"\n"), - cannot_bind_to_port(n,m) then print("Cannot bind to ports: "+n+", "+m+"\n"), - ok(s1,s2) then print("Servers started.\n") - }. - - - - - - *** [5] HTML Formating. - - We need to translate HTML elements as defined above into actual HTML text. - - Actioners require special informations, which must be transmitted when needed by the - 'format' functions: - - - the 'common name', which is used for URLs, - - the HTTP/HTTPS port number, - - the 'state name', which must be transmitted when the actioner is clicked upon, - - the 'form name' (if any) to which the actioner refers. - - If the actioner is off form, and if it refers to a form, the name of that form is - already known by the actioner. On the contrary, if the actioner is 'in form', it refers - implicitly to the form containing it. The name of that form is transmitted to the - 'format' functions called from within the formating of that form. - - - - - *** [5.1] The type 'HTML_Any($T)'. - - The type 'HTML_Any($T)' gathers elements which may be put anywhere in the page. The - parameter $T becomes either 'HTML_Off_Form' or 'HTML_In_Form'. - -type HTML_Any($T): - any_text (List(Text_Option), String the_text), - any_preformated (List(Text_Option), String), - any_paragraph (List(Text_Option), String the_text), - any_image (String url), - any_image (String url, Int32 width, Int32 height), - any_table (List(Table_Option), List(HTML_Row($T))), - any_center ($T), - any_mail_to (String email, $T element), - any_scroller (Int32 width, Int32 height, - Int32 content_width, Int32 content_height, - $T content), - any_fixed_size (HTML_Size width, HTML_Size height, $T content), - any_fixed_size_2 (HTML_Size width, HTML_Size height, String name_of_HTML_file), - any_actioner (Actioner_Connection, - Actioner_Target, - Actioner_Aspect, - String action_name, - List((String,String)) extra_ops, - List(Actioner_Local_Action), - Maybe(String) form_name), - any_foreign_link (List(Text_Option), String url, String name), - any_private_download (String abs_path, String name, String extra_ext, - Maybe((String,List((String,String))))), - any_div (List(DIV_Option), $T element), - any_div_empty (List(DIV_Option)), - any_coreattrs (List(CoreAttrs)). - - - - *** [5.2] Formating a color. - - RGB colors are formatted as '#rrggbb' where rr, gg and bb are two characters - hexadecimal values. - -define String - html_format - ( - RGB color - ) = - if color is rgb(r,g,b) then - "#" + hexadecimal(word8_to_int32(r),2) - + hexadecimal(word8_to_int32(g),2) - + hexadecimal(word8_to_int32(b),2). - - - The following is a very arbitrary definition of the opposite color. The thing which is - important is that it is far from the original, so that characters in 'opposite' color - are clearly visible over the original. - -define RGB - opposite - ( - RGB color - ) = - if color is rgb(r,g,b) then - with r1 = word8_to_int32(r), - with g1 = word8_to_int32(g), - with b1 = word8_to_int32(b), - rgb(truncate_to_word8(255-r1), - truncate_to_word8(255-g1), - truncate_to_word8(255-b1)). - - - - - *** [5.3] Creating buttons. - - We want to be able to create buttons in the form of a pair of images (rollovers) - automatically. We use the JPEG interface, because for the time being Anubis cannot - handle other kinds of images. - - - Computing printed text length. - - define Int32 - printed_text_width - ( - Word8 -> Int32 char_size, - List(Word8) l - ) = - if l is - { - [] then (Int32) 0, - [h . t] then char_size(h) + 1+ printed_text_width(char_size,t) - }. - - define Int32 - printed_text_width - ( - SystemFont font, - String s - ) = - printed_text_width((Word8 c) |-> word8_to_int32(width(get_char_info(font,c))), - explode(s)). - - - - Converting RGB to RGBA. - -define RGBA - to_rgba - ( - RGB color - ) = - if color is rgb(r,g,b) then rgba(r,g,b,255). - - - Drawing a 'relief'. - - define One - draw_relief - ( - RGBAImage dest, - RGBA color, - Int32 contrast, - Int32 x, - Int32 y, - Int32 width, - Int32 height - ) = - with l = lighten(color,contrast), - d = darken(color,contrast), - draw_rectangle(dest,rect(x,y,x+width,y+1),l); - draw_rectangle(dest,rect(x,y+1,x+1,y+height),l); - draw_rectangle(dest,rect(x+width-1,y+1,x+width,y+height),d); - draw_rectangle(dest,rect(x+1,y+height-1,x+width-1,y+height),d). - - - Creating a button background. - - define RGBAImage - create_button_background - ( - RGBA color, - Int32 width, - Int32 height - ) = - with result = create_rgba_image(width,height,color), - draw_relief(result,color,100,0,0,width,height); - draw_relief(result,color,70,1,1,width-2,height-2); - draw_relief(result,color,55,2,2,width-4,height-4); - draw_relief(result,color,35,3,3,width-6,height-6); - draw_relief(result,color,20,4,4,width-8,height-8); - draw_relief(result,color,10,5,5,width-10,height-10); - draw_relief(result,color,5,6,6,width-12,height-12); - result. - - - Drawing the text over the background. - - define One - draw_button_text - ( - RGBAImage image, - String text, - Int32 text_index, - Int32 pixel_x, - Int32 y, - Rectangle clip, - RGBA color, - SystemFont font, - ) = - if nth(text_index,text) is - { - failure then unique, - success(c) then - with cw = draw_system_character(image,clip,pixel_x,y,font,word8_to_int32(c),color), - draw_button_text(image,text,text_index+1,pixel_x+cw+1,y,clip,color,font) - }. - - define One - draw_button_text - ( - RGBAImage image, - String text, - Int32 text_width, - RGBA light_color, - RGBA dark_color, - SystemFont font - ) = - with image_width = width(image), - image_height = height(image), - x_pos = (image_width-text_width)>>1, - clip = rect(0,0,image_width,image_height), - new_light_color = lighten(light_color,150), - new_dark_color = darken(dark_color,40), - draw_button_text(image, text, 0, x_pos+2, 16, clip, new_dark_color, font); - draw_button_text(image, text, 0, x_pos, 14, clip, new_light_color, font). - - - The next function creates the two images for a button. The information given is the - main color of the button, the text of the button and the minimal width (in pixels) of - the button. The function does not create the button if the images already exist. The - two images are stored in the directory 'site_directory/buttons'. The names of the files - are of the form: - - bxxxx_off.jpg - bxxxx_on.jpg - - where the prefix 'b' is to avoid leading '-' which may perturb UNIX commands (like - 'rm'), and where 'xxxx' is created from the given informations by the formula: - - xxxx = web_arg_encode(sha1((color,text,width))) - - Hence, distinct informations give distinct file names. - - - define String // returns xxxx - create_button_images - ( - String site_directory, - RGBA color, - String text, - Int32 width, - SystemFont font - ) = - with xxxx = web_arg_encode(sha1((color,text,width))), - buttons_dir = site_directory+"/public/buttons", - off_filepath = buttons_dir+"/b"+xxxx+"_off.jpg", - on_filepath = buttons_dir+"/b"+xxxx+"_on.jpg", - if file_exists(on_filepath) - then xxxx - else with - text_width = printed_text_width(font,text), - button_width = max(width,text_width+12), - button_height = (Int32)20, - light_color = lighten(color,60), - very_light_color = lighten(light_color,30), - dark_color = darken(color,40), - background_off = - create_button_background(color,button_width,button_height), - background_on = - create_button_background(light_color,button_width,button_height), - - draw_button_text(background_off,text,text_width,very_light_color,dark_color,font); - draw_button_text(background_on, text,text_width,very_light_color,dark_color,font); - forget(write_image_to_JPEG_file(to_JPEG(background_off), - off_filepath, - 100)); - forget(write_image_to_JPEG_file(to_JPEG(background_on), - on_filepath, - 100)); - xxxx. - - - - - - *** [5.4] Formating an actioner. - - An actioner works as follows. Assume first that it refers to a form. When it is clicked - upon, the actioner puts (via 'onMouseDown') the URL into the 'action' attribute of the - form, and submits the form, using the JavaScript command 'form_name.submit()'. If the - actioner does not refer to a form, it fires the URL directly via 'href', because in - that case, the actioner is always an tag. - - The URL itself is composed using the connection sort (same, http or https), the common - name and port number (if needed), the state name, the action name, and the extra - operands, which are put into a query string. It may look like this: - - http://common_name:port/?s=state_name&a=action_name&oname=value... - - Each extra operand is a pair of strings: (name,value). It is formated as: - - &oname=value - - -define String - format_extra_operands - ( - List((String,String)) l - ) = - if l is - { - [ ] then "", - [h . t] then if h is (n,v) then - "&o"+n+"="+v+format_extra_operands(t) - }. - - - It seams that the standard requires "&" instead of "&" ! - - In case the target is another window, we need to format the options for this window. - -define String - format - ( - List(Other_Window_Option) l - ) = - if l is - { - [ ] then "", - [h . t] then if h is - { - resizable then "resizable", - scrollbars then "scrollbars", - width(w) then "width="+w, - height(h) then "height="+h - } + if t is [ ] then "" else (","+format(t)) - }. - - - - Formating choices for a "] - } - ], - - button(url_off,url_on) then - [ if action is - { - url(u) then ["", - "\"",url_off,"\"", - "" - ], - - button(url_off,url_on,w,h) then - [ if action is - { - url(u) then ["", - "\"",url_off,"\"", - "" - ], - - immediate_selector(name,size,choices) then - [ if action is - { - url(u) then ["" - ] - }. - - -define Printable_tree - format_local_popup_button - ( - CommonInfo cinfo, - Actioner_Aspect aspect, - Int32 n, - ) = - if cinfo is info(common_name,http_port,https_port,site_dir,secret) then - [ "", - "", - if aspect is - { - link(opt,text) then [text], - push_button(opt, text) then [text], - button(url_off,url_on) then - [ - "\"",url_off,"\"", - ], - - button(url_off,url_on,w,h) then - [ - "\"",url_off,"\"", - ], - - immediate_selector(name,size,choices) then alert, - - }, - ""]. - - - - - - *** [5.5] Formating a private download link. - - We get the absolute path of the file to be downloaded, and the name under which it - should appear to the client. The function 'format_private_download' creates an - hypertext link for downloading the file. The secured mecanism of private download is - used. This function is called by the function which formats HTML_Any($T) elements. - - -define Printable_tree - format_private_download - ( - CommonInfo cinfo, - String sn, // state name - String abs_path, // absolute file path on server - String name, // name of file as it appears in the browser - String extra, // extra extension - Maybe((String,List((String,String)))) action - - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with private_download_directory = site_directory+"/private_download", - with auth = make_authorization(site_directory,secret,abs_path), - [ - "", - name, - "" - ]. - - - - - *** [5.6] Formating rows and cells in a table. - -define Int32 - percent - ( - Int32 p - ) = - if p < 0 then 0 else if p > 100 then 100 else p. - - - - - Formating cell options. - - - -define String - format - ( - BackgroundOption o - ) = - if o is - { - repeat then "", - repeat_horizontal then "; background-repeat: repeat-x", - repeat_vertical then "; background-repeat: repeat-y", - no_repeat then "; background-repeat: no-repeat", - center then "; background-position: center top" - }. - -define String - format - ( - List(BackgroundOption) l - ) = - if l is - { - [ ] then "", - [h . t] then format(h)+format(t) - }. - - -define String - format - ( - List(Cell_Option) options - ) = - if options is - { - [ ] then "", - [h . t] then - if h is - { - left then " align=left", - h_center then " align=center", - right then " align=right", - top then " valign=top", - v_center then " valign=middle", - bottom then " valign=bottom", - base_line then " valign=baseline", - background_color(c) then " bgcolor=\""+html_format(c)+"\"", - background_image(n,o) then " style=\"background: url("+n+")"+format(o)+"\"", - width(w) then " width=\""+w+"\"", - percentage_width(n) then " width=\""+percent(n)+"%\"", - height(h) then " height="+h, - columns(n) then " colspan="+n, - rows(n) then " rowspan="+n, - nowrap then " nowrap" - } - + format(t) - }. - - - Normalizing a list of cell options (horizontal position must be specified; the default - is 'left'). - -define List(Cell_Option) - normalize - ( - List(Cell_Option) l - ) = - if member(l,left) then l else - if member(l,h_center) then l else - if member(l,right) then l else - [left . l]. - - - Formating cells in a row. - -define Printable_tree - format - ( - List(HTML_Cell($T)) cells, - $T -> Printable_tree format_element - ) = - if cells is - { - [ ] then [ ], - [h . t] then if h is cell(options,element) then - ["", - format_element(element), - "" - . format(t,format_element)] - }. - - - Formating the rows in a table. - -define Printable_tree - format - ( - List(HTML_Row($T)) rows, - $T -> Printable_tree format_element, - ) = - if rows is - { - [ ] then [ ], - [h . t] then if h is row(options,cells) then - ["", - format(cells,format_element), - "" - . format(t,format_element)] - }. - - - -define Printable_tree - format1 - ( - List(TextAreaOption) l - ) = - if l is - { - [] then [], - [h . t] then if h is - { - disabled then [" disabled " . format1(t)] - read_only then [" readonly " . format1(t)] - wrap_lines then [" wrap " . format1(t)] - } - }. - -define Printable_tree - format - ( - List(TextAreaOption) l - ) = - if member(l,wrap_lines) - then format1(l) - else [" wrap=off " . format1(l)]. - - - *** [5.7] Formating elements which may be put anywhere. - - The function below involves the parameter $T which is later instantiated as - 'HTML_In_Form' or as 'HTML_Off_Form'. Now, since there are dictinct 'format' functions - for these two types, and because formating of tables requires recursive calls of such - functions, it is necessary to provide the 'format' function to be called recursively as - an argument. Putting naively a call to 'format' will not work, because the compiler - will look for a function able to format data of type $T (which is at that time distinct - from any other type, including our two types). Such a function does not exist. Hence - the function to be called for formating elements must be passed as a functional - argument (called 'format_element' below). Actually, what we pass is a function taking - a unique argument of type $T. Other informations (like the name of the state) are - already in the function by way of full functionality. - - - Formating text options. They are formated in CSS syntax, to be used within a - 'style=...'. - -define String - format - ( - List(Text_Option) l - ) = - if l is - { - [ ] then "", - [h . t] then if h is - { - size(n) then "font-size:"+n+"pt", - font(fn) then "font-family:"+fn, - color(c) then if c is rgb(r,g,b) then - "color:rgb("+word8_to_int32(r)+","+word8_to_int32(g)+","+word8_to_int32(b)+")", - italic then "font-style:italic", - oblique then "font-style:oblique", - small_capitals then "font-variant:small-caps", - bold then "font-weight:bold", - underlined then "text-decoration:underline", - left_justified then "text-align:left", - right_justified then "text-align:right", - justified then "text-align:justify", - line_through then "text-decoration:line-through", - nowrap then "white-space:nowrap", - class(class_name)then " class=\"" +class_name +"\"" - } + if t is [ ] then "" else ("; "+format(t)) - }. - - - - Formating table options. - -define String - format - ( - List(Table_Option) l, - Bool border_seen - ) = - if l is - { - [ ] then if border_seen then "" else " border=0 cellspacing=0 cellpadding=0", - [h . t] then if h is - { - background_color(c) then " bgcolor=\""+html_format(c)+"\""+format(t,border_seen), - background_image(url) then " background="+url+format(t,border_seen), - border(o,top,i,c) then " border="+o+" cellspacing="+top+" cellpadding="+i+ - //" bordercolor="+format(c)+ - format(t,true), - width(w) then " width=\""+w+"\""+format(t,border_seen), - percentage_width(p) then " width=\""+percent(p)+"%\""+format(t,border_seen), - } - }. - - - - - -define Printable_tree - format_scroller - ( - String sn, - Int32 width, - Int32 height, - Int32 content_width, - Int32 content_height, - Int32 idnum, // identifying the scroller - $T content, - $T -> Printable_tree format_element - ) = - [ - "", - "", - "", - "", - "", - "", - (if content_width > width then - [ - "", - "", - "", - ] else [ ]), - "
", - "
", - "
", - format_element(content), - "
", - "
", - "
", - "", - "", - "", - "
\"sroll
\"scroll
", - "
", - "", - "", - "", - "", - "", - "
\"scroll\"scroll
", - "
" - ]. - - - - define Printable_tree - popup_topbar - ( - CommonInfo cinfo, - String title, - RGB color, - Int32 width, - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with xxxx = web_arg_encode(sha1((title,color,width))), - path = site_directory+"/public/buttons/t"+xxxx+".jpg", - result = (Printable_tree)["\"button\""], - if file_exists(path) then result else - with col = to_rgba(color), - bg = create_button_background(col,width,20), - very_light_color = lighten(col,70), - dark_color = darken(col,40), - title_width = printed_text_width(font,title), - draw_button_text(bg,title,title_width,very_light_color,dark_color,font); - forget(write_image_to_JPEG_file(to_JPEG(bg),path,100)); - result. - - - define Printable_tree - popup_close_button - ( - CommonInfo cinfo, - RGB color, - String div_name, - String state_var_name, - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with xxxx = web_arg_encode(sha1(color)), - path_on = site_directory+"/public/buttons/c"+xxxx+"_on.jpg", - path_off = site_directory+"/public/buttons/c"+xxxx+"_off.jpg", - result = (Printable_tree)["\"button\""], - if file_exists(path_on) then result else - with col = to_rgba(color), - title = "x", - title_width = printed_text_width(font,title), - bg_on = create_button_background(lighten(col,30),20,20), - bg_off = create_button_background(col,20,20), - very_light_color = lighten(col,70), - dark_color = darken(col,40), - draw_button_text(bg_on,title,title_width,very_light_color,dark_color,font); - draw_button_text(bg_off,title,title_width,very_light_color,dark_color,font); - forget(write_image_to_JPEG_file(to_JPEG(bg_on),path_on,100)); - forget(write_image_to_JPEG_file(to_JPEG(bg_off),path_off,100)); - result. - - - - - - The function below formats a datum of type 'HTML_Any($T)'. - -define Printable_tree - format - ( - CommonInfo cinfo, - String sn, // state_name - Var(Int32) ic_v, - HTML_Any($T) element, - $T -> Printable_tree format_element, // able to format a datum of type $T - Bool is_https, - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - if element is - { - any_text(opts,t) then - ["",t,""], - any_preformated(opts,s) then - ["
",s,"
"], - //["
",s,"
"], - any_paragraph(opts,t) then - ["

",t,"

"], - any_image(url) then - ["\"",url,"\""], - any_image(url,w,h) then - ["\"",url,"\""], - any_table(opts,rows) then - ["",format(rows,format_element),"
"], - any_center(e) then - ["
",format_element(e),"
"], - any_mail_to(email,elem) then - ["",format_element(elem),""], - any_scroller(w,h,cw,ch,c) then - format_scroller(sn,w,h,cw,ch,new_idnum(ic_v),c,format_element), - any_fixed_size(w,h,c) then - with url = create_secondary_document(site_directory,secret,sn,format_element,c,w), - ["", - "secondary document", - ""], - any_fixed_size_2(w,h,fn) then - with url = fn+"?zauth="+make_authorization(site_directory,secret, - fn), - ["", - "secondary document", - ""], - any_actioner(c,t,a,an,eo,ja,fn) then - format_actioner(cinfo,sn,c,t,a,an,eo,ja,fn,is_https), - any_foreign_link(options,url,name) then - ["",name,""], - any_private_download(url,name,extra_ext,action) then - format_private_download(cinfo,sn,url,name,extra_ext,action), - any_div(options, e) then - [format_div_option(options), format_element(e),""], - any_div_empty(options) then - [format_div_option(options), ""], - any_coreattrs(attributs) then - [format_coreattrs(attributs)] - }. - - - - - *** [5.8] Formating 'in form' elements. - - - - - -define Printable_tree - format - ( - CommonInfo cinfo, - String fn, // form_name - String sn, // state_name - Var(Int32) ic_v, // idnum counter variable - HTML_In_Form element, - Bool is_https, - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with format_element = (HTML_In_Form e) |-> format(cinfo,fn,sn,ic_v,e,is_https), - if element is - { - literal_pt(t) then t, - literal(t) then [t], - sequence(l) then flat(map(format_element,l)) - text(opts,t) then - format(cinfo,sn,ic_v,any_text(opts,t),format_element,is_https), - preformated(o,s) then - format(cinfo,sn,ic_v,any_preformated(o,s),format_element,is_https), - paragraph(opts,t) then - format(cinfo,sn,ic_v,any_paragraph(opts,t),format_element,is_https), - image(url) then - format(cinfo,sn,ic_v,any_image(url),format_element,is_https), - image(url,w,h) then - format(cinfo,sn,ic_v,any_image(url,w,h),format_element,is_https), - table(opts,rows) then - format(cinfo,sn,ic_v,any_table(opts,rows),format_element,is_https), - center(e) then - format(cinfo,sn,ic_v,any_center(e),format_element,is_https), - mail_to(a,e) then - format(cinfo,sn,ic_v,any_mail_to(a,e),format_element,is_https), - scroller(w,h,cw,ch,c) then - format(cinfo,sn,ic_v,any_scroller(w,h,cw,ch,c),format_element,is_https), - actioner(c,t,a,an,eo,ja) then - format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,success(fn)),format_element,is_https), - foreign_link(options,url,name) then - format(cinfo,sn,ic_v,any_foreign_link(options,url,name),format_element,is_https), - private_download(url,name,extra,action) then - format(cinfo,sn,ic_v,any_private_download(url,name,extra,action),format_element,is_https), - text_input(label_text, label, name,i,w) then - [ "", - ""], - //["  "], - password_input(label_text, label, name,w) then - [ "", - ""], - //["  "], - text_area(opts,n,i,w,h) then - [""], - file_upload(n,w) then - [""], - selector(n,s,cs) then - [""], - selector(n,s,cs,sd) then - [""], - selector_c(n,s,cs) then - [""], - selector_c(n,s,cs,sd) then - [""], - - radio_button(label_text,label,n,v,c) then - [ "", - ""], - check_box(label_text, label,n,c) then - [ "", - ""] - div(options, e) then - format(cinfo,sn,ic_v,any_div(options, e),format_element,is_https), - div_empty(options) then - format(cinfo,sn,ic_v,any_div_empty(options),format_element,is_https), - hidden(name, value) then - [""], - - }. - - - - - - - *** [5.9] Formating 'off form' elements. - - The encryption type 'multipart/form-data' is required for a form containing an upload. - - -define Bool - contains_an_upload - ( - HTML_In_Form form_content - ). - -define Bool - contains_an_upload - ( - HTML_Row(HTML_In_Form) row - ) = - mapor(contains_an_upload, - map(content,cells(row))). - - -define Bool - contains_an_upload - ( - HTML_In_Form form_content - ) = - if form_content is - { - literal_pt(t) then false, - literal(t) then false, - sequence(l) then mapor(contains_an_upload,l) - text(o,t) then false, - preformated(o,s) then false, - paragraph(o,t) then false, - image(u) then false, - image(u,w,h) then false, - table(o,rows) then mapor(contains_an_upload,rows), - center(e) then contains_an_upload(e), - mail_to(m,e) then false, // 'e' may but should not contain an upload - scroller(w,h,cw,ch,e) then contains_an_upload(e), - actioner(c,t,a,an,eo,ja) then false, - foreign_link(o,u,n) then false, - private_download(p,n,e,a) then false, - text_input(lt,l,n,i,w) then false, - password_input(lt,l,n,w) then false, - text_area(o,n,i,w,h) then false, - file_upload(n,w) then true, - selector(n,s,c) then false, - selector(n,s,c,p) then false, - selector_c(n,s,c) then false, - selector_c(n,s,c,p) then false, - radio_button(_,_,n,v,c) then false, - check_box(_,_,n,c) then false, - div(o,c) then false, - div_empty(o) then false, - hidden(_,_) then false - }. - -define String - enctype - ( - HTML_In_Form form_content - ) = - if contains_an_upload(form_content) - then " enctype=multipart/form-data" - else "". - - - -define Printable_tree - format - ( - CommonInfo cinfo, - String sn, // state_name - Var(Int32) ic_v, // 'idnum' counter variable - HTML_Off_Form element, - Bool is_https, - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with format_element = (HTML_Off_Form e) |-> format(cinfo,sn,ic_v,e,is_https), - if element is - { - literal_pt(t) then t, - literal(t) then [t], - sequence(l) then flat(map(format_element,l)), - text(opts,t) then - format(cinfo,sn,ic_v,any_text(opts,t),format_element,is_https), - preformated(o,s) then - format(cinfo,sn,ic_v,any_preformated(o,s),format_element,is_https), - paragraph(opts,t) then - format(cinfo,sn,ic_v,any_paragraph(opts,t),format_element,is_https), - image(url) then - format(cinfo,sn,ic_v,any_image(url),format_element,is_https), - image(url,w,h) then - format(cinfo,sn,ic_v,any_image(url,w,h),format_element,is_https), - table(opts,rows) then - format(cinfo,sn,ic_v,any_table(opts,rows),format_element,is_https), - center(e) then - format(cinfo,sn,ic_v,any_center(e),format_element,is_https), - mail_to(a,e) then - format(cinfo,sn,ic_v,any_mail_to(a,e),format_element,is_https), - scroller(w,h,cw,ch,c) then - format(cinfo,sn,ic_v,any_scroller(w,h,cw,ch,c),format_element,is_https), - fixed_size(w,h,c) then - format(cinfo,sn,ic_v,any_fixed_size(w,h,c),format_element,is_https), - fixed_size_2(w,h,fn) then - format(cinfo,sn,ic_v,any_fixed_size_2(w,h,fn),format_element,is_https), - actioner(c,t,a,an,eo,ja) then - format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,failure),format_element,is_https), - actioner(c,t,a,an,eo,ja,fn) then - format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,success(fn)),format_element,is_https), - foreign_link(options,url,name) then - format(cinfo,sn,ic_v,any_foreign_link(options,url,name),format_element,is_https), - private_download(url,name,extra,action) then - format(cinfo,sn,ic_v,any_private_download(url,name,extra,action),format_element,is_https), - label(n) then [""], - form(fn,attributs, c) then - [ - "
", - // action is set dynamically by - // the actioner using JavaScript - format(cinfo,fn,sn,ic_v,c,is_https), - "
" - ] - div(options, e) then - format(cinfo,sn,ic_v,any_div(options, e),format_element,is_https), - div_empty(options) then - format(cinfo,sn,ic_v,any_div_empty(options),format_element,is_https), - - }. - - - - - *** [5.10] Formating meta-tags. - -define Printable_tree - format_keywords - ( - List(String) l - ) = - if l is - { - [] then [ ], - [h . t] then if t is [] - then [h] - else [h , ", " . format_keywords(t)] - }. - - -define Printable_tree - format - ( - CommonInfo cinfo, - String state_name, - HTML_Meta m, - Bool is_https - ) = - if m is - { - keywords(l) then [""], - refresh(co,ta,an,delay) then - [""], - meta(n,c) then [""], - http_equiv(n,c) then [""], - generic_meta(l) then [" if p is (n,v) then [n,"=\"",v,"\" "], - l)), - ">"], - literal(s) then [s] - }. - - -define Printable_tree - format - ( - CommonInfo cinfo, - String state_name, - List(HTML_Meta) metas, - Bool is_https, - String charset - ) = - if metas is - { - [] then [format(cinfo,state_name,http_equiv("content-type", - "text/html; charset="+charset),is_https)], - [h . t] then [format(cinfo,state_name,h,is_https) - . format(cinfo,state_name,t,is_https,charset)] - }. - - -define Printable_tree - format - ( - Body_Option o - ) = - if o is - { - background_color(c) then [" bgcolor=\"" , (String)html_format(c), "\""], - background_image(n) then [" background=", n], - background_image(n,o) then [" style=\"background: url(",n,")",format(o),"\""] - - }. - - - -define Printable_tree - format - ( - List(Body_Option) l - ) = - if l is - { - [ ] then [ ], - [h . t] then [format(h) . format(t)] - }. - -define Printable_tree - add_css_files - ( - List(CSS_File) l - ) = - if l is - { - [ ] then [ ], - [h . t] then - [ ["\n" ] - . add_css_files(t)] - }. - -define Printable_tree - add_css_styles - ( - List(CSS_Style) css_styles - ) = - - if css_styles is - { - [] then [], - [_._] then [ "" - ] - }. - -define Printable_tree - format - ( - CommonInfo cinfo, - String state_name, - HTML_Page page, - Bool is_https, - String charset - ) = - if cinfo is info(common_name,http_port,https_port,site_directory,secret) then - with ic_v = var((Int32)0), - if page is - { - html_page(title,metas,css_styles, css_files, body) then - if body is body(options,element) then - [ doctype_w3c_header, - "", - "", - add_css_styles(css_styles), - add_css_files(css_files), - "", - "", - "",title,"", // put title - format(cinfo,state_name,metas,is_https,charset), // format the metas - "", - "", // format body options - //"
", - format(cinfo,state_name,ic_v,element,is_https), - //"
", - "", - "" - ] - }. - - - - - - + + + *Project* Anubis + + *Title* Making interactive Web sites. + + *Copyright* Copyright (c) Alain Prouté 2004-2005. + + + *Author* Alain Prouté + + *Revised* January 2005. + + + *Overview* + + In this file we propose simple tools for making well structured interactive and secured + web sites. + + + ----------------------------------- Table of Contents --------------------------------- + + * (1) Structure of a web site. + ** (1.1) Three sorts of data. + ** (1.2) How requests are handled. + ** (1.3) What web pages are made of. + ** (1.4) Actions. + ** (1.5) States. + + * (2) Carrying on. + ** (2.1) Describing your web sites. + ** (2.2) Directories on the server's disk. + ** (2.3) Starting your web sites. + + * (3) The HTML interface. + ** (3.1) Types used by the HTML interface. + ** (3.2) ``in form'' versus ``off form''. + ** (3.3) Defining your own style. + ** (3.4) Actioners and forms. + ** (3.5) Local popup. + + --------------------------------------------------------------------------------------- + + +read tools/basis.anubis +read CXM_common.anubis +read CXM_multihost_http_server.anubis +read CXM_mime.anubis + + + + * (1) Structure of a web site. + + First of all we need to explain what a web site should be made of. Ideally, the + visitor (also called the 'client') should see the web site working as any other + interactive computer software. So, it should be clear that a 'session' (i.e. a visit + to the web site, including the consultation of several pages) is some kind of + conversation between the visitor and the web site, and that the web site should + maintain a 'current state' of this conversation. At each new request (click) from the + visitor, this state must be updated. This whole conversation is called a 'session' and + should not be confused with a single request. + + + + ** (1.1) Three sorts of data. + + All the data needed for putting a web site at work may be dispatched into three + categories: + + 1. Constant data (data that never change). These data may be hard coded into the + Anubis source files of the web site. + + 2. Permanent data (data which always exist independantly of the users connected to + the web site). These data are normally recorded into data bases. + + 3. Session data (data which depend on a particular visitor and which exist only + during the time he visits the web site). These data are stored into so-called + 'states'. + + + It is important to determine which data belongs to which category. This is part of your + design decisions. + + + + ** (1.2) How requests are handled. + + We want to separate the following two functionalities (which are used at each request + (click) during a single session): + + - computing the new state from the previous state and from the client request, and + updating the data base, + + - computing the page to be sent to the client from the new current state and from + the informations in the data base. + + + The next picture shows the structure we have in mind: + + + request +---------+ HTML page (with a hidden state name) + .-------------------| client |<--------------. + | .-----------------| | | + | | previous state +---------+ | + | | name (if any) | + | | | client side + ............................................................................ + | | | server side + | | | + | | .-------------------. | + | | | previous state | | + V V V | | + +---------------+ +---------------+ +--------------+ + | compute state | | server's disk | | compute page | + +---------------+ +---------------+ +--------------+ + ^ | | ^ ^ ^ ^ ^ + | | | | | | | | + | | `--------------------+--------------------' | | + | | new state | | | + read | `------------------------+--------------------' | + write | new state name | + update V | + +-----------+ | + | data base |--------------------------------------------' + +-----------+ read only + + + When the client begins a session, there is no previous state. In this case, a default + 'initial state' is used instead. + + The data base may be updated by 'compute state' box, but should not be update by the + 'compute page' box. The 'compute page' box should be allowed only to read the data + base. + + In this file, all the above stuff is defined, except the 'compute state' and 'compute + page' boxes. You just have to provide the function for computing a new state (compute + state) and the function for computing the page (compute page) from the new state. You + don't have to worry about state names, saving and retrieving states and the like. + + + + + + ** (1.3) What web pages are made of. + + What the client can see in his browser's window may be called a 'page'. Within a page, + we have several sorts of components: + + - 'local' components, i.e. all components which do not open a connection, like + texts, images, etc... possibly using JavaScript programmation, + + - 'actioners', which, when clicked upon, open a connection with our web site; they + may appear as links or buttons, etc... + + - 'foreign links', which when clicked upon, open a connection with another web site + (or ours eventually). + + Of course, what an actioner does is just ask our web site to perform an action. To that + end, the actioner essentially sends the name of the action to be performed. However, it + may be necessary to provide additional informations which may be seen as 'operands' of + the action. In order to attach operands to an action, HTML provides the notion of + 'form'. Indeed, a form contains essentially a set of input fields into which the client + may put values for the required operands of the action, and a submit button, which is + the actioner itself. Notice that a single form may contain several submit buttons, + which simply means that there are several distincts actions taking the same set of + operands. + + Restrictions must be put on the use of all theses gadgets. Indeed, for example, + putting a form within another form is officially meaningless in HTML, and the client's + browser may be seriously disturbed by this. In this file, we propose an interface to + the HTML language, which forbids such meaningless things, simply by imposing a strict + typing of HTML concepts. + + Each web site may be accessible through two communication channels: + + - a non secured channel (HTTP), + - a secured channel (HTTPS). + + Nevertheless, the whole thing should be considered as a single web site. For example, + you may have a secured page, obtained through HTTPS, containing public images obtained + through HTTP. An actioner in a non secured page may open a secured connection, and + conversely. + + Summarizing, a web page is made of local elements, foreign links and actioners. + Actioners receive operands from forms, and they also choose to communicate through the + non secured or through the secured channel. + + + + +-------------------+ + | page | + | | +---------------+ + | +--------------+ | | next page | + | | form | | | (non secured) | + | | +----------+ | | HTTP | | + | | | actioner |---------------------------->| | + | | +----------+ | | +---------------+ + | | | | + | | +----------+ | | +---------------+ + | | | actioner |---------------------------->| next page | + | | +----------+ | | HTTPS | (secured) | + | | | | | | + | +--------------+ | | | + | | +---------------+ + | | + +-------------------+ + + + Notice that actioners need no be necessarily put into forms. In that case, they work as + ordinary links, but they still may receive operands as we shall see. + + + + + ** (1.4) Actions. + + The client opens a new connection with our web site whenever he clicks on an + actioner. The result is that a request is sent, essentially made of a list of 'web + arguments'. Each web argument is a pair (name,value). One of these web arguments, the + 'action' web argument (whose name is "a"), determines the action to be performed. The + other web arguments (not including "s", used to identify the state) are the operands + for this action. + + Hence, the 'compute state' box in the picture above, splits naturally into as many + sub-boxes as there are actions. For this reason, we define the following type for + representing actions (where '$State' is the type representing session informations): + +public type Web_Action($SessionTicket, $State): + http_action (String name, // name of action + (Maybe($State)) -> Bool allow, // true if action allowed + (HTTP_Info http_info, + List(Web_arg) web_args, // actually only 'operands' web arguments + Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it), + https_action (String name, // name of action + (Maybe($State)) -> Bool allow, // true if action allowed + (HTTP_Info http_info, + List(Web_arg) web_args, // actually only 'operands' web arguments + Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it), + http_https_action (String name, + (Maybe($State)) -> Bool allow, // true if action allowed + (HTTP_Info http_info, + List(Web_arg) web_args, + Maybe($State) state) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) do_it). + + 'http_action's are executed only under HTTP, and 'https_action's are executed only + under HTTPS. 'http_https_action's may be executed under both types of connections. + + Each action has a name, which is used to identify the action. Each action also has a + function 'allow' whose job is to verify that the action is allowed in the current + state, and a function 'do_it' for performing the action. The function 'do_it' receives + a lot of informations: + + - 'HTTP informations': + - the IP address of the client, + - the URI requested by the client (after redirection), + - the list of HTTP headers generated by the client's browser, + - the list of web arguments sent by the client (except "s" and "a"), + - the previous state (or the 'initial' or 'ticket expired' state if no previous + state can be found). + + In most cases, HTTP informations are not used. This is the reason why they are gathered + for simplicity into a unique datum of type 'HTTP_Info'. + +// For your convenience, we introduce the following simpler variants: +// +//public define Web_Action($State) +// http_action +// ( +// String name, +// $State -> Bool allow, +// (List(Web_arg),$State) -> $State do_it +// ) = +// http_action(name, +// (Maybe($State) ms) |-> if ms is +// { +// failure then true, +// success(s) then allow(s) +// }, +// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is +// { +// failure then (failure, []), +// success(s2) then (success(do_it(l,s2)), []) +// }). +// +//public define Web_Action($State) +// https_action +// ( +// String name, +// $State -> Bool allow, +// (List(Web_arg),$State) -> $State do_it +// ) = +// https_action(name, +// (Maybe($State) ms) |-> if ms is +// { +// failure then true, +// success(s) then allow(s) +// }, +// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is +// { +// failure then (failure, []), +// success(s2) then (success(do_it(l,s2)), []) +// }). +// +//public define Web_Action($State) +// http_https_action +// ( +// String name, +// $State -> Bool allow, +// (List(Web_arg),$State) -> $State do_it +// ) = +// http_https_action(name, +// (Maybe($State) ms) |-> if ms is +// { +// failure then true, +// success(s) then allow(s) +// }, +// (HTTP_Info h, List(Web_arg) l, Maybe($State) s) |-> if s is +// { +// failure then (failure, []), +// success(s2) then (success(do_it(l,s2)), []) +// }). + + + + When you define your web site, you must provide the list of all the actions of the + site. When a new state has been computed, a graphical representation of this state + must be sent to the client. To that end, you must provide a function (named below + 'compute_page') of type: + + $State -> HTML_Page + + where the type 'HTML_Page' (defined below in this file) abstractly represents HTML + pages. + +public type HTML_Page:... + + It should be clear that states and pages are deeply linked together. Indeed, we really + understand the page shown to the client as a representation of the current state of the + conversation between the client and the web site, but also containing informations + taken from the data bases. + + + + + + ** (1.5) States. + + Now, we explain how you can define the type (say 'State') to be used as an instance of + the type parameter '$State'. The following is just a suggestion. + + Each state determines a page (since 'compute_page' computes a page from a + state). However, some components of the state may be independant of the page. It may be + the case for example for the indication of the natural language used by the + client. Hence, a state should be made of (at least) two parts: + + - informations which are the same for all pages, + - informations which are particular to each page. + + For example, you could define: + + type Page: // one alternative per page, with particular informations + login(...), // in the components + main_page(...), + ...etc... + + Now, the type 'State' could be defined as follows: + + type State: + state(Language, // informations valid for all pages + ..., + Page). // informations particular to a page + + However, if you are making a secured web site within which clients should be identified + (by id and password), it may be a good idea to have two sorts of states, one for non + identified clients and one for identified clients. In this case, define the type + 'State' as follows (this is just a suggestion): + + type State: + non_identified(Language), + identified(String id, + Language, + Page). + + When a request arrives, check if the previous state is 'identified(...)' or + 'non_identified(...)', and don't provide access to certain pages to non identified + clients. This is required for security. + + Some more words on security. If your site needs to identify clients, define the + initial state as 'non_identified(...)'. Construct a 'login' page, and check the id and + password of the client. If the id and password are correct, then change the state of + the client to 'identified(...)'. No other action should be able to do that. Now, be + confident that clients cannot forge states. The only information they have is the name + of a state, not the state itself which is never sent over the network, but only stored + on the server's disk. The name of the state is constructed using strong cryptographical + methods (sha1). If everything (since the 'login' page) is performed under HTTPS, even + state names cannot be seen by a third party. So, if the system retrieves a previous + state of the form 'identified(...)', you can be confident that your client is well + identified, and you can send him confidential informations. + + States have a limited life time. It may happen that a client clicks on a button at a + time its state is out of date. In this case, this system considers that the new state + is a special state named 'ticket expired'. You must provide a function producing this + state when you describe your web site. The page corresponding to this state must just + inform the client that he/she waited a too long time before clicking on a button, and + has to restart (a new conversation) from the begining. + + + + * (2) Carrying on. + + ** (2.1) Describing your web sites. + + Before you may start your web site, you must describe it, i.e. produce a datum of the + opaque type 'Web_Site'. + +public type Web_Site:... + + Producing such a datum may be performed by: + +public define Web_Site + make_web_site_description + ( + List(String) common_names, // for example: ["www.our-business.com", + // "192.168.0.1"] + // the second one is just for testing + String site_directory, // where 'public' and other directories are + // located (should NOT end with '/') + One -> One init, + (HTTP_Info) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) initial_state, + ($State expired, + HTTP_Info, + List(Web_arg), + Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_expired_state, + (HTTP_Info, + List(Web_arg), + Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_lost_state, + List(Web_Action($SessionTicket, $State)) actions, + (Maybe($SessionTicket), Maybe($State)) -> HTML_Page compute_page, + Int32 timeout, // seconds (todo: minutes) + List(Redirection) redirections, + String charset, + List(String) journal_extensions, + List(String) journal_headers, + String authorization_secret, + List(MIME) known_mime_types, + (String action_name, + List(Web_arg) args)-> One before_send_file + ). + + + Explanations: + + 'common_names' is the list of names of the site (the name the browser must send as the + value of the 'Host' HTTP header in order to access the site must be in that list). Such + a name generally looks like this: + + www.somewhere.com + + If you are using HTTPS, you also have an 'X.509 SSL server certificate'. The name of + the site must be exactly the same as the name on the certificate (which is precisely + called the 'common name' in the X.509 jargon). If the two names do not match, the site + will still work, but the transaction will not be transparent to the client. His browser + will complain that the name of the certificate does not match the name of the site, and + he will have to accept the certificate manually. + + 'site_directory' is the absolute path to the directory where the files needed by the + site are located. Usually this directory looks like: + + my_anubis/web_sites/www.somewhere.com + + However, this information is not computed from 'common_name', so that you can change + the common name (for example temporarily, for networking reasons) without loosing + access to the files. + + 'ticket_expired_state(expired_state,http_info,lwa,is_https)' must produce the state + whose graphical representation is a page explaining to the user that its 'ticket' (or + 'session information') has expired, and that he/she must close all popup windows and + start a new session. The arguments of the function contain the previous (expired) + state and all current informations concerning the user. This arguments may be useful + for example for producing the expiration message in the language chosen by the user. + You can also (and this may be much smarter) send a 'ticket prolongation page' + (including a new login for example), and resume the same conversation, since you have + all the pertinent informations at hand. In the case the ticket is definitely lost, the + second fonction 'ticket_lost_state' is used. + + Notice that despite the fact that the parameter $State is involved in the arguments of + the above function, the type 'Web_Site' does not depend on this parameter. This allows + to produce lists of web site descriptions, where each description may be constructed + with a different instance of $State. This is required because distinct sites must have + distinct types of session informations. This is made possible by the fact that the + type is obscure, and the constructor replaced by a function which assembles + 'ticket_expired_state', ticket_lost_state', 'actions' and 'compute_page' into a single + entity not depending on $State. You should have a look to the private part of this file + if you want more precisions about this programming technique. + + 'charset' is a string which will determine the character encoding to be used by the + browser. Typically, this string is one of: "UTF-8", "ISO-8859-1", "Windows-1252", + etc... + + 'before_send_file' is a function which is executed just before the HTTP server sends a + file. It gets an action name and the web arguments received with the request for that + file. Notice that this action name and these web arguments may be put into a + 'private_download' element, and will come back to the server at the time of the + download. + + + + ** (2.2) Directories on the server's disk. + + The description of you site contains the name of the directory within which the + required files are located. This may be for example: + + my_anubis/web_sites/www.our-business.com/ + + This is called the 'site directory' (for the given site). Within the site directory, + the following directories are created by this program: + + states + public + journal + private_download + upload_temporary + + The directory 'states' is used for storing states (session informations). Out of date + states are automatically removed after some time. + + The tree rooted at 'public' contains files that the server is allowed to send to the + clients. For security reasons, the server never sends a file which is not within the + tree whose root is this 'public' directory (except for the 'private download' mecanism; + see 'web/multihost_http_server.anubis'). Also, the MIME type (see 'web/mime.anubis') + must have been recognized before the file may be sent. + + The directory 'journal' contains the jounal files. The roles of the remaining + directories 'private_download' and 'upload_temporary' is explained in + 'multihost_http_server.anubis', where you will also find further informations on + 'public' and 'journal'. + + + + + ** (2.3) Web servers parameters. + + The web servers have several parameters useful for administration. They are described + as follows: + + public type WebServersParameters: + wsparms(Var(Bool) shutdown_required, + + + + + ** (2.4) Starting your web sites. + + When you have described all your web sites (you may want to have several web sites, and + they are distinguished by their 'common name'), you may start them all together using + 'start_web_sites' below. This function returns a result of the following type: + +public type Start_Web_Sites_Result: + cannot_bind_to_port(Int32), + cannot_bind_to_port(Int32,Int32), + ok(Server http_server, + Server https_server). + + Indeed, it may happen that the system cannot bind (begin to listen) to one of the two + ports (or to both). The main reason is that another server is already listening on that + port. Another reason may be that 'anbexec' has not been correctly installed, i.e. that + the 's' bit has not been set for 'user' and 'group' (there is not such problem under + Windows). Also notice that the Linux kernel may need a rather long time (up to several + minutes) before liberating a listening port. Now, if the system can bind to the two + ports, the pair of the two servers is returned. Two tools are useful for manipulating + servers: + + shutdown of type Server -> One + is_down of type Server -> Bool + + They are defined in 'predefined.anubis' (together with the type 'Server'). + + +public define Start_Web_Sites_Result + start_web_sites + ( + Int32 ip_address, // the IP address shared by the web sites + Int32 http_port, // usually: 80 + Int32 https_port, // usually: 443 + String ssl_certificate_common_name, + List(Web_Site) web_sites, // web sites to be started + Var(Bool) shutdown_required + ). + + 'ip_address' is the IP address on which the two servers listen. If you put 0, the + servers listen on all the IP addresses of the machine. This may be useful if the + machine has several network interfaces. + + 'ssl_certificate_common_name' is the common name of the SSL certificate that 'anbexec' + loads when it starts. One instance of 'anbexec' cannot handle more than one SSL server + certificate. This is due to a problem of conception of SSL itself. See the book 'SSL + and TLS' by Eric Rescorla (at Addison Wesley) for more explanations. + + Notice that the number of servers is always 2, regardless of the number of web sites + you are starting. + + The dynamic variable 'shutdown_required' may be used to control the shutdown of the two + servers from within the web site (typically the administration part). The servers will + shutdown as soon as this variable contains 'true'. So you must provide a variable + containing 'false' otherwise your servers will not run. You may also use the primitive + 'must_restart' (see 'predefined.anubis') to control the restarting of your servers. + + + + + + + * (3) The HTML interface. + + We propose an interface to dynamic HTML. Dynamic HTML includes HTML, and a combination + of CSS (Cascading Style Sheet) and JavaScript techniques for making HTML elements more + reactive and attractive on the client side. + + + ** (3.1) Types used by the HTML interface. + + For easy reference, we gather below the definitions of all the types used by the HTML + interface, and we comment them immediately. + + +public type HTML_Size: + absolute(Int32), // in pixels + percentage(Int32). + + +public type Text_Option: + size(Int32), // size of character font to use + font(String), // name of character font to use (such as "helvetica",...) + color(RGB), // color to be used for characters + italic, + oblique, + small_capitals, + bold, + underlined, + left_justified, + right_justified, + justified, // justified on both sides + line_through, + nowrap, + class(String). //CSS class + + A list of 'Text_Option' must be given with each text you want to put in your page. + + This indicate the way of reading text. +public type Reading_Way: + ltr, //the text is readable from "Left To Right" like english + rtl. //the text is readable from "Right To Left" like arabic + + + +public type CoreAttrs: + id (String), + class (String), + style (String), + title (String). + +public type I18n: + lang (String), + dir (Reading_Way). + +public type DIV_Option: + id (String), + class (String), + style (String), + title (String), + lang (String), + dir (Reading_Way). + + + A list of 'DIV_Option' must be given with each DIV you want to put in your page. + + +public type Table_Option: + background_color(RGB), // applied to all cells in the table + background_image(String url), + border(Int32 width_of_outer_edge, // if not present, all values are 0 + Int32 width_of_top_of_relief, + Int32 width_of_inner_edge, + RGB border_color), + width(Int32), // sets a minimal width for the table + percentage_width(Int32). + + +public define Table_Option nude = border(0,0,0,rgb(0,0,0)). + + + A list of 'Table_Option' must be given with each table. + + +public type BackgroundOption: + repeat, // repeat the background in both directions + repeat_horizontal, // repeat the background only horizontally + repeat_vertical, // repeat the background only verticall + no_repeat, // don't repeat the background + center. + + +public type Cell_Option: + left, // put the content of the cell on the left + h_center, // center the content of the cell horizontally + right, // put the content of the cell on the right + top, // put the content of the cell upwards + v_center, // center the content of tye cell vertically, + bottom, // put the content of the cell downwards + base_line, // align the content vertically according to base lines + background_color(RGB), + background_image(String url, BackgroundOption), + width(Int32), // sets a minimal width for the cell + percentage_width(Int32), + height(Int32), // sets a minimal height for the cell + columns(Int32), // lets the cell span over several columns + rows(Int32), // lets the cell span over several rows + nowrap. // do not allow text wrapping within the cell + + A list of 'Cell_Option' must be given with each cell and each row in a table. Options + given with a row apply to all the cells in the row, but are superseded by options given + with cells, which apply only to the cell they are given with. + + +public type HTML_Cell($T): + cell(List(Cell_Option) options, $T content). + + The parameter $T is later instantiated either to 'HTML_In_Form' or to 'HTML_Off_Form', + depending on where you put your table (within a form or not within a form). For your + convenience, we define the following particular case: + +public define HTML_Cell($T) + cell + ( + $T content + ) = + cell([],content). + + + +public type HTML_Row($T): + row(List(Cell_Option) options, List(HTML_Cell($T)) cells). + + Same remark as for 'HTML_Cell($T)'. We define several convenience functions: + +public define HTML_Row($T) + row + ( + List(HTML_Cell($T)) cells + ) = + row([],cells). + +public define HTML_Row($T) + row + ( + HTML_Cell($T) cell + ) = + row([],[cell]). + +public type Actioner_Connection: + same, // use same type of connection as current page + http, // use non secured connection + https. // use secured connection + +public type Other_Window_Option: + resizable, // the new window may be resized by the client + scrollbars, // the new window has scrollbars + width(Int32), // the new window has the specified width + height(Int32). // the new window has the specified height + +public type Actioner_Target: + same, + same (String label), + other(String window_name, List(Other_Window_Option)). + +public type Actioner_Aspect: + link (List(Text_Option),String text), // hypertext link + push_button (List(CoreAttrs),String text), + button (String url_off, String url_on), // rollover button + button (String url_off, String url_on, Int32 w, Int32 h), // idem with size + immediate_selector (String name, Int32 size, List(String) choices). + + +public type Actioner_Local_Action: + close_window. + + +public define Actioner_Aspect + link + ( + String text + ) = + link([],text). + + +public define Actioner_Aspect + link + ( + List(Text_Option) options, + Int32 i + ) = + link(options,integer_to_string(i)). + +public define Actioner_Aspect + link + ( + Int32 i + ) = + link([],i). + +public define Actioner_Aspect + button + ( + String url_img + ) = + button(url_img,url_img). + + + + Actioners are explained in details below. + + +public type TextAreaOption: + disabled, + read_only, + wrap_lines. + +public type HTML_In_Form: + literal_pt (Printable_tree), + literal (String), + sequence (List(HTML_In_Form) items), + text (List(Text_Option), String the_text), + preformated (List(Text_Option), String), + paragraph (List(Text_Option), String the_text), + image (String url), + image (String url, Int32 width, Int32 height), + table (List(Table_Option), List(HTML_Row(HTML_In_Form))), + center (HTML_In_Form), + mail_to (String email, HTML_In_Form element), + scroller (Int32 width, Int32 height, + Int32 content_width, Int32 content_height, + HTML_In_Form content), + actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, + String action_name, List((String,String)) extra_ops, + List(Actioner_Local_Action)), + foreign_link (List(Text_Option), String url, String name), + private_download (String abs_path, String name, String extra_ext, + Maybe((String,List((String,String)))) action), + text_input (String label_text, String label, String name, String init, Int32 width), + password_input (String label_text, String label, String name, Int32 width), + text_area (List(TextAreaOption), String name, String init, Int32 width, Int32 height), + file_upload (String name, Int32 width), + selector (String name, Int32 size, List(String) choices), + selector (String name, Int32 size, List(String) choices, String selected), + // List((String,String)) = List((code,name)) where : + // name appears in selector + // code is the web-arg value + selector_c (String name, Int32 size, List((String,String)) choices), + selector_c (String name, Int32 size, List((String,String)) choices, String selected), + radio_button (String label_text, String label, String name, String value, Bool checked), + check_box (String label_text, String label, String name, Bool checked), + div (List(DIV_Option), HTML_In_Form content), + div_empty (List(DIV_Option)), + hidden (String name, String value). + + + 'HTML_In_Form' defines all the elements you may put within a form. We define a + convenience function: + +public define HTML_In_Form literal(Printable_tree t) = literal_pt(t). + +public define HTML_In_Form + foreign_link + ( + Int32 tsize, + String url, + String name + ) = + foreign_link([size(tsize)],url,name). + +public define HTML_In_Form + actioner + ( + Actioner_Connection conn, + Actioner_Target targ, + Actioner_Aspect asp, + String action_name, + List((String,String)) extra_ops + ) = + actioner(conn,targ,asp,action_name,extra_ops,[]). + + + +public define HTML_In_Form + text_area + ( + String name, + String init, + Int32 width, + Int32 height + ) = + text_area([],name,init,width,height). + +public define HTML_In_Form + table + ( + List(HTML_Row(HTML_In_Form)) rows + ) = + table([],rows). + + +public define HTML_In_Form + private_download + ( + String abs_path, + String name, + String extra_ext + ) = + private_download(abs_path,name,extra_ext,failure). + +public define HTML_In_Form + private_download + ( + String abs_path, + String name, + String extra_ext, + String action_name, + List((String,String)) args + ) = + private_download(abs_path,name,extra_ext,success((action_name,args))). + +public define HTML_In_Form + text + ( + String s + ) = + text([],s). + + + + +public type HTML_Off_Form: + literal_pt (Printable_tree), + literal (String), + sequence (List(HTML_Off_Form) items), + text (List(Text_Option), String the_text), + preformated (List(Text_Option), String), + paragraph (List(Text_Option), String the_text), + image (String url), + image (String url, Int32 width, Int32 height), + table (List(Table_Option), List(HTML_Row(HTML_Off_Form))), + center (HTML_Off_Form), + mail_to (String email, HTML_Off_Form element), + scroller (Int32 width, Int32 height, + Int32 content_width, Int32 content_height, + HTML_Off_Form content), + fixed_size (HTML_Size width, HTML_Size height, HTML_Off_Form content), + fixed_size_2 (HTML_Size width, HTML_Size height, String name_of_HTML_file), + actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, + String action_name, List((String,String)) extra_ops, + List(Actioner_Local_Action)), + actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect, + String action_name, List((String,String)) extra_ops, + List(Actioner_Local_Action), String form_name), + foreign_link (List(Text_Option), String url, String name), + private_download (String abs_path, String name, String extra_ext, + Maybe((String,List((String,String)))) action), + label (String name), + form (String form_name, List(CoreAttrs), HTML_In_Form content), + div (List(DIV_Option), HTML_Off_Form content), + div_empty (List(DIV_Option)). + + 'HTML_Off_Form' defines all the elements you may put outside any form. + + +public define HTML_Off_Form literal(Printable_tree t) = literal_pt(t). +public define HTML_Off_Form fixed_size(HTML_Size width, HTML_Size height, String name_of_HTML_file) + = fixed_size_2(width,height,name_of_HTML_file). + + +public define HTML_Off_Form + foreign_link + ( + Int32 tsize, + String url, + String name + ) = + foreign_link([size(tsize)],url,name). + + +public define HTML_Off_Form + actioner + ( + Actioner_Connection conn, + Actioner_Target targ, + Actioner_Aspect asp, + String action_name, + List((String,String)) extra_ops + ) = + actioner(conn,targ,asp,action_name,extra_ops,[]). + +public define HTML_Off_Form + table + ( + List(HTML_Row(HTML_Off_Form)) rows + ) = + table([],rows). + + + + We add two convenience functions for 'row'. The reason why we add two functions, one + for 'HTML_In_Form' and one for 'HTML_Off_Form', is that adding a schema with an + arbitrary '$T' creates too many ambiguities. This is due to the fact that, if we do so, + the arguments of the function do not refer to any of the types defined here. + +public define HTML_Row(HTML_In_Form) + row + ( + HTML_In_Form content + ) = + row([],[cell([],content)]). + +public define HTML_Row(HTML_Off_Form) + row + ( + HTML_Off_Form content + ) = + row([],[cell([],content)]). + + +public define HTML_Off_Form + private_download + ( + String abs_path, + String name, + String extra_ext + ) = + private_download(abs_path,name,extra_ext,failure). + + +public define HTML_Off_Form + private_download + ( + String abs_path, + String name, + String extra_ext, + String action_name, + List((String,String)) args + ) = + private_download(abs_path,name,extra_ext,success((action_name,args))). + +public define HTML_Off_Form + text + ( + List(Text_Option) lto, + Int32 i + ) = + text(lto,integer_to_string(i)). + + +public define HTML_Off_Form + text + ( + Int32 i + ) = + text([],i). + +public define HTML_Off_Form + text + ( + String s + ) = + text([],s). + + - Cell a gap between two other cells : + +public define HTML_Cell(HTML_Off_Form) + width_gap + ( + Int32 w + ) = + cell([width(w)],text([],"")). + +public define HTML_Cell(HTML_In_Form) + width_gap + ( + Int32 w + ) = + cell([width(w)],text([],"")). + + + - Row a gap between two other rows : + +public define HTML_Row(HTML_Off_Form) + height_gap + ( + Int32 h + ) = + row([],[cell([height(h)],text([],""))]). + +public define HTML_Row(HTML_In_Form) + height_gap + ( + Int32 h + ) = + row([],[cell([height(h)],text([],""))]). + + + Notice that the two types have alternatives in common (same name, same arguments types, + up to the value of the parameter $T), which correspond to elements which may be put + anywhere in the page. + + + +public type CSS_Style: + text_options(List(Text_Option)). + +public type CSS_File: + css_file(String file_name). + +define String + format + ( + List(Text_Option) l + ). + + + + +define Printable_tree + format_css_styles + ( + List(CSS_Style) l + ) = + if l is + { + [ ] then [ ], + [h . t] then + [ if h is + { + text_options(tos) then + [" body, span, p { ", format(tos), " }\n" ] + } + . format_css_styles(t)] + }. + + + +public type HTML_Meta: + keywords (List(String)), + refresh (Actioner_Connection connection, + Actioner_Target target, + String action_name, + Int32 delay), // in seconds + meta (String name, String content), + http_equiv (String name, String content), + generic_meta (List((String,String))), + literal (String). + + Meta tags are put in the 'head' of the HTML page. + + +public type Body_Option: + background_color (RGB), + background_image (String url), + background_image (String url, List(BackgroundOption)). + +public type HTML_Body: + body(List(Body_Option) options, HTML_Off_Form content). + + +public type HTML_Page: + html_page(String title, + List(HTML_Meta) meta_tags, + List(CSS_Style) styles, + List(CSS_File) css_files, + HTML_Body body). + +public define HTML_Page + html_page + ( + String title, + List(HTML_Meta) metas, + HTML_Body body + ) = + html_page(title,metas,[],[],body). + +public define HTML_Page + html_page + ( + String title, + List(HTML_Meta) metas, + List(CSS_Style) styles, + HTML_Body body + ) = + html_page(title, metas, styles, [], body). + + 'HTML_Page' represents the final product of the construction of a web page. + + + + + *** (3.2) ``in form'' versus ``off form''. + + There is a variety of HTML elements: texts, buttons, links, forms, inputs, etc... Some + of them may have a content, which is yet another HTML element (or several). Hence, it + is meaningful to say that an element is 'within' another one. Now, putting any element + within any other one may be meaningless. For example, an input element must be put + within a form (otherwise, it is useless), and a from within another form has no precise + meaning (and is forbidden by the HTML specification). + + Actually, the main criterium is ``within a form or not within a form''. So, HTML + elements in a given page are separated into two categories: those who are within a + form, and the others. Nevertheless, there are elements which may belong to both + categories, like images and texts. We want to make use of the strong typing mecanism of + Anubis in order to forbid non meaningful placement of elements. + + The type 'HTML_In_Form' defines elements to be put within forms. Similarly, + 'HTML_Off_Form' defines elements not to be put within forms. Both types are recursive, + and 'HTML_Off_Form' refers to 'HTML_In_Form' (via the 'form' alternative, of course), + but the two types are not cross recursive. This is the reason why it is impossible to + put a form within a form. In order to construct a web page, you essentially have to + produce a datum of type 'HTML_Off_Form' (maybe containing data of type 'HTML_In_Form'). + + In practice, you don't have to worry so much about these two types, because elements + which may be put anywhere are constructed for both types by functions with the same + name and the same arguments. Hence, for both types, you just write the same thing. You + are warned by the compiler only when you try to put an element at a place it is not + allowed. + + + + *** (3.3) Defining your own style. + + We provide generic tools for constructing HTML elements. However, your web site needs + to have a ``style''. + + To that end, you need to write down a set of ``styling functions'', using the tools + defined here. These styling functions allow the introduction of your colors and other + visual characteristics into the constructed elements once and for all. For example, you + may want all your texts to be rendered in the ``Helvetica'' font, in size 14 and using + some 'text_color'. You may write something like this: + + define RGB text_color = rgb(10,40,40). + + define HTML_Off_Form + text + ( + String the_text + ) = + text([font("helvetica"),size(14),color(text_color)], + the_text). + + (and the same one for type 'HTML_In_Form') so that in order to put a piece of text in a + page, you just write: + + text("... some text ...") + + and you don't have to provide the font, size and color for each text. If you want to + have several styles of text presentation, you just write several sets of such + convenience functions. This also suggests a trick. You may want for example different + colors for 'in form' texts and 'off form' texts. This may be achieved automatically by + defining two functions as above, with the same name and same argument type, but + returning either a 'HTML_In_Form' or a 'HTML_Off_Form'. + + If this preliminary work is well done, you will not waste your time later when you + concentrate on the actual informational content of your pages. + + This general principle should be applied to all sorts of elements. This is the best + thing to do in order to separate the functions defining the visual style from the + functions defining the informational content itself, so that changing the style without + changing the content becomes easy. This is also the best way for having a clean and + easily readable source for your web site. + + + + + + *** (3.4) Actioners and forms. + + We have gathered several notions from HTML into that of an 'actioner'. An actioner is + an HTML element which opens a connection to our server when clicked upon. Actioners may + have different visual aspects. They may look like hypertext links or like buttons + (rollovers), or even like selectors (with immediate action). In any case, their + behavior is the same: they open a connection to our server, and send a set of 'web + arguments', i.e. pairs 'name=value'. Among these web arguments, one of them denotes + the action to be performed, and the others should be considered as operands for this + action. Actually, the precise behavior of the actioner has several variants. + + The connection with the server may be secured (HTTPS) or non secured (HTTP). See the + type 'Actioner_Connection' above. + + You must also choose where the answer must be rendered. This may be in the same window + or in another window (or frame). If it is in another window, the name of that window + must be given. If the window does not exist, the browser will create it. Optionally, + you may give the dimensions of the new window and other characteristics. See the type + 'Actioner_Target' above. + + The actioner also has a visual aspect. See the type 'Actioner_Aspect' above. In the + case of a rollover button, you provide the URLs of two images (of the same size) + representing the button: + + url_off: to be used when the mouse is not over the button, + url_on: to be used when the mouse is over the button. + + You can also create rollover buttons without creating images. Just use the second + alternative named 'button'. The server creates the images automatically. + + The purpose of forms is just to give operands to actioners. If the actioner is placed + within a form, all the input elements which are within this form provide operands to + the actioner (except sometimes when they are not set by the client). If it is not put + within a form, the actioner gets no operand, except if the name of a form is explicitly + given, in which case the actioner gets all the inputs from that form as + operands. Furthermore, you may want to give extra operands to the actioner. This may be + useful for separating families of actioners with the same action name. Extra operands + 'name=value' must be given in the form of pairs '(name,value)'. + + Notice that the name of a form may be used by an actioner which is off the form, so as + to get the operands provided by this form. Also notice that several actioners may refer + to the same form, being either in the form, or referring to the form from the + outside. These actioners simply get the same set of operands, even if they correspond + to distinct actions. + + Input elements may be put only within a form. + + + + *** (3.5) Local popup. + + This element looks like a link or a rollover button. When this button is clicked upon, + a 'popup window' appears. Actually, this popup window is just a layer in the same HTML + page, which becomes suddenly visible. It is realized with a '
' HTML tag. In + particular, clicking on the button does not open any connection. This is why it is + called 'local'. The arguments have the following roles: + + Actioner_Aspect aspect of the button (same semantics as for actioners) + content content of the popup window + x, y, position of the popup window on the HTML page (not relative + to the button but to the page itself) + title title of the popup window + color color of the title bar and close button in the popup window. + A lightened version of this color is used for the background + of the popup window. + width width of the title bar + + + + + + --- That's all for the public part ! -------------------------------------------------- + + + + + + ----------------------------------- Table of Contents --------------------------------- + + *** [1] States. + *** [1.1] Saving and retrieving states. + *** [1.2] Deleting out of date states. + + *** [2] Tools. + *** [2.1] Directories. + *** [2.2] Secondary documents. + + *** [3] Managing web arguments. + *** [3.1] Prefixing web arguments names. + *** [3.2] Separating web arguments. + *** [3.3] Applying an action. + + *** [4] Web site descriptions and the 'awp handlers'. + *** [4.1] The type 'Web_Site'. + *** [4.2] Making a web site description. + *** [4.3] Starting the servers. + + *** [5] HTML Formating. + *** [5.1] The type 'HTML_Any($T)'. + *** [5.2] Formating a color. + *** [5.3] Creating buttons. + *** [5.4] Formating an actioner. + *** [5.5] Formating a private download link. + *** [5.6] Formating rows and cells in a table. + *** [5.7] Formating elements which may be put anywhere. + *** [5.8] Formating 'in form' elements. + *** [5.9] Formating 'off form' elements. + *** [5.10] Formating meta-tags. + + --------------------------------------------------------------------------------------- + + + + +public define String + doctype_w3c_header + = + "\n". + + + + + *** [1] States. + + We have to define functions for saving a state, retrieving a state, deleting out of + date states. We need one such function per web site. The types of the first two + functions depend on the parameter $State. This is not the case of the third one. The + fact that the instance of $State is variable from one web sites to the other implies + rather subtle manipulations using full functionality. + + + *** [1.1] Saving and retrieving states. + + Each state is saved into a file on the server's disk (in the directory represented by + the symbol 'state_directory', which is 'my_anubis/web_sites/common_name/states'). The + state is saved together with a time stamp whose value is obtained by adding the current + time to the given timeout for states. The state receives a name obtained by hashing + (using sha1) the content of the file itself, and then encoding the hash with + 'web_arg_encode'. The name of the file into which the state is saved is the + concatenation of "s" and the name of the state. + +read CXM_web_arg_encode.anubis + + The tool below constructs the function which is able to save a state on the server's + disk. + +define (Maybe($State) s) -> String // the function constructed returns the name of the state + make_save_state_function + ( + Int32 timeout, + String state_directory + ) = + (Maybe($State) mbs) |-> + if mbs is + { + failure then "", + success(s) then + with time_stamp = now+timeout, + to_be_saved = (time_stamp,s), + state_name = web_arg_encode(sha1(s)), + if save(to_be_saved,state_directory+"/s"+state_name) is ok + then state_name + else (print("Cannot create state file in '"+state_directory+"'.\n"); "") + }. + + + When a request arrives, we need to retrieve the previous state from the server's + disk. We receive the name of that state. If the state is out of date, the state file is + kept 3 days, and then deleted. + +type PreviousState($State): + not_found, // cannot retrieve the previous state + out_of_date($State), // the previous state is out of date + still_valid($State). // the previous state is still valid + +define (String state_name) -> PreviousState($State) + make_retrieve_state_function + ( + String state_directory + ) = + (String state_name) |-> + with file_path = state_directory+"/s"+state_name, + if (RetrieveResult((Int32,$State)))retrieve(file_path) is ok(d) + then ( + if d is (time_stamp,s) then + if time_stamp < now + then ( + forget(remove(file_path)); + out_of_date(s) + ) + else still_valid(s) // state has been successfully retrieved + ) + else not_found. + + + + *** [1.2] Deleting out of date states. + + We also need to delete states which are out of date and which will never be deleted by + the above method. This may be performed by a machine doing this periodically (say once + per states life time period). + +define (List(String) file_names) -> One + make_delete_out_of_date_states_function + ( + Maybe($State) dummy, + String state_directory + ) = + (List(String) file_names) |-df-> + if file_names is + { + [ ] then unique, + [h . t] then + with file_path = state_directory+"/"+h, + if (RetrieveResult((Int32,$State)))retrieve(file_path) is ok(d) + then ( + if d is (time_stamp,data) then + if time_stamp < now + then (forget(remove(file_path)); df(t)) + else df(t) + ) + else (forget(remove(file_path)); df(t)) + }. + + + The 'labelled arrow' |-df-> is documented in 'documentation/en/anubis_doc.txt'. + + Note: The argument 'dummy' (of type Maybe($State)) is not used in the body of the + function (hence its name). Nevertheless, it is required. Indeed, the Anubis compiler + does not accept a parameter in the body of a function (here the parameter is required + by the use of 'retrieve') if this parameter does not appear in the type of the + function. This is because this would create ambiguities that no explicit typing may + ever resolve. If you put a double slash in front of the declaration of 'dummy' above, + and if you compile this file, you will get a message like this one: + + Error in 'making_a_web_site.anubis', line 1300, column 7: + A definition may not contain parameters which are not present + in the declaration part (hidden parameters): + $State + + The type of the function constructed by 'make_delete_out_of_date_states_function' is + independant of the parameter $State. This is important because this allows to create + the list of such functions for all web sites. From this list, it is possible to call + the functions one after the other, so deleting out of date states for all web + sites. Actually, the next function receives a list of pairs (state_directory,function), + one for each web site. + +define One + delete_out_of_date_states // for all web sites + ( + List((String, List(String) -> One)) directories_and_functions + ) = + if directories_and_functions is + { + [ ] then unique, + [h . t] then if h is (state_directory,function) then + function(directory_list(state_directory,"s*")); + delete_out_of_date_states(t) + }. + + + The above function must be called periodically in a separate virtual machine. The + period we have choosen is (rather logically) the life time of states itself. This may + be achieved by an 'infinite' loop, using a 'sleep(timeout)'. However, the loop must not + be really infinite, because the servers may be shutdown. Hence, our loop must test + (rather frequently; say every second) if the servers are down. If they are, the loop + must be exited. + +define One + delete_states_loop + ( + List((String,List(String) -> One)) directories_and_functions, + Int32 timeout, + Int32 next_time, + Server http_server, + Server https_server, + Var(Bool) shutdown_required + ) = + if *shutdown_required + then (shutdown(http_server); shutdown(https_server)) + else unique; + if (is_down(http_server) & is_down(https_server)) + then unique + else if now > next_time + then + ( + delete_out_of_date_states(directories_and_functions); + delete_states_loop(directories_and_functions, + timeout, + now+timeout, + http_server, + https_server, + shutdown_required) + ) + else + ( + sleep(1000); // sleep just one second and try again + delete_states_loop(directories_and_functions, + timeout, + next_time, + http_server, + https_server, + shutdown_required) + ). + + The above loop must be run in a separate virtual machine. This will be done just after + the two servers are started. + + + + + + *** [2] Tools. + + *** [2.1] Directories. + + We need a tool for creating directories (if needed). + + (This tool has been moved to 'tools/basis.anubis'). + + + + + *** [2.2] Secondary documents. + + Some HTML elements (like '', '') cannot receive their content directly + from the current document, but only through an URL. For this reason, we implement a + mecanism for creating secondary documents on the fly. To that end we use the 'private + download' mecanism. + + A secondary document is formated by the same functions as the main document itself. The + next function takes an 'off form' element, creates the file containing the secondary + document in HTML format, and returns the URL at which the document will be available. + +define String + create_secondary_document + ( + String sd, // site directory + String as, // authorization_secret + String sn, // state name + $T -> Printable_tree format_element, + $T content, + HTML_Size width + ) = + with private_download_directory = sd+"/private_download", + hash = web_arg_encode(sha1(content)), + file_content = (Printable_tree) + [doctype_w3c_header, + "
", + format_element(content), + "
" + ], + file_name = "sd"+hash+".html", + file_path = private_download_directory+"/"+file_name, + if write_to_file(file_path,file_content) is + { + cannot_open_file then print("Cannot open file '"+file_path+"'.\n"); "", + write_error(n) then print("Error writing file '"+file_path+"'.\n"); "", + ok then file_name+"?zauth="+ + make_authorization(sd,as,private_download_directory+"/"+file_name) + }. + + + + + *** [2.3] Generating unique ids. + + In order to uniquely name object for JavaScript we generate unique ids from a counter. + +define Int32 + new_idnum + ( + Var(Int32) ic_v // 'idnum' counter variable + ) = + protect + with result = *ic_v+1, + ic_v <- result; + result. + + + + + + + *** [3] Managing web arguments. + + Web arguments are those pairs 'name=value' which are transmitted through the HTTP + protocol. We need precise naming conventions for these web arguments. + + + + *** [3.1] Prefixing web arguments names. + + We want to assign different roles to web arguments, and we also want to be able to + recognize its role directly from the name of a web argument. The name "s" is reserved + for the web argument whose value is the name of the current state. The name "a" is + reserved for the web argument whose value is the name of the action to be + performed. Other web arguments receive arbitrary names, and in order to avoid clashes, + these names are prefixed by: + + "p" for names of password inputs, + "o" for other web arguments + + The reason why password input names have a distinct prefix is that this allows the HTTP + server to hide the passwords on the console of the server and in the journal. + + + + + *** [3.2] Separating web arguments. + + When a new request arrives, we need to separate the web arguments, that is to say: + + - find the value of "s", and recover the corresponding state, + - find the value of "a", which is the name of the action to be performed, + - get the list of all the remaining web arguments (operands of the action). + + We must also determine if the previous state may be recovered. If it is not the case + (either because the previous state name is invalid, or the previous state is out of + date), we must check if there is an action name. Indeed, the presence of an action name + indicates that the user has clicked on one of our buttons or links. If on the contrary + there is no action name the user has just entered our address in his browser. In this + last case, we must send the first page of our site (maybe a 'login' page), but if there + is an action, we must send a page just saying that the session ticket has expired. If + the previous state is recovered and there is no action, the new state is the same as + the previous state. + + The result of the separation of the web arguments is of type: + +type Separated_Web_Args($State): + swa(Maybe(PreviousState($State)) previous_state, + Maybe(String) action_name, + List(Web_arg) operands). + + + + The next function constructs the function which separates the web arguments. + +define (List(Web_arg) lwa) -> Separated_Web_Args($State) + make_separate_web_args_function + ( + String state_directory, + String -> PreviousState($State) retrieve_state + ) = + (List(Web_arg) lwa) |-swaf-> + if lwa is + { + [ ] then + // + // no web arg found => no previous state and no action + // + swa(failure,failure,[]), + + [wa_1 . wa_others] then + // + // at least one web arg => + // separate other web args, and insert the first one as needed + // + if (Separated_Web_Args($State))swaf(wa_others) is + { + swa(ps1, // possible previous state + an1, // maybe an action name + op1) // operands so far + then + if wa_1 is + { + web_arg(n,v) then + with prefix = if substr(n,0,4) = "amp;" then substr(n,4,1) else substr(n,0,1), + name_start = (Int32)(if substr(n,0,4) = "amp;" then 5 else 1), + if prefix = "s" then + swa(success(retrieve_state(v)),an1,op1) else + if prefix = "a" then + swa(ps1,success(v),op1) else + if prefix = "t" then + swa(ps1,an1,[web_arg("target",v) . op1]) else + if prefix = "p" then + swa(ps1,an1,[web_arg(substr(n,name_start,length(n)-name_start),v) . op1]) else + if prefix = "o" then + swa(ps1,an1,[web_arg(substr(n,name_start,length(n)-name_start),v) . op1]) else + swa(ps1,an1,op1), + + upload(n,v,t) then + swa(ps1,an1,[upload(substr(n,1,length(n)-1),v,t) . op1]) + }} + }. + + + + + + + *** [3.3] Applying an action. + + When the web arguments are separated (and their names cleaned up from prefixes), we may + apply the action to the operands and the current state. We search for the action to be + applied in the list of actions. If no action is found, the new state is the same as + the previous state. Also, we deny the application of an HTTP action if the request + arrives through the HTTPS channel and conversely. + + +define (Maybe($State) previous, + String action_name, + HTTP_Info http_info, + List(Web_arg) lwa, + Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) + make_apply_action_function + ( + List(Web_Action($SessionTicket, $State)) actions + ) = + with f = + (Maybe($State) previous, + String action_name, + HTTP_Info http_info, + List(Web_arg) lwa, + Bool is_https, + List(Web_Action($SessionTicket, $State)) actions) |-f-> + if actions is + { + [ ] then (print("action '"+action_name+ + "' not found.\n"); (failure, previous, [])), + [ac1 . others] then if ac1 is + { + http_action(an,allow,do_it) then + if an = action_name + then if is_https + then (print("HTTP action '"+an+ + "' called through HTTPS (denied).\n"); + (failure, previous, [])) + else if allow(previous) + then do_it(http_info,lwa,previous) + else (failure, previous, []) + else f(previous,action_name,http_info,lwa,is_https,others), + + https_action(an,allow,do_it) then + if an = action_name + then if is_https + then if allow(previous) + then do_it(http_info,lwa,previous) + else (failure, previous, []) + else (print("HTTPS action '"+an+ + "' called through HTTP (denied).\n"); + (failure, previous, [])) + else f(previous,action_name,http_info,lwa,is_https,others), + + http_https_action(an,allow,do_it) then + if an = action_name + then if allow(previous) + then do_it(http_info,lwa,previous) + else (failure, previous, []) + else f(previous,action_name,http_info,lwa,is_https,others), + + } + }, + (Maybe($State) previous, + String action_name, + HTTP_Info http_info, + List(Web_arg) lwa, + Bool is_https) |-> + f(previous,action_name,http_info,lwa,is_https,actions). + + + + + + + + + *** [4] Web site descriptions and the 'awp handlers'. + + *** [4.1] The type 'Web_Site'. + + The type 'Web_Site_Description' is defined in 'web/multihost_http_server.anubis'. We + need another one, because, we have some extra informations to record for each site. + +public type Web_Site: + web_site((Int32,Int32) -> Web_Site_Description description, + List(String) -> One delete_out_of_date). + + + + + *** [4.2] Making a web site description. + + Below is the function which creates a web site description. It first creates (if + needed) the directories for the site, then constructs the tool functions for the site, + and the site handler. Finally, it constructs the web site description. + + We gather common (constant) informations in the following type: + +type CommonInfo: + info(String common_name, + Int32 http_port, + Int32 https_port, + String site_directory, + String authorization_secret + ). + + We need a forward declaration. + +public define Printable_tree + format + ( + CommonInfo cinfo, + String state_name, + HTML_Page page, + Bool is_https, + String charset + ). + + +define Printable_tree + format + ( + HTML_Size s + ) = + if s is + { + absolute(x) then ["\"",x,"\""], + percentage(x) then ["\"",x,"%\""] + }. + +define Printable_tree + top_redirection_page + ( + String common_name + ) = + [ doctype_w3c_header, + "", + "", + "" + ]. + +public define Web_Site + make_web_site_description + ( + List(String) common_names, // for example: ["www.our-business.com"] + String site_directory, + One -> One init, + (HTTP_Info) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) initial_state, + ($State expired, + HTTP_Info, + List(Web_arg), + Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_expired_state, + (HTTP_Info, + List(Web_arg), + Bool is_https) -> (Maybe($SessionTicket), Maybe($State), List(HTTP_header)) ticket_lost_state, + List(Web_Action($SessionTicket, $State)) actions, + (Maybe($SessionTicket), Maybe($State)) -> HTML_Page compute_page, + Int32 timeout, + List(Redirection) redirections, + String charset, + List(String) journal_extensions, + List(String) journal_headers, + String secret, + List(MIME) known_mime_types, + (String action_name, + List(Web_arg) args) -> One before_send_file + ) = + init(unique); + + + // + // make required directories (if needed) + // + with web_sites_directory = make_directory(my_anubis_directory+"/web_sites"), + base_directory = make_directory(site_directory), + state_directory = make_directory(site_directory+"/states"), + forget(make_directory(site_directory+"/public")); + // + // construct tool functions + // + with save_state = make_save_state_function(timeout,state_directory), + retrieve_state = make_retrieve_state_function(state_directory), + separate_web_args = make_separate_web_args_function(state_directory,retrieve_state), + apply_action = make_apply_action_function(actions), + // + // construct the site handler + // + site_handler = (Int32 http_port, Int32 https_port) |-> + ((String host_name, + HTTP_Info http_info, + List(Web_arg) lwa, + Bool is_https) |-> + ((List(HTTP_header),Printable_tree)) + if separate_web_args(lwa) is + { + swa(mb_previous_state,mb_action_name,operands) then + with state_and_headers = if mb_previous_state is + { + failure then + if mb_action_name is + { + failure then initial_state(http_info), + success(action_name) then + apply_action(failure,action_name,http_info,operands,is_https) + }, + success(previous_state) then if previous_state is + { + not_found then + if mb_action_name is + { + failure then initial_state(http_info), + success(_) then + ticket_lost_state(http_info,lwa,is_https) + }, + + out_of_date(state) then + ticket_expired_state(state,http_info,lwa,is_https), + + still_valid(state) then + if mb_action_name is + { + failure then (failure, success(state), []), + success(action_name) then + apply_action(success(state),action_name,http_info,operands,is_https) + } + } + }, + if state_and_headers is (session_ticket, mb_new_state, headers) then + with state_name = save_state(mb_new_state), + (headers, + format(info(host_name, http_port, https_port, site_directory, secret), + state_name, + compute_page(session_ticket, mb_new_state), + is_https, + charset)) + }), + // + // make the delete_out_of_date function + // + delete_out_of_date = + make_delete_out_of_date_states_function((Maybe($State))failure, + site_directory+"/states"), + // + // construct the web site description + // + web_site((Int32 http_port, Int32 https_port) |-> + web_site_description(common_names, + site_directory, + redirections, + charset, + journal_extensions, + journal_headers, + secret, + known_mime_types, + site_handler(http_port,https_port), + (List(Web_arg) lwa) |-> if separate_web_args(lwa) is + swa(mb_previous_state,mb_action_name,operands) then + if mb_action_name is + { + failure then unique + success(an) then before_send_file(an,operands) + }), + delete_out_of_date). + + + + + *** [4.3] Starting the servers. + +public define Start_Web_Sites_Result + start_web_sites + ( + Int32 ip_address, // the IP address shared by the web sites + Int32 http_port, // usually: 80 + Int32 https_port, // usually: 443 + String ssl_certificate_common_name, + List(Web_Site) web_sites, // web sites to be started + Var(Bool) shutdown_required + ) = + with get_description = (Web_Site ws) |-> description(ws)(http_port,https_port), + with http_server_r = + start_http_server(ip_address,http_port, + map(get_description,web_sites), + load_denial_of_service_info), + with https_server_r = + start_https_server(ip_address,https_port, + ssl_certificate_common_name, + map(get_description,web_sites), + load_denial_of_service_info), + if http_server_r is ok(http_server) + then + ( + if https_server_r is ok(https_server) + then + ( + start_http_servers_tasks(map(get_description,web_sites), + [http_server,https_server], + 600); // period of 10 minutes + delegate + delete_states_loop( + map((Web_Site ws) |-> + (site_directory(description(ws)(http_port,https_port))+ + "/states",delete_out_of_date(ws)), + web_sites), + 3600*24*3, // keep out of date states 3 days + now, + http_server, + https_server, + shutdown_required), + ok(http_server,https_server) + ) + else cannot_bind_to_port(https_port) + ) + else + ( + if https_server_r is ok(https_server) + then cannot_bind_to_port(http_port) + else cannot_bind_to_port(http_port,https_port) + ). + + +public define One + start_web_sites + ( + Int32 ip_address, // the IP address shared by the web sites + Int32 http_port, // usually: 80 + Int32 https_port, // usually: 443 + String ssl_certificate_common_name, + List(Web_Site) web_sites, // web sites to be started + Var(Bool) shutdown_required + ) = + if (Start_Web_Sites_Result)start_web_sites(ip_address, + http_port, + https_port, + ssl_certificate_common_name, + web_sites, + shutdown_required) is + { + cannot_bind_to_port(n) then print("Cannot bind to port: "+n+"\n"), + cannot_bind_to_port(n,m) then print("Cannot bind to ports: "+n+", "+m+"\n"), + ok(s1,s2) then print("Servers started.\n") + }. + + + + + + *** [5] HTML Formating. + + We need to translate HTML elements as defined above into actual HTML text. + + Actioners require special informations, which must be transmitted when needed by the + 'format' functions: + + - the 'common name', which is used for URLs, + - the HTTP/HTTPS port number, + - the 'state name', which must be transmitted when the actioner is clicked upon, + - the 'form name' (if any) to which the actioner refers. + + If the actioner is off form, and if it refers to a form, the name of that form is + already known by the actioner. On the contrary, if the actioner is 'in form', it refers + implicitly to the form containing it. The name of that form is transmitted to the + 'format' functions called from within the formating of that form. + + + + + *** [5.1] The type 'HTML_Any($T)'. + + The type 'HTML_Any($T)' gathers elements which may be put anywhere in the page. The + parameter $T becomes either 'HTML_Off_Form' or 'HTML_In_Form'. + +type HTML_Any($T): + any_text (List(Text_Option), String the_text), + any_preformated (List(Text_Option), String), + any_paragraph (List(Text_Option), String the_text), + any_image (String url), + any_image (String url, Int32 width, Int32 height), + any_table (List(Table_Option), List(HTML_Row($T))), + any_center ($T), + any_mail_to (String email, $T element), + any_scroller (Int32 width, Int32 height, + Int32 content_width, Int32 content_height, + $T content), + any_fixed_size (HTML_Size width, HTML_Size height, $T content), + any_fixed_size_2 (HTML_Size width, HTML_Size height, String name_of_HTML_file), + any_actioner (Actioner_Connection, + Actioner_Target, + Actioner_Aspect, + String action_name, + List((String,String)) extra_ops, + List(Actioner_Local_Action), + Maybe(String) form_name), + any_foreign_link (List(Text_Option), String url, String name), + any_private_download (String abs_path, String name, String extra_ext, + Maybe((String,List((String,String))))), + any_div (List(DIV_Option), $T element), + any_div_empty (List(DIV_Option)), + any_coreattrs (List(CoreAttrs)). + + + + *** [5.2] Formating a color. + + RGB colors are formatted as '#rrggbb' where rr, gg and bb are two characters + hexadecimal values. + +define String + html_format + ( + RGB color + ) = + if color is rgb(r,g,b) then + "#" + hexadecimal(word8_to_int32(r),2) + + hexadecimal(word8_to_int32(g),2) + + hexadecimal(word8_to_int32(b),2). + + + The following is a very arbitrary definition of the opposite color. The thing which is + important is that it is far from the original, so that characters in 'opposite' color + are clearly visible over the original. + +define RGB + opposite + ( + RGB color + ) = + if color is rgb(r,g,b) then + with r1 = word8_to_int32(r), + with g1 = word8_to_int32(g), + with b1 = word8_to_int32(b), + rgb(truncate_to_word8(255-r1), + truncate_to_word8(255-g1), + truncate_to_word8(255-b1)). + + + + + *** [5.3] Creating buttons. + + We want to be able to create buttons in the form of a pair of images (rollovers) + automatically. We use the JPEG interface, because for the time being Anubis cannot + handle other kinds of images. + + + Computing printed text length. + + define Int32 + printed_text_width + ( + Word8 -> Int32 char_size, + List(Word8) l + ) = + if l is + { + [] then (Int32) 0, + [h . t] then char_size(h) + 1+ printed_text_width(char_size,t) + }. + + define Int32 + printed_text_width + ( + SystemFont font, + String s + ) = + printed_text_width((Word8 c) |-> word8_to_int32(width(get_char_info(font,c))), + explode(s)). + + + + Converting RGB to RGBA. + +define RGBA + to_rgba + ( + RGB color + ) = + if color is rgb(r,g,b) then rgba(r,g,b,255). + + + Drawing a 'relief'. + + define One + draw_relief + ( + RGBAImage dest, + RGBA color, + Int32 contrast, + Int32 x, + Int32 y, + Int32 width, + Int32 height + ) = + with l = lighten(color,contrast), + d = darken(color,contrast), + draw_rectangle(dest,rect(x,y,x+width,y+1),l); + draw_rectangle(dest,rect(x,y+1,x+1,y+height),l); + draw_rectangle(dest,rect(x+width-1,y+1,x+width,y+height),d); + draw_rectangle(dest,rect(x+1,y+height-1,x+width-1,y+height),d). + + + Creating a button background. + + define RGBAImage + create_button_background + ( + RGBA color, + Int32 width, + Int32 height + ) = + with result = create_rgba_image(width,height,color), + draw_relief(result,color,100,0,0,width,height); + draw_relief(result,color,70,1,1,width-2,height-2); + draw_relief(result,color,55,2,2,width-4,height-4); + draw_relief(result,color,35,3,3,width-6,height-6); + draw_relief(result,color,20,4,4,width-8,height-8); + draw_relief(result,color,10,5,5,width-10,height-10); + draw_relief(result,color,5,6,6,width-12,height-12); + result. + + + Drawing the text over the background. + + define One + draw_button_text + ( + RGBAImage image, + String text, + Int32 text_index, + Int32 pixel_x, + Int32 y, + Rectangle clip, + RGBA color, + SystemFont font, + ) = + if nth(text_index,text) is + { + failure then unique, + success(c) then + with cw = draw_system_character(image,clip,pixel_x,y,font,word8_to_int32(c),color), + draw_button_text(image,text,text_index+1,pixel_x+cw+1,y,clip,color,font) + }. + + define One + draw_button_text + ( + RGBAImage image, + String text, + Int32 text_width, + RGBA light_color, + RGBA dark_color, + SystemFont font + ) = + with image_width = width(image), + image_height = height(image), + x_pos = (image_width-text_width)>>1, + clip = rect(0,0,image_width,image_height), + new_light_color = lighten(light_color,150), + new_dark_color = darken(dark_color,40), + draw_button_text(image, text, 0, x_pos+2, 16, clip, new_dark_color, font); + draw_button_text(image, text, 0, x_pos, 14, clip, new_light_color, font). + + + The next function creates the two images for a button. The information given is the + main color of the button, the text of the button and the minimal width (in pixels) of + the button. The function does not create the button if the images already exist. The + two images are stored in the directory 'site_directory/buttons'. The names of the files + are of the form: + + bxxxx_off.jpg + bxxxx_on.jpg + + where the prefix 'b' is to avoid leading '-' which may perturb UNIX commands (like + 'rm'), and where 'xxxx' is created from the given informations by the formula: + + xxxx = web_arg_encode(sha1((color,text,width))) + + Hence, distinct informations give distinct file names. + + + define String // returns xxxx + create_button_images + ( + String site_directory, + RGBA color, + String text, + Int32 width, + SystemFont font + ) = + with xxxx = web_arg_encode(sha1((color,text,width))), + buttons_dir = site_directory+"/public/buttons", + off_filepath = buttons_dir+"/b"+xxxx+"_off.jpg", + on_filepath = buttons_dir+"/b"+xxxx+"_on.jpg", + if file_exists(on_filepath) + then xxxx + else with + text_width = printed_text_width(font,text), + button_width = max(width,text_width+12), + button_height = (Int32)20, + light_color = lighten(color,60), + very_light_color = lighten(light_color,30), + dark_color = darken(color,40), + background_off = + create_button_background(color,button_width,button_height), + background_on = + create_button_background(light_color,button_width,button_height), + + draw_button_text(background_off,text,text_width,very_light_color,dark_color,font); + draw_button_text(background_on, text,text_width,very_light_color,dark_color,font); + forget(write_image_to_JPEG_file(to_JPEG(background_off), + off_filepath, + 100)); + forget(write_image_to_JPEG_file(to_JPEG(background_on), + on_filepath, + 100)); + xxxx. + + + + + + *** [5.4] Formating an actioner. + + An actioner works as follows. Assume first that it refers to a form. When it is clicked + upon, the actioner puts (via 'onMouseDown') the URL into the 'action' attribute of the + form, and submits the form, using the JavaScript command 'form_name.submit()'. If the + actioner does not refer to a form, it fires the URL directly via 'href', because in + that case, the actioner is always an tag. + + The URL itself is composed using the connection sort (same, http or https), the common + name and port number (if needed), the state name, the action name, and the extra + operands, which are put into a query string. It may look like this: + + http://common_name:port/?s=state_name&a=action_name&oname=value... + + Each extra operand is a pair of strings: (name,value). It is formated as: + + &oname=value + + +define String + format_extra_operands + ( + List((String,String)) l + ) = + if l is + { + [ ] then "", + [h . t] then if h is (n,v) then + "&o"+n+"="+v+format_extra_operands(t) + }. + + + It seams that the standard requires "&" instead of "&" ! + + In case the target is another window, we need to format the options for this window. + +define String + format + ( + List(Other_Window_Option) l + ) = + if l is + { + [ ] then "", + [h . t] then if h is + { + resizable then "resizable", + scrollbars then "scrollbars", + width(w) then "width="+w, + height(h) then "height="+h + } + if t is [ ] then "" else (","+format(t)) + }. + + + + Formating choices for a "] + } + ], + + button(url_off,url_on) then + [ if action is + { + url(u) then ["", + "\"",url_off,"\"", + "" + ], + + button(url_off,url_on,w,h) then + [ if action is + { + url(u) then ["", + "\"",url_off,"\"", + "" + ], + + immediate_selector(name,size,choices) then + [ if action is + { + url(u) then ["" + ] + }. + + +define Printable_tree + format_local_popup_button + ( + CommonInfo cinfo, + Actioner_Aspect aspect, + Int32 n, + ) = + if cinfo is info(common_name,http_port,https_port,site_dir,secret) then + [ "", + "", + if aspect is + { + link(opt,text) then [text], + push_button(opt, text) then [text], + button(url_off,url_on) then + [ + "\"",url_off,"\"", + ], + + button(url_off,url_on,w,h) then + [ + "\"",url_off,"\"", + ], + + immediate_selector(name,size,choices) then alert, + + }, + ""]. + + + + + + *** [5.5] Formating a private download link. + + We get the absolute path of the file to be downloaded, and the name under which it + should appear to the client. The function 'format_private_download' creates an + hypertext link for downloading the file. The secured mecanism of private download is + used. This function is called by the function which formats HTML_Any($T) elements. + + +define Printable_tree + format_private_download + ( + CommonInfo cinfo, + String sn, // state name + String abs_path, // absolute file path on server + String name, // name of file as it appears in the browser + String extra, // extra extension + Maybe((String,List((String,String)))) action + + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with private_download_directory = site_directory+"/private_download", + with auth = make_authorization(site_directory,secret,abs_path), + [ + "", + name, + "" + ]. + + + + + *** [5.6] Formating rows and cells in a table. + +define Int32 + percent + ( + Int32 p + ) = + if p < 0 then 0 else if p > 100 then 100 else p. + + + + + Formating cell options. + + + +define String + format + ( + BackgroundOption o + ) = + if o is + { + repeat then "", + repeat_horizontal then "; background-repeat: repeat-x", + repeat_vertical then "; background-repeat: repeat-y", + no_repeat then "; background-repeat: no-repeat", + center then "; background-position: center top" + }. + +define String + format + ( + List(BackgroundOption) l + ) = + if l is + { + [ ] then "", + [h . t] then format(h)+format(t) + }. + + +define String + format + ( + List(Cell_Option) options + ) = + if options is + { + [ ] then "", + [h . t] then + if h is + { + left then " align=left", + h_center then " align=center", + right then " align=right", + top then " valign=top", + v_center then " valign=middle", + bottom then " valign=bottom", + base_line then " valign=baseline", + background_color(c) then " bgcolor=\""+html_format(c)+"\"", + background_image(n,o) then " style=\"background: url("+n+")"+format(o)+"\"", + width(w) then " width=\""+w+"\"", + percentage_width(n) then " width=\""+percent(n)+"%\"", + height(h) then " height="+h, + columns(n) then " colspan="+n, + rows(n) then " rowspan="+n, + nowrap then " nowrap" + } + + format(t) + }. + + + Normalizing a list of cell options (horizontal position must be specified; the default + is 'left'). + +define List(Cell_Option) + normalize + ( + List(Cell_Option) l + ) = + if member(l,left) then l else + if member(l,h_center) then l else + if member(l,right) then l else + [left . l]. + + + Formating cells in a row. + +define Printable_tree + format + ( + List(HTML_Cell($T)) cells, + $T -> Printable_tree format_element + ) = + if cells is + { + [ ] then [ ], + [h . t] then if h is cell(options,element) then + ["", + format_element(element), + "" + . format(t,format_element)] + }. + + + Formating the rows in a table. + +define Printable_tree + format + ( + List(HTML_Row($T)) rows, + $T -> Printable_tree format_element, + ) = + if rows is + { + [ ] then [ ], + [h . t] then if h is row(options,cells) then + ["", + format(cells,format_element), + "" + . format(t,format_element)] + }. + + + +define Printable_tree + format1 + ( + List(TextAreaOption) l + ) = + if l is + { + [] then [], + [h . t] then if h is + { + disabled then [" disabled " . format1(t)] + read_only then [" readonly " . format1(t)] + wrap_lines then [" wrap " . format1(t)] + } + }. + +define Printable_tree + format + ( + List(TextAreaOption) l + ) = + if member(l,wrap_lines) + then format1(l) + else [" wrap=off " . format1(l)]. + + + *** [5.7] Formating elements which may be put anywhere. + + The function below involves the parameter $T which is later instantiated as + 'HTML_In_Form' or as 'HTML_Off_Form'. Now, since there are dictinct 'format' functions + for these two types, and because formating of tables requires recursive calls of such + functions, it is necessary to provide the 'format' function to be called recursively as + an argument. Putting naively a call to 'format' will not work, because the compiler + will look for a function able to format data of type $T (which is at that time distinct + from any other type, including our two types). Such a function does not exist. Hence + the function to be called for formating elements must be passed as a functional + argument (called 'format_element' below). Actually, what we pass is a function taking + a unique argument of type $T. Other informations (like the name of the state) are + already in the function by way of full functionality. + + + Formating text options. They are formated in CSS syntax, to be used within a + 'style=...'. + +define String + format + ( + List(Text_Option) l + ) = + if l is + { + [ ] then "", + [h . t] then if h is + { + size(n) then "font-size:"+n+"pt", + font(fn) then "font-family:"+fn, + color(c) then if c is rgb(r,g,b) then + "color:rgb("+word8_to_int32(r)+","+word8_to_int32(g)+","+word8_to_int32(b)+")", + italic then "font-style:italic", + oblique then "font-style:oblique", + small_capitals then "font-variant:small-caps", + bold then "font-weight:bold", + underlined then "text-decoration:underline", + left_justified then "text-align:left", + right_justified then "text-align:right", + justified then "text-align:justify", + line_through then "text-decoration:line-through", + nowrap then "white-space:nowrap", + class(class_name)then " class=\"" +class_name +"\"" + } + if t is [ ] then "" else ("; "+format(t)) + }. + + + + Formating table options. + +define String + format + ( + List(Table_Option) l, + Bool border_seen + ) = + if l is + { + [ ] then if border_seen then "" else " border=0 cellspacing=0 cellpadding=0", + [h . t] then if h is + { + background_color(c) then " bgcolor=\""+html_format(c)+"\""+format(t,border_seen), + background_image(url) then " background="+url+format(t,border_seen), + border(o,top,i,c) then " border="+o+" cellspacing="+top+" cellpadding="+i+ + //" bordercolor="+format(c)+ + format(t,true), + width(w) then " width=\""+w+"\""+format(t,border_seen), + percentage_width(p) then " width=\""+percent(p)+"%\""+format(t,border_seen), + } + }. + + + + + +define Printable_tree + format_scroller + ( + String sn, + Int32 width, + Int32 height, + Int32 content_width, + Int32 content_height, + Int32 idnum, // identifying the scroller + $T content, + $T -> Printable_tree format_element + ) = + [ + "", + "", + "", + "", + "", + "", + (if content_width > width then + [ + "", + "", + "", + ] else [ ]), + "
", + "
", + "
", + format_element(content), + "
", + "
", + "
", + "", + "", + "", + "
\"sroll
\"scroll
", + "
", + "", + "", + "", + "", + "", + "
\"scroll\"scroll
", + "
" + ]. + + + + define Printable_tree + popup_topbar + ( + CommonInfo cinfo, + String title, + RGB color, + Int32 width, + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with xxxx = web_arg_encode(sha1((title,color,width))), + path = site_directory+"/public/buttons/t"+xxxx+".jpg", + result = (Printable_tree)["\"button\""], + if file_exists(path) then result else + with col = to_rgba(color), + bg = create_button_background(col,width,20), + very_light_color = lighten(col,70), + dark_color = darken(col,40), + title_width = printed_text_width(font,title), + draw_button_text(bg,title,title_width,very_light_color,dark_color,font); + forget(write_image_to_JPEG_file(to_JPEG(bg),path,100)); + result. + + + define Printable_tree + popup_close_button + ( + CommonInfo cinfo, + RGB color, + String div_name, + String state_var_name, + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with xxxx = web_arg_encode(sha1(color)), + path_on = site_directory+"/public/buttons/c"+xxxx+"_on.jpg", + path_off = site_directory+"/public/buttons/c"+xxxx+"_off.jpg", + result = (Printable_tree)["\"button\""], + if file_exists(path_on) then result else + with col = to_rgba(color), + title = "x", + title_width = printed_text_width(font,title), + bg_on = create_button_background(lighten(col,30),20,20), + bg_off = create_button_background(col,20,20), + very_light_color = lighten(col,70), + dark_color = darken(col,40), + draw_button_text(bg_on,title,title_width,very_light_color,dark_color,font); + draw_button_text(bg_off,title,title_width,very_light_color,dark_color,font); + forget(write_image_to_JPEG_file(to_JPEG(bg_on),path_on,100)); + forget(write_image_to_JPEG_file(to_JPEG(bg_off),path_off,100)); + result. + + + + + + // The function below formats a datum of type 'HTML_Any($T)'. + +define Printable_tree + format + ( + CommonInfo cinfo, + String sn, // state_name + Var(Int32) ic_v, + HTML_Any($T) element, + $T -> Printable_tree format_element, // able to format a datum of type $T + Bool is_https, + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + if element is + { + any_text(opts,t) then + ["",t,""], + any_preformated(opts,s) then + ["
",s,"
"], + //["
",s,"
"], + any_paragraph(opts,t) then + ["

",t,"

"], + any_image(url) then + ["\"",url,"\""], + any_image(url,w,h) then + ["\"",url,"\""], + any_table(opts,rows) then + ["",format(rows,format_element),"
"], + any_center(e) then + ["
",format_element(e),"
"], + any_mail_to(email,elem) then + ["",format_element(elem),""], + any_scroller(w,h,cw,ch,c) then + format_scroller(sn,w,h,cw,ch,new_idnum(ic_v),c,format_element), + any_fixed_size(w,h,c) then + with url = create_secondary_document(site_directory,secret,sn,format_element,c,w), + ["", + "secondary document", + ""], + any_fixed_size_2(w,h,fn) then + with url = fn+"?zauth="+make_authorization(site_directory,secret, + fn), + ["", + "secondary document", + ""], + any_actioner(c,t,a,an,eo,ja,fn) then + format_actioner(cinfo,sn,c,t,a,an,eo,ja,fn,is_https), + any_foreign_link(options,url,name) then + ["",name,""], + any_private_download(url,name,extra_ext,action) then + format_private_download(cinfo,sn,url,name,extra_ext,action), + any_div(options, e) then + [format_div_option(options), format_element(e),""], + any_div_empty(options) then + [format_div_option(options), ""], + any_coreattrs(attributs) then + [format_coreattrs(attributs)] + }. + + + + + // *** [5.8] Formating 'in form' elements. + + + + + +define Printable_tree + format + ( + CommonInfo cinfo, + String fn, // form_name + String sn, // state_name + Var(Int32) ic_v, // idnum counter variable + HTML_In_Form element, + Bool is_https, + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with format_element = (HTML_In_Form e) |-> format(cinfo,fn,sn,ic_v,e,is_https), + if element is + { + literal_pt(t) then t, + literal(t) then [t], + sequence(l) then flat(map(format_element,l)) + text(opts,t) then + format(cinfo,sn,ic_v,any_text(opts,t),format_element,is_https), + preformated(o,s) then + format(cinfo,sn,ic_v,any_preformated(o,s),format_element,is_https), + paragraph(opts,t) then + format(cinfo,sn,ic_v,any_paragraph(opts,t),format_element,is_https), + image(url) then + format(cinfo,sn,ic_v,any_image(url),format_element,is_https), + image(url,w,h) then + format(cinfo,sn,ic_v,any_image(url,w,h),format_element,is_https), + table(opts,rows) then + format(cinfo,sn,ic_v,any_table(opts,rows),format_element,is_https), + center(e) then + format(cinfo,sn,ic_v,any_center(e),format_element,is_https), + mail_to(a,e) then + format(cinfo,sn,ic_v,any_mail_to(a,e),format_element,is_https), + scroller(w,h,cw,ch,c) then + format(cinfo,sn,ic_v,any_scroller(w,h,cw,ch,c),format_element,is_https), + actioner(c,t,a,an,eo,ja) then + format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,success(fn)),format_element,is_https), + foreign_link(options,url,name) then + format(cinfo,sn,ic_v,any_foreign_link(options,url,name),format_element,is_https), + private_download(url,name,extra,action) then + format(cinfo,sn,ic_v,any_private_download(url,name,extra,action),format_element,is_https), + text_input(label_text, label, name,i,w) then + [ "", + ""], + //["  "], + password_input(label_text, label, name,w) then + [ "", + ""], + //["  "], + text_area(opts,n,i,w,h) then + [""], + file_upload(n,w) then + [""], + selector(n,s,cs) then + [""], + selector(n,s,cs,sd) then + [""], + selector_c(n,s,cs) then + [""], + selector_c(n,s,cs,sd) then + [""], + + radio_button(label_text,label,n,v,c) then + [ "", + ""], + check_box(label_text, label,n,c) then + [ "", + ""] + div(options, e) then + format(cinfo,sn,ic_v,any_div(options, e),format_element,is_https), + div_empty(options) then + format(cinfo,sn,ic_v,any_div_empty(options),format_element,is_https), + hidden(name, value) then + [""], + + }. + + + + + + + *** [5.9] Formating 'off form' elements. + + The encryption type 'multipart/form-data' is required for a form containing an upload. + + +define Bool + contains_an_upload + ( + HTML_In_Form form_content + ). + +define Bool + contains_an_upload + ( + HTML_Row(HTML_In_Form) row + ) = + mapor(contains_an_upload, + map(content,cells(row))). + + +define Bool + contains_an_upload + ( + HTML_In_Form form_content + ) = + if form_content is + { + literal_pt(t) then false, + literal(t) then false, + sequence(l) then mapor(contains_an_upload,l) + text(o,t) then false, + preformated(o,s) then false, + paragraph(o,t) then false, + image(u) then false, + image(u,w,h) then false, + table(o,rows) then mapor(contains_an_upload,rows), + center(e) then contains_an_upload(e), + mail_to(m,e) then false, // 'e' may but should not contain an upload + scroller(w,h,cw,ch,e) then contains_an_upload(e), + actioner(c,t,a,an,eo,ja) then false, + foreign_link(o,u,n) then false, + private_download(p,n,e,a) then false, + text_input(lt,l,n,i,w) then false, + password_input(lt,l,n,w) then false, + text_area(o,n,i,w,h) then false, + file_upload(n,w) then true, + selector(n,s,c) then false, + selector(n,s,c,p) then false, + selector_c(n,s,c) then false, + selector_c(n,s,c,p) then false, + radio_button(_,_,n,v,c) then false, + check_box(_,_,n,c) then false, + div(o,c) then false, + div_empty(o) then false, + hidden(_,_) then false + }. + +define String + enctype + ( + HTML_In_Form form_content + ) = + if contains_an_upload(form_content) + then " enctype=multipart/form-data" + else "". + + + +define Printable_tree + format + ( + CommonInfo cinfo, + String sn, // state_name + Var(Int32) ic_v, // 'idnum' counter variable + HTML_Off_Form element, + Bool is_https, + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with format_element = (HTML_Off_Form e) |-> format(cinfo,sn,ic_v,e,is_https), + if element is + { + literal_pt(t) then t, + literal(t) then [t], + sequence(l) then flat(map(format_element,l)), + text(opts,t) then + format(cinfo,sn,ic_v,any_text(opts,t),format_element,is_https), + preformated(o,s) then + format(cinfo,sn,ic_v,any_preformated(o,s),format_element,is_https), + paragraph(opts,t) then + format(cinfo,sn,ic_v,any_paragraph(opts,t),format_element,is_https), + image(url) then + format(cinfo,sn,ic_v,any_image(url),format_element,is_https), + image(url,w,h) then + format(cinfo,sn,ic_v,any_image(url,w,h),format_element,is_https), + table(opts,rows) then + format(cinfo,sn,ic_v,any_table(opts,rows),format_element,is_https), + center(e) then + format(cinfo,sn,ic_v,any_center(e),format_element,is_https), + mail_to(a,e) then + format(cinfo,sn,ic_v,any_mail_to(a,e),format_element,is_https), + scroller(w,h,cw,ch,c) then + format(cinfo,sn,ic_v,any_scroller(w,h,cw,ch,c),format_element,is_https), + fixed_size(w,h,c) then + format(cinfo,sn,ic_v,any_fixed_size(w,h,c),format_element,is_https), + fixed_size_2(w,h,fn) then + format(cinfo,sn,ic_v,any_fixed_size_2(w,h,fn),format_element,is_https), + actioner(c,t,a,an,eo,ja) then + format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,failure),format_element,is_https), + actioner(c,t,a,an,eo,ja,fn) then + format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,success(fn)),format_element,is_https), + foreign_link(options,url,name) then + format(cinfo,sn,ic_v,any_foreign_link(options,url,name),format_element,is_https), + private_download(url,name,extra,action) then + format(cinfo,sn,ic_v,any_private_download(url,name,extra,action),format_element,is_https), + label(n) then [""], + form(fn,attributs, c) then + [ + "
", + // action is set dynamically by + // the actioner using JavaScript + format(cinfo,fn,sn,ic_v,c,is_https), + "
" + ] + div(options, e) then + format(cinfo,sn,ic_v,any_div(options, e),format_element,is_https), + div_empty(options) then + format(cinfo,sn,ic_v,any_div_empty(options),format_element,is_https), + + }. + + + + + *** [5.10] Formating meta-tags. + +define Printable_tree + format_keywords + ( + List(String) l + ) = + if l is + { + [] then [ ], + [h . t] then if t is [] + then [h] + else [h , ", " . format_keywords(t)] + }. + + +define Printable_tree + format + ( + CommonInfo cinfo, + String state_name, + HTML_Meta m, + Bool is_https + ) = + if m is + { + keywords(l) then [""], + refresh(co,ta,an,delay) then + [""], + meta(n,c) then [""], + http_equiv(n,c) then [""], + generic_meta(l) then [" if p is (n,v) then [n,"=\"",v,"\" "], + l)), + ">"], + literal(s) then [s] + }. + + +define Printable_tree + format + ( + CommonInfo cinfo, + String state_name, + List(HTML_Meta) metas, + Bool is_https, + String charset + ) = + if metas is + { + [] then [format(cinfo,state_name,http_equiv("content-type", + "text/html; charset="+charset),is_https)], + [h . t] then [format(cinfo,state_name,h,is_https) + . format(cinfo,state_name,t,is_https,charset)] + }. + + +define Printable_tree + format + ( + Body_Option o + ) = + if o is + { + background_color(c) then [" bgcolor=\"" , (String)html_format(c), "\""], + background_image(n) then [" background=", n], + background_image(n,o) then [" style=\"background: url(",n,")",format(o),"\""] + + }. + + + +define Printable_tree + format + ( + List(Body_Option) l + ) = + if l is + { + [ ] then [ ], + [h . t] then [format(h) . format(t)] + }. + +define Printable_tree + add_css_files + ( + List(CSS_File) l + ) = + if l is + { + [ ] then [ ], + [h . t] then + [ ["\n" ] + . add_css_files(t)] + }. + +define Printable_tree + add_css_styles + ( + List(CSS_Style) css_styles + ) = + + if css_styles is + { + [] then [], + [_._] then [ "" + ] + }. + +define Printable_tree + format + ( + CommonInfo cinfo, + String state_name, + HTML_Page page, + Bool is_https, + String charset + ) = + if cinfo is info(common_name,http_port,https_port,site_directory,secret) then + with ic_v = var((Int32)0), + if page is + { + html_page(title,metas,css_styles, css_files, body) then + if body is body(options,element) then + [ doctype_w3c_header, + "", + "", + add_css_styles(css_styles), + add_css_files(css_files), + "", + "", + "",title,"", // put title + format(cinfo,state_name,metas,is_https,charset), // format the metas + "", + "", // format body options + //"
", + format(cinfo,state_name,ic_v,element,is_https), + //"
", + "", + "" + ] + }. + + + + + + diff --git a/calexium_lib/web/CXM_multihost_http_server.anubis b/calexium_lib/web/CXM_multihost_http_server.anubis index cade615..6019c62 100644 --- a/calexium_lib/web/CXM_multihost_http_server.anubis +++ b/calexium_lib/web/CXM_multihost_http_server.anubis @@ -1,3692 +1,3692 @@ - - *Project* The Anubis Project - - *Title* A Multi Host HTTP/HTTPS Server - - *Copyright* Copyright (c) Alain Prouté 2003. - - - *Author* Alain Prouté - - - *Revised* August 2005. - - - - *Overviews* - In this file a HTTP/HTTPS server is defined, which is able to handle multiple hosts - (virtual hosts). It answers HTTP/HTTPS requests, sends files (images or any other kind - of file), constructs HTML pages on the fly using informations received from the client - (when the URI ends by '.awp'), handles uploading of files and redirections. It is - multitasking by itself, and can handle any number of sites and clients simultaneously. - It should better be used in conjunction with 'making_a_web_site.anubis' to be found in - the same directory. If you use 'web/making_a_web_site.anubis', you don't need to read - this file. - - - ----------------------------------- Table of Contents --------------------------------- - - *** (1) Multihosting and redirections. - *** (2) The incompatibility between SSL and virtual hosts. - *** (3) HTTP headers and web arguments. - *** (4) Site descriptions. - *** (5) Protection against denial of service attacks. - *** (6) Starting your HTTP and HTTPS servers. - *** (7) Private download. - *** (8) About web argument names. - *** (9) A web dispatcher. - - --------------------------------------------------------------------------------------- - - - - - *** (1) Multihosting and redirections. - - This HTTP/HTTPS server can handle several host (also called 'virtual hosts'), in other - words, you may have several sites on the same server, with the same IP address and same - port numbers, but distinct 'host names'. - - A HTTP request sent by a browser contains the following informations: - - - a 'host name', - - an URI (Uniform Resource Identifier), - - HTTP headers, - - web arguments (in the form 'name=value'). - - Actually, the host name is just the value of the HTTP header whose name is 'Host'. The - host name indicates which site is requested. Hence, it is the primary information for - branching to the right site. If there is no 'Host' HTTP header in the request, the - request is denied. - - From now on, we may assume that the host is determined, and consequently that we are - concerned by only one site. Each site has his own directories on the server's - disk. - - Each site also has a list of 'redirections'. A redirection is a triplet, like this one: - - redirect("/", "www.our-business.com", "/homepage.awp") - - meaning that if the host is "www.our-business.com", and if the requested URI is "/", - then the URI to be served is "/homepage.awp". 'redirect' is a constructor of the type - 'Redirection' defined in 'web/common.anubis'. - - Now, an URI may end by ".awp" (meaning 'Anubis Web Page') or not. If it does, the - server understands that an HTML page must be constructed on the fly, and to that end it - calls the 'awp handler' of the site. Otherwise, the URI must end by a known extension, - like ".jpg", ".png", ".txt", etc... and represents a file path relative to the - 'public' directory of the site. If these conditions are satisfied, the file is sent to - the client. Known extensions are recorded in 'web/mime.anubis'. - - - - - *** (2) The incompatibility between SSL and virtual hosts. - - Handling virtual hosts makes a problem under SSL (i.e. when using HTTPS), which is due - to the fact that the guys at Netscape who designed SSL probably did not have the - question of virtual hosts in mind. Indeed, the SSL handshake is completed before the - server can know about the value of the 'Host' HTTP header, so that it cannot know which - server certificate must be sent to the client. This makes a problem, because the - browser will not accept a certificate whose common name does not correspond to the name - of the requested host. The user will have to accept the certificate manually, which is - not good for the security image of the site. This problem has at least two solutions - (as far as Anubis is concerned). - - Solution 1. Arrange so that the network interface on which the server is listening - has at least as many different IP addresses as you have virtual hosts. Such - supplementary IP addresses are called 'IP Aliases'. In this case, start one HTTPS - server for each virtual host, each one listening on a different address. For the time - being, this method is applicable under Anubis only if you start as many instances of - 'anbexec' as you have virtual hosts, because each instance of 'anbexec' can handle only - one server certificate. Of course, getting IP aliases is another problem to be solved - with your Internet provider. - - Solution 2. We propose a simple solution, using only one server certificate (hence - only one instance of 'anbexec'). Since, we have only one server certificate, we must - introduce a notion of 'main host', i.e. a host containing all other 'virtual - hosts'. The unique server certificate belong to the main host, so that only the main - host is identified by the client. The client must trust the main host and be confident - that the main host redirects him to the right virtual host. Actually, the process will - be transparent to the client, except that the client will see the name of the main host - instead of the name of the virtual host in the 'location' field of the browser. - - So, assume that the name of main host is 'www.securedhost.com', and that the names of - the virtual hosts are: - - actual name simplified name - ----------------------------------------------------- - www.virtual1.com virtual1 - www.virtual2.com virtual2 - www.virtual3.com virtual3 - - Then the (confidential) document '/doc/my_document.pdf' on 'www.virtual2.com' will have - the URL: - - https://www.securedhost.com/virtual2/doc/my_document.pdf - - In order to work transparently, this solution must combine HTTP and HTTPS. Indeed, the - vitual host must have a first page reachable under HTTP, through the URL: - - http://www.virtual2.com/ - - The HTTP server will redirect this URL to the awp handler of virtual host 'virtual2'. - The handler of this virtual host is able to generate a first page containing the - following HTML meta: - - , - - so that the client is immediately redirected to the main host under HTTPS (hence - accepting tranparently the server certificate). The awp handler of 'virtual2' then - redirects this URL to the home page (maybe a login page) of 'virtual2'. - - See 'web/making_a_web_site.anubis' for the sequel of this story. - - - - - - *** (3) HTTP headers and web arguments. - - Each HTTP request which arrives on the server contains a request line followed by a - series of HTTP headers. Each HTTP header is a pair '(name,value)' assigning a value to - a name. The type 'HTTP_header' is defined in 'web/common.anubis'. - - The request may also have a 'body'. The body contains either 'web arguments' or - uploaded files (or both). The request line itself may also contain web arguments (in a - so-called 'query string'). Like HTTP headers, 'web arguments' are pairs - '(name,value)', but the difference is that these pairs are generated by the page within - which the client clicks, while HTTP headers are generated by the browser itself. The - type 'Web_arg' is defined in 'web/common.anubis'. It has two alternatives, one for - ordinary web arguments (pairs) and one for uploaded files. - -read CXM_common.anubis -read tools/basis.anubis -read CXM_mime.anubis - - - - *** (4) Site descriptions. - - The type HTTP_Info gathers informations comming along with the client's request. These - informations are rarely used for composing HTML pages. Nevertheless, they are at your - disposal. - -public type HTTP_Info: - http_info - ( - Int32 ip_address, // IP address of the client - String uri, // URI requested by the client - List(HTTP_header) http_headers, // HTTP headers sent by the client - One -> String generate_trust_ticket // may be used against denial of - // service attacks - ). - - - - Each site is described by a 'web site description', which is a datum of type - 'Web_Site_Description'. - -public type Web_Site_Description: - web_site_description( - List(String) common_names, - String site_directory, - List(Redirection) redirections, - String charset, - List(String) journal_extensions, - List(String) journal_headers, - String authorization_secret, - List(MIME) known_mime_types, - (String host_name, - HTTP_Info http_info, - List(Web_arg) lwa, - Bool is_https) -> (List(HTTP_header), - Printable_tree) awp_handler, - (List(Web_arg) lwa) -> One before_send_file). - - The component 'common_names' is the list of names of the site, like for example - "www.our-business.com". The reason why we have a list of common names instead of a - single common name, is that it may be useful to have a common name like "192.168.0.1" - for testing. - - 'charset' is a string which will determine the character encoding to be used by the - browser. Typically, this string is one of: "UTF-8", "ISO-8859-1", "Windows-1252", - etc... - - 'journal_extensions' is the list of URI extensions for which you want a log in the - journal (and on the console). When a request arrives, and if the extension is a member - of this list, a message is printed into the journal of the site including the date, the - IP address of the client, the complete HTTP request line. The HTTP headers whose name - is a member of 'journal_headers' are also printed in the journal. A reasonable minimum - for these two components is: - - [".awp"] for journal_extensions - ["user-agent"] for journal_headers - - 'authorization_secret' is a string which should just be unguessable. You may choose - something like (but don't choose this one !): - - "Hg8kJe42gCML9jNH-74" - - i.e. a sequence of characters typed at random, long enough to be unguessable. This is - used by the 'private download' mecanism, which is discussed later in this file. - - The component 'awp_handler' is a function of type: - - (String host_name, - HTTP_Info http_info, - List(Web_arg) web_args, - Bool is_https) -> Printable_tree - - ('Printable_tree' is a substitute for 'String' and is defined in - 'tools/basis.anubis'). This function is the 'awp handler' for the site. When the URI - ends by ".awp", this function is called, and the result (an HTML page) is sent to the - client over the connection. The last operand to this function is a boolean which is - 'true' when the requests arrives through the HTTPS channel, and 'false' when it arrives - through the HTTP channel. - - - - - - - - *** (5) Protection against denial of service attacks. - - We need to protect our servers against 'denial of service' attacks. The attack may be - send automatically from machines which are infested by viruses. In that case, our - server is saturated of connections (all virtual machines at work), but nothing is - comming on the connections. In order to avoid this problem, we propose the following: - - (1) Limit the number of simultaneous connections (say to 100). - (2) Close a connection if the request is not complete after say 10 seconds. - (3) Close the connection if the request is bigger than a given size (normal requests - are small except when there are uploaded files. - (4) Close the connection during the sending of the answer if the client is waiting - too much. - (5) Record all IP addresses with which we have encountered one of the problems above. - (6) Immediately close the connections if the IP address is in our list. - (7) Remove an address from the list only after 5 minutes of inactivity of this - address. - (8) Maintain a list of reliable IP addresses. - - Of course, all the above are approximative solutions which may in some circumstances - become either cumbersome or also partially block the system. So, it is needed to have a - set of dynamically modifiable parameters in order to master the behavior of this - mecanism. - - - Each dubious IP address is recorded together with its last activity time. - -public type DubiousIP: - dubious_ip (Int32 address, - Int32 last_activity). - - -public type DenialOfService: - denial_of_service(Var(Int32) max_connections, - Var(Int32) request_line_delay, // seconds - Var(Int32) headers_delay, - Var(Int32) answer_delay, - Var(List(DubiousIP)) list_of_dubious, - Var(List(Int32)) reliable_addresses). - - The informations in this set of variables are stored serialized into the file - 'my_anubis/web_sites/dos_info'. If this file does not exist a set if variables with - default values is created. The values are saved on the disk each time they are - modified. - -public define DenialOfService load_denial_of_service_info. - - - - *** (6) Starting your HTTP and HTTPS servers. - - When your web site descriptions are ready, you can start a pair of servers (a HTTP - server and a HTTPS server) for serving your web sites. Notice that there are always - two servers, regardless of the number of web sites, and that each web sites normally - uses the two servers. - - -public define StartServerResult - start_http_server - ( - Int32 ip_address, - Int32 http_port, - List(Web_Site_Description) web_sites, - DenialOfService dos - ). - -public define StartServerResult - start_https_server - ( - Int32 ip_address, - Int32 https_port, - String certificate_common_name, - List(Web_Site_Description) web_sites, - DenialOfService dos - ). - - The first argument 'ip_address' is the IP address on which the servers listen. If you - put 0, the servers listen on all adresses of the machine (which is useful if the - machine has several network interfaces). Otherwise, use the function 'ip_address' - defined in 'tools/basis.anubis' for composing a particular IP address. - - The next arguments are the port numbers for HTTP and HTTPS. The usual values are 80 and - 443, but you may have reasons to choose other values. - - The next argument is the list of your web site descriptions. All the sites described in - this list will be accessible on the server. - - The argument 'dos' is a set of dynamic variables containing the informations for - protecting the servers against denial of service attacks. - - - - - - - *** (7) Private download. - - It may happen that you want to propose private files for download. This means that such - a file could be downloaded only by the authorized person, and should not be seen by any - other one. This feature can be used only under HTTPS, not under HTTP. - - The file may be located anywhere on the server. Hence, the file has a complete absolute - path, like for example: - - /home/georges/my_documents/my_text.pdf - - which has nothing to do with the directories of the web server. Now, you may also want - to show another path or simply just a name to the client, not the actual absolute path - above, which may need to remain secret. So for example, the same file may appear to the - client as: - - informations.pdf - - The page must provide a link with an authorization. The authorization is just a web - argument, whose name is "zauth". The value of this web argument is computed by hashing - some secret string (known only from the programmer of the web site) with the absolute - path of the file. The HTTPS request will have the form: - - GET /informations.pdf?zauth=d38161f5b4e87e2d46e06ff8b3e233be563794d1 - - The server will search for a file named - - zd38161f5b4e87e2d46e06ff8b3e233be563794d1 - - (i.e. "z" concatenated with the value of the authorization) in the subdirectory - 'private_download' of the site directory. This file contains the absolute path of the - file, i.e: - - /home/georges/my_documents/my_text.pdf - - At that point, the server may hash the secret string and the absolute path together, to - check if the client is authorized to download the file. If it is the case, it sends the - file (the MIME type is declared as 'application/octet-stream' if it is not recognized). - The file is sent under the visible name. - - The server creates automatically the subdirectory 'private_download/' within the 'site - directory' (for each web site) if it does not already exist. Files in this directory - are deleted when they become too old (for example, after 3 days of life). - - Here is the function for computing the value of the authorization, and for making the - authorization file in 'private_download'. - -public define String - make_authorization - ( - String site_directory, - String authorization_secret, // known only by the programmer of the web site - String absolute_path // on server - ). - - See 'web/making_a_web_site.anubis' for the construction of the link for downloading. - - - - - - - - - *** (8) About web argument names. - - The server reserves the name "zauth" for the authorization in the private download - mecanism. Also, if the name of a web arguments begins by "p" (like 'password'), it does - not print the value of the web argument neither on the console or in the journal. A - good politics is to prefix all web arguments by letters distinct from 'p' and 'z'. This - method is used in 'web/making_a_web_site.anubis'. This will avoid clashes of names. - - - - - - - *** (9) A web dispatcher. - - For hosting several sites you may prefer another method which we now describe. We start - a HTTP server on port 80 (or on another port). This server is called the - ``dispatcher''. When a requests arrives, the dispatcher examines the ``host'' HTTP - header, so that it gets the name of the requested host. Then it sends to the client a - page like this one: - - - - - - - - - - where the URL represented by '...' is the URL of the requested site. This URL may have - the same IP address as the dispatcher, except that the port number is different. It may - also have a different IP address. - - The dispatcher uses the file 'my_anubis/web_sites/dispatcher.info'. This file contains - a serialized datum of type 'List(DispatcherInfo)'. - -public type DispatcherInfo: - site(String common_name, - Int32 http_port). - - The dispatcher does not write into this file. It reads it when it starts, and rereads - it each time the date of last modification of the file changes, so that the dispatcher - always has up to date data. The file may be managed (written and updated) by another - program. - - So, for each site, the dispatcher knows the common name (needed to recognize the 'host' - HTTP header), and the pair (ip_address,port) used by the actual site for HTTP. The - dispatcher does not worry about HTTPS. HTTPS must be managed by the actual site. - - The dispatcher is started by: - -public define One - start_web_dispatcher - ( - Int32 ip_address, // address for listening (typically 0) - Int32 port, // typically 80 - DenialOfService dos - ). - - A command line tool for managing the file 'my_anubis/web_sites/dispatcher.info' is also - provided: - -global define One - manage_web_dispatcher - ( - List(String) args - ). - - - - - - - - --- That's all for the public part ! -------------------------------------------------- - - - - - - - - ----------------------------------- Table of Contents --------------------------------- - - *** [1] Types which are private to this file. - - *** [2] Tools. - *** [2.1] Formating an error message. - *** [2.2] Converting IP addresses. - *** [2.3] Reading and unputting characters. - *** [2.4] Reading and discarding characters. - *** [2.5] Reading a character string. - *** [2.6] Padding integers with zeros. - *** [2.7] Converting web arguments to ASCII. - *** [2.8] Server description. - - *** [3] Managing the journal. - *** [3.1] Naming journal files. - *** [3.2] Formating HTTP headers. - *** [3.3] Formating web arguments. - *** [3.4] Formating the whole request. - *** [3.5] Putting it in the journal file (and on the console). - - *** [4] Reading the HTTP request. - *** [4.1] Skipping leading blanks. - *** [4.2] Reading a new line. - *** [4.3] Reading a 'word'. - *** [4.4] Separating the URI from the query string. - *** [4.5] Reading the web arguments. - *** [4.7] Reading the request line. - *** [4.8] Reading the HTTP headers. - *** [4.9] Getting the size of the request's body. - *** [4.10] Reading the body of the request. - - *** [5] Making the HTTP answer. - *** [5.1] Avoiding illegal URIs. - *** [5.2] Managing authorizations for downloading private files. - *** [5.3] Recognizing MIME types. - *** [5.4] Formating HTTP headers. - *** [5.5] Sending a file. - *** [5.6] Answering a www-url encoded request. - *** [5.7] Answering a multipart/form-data encoded request. - *** [5.7.1] Finding the boundary. - *** [5.7.2] Reading attributes from a multipart entity. - *** [5.7.3] Creating a temporary filename for an uploaded file. - *** [5.7.4] Saving an uploaded file under a temporary filename. - *** [5.7.5] Removing the path from a file name. - *** [5.7.6] Reading a multipart entity. - *** [5.8] Handling redirections. - *** [5.9] Answering both sorts of requests. - - *** [6] The HTTP/HTTPS servers. - *** [6.1] The HTTP request handler. - *** [6.2] Server's tasks. - *** [6.3] Starting the HTTP/HTTPS servers. - - *** [7] The web dispatcher. - *** [7.1] The dispatcher server. - *** [7.2] The dispatcher web site. - *** [7.3] Managing the info file. - - --------------------------------------------------------------------------------------- - - - - -read tools/basis.anubis -read tools/findstring.anubis -read tools/connections.anubis - - - - - - *** [1] Types which are private to this file. - - We use the following self-explanatory types. - -type Error: - cannot_read_from_connection, - not_get_or_post_request(String), - end_of_line_expected, - incorrect_content_length_value, - colon_expected, - timeout(Int32). - -type HTTP_RequestType: - get, - post. - -type HTTP_RequestLine: - request_line (HTTP_RequestType type, - String uri, - List(Web_arg) query_string). - -type EncodingType: - www_url, - multipart_form_data. - - - - - - *** [2] Tools. - - *** [2.1] Formating an error message. - - The next function formats an error message. - -define String - format - ( - Error msg - ) = - if msg is - { - cannot_read_from_connection then - "Cannot read from connection.\n", - not_get_or_post_request(s) then - "The request did not begin by 'GET' or 'POST': "+s+".\n", - end_of_line_expected then - "End of line expected.\n", - incorrect_content_length_value then - "Incorrect value for HTTP header 'Content-Length'.\n", - colon_expected then - "':' was expected.\n", - timeout(n) then - //"time out: "+n+"\n" - //"time out.\n" - "" - }. - - - - - - - *** [2.2] Converting IP addresses. - - We need two conversion functions for IP addresses: - - (Word8,Word8,Word8,Word8) --> Int32 ip_address - Int32 --> String ip_addr_to_string - - These conversions are defined in 'tools/basis.anubis'. - - - - - - - - - *** [2.3] Reading and unputting characters. - - We need a mecanism for unputting several characters (actually at least 3). This is - because when reading the client connection, we must sometimes go ahead several - characters, and virtually put them back into the connection, so that they can be - reread. Of course, we do not send them back to the client. We store them in a list - (hold by the variable 'unput_chars'), and we manage this list, so that characters may - be virtually put back in the connection (this is called 'unputting'). - -variable List(Word8) unput_chars = []. - - The most recently read one is the head of list. Fortunately, this variable is private - to this virtual machine (hence to this client). - - -define One - unput // unputting a character (add it in front of the list) - ( - Word8 character - ) = - unput_chars <- (List(Word8))[character . *unput_chars]. - - - -define One record_dubious_IP(Int32 addr,DenialOfService dos). - -variable Int32 sttm = 0. // contains the start time for this connection. - -define Result(Error,Word8) - record_dubious_connection - ( - Connection conn, - Int32 dead_line, - DenialOfService dos, - ) = - if remote_IP_address_and_port(conn) is (addr,port) then - record_dubious_IP(addr,dos); - print("Recording IP address "+ip_addr_to_string(addr)+ - " as dubious after "+(dead_line-*sttm)+" seconds. Total: "+ - length(*list_of_dubious(dos))+"\n"); - error(timeout(dead_line)). - - -define Result(Error,Word8) - read_one_byte - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - //if now > dead_line then record_dubious_connection(connection,dead_line,dos) else - if read(connection,1,600) is // the connection is closed after 10 minutes of inactivity - { - error then error(cannot_read_from_connection), - timeout then error(timeout(600)), - //record_dubious_connection(connection,dead_line,dos), - ok(ba) then if nth(0,ba) is - { - failure then error(cannot_read_from_connection), - success(c) then ok(c) - } - }. - - -define Result(Error,Word8) - next_char // reading a character (check the list first, and read on the connection - // only when the list is empty). - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if *unput_chars is - { - [ ] then read_one_byte(connection,dead_line,dos), - - [h . t] then - unput_chars <- t; - ok(h) - }. - - - - - - - - *** [2.4] Reading and discarding characters. - - The next function reads the specified number of bytes (this is the same as - 'characters') from the connection and discards them. This is used for discarding CR LF - just before the body of a request. - -define Result(Error,One) - read_and_ignore - ( - Connection connection, // to client - Int32 dead_line, - Int32 number_of_characters, // number of characters to read and ignore - DenialOfService dos - ) = - if number_of_characters =< 0 then ok(unique) else - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then read_and_ignore(connection,dead_line,number_of_characters-1,dos) - }. - - - - - - - - *** [2.5] Reading a character string. - - Sometimes values of HTTP attributes or web args are presented in the form of double - quoted strings. The next function handles the reading of such things. The leading - double quote is already read in. We must read subsequent characters until the next non - backslashed double quote. - -define Result(Error,String) - read_string - ( - Connection connection, // connection with the client - Int32 dead_line, - List(Word8) so_far, // characters read so far (in reverse order) - DenialOfService dos - ) = - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if c = '\\' - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(d) then - if d = '\"' - then read_string(connection,dead_line,['\"' . so_far],dos) - else read_string(connection,dead_line,[d, c . so_far],dos) - } - else if c = '\"' - then ok(implode(reverse(so_far))) - else read_string(connection,dead_line,[c . so_far],dos) - }. - - - - - - - - *** [2.6] Padding integers with zeros. - - 'zero_pad_2' transforms an integer (which is assumed to be between 0 and 99) into a - string with exactly two digits. This is used for formating days, hours, minutes and - seconds. - -define String - zero_pad_2 - ( - Int32 n - ) = - with s = integer_to_string(n), - if length(s) < 2 - then "0"+s - else s. - - - - - - - - *** [2.7] Converting web arguments to ASCII. - - The function 'web_to_ascii' gets a character string and replaces web encoding by normal - ASCII encoding. This amounts to replacing: - - + by blank - %xx by the character whose ASCII code is xx in hexadecimal - - Note: We assume that '9' < 'A' (which is the case for ASCII code). - - - -define Word8 - web_decode - ( - Word8 x1, - Word8 x2 - ) = - with z1 = word8_to_int32(x1), - n1 = if z1 =< '9' then (z1 - '0') else (z1 - 'A' + 10), - z2 = word8_to_int32(x2), - n2 = if z2 =< '9' then (z2 - '0') else (z2 - 'A' + 10), - n = (n1 << 4) + n2, - truncate_to_word8(n). - - - -define String - web_to_ascii - ( - String web_string, - Int32 n, // current position in web_string - List(Word8) so_far - ) = - if nth(n,web_string) is - { - failure then implode(reverse(so_far)), - success(c) then - if c = '+' - then web_to_ascii(web_string,n+1,[' ' . so_far]) - else if c = '%' - then if nth(n+1,web_string) is - { - failure then implode(reverse(so_far)), - success(x1) then if nth(n+2,web_string) is - { - failure then implode(reverse(so_far)), - success(x2) then web_to_ascii(web_string,n+3,[web_decode(x1,x2) . so_far]) - } - } - else web_to_ascii(web_string,n+1,[c . so_far]) - }. - - - - - - - - - *** [3] Managing the journal. - - Concurrently working machines should not try to access the same file at the same - time. This problem may be solved by using the 'protect' mecanism. - - - - *** [3.1] Naming journal files. - - Since journal messages are rather prolific, we should have at least one file per - hour. Hence, the name of a journal file must be constructed from the current year, - month, day and hour. For example, it may be: - - 2003_03_12_19 - - (this is for the journal of 7 PM to 8 PM, 2003/mar/12). - -define String - make_current_journal_file_name - = - if convert_time(now) is date_and_time(y,m,d,h,_,_,_,_,_) then - integer_to_string(y)+"_"+ - zero_pad_2(m)+"_"+ - zero_pad_2(d)+"_"+ - zero_pad_2(h). - - - - - - - - *** [3.2] Formating HTTP headers. - - HTTP headers may be shown on the console or written in the journal. The function below - formats a list of HTTP headers. - -define String - show_format - ( - Web_Site_Description desc, - List(HTTP_header) headers, - ) = - if headers is - { - [ ] then "", - [h . t] then if h is http_header(name,value) then - if member(journal_headers(desc),name) - then " | "+name+": "+value+"\n"+show_format(desc,t) - else show_format(desc,t) - }. - - - - - - - *** [3.3] Formating web arguments. - - The same thing for web arguments. - -define String - show_format - ( - List(Web_arg) lwa - ) = - if lwa is - { - [ ] then "", - [h . t] then if h is - { - web_arg(n,v) then - " | "+n+"="+(if nth(0,n) = success('p') then "" else v)+"\n"+show_format(t), - upload(n,fn,tfn) then - " | "+n+"="+fn+" (uploaded as '"+tfn+"')\n"+show_format(t) - } - }. - - - - - - - *** [3.4] Formating the whole request. - - It is cheap to transform month numbers into abbreviated month names. This enhances the - readability of the journal. - -define String - format_month - ( - Int32 m - ) = - if m = 1 then "jan" else - if m = 2 then "feb" else - if m = 3 then "mar" else - if m = 4 then "apr" else - if m = 5 then "may" else - if m = 6 then "jun" else - if m = 7 then "jul" else - if m = 8 then "aug" else - if m = 9 then "sep" else - if m = 10 then "oct" else - if m = 11 then "nov" else - if m = 12 then "dec" else - "???". - - - Below we format a whole HTTP request. This may give this (actually, it depends on how - you defined the values of 'journal_headers' and 'journal_extensions'): - - [3] 2003/mar/10 10:06:57 from 123.456.123.456: /homepage.awp - | host: www.the-best-one.com - | user-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 - - The leading number between brackets is the number of the virtual machine which served - the URI. - -define String - format_request - ( - Web_Site_Description desc, - Connection client_connection, - HTTP_RequestLine request_line, - List(HTTP_header) headers, - List(Web_arg) web_args - ) = - with dt = convert_time(now), - if remote_IP_address_and_port(client_connection) is (addr,port) then - integer_to_string(year(dt))+"/"+format_month(month(dt))+"/"+zero_pad_2(day(dt))+" "+ - zero_pad_2(hour(dt))+":"+zero_pad_2(minute(dt))+":"+zero_pad_2(second(dt))+ - " from "+ip_addr_to_string(addr)+ - ": "+uri(request_line)+"\n"+ - show_format(desc,headers)+ - show_format(web_args). - - - - - - - - *** [3.5] Putting it in the journal file (and on the console). - - We must not forget to 'protect' this operation, so that the messages of two machines - (working for the same site) will not be mixed together. - -define One - log_journal_msg - ( - Web_Site_Description desc, - String msg, - ) = - with msg = to_byte_array("["+virtual_machine_id+"] "+msg+"\n"), - protect - ( - if file(site_directory(desc)+"/journal/"+make_current_journal_file_name,append) is - { - failure then unique, - success(journal_file) then - forget(reliable_write(file(journal_file),msg)) - }; - forget(reliable_write(file(stdout),msg)) - ). - - - - - - - - *** [4] Reading the HTTP request. - - - *** [4.1] Skipping leading blanks. - - One of the peculiarities of HTTP is that the characters 13 (carriage return) and 10 - (line feed) followed by either a space (32) or a tab (9), is considered as a blank not - containing any new line. 'skip_http_blanks' must skip all blanks characters until the - first non blank character, which should not be read in. Obviously, because of the above - peculiarity, we need at least 3 characters of lookahead to do this. In other words, we - must be able to unput at least 3 characters (hopefully we are). - - Strictly blanks characters are 'space' and 'tab'. - -define Bool - is_strict_blank - ( - Word8 c - ) = - if c = ' ' then true else c = '\t'. - - - On the contrary, blanks include 13 and 10. - -define Bool - is_blank - ( - Word8 c - ) = - if c = ' ' then true else - if c = '\t' then true else - if c = 13 then true else - c = 10. - - - Skipping HTTP blanks. - -define Result(Error,One) - skip_http_blanks - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if is_strict_blank(c) - then skip_http_blanks(connection,dead_line,dos) - else if c = 13 - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), // (unput(c); ok(unique)), - ok(d) then - if d = 10 - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), // (unput(d); unput(c); ok(unique)), - ok(e) then - if is_strict_blank(e) - then skip_http_blanks(connection,dead_line,dos) - else (unput(e); unput(d); unput(c); ok(unique)) - } - else (unput(d); unput(c); ok(unique)) - } - else (unput(c); ok(unique)) - }. - - - - - - - - - *** [4.2] Reading a new line. - - Normally in HTTP a new line is the sequence 13 10 (carriage return line feed), not - followed by a space or tabulator. If it is followed by a space or tabulator, the three - characters are considered blanks, and no new line has been read. Before trying to read - a new line, we first skip leading spaces and tabs. Then we try to read 13 and 10, and - we read another character. if this character is space or tab, we consider we have read - only blanks and we continue reading in order to find our new line. Otherwise, we unput - this character (which may be for example the first character of the name of the next - header), and answer that we have seen a new line. - - Warning: we must not use this function for reading the last pair (13,10) before the - beginning of the body, because if the body is empty, there is no character to read - after this pair, so that the server could wait for a character which will never - come. This is the reason for 'read_and_ignore' above, which is used precisely for - reading that last (13,10) pair. - -define Result(Error,One) - read_new_line - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if skip_http_blanks(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if c = 13 - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(d) then - if d = 10 - then ok(unique) - else (unput(d); - unput(c); - error(end_of_line_expected)) - } - else (unput(c); - error(end_of_line_expected)) - }}. - - - - - - - - - *** [4.3] Reading a 'word'. - - A 'word' is a sequence of characters which begins either by a double quote or not by a - double quote. (However, any leading blanks are read in and ignored. This is - accomplished by 'skip_http_blanks'.) If it begins by a double quote, it is read like a - string, i.e. it ends at the next (non backslashed) double quote. Otherwise, it is - right delimited by any character which may be considered as 'blank'. If the word is - double quoted, the closing double quote is read in. On the contrary, if the word is not - double quoted, the right delimiting blank character is not read in (it is 'unput' back - into the connection), and may be read in again. This is needed because carriage return - or line feed which are 'blank', also have a meaning in HTTP. - -define Result(Error,String) - read_word_aux - ( - Connection connection, - Int32 dead_line, - List(Word8) so_far, - DenialOfService dos - ) = - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if is_blank(c) - then (unput(c); - ok(implode(reverse(so_far)))) - else read_word_aux(connection,dead_line,[c . so_far],dos) - }. - -define Result(Error,String) - read_word - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if skip_http_blanks(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if c = '\"' - then read_string(connection,dead_line,[],dos) - else read_word_aux(connection,dead_line,[c],dos) - } - }. - - - - - - - - - *** [4.4] Separating the URI from the query string. - - A 'query string' may be postfixed to the URI, just after a question mark. For example, - the client may send the following request: - - GET /catalog.awp?item=3&color=blue - - We separate this into an URI: "/catalog.awp" and the string: "item=3&color=blue" which - will be later transformed into the list: - - [web_arg("item","3"),web_arg("color","blue")] - - -define (String,String) - separate_uri_from_query_string - ( - String uri_and_query_string, - Int32 n - ) = - if nth(n,uri_and_query_string) is - { - failure then (uri_and_query_string,""), - success(c) then - if c = '?' - then (substr(uri_and_query_string,0,n), - substr(uri_and_query_string,n+1,length(uri_and_query_string)-(n+1))) - else separate_uri_from_query_string(uri_and_query_string,n+1) - }. - - - - - - - - - - *** [4.5] Reading the web arguments. - - HTTP/HTTPS requests are sent in one of two formats: - - (1) www-url encoded - (2) multipart/form-data encoded - - The first one is the normal (historical) way of encoding. The second one is required - for uploading files. A server which is supposed to accept upload of files must handle - both formats. The first thing to do is to decide the format of the request. This is - easily done by examining the HTTP headers. If we find the header: - - Content-Type: multipart/form-data - - the request is multipart/form-data encoded. Otherwise, it is 'www-url' encoded. We - first consider 'www-url' encoded requests. - - For a 'www-url' encoded request, the web argument are either in the query string or in - the body of the request, or both. The format is the same for both: - - name=value&name=value&... - - However, we may also have - - name - name= - name=&... - name&... - - i.e. some parts may be missing. Hence, we must be careful. - - Furthermore, web arguments must be translated from web to ASCII when www-url encoded. - -define Bool - is_ampersand_or_equal - ( - Word8 c - ) = - if c = '&' then true else c = '='. - - - - The function 'read_name_or_value' reads the string 's' starting at position 'n' until - either the end of the string or the first '&' or '='. - -define String - read_name_or_value - ( - String s, - Int32 start, - Int32 i - ) = - if nth(i,s) is - { - failure then substr(s,start,i - start), - success(c) then - if is_ampersand_or_equal(c) - then substr(s,start,i-start) // the separator is not included - else read_name_or_value(s,start,i+1) - }. - - -define List(Web_arg) - read_www_url_encoded_web_args - ( - String s, - Int32 start, - ) = - with first = read_name_or_value(s,start,start), - if first = "" - then [] - else with i = start+length(first), - if nth(i,s) is - { - failure then [web_arg(first,"")], - success(c) then - if c = '&' - then [web_arg(first,"") . read_www_url_encoded_web_args(s,i+1)] - else if c = '=' - then with second1 = read_name_or_value(s,i+1,i+1), - // print("\""+second1+"\"\n"); - with second = web_to_ascii(second1,0,[]), - [web_arg(first,second) . read_www_url_encoded_web_args(s,i+length(second1)+2)] - else alert - }. - - - - - - *** [4.7] Reading the request line. - - 'read_request_line' reads three words and a new line from the connection. It tries to - recognize "GET" or "POST" in the first word, separates the URI from the query string in - the second word, transforms the query string into a list of 'Web_arg', and finally - returns a datum of type 'HTTP_RequestLine' if no error arose. - - -define Result(Error,HTTP_RequestType) - identify_get_or_post - ( - String s - ) = - with s = to_lower(s), - if s = "get" then ok(get) else - if s = "post" then ok(post) else - error(not_get_or_post_request(s)). - -define Result(Error,HTTP_RequestLine) - read_request_line - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if read_word(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(get_or_post) then if read_word(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(uri_and_query_string) then if read_word(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(http_version) then if read_new_line(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then if separate_uri_from_query_string(uri_and_query_string,0) is - (uri,query_string) then if identify_get_or_post(get_or_post) is - { - error(msg) then error(msg), - ok(request_type) then - ok(request_line(request_type,uri,read_www_url_encoded_web_args(query_string,0))) - } - } - } - } - }. - - - - - - - - *** [4.8] Reading the HTTP headers. - - Each header is made of a name (containing only letters, the underscore, digits and the - minus sign), a colon, a value, and a new line. The first empty line ends the headers. - - - The next function tests characters acceptable in a header name. - -define Bool - is_header_name_char - ( - Word8 c - ) = - with n = word8_to_int32(c), - if ('a' =< n & n =< 'z') then true else - if ('A' =< n & n =< 'Z') then true else - if ('0' =< n & n =< '9') then true else - if c = '-' then true else - c = '_'. - -define Result(Error,String) - read_header_name - ( - Connection connection, - Int32 dead_line, - List(Word8) so_far, - DenialOfService dos - ) = - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if is_header_name_char(c) - then read_header_name(connection,dead_line,[to_lower(c) . so_far],dos) - else unput(c); ok(implode(reverse(so_far))) - }. - -define Result(Error,One) - skip_colon - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if skip_http_blanks(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if c = ':' - then ok(unique) - else error(colon_expected) - }}. - - -define Result(Error,String) - read_header_value - ( - Connection connection, - Int32 dead_line, - List(Word8) so_far, - DenialOfService dos - ) = - if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(c) then - if c = 13 - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(d) then - if d = 10 - then if next_char(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(e) then - if is_strict_blank(e) - then read_header_value(connection,dead_line,[e . so_far],dos) - else (unput(e); ok(implode(reverse(so_far)))) - } - else read_header_value(connection,dead_line,[d, c . so_far],dos) - } - else read_header_value(connection,dead_line,[c . so_far],dos) - }. - - - Reading a single header. - -define Result(Error,Maybe(HTTP_header)) - read_header - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if read_header_name(connection,dead_line,[],dos) is - { - error(msg) then error(msg), - ok(name) then - if name = "" then - if read_and_ignore(connection,dead_line,2,dos) /* 13 and 10 */ is - { - error(msg) then error(msg), - ok(_) then // this is the blank line - ok(failure) // end of headers - } - else if skip_colon(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then if skip_http_blanks(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(_) then if read_header_value(connection,dead_line,[],dos) is - { - error(msg) then error(msg), - ok(value) then - ok(success(http_header(name,value))) - } - } - } - }. - - - - Reading all the headers. - -define Result(Error,List(HTTP_header)) - read_http_headers - ( - Connection connection, - Int32 dead_line, - DenialOfService dos - ) = - if read_header(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(mbh) then if mbh is - { - failure then ok([ ]), - success(header) then - if read_http_headers(connection,dead_line,dos) is - { - error(msg) then error(msg), - ok(others) then ok([header . others]) - } - } - }. - - - - - - - - *** [4.9] Getting the size of the request's body. - - The size of the body of the request is given under the 'Content-Length' header. If this - header is not present, the size is assumed to be zero. - -define Result(Error,Int32) - get_body_size - ( - List(HTTP_header) headers - ) = - if headers is - { - [ ] then ok(0), - [h . t] then if h is http_header(name,value) then - if name = "content-length" - then if string_to_integer(value) is - { - failure then error(incorrect_content_length_value), - success(n) then ok(n) - } - else get_body_size(t) - }. - - - - - - - - - - - *** [4.10] Reading the body of the request. - - The body of the request may be very big (it contains uploaded files, if any). We read - it using the primitive 'read', which returns the number of bytes read, which may be - less than the number of bytes we wanted to read. This is not an error, but simply due - to the fact the buffer associated with the connection in the Linux (or MS-Windows) - kernel has a limited size. Hence, we must read bytes again until we have read the - required number of bytes. However, if the number of bytes read is zero, the connection - may be broken. In that case, we must not try to read indefinitely. On the contrary, we - make at most 10 retries, with a small sleeping time between any two of them. - -define Result(Error,ByteArray) - read_http_body - ( - Connection connection, - Int32 body_size, - ByteArray so_far, // when calling this function, 'so_far' is the empty byte array - Int32 retries // this function is called with retries = 10 - ) = - if body_size = 0 then ok(constant_byte_array(0,0)) else - if retries =< 0 then error(cannot_read_from_connection) else - if read(connection,body_size,60) is - { - error then error(cannot_read_from_connection), - timeout then error(timeout(60)), - ok(new_bytes) then with - ba = so_far + new_bytes, // contains all the bytes read so far - nr = length(ba), // total read since the beginning - nn = length(new_bytes), // number of bytes just read - if nr < body_size // must read more bytes - then if nn > 0 // if connection seems to work - then read_http_body(connection,body_size,ba,1000) // continue reading - else sleep(100); // otherwise, sleep 1/10 of second - read_http_body(connection,body_size,ba, // and retry reading - retries-1) // but no more than 10 times - else ok(ba) // required number of bytes has been read - }. - - - Note: During sleeping, 'anbexec' runs other machines. Actually, calling 'sleep', even - for one millisecond, is some way of giving up explicitly, so that other virtual - machines may work. - - - - - - - - - - - - - *** [5] Making the HTTP answer. - - At that point we have read the request line, the headers and the body of the - request, and we must decide what to do. - - Actually, we can do one of the following: - - - send a file, - - execute 'tickets_and_web_page' in case of an ".awp" URI. - - The uploaded file (which are in the body of the request) are saved into temporary files - below. - - - - - - *** [5.1] Avoiding illegal URIs. - - For security reasons, we must avoid illegal URIs, for example those which may climb up - in the file hierarchy. First we accept only few characters in URIs. - -define Bool - is_legal_uri_char - ( - Word8 c - ) = - with n = word8_to_int32(c), - if ('a' =< n & n =< 'z') then true else // accept 'a' to 'z' - if ('A' =< n & n =< 'Z') then true else // accept 'A' to 'Z' - if ('0' =< n & n =< '9') then true else // accept '0' to '9' - if c = '.' then true else // accept '.' '-' '/' and '_' - if c = '-' then true else - if c = '/' then true else - c = '_'. - - We do not accept ~ which is some way of climbing. Of course, we cannot disallow single - dots, which are most often present in legal URIs, but we must avoid double dots .. - which mean 'climb up'. - -define Bool - is_illegal_uri - ( - String uri, - Int32 n - ) = - if nth(n,uri) is - { - failure then false, - success(c) then - if c = '.' // first dot - then if nth(n+1,uri) is - { - failure then false, - success(d) then - if d = '.' // second dot - then true - else is_illegal_uri(uri,n+1) - } - else is_illegal_uri(uri,n+1) - }. - - - - - - - *** [5.2] Managing authorizations for downloading private files. - - Computing the authorization and making the authorization file (containing the absolute - path of the file on the server). - - -define String - compute_authorization - ( - String authorization_secret, - String absolute_path - ) = - to_ascii(sha1((authorization_secret, - absolute_path))). - - -public define String - make_authorization - ( - String site_directory, - String authorization_secret, - String absolute_path - ) = - with private_download_dir = site_directory+"/private_download", - auth = compute_authorization(authorization_secret, - absolute_path), - forget(save(absolute_path, - private_download_dir+"/z"+auth)); - auth. - - - The function 'send_file' defined below handles the recognition of authorizations. - - - - - - *** [5.3] Recognizing MIME types. - - The extension of the (redirected) URI must be either ".awp" or recognized as associated - to a MIME type. Otherwise, the server will not send the file. This is for security, but - also because, we must generate a 'Content-Type' header in the answer, with the right - MIME type. - -define String - get_uri_extension_aux - ( - String uri, - Int32 n // used for searching backwards - ) = - if nth(n,uri) is - { - failure then "", - success(c) then - if c = '.' then substr(uri,n,length(uri)-n) - else if c = '/' then "" - else get_uri_extension_aux(uri,n-1) - }. - -public define String - get_uri_extension - ( - String uri - ) = - get_uri_extension_aux(uri, - length(uri)-1). // search starts at the right end - - - -define Maybe(String) - recognize_mime_type_from_ext - ( - String ext, - List(MIME) l - ) = - if l is - { - [ ] then success("application/octet-stream"), // failure, - [h . t] then if h is mime(mime_type,extension) then - if ext = extension - then success(mime_type) - else recognize_mime_type_from_ext(ext,t) - }. - -define Maybe(String) - recognize_mime_type_from_uri - ( - Web_Site_Description desc, - String uri - ) = - recognize_mime_type_from_ext(get_uri_extension(uri),known_mime_types(desc)). - - - - - - - - - *** [5.4] Formating HTTP headers. - - This is the formating for sending to the client (hence, it has nothing to do with the - component 'journal_headers' in the web site description). - -define Printable_tree - format_headers - ( - List(HTTP_header) headers - ) = - if headers is - { - [ ] then [ ], - [h . t] then if h is http_header(name,value) then - [name,": ",value,crlf . format_headers(t)] - }. - - - - - - - *** [5.5] Sending a file. - - We send 2 headers 'Content-Type' and 'Content-Length'. - -define List(HTTP_header) - headers_for_send_file - ( - String mime_type, - Int32 size, - ) = - [ - http_header("Content-Type",mime_type), - http_header("Content-Length",integer_to_string(size)), - ]. - - - - Sending the body of the answer (i.e. the file itself). - -define One - send_file_body - ( - Web_Site_Description desc, - Connection connection, // connection with the client - Connection file, // file to be sent already opened - Int32 size, // size of file - Int32 sent, // bytes already sent - String filename // name of file - ) = - if sent >= size then unique else - if read(file,min(10000,size-sent),60) is - { - error then log_journal_msg(desc,"Cannot read from file '"+filename+"'.\n"), - timeout then log_journal_msg(desc,"Cannot read from file timeoput'"+filename+"'.\n"), - ok(ba) then - with nr = length(ba), // get the number of bytes read - if reliable_write(connection,ba) is - { - failure then log_journal_msg(desc,"Cannot write into connection.\n"), - success(nw) then - send_file_body(desc,connection,file,size,sent+nw,filename) - } - }. - - - - - Sending the answer line, the headers and the body. - -define One - send_file - ( - Web_Site_Description desc, - Connection connection, - List(HTTP_header) headers, - Int32 size, - Connection file, - String filename, - One -> One action_before_send_file - ) = - action_before_send_file(unique); - forget(reliable_write(connection,to_byte_array("HTTP/1.1 200 OK"+crlf))); - forget(reliable_write(connection,[format_headers(headers) , crlf])); - send_file_body(desc,connection,file,size,0,filename). - - - - Checking if a connection is under SSL. - -define Bool - is_SSL - ( - Connection c - ) = - if c is - { - file_r(_) then false, - file_w(_) then false, - file_rw(_) then false, - tcp(_) then false, - ssl(_) then true - }. - - - - Before opening and sending a file, we check the MIME type. It must be recognized, - except if there is a valid authorization for private download. - -define One - send_file - ( - Web_Site_Description desc, - Connection connection, - String uri, - Maybe(String) mbauthorization, - One -> One action_before_send_file - ) = - if mbauthorization is - { - //--- file without authorization: take it from public --- - failure then if recognize_mime_type_from_uri(desc,uri) is - { - failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), - success(mime_type) then - with path = site_directory(desc)+"/public"+uri, - if (Maybe(RStream))connect to file path is - { - failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), - success(f) then with size = file_size(f), - send_file(desc, - connection, - headers_for_send_file(mime_type,size), - size, - file(f), - uri, - action_before_send_file) - } - }, - - //--- file with authorization: apply 'private download' mecanism --- - success(authorization) then - with private_download_dir = site_directory(desc)+"/private_download", - if (RetrieveResult(String))retrieve(private_download_dir+"/z"+authorization) - is ok(absolute_path) - then ( - with new_hash = compute_authorization(authorization_secret(desc), - absolute_path), - if (Maybe(RStream))connect to file absolute_path is - { - failure then log_journal_msg(desc,"Cannot find file '"+absolute_path+"'.\n"), - success(f) then with size = file_size(f), - send_file(desc, - connection, - headers_for_send_file(if recognize_mime_type_from_uri(desc,uri) is - { - failure then "application/octet-stream" - success(mime_type) then mime_type - }, - size), - size, - file(f), - uri, - action_before_send_file) - } - ) - else log_journal_msg(desc,"Cannot find or read authorization file.\n") - }. - - - define One - send_file - ( - Web_Site_Description desc, - Connection connection, - String uri, - Maybe(String) mbauthorization, - One -> One action_before_send_file - ) = - if mbauthorization is - { - //--- file without authorization: take it from public --- - failure then if recognize_mime_type_from_uri(desc,uri) is - { - failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), - success(mime_type) then - with path = site_directory(desc)+"/public"+uri, - if (Maybe(RStream))connect to file path is - { - failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), - success(f) then with size = file_size(f), - send_file(desc, - connection, - headers_for_send_file(mime_type,size), - size, - file(f), - uri, - action_before_send_file) - } - }, - - //--- file with authorization: apply 'private download' mecanism --- - success(authorization) then - if is_SSL(connection) - then ( - with private_download_dir = site_directory(desc)+"/private_download", - if (RetrieveResult(String))retrieve(private_download_dir+"/z"+authorization) - is ok(absolute_path) - then ( - with new_hash = compute_authorization(authorization_secret(desc), - absolute_path), - if (Maybe(RStream))connect to file absolute_path is - { - failure then log_journal_msg(desc,"Cannot find file '"+absolute_path+"'.\n"), - success(f) then with size = file_size(f), - send_file(desc, - connection, - headers_for_send_file(if recognize_mime_type_from_uri(desc,uri) is - { - failure then "application/octet-stream" - success(mime_type) then mime_type - }, - size), - size, - file(f), - uri, - action_before_send_file) - } - ) - else log_journal_msg(desc,"Cannot find or read authorization file.\n") - ) - else //-- file with authorization, but under HTTP: take it from private_download - if recognize_mime_type_from_uri(desc,uri) is - { - failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), - success(mime_type) then - with path = site_directory(desc)+"/private_download"+uri, - if (Maybe(RStream))connect to file path is - { - failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), - success(f) then with size = file_size(f), - send_file(desc, - connection, - headers_for_send_file(mime_type,size), - size, - file(f), - uri, - action_before_send_file) - } - } - }. - - - - - - - - *** [5.6] Answering a www-url encoded request. - - Standard headers are for answering ".awp" requests. - -define List(HTTP_header) - standard_headers - ( - Int32 answer_body_size, - String charset - ) = - [ - //http_header("Content-Type","text/html"), - http_header("Content-Type","text/html; charset="+charset), - http_header("Content-length",integer_to_string(answer_body_size)) - ]. - - -define One - www_url_answer - ( - String host_name, - Web_Site_Description desc, - Connection connection, // with the client - Int32 ip_addr, // of the client - HTTP_RequestLine request_line, - List(HTTP_header) headers, - ByteArray body, - One -> String generate_tt // trust ticket generation - ) = - with all_web_args = query_string(request_line) + - read_www_url_encoded_web_args(to_string(body),0), - uri = uri(request_line), - ext = get_uri_extension(uri), - (if member(journal_extensions(desc),ext) - then log_journal_msg(desc, - format_request(desc,connection,request_line,headers,all_web_args)) - else unique); - if is_illegal_uri(uri,0) - then log_journal_msg(desc,"Received illegal URI: "+uri+"\n") - else (if (ext = ".awp" | ext = "") - then (with answer_headers_body = awp_handler(desc)(host_name, - http_info(ip_addr,uri,headers,generate_tt), - all_web_args, - is_SSL(connection)), - if answer_headers_body is (additional_headers,answer_body) then - forget(reliable_write(connection, - [ "HTTP/1.1 200 OK", crlf, - format_headers(standard_headers(length(answer_body),charset(desc))), - format_headers(additional_headers), - crlf . - answer_body]))) - else (send_file(desc, - connection, - uri, - if web_arg_value(all_web_args,"zauth") is - { - not_found then failure, - found(v) then success(v) - }, - (One u) |-> before_send_file(desc)(all_web_args)))). - - - - - - - - *** [5.7] Answering a multipart/form-data encoded request. - - In order to support upload of files, we must be able to read web arguments which are - encoded in a multipart/form-data body. The first thing to do is to find the - boundary. The boundary is a special string which delimits the various parts of the - 'multipart' body. It is found within the value of the 'Content-Type' HTTP header, as - the value of the 'boundary' attribute. - - - - - - *** [5.7.1] Finding the boundary. - - Hence, we just have to find the string 'boundary=' within the value of the - 'Content-Type' header, and read the value of the boundary from there. - -define Bool - delimits_boundary - ( - Word8 c - ) = - if c = ' ' then true else - if c = 13 then true else - if c = 10 then true else - if c = 0 then true else - if c = ',' then true else - c = ';'. - - -define Maybe(String) - get_boundary_value_3 - ( - String s, - Int32 i, - List(Word8) so_far - ) = - if nth(i,s) is - { - failure then success(implode(reverse(so_far))), - success(c) then - if delimits_boundary(c) - then success(implode(reverse(so_far))) - else get_boundary_value_3(s,i+1,[c . so_far]) - }. - - - -define Maybe(String) - get_boundary_value_2 - ( - String s, - Int32 i, - ) = - if nth(i,s) is - { - failure then failure, - success(c) then - if is_blank(c) - then get_boundary_value_2(s,i+1) - else get_boundary_value_3(s,i+1,[c]) - }. - -define Maybe(String) - get_boundary_value_1 - ( - String s, // string into which we must find '= ...' - Int32 i // position of start of search - ) = - if nth(i,s) is - { - failure then failure, - success(c) then - if is_blank(c) - then get_boundary_value_1(s,i+1) - else if c = '=' - then get_boundary_value_2(s,i+1) - else failure - }. - - -define Maybe(String) - get_boundary - ( - String content_type_header_value - ) = - if find("boundary",content_type_header_value,0) is - { - failure then failure, - success(n) then // 'boundary' has been found at position n - get_boundary_value_1(content_type_header_value,n+8) - }. - -define Maybe(String) - get_boundary - ( - List(HTTP_header) headers - ) = - if headers is - { - [ ] then failure, - [h . t] then if h is http_header(name,value) then - if name = "content-type" - then get_boundary(value) - else get_boundary(t) - }. - - - - - - - - - *** [5.7.2] Reading attributes from a multipart entity. - - Entities in a multipart/form-data body are separated by instances of the string: - - --bbbbb - - where bbbbb is the boundary computed above. Actually, the body has the form: - - --bbbbb - - --bbbbb - - --bbbbb - ... - --bbbbb - - --bbbbb - - - We have to extract an entity which is in the body between offsets 'start' and 'end' - (computed when boundaries have been localized). The entity itself is made of two parts: - headers and body. The body is separated from the headers by a blank line. This blank - line (a double crlf) marks the beginning of the body of the entity. Within the headers - of the entity, we look for a 'Content-Disposition' header, which should look like this: - - Content-Disposition: form-data; name="..."; filename="..." crlf - - We are just interested in the name and the file name. Hence we first search - 'Content-Disposition', then we search 'name' and read the value, and we do the same for - 'filename'. - - If the 'filename' attribute is not present, the web arg is an ordinary one, otherwise, - it is an uploaded file. - - - Below is a variant of 'find' (see 'tools/findstring.anubis'), with an extra 'end' - argument. - -define Maybe(Int32) - find - ( - String what, - ByteArray where, - Int32 start, - Int32 end - ) = - if find(to_byte_array(what),where,start) is - { - failure then failure, - success(n) then - if n+length(what) >= end - then failure - else success(n) - }. - - -define String - read_attribute_value - ( - ByteArray where, - Int32 start, - Int32 end, - List(Word8) so_far - ) = - if start >= end then implode(reverse(so_far)) else - if nth(start,where) is - { - failure then implode(reverse(so_far)), - success(c) then - if c = '\"' - then implode(reverse(so_far)) - else read_attribute_value(where,start+1,end,[c . so_far]) - }. - -define Maybe(String) - find_attribute - ( - String name, - ByteArray where, - Int32 start, - Int32 end - ) = - with name = name+"=\"", - if find(to_byte_array(name),where,start) is - { - failure then failure, - success(n) then - if n+length(name) >= end - then failure - else success(read_attribute_value(where,n+length(name),end,[])) - }. - - - -define Maybe((String,Maybe(String))) - find_name_and_filename - ( - ByteArray body, - Int32 start, - Int32 end - ) = - if find(to_byte_array("Content-Disposition"),body,start) is - { - failure then failure, - success(n) then - if find_attribute("name",body,n+19,end) is - { - failure then failure, - success(name_value) then if find_attribute("filename",body,n+19,end) is - { - failure then success((name_value,failure)), - success(filename_value) then success((name_value,success(filename_value))) - } - } - }. - - - - - - - - - - - *** [5.7.3] Creating a temporary filename for an uploaded file. - -variable Int32 uploaded_file_count = 0. - - This variable is local to the virtual machine. Hence, its value is 0 each time a new - requests arrives. Temporary uploaded files are stored in the directory represented by - 'upload_temporary_directory'. The filenames have the form: - - _m_n - - where 'm' is the number of the virtual machine, and 'n' a number obtained by - incrementing 'uploaded_file_count'. Notice that the program must do something with this - file (move it to some directory/name), otherwise, it will probably be overwritten the - next time the same machine works. - - - - - - - *** [5.7.4] Saving an uploaded file under a temporary filename. - -define Maybe(String) // returns the temporary file name - save_uploaded_file - ( - Web_Site_Description desc, - ByteArray body, - Int32 start, - Int32 end - ) = - uploaded_file_count <- 1 + *uploaded_file_count; - with tfn = "_"+integer_to_string(virtual_machine_id)+"_"+integer_to_string(*uploaded_file_count), - if (Maybe(WStream))connect to file site_directory(desc)+"/upload_temporary/"+tfn is - { - failure then failure, - success(f) then - if reliable_write(file(f),extract(body,start,end)) is - { - failure then failure, - success(nw) then - if nw = end - start - then success(tfn) - else failure - } - }. - - - - - - - - - *** [5.7.5] Removing the path from a file name. - - When a file is uploaded, the browser sends the complete path of the file on the client - machine as the file name. Actually, this is not quite normal. Nevertheless, we need to - remove the path, and keep only the file name. This is achieved by 'remove_path' below. - -define Int32 - file_name_begin - ( - String full_name, - Int32 i - ) = - if nth(i,full_name) is - { - failure then 0, - success(c) then - if c = '/' then i+1 else - if c = '\\' then i+1 else - file_name_begin(full_name,i-1) - }. - -define String - remove_path - ( - String full_name - ) = - with l = length(full_name), - b = file_name_begin(full_name,l-1), - substr(full_name,b,l-b). - - - - - - *** [5.7.6] Reading a multipart entity. - -define Maybe(Web_arg) - get_multipart_entity - ( - Web_Site_Description desc, - ByteArray body, - Int32 start, - Int32 end - ) = - if find(to_byte_array(crlf+crlf),body,start) is - { - failure then failure, - success(k) then - if k >= end // must be within this entity, not the next one - then failure - else if find_name_and_filename(body,start,k) is - { - failure then failure, - success(n_mbfn) then if n_mbfn is (name,mbfn) then - if mbfn is - { - failure then - success(web_arg(name,to_string(extract(body,k+4,end-2)))), - // we must substract 2 to end because of crlf just before the boundary - - success(fn) then - if save_uploaded_file(desc,body,k+4,end-2) is - { - failure then failure, - success(tfn) then - success(upload(name,remove_path(fn), - site_directory(desc)+"/upload_temporary/"+tfn)) - - } - } - } - }. - - - -define List(Web_arg) - read_multipart_form_data_encoded_web_args - ( - Web_Site_Description desc, - ByteArray body, - ByteArray __boundary, - Int32 i, - ) = - if find(__boundary,body,i) is - { - failure then [ ], - success(n) then - if find(__boundary,body,n+length(__boundary)) is - { - failure then [ ], - success(m) then - if get_multipart_entity(desc,body,n+length(__boundary),m) is - { - failure then [ ], - success(wa) then - [wa . read_multipart_form_data_encoded_web_args(desc,body,__boundary,m)] - } - } - }. - - - -define One - multipart_form_data_answer - ( - String host_name, - Web_Site_Description desc, - Connection connection, - Int32 ip_addr, - HTTP_RequestLine request_line, - List(HTTP_header) headers, - ByteArray body, - One -> String generate_tt - ) = - if get_boundary(headers) is - { - failure then unique, - success(boundary) then - with all_web_args = query_string(request_line) + - read_multipart_form_data_encoded_web_args(desc, - body, - to_byte_array("--"+boundary), - 0), - uri = uri(request_line), - ext = get_uri_extension(uri), - log_journal_msg(desc, - format_request(desc,connection,request_line,headers,all_web_args)); - if is_illegal_uri(uri,0) - then log_journal_msg(desc,"Received illegal URI: "+uri+"\n") - else - if (ext = ".awp" | ext = "") then - (with answer_headers_body = awp_handler(desc)(host_name, - http_info(ip_addr,uri,headers,generate_tt), - all_web_args, - is_SSL(connection)), - if answer_headers_body is (additional_headers,answer_body) then - forget(reliable_write(connection, - [ "HTTP/1.1 200 OK",crlf, - format_headers(standard_headers(length(answer_body),charset(desc))), - format_headers(additional_headers), - crlf . - answer_body]))) - else unique - }. - - - - - - - - - *** [5.8] Handling redirections. - - 'redirections' (of type 'List(Redirection)') contains redirection directives. Each one - has the form: - - redirect(required_uri,required_host,corresponding_uri). - - The host required by the client may be found in the 'Host' HTTP header. The URI - required by the client is given below as 'uri'. We just have to find the required host - in the headers, and to find the corresponding redirection directive. - - - In the next fonction, the required host and URI are known. We just have to search in - the 'redirections' list. - -define String - handle_redirection - ( - String required_uri, - String required_host, - List(Redirection) redirections - ) = - if redirections is - { - [ ] then required_uri, - [h . t] then if h is redirect(uri,host,target) then - if host = required_host - then if uri = required_uri - then target - else handle_redirection(required_uri,required_host,t) - else handle_redirection(required_uri,required_host,t) - }. - - - - The host name may be encumbered by a port number, like - - www.our-business.com:1607 - - We must remove this port number, otherwise the host name may not be recognized. - -define String - strip_port - ( - String name, - Int32 i - ) = - if nth(i,name) is - { - failure then name, - success(c) then - if c = ':' - then substr(name,0,i) - else strip_port(name,i+1) - }. - - - - - - Finding the 'Host' header. No redirection is performed if this header is not found. - -define String - handle_redirection // returns the redirected URI - ( - List(Redirection) redirections, - String uri, // original URI - List(HTTP_header) headers - ) = - if headers is - { - [ ] then uri, - [h . t] then if h is http_header(name,value) then - if name = "host" - then handle_redirection(uri,strip_port(value,0),redirections) - else handle_redirection(redirections,uri,t) - }. - - - - - - - - - *** [5.9] Answering both sorts of requests. - - We must decide if the request is www-url encoded or multipart/form-data encoded. This - is achieved through the header 'Content-Type'. - -define EncodingType - get_encoding_type - ( - List(HTTP_header) headers - ) = - if headers is - { - [ ] then www_url, // this is the default - [h . t] then if h is http_header(name,value) then - if name = "content-type" - then if find("multipart/form-data",value,0) is - { - failure then www_url, - success(_) then multipart_form_data - } - else get_encoding_type(t) - }. - - - -define One - send_answer - ( - String host_name, - Web_Site_Description desc, - Connection connection, - HTTP_RequestLine rqline, - List(HTTP_header) headers, - ByteArray body, - One -> String generate_tt - ) = - if rqline is request_line(type,uri,qstring) then - with rqline = request_line(type,handle_redirection(redirections(desc),uri,headers),qstring), - if remote_IP_address_and_port(connection) is (ip_addr,_) then - if get_encoding_type(headers) is - { - www_url then - www_url_answer(host_name,desc,connection,ip_addr,rqline,headers,body,generate_tt), - multipart_form_data then - multipart_form_data_answer(host_name,desc,connection,ip_addr,rqline,headers,body,generate_tt) - }. - - - - - - - - *** [6] The HTTP/HTTPS server. - - The command 'start_server' (declared in 'predefined.anubis') starts a virtual machine - which opens a server TCP/IP connection, and which continuously listens to this - connection. When a request arrives, this machine delegates the work of deciphering and - answering the request to another virtual machine, and continues to listen. The job of - the delegated machine is defined by the HTTP request handler below. - - - - - - *** [6.1] Determining the requested host. - - When a request arrives to one of our two servers, we must decide which site (host) is - requested. - -define Maybe(String) - get_host_header_value - ( - List(HTTP_header) headers - ) = - if headers is - { - [ ] then failure, - [h . t] then if h is http_header(name,value) then - if name = "host" - then success(strip_port(value,0)) - else get_host_header_value(t) - }. - -define Maybe((String,Web_Site_Description)) - get_site - ( - String requested_host, - List(Web_Site_Description) sites - ) = - if sites is - { - [ ] then print("Requested host '"+requested_host+"' does not exist.\n"); failure, - [site1 . others] then - if site1 is web_site_description(common_names,_,_,_,_,_,_,_,_,_) then - if member(common_names,requested_host) - then success((requested_host,site1)) - else get_site(requested_host,others) - }. - - -define Maybe((String,Web_Site_Description)) - get_site - ( - List(HTTP_header) headers, - List(Web_Site_Description) sites - ) = - if get_host_header_value(headers) is - { - failure then print("No 'Host' HTTP header.\n"); failure, - success(requested_host) then - //here we treat the case with only one site. hence we accept any host request - //print("*** there is " +length(sites) + " sites \n"); - if length(sites) = 1 then - with site = force_nth(0, sites), - //print("ONE server OK\n"); - success((requested_host, site)) - else - get_site(requested_host,sites) - }. - - - - - - *** [6.2] The HTTP request handler. - - Here is the HTTP/HTTPS handler. It is called at each new request in a separate virtual - machine. It reads the headers of the HTTP request, determines the host, determines body - size, reads the body of the HTTP request, and answers the request. - - - -define One -> String make_generate_trust_ticket(DenialOfService dos). - - -define One - http_https_handler - ( - List(Web_Site_Description) sites, - Connection connection, - Bool is_https, - DenialOfService dos - ) = - with start_time = (Int32)now, - sttm <- start_time; - if dos is denial_of_service(mc_v,rld_v,hd_v,ad_v,ld_v,ra_v) then - if remote_IP_address_and_port(connection) is (ip_addr,port) then - if read_request_line(connection,start_time+*rld_v,dos) is - { - error(msg) then print(format(msg)), - ok(request_line) then - if read_http_headers(connection,start_time+*hd_v,dos) is - { - error(msg) then print(format(msg)), - ok(headers) then if get_site(headers,sites) is - { - failure then unique, - success(p) then if p is (host_name,desc) then - if get_body_size(headers) is - { - error(msg) then log_journal_msg(desc,format(msg)), - ok(body_size) then - if read_http_body(connection,body_size,constant_byte_array(0,0),1000) is - { - error(msg) then log_journal_msg(desc,format(msg)), - ok(body) then - send_answer(host_name,desc,connection,request_line,headers,body, - make_generate_trust_ticket(dos)) - } - } - } - } - }. - - - Below are the two tools for constructing the handlers required by 'start_server' and - 'start_ssl_server' (see 'predefined.anubis'). - -define Bool is_dubious_IP(Int32 ip, DenialOfService dos). - -define Server -> ((RWStream) -> One) - make_http_handler - ( - List(Web_Site_Description) sites, - DenialOfService dos - ) = - (Server server) |-> (RWStream connection) |-> - if remote_IP_address_and_port(connection) is (addr,_) then - if is_dubious_IP(addr,dos) - then print("Rejecting dubious IP address "+ip_addr_to_string(addr)+"\n") - else http_https_handler(sites,tcp(connection),false,dos). - -define Server -> (SSL_Connection -> One) - make_https_handler - ( - List(Web_Site_Description) sites, - DenialOfService dos - ) = - (Server server) |-> (SSL_Connection connection) |-> - http_https_handler(sites,ssl(connection),true,dos). - - - - - *** [6.3] Server's tasks. - - Some tasks must be executed periodically, for example for cleaning up directories from - short life time files. - - The next function removes from the given directory (and recursively from its - subdirectories) all the files which are more than 10 minutes old. - -define One - cleanup_directory_10mn - ( - String dir // path of private download directory (or subdirectory) with trailing slash - ) = - forget(map((FileDescription fd) |-> if fd is - { - no_info(name) then forget(remove(dir+name)), - file(name,_,_,d) then if d+600 < now then forget(remove(dir+name)) else unique, - link(name,_,_,d) then if d+600 < now then forget(remove(dir+name)) else unique, - directory(name,_,_) then cleanup_directory_10mn(dir+name+"/"), - }, - directory_full_list(dir,"*","*","*"))). - - -define One - http_servers_tasks - ( - List(Web_Site_Description) sites, - List(Server) servers, - Int32 period, - Int32 next_time, - ) = - if mapand(is_down,servers) - then unique - else if now > next_time - then - ( - /* - forget(map((Web_Site_Description wsd) |-> - cleanup_directory_10mn(site_directory(wsd)+"/private_download/"), - sites)); - */ - http_servers_tasks(sites,servers,period,next_time+period) - ) - else - ( - sleep(1000); - http_servers_tasks(sites,servers,period,next_time) - ). - - -public define One - start_http_servers_tasks - ( - List(Web_Site_Description) sites, - List(Server) servers, - Int32 period - ) = - delegate http_servers_tasks(sites,servers,period,now), - unique. - - - - - *** [6.4] Protection against 'denial of service' attacks. - - - *** [6.4.1] Counting connections. - -define Bool // returns false if the counter cannot be incremented (too many connections) - increment_connections_counter - ( - Var(Int32) counter - ) = - protect with n = *counter, - if n >= 100 - then false - else (counter <- (*counter)+1); true. - -define One - decrement_connections_counter - ( - Var(Int32) counter - ) = - protect counter <- (*counter)-1. - - - - - - *** [6.4.2] Recording dubious IP addresses. - - -define List(DubiousIP) - record_dubious_IP - ( - Int32 ip, - List(DubiousIP) l - ) = - if l is - { - [ ] then [dubious_ip(ip,now)], - [h . t] then if h is dubious_ip(addr,time) then - if addr = ip - then [dubious_ip(addr,now) . t] - else [h . record_dubious_IP(ip,t)] - }. - - -define One - record_dubious_IP - ( - Int32 dubious_IP, - Var(List(DubiousIP)) v - ) = - protect v <- record_dubious_IP(dubious_IP,*v). - - -define One - record_dubious_IP - ( - Int32 addr, - DenialOfService dos - ) = - record_dubious_IP(addr,list_of_dubious(dos)). - - -public define DenialOfService - load_denial_of_service_info - = - if (RetrieveResult(DenialOfService))retrieve(my_anubis_directory+"/web_sites/dos_info") is - ok(dos) then dos else denial_of_service( - var(100), - var(1000), - var(1500), - var(2000), - var([]), - var([])). - - - - - *** [6.4.3] Testing if an address is dubious. - -define Bool - is_dubious_IP - ( - Int32 ip, - List(DubiousIP) l - ) = - if l is - { - [ ] then false, - [h . t] then if h is dubious_ip(addr,time) then - if ip = addr - then true - else is_dubious_IP(ip,t) - }. - - -define Bool - is_dubious_IP - ( - Int32 ip, - DenialOfService dos - ) = - if dos is - { - denial_of_service(mc_v,rld_v,hd_v,ad_v,ld_v,ra_v) then - if member(*ra_v,ip) then false else - is_dubious_IP(ip,*ld_v) - }. - - - - - *** [6.4.4] Removing inactive dubious IP addresses. - -define List(DubiousIP) - remove_inactive_dubious_IP - ( - List(DubiousIP) l, - Int32 ref_time, - ) = - if l is - { - [ ] then [ ], - [h . t] then if h is dubious_ip(addr,time) then - if time < ref_time - then (print(ip_addr_to_string(addr)+" removed from dubious addresses list.\n"); - remove_inactive_dubious_IP(t,ref_time)) - else [h . remove_inactive_dubious_IP(t,ref_time)] - }. - -define One - remove_inactive_dubious_IP - ( - Var(List(DubiousIP)) v - ) = - protect - with ref_time = now - 600, // 10 minutes - v <- remove_inactive_dubious_IP(*v,ref_time). - - - The above function will be executed periodically by the servers's tasks machine. - - - - *** [6.4.5] Making the function for generating trust tickets. - -define One -> String - make_generate_trust_ticket - ( - DenialOfService dos - ) = - (One _) |-> "". - - - - - - - - *** [6.5] Starting the HTTP/HTTPS server. - - - The next function creates the directories for all sites (if they don't already exist). - -define One - create_directories - ( - List(Web_Site_Description) sites - ) = - if sites is - { - [ ] then unique, - [s1 . others] then - with site_dir = site_directory(s1), - forget(make_directory(site_dir+"/public",default_directory_mode)); - forget(make_directory(site_dir+"/upload_temporary",default_directory_mode)); - forget(make_directory(site_dir+"/private_download",default_directory_mode)); - forget(make_directory(site_dir+"/journal",default_directory_mode)); - create_directories(others) - }. - - - - - - Below are the commands for starting an HTTP server and an HTTPS server. - - -define StartServerResult - start_http_server - ( - Int32 ip_address, - Int32 port, - Server -> ((RWStream) -> One) handler, - Int32 retries, - DenialOfService dos - ) = - if start_server(ip_address, - port, - handler, - identity) is ok(server) - then print(" \r"); - ok(server) - else print("Port "+port+": retry number "+retries+"\r"); - sleep(1000); - start_http_server(ip_address,port,handler,retries+1,dos). - -public define StartServerResult - start_http_server - ( - Int32 ip_address, - Int32 port, - List(Web_Site_Description) sites, - DenialOfService dos - ) = - create_directories(sites); - start_http_server(ip_address,port, - make_http_handler(sites,dos), - 0, - dos). - - - For the HTTPS server, we have a problem which is due to the fact that 'anbexec' is not - yet able to manipulate several SSL server certificates. 'anbexec' and - 'predefined.anubis' must be changed. Sorry ! This will be done as soon as possible. The - 'solution' for the time being is to provide the common name of the unique SSL server - certificate. - - -define StartServerResult - start_https_server - ( - Int32 ip_address, - Int32 port, - String certificate_common_name, - Server -> (SSL_Connection -> One) handler, - Int32 retries, - DenialOfService dos - ) = - if start_ssl_server(ip_address, - port, - certificate_common_name, - handler, - identity) is ok(server) - then print(" \r"); - ok(server) - else print("Port "+port+": retry number "+retries+"\r"); - sleep(1000); - start_https_server(ip_address,port, - certificate_common_name, - handler,retries+1, - dos). - - -public define StartServerResult - start_https_server - ( - Int32 ip_address, - Int32 port, - String certificate_common_name, // of SSL server certificate - List(Web_Site_Description) sites, - DenialOfService dos - ) = - create_directories(sites); - start_https_server(ip_address,port,certificate_common_name, - make_https_handler(sites,dos), - 0,dos). - - - - - - - - - - *** [7] The web dispatcher. - - - *** [7.1] The dispatcher server. - -define One - send_dispatching_page - ( - RWStream conn, - String common_name, - Int32 port - ) = - print("Dispatching '"+common_name+"' to port "+port+"\n"); - forget(reliable_write(conn,to_byte_array( - "" - ))). - - - -define Maybe(DispatcherInfo) - find_host - ( - List(DispatcherInfo) l, - String host - ) = - if l is - { - [ ] then failure, - [h . t] then if h is site(name,port) then - if name = host - then success(h) - else find_host(t,host) - }. - - - -define Server -> ((RWStream) -> One) - make_dispatcher_handler - ( - Var(List(DispatcherInfo)) info_v, - DenialOfService dos - ) = - (Server server) |-> (RWStream conn) |-> - with start_time = (Int32)now, - if read_request_line(tcp(conn),start_time+*request_line_delay(dos),dos) is - { - error(msg) then print(format(msg)), - ok(request_line) then - if read_http_headers(tcp(conn),start_time+*headers_delay(dos),dos) is - { - error(msg) then print(format(msg)), - ok(headers) then if get_host_header_value(headers) is - { - failure then print("No 'HOST' HTTP header.\n"), - success(host) then - if find_host(*info_v,host) is - { - failure then print("Host: '"+host+"' not registered.\n"), - success(s) then if s is site(common_name,ip_port) then - send_dispatching_page(conn,common_name,ip_port) - } - } - } - }. - - -define One - dispatcher_update_error - ( - String file_path - ) = - print("web_dispatcher: unable to reread file: '"+file_path+"'.\n"). - - -define Bool - dispatcher_update_data - ( - String info_file_path, - Var(List(DispatcherInfo)) info_v, - Var(Int32) info_date_v - ) = - if directory_full_list(my_anubis_directory+"/web_sites","dispatcher.info","","") is - { - [ ] then false, - [h . t] then if h is - { - no_info(n) then false, - file(n,_,_,d) then if n = "dispatcher.info" - then (info_date_v <- d; - if (RetrieveResult(List(DispatcherInfo)))retrieve(info_file_path) is - { - cannot_find_file then false, - read_error then false, - type_error then false, - ok(info) then info_v <- info; true - }) - else false, - link(_,_,_,_) then false, - directory(_,_,_) then false - } - }. - - - - The loop within which the dispatcher updates its data every 3 seconds: - -define One - dispatcher_update_task - ( - String info_file_path, - Var(List(DispatcherInfo)) info_v, - Var(Int32) info_date_v - ) = - sleep(3000); - (if dispatcher_update_data(info_file_path,info_v,info_date_v) - then unique - else dispatcher_update_error(info_file_path)); - dispatcher_update_task(info_file_path,info_v,info_date_v). - - -public define One - start_web_dispatcher - ( - Int32 ip_address, // address for listening (typically 0: listen on all interfaces) - Int32 http_port, // typically 80 - DenialOfService dos - ) = - with info_file_path = my_anubis_directory+"/web_sites/dispatcher.info", - info_v = var((List(DispatcherInfo))[]), - info_date_v = var((Int32)0), - if dispatcher_update_data(info_file_path,info_v,info_date_v) - then if start_server(ip_address, - http_port, - make_dispatcher_handler(info_v,dos), - (One u)|->u) is - { - cannot_create_the_socket then - print("Cannot create the socket for HTTP server.\n"), - cannot_bind_to_port then - print("Cannot bind HTTP server to port "+http_port+".\n"), - cannot_listen_on_port then - print("HTTP server cannot listen on port "+http_port+".\n"), - ok(http_server) then - dispatcher_update_task(info_file_path,info_v,info_date_v) - } - else dispatcher_update_error(info_file_path). - - - - *** [7.2] The dispatcher web site. - -global define One - web_dispatcher - ( - List(String) args - ) = - start_web_dispatcher(0,80,load_denial_of_service_info). - - - - - - - *** [7.3] Managing the info file. - -define Int32 - register_ip_address - = - if ip_address(prompt(" numerical IP address (for HTTP): ")) is - { - failure then print(" *** Error: incorrect IP address.\n"); - register_ip_address, - success(n) then n - }. - - -define Int32 - register_ip_port - = - if string_to_integer(prompt(" IP port (for HTTP): ")) is - { - failure then print(" *** Error: incorrect IP port.\n"); - register_ip_port, - success(p) then if (0 =< p & p =< 65535) - then p - else print(" *** Error: IP port out of bounds.\n"); - register_ip_port - }. - - -define One - register_new_site - ( - Var(List(DispatcherInfo)) info_v - ) = - print("\n"); - print(" Registering a new site:\n"); - with name = prompt(" Site name: "), - with addr = register_ip_address, - with port = register_ip_port, - (protect info_v <- [site(name,port) . *info_v]); - print(" Site "+name+" at "+ip_addr_to_string(addr)+":"+port+" added\n (but not saved to disk).\n"). - - -define List(DispatcherInfo) - find_sites - ( - List(DispatcherInfo) l, - String name - ) = - if l is - { - [ ] then [ ], - [h . t] then if h is site(n,_) then - if find(name,n,0) is - { - failure then find_sites(t,name), - success(_) then [h . find_sites(t,name)] - } - }. - - -define String - pad - ( - String s, - Int32 l - ) = - if length(s) >= l - then s - else s+constant_string(l-length(s),' '). - - - -define One - show_sites_1 - ( - List(DispatcherInfo) l, - Int32 i - ) = - if l is - { - [ ] then unique, - [h . t] then if h is site(name,port) then - print(" ["+i+"] "+pad(name,40)+" "+" "+port+"\n"); - show_sites_1(t,i+1) - }. - - -define One - show_sites - ( - List(DispatcherInfo) l, - Int32 i - ) = - print(" Name Port\n"); - print(" --------------------------------------------------------\n"); - show_sites_1(l,i). - -define List(DispatcherInfo) - replace_info - ( - List(DispatcherInfo) l, - String site_name, - Int32 new_port - ) = - if l is - { - [ ] then alert, - [h . t] then if h is site(n,_) then - if n = site_name - then [site(n,new_port) . t] - else [h . replace_info(t,site_name,new_port)] - }. - -define List(DispatcherInfo) - delete_info - ( - List(DispatcherInfo) l, - String site_name, - ) = - if l is - { - [ ] then alert, - [h . t] then if h is site(n,_) then - if n = site_name - then t - else [h . delete_info(t,site_name)] - }. - - -define One - update_site - ( - Var(List(DispatcherInfo)) info_v, - String site_name, - Int32 old_port - ) = - print("\n"); - print(" Updating site '"+site_name+"': (currently: "+old_port+")\n"); - with new_port = register_ip_port, - answer = prompt(" Update '"+site_name+"' as: "+new_port+" [Y/N] ? "), - if (answer = "Y" | answer = "y") - then info_v <- replace_info(*info_v,site_name,new_port) - else unique. - - - -define Bool - compare - ( - DispatcherInfo d1, - DispatcherInfo d2 - ) = - if d1 is site(n1,_) then - if d2 is site(n2,_) then - string_less(n1,n2). - - - -define One - update_site - ( - Var(List(DispatcherInfo)) info_v - ) = - print("\n"); - with prefix = prompt(" Search for site to update: "), - if find_sites(*info_v,prefix) is - { - [ ] then print(" No site found.\n"); - update_site(info_v), - [h . t] then - show_sites(qsort([h . t],compare),1); - with i1 = prompt(" Choose a site to update [1/.../"+(length(t)+1)+"]: "), - if string_to_integer(i1) is - { - failure then print(" *** Error: site number not recognized.\n"); - update_site(info_v), - success(ii1) then if nth(ii1-1,*info_v) is - { - failure then print(" *** Error: site number "+i1+" does not exist.\n"); - update_site(info_v), - success(site_info) then if site_info is site(name,old_port) then - update_site(info_v,name,old_port) - } - } - }. - - -define One - delete_site - ( - Var(List(DispatcherInfo)) info_v, - String site_name, - Int32 old_port - ) = - print("\n"); - print(" Deleting site '"+site_name+"': (currently: "+old_port+")\n"); - with answer = prompt(" Are you sure you want to delete site: '"+site_name+"' [Y/N] ? "), - if (answer = "Y" | answer = "y") - then info_v <- delete_info(*info_v,site_name) - else print(" Site '"+site_name+"' not deleted.\n"). - - -define One - delete_site - ( - Var(List(DispatcherInfo)) info_v - ) = - print("\n"); - with prefix = prompt(" Search for site to delete: "), - if find_sites(*info_v,prefix) is - { - [ ] then print(" No site found.\n"); - delete_site(info_v), - [h . t] then - show_sites(qsort([h . t],compare),1); - with i1 = prompt(" Choose a site to delete [1/.../"+(length(t)+1)+"]: "), - if string_to_integer(i1) is - { - failure then print(" *** Error: site number not recognized.\n"); - delete_site(info_v), - success(ii1) then if nth(ii1-1,*info_v) is - { - failure then print(" *** Error: site number "+i1+" does not exist.\n"); - delete_site(info_v), - success(site_info) then if site_info is site(name,old_port) then - delete_site(info_v,name,old_port) - } - } - }. - - -define One - manager - ( - Var(List(DispatcherInfo)) info_v, - String file_path - ) = - print("\n"); - print(" --- Welcome to the Web Dispatcher Manager ---\n"); - with l = length(*info_v), - print(" "+l+" site"+(if l>1 then "s" else "")+" currently registred.\n"); - print(" [L] List registered sites.\n"); - print(" [R] Register a new site.\n"); - print(" [U] Update a registred site.\n"); - print(" [D] Delete a registred site.\n"); - with propose_write_v = var((Bool)true), - action = prompt(" Choose an action [L/R/U/D]: "), - (if (action = "L" | action = "l") then (show_sites(*info_v,1); propose_write_v <- false) else - if (action = "R" | action = "r") then register_new_site(info_v) else - if (action = "U" | action = "u") then update_site(info_v) else - if (action = "D" | action = "d") then delete_site(info_v) else - print("Action not recognized.\n")); - print("\n"); - if *propose_write_v then - with result = prompt(" Write modifications to data base [Y/N] ?"), - if (result = "Y" | result = "y") - then if save(*info_v,file_path) is - { - cannot_open_file then print(" File '"+file_path+"' not found.\n"), - write_error then print(" Error while writing file '"+file_path+"'.\n"), - ok then print(" Data base has been modified.\n") - } - else print(" Data base not modified.\n") - else unique. - - - -global define One - manage_web_dispatcher - ( - List(String) args - ) = - with info_v = var((List(DispatcherInfo))[]), - with file_path = my_anubis_directory+"/web_sites/dispatcher.info", - if (RetrieveResult(List(DispatcherInfo)))retrieve(file_path) is - { - cannot_find_file then print("File '"+file_path+"' does not exist.\n"); - with answer = prompt("Create it [Y/N] ? "), - if (answer = "Y" | answer = "y") - then if save((List(DispatcherInfo))[],file_path) is - { - cannot_open_file then - print("Cannot create file '"+file_path+"'.\n"), - write_error then - print("Error while creating file '"+file_path+"'.\n"), - ok then manager(info_v,file_path) - } - else unique, - read_error then print("Error while reading file '"+file_path+"'.\n"), - type_error then print("File '"+file_path+"' is corrupted.\n"), - ok(info) then info_v <- info; - manager(info_v,file_path) - }. - - - - - + + *Project* The Anubis Project + + *Title* A Multi Host HTTP/HTTPS Server + + *Copyright* Copyright (c) Alain Prouté 2003. + + + *Author* Alain Prouté + + + *Revised* August 2005. + + + + *Overviews* + In this file a HTTP/HTTPS server is defined, which is able to handle multiple hosts + (virtual hosts). It answers HTTP/HTTPS requests, sends files (images or any other kind + of file), constructs HTML pages on the fly using informations received from the client + (when the URI ends by '.awp'), handles uploading of files and redirections. It is + multitasking by itself, and can handle any number of sites and clients simultaneously. + It should better be used in conjunction with 'making_a_web_site.anubis' to be found in + the same directory. If you use 'web/making_a_web_site.anubis', you don't need to read + this file. + + + ----------------------------------- Table of Contents --------------------------------- + + *** (1) Multihosting and redirections. + *** (2) The incompatibility between SSL and virtual hosts. + *** (3) HTTP headers and web arguments. + *** (4) Site descriptions. + *** (5) Protection against denial of service attacks. + *** (6) Starting your HTTP and HTTPS servers. + *** (7) Private download. + *** (8) About web argument names. + *** (9) A web dispatcher. + + --------------------------------------------------------------------------------------- + + + + + *** (1) Multihosting and redirections. + + This HTTP/HTTPS server can handle several host (also called 'virtual hosts'), in other + words, you may have several sites on the same server, with the same IP address and same + port numbers, but distinct 'host names'. + + A HTTP request sent by a browser contains the following informations: + + - a 'host name', + - an URI (Uniform Resource Identifier), + - HTTP headers, + - web arguments (in the form 'name=value'). + + Actually, the host name is just the value of the HTTP header whose name is 'Host'. The + host name indicates which site is requested. Hence, it is the primary information for + branching to the right site. If there is no 'Host' HTTP header in the request, the + request is denied. + + From now on, we may assume that the host is determined, and consequently that we are + concerned by only one site. Each site has his own directories on the server's + disk. + + Each site also has a list of 'redirections'. A redirection is a triplet, like this one: + + redirect("/", "www.our-business.com", "/homepage.awp") + + meaning that if the host is "www.our-business.com", and if the requested URI is "/", + then the URI to be served is "/homepage.awp". 'redirect' is a constructor of the type + 'Redirection' defined in 'web/common.anubis'. + + Now, an URI may end by ".awp" (meaning 'Anubis Web Page') or not. If it does, the + server understands that an HTML page must be constructed on the fly, and to that end it + calls the 'awp handler' of the site. Otherwise, the URI must end by a known extension, + like ".jpg", ".png", ".txt", etc... and represents a file path relative to the + 'public' directory of the site. If these conditions are satisfied, the file is sent to + the client. Known extensions are recorded in 'web/mime.anubis'. + + + + + *** (2) The incompatibility between SSL and virtual hosts. + + Handling virtual hosts makes a problem under SSL (i.e. when using HTTPS), which is due + to the fact that the guys at Netscape who designed SSL probably did not have the + question of virtual hosts in mind. Indeed, the SSL handshake is completed before the + server can know about the value of the 'Host' HTTP header, so that it cannot know which + server certificate must be sent to the client. This makes a problem, because the + browser will not accept a certificate whose common name does not correspond to the name + of the requested host. The user will have to accept the certificate manually, which is + not good for the security image of the site. This problem has at least two solutions + (as far as Anubis is concerned). + + Solution 1. Arrange so that the network interface on which the server is listening + has at least as many different IP addresses as you have virtual hosts. Such + supplementary IP addresses are called 'IP Aliases'. In this case, start one HTTPS + server for each virtual host, each one listening on a different address. For the time + being, this method is applicable under Anubis only if you start as many instances of + 'anbexec' as you have virtual hosts, because each instance of 'anbexec' can handle only + one server certificate. Of course, getting IP aliases is another problem to be solved + with your Internet provider. + + Solution 2. We propose a simple solution, using only one server certificate (hence + only one instance of 'anbexec'). Since, we have only one server certificate, we must + introduce a notion of 'main host', i.e. a host containing all other 'virtual + hosts'. The unique server certificate belong to the main host, so that only the main + host is identified by the client. The client must trust the main host and be confident + that the main host redirects him to the right virtual host. Actually, the process will + be transparent to the client, except that the client will see the name of the main host + instead of the name of the virtual host in the 'location' field of the browser. + + So, assume that the name of main host is 'www.securedhost.com', and that the names of + the virtual hosts are: + + actual name simplified name + ----------------------------------------------------- + www.virtual1.com virtual1 + www.virtual2.com virtual2 + www.virtual3.com virtual3 + + Then the (confidential) document '/doc/my_document.pdf' on 'www.virtual2.com' will have + the URL: + + https://www.securedhost.com/virtual2/doc/my_document.pdf + + In order to work transparently, this solution must combine HTTP and HTTPS. Indeed, the + vitual host must have a first page reachable under HTTP, through the URL: + + http://www.virtual2.com/ + + The HTTP server will redirect this URL to the awp handler of virtual host 'virtual2'. + The handler of this virtual host is able to generate a first page containing the + following HTML meta: + + , + + so that the client is immediately redirected to the main host under HTTPS (hence + accepting tranparently the server certificate). The awp handler of 'virtual2' then + redirects this URL to the home page (maybe a login page) of 'virtual2'. + + See 'web/making_a_web_site.anubis' for the sequel of this story. + + + + + + *** (3) HTTP headers and web arguments. + + Each HTTP request which arrives on the server contains a request line followed by a + series of HTTP headers. Each HTTP header is a pair '(name,value)' assigning a value to + a name. The type 'HTTP_header' is defined in 'web/common.anubis'. + + The request may also have a 'body'. The body contains either 'web arguments' or + uploaded files (or both). The request line itself may also contain web arguments (in a + so-called 'query string'). Like HTTP headers, 'web arguments' are pairs + '(name,value)', but the difference is that these pairs are generated by the page within + which the client clicks, while HTTP headers are generated by the browser itself. The + type 'Web_arg' is defined in 'web/common.anubis'. It has two alternatives, one for + ordinary web arguments (pairs) and one for uploaded files. + +read CXM_common.anubis +read tools/basis.anubis +read CXM_mime.anubis + + + + *** (4) Site descriptions. + + The type HTTP_Info gathers informations comming along with the client's request. These + informations are rarely used for composing HTML pages. Nevertheless, they are at your + disposal. + +public type HTTP_Info: + http_info + ( + Int32 ip_address, // IP address of the client + String uri, // URI requested by the client + List(HTTP_header) http_headers, // HTTP headers sent by the client + One -> String generate_trust_ticket // may be used against denial of + // service attacks + ). + + + + Each site is described by a 'web site description', which is a datum of type + 'Web_Site_Description'. + +public type Web_Site_Description: + web_site_description( + List(String) common_names, + String site_directory, + List(Redirection) redirections, + String charset, + List(String) journal_extensions, + List(String) journal_headers, + String authorization_secret, + List(MIME) known_mime_types, + (String host_name, + HTTP_Info http_info, + List(Web_arg) lwa, + Bool is_https) -> (List(HTTP_header), + Printable_tree) awp_handler, + (List(Web_arg) lwa) -> One before_send_file). + + The component 'common_names' is the list of names of the site, like for example + "www.our-business.com". The reason why we have a list of common names instead of a + single common name, is that it may be useful to have a common name like "192.168.0.1" + for testing. + + 'charset' is a string which will determine the character encoding to be used by the + browser. Typically, this string is one of: "UTF-8", "ISO-8859-1", "Windows-1252", + etc... + + 'journal_extensions' is the list of URI extensions for which you want a log in the + journal (and on the console). When a request arrives, and if the extension is a member + of this list, a message is printed into the journal of the site including the date, the + IP address of the client, the complete HTTP request line. The HTTP headers whose name + is a member of 'journal_headers' are also printed in the journal. A reasonable minimum + for these two components is: + + [".awp"] for journal_extensions + ["user-agent"] for journal_headers + + 'authorization_secret' is a string which should just be unguessable. You may choose + something like (but don't choose this one !): + + "Hg8kJe42gCML9jNH-74" + + i.e. a sequence of characters typed at random, long enough to be unguessable. This is + used by the 'private download' mecanism, which is discussed later in this file. + + The component 'awp_handler' is a function of type: + + (String host_name, + HTTP_Info http_info, + List(Web_arg) web_args, + Bool is_https) -> Printable_tree + + ('Printable_tree' is a substitute for 'String' and is defined in + 'tools/basis.anubis'). This function is the 'awp handler' for the site. When the URI + ends by ".awp", this function is called, and the result (an HTML page) is sent to the + client over the connection. The last operand to this function is a boolean which is + 'true' when the requests arrives through the HTTPS channel, and 'false' when it arrives + through the HTTP channel. + + + + + + + + *** (5) Protection against denial of service attacks. + + We need to protect our servers against 'denial of service' attacks. The attack may be + send automatically from machines which are infested by viruses. In that case, our + server is saturated of connections (all virtual machines at work), but nothing is + comming on the connections. In order to avoid this problem, we propose the following: + + (1) Limit the number of simultaneous connections (say to 100). + (2) Close a connection if the request is not complete after say 10 seconds. + (3) Close the connection if the request is bigger than a given size (normal requests + are small except when there are uploaded files. + (4) Close the connection during the sending of the answer if the client is waiting + too much. + (5) Record all IP addresses with which we have encountered one of the problems above. + (6) Immediately close the connections if the IP address is in our list. + (7) Remove an address from the list only after 5 minutes of inactivity of this + address. + (8) Maintain a list of reliable IP addresses. + + Of course, all the above are approximative solutions which may in some circumstances + become either cumbersome or also partially block the system. So, it is needed to have a + set of dynamically modifiable parameters in order to master the behavior of this + mecanism. + + + Each dubious IP address is recorded together with its last activity time. + +public type DubiousIP: + dubious_ip (Int32 address, + Int32 last_activity). + + +public type DenialOfService: + denial_of_service(Var(Int32) max_connections, + Var(Int32) request_line_delay, // seconds + Var(Int32) headers_delay, + Var(Int32) answer_delay, + Var(List(DubiousIP)) list_of_dubious, + Var(List(Int32)) reliable_addresses). + + The informations in this set of variables are stored serialized into the file + 'my_anubis/web_sites/dos_info'. If this file does not exist a set if variables with + default values is created. The values are saved on the disk each time they are + modified. + +public define DenialOfService load_denial_of_service_info. + + + + *** (6) Starting your HTTP and HTTPS servers. + + When your web site descriptions are ready, you can start a pair of servers (a HTTP + server and a HTTPS server) for serving your web sites. Notice that there are always + two servers, regardless of the number of web sites, and that each web sites normally + uses the two servers. + + +public define StartServerResult + start_http_server + ( + Int32 ip_address, + Int32 http_port, + List(Web_Site_Description) web_sites, + DenialOfService dos + ). + +public define StartServerResult + start_https_server + ( + Int32 ip_address, + Int32 https_port, + String certificate_common_name, + List(Web_Site_Description) web_sites, + DenialOfService dos + ). + + The first argument 'ip_address' is the IP address on which the servers listen. If you + put 0, the servers listen on all adresses of the machine (which is useful if the + machine has several network interfaces). Otherwise, use the function 'ip_address' + defined in 'tools/basis.anubis' for composing a particular IP address. + + The next arguments are the port numbers for HTTP and HTTPS. The usual values are 80 and + 443, but you may have reasons to choose other values. + + The next argument is the list of your web site descriptions. All the sites described in + this list will be accessible on the server. + + The argument 'dos' is a set of dynamic variables containing the informations for + protecting the servers against denial of service attacks. + + + + + + + *** (7) Private download. + + It may happen that you want to propose private files for download. This means that such + a file could be downloaded only by the authorized person, and should not be seen by any + other one. This feature can be used only under HTTPS, not under HTTP. + + The file may be located anywhere on the server. Hence, the file has a complete absolute + path, like for example: + + /home/georges/my_documents/my_text.pdf + + which has nothing to do with the directories of the web server. Now, you may also want + to show another path or simply just a name to the client, not the actual absolute path + above, which may need to remain secret. So for example, the same file may appear to the + client as: + + informations.pdf + + The page must provide a link with an authorization. The authorization is just a web + argument, whose name is "zauth". The value of this web argument is computed by hashing + some secret string (known only from the programmer of the web site) with the absolute + path of the file. The HTTPS request will have the form: + + GET /informations.pdf?zauth=d38161f5b4e87e2d46e06ff8b3e233be563794d1 + + The server will search for a file named + + zd38161f5b4e87e2d46e06ff8b3e233be563794d1 + + (i.e. "z" concatenated with the value of the authorization) in the subdirectory + 'private_download' of the site directory. This file contains the absolute path of the + file, i.e: + + /home/georges/my_documents/my_text.pdf + + At that point, the server may hash the secret string and the absolute path together, to + check if the client is authorized to download the file. If it is the case, it sends the + file (the MIME type is declared as 'application/octet-stream' if it is not recognized). + The file is sent under the visible name. + + The server creates automatically the subdirectory 'private_download/' within the 'site + directory' (for each web site) if it does not already exist. Files in this directory + are deleted when they become too old (for example, after 3 days of life). + + Here is the function for computing the value of the authorization, and for making the + authorization file in 'private_download'. + +public define String + make_authorization + ( + String site_directory, + String authorization_secret, // known only by the programmer of the web site + String absolute_path // on server + ). + + See 'web/making_a_web_site.anubis' for the construction of the link for downloading. + + + + + + + + + *** (8) About web argument names. + + The server reserves the name "zauth" for the authorization in the private download + mecanism. Also, if the name of a web arguments begins by "p" (like 'password'), it does + not print the value of the web argument neither on the console or in the journal. A + good politics is to prefix all web arguments by letters distinct from 'p' and 'z'. This + method is used in 'web/making_a_web_site.anubis'. This will avoid clashes of names. + + + + + + + *** (9) A web dispatcher. + + For hosting several sites you may prefer another method which we now describe. We start + a HTTP server on port 80 (or on another port). This server is called the + ``dispatcher''. When a requests arrives, the dispatcher examines the ``host'' HTTP + header, so that it gets the name of the requested host. Then it sends to the client a + page like this one: + + + + + + + + + + where the URL represented by '...' is the URL of the requested site. This URL may have + the same IP address as the dispatcher, except that the port number is different. It may + also have a different IP address. + + The dispatcher uses the file 'my_anubis/web_sites/dispatcher.info'. This file contains + a serialized datum of type 'List(DispatcherInfo)'. + +public type DispatcherInfo: + site(String common_name, + Int32 http_port). + + The dispatcher does not write into this file. It reads it when it starts, and rereads + it each time the date of last modification of the file changes, so that the dispatcher + always has up to date data. The file may be managed (written and updated) by another + program. + + So, for each site, the dispatcher knows the common name (needed to recognize the 'host' + HTTP header), and the pair (ip_address,port) used by the actual site for HTTP. The + dispatcher does not worry about HTTPS. HTTPS must be managed by the actual site. + + The dispatcher is started by: + +public define One + start_web_dispatcher + ( + Int32 ip_address, // address for listening (typically 0) + Int32 port, // typically 80 + DenialOfService dos + ). + + A command line tool for managing the file 'my_anubis/web_sites/dispatcher.info' is also + provided: + + global define One + manage_web_dispatcher + ( + List(String) args + ). + + + + + + + + --- That's all for the public part ! -------------------------------------------------- + + + + + + + + ----------------------------------- Table of Contents --------------------------------- + + *** [1] Types which are private to this file. + + *** [2] Tools. + *** [2.1] Formating an error message. + *** [2.2] Converting IP addresses. + *** [2.3] Reading and unputting characters. + *** [2.4] Reading and discarding characters. + *** [2.5] Reading a character string. + *** [2.6] Padding integers with zeros. + *** [2.7] Converting web arguments to ASCII. + *** [2.8] Server description. + + *** [3] Managing the journal. + *** [3.1] Naming journal files. + *** [3.2] Formating HTTP headers. + *** [3.3] Formating web arguments. + *** [3.4] Formating the whole request. + *** [3.5] Putting it in the journal file (and on the console). + + *** [4] Reading the HTTP request. + *** [4.1] Skipping leading blanks. + *** [4.2] Reading a new line. + *** [4.3] Reading a 'word'. + *** [4.4] Separating the URI from the query string. + *** [4.5] Reading the web arguments. + *** [4.7] Reading the request line. + *** [4.8] Reading the HTTP headers. + *** [4.9] Getting the size of the request's body. + *** [4.10] Reading the body of the request. + + *** [5] Making the HTTP answer. + *** [5.1] Avoiding illegal URIs. + *** [5.2] Managing authorizations for downloading private files. + *** [5.3] Recognizing MIME types. + *** [5.4] Formating HTTP headers. + *** [5.5] Sending a file. + *** [5.6] Answering a www-url encoded request. + *** [5.7] Answering a multipart/form-data encoded request. + *** [5.7.1] Finding the boundary. + *** [5.7.2] Reading attributes from a multipart entity. + *** [5.7.3] Creating a temporary filename for an uploaded file. + *** [5.7.4] Saving an uploaded file under a temporary filename. + *** [5.7.5] Removing the path from a file name. + *** [5.7.6] Reading a multipart entity. + *** [5.8] Handling redirections. + *** [5.9] Answering both sorts of requests. + + *** [6] The HTTP/HTTPS servers. + *** [6.1] The HTTP request handler. + *** [6.2] Server's tasks. + *** [6.3] Starting the HTTP/HTTPS servers. + + *** [7] The web dispatcher. + *** [7.1] The dispatcher server. + *** [7.2] The dispatcher web site. + *** [7.3] Managing the info file. + + --------------------------------------------------------------------------------------- + + + + +read tools/basis.anubis +read tools/findstring.anubis +read tools/connections.anubis + + + + + + *** [1] Types which are private to this file. + + We use the following self-explanatory types. + +type Error: + cannot_read_from_connection, + not_get_or_post_request(String), + end_of_line_expected, + incorrect_content_length_value, + colon_expected, + timeout(Int32). + +type HTTP_RequestType: + get, + post. + +type HTTP_RequestLine: + request_line (HTTP_RequestType type, + String uri, + List(Web_arg) query_string). + +type EncodingType: + www_url, + multipart_form_data. + + + + + + *** [2] Tools. + + *** [2.1] Formating an error message. + + The next function formats an error message. + +define String + format + ( + Error msg + ) = + if msg is + { + cannot_read_from_connection then + "Cannot read from connection.\n", + not_get_or_post_request(s) then + "The request did not begin by 'GET' or 'POST': "+s+".\n", + end_of_line_expected then + "End of line expected.\n", + incorrect_content_length_value then + "Incorrect value for HTTP header 'Content-Length'.\n", + colon_expected then + "':' was expected.\n", + timeout(n) then + //"time out: "+n+"\n" + //"time out.\n" + "" + }. + + + + + + + *** [2.2] Converting IP addresses. + + We need two conversion functions for IP addresses: + + (Word8,Word8,Word8,Word8) --> Int32 ip_address + Int32 --> String ip_addr_to_string + + These conversions are defined in 'tools/basis.anubis'. + + + + + + + + + *** [2.3] Reading and unputting characters. + + We need a mecanism for unputting several characters (actually at least 3). This is + because when reading the client connection, we must sometimes go ahead several + characters, and virtually put them back into the connection, so that they can be + reread. Of course, we do not send them back to the client. We store them in a list + (hold by the variable 'unput_chars'), and we manage this list, so that characters may + be virtually put back in the connection (this is called 'unputting'). + +variable List(Word8) unput_chars = []. + + The most recently read one is the head of list. Fortunately, this variable is private + to this virtual machine (hence to this client). + + +define One + unput // unputting a character (add it in front of the list) + ( + Word8 character + ) = + unput_chars <- (List(Word8))[character . *unput_chars]. + + + +define One record_dubious_IP(Int32 addr,DenialOfService dos). + +variable Int32 sttm = 0. // contains the start time for this connection. + +define Result(Error,Word8) + record_dubious_connection + ( + Connection conn, + Int32 dead_line, + DenialOfService dos, + ) = + if remote_IP_address_and_port(conn) is (addr,port) then + record_dubious_IP(addr,dos); + print("Recording IP address "+ip_addr_to_string(addr)+ + " as dubious after "+(dead_line-*sttm)+" seconds. Total: "+ + length(*list_of_dubious(dos))+"\n"); + error(timeout(dead_line)). + + +define Result(Error,Word8) + read_one_byte + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + //if now > dead_line then record_dubious_connection(connection,dead_line,dos) else + if read(connection,1,600) is // the connection is closed after 10 minutes of inactivity + { + error then error(cannot_read_from_connection), + timeout then error(timeout(600)), + //record_dubious_connection(connection,dead_line,dos), + ok(ba) then if nth(0,ba) is + { + failure then error(cannot_read_from_connection), + success(c) then ok(c) + } + }. + + +define Result(Error,Word8) + next_char // reading a character (check the list first, and read on the connection + // only when the list is empty). + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if *unput_chars is + { + [ ] then read_one_byte(connection,dead_line,dos), + + [h . t] then + unput_chars <- t; + ok(h) + }. + + + + + + + + *** [2.4] Reading and discarding characters. + + The next function reads the specified number of bytes (this is the same as + 'characters') from the connection and discards them. This is used for discarding CR LF + just before the body of a request. + +define Result(Error,One) + read_and_ignore + ( + Connection connection, // to client + Int32 dead_line, + Int32 number_of_characters, // number of characters to read and ignore + DenialOfService dos + ) = + if number_of_characters =< 0 then ok(unique) else + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then read_and_ignore(connection,dead_line,number_of_characters-1,dos) + }. + + + + + + + + *** [2.5] Reading a character string. + + Sometimes values of HTTP attributes or web args are presented in the form of double + quoted strings. The next function handles the reading of such things. The leading + double quote is already read in. We must read subsequent characters until the next non + backslashed double quote. + +define Result(Error,String) + read_string + ( + Connection connection, // connection with the client + Int32 dead_line, + List(Word8) so_far, // characters read so far (in reverse order) + DenialOfService dos + ) = + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if c = '\\' + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(d) then + if d = '\"' + then read_string(connection,dead_line,['\"' . so_far],dos) + else read_string(connection,dead_line,[d, c . so_far],dos) + } + else if c = '\"' + then ok(implode(reverse(so_far))) + else read_string(connection,dead_line,[c . so_far],dos) + }. + + + + + + + + *** [2.6] Padding integers with zeros. + + 'zero_pad_2' transforms an integer (which is assumed to be between 0 and 99) into a + string with exactly two digits. This is used for formating days, hours, minutes and + seconds. + +define String + zero_pad_2 + ( + Int32 n + ) = + with s = integer_to_string(n), + if length(s) < 2 + then "0"+s + else s. + + + + + + + + *** [2.7] Converting web arguments to ASCII. + + The function 'web_to_ascii' gets a character string and replaces web encoding by normal + ASCII encoding. This amounts to replacing: + + + by blank + %xx by the character whose ASCII code is xx in hexadecimal + + Note: We assume that '9' < 'A' (which is the case for ASCII code). + + + +define Word8 + web_decode + ( + Word8 x1, + Word8 x2 + ) = + with z1 = word8_to_int32(x1), + n1 = if z1 =< '9' then (z1 - '0') else (z1 - 'A' + 10), + z2 = word8_to_int32(x2), + n2 = if z2 =< '9' then (z2 - '0') else (z2 - 'A' + 10), + n = (n1 << 4) + n2, + truncate_to_word8(n). + + + +define String + web_to_ascii + ( + String web_string, + Int32 n, // current position in web_string + List(Word8) so_far + ) = + if nth(n,web_string) is + { + failure then implode(reverse(so_far)), + success(c) then + if c = '+' + then web_to_ascii(web_string,n+1,[' ' . so_far]) + else if c = '%' + then if nth(n+1,web_string) is + { + failure then implode(reverse(so_far)), + success(x1) then if nth(n+2,web_string) is + { + failure then implode(reverse(so_far)), + success(x2) then web_to_ascii(web_string,n+3,[web_decode(x1,x2) . so_far]) + } + } + else web_to_ascii(web_string,n+1,[c . so_far]) + }. + + + + + + + + + *** [3] Managing the journal. + + Concurrently working machines should not try to access the same file at the same + time. This problem may be solved by using the 'protect' mecanism. + + + + *** [3.1] Naming journal files. + + Since journal messages are rather prolific, we should have at least one file per + hour. Hence, the name of a journal file must be constructed from the current year, + month, day and hour. For example, it may be: + + 2003_03_12_19 + + (this is for the journal of 7 PM to 8 PM, 2003/mar/12). + +define String + make_current_journal_file_name + = + if convert_time(now) is date_and_time(y,m,d,h,_,_,_,_,_) then + integer_to_string(y)+"_"+ + zero_pad_2(m)+"_"+ + zero_pad_2(d)+"_"+ + zero_pad_2(h). + + + + + + + + *** [3.2] Formating HTTP headers. + + HTTP headers may be shown on the console or written in the journal. The function below + formats a list of HTTP headers. + +define String + show_format + ( + Web_Site_Description desc, + List(HTTP_header) headers, + ) = + if headers is + { + [ ] then "", + [h . t] then if h is http_header(name,value) then + if member(journal_headers(desc),name) + then " | "+name+": "+value+"\n"+show_format(desc,t) + else show_format(desc,t) + }. + + + + + + + *** [3.3] Formating web arguments. + + The same thing for web arguments. + +define String + show_format + ( + List(Web_arg) lwa + ) = + if lwa is + { + [ ] then "", + [h . t] then if h is + { + web_arg(n,v) then + " | "+n+"="+(if nth(0,n) = success('p') then "" else v)+"\n"+show_format(t), + upload(n,fn,tfn) then + " | "+n+"="+fn+" (uploaded as '"+tfn+"')\n"+show_format(t) + } + }. + + + + + + + *** [3.4] Formating the whole request. + + It is cheap to transform month numbers into abbreviated month names. This enhances the + readability of the journal. + +define String + format_month + ( + Int32 m + ) = + if m = 1 then "jan" else + if m = 2 then "feb" else + if m = 3 then "mar" else + if m = 4 then "apr" else + if m = 5 then "may" else + if m = 6 then "jun" else + if m = 7 then "jul" else + if m = 8 then "aug" else + if m = 9 then "sep" else + if m = 10 then "oct" else + if m = 11 then "nov" else + if m = 12 then "dec" else + "???". + + + Below we format a whole HTTP request. This may give this (actually, it depends on how + you defined the values of 'journal_headers' and 'journal_extensions'): + + [3] 2003/mar/10 10:06:57 from 123.456.123.456: /homepage.awp + | host: www.the-best-one.com + | user-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 + + The leading number between brackets is the number of the virtual machine which served + the URI. + +define String + format_request + ( + Web_Site_Description desc, + Connection client_connection, + HTTP_RequestLine request_line, + List(HTTP_header) headers, + List(Web_arg) web_args + ) = + with dt = convert_time(now), + if remote_IP_address_and_port(client_connection) is (addr,port) then + integer_to_string(year(dt))+"/"+format_month(month(dt))+"/"+zero_pad_2(day(dt))+" "+ + zero_pad_2(hour(dt))+":"+zero_pad_2(minute(dt))+":"+zero_pad_2(second(dt))+ + " from "+ip_addr_to_string(addr)+ + ": "+uri(request_line)+"\n"+ + show_format(desc,headers)+ + show_format(web_args). + + + + + + + + *** [3.5] Putting it in the journal file (and on the console). + + We must not forget to 'protect' this operation, so that the messages of two machines + (working for the same site) will not be mixed together. + +define One + log_journal_msg + ( + Web_Site_Description desc, + String msg, + ) = + with msg = to_byte_array("["+virtual_machine_id+"] "+msg+"\n"), + protect + ( + if file(site_directory(desc)+"/journal/"+make_current_journal_file_name,append) is + { + failure then unique, + success(journal_file) then + forget(reliable_write(file(journal_file),msg)) + }; + forget(reliable_write(file(stdout),msg)) + ). + + + + + + + + *** [4] Reading the HTTP request. + + + *** [4.1] Skipping leading blanks. + + One of the peculiarities of HTTP is that the characters 13 (carriage return) and 10 + (line feed) followed by either a space (32) or a tab (9), is considered as a blank not + containing any new line. 'skip_http_blanks' must skip all blanks characters until the + first non blank character, which should not be read in. Obviously, because of the above + peculiarity, we need at least 3 characters of lookahead to do this. In other words, we + must be able to unput at least 3 characters (hopefully we are). + + Strictly blanks characters are 'space' and 'tab'. + +define Bool + is_strict_blank + ( + Word8 c + ) = + if c = ' ' then true else c = '\t'. + + + On the contrary, blanks include 13 and 10. + +define Bool + is_blank + ( + Word8 c + ) = + if c = ' ' then true else + if c = '\t' then true else + if c = 13 then true else + c = 10. + + + Skipping HTTP blanks. + +define Result(Error,One) + skip_http_blanks + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if is_strict_blank(c) + then skip_http_blanks(connection,dead_line,dos) + else if c = 13 + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), // (unput(c); ok(unique)), + ok(d) then + if d = 10 + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), // (unput(d); unput(c); ok(unique)), + ok(e) then + if is_strict_blank(e) + then skip_http_blanks(connection,dead_line,dos) + else (unput(e); unput(d); unput(c); ok(unique)) + } + else (unput(d); unput(c); ok(unique)) + } + else (unput(c); ok(unique)) + }. + + + + + + + + + *** [4.2] Reading a new line. + + Normally in HTTP a new line is the sequence 13 10 (carriage return line feed), not + followed by a space or tabulator. If it is followed by a space or tabulator, the three + characters are considered blanks, and no new line has been read. Before trying to read + a new line, we first skip leading spaces and tabs. Then we try to read 13 and 10, and + we read another character. if this character is space or tab, we consider we have read + only blanks and we continue reading in order to find our new line. Otherwise, we unput + this character (which may be for example the first character of the name of the next + header), and answer that we have seen a new line. + + Warning: we must not use this function for reading the last pair (13,10) before the + beginning of the body, because if the body is empty, there is no character to read + after this pair, so that the server could wait for a character which will never + come. This is the reason for 'read_and_ignore' above, which is used precisely for + reading that last (13,10) pair. + +define Result(Error,One) + read_new_line + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if skip_http_blanks(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if c = 13 + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(d) then + if d = 10 + then ok(unique) + else (unput(d); + unput(c); + error(end_of_line_expected)) + } + else (unput(c); + error(end_of_line_expected)) + }}. + + + + + + + + + *** [4.3] Reading a 'word'. + + A 'word' is a sequence of characters which begins either by a double quote or not by a + double quote. (However, any leading blanks are read in and ignored. This is + accomplished by 'skip_http_blanks'.) If it begins by a double quote, it is read like a + string, i.e. it ends at the next (non backslashed) double quote. Otherwise, it is + right delimited by any character which may be considered as 'blank'. If the word is + double quoted, the closing double quote is read in. On the contrary, if the word is not + double quoted, the right delimiting blank character is not read in (it is 'unput' back + into the connection), and may be read in again. This is needed because carriage return + or line feed which are 'blank', also have a meaning in HTTP. + +define Result(Error,String) + read_word_aux + ( + Connection connection, + Int32 dead_line, + List(Word8) so_far, + DenialOfService dos + ) = + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if is_blank(c) + then (unput(c); + ok(implode(reverse(so_far)))) + else read_word_aux(connection,dead_line,[c . so_far],dos) + }. + +define Result(Error,String) + read_word + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if skip_http_blanks(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if c = '\"' + then read_string(connection,dead_line,[],dos) + else read_word_aux(connection,dead_line,[c],dos) + } + }. + + + + + + + + + *** [4.4] Separating the URI from the query string. + + A 'query string' may be postfixed to the URI, just after a question mark. For example, + the client may send the following request: + + GET /catalog.awp?item=3&color=blue + + We separate this into an URI: "/catalog.awp" and the string: "item=3&color=blue" which + will be later transformed into the list: + + [web_arg("item","3"),web_arg("color","blue")] + + +define (String,String) + separate_uri_from_query_string + ( + String uri_and_query_string, + Int32 n + ) = + if nth(n,uri_and_query_string) is + { + failure then (uri_and_query_string,""), + success(c) then + if c = '?' + then (substr(uri_and_query_string,0,n), + substr(uri_and_query_string,n+1,length(uri_and_query_string)-(n+1))) + else separate_uri_from_query_string(uri_and_query_string,n+1) + }. + + + + + + + + + + *** [4.5] Reading the web arguments. + + HTTP/HTTPS requests are sent in one of two formats: + + (1) www-url encoded + (2) multipart/form-data encoded + + The first one is the normal (historical) way of encoding. The second one is required + for uploading files. A server which is supposed to accept upload of files must handle + both formats. The first thing to do is to decide the format of the request. This is + easily done by examining the HTTP headers. If we find the header: + + Content-Type: multipart/form-data + + the request is multipart/form-data encoded. Otherwise, it is 'www-url' encoded. We + first consider 'www-url' encoded requests. + + For a 'www-url' encoded request, the web argument are either in the query string or in + the body of the request, or both. The format is the same for both: + + name=value&name=value&... + + However, we may also have + + name + name= + name=&... + name&... + + i.e. some parts may be missing. Hence, we must be careful. + + Furthermore, web arguments must be translated from web to ASCII when www-url encoded. + +define Bool + is_ampersand_or_equal + ( + Word8 c + ) = + if c = '&' then true else c = '='. + + + + The function 'read_name_or_value' reads the string 's' starting at position 'n' until + either the end of the string or the first '&' or '='. + +define String + read_name_or_value + ( + String s, + Int32 start, + Int32 i + ) = + if nth(i,s) is + { + failure then substr(s,start,i - start), + success(c) then + if is_ampersand_or_equal(c) + then substr(s,start,i-start) // the separator is not included + else read_name_or_value(s,start,i+1) + }. + + +define List(Web_arg) + read_www_url_encoded_web_args + ( + String s, + Int32 start, + ) = + with first = read_name_or_value(s,start,start), + if first = "" + then [] + else with i = start+length(first), + if nth(i,s) is + { + failure then [web_arg(first,"")], + success(c) then + if c = '&' + then [web_arg(first,"") . read_www_url_encoded_web_args(s,i+1)] + else if c = '=' + then with second1 = read_name_or_value(s,i+1,i+1), + // print("\""+second1+"\"\n"); + with second = web_to_ascii(second1,0,[]), + [web_arg(first,second) . read_www_url_encoded_web_args(s,i+length(second1)+2)] + else alert + }. + + + + + + *** [4.7] Reading the request line. + + 'read_request_line' reads three words and a new line from the connection. It tries to + recognize "GET" or "POST" in the first word, separates the URI from the query string in + the second word, transforms the query string into a list of 'Web_arg', and finally + returns a datum of type 'HTTP_RequestLine' if no error arose. + + +define Result(Error,HTTP_RequestType) + identify_get_or_post + ( + String s + ) = + with s = to_lower(s), + if s = "get" then ok(get) else + if s = "post" then ok(post) else + error(not_get_or_post_request(s)). + +define Result(Error,HTTP_RequestLine) + read_request_line + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if read_word(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(get_or_post) then if read_word(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(uri_and_query_string) then if read_word(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(http_version) then if read_new_line(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then if separate_uri_from_query_string(uri_and_query_string,0) is + (uri,query_string) then if identify_get_or_post(get_or_post) is + { + error(msg) then error(msg), + ok(request_type) then + ok(request_line(request_type,uri,read_www_url_encoded_web_args(query_string,0))) + } + } + } + } + }. + + + + + + + + *** [4.8] Reading the HTTP headers. + + Each header is made of a name (containing only letters, the underscore, digits and the + minus sign), a colon, a value, and a new line. The first empty line ends the headers. + + + The next function tests characters acceptable in a header name. + +define Bool + is_header_name_char + ( + Word8 c + ) = + with n = word8_to_int32(c), + if ('a' =< n & n =< 'z') then true else + if ('A' =< n & n =< 'Z') then true else + if ('0' =< n & n =< '9') then true else + if c = '-' then true else + c = '_'. + +define Result(Error,String) + read_header_name + ( + Connection connection, + Int32 dead_line, + List(Word8) so_far, + DenialOfService dos + ) = + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if is_header_name_char(c) + then read_header_name(connection,dead_line,[to_lower(c) . so_far],dos) + else unput(c); ok(implode(reverse(so_far))) + }. + +define Result(Error,One) + skip_colon + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if skip_http_blanks(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if c = ':' + then ok(unique) + else error(colon_expected) + }}. + + +define Result(Error,String) + read_header_value + ( + Connection connection, + Int32 dead_line, + List(Word8) so_far, + DenialOfService dos + ) = + if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(c) then + if c = 13 + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(d) then + if d = 10 + then if next_char(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(e) then + if is_strict_blank(e) + then read_header_value(connection,dead_line,[e . so_far],dos) + else (unput(e); ok(implode(reverse(so_far)))) + } + else read_header_value(connection,dead_line,[d, c . so_far],dos) + } + else read_header_value(connection,dead_line,[c . so_far],dos) + }. + + + Reading a single header. + +define Result(Error,Maybe(HTTP_header)) + read_header + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if read_header_name(connection,dead_line,[],dos) is + { + error(msg) then error(msg), + ok(name) then + if name = "" then + if read_and_ignore(connection,dead_line,2,dos) /* 13 and 10 */ is + { + error(msg) then error(msg), + ok(_) then // this is the blank line + ok(failure) // end of headers + } + else if skip_colon(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then if skip_http_blanks(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(_) then if read_header_value(connection,dead_line,[],dos) is + { + error(msg) then error(msg), + ok(value) then + ok(success(http_header(name,value))) + } + } + } + }. + + + + Reading all the headers. + +define Result(Error,List(HTTP_header)) + read_http_headers + ( + Connection connection, + Int32 dead_line, + DenialOfService dos + ) = + if read_header(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(mbh) then if mbh is + { + failure then ok([ ]), + success(header) then + if read_http_headers(connection,dead_line,dos) is + { + error(msg) then error(msg), + ok(others) then ok([header . others]) + } + } + }. + + + + + + + + *** [4.9] Getting the size of the request's body. + + The size of the body of the request is given under the 'Content-Length' header. If this + header is not present, the size is assumed to be zero. + +define Result(Error,Int32) + get_body_size + ( + List(HTTP_header) headers + ) = + if headers is + { + [ ] then ok(0), + [h . t] then if h is http_header(name,value) then + if name = "content-length" + then if string_to_integer(value) is + { + failure then error(incorrect_content_length_value), + success(n) then ok(n) + } + else get_body_size(t) + }. + + + + + + + + + + + *** [4.10] Reading the body of the request. + + The body of the request may be very big (it contains uploaded files, if any). We read + it using the primitive 'read', which returns the number of bytes read, which may be + less than the number of bytes we wanted to read. This is not an error, but simply due + to the fact the buffer associated with the connection in the Linux (or MS-Windows) + kernel has a limited size. Hence, we must read bytes again until we have read the + required number of bytes. However, if the number of bytes read is zero, the connection + may be broken. In that case, we must not try to read indefinitely. On the contrary, we + make at most 10 retries, with a small sleeping time between any two of them. + +define Result(Error,ByteArray) + read_http_body + ( + Connection connection, + Int32 body_size, + ByteArray so_far, // when calling this function, 'so_far' is the empty byte array + Int32 retries // this function is called with retries = 10 + ) = + if body_size = 0 then ok(constant_byte_array(0,0)) else + if retries =< 0 then error(cannot_read_from_connection) else + if read(connection,body_size,60) is + { + error then error(cannot_read_from_connection), + timeout then error(timeout(60)), + ok(new_bytes) then with + ba = so_far + new_bytes, // contains all the bytes read so far + nr = length(ba), // total read since the beginning + nn = length(new_bytes), // number of bytes just read + if nr < body_size // must read more bytes + then if nn > 0 // if connection seems to work + then read_http_body(connection,body_size,ba,1000) // continue reading + else sleep(100); // otherwise, sleep 1/10 of second + read_http_body(connection,body_size,ba, // and retry reading + retries-1) // but no more than 10 times + else ok(ba) // required number of bytes has been read + }. + + + Note: During sleeping, 'anbexec' runs other machines. Actually, calling 'sleep', even + for one millisecond, is some way of giving up explicitly, so that other virtual + machines may work. + + + + + + + + + + + + + *** [5] Making the HTTP answer. + + At that point we have read the request line, the headers and the body of the + request, and we must decide what to do. + + Actually, we can do one of the following: + + - send a file, + - execute 'tickets_and_web_page' in case of an ".awp" URI. + + The uploaded file (which are in the body of the request) are saved into temporary files + below. + + + + + + *** [5.1] Avoiding illegal URIs. + + For security reasons, we must avoid illegal URIs, for example those which may climb up + in the file hierarchy. First we accept only few characters in URIs. + +define Bool + is_legal_uri_char + ( + Word8 c + ) = + with n = word8_to_int32(c), + if ('a' =< n & n =< 'z') then true else // accept 'a' to 'z' + if ('A' =< n & n =< 'Z') then true else // accept 'A' to 'Z' + if ('0' =< n & n =< '9') then true else // accept '0' to '9' + if c = '.' then true else // accept '.' '-' '/' and '_' + if c = '-' then true else + if c = '/' then true else + c = '_'. + + We do not accept ~ which is some way of climbing. Of course, we cannot disallow single + dots, which are most often present in legal URIs, but we must avoid double dots .. + which mean 'climb up'. + +define Bool + is_illegal_uri + ( + String uri, + Int32 n + ) = + if nth(n,uri) is + { + failure then false, + success(c) then + if c = '.' // first dot + then if nth(n+1,uri) is + { + failure then false, + success(d) then + if d = '.' // second dot + then true + else is_illegal_uri(uri,n+1) + } + else is_illegal_uri(uri,n+1) + }. + + + + + + + *** [5.2] Managing authorizations for downloading private files. + + Computing the authorization and making the authorization file (containing the absolute + path of the file on the server). + + +define String + compute_authorization + ( + String authorization_secret, + String absolute_path + ) = + to_ascii(sha1((authorization_secret, + absolute_path))). + + +public define String + make_authorization + ( + String site_directory, + String authorization_secret, + String absolute_path + ) = + with private_download_dir = site_directory+"/private_download", + auth = compute_authorization(authorization_secret, + absolute_path), + forget(save(absolute_path, + private_download_dir+"/z"+auth)); + auth. + + + The function 'send_file' defined below handles the recognition of authorizations. + + + + + + *** [5.3] Recognizing MIME types. + + The extension of the (redirected) URI must be either ".awp" or recognized as associated + to a MIME type. Otherwise, the server will not send the file. This is for security, but + also because, we must generate a 'Content-Type' header in the answer, with the right + MIME type. + +define String + get_uri_extension_aux + ( + String uri, + Int32 n // used for searching backwards + ) = + if nth(n,uri) is + { + failure then "", + success(c) then + if c = '.' then substr(uri,n,length(uri)-n) + else if c = '/' then "" + else get_uri_extension_aux(uri,n-1) + }. + +public define String + get_uri_extension + ( + String uri + ) = + get_uri_extension_aux(uri, + length(uri)-1). // search starts at the right end + + + +define Maybe(String) + recognize_mime_type_from_ext + ( + String ext, + List(MIME) l + ) = + if l is + { + [ ] then success("application/octet-stream"), // failure, + [h . t] then if h is mime(mime_type,extension) then + if ext = extension + then success(mime_type) + else recognize_mime_type_from_ext(ext,t) + }. + +define Maybe(String) + recognize_mime_type_from_uri + ( + Web_Site_Description desc, + String uri + ) = + recognize_mime_type_from_ext(get_uri_extension(uri),known_mime_types(desc)). + + + + + + + + + *** [5.4] Formating HTTP headers. + + This is the formating for sending to the client (hence, it has nothing to do with the + component 'journal_headers' in the web site description). + +define Printable_tree + format_headers + ( + List(HTTP_header) headers + ) = + if headers is + { + [ ] then [ ], + [h . t] then if h is http_header(name,value) then + [name,": ",value,crlf . format_headers(t)] + }. + + + + + + + *** [5.5] Sending a file. + + We send 2 headers 'Content-Type' and 'Content-Length'. + +define List(HTTP_header) + headers_for_send_file + ( + String mime_type, + Int32 size, + ) = + [ + http_header("Content-Type",mime_type), + http_header("Content-Length",integer_to_string(size)), + ]. + + + + Sending the body of the answer (i.e. the file itself). + +define One + send_file_body + ( + Web_Site_Description desc, + Connection connection, // connection with the client + Connection file, // file to be sent already opened + Int32 size, // size of file + Int32 sent, // bytes already sent + String filename // name of file + ) = + if sent >= size then unique else + if read(file,min(10000,size-sent),60) is + { + error then log_journal_msg(desc,"Cannot read from file '"+filename+"'.\n"), + timeout then log_journal_msg(desc,"Cannot read from file timeoput'"+filename+"'.\n"), + ok(ba) then + with nr = length(ba), // get the number of bytes read + if reliable_write(connection,ba) is + { + failure then log_journal_msg(desc,"Cannot write into connection.\n"), + success(nw) then + send_file_body(desc,connection,file,size,sent+nw,filename) + } + }. + + + + + Sending the answer line, the headers and the body. + +define One + send_file + ( + Web_Site_Description desc, + Connection connection, + List(HTTP_header) headers, + Int32 size, + Connection file, + String filename, + One -> One action_before_send_file + ) = + action_before_send_file(unique); + forget(reliable_write(connection,to_byte_array("HTTP/1.1 200 OK"+crlf))); + forget(reliable_write(connection,[format_headers(headers) , crlf])); + send_file_body(desc,connection,file,size,0,filename). + + + + Checking if a connection is under SSL. + +define Bool + is_SSL + ( + Connection c + ) = + if c is + { + file_r(_) then false, + file_w(_) then false, + file_rw(_) then false, + tcp(_) then false, + ssl(_) then true + }. + + + + Before opening and sending a file, we check the MIME type. It must be recognized, + except if there is a valid authorization for private download. + +define One + send_file + ( + Web_Site_Description desc, + Connection connection, + String uri, + Maybe(String) mbauthorization, + One -> One action_before_send_file + ) = + if mbauthorization is + { + //--- file without authorization: take it from public --- + failure then if recognize_mime_type_from_uri(desc,uri) is + { + failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), + success(mime_type) then + with path = site_directory(desc)+"/public"+uri, + if (Maybe(RStream))connect to file path is + { + failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), + success(f) then with size = file_size(f), + send_file(desc, + connection, + headers_for_send_file(mime_type,size), + size, + file(f), + uri, + action_before_send_file) + } + }, + + //--- file with authorization: apply 'private download' mecanism --- + success(authorization) then + with private_download_dir = site_directory(desc)+"/private_download", + if (RetrieveResult(String))retrieve(private_download_dir+"/z"+authorization) + is ok(absolute_path) + then ( + with new_hash = compute_authorization(authorization_secret(desc), + absolute_path), + if (Maybe(RStream))connect to file absolute_path is + { + failure then log_journal_msg(desc,"Cannot find file '"+absolute_path+"'.\n"), + success(f) then with size = file_size(f), + send_file(desc, + connection, + headers_for_send_file(if recognize_mime_type_from_uri(desc,uri) is + { + failure then "application/octet-stream" + success(mime_type) then mime_type + }, + size), + size, + file(f), + uri, + action_before_send_file) + } + ) + else log_journal_msg(desc,"Cannot find or read authorization file.\n") + }. + + + define One + send_file + ( + Web_Site_Description desc, + Connection connection, + String uri, + Maybe(String) mbauthorization, + One -> One action_before_send_file + ) = + if mbauthorization is + { + //--- file without authorization: take it from public --- + failure then if recognize_mime_type_from_uri(desc,uri) is + { + failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), + success(mime_type) then + with path = site_directory(desc)+"/public"+uri, + if (Maybe(RStream))connect to file path is + { + failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), + success(f) then with size = file_size(f), + send_file(desc, + connection, + headers_for_send_file(mime_type,size), + size, + file(f), + uri, + action_before_send_file) + } + }, + + //--- file with authorization: apply 'private download' mecanism --- + success(authorization) then + if is_SSL(connection) + then ( + with private_download_dir = site_directory(desc)+"/private_download", + if (RetrieveResult(String))retrieve(private_download_dir+"/z"+authorization) + is ok(absolute_path) + then ( + with new_hash = compute_authorization(authorization_secret(desc), + absolute_path), + if (Maybe(RStream))connect to file absolute_path is + { + failure then log_journal_msg(desc,"Cannot find file '"+absolute_path+"'.\n"), + success(f) then with size = file_size(f), + send_file(desc, + connection, + headers_for_send_file(if recognize_mime_type_from_uri(desc,uri) is + { + failure then "application/octet-stream" + success(mime_type) then mime_type + }, + size), + size, + file(f), + uri, + action_before_send_file) + } + ) + else log_journal_msg(desc,"Cannot find or read authorization file.\n") + ) + else //-- file with authorization, but under HTTP: take it from private_download + if recognize_mime_type_from_uri(desc,uri) is + { + failure then log_journal_msg(desc,"No MIME type found for '"+uri+"'.\n"), + success(mime_type) then + with path = site_directory(desc)+"/private_download"+uri, + if (Maybe(RStream))connect to file path is + { + failure then log_journal_msg(desc,"Cannot find file '"+path+"'.\n"), + success(f) then with size = file_size(f), + send_file(desc, + connection, + headers_for_send_file(mime_type,size), + size, + file(f), + uri, + action_before_send_file) + } + } + }. + + + + + + + + *** [5.6] Answering a www-url encoded request. + + Standard headers are for answering ".awp" requests. + +define List(HTTP_header) + standard_headers + ( + Int32 answer_body_size, + String charset + ) = + [ + //http_header("Content-Type","text/html"), + http_header("Content-Type","text/html; charset="+charset), + http_header("Content-length",integer_to_string(answer_body_size)) + ]. + + +define One + www_url_answer + ( + String host_name, + Web_Site_Description desc, + Connection connection, // with the client + Int32 ip_addr, // of the client + HTTP_RequestLine request_line, + List(HTTP_header) headers, + ByteArray body, + One -> String generate_tt // trust ticket generation + ) = + with all_web_args = query_string(request_line) + + read_www_url_encoded_web_args(to_string(body),0), + uri = uri(request_line), + ext = get_uri_extension(uri), + (if member(journal_extensions(desc),ext) + then log_journal_msg(desc, + format_request(desc,connection,request_line,headers,all_web_args)) + else unique); + if is_illegal_uri(uri,0) + then log_journal_msg(desc,"Received illegal URI: "+uri+"\n") + else (if (ext = ".awp" | ext = "") + then (with answer_headers_body = awp_handler(desc)(host_name, + http_info(ip_addr,uri,headers,generate_tt), + all_web_args, + is_SSL(connection)), + if answer_headers_body is (additional_headers,answer_body) then + forget(reliable_write(connection, + [ "HTTP/1.1 200 OK", crlf, + format_headers(standard_headers(length(answer_body),charset(desc))), + format_headers(additional_headers), + crlf . + answer_body]))) + else (send_file(desc, + connection, + uri, + if web_arg_value(all_web_args,"zauth") is + { + not_found then failure, + found(v) then success(v) + }, + (One u) |-> before_send_file(desc)(all_web_args)))). + + + + + + + + *** [5.7] Answering a multipart/form-data encoded request. + + In order to support upload of files, we must be able to read web arguments which are + encoded in a multipart/form-data body. The first thing to do is to find the + boundary. The boundary is a special string which delimits the various parts of the + 'multipart' body. It is found within the value of the 'Content-Type' HTTP header, as + the value of the 'boundary' attribute. + + + + + + *** [5.7.1] Finding the boundary. + + Hence, we just have to find the string 'boundary=' within the value of the + 'Content-Type' header, and read the value of the boundary from there. + +define Bool + delimits_boundary + ( + Word8 c + ) = + if c = ' ' then true else + if c = 13 then true else + if c = 10 then true else + if c = 0 then true else + if c = ',' then true else + c = ';'. + + +define Maybe(String) + get_boundary_value_3 + ( + String s, + Int32 i, + List(Word8) so_far + ) = + if nth(i,s) is + { + failure then success(implode(reverse(so_far))), + success(c) then + if delimits_boundary(c) + then success(implode(reverse(so_far))) + else get_boundary_value_3(s,i+1,[c . so_far]) + }. + + + +define Maybe(String) + get_boundary_value_2 + ( + String s, + Int32 i, + ) = + if nth(i,s) is + { + failure then failure, + success(c) then + if is_blank(c) + then get_boundary_value_2(s,i+1) + else get_boundary_value_3(s,i+1,[c]) + }. + +define Maybe(String) + get_boundary_value_1 + ( + String s, // string into which we must find '= ...' + Int32 i // position of start of search + ) = + if nth(i,s) is + { + failure then failure, + success(c) then + if is_blank(c) + then get_boundary_value_1(s,i+1) + else if c = '=' + then get_boundary_value_2(s,i+1) + else failure + }. + + +define Maybe(String) + get_boundary + ( + String content_type_header_value + ) = + if find("boundary",content_type_header_value,0) is + { + failure then failure, + success(n) then // 'boundary' has been found at position n + get_boundary_value_1(content_type_header_value,n+8) + }. + +define Maybe(String) + get_boundary + ( + List(HTTP_header) headers + ) = + if headers is + { + [ ] then failure, + [h . t] then if h is http_header(name,value) then + if name = "content-type" + then get_boundary(value) + else get_boundary(t) + }. + + + + + + + + + *** [5.7.2] Reading attributes from a multipart entity. + + Entities in a multipart/form-data body are separated by instances of the string: + + --bbbbb + + where bbbbb is the boundary computed above. Actually, the body has the form: + + --bbbbb + + --bbbbb + + --bbbbb + ... + --bbbbb + + --bbbbb + + + We have to extract an entity which is in the body between offsets 'start' and 'end' + (computed when boundaries have been localized). The entity itself is made of two parts: + headers and body. The body is separated from the headers by a blank line. This blank + line (a double crlf) marks the beginning of the body of the entity. Within the headers + of the entity, we look for a 'Content-Disposition' header, which should look like this: + + Content-Disposition: form-data; name="..."; filename="..." crlf + + We are just interested in the name and the file name. Hence we first search + 'Content-Disposition', then we search 'name' and read the value, and we do the same for + 'filename'. + + If the 'filename' attribute is not present, the web arg is an ordinary one, otherwise, + it is an uploaded file. + + + Below is a variant of 'find' (see 'tools/findstring.anubis'), with an extra 'end' + argument. + +define Maybe(Int32) + find + ( + String what, + ByteArray where, + Int32 start, + Int32 end + ) = + if find(to_byte_array(what),where,start) is + { + failure then failure, + success(n) then + if n+length(what) >= end + then failure + else success(n) + }. + + +define String + read_attribute_value + ( + ByteArray where, + Int32 start, + Int32 end, + List(Word8) so_far + ) = + if start >= end then implode(reverse(so_far)) else + if nth(start,where) is + { + failure then implode(reverse(so_far)), + success(c) then + if c = '\"' + then implode(reverse(so_far)) + else read_attribute_value(where,start+1,end,[c . so_far]) + }. + +define Maybe(String) + find_attribute + ( + String name, + ByteArray where, + Int32 start, + Int32 end + ) = + with name = name+"=\"", + if find(to_byte_array(name),where,start) is + { + failure then failure, + success(n) then + if n+length(name) >= end + then failure + else success(read_attribute_value(where,n+length(name),end,[])) + }. + + + +define Maybe((String,Maybe(String))) + find_name_and_filename + ( + ByteArray body, + Int32 start, + Int32 end + ) = + if find(to_byte_array("Content-Disposition"),body,start) is + { + failure then failure, + success(n) then + if find_attribute("name",body,n+19,end) is + { + failure then failure, + success(name_value) then if find_attribute("filename",body,n+19,end) is + { + failure then success((name_value,failure)), + success(filename_value) then success((name_value,success(filename_value))) + } + } + }. + + + + + + + + + + + *** [5.7.3] Creating a temporary filename for an uploaded file. + +variable Int32 uploaded_file_count = 0. + + This variable is local to the virtual machine. Hence, its value is 0 each time a new + requests arrives. Temporary uploaded files are stored in the directory represented by + 'upload_temporary_directory'. The filenames have the form: + + _m_n + + where 'm' is the number of the virtual machine, and 'n' a number obtained by + incrementing 'uploaded_file_count'. Notice that the program must do something with this + file (move it to some directory/name), otherwise, it will probably be overwritten the + next time the same machine works. + + + + + + + *** [5.7.4] Saving an uploaded file under a temporary filename. + +define Maybe(String) // returns the temporary file name + save_uploaded_file + ( + Web_Site_Description desc, + ByteArray body, + Int32 start, + Int32 end + ) = + uploaded_file_count <- 1 + *uploaded_file_count; + with tfn = "_"+integer_to_string(virtual_machine_id)+"_"+integer_to_string(*uploaded_file_count), + if (Maybe(WStream))connect to file site_directory(desc)+"/upload_temporary/"+tfn is + { + failure then failure, + success(f) then + if reliable_write(file(f),extract(body,start,end)) is + { + failure then failure, + success(nw) then + if nw = end - start + then success(tfn) + else failure + } + }. + + + + + + + + + *** [5.7.5] Removing the path from a file name. + + When a file is uploaded, the browser sends the complete path of the file on the client + machine as the file name. Actually, this is not quite normal. Nevertheless, we need to + remove the path, and keep only the file name. This is achieved by 'remove_path' below. + +define Int32 + file_name_begin + ( + String full_name, + Int32 i + ) = + if nth(i,full_name) is + { + failure then 0, + success(c) then + if c = '/' then i+1 else + if c = '\\' then i+1 else + file_name_begin(full_name,i-1) + }. + +define String + remove_path + ( + String full_name + ) = + with l = length(full_name), + b = file_name_begin(full_name,l-1), + substr(full_name,b,l-b). + + + + + + *** [5.7.6] Reading a multipart entity. + +define Maybe(Web_arg) + get_multipart_entity + ( + Web_Site_Description desc, + ByteArray body, + Int32 start, + Int32 end + ) = + if find(to_byte_array(crlf+crlf),body,start) is + { + failure then failure, + success(k) then + if k >= end // must be within this entity, not the next one + then failure + else if find_name_and_filename(body,start,k) is + { + failure then failure, + success(n_mbfn) then if n_mbfn is (name,mbfn) then + if mbfn is + { + failure then + success(web_arg(name,to_string(extract(body,k+4,end-2)))), + // we must substract 2 to end because of crlf just before the boundary + + success(fn) then + if save_uploaded_file(desc,body,k+4,end-2) is + { + failure then failure, + success(tfn) then + success(upload(name,remove_path(fn), + site_directory(desc)+"/upload_temporary/"+tfn)) + + } + } + } + }. + + + +define List(Web_arg) + read_multipart_form_data_encoded_web_args + ( + Web_Site_Description desc, + ByteArray body, + ByteArray __boundary, + Int32 i, + ) = + if find(__boundary,body,i) is + { + failure then [ ], + success(n) then + if find(__boundary,body,n+length(__boundary)) is + { + failure then [ ], + success(m) then + if get_multipart_entity(desc,body,n+length(__boundary),m) is + { + failure then [ ], + success(wa) then + [wa . read_multipart_form_data_encoded_web_args(desc,body,__boundary,m)] + } + } + }. + + + +define One + multipart_form_data_answer + ( + String host_name, + Web_Site_Description desc, + Connection connection, + Int32 ip_addr, + HTTP_RequestLine request_line, + List(HTTP_header) headers, + ByteArray body, + One -> String generate_tt + ) = + if get_boundary(headers) is + { + failure then unique, + success(boundary) then + with all_web_args = query_string(request_line) + + read_multipart_form_data_encoded_web_args(desc, + body, + to_byte_array("--"+boundary), + 0), + uri = uri(request_line), + ext = get_uri_extension(uri), + log_journal_msg(desc, + format_request(desc,connection,request_line,headers,all_web_args)); + if is_illegal_uri(uri,0) + then log_journal_msg(desc,"Received illegal URI: "+uri+"\n") + else + if (ext = ".awp" | ext = "") then + (with answer_headers_body = awp_handler(desc)(host_name, + http_info(ip_addr,uri,headers,generate_tt), + all_web_args, + is_SSL(connection)), + if answer_headers_body is (additional_headers,answer_body) then + forget(reliable_write(connection, + [ "HTTP/1.1 200 OK",crlf, + format_headers(standard_headers(length(answer_body),charset(desc))), + format_headers(additional_headers), + crlf . + answer_body]))) + else unique + }. + + + + + + + + + *** [5.8] Handling redirections. + + 'redirections' (of type 'List(Redirection)') contains redirection directives. Each one + has the form: + + redirect(required_uri,required_host,corresponding_uri). + + The host required by the client may be found in the 'Host' HTTP header. The URI + required by the client is given below as 'uri'. We just have to find the required host + in the headers, and to find the corresponding redirection directive. + + + In the next fonction, the required host and URI are known. We just have to search in + the 'redirections' list. + +define String + handle_redirection + ( + String required_uri, + String required_host, + List(Redirection) redirections + ) = + if redirections is + { + [ ] then required_uri, + [h . t] then if h is redirect(uri,host,target) then + if host = required_host + then if uri = required_uri + then target + else handle_redirection(required_uri,required_host,t) + else handle_redirection(required_uri,required_host,t) + }. + + + + The host name may be encumbered by a port number, like + + www.our-business.com:1607 + + We must remove this port number, otherwise the host name may not be recognized. + +define String + strip_port + ( + String name, + Int32 i + ) = + if nth(i,name) is + { + failure then name, + success(c) then + if c = ':' + then substr(name,0,i) + else strip_port(name,i+1) + }. + + + + + + Finding the 'Host' header. No redirection is performed if this header is not found. + +define String + handle_redirection // returns the redirected URI + ( + List(Redirection) redirections, + String uri, // original URI + List(HTTP_header) headers + ) = + if headers is + { + [ ] then uri, + [h . t] then if h is http_header(name,value) then + if name = "host" + then handle_redirection(uri,strip_port(value,0),redirections) + else handle_redirection(redirections,uri,t) + }. + + + + + + + + + *** [5.9] Answering both sorts of requests. + + We must decide if the request is www-url encoded or multipart/form-data encoded. This + is achieved through the header 'Content-Type'. + +define EncodingType + get_encoding_type + ( + List(HTTP_header) headers + ) = + if headers is + { + [ ] then www_url, // this is the default + [h . t] then if h is http_header(name,value) then + if name = "content-type" + then if find("multipart/form-data",value,0) is + { + failure then www_url, + success(_) then multipart_form_data + } + else get_encoding_type(t) + }. + + + +define One + send_answer + ( + String host_name, + Web_Site_Description desc, + Connection connection, + HTTP_RequestLine rqline, + List(HTTP_header) headers, + ByteArray body, + One -> String generate_tt + ) = + if rqline is request_line(type,uri,qstring) then + with rqline = request_line(type,handle_redirection(redirections(desc),uri,headers),qstring), + if remote_IP_address_and_port(connection) is (ip_addr,_) then + if get_encoding_type(headers) is + { + www_url then + www_url_answer(host_name,desc,connection,ip_addr,rqline,headers,body,generate_tt), + multipart_form_data then + multipart_form_data_answer(host_name,desc,connection,ip_addr,rqline,headers,body,generate_tt) + }. + + + + + + + + *** [6] The HTTP/HTTPS server. + + The command 'start_server' (declared in 'predefined.anubis') starts a virtual machine + which opens a server TCP/IP connection, and which continuously listens to this + connection. When a request arrives, this machine delegates the work of deciphering and + answering the request to another virtual machine, and continues to listen. The job of + the delegated machine is defined by the HTTP request handler below. + + + + + + *** [6.1] Determining the requested host. + + When a request arrives to one of our two servers, we must decide which site (host) is + requested. + +define Maybe(String) + get_host_header_value + ( + List(HTTP_header) headers + ) = + if headers is + { + [ ] then failure, + [h . t] then if h is http_header(name,value) then + if name = "host" + then success(strip_port(value,0)) + else get_host_header_value(t) + }. + +define Maybe((String,Web_Site_Description)) + get_site + ( + String requested_host, + List(Web_Site_Description) sites + ) = + if sites is + { + [ ] then print("Requested host '"+requested_host+"' does not exist.\n"); failure, + [site1 . others] then + if site1 is web_site_description(common_names,_,_,_,_,_,_,_,_,_) then + if member(common_names,requested_host) + then success((requested_host,site1)) + else get_site(requested_host,others) + }. + + +define Maybe((String,Web_Site_Description)) + get_site + ( + List(HTTP_header) headers, + List(Web_Site_Description) sites + ) = + if get_host_header_value(headers) is + { + failure then print("No 'Host' HTTP header.\n"); failure, + success(requested_host) then + //here we treat the case with only one site. hence we accept any host request + //print("*** there is " +length(sites) + " sites \n"); + if length(sites) = 1 then + with site = force_nth(0, sites), + //print("ONE server OK\n"); + success((requested_host, site)) + else + get_site(requested_host,sites) + }. + + + + + + *** [6.2] The HTTP request handler. + + Here is the HTTP/HTTPS handler. It is called at each new request in a separate virtual + machine. It reads the headers of the HTTP request, determines the host, determines body + size, reads the body of the HTTP request, and answers the request. + + + +define One -> String make_generate_trust_ticket(DenialOfService dos). + + +define One + http_https_handler + ( + List(Web_Site_Description) sites, + Connection connection, + Bool is_https, + DenialOfService dos + ) = + with start_time = (Int32)now, + sttm <- start_time; + if dos is denial_of_service(mc_v,rld_v,hd_v,ad_v,ld_v,ra_v) then + if remote_IP_address_and_port(connection) is (ip_addr,port) then + if read_request_line(connection,start_time+*rld_v,dos) is + { + error(msg) then print(format(msg)), + ok(request_line) then + if read_http_headers(connection,start_time+*hd_v,dos) is + { + error(msg) then print(format(msg)), + ok(headers) then if get_site(headers,sites) is + { + failure then unique, + success(p) then if p is (host_name,desc) then + if get_body_size(headers) is + { + error(msg) then log_journal_msg(desc,format(msg)), + ok(body_size) then + if read_http_body(connection,body_size,constant_byte_array(0,0),1000) is + { + error(msg) then log_journal_msg(desc,format(msg)), + ok(body) then + send_answer(host_name,desc,connection,request_line,headers,body, + make_generate_trust_ticket(dos)) + } + } + } + } + }. + + + Below are the two tools for constructing the handlers required by 'start_server' and + 'start_ssl_server' (see 'predefined.anubis'). + +define Bool is_dubious_IP(Int32 ip, DenialOfService dos). + +define Server -> ((RWStream) -> One) + make_http_handler + ( + List(Web_Site_Description) sites, + DenialOfService dos + ) = + (Server server) |-> (RWStream connection) |-> + if remote_IP_address_and_port(connection) is (addr,_) then + if is_dubious_IP(addr,dos) + then print("Rejecting dubious IP address "+ip_addr_to_string(addr)+"\n") + else http_https_handler(sites,tcp(connection),false,dos). + +define Server -> (SSL_Connection -> One) + make_https_handler + ( + List(Web_Site_Description) sites, + DenialOfService dos + ) = + (Server server) |-> (SSL_Connection connection) |-> + http_https_handler(sites,ssl(connection),true,dos). + + + + + *** [6.3] Server's tasks. + + Some tasks must be executed periodically, for example for cleaning up directories from + short life time files. + + The next function removes from the given directory (and recursively from its + subdirectories) all the files which are more than 10 minutes old. + +define One + cleanup_directory_10mn + ( + String dir // path of private download directory (or subdirectory) with trailing slash + ) = + forget(map((FileDescription fd) |-> if fd is + { + no_info(name) then forget(remove(dir+name)), + file(name,_,_,d) then if d+600 < now then forget(remove(dir+name)) else unique, + link(name,_,_,d) then if d+600 < now then forget(remove(dir+name)) else unique, + directory(name,_,_) then cleanup_directory_10mn(dir+name+"/"), + }, + directory_full_list(dir,"*","*","*"))). + + +define One + http_servers_tasks + ( + List(Web_Site_Description) sites, + List(Server) servers, + Int32 period, + Int32 next_time, + ) = + if mapand(is_down,servers) + then unique + else if now > next_time + then + ( + /* + forget(map((Web_Site_Description wsd) |-> + cleanup_directory_10mn(site_directory(wsd)+"/private_download/"), + sites)); + */ + http_servers_tasks(sites,servers,period,next_time+period) + ) + else + ( + sleep(1000); + http_servers_tasks(sites,servers,period,next_time) + ). + + +public define One + start_http_servers_tasks + ( + List(Web_Site_Description) sites, + List(Server) servers, + Int32 period + ) = + delegate http_servers_tasks(sites,servers,period,now), + unique. + + + + + *** [6.4] Protection against 'denial of service' attacks. + + + *** [6.4.1] Counting connections. + +define Bool // returns false if the counter cannot be incremented (too many connections) + increment_connections_counter + ( + Var(Int32) counter + ) = + protect with n = *counter, + if n >= 100 + then false + else (counter <- (*counter)+1); true. + +define One + decrement_connections_counter + ( + Var(Int32) counter + ) = + protect counter <- (*counter)-1. + + + + + + *** [6.4.2] Recording dubious IP addresses. + + +define List(DubiousIP) + record_dubious_IP + ( + Int32 ip, + List(DubiousIP) l + ) = + if l is + { + [ ] then [dubious_ip(ip,now)], + [h . t] then if h is dubious_ip(addr,time) then + if addr = ip + then [dubious_ip(addr,now) . t] + else [h . record_dubious_IP(ip,t)] + }. + + +define One + record_dubious_IP + ( + Int32 dubious_IP, + Var(List(DubiousIP)) v + ) = + protect v <- record_dubious_IP(dubious_IP,*v). + + +define One + record_dubious_IP + ( + Int32 addr, + DenialOfService dos + ) = + record_dubious_IP(addr,list_of_dubious(dos)). + + +public define DenialOfService + load_denial_of_service_info + = + if (RetrieveResult(DenialOfService))retrieve(my_anubis_directory+"/web_sites/dos_info") is + ok(dos) then dos else denial_of_service( + var(100), + var(1000), + var(1500), + var(2000), + var([]), + var([])). + + + + + *** [6.4.3] Testing if an address is dubious. + +define Bool + is_dubious_IP + ( + Int32 ip, + List(DubiousIP) l + ) = + if l is + { + [ ] then false, + [h . t] then if h is dubious_ip(addr,time) then + if ip = addr + then true + else is_dubious_IP(ip,t) + }. + + +define Bool + is_dubious_IP + ( + Int32 ip, + DenialOfService dos + ) = + if dos is + { + denial_of_service(mc_v,rld_v,hd_v,ad_v,ld_v,ra_v) then + if member(*ra_v,ip) then false else + is_dubious_IP(ip,*ld_v) + }. + + + + + *** [6.4.4] Removing inactive dubious IP addresses. + +define List(DubiousIP) + remove_inactive_dubious_IP + ( + List(DubiousIP) l, + Int32 ref_time, + ) = + if l is + { + [ ] then [ ], + [h . t] then if h is dubious_ip(addr,time) then + if time < ref_time + then (print(ip_addr_to_string(addr)+" removed from dubious addresses list.\n"); + remove_inactive_dubious_IP(t,ref_time)) + else [h . remove_inactive_dubious_IP(t,ref_time)] + }. + +define One + remove_inactive_dubious_IP + ( + Var(List(DubiousIP)) v + ) = + protect + with ref_time = now - 600, // 10 minutes + v <- remove_inactive_dubious_IP(*v,ref_time). + + + The above function will be executed periodically by the servers's tasks machine. + + + + *** [6.4.5] Making the function for generating trust tickets. + +define One -> String + make_generate_trust_ticket + ( + DenialOfService dos + ) = + (One _) |-> "". + + + + + + + + *** [6.5] Starting the HTTP/HTTPS server. + + + The next function creates the directories for all sites (if they don't already exist). + +define One + create_directories + ( + List(Web_Site_Description) sites + ) = + if sites is + { + [ ] then unique, + [s1 . others] then + with site_dir = site_directory(s1), + forget(make_directory(site_dir+"/public",default_directory_mode)); + forget(make_directory(site_dir+"/upload_temporary",default_directory_mode)); + forget(make_directory(site_dir+"/private_download",default_directory_mode)); + forget(make_directory(site_dir+"/journal",default_directory_mode)); + create_directories(others) + }. + + + + + + Below are the commands for starting an HTTP server and an HTTPS server. + + +define StartServerResult + start_http_server + ( + Int32 ip_address, + Int32 port, + Server -> ((RWStream) -> One) handler, + Int32 retries, + DenialOfService dos + ) = + if start_server(ip_address, + port, + handler, + identity) is ok(server) + then print(" \r"); + ok(server) + else print("Port "+port+": retry number "+retries+"\r"); + sleep(1000); + start_http_server(ip_address,port,handler,retries+1,dos). + +public define StartServerResult + start_http_server + ( + Int32 ip_address, + Int32 port, + List(Web_Site_Description) sites, + DenialOfService dos + ) = + create_directories(sites); + start_http_server(ip_address,port, + make_http_handler(sites,dos), + 0, + dos). + + + For the HTTPS server, we have a problem which is due to the fact that 'anbexec' is not + yet able to manipulate several SSL server certificates. 'anbexec' and + 'predefined.anubis' must be changed. Sorry ! This will be done as soon as possible. The + 'solution' for the time being is to provide the common name of the unique SSL server + certificate. + + +define StartServerResult + start_https_server + ( + Int32 ip_address, + Int32 port, + String certificate_common_name, + Server -> (SSL_Connection -> One) handler, + Int32 retries, + DenialOfService dos + ) = + if start_ssl_server(ip_address, + port, + certificate_common_name, + handler, + identity) is ok(server) + then print(" \r"); + ok(server) + else print("Port "+port+": retry number "+retries+"\r"); + sleep(1000); + start_https_server(ip_address,port, + certificate_common_name, + handler,retries+1, + dos). + + +public define StartServerResult + start_https_server + ( + Int32 ip_address, + Int32 port, + String certificate_common_name, // of SSL server certificate + List(Web_Site_Description) sites, + DenialOfService dos + ) = + create_directories(sites); + start_https_server(ip_address,port,certificate_common_name, + make_https_handler(sites,dos), + 0,dos). + + + + + + + + + + *** [7] The web dispatcher. + + + *** [7.1] The dispatcher server. + +define One + send_dispatching_page + ( + RWStream conn, + String common_name, + Int32 port + ) = + print("Dispatching '"+common_name+"' to port "+port+"\n"); + forget(reliable_write(conn,to_byte_array( + "" + ))). + + + +define Maybe(DispatcherInfo) + find_host + ( + List(DispatcherInfo) l, + String host + ) = + if l is + { + [ ] then failure, + [h . t] then if h is site(name,port) then + if name = host + then success(h) + else find_host(t,host) + }. + + + +define Server -> ((RWStream) -> One) + make_dispatcher_handler + ( + Var(List(DispatcherInfo)) info_v, + DenialOfService dos + ) = + (Server server) |-> (RWStream conn) |-> + with start_time = (Int32)now, + if read_request_line(tcp(conn),start_time+*request_line_delay(dos),dos) is + { + error(msg) then print(format(msg)), + ok(request_line) then + if read_http_headers(tcp(conn),start_time+*headers_delay(dos),dos) is + { + error(msg) then print(format(msg)), + ok(headers) then if get_host_header_value(headers) is + { + failure then print("No 'HOST' HTTP header.\n"), + success(host) then + if find_host(*info_v,host) is + { + failure then print("Host: '"+host+"' not registered.\n"), + success(s) then if s is site(common_name,ip_port) then + send_dispatching_page(conn,common_name,ip_port) + } + } + } + }. + + +define One + dispatcher_update_error + ( + String file_path + ) = + print("web_dispatcher: unable to reread file: '"+file_path+"'.\n"). + + +define Bool + dispatcher_update_data + ( + String info_file_path, + Var(List(DispatcherInfo)) info_v, + Var(Int32) info_date_v + ) = + if directory_full_list(my_anubis_directory+"/web_sites","dispatcher.info","","") is + { + [ ] then false, + [h . t] then if h is + { + no_info(n) then false, + file(n,_,_,d) then if n = "dispatcher.info" + then (info_date_v <- d; + if (RetrieveResult(List(DispatcherInfo)))retrieve(info_file_path) is + { + cannot_find_file then false, + read_error then false, + type_error then false, + ok(info) then info_v <- info; true + }) + else false, + link(_,_,_,_) then false, + directory(_,_,_) then false + } + }. + + + + The loop within which the dispatcher updates its data every 3 seconds: + +define One + dispatcher_update_task + ( + String info_file_path, + Var(List(DispatcherInfo)) info_v, + Var(Int32) info_date_v + ) = + sleep(3000); + (if dispatcher_update_data(info_file_path,info_v,info_date_v) + then unique + else dispatcher_update_error(info_file_path)); + dispatcher_update_task(info_file_path,info_v,info_date_v). + + +public define One + start_web_dispatcher + ( + Int32 ip_address, // address for listening (typically 0: listen on all interfaces) + Int32 http_port, // typically 80 + DenialOfService dos + ) = + with info_file_path = my_anubis_directory+"/web_sites/dispatcher.info", + info_v = var((List(DispatcherInfo))[]), + info_date_v = var((Int32)0), + if dispatcher_update_data(info_file_path,info_v,info_date_v) + then if start_server(ip_address, + http_port, + make_dispatcher_handler(info_v,dos), + (One u)|->u) is + { + cannot_create_the_socket then + print("Cannot create the socket for HTTP server.\n"), + cannot_bind_to_port then + print("Cannot bind HTTP server to port "+http_port+".\n"), + cannot_listen_on_port then + print("HTTP server cannot listen on port "+http_port+".\n"), + ok(http_server) then + dispatcher_update_task(info_file_path,info_v,info_date_v) + } + else dispatcher_update_error(info_file_path). + + + + *** [7.2] The dispatcher web site. + + global define One + web_dispatcher + ( + List(String) args + ) = + start_web_dispatcher(0,80,load_denial_of_service_info). + + + + + + + *** [7.3] Managing the info file. + +define Int32 + register_ip_address + = + if ip_address(prompt(" numerical IP address (for HTTP): ")) is + { + failure then print(" *** Error: incorrect IP address.\n"); + register_ip_address, + success(n) then n + }. + + +define Int32 + register_ip_port + = + if string_to_integer(prompt(" IP port (for HTTP): ")) is + { + failure then print(" *** Error: incorrect IP port.\n"); + register_ip_port, + success(p) then if (0 =< p & p =< 65535) + then p + else print(" *** Error: IP port out of bounds.\n"); + register_ip_port + }. + + +define One + register_new_site + ( + Var(List(DispatcherInfo)) info_v + ) = + print("\n"); + print(" Registering a new site:\n"); + with name = prompt(" Site name: "), + with addr = register_ip_address, + with port = register_ip_port, + (protect info_v <- [site(name,port) . *info_v]); + print(" Site "+name+" at "+ip_addr_to_string(addr)+":"+port+" added\n (but not saved to disk).\n"). + + +define List(DispatcherInfo) + find_sites + ( + List(DispatcherInfo) l, + String name + ) = + if l is + { + [ ] then [ ], + [h . t] then if h is site(n,_) then + if find(name,n,0) is + { + failure then find_sites(t,name), + success(_) then [h . find_sites(t,name)] + } + }. + + +define String + pad + ( + String s, + Int32 l + ) = + if length(s) >= l + then s + else s+constant_string(l-length(s),' '). + + + +define One + show_sites_1 + ( + List(DispatcherInfo) l, + Int32 i + ) = + if l is + { + [ ] then unique, + [h . t] then if h is site(name,port) then + print(" ["+i+"] "+pad(name,40)+" "+" "+port+"\n"); + show_sites_1(t,i+1) + }. + + +define One + show_sites + ( + List(DispatcherInfo) l, + Int32 i + ) = + print(" Name Port\n"); + print(" --------------------------------------------------------\n"); + show_sites_1(l,i). + +define List(DispatcherInfo) + replace_info + ( + List(DispatcherInfo) l, + String site_name, + Int32 new_port + ) = + if l is + { + [ ] then alert, + [h . t] then if h is site(n,_) then + if n = site_name + then [site(n,new_port) . t] + else [h . replace_info(t,site_name,new_port)] + }. + +define List(DispatcherInfo) + delete_info + ( + List(DispatcherInfo) l, + String site_name, + ) = + if l is + { + [ ] then alert, + [h . t] then if h is site(n,_) then + if n = site_name + then t + else [h . delete_info(t,site_name)] + }. + + +define One + update_site + ( + Var(List(DispatcherInfo)) info_v, + String site_name, + Int32 old_port + ) = + print("\n"); + print(" Updating site '"+site_name+"': (currently: "+old_port+")\n"); + with new_port = register_ip_port, + answer = prompt(" Update '"+site_name+"' as: "+new_port+" [Y/N] ? "), + if (answer = "Y" | answer = "y") + then info_v <- replace_info(*info_v,site_name,new_port) + else unique. + + + +define Bool + compare + ( + DispatcherInfo d1, + DispatcherInfo d2 + ) = + if d1 is site(n1,_) then + if d2 is site(n2,_) then + string_less(n1,n2). + + + +define One + update_site + ( + Var(List(DispatcherInfo)) info_v + ) = + print("\n"); + with prefix = prompt(" Search for site to update: "), + if find_sites(*info_v,prefix) is + { + [ ] then print(" No site found.\n"); + update_site(info_v), + [h . t] then + show_sites(qsort([h . t],compare),1); + with i1 = prompt(" Choose a site to update [1/.../"+(length(t)+1)+"]: "), + if string_to_integer(i1) is + { + failure then print(" *** Error: site number not recognized.\n"); + update_site(info_v), + success(ii1) then if nth(ii1-1,*info_v) is + { + failure then print(" *** Error: site number "+i1+" does not exist.\n"); + update_site(info_v), + success(site_info) then if site_info is site(name,old_port) then + update_site(info_v,name,old_port) + } + } + }. + + +define One + delete_site + ( + Var(List(DispatcherInfo)) info_v, + String site_name, + Int32 old_port + ) = + print("\n"); + print(" Deleting site '"+site_name+"': (currently: "+old_port+")\n"); + with answer = prompt(" Are you sure you want to delete site: '"+site_name+"' [Y/N] ? "), + if (answer = "Y" | answer = "y") + then info_v <- delete_info(*info_v,site_name) + else print(" Site '"+site_name+"' not deleted.\n"). + + +define One + delete_site + ( + Var(List(DispatcherInfo)) info_v + ) = + print("\n"); + with prefix = prompt(" Search for site to delete: "), + if find_sites(*info_v,prefix) is + { + [ ] then print(" No site found.\n"); + delete_site(info_v), + [h . t] then + show_sites(qsort([h . t],compare),1); + with i1 = prompt(" Choose a site to delete [1/.../"+(length(t)+1)+"]: "), + if string_to_integer(i1) is + { + failure then print(" *** Error: site number not recognized.\n"); + delete_site(info_v), + success(ii1) then if nth(ii1-1,*info_v) is + { + failure then print(" *** Error: site number "+i1+" does not exist.\n"); + delete_site(info_v), + success(site_info) then if site_info is site(name,old_port) then + delete_site(info_v,name,old_port) + } + } + }. + + +define One + manager + ( + Var(List(DispatcherInfo)) info_v, + String file_path + ) = + print("\n"); + print(" --- Welcome to the Web Dispatcher Manager ---\n"); + with l = length(*info_v), + print(" "+l+" site"+(if l>1 then "s" else "")+" currently registred.\n"); + print(" [L] List registered sites.\n"); + print(" [R] Register a new site.\n"); + print(" [U] Update a registred site.\n"); + print(" [D] Delete a registred site.\n"); + with propose_write_v = var((Bool)true), + action = prompt(" Choose an action [L/R/U/D]: "), + (if (action = "L" | action = "l") then (show_sites(*info_v,1); propose_write_v <- false) else + if (action = "R" | action = "r") then register_new_site(info_v) else + if (action = "U" | action = "u") then update_site(info_v) else + if (action = "D" | action = "d") then delete_site(info_v) else + print("Action not recognized.\n")); + print("\n"); + if *propose_write_v then + with result = prompt(" Write modifications to data base [Y/N] ?"), + if (result = "Y" | result = "y") + then if save(*info_v,file_path) is + { + cannot_open_file then print(" File '"+file_path+"' not found.\n"), + write_error then print(" Error while writing file '"+file_path+"'.\n"), + ok then print(" Data base has been modified.\n") + } + else print(" Data base not modified.\n") + else unique. + + + +global define One + manage_web_dispatcher + ( + List(String) args + ) = + with info_v = var((List(DispatcherInfo))[]), + with file_path = my_anubis_directory+"/web_sites/dispatcher.info", + if (RetrieveResult(List(DispatcherInfo)))retrieve(file_path) is + { + cannot_find_file then print("File '"+file_path+"' does not exist.\n"); + with answer = prompt("Create it [Y/N] ? "), + if (answer = "Y" | answer = "y") + then if save((List(DispatcherInfo))[],file_path) is + { + cannot_open_file then + print("Cannot create file '"+file_path+"'.\n"), + write_error then + print("Error while creating file '"+file_path+"'.\n"), + ok then manager(info_v,file_path) + } + else unique, + read_error then print("Error while reading file '"+file_path+"'.\n"), + type_error then print("File '"+file_path+"' is corrupted.\n"), + ok(info) then info_v <- info; + manager(info_v,file_path) + }. + + + + + -- libgit2 0.21.4