Anubis / Anubis


 *Project*                                  Anubis
   
 *Title*                           A simple notion of stream.
   
 *Copyright*                     Copyright (c) Alain Prouté 2005. 


 *Author*       Alain Prouté
                David René
   
 *Overview*  
   
   A 'stream'  is an object  (in the sens of  OOP) from which  bytes may be read,  or into
   which bytes may be written.  We have defined the opaque type:

public type Stream:...

   in a  'implementation independant' way.   This means that  the actual source  or target
   behind  a   stream  may  be  a   file,  a  network  connection,   a  character  string,
   etc... Whatever it is, the stream is used the same way by the tools provided here.
   
   Different streams  may have different  capabilities. For example,  a stream may  not be
   writable. A  tool returns the capabilities  of the stream. If  you use a  function on a
   stream not having the corresponding capability, the function has just no effect. 
   
   
   *** Creating a stream. 
   
   As said above, a stream may be  created from various sources.  We provide below several
   functions for creating streams:
   
public define Stream      make_stream(RStream     file). 
public define Stream      make_stream(WStream     file). 
public define Stream      make_stream(RWStream    file).
public define Stream      make_stream(String         s).    

   You may also create a 'pipe' (see below). 
   
   
   
   *** Getting the capabilities. 
   
public type StreamCapability:
   readable,
   writable,
   seekable.
   
public define List(StreamCapability)   get_capabilities(Stream s). 
   
   
   
   
   *** Reading from a stream. 
   
public define Maybe(Word8)    read_byte        (Stream s). 
public define Maybe(String)   read_line        (Stream s).
public define List(String)    read_lines       (Stream s).
public define One             skip_blanks      (Stream s).
public define Maybe(String)   read_symbol      (Stream s).   
public define Maybe(String)   read_integer     (Stream s). 
public define List($T)        read_several     (Stream s, Stream -> Maybe($T) read_one).  
public define Maybe(List($T)) read_list        (Stream s, Stream -> Maybe($T) read_one). 
   
   
   'read_byte' waits until a byte can be  read and returns it or returns 'failure' in case
   no more byte will never be read from the stream.
   
   'read_line' reads from the current position to  end of current line. The end of line is
      any of:     LF
                  CR LF
                  CR
                  end of input
   
   'skip_blanks' skips blank  characters until the next non blank  character (which is not
      read)
   
   'read_symbol' reads  a symbol, skiping leading blanks  if necessary. A symbol  is a non
      empty concatenation of letters from 'a' to 'z', 'A' to 'Z', '0' to '9' and '_'. 
   
   'read_integer' is like 'read_symbol' but accepts only the characters '0' to '9'. 
   
   'read_several' applies 'read_one' repeatedly until it returns 'failure'. 'read_several'
      returns the list of all the entities read by 'read_one' (in their natural order). 
   
   'read_list'  expects an  opening parenthese,  then applies  'read_one'  repeatedly, and
      expects a closing parenthese. It returns the list of entities read by 'read_one'. 

      
   
   You may also unput an arbitrary number of bytes in the stream:
   
public define One        unput_byte       (Word8 c, Stream s).   
public define One        unput_string     (String str, Stream s).   
   If   the  bytes   unput  are   not  those   read  from   the  stream   the   result  is
   unpredictable. Unputing a '\n' may induce a wrong column count. 

   
   
   *** Informations about the current state of the stream. 
   
public define Int           current_offset   (Stream s).
public define Int           current_line     (Stream s).
public define Int           current_column   (Stream s). 
 public define Bool         at_end_of_input  (Stream s).    
          
   'current_offset' returns the number of bytes read so far.
   
   'current_line'   returns the number of '\n' read so_far.
   
   'current_column' returns the number of bytes read since the last '\n' (or the beginning
       of the input).
   

   
   *** Writting into a stream. 
   
public define Maybe(One)    write_byte       (Stream  s, Word8  byte). 
public define Maybe(One)    write_string     (Stream  s, String t).    
public define Maybe(One)    write_end        (Stream s).    
   
   'write_end' has some effect only for pipes (see below). 
   
   
   
   *** Making a pipe.
   
   A  'pipe' is  a stream  into which  we  may write  bytes, and  from which  we may  read
   bytes. All the bytes written into the pipe are readable from the pipe, which works like
   a FIFO  stack. The  pipe does  not handle offsets,  line and  column numbers  (they are
   always 0).
   
public define Stream
   make_pipe
     (
       Int buffer_size    
     ). 
   
   The size of the  buffer is the maximal number of bytes which may  be stored at the same
   time into  the pipe.   The size of  this buffer  never changes.  When  it is  full, the
   writting of new  bytes must wait until bytes  are read from the pipe, so  that there is
   some place for writting  new bytes.  Similarly, when the pipe is  empty, the reading of
   the  pipe waits until  a byte  is written  into it,  except if  a 'write_end'  has been
   executed on the pipe, in which case 'read_byte' returns 'failure'. When 'write_end' has
   been executed, it is no more possible to write into the pipe.
   
   As a consequence, pipes should always  be read and written by distinct virtual machines
   (use 'delegate' for launching a new virtual machine).

   
   
   --- That's all for the public part ! --------------------------------------------------

read tools/basis.anubis
   
   
   
   
public type Stream:
   stream(One -> Maybe(Word8)       rb,       // read a byte
          Word8 -> Maybe(One)       wb,       // write a byte
          Word8 -> One              ub,       // unput a byte
          One -> Maybe(One)         we,       // write an end
          One -> Maybe(String)      rl,       // read a line
          One -> Int                of,       // offset
          One -> Int                ln,       // line
          One -> Int                cl).      // column
   
         

define One -> Maybe(Word8)
   make_rb   // making the 'rb' function 
     (
       RStream            file,
       Var(List(Word8))   v, 
       Var(Int)           o, 
       Var(Int)           l,
       Var(Int)           c
     ) =
   (One u) |-> if *v is 
      {
        [ ] then if *file is 
          {
            failure then failure, 
            success(byte) then o <- *o+1;      //increment the offset counter
                               (if byte = '\n' then (l <- *l+1; c <- 0) 
                                               //and if it is a new line increment the line counter
                                               else c <- *c+1);
                               success(byte)   
          }, 
        [h . t] then v <- t; 
                     o <- *o+1; 
                     (if h = '\n' then (l <- *l+1; c <- 0)
                                  else c <- *c+1); 
                     success(h)
     }. 
   

   
define Word8 -> One
   make_ub     // making the 'ub' function 
     (
       Var(List(Word8))   v, 
       Var(Int)        o, 
       Var(Int)        l,
       Var(Int)        c
     ) =
   (Word8 byte) |-> v <- [byte . *v]; 
                   o <- *o-1;
                   (if byte = '\n' then (l <- *l-1; c <- 0)
                                   else c <- *c-1).

   *Description*  'read_text_byte' transforms  a  function of  type  'One ->  Maybe(Word8)'
   reading bytes  in binary mode into  a function of the  same type reading  bytes in text
   mode. By convention all ends of lines are returned as '\n'. 
   
define One -> Maybe(Word8)
   read_text_byte
     (
       One -> Maybe(Word8) rbs,
       Word8 -> One        unput
     ) =
   (One u) |-> 
     if rbs(unique) is 
       {
         failure    then failure, 
         success(c) then 
           if c = '\r'
           then if rbs(unique) is 
             {
               failure     then success('\n'),
               success(d)  then 
                 if d = '\n'
                 then success('\n')
                 else unput(d); success('\n')
             }
           else success(c)
                /* actually: 
                if c = '\n'
                then success('\n')
                else success(c)
                */
       }.

define Maybe(String)
   read_line
     (
       One -> Maybe(Word8)    rtb,       // this function reads bytes in text mode
       List(Word8)            so_far
     ) =
   if rtb(unique) is 
     {
       failure     then if so_far is []
                        then failure 
                        else success(implode(reverse(so_far))), 
       success(c)  then 
         if c = '\n'
         then success(implode(reverse([c . so_far])))
         else read_line(rtb,[c . so_far])
     }. 
      
   
public define Stream
   make_stream
     (
       RStream    file
     ) =
   with v = var((List(Word8))[]),     // stack of unput bytes
        o = var((Int)0),           // offset
        l = var((Int)1),           // line
        c = var((Int)0),           // column
        rb = make_rb(file,v,o,l,c),
        ub = make_ub(v,o,l,c),
   stream(
     rb, 
     (Word8 w) |-> failure, 
     ub, 
     (One u) |-> failure, 
     (One _) |-> 
          if *v is [] then
            if read_line(file, 4096, 0) is ok(line) then l <- *l + 1; c <- 0; o <- tell(file); success(line) else failure
          else
            read_line(read_text_byte(rb,ub),[]),
     (One u) |-> *o, 
     (One u) |-> *l,
     (One u) |-> *c
         ). 
   
public define Stream
   make_stream
     (
       RWStream    file
     ) =
   with v = var((List(Word8))[]),     // stack of unput bytes
        o = var((Int)0),           // offset
        l = var((Int)1),           // line
        c = var((Int)0),           // column
        rb = make_rb(weaken(file),v,o,l,c),
        ub = make_ub(v,o,l,c),
   stream(
     rb, 
     (Word8 w) |-> file <- w, 
     ub, 
     (One u) |-> failure, 
     (One _) |-> 
          if *v is [] then
            if read_line(weaken(file), 4096, 0) is ok(line) then l <- *l + 1; c <- 0; o <- tell(file); success(line) else failure
          else
            read_line(read_text_byte(rb,ub),[]),
     (One u) |-> *o, 
     (One u) |-> *l,
     (One u) |-> *c
         ). 
   
public define Stream
   make_stream
     (
       WStream    file
     ) =
   stream(
     (One u) |-> failure,
     (Word8 c) |-> file <- c,
     (Word8 c) |-> unique, 
     (One u) |-> failure, 
     (One u) |-> failure, 
     (One u) |-> 0,
     (One u) |-> 0, 
     (One u) |-> 0
   ).
   
public define Stream
   make_stream
     (
       String s
     ) =
   with i = var((Int)0), 
        l = var((Int)1), 
        c = var((Int)0), 
        b = var((List(Word8))[]), 
        d = var((List(String))[]),
        n = var((Int)0), 
        // functions
        rb = (One u) |-> if nth(*i,s) is 
                {
                  failure then failure, 
                  success(byte) then i <- *i+1; 
                                     (if byte = '\n' then (l <- *l+1; c <- 0)
                                                     else c <- *c+1);
                                     success(byte)
                },
        ub = (Word8 byte) |-> i <- *i-1;
                (if byte = '\n' then (l <- *l-1; c <- 0)
                                else c <- *c-1),

   stream(
          // reading the next byte:
          rb,
   
          // writing a byte
          (Word8 w) |-> 
             b <- [w . *b];
             n <- *n + 1; 
             if *n >= 100
             then 
               (protect
                d <- [implode(reverse(*b)) . *d];
                b <- [];
                n <- 0;
                success(unique)
               )
             else success(unique), 
   
          // unputting a byte:
          ub,
          
          // no end to write 
          (One u) |-> failure, 
   
          // reading line
          (One _) |-> read_line(read_text_byte(rb,ub),[]),
          // getting the offset:
          (One u) |-> *i,
   
          // getting the line:
          (One u) |-> *l,
    
          // getting the column
          (One u) |-> *c
         ). 
   
   
   

public define Int      
   current_offset
     (
       Stream s
     ) =
   of(s)(unique).    
   
   
public define Int
   current_line
     (
       Stream s
     ) =
   ln(s)(unique). 
   
public define Int
   current_column
     (
       Stream s
     ) =
   cl(s)(unique). 
   
   
public define Maybe(Word8)
   read_byte
     (
       Stream s
     ) =
   rb(s)(unique). 
   
public define One
   unput_byte
     (
       Word8 c, 
       Stream s
     ) =
   ub(s)(c). 
   
public define One
   unput_string
     (
       String string, 
       Stream s
     ) =
   map_forget((Word8 c) |-> ub(s)(c), explode(string))   
   .    


public define Maybe(String)
   read_line
     (
       Stream s
     ) =
   rl(s)(unique).
   //read_line(read_text_byte(rb(s),ub(s)),[]). 
   

define List(String) 
  read_lines
  (
    Stream s,
    List(String) tmp_list,
  ) = 
  if read_line(s) is
  {
    failure       then reverse(tmp_list),
    success(line) then read_lines(s, [line . tmp_list])
  }.

	
public define List(String) 
  read_lines
  (
    Stream s
  ) = 
  read_lines(s, []).

public define One
  unput_line
  (
    Stream  s,
    String  line
  )=
  forget(map((Word8 c) |-> ub(s)(c) , reverse(explode(line)))).
  
public define One
   skip_blanks
     (
       Stream     s
     ) =
   if rb(s)(unique) is 
     {
       failure then unique, 
       success(c) then 
         if c +=< ' '
         then skip_blanks(s)
         else ub(s)(c)
     }. 
   
   
   
define Maybe(String)
   read_symbol
     (
       Stream         s,
       List(Word8)     so_far,
       Word8 -> Bool   is_acceptable_char
     ) =
   if rb(s)(unique) is 
     {
       failure then if so_far is 
         {
           [ ] then failure, 
           [_ . _] then success(implode(reverse(so_far)))
         },
       success(c) then 
         if is_acceptable_char(c)
         then read_symbol(s,[c . so_far],is_acceptable_char)
         else 
           ub(s)(c); 
           if so_far is 
             {
               [ ] then failure, 
               [_ . _] then success(implode(reverse(so_far)))
             }
     }.
   
   
define Bool
   is_symbol_char
     (
       Word8   c
     ) =
   if c = '_' then true else
   if ('a' +=< c & c +=< 'z') then true else
   if ('A' +=< c & c +=< 'Z') then true else
   ('0' +=< c & c +=< '9').
   
define Bool
   is_integer_char
     (
       Word8   c
     ) =
   '0' +=< c & c +=< '9'.
   
   
public define Maybe(String)
   read_symbol
     (
       Stream     s
     ) =
   skip_blanks(s);
   read_symbol(s,[],is_symbol_char).    
   
public define Maybe(String)
   read_integer
     (
       Stream     s
     ) =
   skip_blanks(s);
   read_symbol(s,[],is_integer_char).    
   
   
define List($T)
   read_several
     (
       Stream                   s, 
       Stream -> Maybe($T)      read_one,
       List($T)                 so_far
     ) =
   if read_one(s) is 
     {
       failure then reverse(so_far),
       success(u) then read_several(s,read_one,[u . so_far])
     }.
   
   
   
public define List($T)
   read_several
     (
       Stream                   s, 
       Stream -> Maybe($T)      read_one
     ) =
   read_several(s,read_one,[]).    
   
   
define List($T)
   read_list
     (
       Stream                   s, 
       Stream -> Maybe($T)      read_one,
       List($T)                 so_far
     ) =
   if read_one(s) is 
     {
       failure then 
         skip_blanks(s); 
         if rb(s)(unique) is 
           {
             failure then reverse(so_far),
             success(c) then
               if c = ')'
               then reverse(so_far)
               else (ub(s)(c); reverse(so_far))
           },
       success(u) then read_list(s,read_one,[u . so_far])
     }.
   
public define Maybe(List($T))   
   read_list
     (
       Stream                   s, 
       Stream -> Maybe($T)      read_one
     ) =
   skip_blanks(s); 
   if rb(s)(unique) is 
     {
       failure then failure, 
       success(c) then 
         if c = '('
         then success(read_list(s,read_one,[]))
         else ub(s)(c); failure
     }.
   
   
public define Maybe(One)
   write_byte
     (
       Stream  s, 
       Word8  byte
     ) =
   wb(s)(byte). 

   
   
define Maybe(One)    
   write_string     
     (
       Word8 -> Maybe(One)  wb, 
       String              t,
       Int               i
     ) =    
   if nth(i,t) is 
     {
       failure then success(unique), 
       success(c) then 
         if wb(c) is 
           {
             failure then failure, 
             success(_) then write_string(wb,t,i+1)
           }
     }.
   
   
public define Maybe(One)    
   write_string
     (
       Stream      s, 
       String      t
     ) =    
   write_string(wb(s),t,0). 
   

   
   *** Pipes. 
   
   
define One sleep = checking every 1 milliseconds, wait for true then unique. 
   
   The buffer is 'circular'. There are two situations:
   
        -------------bbbbbbbbbbbbbbbbbbbbbb------------------
                     ^                     ^
                     |                     | 
                     |                     next_to_write
                     next_to_read
   
   
        bbbbbbbbbbbbb-------------------bbbbbbbbbbbbbbbbbbbbb
                     ^                  ^
                     |                  |
                     |                  next_to_read
                     next_to_write
   
   where 'b' is a stored  byte, and '-' a free slot. There is always  at least 1 free slot
   in the  buffer so that 'next_to_read =  next_to_write' means that the  buffer is empty.
   If the buffer is not empty, 'next_to_read' always points to an actual byte.
   
   
   
public define Stream
   make_pipe
     (
       Int buffer_size   // actually this will be 1 less than the actual buffer size
     ) =
   with          bs = if buffer_size < 10 then 10 else buffer_size, 
             buffer = constant_byte_array(bs + 1, 0), 
       next_to_read = var((Int)0),    // offset (into buffer) of next byte to be read
      next_to_write = var((Int)0),    // offset (into buffer) of next byte to be written
             at_end = var((Bool)false), 
         unput_list = var((List(Word8))[]), 
   stream
     (
     /* rb */ 
     (One u) |-rb-> if *unput_list is 
                      {
                        [ ] then with ntr = *next_to_read, 
                          if ntr = *next_to_write
                          then (if *at_end then failure else (sleep; rb(u)))
                          else with result = force_nth(ntr,buffer),
                                 next_to_read <- (if ntr = bs then 0 else ntr + 1);
                                 success(result), 
                        [h . t] then 
                          unput_list <- t; 
                          success(h)
                      },
   
     /* wb */ 
     (Word8 c) |-wb-> if *at_end then failure else 
                     with ntw = *next_to_write,
                          ntr = *next_to_read, 
                       if ntw + 1 = ntr
                       then /* full */ (sleep; wb(c))
                       else if ntr = 0
                            then if ntw = bs
                                 then /* full */ (sleep; wb(c))
                                 else (forget(put(buffer,ntw,c));
                                       next_to_write <- ntw + 1;
                                       success(unique))
                            else if ntw = bs
                                 then (forget(put(buffer,ntw,c)); 
                                       next_to_write <- 0;
                                       success(unique))
                                 else (forget(put(buffer,ntw,c));
                                       next_to_write <- ntw + 1;
                                       success(unique)),
   
     /* ub */
     (Word8 c) |-> unput_list <- [c . *unput_list], 
   
     /* we */
     (One u) |-> success(at_end <- true), 
     
     /* rl */
     (One _) |-> failure, // TODO implement this
   
     /* of */
     (One u) |-> 0, 
   
     /* ln */
     (One u) |-> 0, 
   
     /* cl */
     (One u) |-> 0
     ). 
   
   
public define Maybe(One)
   write_end
     (
       Stream s
     ) =
   we(s)(unique).