Anubis / Anubis

Commit 1a5d477a9e8f851594fb9f4c81a18dd8d04d675b

Authored by Alain Prouté
1 parent 81bbc2e5

-

Showing 8 changed files with 1076 additions and 310 deletions   Show diff stats
Inline Side-by-side
anubis_dev/library/data_base/metaSQL.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -12,19 +12,15 @@
12 12 Anubis source file to be part of your own program.
13 13  
14 14 The generated program is independant of the particular database you are using. It generates
15   - standard SQL commands. The only thing you have to provide is the 'query function' of type:
  15 + standard SQL commands. The only thing you have to provide are the (database software dependant)
  16 + informations which will permit the construction of the 'query function'. The details are
  17 + given below.
16 18  
17   - (String sql_query) -> MetaSQL_Answer
18   -
19   - suitable for your database software. How this query function is to be written is
20   - explained below. The instance of this function suitable for PostgreSQL is available
21   - in 'MetaSQL_Postgres.anubis'.
22   -
23   - If you use this metaprogram, you can use a database in your program without knowing
24   - anything to the SQL language. Nevertheless, you need to have basic notions about
25   - databases, and in particular about how to design well behaved databases.
  19 + If you use this metaprogram, you can use a database with only an approximate knowledge
  20 + of the SQL language. Nevertheless, you need to have basic notions about databases, and
  21 + in particular about how to design well behaved databases.
26 22  
27   - In an Anubis program, a row of data comming from a table should be a single datum whose
  23 + In an Anubis program, a row of data comming from a SELECT request should be a single datum whose
28 24 type has only one alternative within which each component corresponds to a datum of this
29 25 row. This is of course not difficult to transform a row got from a network connection
30 26 with a database into such a single datum. However, this transformation may encounter
... ... @@ -33,22 +29,20 @@
33 29 which must be an integer cannot be converted using 'decimal_scan'), etc. As a consequence,
34 30 writing down safe and complete translation functions in Anubis, taking into account
35 31 all possible errors (even if they are not likely to occur) is a huge and boring work.
36   - The purpose of this metaprogram is to generate these functions automatically from a
  32 + One of the purposes of this metaprogram is to generate these functions automatically from a
37 33 formal description of the database and of the queries you want to execute.
38 34  
39 35 This metaprogram also provides tools for initializing the database or automatically
40   - modifying it in case you change its description.
  36 + modifying it in case you change its description. It also provides a web interface
  37 + for administering your database.
41 38  
42 39 This metaprogram also provides other avantages. It is able to check that queries
43   - satisfy some necessary conditions (for example, that 'INSERT' has an actual value for
44   - each NOT NULL column), so that the corresponding error cannot happen at execution time.
  40 + satisfy some required conditions (for example, that 'INSERT' gets an actual value for
  41 + each NOT NULL column), so that the corresponding error will not happen at execution time.
45 42 It also checks properties of the database description itself. The list of verifications
46   - provided by this program is listed below.
47   -
48   - In some sens, the present metaprogram puts your database programming under the control
49   - of the Anubis compiler, so that this part of your program benefits of the accurate
50   - type checking of Anubis.
  43 + provided by this program is given below.
51 44  
  45 +
52 46  
53 47 ---------------- Table of Contents ------------------------------------------------
54 48  
... ... @@ -56,15 +50,29 @@
56 50 *** (1) How it works.
57 51 *** (2) Classifying SQL commands.
58 52 *** (3) Formal description of the database.
59   - *** (4) Formal description of queries.
  53 + *** (3.1) 'MetaSQL_Type' (data types of table columns).
  54 + *** (3.2) 'MetaSQL_Attr' (attributes of columns).
  55 + *** (3.3) 'MetaSQL_PrimKey' (chosing to have a primary key column in a table).
  56 + *** (3.4) 'MetaSQL_ColStruct' (describing a column in a table).
  57 + *** (3.5) 'MetaSQL_TabStruct' (describing a table).
  58 + *** (3.6) 'MetaSQL_DBStruct' (describing a whole database).
  59 + *** (4) Formal description of queries (metaqueries).
  60 + *** (4.1) 'MetaSQL_Column' (naming columns in WHERE clauses).
  61 + *** (4.2) 'MetaSQL_Var' (introducing variables in WHERE clauses).
  62 + *** (4.3) 'MetaSQL_Where' (describing a WHERE clause).
  63 + *** (4.4) 'MetaSQL_Range' (getting only a range of rows in a SELECT query).
  64 + *** (4.5) 'MetaSQL_DML' (describing queries for the DML part of SQL).
  65 + *** (4.6) Convenience functions for SELECT queries.
60 66 *** (5) The function 'make_queries'.
61 67 *** (6) Cursors.
62 68 *** (7) Transactions.
63 69 *** (8) Providing the 'query function'.
64   - *** (9) Considerations relative to the use of metaSQL in a web site.
65   - *** (10) Verifications performed by this metaprogram.
66   - *** (10.1) Cycles of references in the database.
67   - *** (10.2) Values provided to _INSERT.
  70 + *** (9) Verifications performed by this metaprogram.
  71 + *** (9.1) Cycles of references in the database.
  72 + *** (9.2) Values provided to _INSERT.
  73 + *** (9.3) Coherence of WHERE clauses.
  74 + *** (10) Creating the database and modifying its structure.
  75 + *** (11) Database administration through the web.
68 76  
69 77 -----------------------------------------------------------------------------------
70 78  
... ... @@ -88,7 +96,7 @@
88 96 within which you write what is shown below. It is recommanded to make a directory 'generated'
89 97 within your source directory, where automatically generated files will be put. This helps
90 98 to distinguish manually written Anubis files from automatically generated Anubis files
91   - (which are not true source files).
  99 + (which are not true source files, and which should never be modified by hand).
92 100  
93 101 --- The file 'meta_queries.anubis' ----------------------------------------------
94 102  
... ... @@ -106,7 +114,7 @@
106 114 ... describe your database here (see below) ...
107 115  
108 116 --- Description of all my queries:
109   - define List(MetaSQL_DML) // DML stands for 'Data Manuipulation Language' (a part of SQL)
  117 + define List(MetaSQL_DML) // DML stands for 'Data Manipulation Language' (a part of SQL)
110 118 my_queries
111 119 =
112 120 [
... ... @@ -127,11 +135,13 @@
127 135  
128 136 --- Generate the file 'generated/queries.anubis':
129 137 execute anbexec make_my_queries
130   - read generated/queries.anubis // this is just for testing that the generated file
131   - // compiles ok. In any case, this does not waste
132   - // compilation time since during a compilation
133   - // each file is read only once regardless of
134   - // the number of 'read' of this file.
  138 +
  139 + --- Check it:
  140 + This is just for testing that the generated file compiles ok. In any case, this does not
  141 + waste compilation time since during a compilation each file is read only once regardless
  142 + of the number of 'read' of this file.
  143 +
  144 + read generated/queries.anubis
135 145  
136 146 --- End of 'meta_queries.anubis' ------------------------------------------------
137 147  
... ... @@ -187,6 +197,8 @@
187 197  
188 198 *** (3) Formal description of the database.
189 199  
  200 + *** (3.1) 'MetaSQL_Type' (data types of table columns).
  201 +
190 202 Sorts of data we can find in the database are listed below. We consider that the
191 203 fact of being 'nullable' is a matter of type, hence the following definition:
192 204  
... ... @@ -212,20 +224,25 @@ public type MetaSQL_Type:
212 224  
213 225 From the point of view of your Anubis program, these data will be of types:
214 226  
215   - binary --> ByteArray
216   - boolean --> Bool
217   - date --> Int (number of seconds since the 'epoch': 1970-01-01:00:00:00)
218   - foreign --> MetaSQL_Id
219   - integer16 --> Word16
220   - integer32 --> Word32
221   - integer --> Int
222   - text --> String
223   - vartext --> String
  227 + Name | Anubis type | Type within the database
  228 + ----------+-----------------+--------------------------------------------------
  229 + binary | ByteArray | text (the byte array base64 encoded)
  230 + boolean | Bool | boolean
  231 + date | String | text (date in ISO-8601 format: yyyy-mm-dd hh:mm:ss)
  232 + foreign | MetaSQL_Id | 32 bits (or 64 bits) integer
  233 + integer16 | Word16 | 16 bits integer
  234 + integer32 | Word32 | 32 bits integer
  235 + integer | Int | arbitrary size integer (numeric)
  236 + text | String | max size text
  237 + vartext | String | arbitrary size text
224 238  
225   - and similarly for nullable types, with a 'Maybe':
  239 + and similarly for nullable types, except that the Anubis type is wrapped into a 'MaybeNull(...)'
  240 + (the type schema 'MaybeNull()' is defined in 'library/data_base/MetaSQL_tools.anubis').
  241 + Conversion tools between dates in ISO-8601 format and 'Int' (number of seconds since the epoch)
  242 + are provided in 'library/tools/ISO-8601.anubis'.
226 243  
227   - binary_or_null --> Maybe(ByteArray)
228   - ... etc ...
  244 +
  245 + *** (3.2) 'MetaSQL_Attr' (attributes of columns).
229 246  
230 247 Possible attributes of columns: (each column has a list of such attributes)
231 248  
... ... @@ -233,11 +250,18 @@ public type MetaSQL_Attr:
233 250 unique, // by default, the values in a column are not required to be all different
234 251 indexed. // by default, a column is not indexed
235 252  
  253 + Of course, 'NOT NULL' is not an alternative of 'MetaSQL_Attr' because it is already coded into the
  254 + database type itself (type 'MetaSQL_Type' above).
  255 +
  256 +
  257 + *** (3.3) 'MetaSQL_PrimKey' (chosing to have a primary key column in a table).
  258 +
236 259 'MetaSQL_PrimKey': does the table have a primary key or not ?
237 260  
238 261 public type MetaSQL_PrimKey:
239   - with_pk, // with primary key
240   - no_pk. // without primary key
  262 + with_pk_32, // with 32 bits primary key
  263 + with_pk_64, // with 64 bits primary key
  264 + no_pk. // without primary key
241 265  
242 266 Note: A primary key column is always named 'id', it is of type MetaSQL_Id as far as Anubis
243 267 is concerned, and has attributes 'unique', 'not_null' and 'indexed'. Futhermore, it
... ... @@ -249,10 +273,12 @@ public type MetaSQL_PrimKey:
249 273 column with attribute 'unique'. For example, in a web site, where users may be
250 274 registered, they can be uniquely identified by their email address. Nevertheless, despite
251 275 the fact that the column 'email' has attribute 'unique', you should still have an 'id'
252   - column. The two columns do not have equivalent behaviors as demonstrated below. Indeed,
253   - if you get an email address via the network (someone filling up a registration form for example)
254   - you have to check if this address is already in the database. To that end, you use a
255   - SELECT command, with condition email = ..., and the number of rows in the answer can be:
  276 + column. Of course, this allows users to change their email address, but there are more
  277 + general reasons. The two columns 'id' and 'email' do not have equivalent behaviors as
  278 + demonstrated below. Indeed, if you get an email address via the network (someone filling
  279 + up a registration form for example) you have to check if this address is already in the
  280 + database. To that end, you use a SELECT command, with condition email = ..., and the
  281 + number of rows in the answer can be:
256 282  
257 283 0 the email address is not yet registered
258 284 1 the email address is already registered
... ... @@ -261,7 +287,7 @@ public type MetaSQL_PrimKey:
261 287 Now, if you have the 'id' of a user at hand, this 'id' comes necessarily from the database.
262 288 This means that the corresponding row always exists, and the number of rows returned by
263 289 a SELECT command, with condition id = ... must always return one and only one row. The case
264   - it returns 0 row is an now error. (In case you delete a row, you must of course abandon
  290 + it returns 0 row is now an error. (In case you delete a row, you must of course abandon
265 291 the corresponding id which becomes non significant at that moment.)
266 292  
267 293 Assuming that the Anubis type of a row is ROW (you choose this name yourself, but not the
... ... @@ -273,19 +299,27 @@ public type MetaSQL_PrimKey:
273 299 and at the corresponding generated file to check this affirmation.
274 300  
275 301  
  302 + *** (3.4) 'MetaSQL_ColStruct' (describing a column in a table).
  303 +
276 304 Description of a column:
277 305  
278 306 public type MetaSQL_ColStruct:
279   - col (String name, // ordinary column (i.e. all but 'id')
  307 + col (String name, // name of ordinary column (i.e. all but 'id')
280 308 MetaSQL_Type type,
281 309 List(MetaSQL_Attr) attributes).
282 310  
  311 +
  312 + *** (3.5) 'MetaSQL_TabStruct' (describing a table).
  313 +
283 314 Description of a table:
284 315  
285 316 public type MetaSQL_TabStruct:
286 317 table (String name, // name of the table
287 318 MetaSQL_PrimKey pk, // with or without primary key column
288   - List(MetaSQL_ColStruct) columns). // other than the primary key column if there is one
  319 + List(MetaSQL_ColStruct) columns). // columns other than the primary key column (if any)
  320 +
  321 +
  322 + *** (3.6) 'MetaSQL_DBStruct' (describing a whole database).
289 323  
290 324 Description of a whole database:
291 325  
... ... @@ -297,7 +331,7 @@ public type MetaSQL_DBStruct:
297 331  
298 332  
299 333  
300   - *** (4) Formal description of queries.
  334 + *** (4) Formal description of queries (metaqueries).
301 335  
302 336 SQL commands from the DML are represented by the following type:
303 337  
... ... @@ -306,7 +340,10 @@ public type MetaSQL_DML:... (defined below)
306 340 This type has alternatives corresponding to INSERT, SELECT, UPDATE and DELETE, but the
307 341 components need auxiliary types, that we now define.
308 342  
309   - The 'SELECT' query may refer to several columns not all in the same table ('joins').
  343 +
  344 + *** (4.1) 'MetaSQL_Column' (naming columns in WHERE clauses).
  345 +
  346 + A 'SELECT' query may refer to several columns not all in the same table ('joins').
310 347 Hence, we must define the notion of 'selection of columns'. This is a list of data of type:
311 348  
312 349 public type MetaSQL_Column:
... ... @@ -317,13 +354,19 @@ public type MetaSQL_Column:
317 354 (used as end of paragraph or in the list notation '[_ . _]') is not the same operator as a dot not
318 355 followed by a space (used here).
319 356  
320   - Most queries include a condition, also called a 'where clause', which contains the 'variables'
  357 +
  358 + *** (4.2) 'MetaSQL_Var' (introducing variables in WHERE clauses).
  359 +
  360 + Most queries include a condition, also called a 'WHERE clause', which contains the 'variables'
321 361 taking different values at each call.
322 362  
323 363 public type MetaSQL_Var: // variables for queries
324 364 var(String name).
325 365  
326   -public type MetaSQL_Where: // formal description of 'WHERE' clause
  366 +
  367 + *** (4.3) 'MetaSQL_Where' (describing a WHERE clause).
  368 +
  369 +public type MetaSQL_Where: // formal description of a 'WHERE' clause
327 370 MetaSQL_Column = MetaSQL_Var, // equality of values
328 371 MetaSQL_Column < MetaSQL_Var, // comparison for numbers, dates, ...
329 372 MetaSQL_Var < MetaSQL_Column,
... ... @@ -339,7 +382,7 @@ public type MetaSQL_Where: // formal description of &#39;WHERE&#39; clause
339 382 ~ MetaSQL_Where. // 'NOT'
340 383  
341 384 The same one, but with column names without table name (used when there is only one table
342   - in a SELECT). Actually, this is just to put some syntactic sugar.
  385 + involved in a SELECT). Actually, this is just to put some syntactic sugar.
343 386  
344 387 public type MetaSQL_Where1:
345 388 String = MetaSQL_Var, // equality of values
... ... @@ -356,6 +399,9 @@ public type MetaSQL_Where1:
356 399 MetaSQL_Where1 | MetaSQL_Where1, // 'OR'
357 400 ~ MetaSQL_Where1. // 'NOT'
358 401  
  402 +
  403 + *** (4.4) 'MetaSQL_Range' (getting only a range of rows in a SELECT query).
  404 +
359 405 In a SELECT statement, you may not want to get all rows matching the condition but only
360 406 a number of rows starting at some row, i.e. a 'range' of rows. Notice that using this
361 407 is less efficient than using cursors, since the rows will be tested from the beginning
... ... @@ -367,10 +413,12 @@ public type MetaSQL_Range:
367 413 range(Int start, Int how_many). // get only a range of rows
368 414  
369 415  
370   - Below is the definition of the type MetaSQL_DML. Each alternative has a 'query_name' component
  416 + *** (4.5) 'MetaSQL_DML' (describing queries for the DML part of SQL).
  417 +
  418 + Below is the definition of the type 'MetaSQL_DML'. Each alternative has a 'query_name' component
371 419 which becomes the name of the function to be called for executing the query. Things which seem
372 420 to be missing in this definition (like form example the values to be inserted in the case of
373   - _INSERT) become arguments to the generated function (with the types according to the description
  421 + _INSERT) become arguments to the generated function (typed according to the description
374 422 of the database).
375 423  
376 424 public type MetaSQL_DML:
... ... @@ -378,16 +426,18 @@ public type MetaSQL_DML:
378 426 (
379 427 String query_name,
380 428 String table_name, // the name of the table into which rows must be inserted
381   - List(String) columns_names // all 'NOT NULL' columns must be present
  429 + List(String) columns_names // all 'NOT NULL' columns (except 'id') must be present
  430 + // and 'id' must not be present
382 431 // values are provided as arguments to the generated function
383 432 ),
384 433 _SELECT
385 434 (
386 435 String query_name,
387   - String output_type_name, // name for the output type of the generated function
  436 + String output_type_name, // name for the output type (for rows) of the generated function
388 437 List(MetaSQL_Column) columns, // a selection of columns from several tables
389 438 MetaSQL_Where condition,
390 439 MetaSQL_Range range
  440 + // some convenience variants of this are provided (see below)
391 441 ),
392 442 _UPDATE
393 443 (
... ... @@ -404,18 +454,21 @@ public type MetaSQL_DML:
404 454 MetaSQL_Where1 condition
405 455 ).
406 456  
  457 +
407 458 Note: the names of the queries need not be unique provided the generated functions may be
408 459 distinguished by their types.
409 460  
410   - Note: the name of the output type in _SELECT queries need not be unique provided that
  461 + Note: the name of the output type for rows in _SELECT queries need not be unique provided that
411 462 any two usages of this name correspond to types with the same components. In other words,
412 463 if you use the same name for the output type in two _SELECT queries, ensure that the
413   - selections of columns (3rd argument) are identical. Anyway, if this is not the case, this
414   - will detected by the Anubis compiler at the compilation of the generated file.
  464 + selections of columns (3rd argument) are identical. Anyway, if it is not the case, this
  465 + will be detected by the Anubis compiler at the compilation of the generated file.
  466 +
415 467  
  468 + *** (4.6) Convenience functions for SELECT queries.
416 469  
417 470 In case there is only one table used in a SELECT, you can use this more handy function,
418   - so that you can write everywhere "price" instead of "product"."price".
  471 + so that, for example, you can write "price" instead of "product"."price".
419 472  
420 473 public define MetaSQL_DML
421 474 _SELECT
... ... @@ -428,19 +481,31 @@ public define MetaSQL_DML
428 481 MetaSQL_Range range
429 482 ).
430 483  
  484 + The same one without the 'range' argument (assumed to be 'all'):
431 485  
  486 +public define MetaSQL_DML
  487 + _SELECT
  488 + (
  489 + String query_name,
  490 + String output_type_name,
  491 + String table_name,
  492 + List(String) columns_names,
  493 + MetaSQL_Where1 condition
  494 + ).
  495 +
432 496  
433 497  
434 498 *** (5) The function 'make_queries'.
435 499  
436   - The function 'make_queries' is declared as follows:
  500 + The function 'make_queries' (the main function for generating the target file)
  501 + is declared as follows:
437 502  
438 503 public define One
439 504 make_queries
440 505 (
441   - String target_file_path,
  506 + String target_file_path, // for example "generated/queries.anubis"
442 507 List(String) files_to_be_read,
443   - List(MetaSQL_DML) queries,
  508 + List(MetaSQL_DML) metaqueries,
444 509 MetaSQL_DBStruct database_description
445 510 ).
446 511  
... ... @@ -452,13 +517,9 @@ public define One
452 517 (4) the description of your database.
453 518  
454 519  
455   -
456   -
457   -
458 520 *** (6) Cursors.
459 521  
460   -
461   -
  522 + Not yet implemented in metaSQL.
462 523  
463 524  
464 525  
... ... @@ -468,12 +529,45 @@ public define One
468 529 several queries as an 'atomic' operation, that is to say an operation which may either have no effect
469 530 at all on the database it it fails, or which is correctly completed if it succeeds. In other words,
470 531 executing the sequence of queries only partially becomes impossible, so that the integrity of the
471   - database is warranted.
472   -
473   -
474   -
475   -
476   -
  532 + database is warranted, and other users of the database can see the final result only (or nothing at
  533 + all if the sequence fails at some point).
  534 +
  535 + Define a function, say 'sequence', of type '(String -> MetaSQL_Answer) -> Result(String,T)' where
  536 + the type T is the type of the result you want to get after the sequence of queries is successfully
  537 + executed. The function 'sequence' is supposed to send several queries to the database. It must return the
  538 + value 'error(msg)' (where 'msg' is an error message) if the sequence fails. Otherwise, it must return
  539 + 'ok(result)', where 'result' is the wanted result.
  540 +
  541 + In order to wrap this sequence of queries into a transaction just write:
  542 +
  543 + transaction(qf)(sequence)
  544 +
  545 + where 'qf' is the 'query function' (of type String -> MetaSQL_Answer). This term is of type
  546 + 'Result(String,T)'. If it returns 'error(msg)', the database was not affected by its execution.
  547 + If it returns 'ok(result)', the complete sequence of queries was successfully executed.
  548 +
  549 + Notice that the function 'sequence' should decide to send a given query only if the previous ones
  550 + in the sequence are successful. This is the reason why the sequence cannot be just a list of
  551 + queries (or a list of descriptions of queries), but must be a function written outside the description
  552 + of queries, and using all the expressiveness of the language.
  553 +
  554 + The function 'transaction' is defined in 'library/data_base/metaSQL_tools.anubis'. It is a good
  555 + example of the expressiveness of functionnal programming. It is of type:
  556 +
  557 + (String -> MetaSQL_Answer) -> (((String -> MetaSQL_Answer) -> Result(String,$T)) -> Result(String,$T))
  558 +
  559 + i.e. of type Q -> ((Q -> R) -> R) where Q is the type of the query function and R the type
  560 + 'Result(String,T)'. So, it takes two successive arguments. The first one of type Q is the query function
  561 + which is needed to have access to the database, and the second one of type Q -> R is the 'sequence' function.
  562 + What 'transaction' does is just:
  563 +
  564 + - send a 'BEGIN TRANSACTION' using the query function,
  565 + - apply the sequence function (of type Q -> R) to the query function (of type Q); this sends the
  566 + sequence of queries and yields a result ot type T,
  567 + - depending on the result of the previous step ('error' or 'ok'), send either a 'ROLLBACK' or a
  568 + 'COMMIT' to the database,
  569 + - return the appropriate result.
  570 +
477 571  
478 572 *** (8) Providing the 'query function'.
479 573  
... ... @@ -490,7 +584,7 @@ public define One
490 584 Hence, you must construct it using the arrow |-> within a context where this connection is visible.
491 585 For example, you must have a function looking like this one:
492 586  
493   - define Maybe((String) -> MetaSQL_Answer)
  587 + define Maybe(String -> MetaSQL_Answer)
494 588 make_query_function
495 589 (
496 590 ... // arguments needed for opening the connection to the database
... ... @@ -511,25 +605,26 @@ public define One
511 605 })
512 606 }.
513 607  
514   - In this example, the functions 'open_db_connection', 'send_query', 'read_answer' must be programmed
515   - by yourself.
  608 + In this example, the functions 'open_db_connection', 'send_query' and 'read_answer' are of your own.
516 609  
517   - See the file 'metaSQL_postgres.anubis' for an actual working example.
518   -
519   -
520   -
521   - *** (9) Considerations relative to the use of metaSQL in a web site.
522   -
  610 + Note: the same query function can be used by several Anubis virtual machines. Indeed, the functions
  611 + generated by this metaprogram (and the function 'transaction' defined in 'metaSQL_tool.anubis') use
  612 + the Anubis 'protect' mecanism in order to ensure that the sequence consisting in sending a request
  613 + and getting the answer cannot be mixed with another such sequence. Hence, if your program isa web site
  614 + for example, you can create the query function when the program starts and use it until the program
  615 + stops.
523 616  
  617 + See the file 'metaSQL_postgres.anubis' for an actual working example of how to construct a query
  618 + function.
524 619  
525   -
526 620  
527   - *** (10) Verifications performed by this metaprogram.
  621 +
  622 + *** (9) Verifications performed by this metaprogram.
528 623  
529 624 This metaprogram check that all the conditions below are fulfilled. If it is not the case,
530 625 error messages are sent (and the target Anubis file is not generated).
531 626  
532   - *** (10.1) Cycles of references in the database.
  627 + *** (9.1) Cycles of references in the database.
533 628  
534 629 The foreign(_) data type implements references between tables. These references may create loops
535 630 (or cycles), including loops of length 1 (self references) in the case the table refers to itself
... ... @@ -542,56 +637,161 @@ public define One
542 637 row in this table. Indeed, since the reference is not null, it must be provided as an argument
543 638 which is impossible when there is still no row at all in the table.
544 639  
  640 + Actually this rule ensures that the references between your tables are well-founded.
545 641  
546 642  
547   - *** (10.2) Values provided to _INSERT.
  643 + *** (9.2) Values provided to _INSERT.
548 644  
549 645 An INSERT command must provide values for all non null columns. Thus, the list of column names
550 646 must contain the names of all non null columns. Furthermore, the names must be in the same order
551   - as in the description of the table, and must not appear twice in the list.
  647 + as in the description of the table, and must not appear twice in the list. The name 'id' must
  648 + also not appear in the list since the database itself generates the value for this column.
552 649  
553 650  
  651 + *** (9.3) Coherence of WHERE clauses.
  652 +
  653 + WHERE clauses must satisfy some (obvious) conditions, such as the fact that comparisons must apply
  654 + to type for which they are meaningful. Also the 'is_null' and 'is_not_null' tests can only apply
  655 + to nullable columns.
554 656  
555 657  
556   - --- That's all for the public part !----------------------------------------------------------------
  658 + *** (10) Creating the database and modifying its structure.
557 659  
  660 + Since your program contains a detailed description of your database, it is possible to automate
  661 + the creation of the database and the modifications of its structure. Use the function 'database_check'
  662 + for this purpose. It should be executed each time you restart your program. This function, which
  663 + is defined in 'library/data_base/metaSQL_tools.anubis', takes the following arguments:
558 664  
559   - *** [] Tools.
560   - *** [] Formatting error messages.
561   - *** [] Finding a table by its name.
562   - *** [] _SELECT with only one table.
  665 + - a query function (in order to access the database),
  666 + - the description of the database.
563 667  
564   - *** [] Verifications concerning the description of the database and queries.
565   - *** [] Forbidden cycles in database references (also checks for duplicate table names).
566   - *** [] Checking for missing arguments in INSERT.
567   - *** [] Launching all verifications.
  668 + It returns a result of type 'Result(List(String),One)'. In other words, if something fails,
  669 + you get a list of error messages, otherwise execution is transparent.
568 670  
569   - *** [2] Generating the target file.
570   - *** [2.1] Tools.
571   - *** [] Converting a database type into the corresponding Anubis type:
572   - ***[] Generating the names of conversion functions.
573   - *** [] Getting the column structure from a column notation
574   - *** [2.2] Formatting a variable in a WHERE clause.
575   - *** [] Formatting a WHERE clause.
576   - *** [] Gathering generated function's arguments comming from WHERE clauses.
577   - *** [] Formatting generated function's arguments comming from WHERE clauses.
  671 + 'database_check' first of all checks the coherence of your database description, i.e. it
  672 + executes the verifications explained in section (9).
578 673  
579   - *** [] Carrying on.
580   - *** [] Generating 'INSERT' queries.
581   - *** [] Generating SELECT queries.
582   - *** [] Generating UPDATE queries.
583   - *** [] Generating DELETE queries.
584   - *** [] Making all sorts of query functions.
585   - *** [] The 'make_queries' function.
  674 + If the database does not exist, 'database_check' creates it. It also creates, within the
  675 + database, a special table (whose name is 'MetaSQL_Description') with a column of type binary.
  676 + In this table, 'database_check' records the structure of the database, in other words the serialization
  677 + of your database description. If you stop your program and change the description of the database,
  678 + 'database_check' will be able to compare the current description with the previous one. As a
  679 + consequence, it will be able to modify the structure of the database. Such modifications may involve:
  680 +
  681 + - creating or deleting a table,
  682 + - creating or deleting a column in a table,
  683 + - modifying the type of a column,
  684 + - creating or deleting an index.
  685 +
  686 + Of course, the previous structure of the database and data in deleted tables and columns are lost
  687 + during these operations. During these operations, 'database_check' locks the database (if possible).
  688 + After these operations are completed, 'database_check' updates the content of the special table
  689 + 'MetaSQL_Description'.
  690 +
  691 + Note: It is possible to change the structure of the database without stopping your program. Indeed,
  692 + instead of including the description of the database within your source files, store a serialization
  693 + of it into a file. This file can be read and unserialized (on demand) from within your running program.
  694 + At that moment, a 'database_check' could be executed, and the database will be modified. If your program
  695 + is a web site, it is likely that you will have to forbid normal client connections during that time.
  696 +
  697 +
  698 +
  699 + *** (11) Database administration through the web.
  700 +
  701 + The file 'library/data_base/metaSQL_admin.anubis' contains a function 'web_admin' of type:
  702 +
  703 + (One -> String -> MetaSQL_Answer) -> Web_Site
  704 +
  705 + (where the type 'Web_Site' is described in 'library/web/making_a_web_site.anubis'). You can
  706 + include this web site in your program. It is suitable for administering your database. The argument
  707 + of type 'One -> String -> MetaSQL_Answer' (i.e. of type 'One -> (String -> MetaSQL_Answer)') must
  708 + be a function constructing a query function each time it is applied to 'unique'. When this web site
  709 + is started the first time, a strong password for the administrator is asked for on the command line.
  710 + A hash of this password is stored into the table 'MetaSQL_Description'.
  711 +
  712 +
  713 +
586 714  
587 715  
  716 + --- That's all for the public part !----------------------------------------------------------------
  717 +
  718 +
  719 +
  720 + *** [1] Tools.
  721 + *** [1.1] Concatening strings which may not exist.
  722 + *** [1.2] A test for knowing if a database type is 'nullable'.
  723 + *** [1.3] Formatting error messages sent by this metaprogram.
  724 + *** [1.4] Formatting error messages to be sent by the generated program.
  725 + *** [1.5] Finding a table by its name.
  726 + *** [1.6] Finding a column.
  727 + *** [1.7] _SELECT with only one table.
  728 + *** [2] Verifications concerning the description of the database and queries.
  729 + *** [2.1] Forbidden cycles in database references (also checks for duplicate table names).
  730 + *** [2.2] Checking for missing arguments in INSERT.
  731 + *** [2.3] ???
  732 + *** [2.4] Launching all verifications.
  733 + *** [3] Generating the target file.
  734 + *** [3.1] Converting a database type into the corresponding Anubis type:
  735 + *** [3.2] Getting the column structure from a column notation
  736 + *** [3.3] Generating the names of conversion functions.
  737 + *** [3.4] Formatting a variable in a WHERE clause.
  738 + *** [3.5] Formatting a WHERE clause.
  739 + *** [3.6] Gathering generated function's arguments comming from WHERE clauses.
  740 + *** [3.7] Formatting generated function's arguments comming from WHERE clauses.
  741 + *** [4] Carrying on.
  742 + *** [4.1] Generating 'INSERT' queries.
  743 + *** [4.2] Generating SELECT queries.
  744 + *** [4.3] Generating UPDATE queries.
  745 + *** [4.4] Generating DELETE queries.
  746 + *** [4.5] Generating all sorts of query functions.
  747 + *** [4.6] The 'make_queries' function.
  748 +
588 749 read tools/basis.anubis
589 750 read system/string.anubis
  751 +
590 752  
591   - read data_base/metaSQL_tools.anubis
  753 + *** [1] Tools.
  754 +
  755 + *** [1.1] Concatening strings which may not exist.
  756 +
  757 + The concatenation function '+' for strings is defined in 'tools/basis.anubis'. Here we define
  758 + an extension of it to strings which may not exist (i.e. data of type 'Maybe(String)'). Of course,
  759 + the result is always of type 'Maybe(String)'.
  760 +
  761 +define Maybe(String)
  762 + Maybe(String) s + String t
  763 + =
  764 + if s is
  765 + {
  766 + failure then failure,
  767 + success(s1) then success(s1+t)
  768 + }.
592 769  
  770 +define Maybe(String)
  771 + String s + Maybe(String) t
  772 + =
  773 + if t is
  774 + {
  775 + failure then failure,
  776 + success(t1) then success(s+t1)
  777 + }.
593 778  
594   - *** [] Tools.
  779 +define Maybe(String)
  780 + Maybe(String) s + Maybe(String) t
  781 + =
  782 + if s is
  783 + {
  784 + failure then failure,
  785 + success(s1) then if t is
  786 + {
  787 + failure then failure,
  788 + success(t1) then success(s1+t1)
  789 + }
  790 + }.
  791 +
  792 +
  793 +
  794 + *** [1.2] A test for knowing if a database type is 'nullable'.
595 795  
596 796 define Bool
597 797 nullable
... ... @@ -621,7 +821,7 @@ define Bool
621 821 }.
622 822  
623 823  
624   - *** [] Formatting error messages sent by this metaprogram.
  824 + *** [1.3] Formatting error messages sent by this metaprogram.
625 825  
626 826 define String
627 827 error
... ... @@ -631,7 +831,7 @@ define String
631 831 "\n* * * Error: "+msg+"\n".
632 832  
633 833  
634   - *** [] Formatting error messages to be sent by the generated program.
  834 + *** [1.4] Formatting error messages to be sent by the generated program.
635 835  
636 836 define String
637 837 make_error
... ... @@ -641,9 +841,9 @@ define String
641 841 "error(\""+msg+" in \"+__FILE__+\":\"+to_decimal(__LINE__))".
642 842  
643 843  
644   - *** [] Finding a table by its name.
  844 + *** [1.5] Finding a table by its name.
645 845  
646   -define Maybe(MetaSQL_TabStruct)
  846 +public define Maybe(MetaSQL_TabStruct)
647 847 find_table
648 848 (
649 849 String table_name,
... ... @@ -660,8 +860,13 @@ define Maybe(MetaSQL_TabStruct)
660 860 }.
661 861  
662 862  
663   -define Maybe(MetaSQL_ColStruct)
664   - find_column_aux
  863 +
  864 + *** [1.6] Finding a column.
  865 +
  866 + The next function is public because used in 'web/metaWEB_Form.anubis'.
  867 +
  868 +public define Maybe(MetaSQL_ColStruct)
  869 + find_column
665 870 (
666 871 String col_name,
667 872 String table_name,
... ... @@ -673,7 +878,7 @@ define Maybe(MetaSQL_ColStruct)
673 878 [c . others] then
674 879 if name(c) = col_name
675 880 then success(c)
676   - else find_column_aux(col_name,table_name,others)
  881 + else find_column(col_name,table_name,others)
677 882 }.
678 883  
679 884  
... ... @@ -686,14 +891,19 @@ define Maybe(MetaSQL_ColStruct)
686 891 if tab is table(name,pk,cols) then
687 892 if pk is
688 893 {
689   - with_pk then if col_name = "id"
690   - then success(col("id",foreign(name),[]))
691   - else find_column_aux(col_name,name,cols),
692   - no_pk then find_column_aux(col_name,name,cols)
  894 + with_pk_32 then if col_name = "id"
  895 + then success(col("id",foreign(name),[]))
  896 + else find_column(col_name,name,cols),
  897 + with_pk_64 then if col_name = "id"
  898 + then success(col("id",foreign(name),[]))
  899 + else find_column(col_name,name,cols),
  900 + no_pk then find_column(col_name,name,cols)
693 901 }.
  902 +
  903 +
694 904  
695   - *** [] _SELECT with only one table.
696   -
  905 + *** [1.7] _SELECT with only one table.
  906 +
697 907 'condition' must be converted from MetaSQL_Where1 to MetaSQL_Where.
698 908  
699 909 define MetaSQL_Where
... ... @@ -735,17 +945,30 @@ public define MetaSQL_DML
735 945 convert(table_name,condition),
736 946 range).
737 947  
  948 + The same one without the 'range' argument:
738 949  
  950 +public define MetaSQL_DML
  951 + _SELECT
  952 + (
  953 + String query_name,
  954 + String output_type_name,
  955 + String table_name,
  956 + List(String) columns_names,
  957 + MetaSQL_Where1 condition
  958 + ) =
  959 + _SELECT(query_name,
  960 + output_type_name,
  961 + map((String c) |-> table_name.c ,columns_names),
  962 + convert(table_name,condition),
  963 + all).
739 964  
740 965  
741   -
  966 +
742 967  
743 968  
744   - *** [] Verifications concerning the description of the database and queries.
745   -
  969 + *** [2] Verifications concerning the description of the database and queries.
746 970  
747   -
748   - *** [] Forbidden cycles in database references (also checks for duplicate table names).
  971 + *** [2.1] Forbidden cycles in database references (also checks for duplicate table names).
749 972  
750 973 define Maybe(String)
751 974 refers_forwards
... ... @@ -829,7 +1052,7 @@ define Bool // returns &#39;true&#39; if there is at least one forbidden cycle.
829 1052  
830 1053  
831 1054  
832   - *** [] Checking for missing arguments in INSERT.
  1055 + *** [2.2] Checking for missing arguments in INSERT.
833 1056  
834 1057 define (List(String) missing_names,
835 1058 List(String) non_existing_names)
... ... @@ -872,7 +1095,12 @@ define Bool
872 1095 }.
873 1096  
874 1097  
875   - *** [] Launching all verifications.
  1098 + *** [2.3] Any table must have at least one ordinary column.
  1099 +
  1100 + *** [2.4]
  1101 +
  1102 +
  1103 + *** [2.?] Launching all verifications.
876 1104  
877 1105 public define Bool
878 1106 check_description
... ... @@ -888,48 +1116,11 @@ public define Bool
888 1116  
889 1117  
890 1118  
891   - *** [2] Generating the target file.
892   -
893   -
894   - *** [2.1] Tools.
895   -
896   -define Maybe(String)
897   - Maybe(String) s + String t
898   - =
899   - if s is
900   - {
901   - failure then failure,
902   - success(s1) then success(s1+t)
903   - }.
904   -
905   -define Maybe(String)
906   - String s + Maybe(String) t
907   - =
908   - if t is
909   - {
910   - failure then failure,
911   - success(t1) then success(s+t1)
912   - }.
913   -
914   -define Maybe(String)
915   - Maybe(String) s + Maybe(String) t
916   - =
917   - if s is
918   - {
919   - failure then failure,
920   - success(s1) then if t is
921   - {
922   - failure then failure,
923   - success(t1) then success(s1+t1)
924   - }
925   - }.
926   -
927   -
928   -
  1119 + *** [3] Generating the target file.
929 1120  
930   - *** [] Converting a database type into the corresponding Anubis type:
  1121 + *** [3.1] Converting a database type into the corresponding Anubis type:
931 1122  
932   -define String
  1123 +public define String // used in 'web/metaWEB_Form.anubis'
933 1124 to_Anubis
934 1125 (
935 1126 MetaSQL_Type typ
... ... @@ -957,7 +1148,7 @@ define String
957 1148 }.
958 1149  
959 1150  
960   - *** [] Getting the column structure from a column notation
  1151 + *** [3.2] Getting the column structure from a column notation
961 1152  
962 1153 define Maybe(MetaSQL_ColStruct)
963 1154 col_struct
... ... @@ -973,7 +1164,7 @@ define Maybe(MetaSQL_ColStruct)
973 1164 }.
974 1165  
975 1166  
976   - ***[] Generating the names of conversion functions.
  1167 + *** [3.3] Generating the names of conversion functions.
977 1168  
978 1169 type ConversionName:
979 1170 convname(Bool is_nullable,
... ... @@ -1020,7 +1211,7 @@ define Maybe(ConversionName)
1020 1211 }.
1021 1212  
1022 1213  
1023   - *** [2.2] Formatting a variable in a WHERE clause.
  1214 + *** [3.4] Formatting a variable in a WHERE clause.
1024 1215  
1025 1216 A variable (of type MetaSQL_Var) occurs in WHERE clauses, where it has to be replaced by an
1026 1217 actual value. This value has to be formatted into a character string. However, each variable
... ... @@ -1107,7 +1298,7 @@ define Maybe(String)
1107 1298 if c is table.column then format_variable(name(v),table,column,db).
1108 1299  
1109 1300  
1110   - *** [] Formatting a WHERE clause.
  1301 + *** [3.5] Formatting a WHERE clause.
1111 1302  
1112 1303 For WHERE clauses, we need to adapt format_variable.
1113 1304  
... ... @@ -1193,11 +1384,12 @@ define Maybe(String)
1193 1384 }.
1194 1385  
1195 1386  
1196   - *** [] Gathering generated function's arguments comming from WHERE clauses.
  1387 + *** [3.6] Gathering generated function's arguments comming from WHERE clauses.
1197 1388  
1198 1389 type Function_Arg:
1199 1390 arg(MetaSQL_Type type,
1200   - String name).
  1391 + String name,
  1392 + String comment).
1201 1393  
1202 1394 define Maybe(List($T))
1203 1395 merge
... ... @@ -1225,7 +1417,7 @@ define Maybe(List(Function_Arg))
1225 1417 if col_struct(c,db) is
1226 1418 {
1227 1419 failure then failure,
1228   - success(cs) then success([arg(type(cs),name(v))])
  1420 + success(cs) then success([arg(type(cs),name(v),name(cs))])
1229 1421 }.
1230 1422  
1231 1423 define Maybe(List(Function_Arg))
... ... @@ -1251,7 +1443,8 @@ define Maybe(List(Function_Arg))
1251 1443 ~ w1 then gather_where_args(w1,db)
1252 1444 }.
1253 1445  
1254   - *** [] Formatting generated function's arguments comming from WHERE clauses.
  1446 +
  1447 + *** [3.7] Formatting generated function's arguments comming from WHERE clauses.
1255 1448  
1256 1449 define Maybe(String)
1257 1450 where_args_dec
... ... @@ -1267,7 +1460,7 @@ define Maybe(String)
1267 1460 {
1268 1461 [ ] then success("One dummy"),
1269 1462 [_ . _] then success(concat(
1270   - map((Function_Arg a) |-> if a is arg(type,name) then to_Anubis(type)+" "+name,
  1463 + map((Function_Arg a) |-> if a is arg(type,name,com) then to_Anubis(type)+" "+name,
1271 1464 args),
1272 1465 ",\n"+constant_string(spaces,' ')))
1273 1466 }
... ... @@ -1276,17 +1469,16 @@ define Maybe(String)
1276 1469  
1277 1470  
1278 1471  
1279   - *** [] Carrying on.
1280   -
  1472 + *** [4] Carrying on.
1281 1473  
1282   - *** [] Generating 'INSERT' queries.
  1474 + *** [4.1] Generating 'INSERT' queries.
1283 1475  
1284 1476 define String
1285 1477 format_value
1286 1478 (
1287 1479 Function_Arg a
1288 1480 ) =
1289   - if a is arg(type,name) then format_variable(name,type).
  1481 + if a is arg(type,name,_) then format_variable(name,type).
1290 1482  
1291 1483 define Maybe(List(Function_Arg))
1292 1484 function_args
... ... @@ -1305,7 +1497,7 @@ define Maybe(List(Function_Arg))
1305 1497 if function_args(tab,others) is
1306 1498 {
1307 1499 failure then failure,
1308   - success(ot) then success([arg(type,name) . ot])
  1500 + success(ot) then success([arg(type,name,name) . ot])
1309 1501 }
1310 1502 }
1311 1503 }.
... ... @@ -1329,7 +1521,7 @@ define String
1329 1521 List(Function_Arg) args,
1330 1522 Int spaces,
1331 1523 ) =
1332   - concat(map((Function_Arg a) |-> if a is arg(t,n) then to_Anubis(t)+" "+n,args),
  1524 + concat(map((Function_Arg a) |-> if a is arg(t,n,c) then to_Anubis(t)+" "+n,args),
1333 1525 ",\n"+constant_string(spaces,' ')).
1334 1526  
1335 1527  
... ... @@ -1358,7 +1550,7 @@ define Maybe(String)
1358 1550 then "."
1359 1551 else " =\n"+
1360 1552 " ("+function_args_dec(args,4)+") |-> \n"+
1361   - " if query_function(\n"+
  1553 + " send_protected(false,(String -> MetaSQL_Answer qf) |-> if qf(\n"+
1362 1554 " \"INSERT INTO "+table_name+" ("+
1363 1555 concat(column_names,",")+")\n VALUES ("+
1364 1556 concat(map((Function_Arg a) |-> "'\"+"+format_value(a)+"+\"'",args),
... ... @@ -1368,14 +1560,14 @@ define Maybe(String)
1368 1560 " command_complete then ok(unique),\n"+
1369 1561 " table(rows) then "+make_error("Unexpected answer")+",\n"+
1370 1562 " informations(info) then "+make_error("Unexpected answer")+"\n"+
1371   - " }.\n"
  1563 + " },query_function).\n"
1372 1564 )+
1373 1565 "\n\n")
1374 1566 }.
1375 1567  
1376 1568  
1377 1569  
1378   - *** [] Generating SELECT queries.
  1570 + *** [4.2] Generating SELECT queries.
1379 1571  
1380 1572 The condition may imply that there must be one and exactly one row in the result.
1381 1573 This is based on the fact that the value of a primary key should be always
... ... @@ -1431,7 +1623,8 @@ define Maybe(List(Function_Arg))
1431 1623 select_components
1432 1624 (
1433 1625 List(MetaSQL_Column) col_names,
1434   - MetaSQL_DBStruct db
  1626 + MetaSQL_DBStruct db,
  1627 + Int i
1435 1628 ) =
1436 1629 if col_names is
1437 1630 {
... ... @@ -1444,26 +1637,26 @@ define Maybe(List(Function_Arg))
1444 1637 {
1445 1638 failure then failure,
1446 1639 success(cs) then if cs is col(cname,ctype,attr) then
1447   - if select_components(others,db) is
  1640 + if select_components(others,db,i+1) is
1448 1641 {
1449 1642 failure then failure,
1450   - success(oc) then success([arg(ctype,cname) . oc])
  1643 + success(oc) then success([arg(ctype,"_"+to_decimal(i),cname) . oc])
1451 1644 }
1452 1645 }
1453 1646 }
1454 1647 }.
1455 1648  
1456   -define Maybe(String)
  1649 + define Maybe(String)
1457 1650 select_components
1458 1651 (
1459 1652 List(MetaSQL_Column) col_names,
1460 1653 MetaSQL_DBStruct db
1461 1654 ) =
1462   - if (Maybe(List(Function_Arg)))select_components(col_names,db) is
  1655 + if (Maybe(List(Function_Arg)))select_components(col_names,db,0) is
1463 1656 {
1464 1657 failure then failure,
1465 1658 success(comps) then success(
1466   - concat(map((Function_Arg a) |-> if a is arg(t,n) then to_Anubis(t)+" "+n,
  1659 + concat(map((Function_Arg a) |-> if a is arg(t,n,c) then to_Anubis(t)+" "+n+" /* "+c+" */",
1467 1660 comps),
1468 1661 ",\n "))
1469 1662 }.
... ... @@ -1514,7 +1707,17 @@ define Maybe(String)
1514 1707 }.
1515 1708  
1516 1709  
1517   -
  1710 +define List(String)
  1711 + collect_table_names
  1712 + (
  1713 + List(MetaSQL_Column) l
  1714 + ) =
  1715 + if l is
  1716 + {
  1717 + [ ] then [ ],
  1718 + [h . t] then with others = collect_table_names(t),
  1719 + if h is tn.cn then if tn:others then others else [tn . others]
  1720 + }.
1518 1721  
1519 1722 define Maybe(String)
1520 1723 make_SELECT_func
... ... @@ -1570,12 +1773,14 @@ define Maybe(String)
1570 1773 (if declare_only
1571 1774 then success(".")
1572 1775 else " =\n"+
1573   - " ("+where_args_dec(condition,4,db)+") |-> if query_function(\n"+
1574   - " \"SELECT ("+
  1776 + " ("+where_args_dec(condition,4,db)+") |->\n"+
  1777 + " send_protected(false,(String -> MetaSQL_Answer qf) |-> if qf(\n"+
  1778 + " \"SELECT "+
1575 1779 concat(map((MetaSQL_Column c) |-> if c is tn.cn then tn+"."+cn,
1576 1780 col_names),
1577   - ",\n ")+
1578   - ")\n WHERE "+
  1781 + ",\n ")+"\n"+
  1782 + " FROM "+concat(collect_table_names(col_names),",")+" \n"+
  1783 + " WHERE "+
1579 1784 format(condition,db)+";\") is\n"+
1580 1785 " {\n"+
1581 1786 " error(msg) then error(msg),\n"+
... ... @@ -1609,12 +1814,12 @@ define Maybe(String)
1609 1814 else " map_escape("+query_name+"_aux,rows)\n"
1610 1815 )+
1611 1816 " informations(info) then "+make_error("Unexpected answer")+"\n"+
1612   - " }.\n\n").
  1817 + " },query_function).\n\n").
1613 1818  
1614 1819  
1615 1820  
1616 1821  
1617   - *** [] Generating UPDATE queries.
  1822 + *** [4.3] Generating UPDATE queries.
1618 1823  
1619 1824 define Maybe(String)
1620 1825 make_UPDATE_func
... ... @@ -1629,7 +1834,7 @@ define Maybe(String)
1629 1834  
1630 1835  
1631 1836  
1632   - *** [] Generating DELETE queries.
  1837 + *** [4.4] Generating DELETE queries.
1633 1838  
1634 1839 define Maybe(String)
1635 1840 make_DELETE_func
... ... @@ -1653,7 +1858,7 @@ define Maybe(String)
1653 1858 then success(".")
1654 1859 else " =\n"+
1655 1860 " ("+where_args_dec(cond,4,db)+") |-> \n"+
1656   - " if query_function(\n"+
  1861 + " send_protected(false,(String -> MetaSQL_Answer qf) |-> if qf(\n"+
1657 1862 " \"DELETE FROM "+table_name+"\n WHERE "+
1658 1863 format(cond,db)+";\") is\n"+
1659 1864 " {\n"+
... ... @@ -1661,28 +1866,73 @@ define Maybe(String)
1661 1866 " command_complete then ok(unique),\n"+
1662 1867 " table(rows) then "+make_error("Unexpected answer")+",\n"+
1663 1868 " informations(info) then "+make_error("Unexpected answer")+"\n"+
1664   - " }.\n")+
  1869 + " },query_function).\n")+
1665 1870 "\n\n".
1666 1871  
1667 1872  
1668 1873  
1669   - *** [] Making all sorts of query functions.
  1874 + *** [4.5] Generating all sorts of query functions.
  1875 +
  1876 + A test for knowing if a given type definition has already been written to the target file.
  1877 +
  1878 +type TypeSeen:
  1879 + type_seen
  1880 + (String name,
  1881 + List(Function_Arg) components).
  1882 +
  1883 +define Bool
  1884 + compare
  1885 + (
  1886 + Function_Arg a1,
  1887 + Function_Arg a2
  1888 + ) =
  1889 + if a1 is arg(n1,c1,_) then
  1890 + if a2 is arg(n2,c2,_) then
  1891 + (n1 = n2) & (c1 = c2).
1670 1892  
1671 1893 define Bool
  1894 + compare
  1895 + (
  1896 + List(Function_Arg) a1,
  1897 + List(Function_Arg) a2
  1898 + ) =
  1899 + if a1 is
  1900 + {
  1901 + [ ] then a2 = [ ],
  1902 + [h1 . t1] then if a2 is
  1903 + {
  1904 + [ ] then false,
  1905 + [h2 . t2] then
  1906 + if compare(h1,h2)
  1907 + then compare(t1,t2)
  1908 + else false
  1909 + }
  1910 + }.
  1911 +
  1912 +define Bool
  1913 + compare
  1914 + (
  1915 + TypeSeen t1,
  1916 + TypeSeen t2
  1917 + ) =
  1918 + if t1 is type_seen(n1,c1) then
  1919 + if t2 is type_seen(n2,c2) then
  1920 + (n1 = n2) & (compare(c1,c2)).
  1921 +
  1922 +
  1923 +define Bool
1672 1924 type_already_seen
1673 1925 (
1674   - String type_name,
1675   - String components,
1676   - List((String,String)) already_seen
  1926 + TypeSeen ts,
  1927 + List(TypeSeen) already_seen
1677 1928 ) =
1678 1929 if already_seen is
1679 1930 {
1680 1931 [ ] then false,
1681   - [h . t] then
1682   - if h = (type_name,components)
1683   - then true
1684   - else type_already_seen(type_name,components,t)
1685   - }.
  1932 + [h . t] then if compare(h,ts)
  1933 + then true
  1934 + else type_already_seen(ts,t)
  1935 + }.
1686 1936  
1687 1937  
1688 1938 define Maybe(String)
... ... @@ -1691,7 +1941,7 @@ define Maybe(String)
1691 1941 List(MetaSQL_DML) queries,
1692 1942 MetaSQL_DBStruct db,
1693 1943 Bool declare_only,
1694   - List((String,String)) types_so_far
  1944 + List(TypeSeen) types_so_far
1695 1945 ) =
1696 1946 if queries is
1697 1947 {
... ... @@ -1711,23 +1961,27 @@ define Maybe(String)
1711 1961 },
1712 1962  
1713 1963 _SELECT(query_name,out_type,col_names,condition,range) then
1714   - if (Maybe(String))select_components(col_names,db) is
  1964 + if (Maybe(List(Function_Arg)))select_components(col_names,db,0) is
1715 1965 {
1716 1966 failure then failure,
1717 1967 success(comps) then
  1968 + with scomps =
  1969 + concat(map((Function_Arg a) |-> if a is arg(t,n,c) then to_Anubis(t)+" "+n+" /* "+c+" */",
  1970 + comps),",\n "),
1718 1971 if make_SELECT_func(query_name,
1719 1972 out_type,
1720 1973 col_names,
1721 1974 condition,
1722   - comps,
  1975 + scomps,
1723 1976 range,
1724 1977 db,
1725 1978 declare_only,
1726   - type_already_seen(out_type,comps,types_so_far)) is
  1979 + type_already_seen(type_seen(out_type,comps),types_so_far)) is
1727 1980 {
1728 1981 failure then failure,
1729 1982 success(f) then
1730   - if make_query_functions(others,db,declare_only,[(out_type,comps) . types_so_far]) is
  1983 + if make_query_functions(others,db,declare_only,
  1984 + [type_seen(out_type,comps) . types_so_far]) is
1731 1985 {
1732 1986 failure then failure,
1733 1987 success(o) then success(f+o)
... ... @@ -1762,7 +2016,7 @@ define Maybe(String)
1762 2016  
1763 2017  
1764 2018  
1765   - *** [] The 'make_queries' function.
  2019 + *** [4.6] The 'make_queries' function.
1766 2020  
1767 2021 public define One
1768 2022 make_queries
... ... @@ -1788,6 +2042,7 @@ public define One
1788 2042 "\n This file was generated automatically by the metaprogram 'data_base/metaSQL.anubis'.\n"+
1789 2043 " Don't modify it. Your modifications would be crushed.\n\n"+
1790 2044 "read tools/basis.anubis\n"+
  2045 + "read data_base/metaSQL_common.anubis\n"+
1791 2046 "read data_base/metaSQL_tools.anubis\n"+
1792 2047 concat(map((String f) |-> "read "+f,files_to_be_read),"\n")+
1793 2048 "\n\n"+
... ... @@ -1801,9 +2056,13 @@ public define One
1801 2056  
1802 2057  
1803 2058  
1804   -
1805   -
1806   -
1807   -
1808   -
1809   -
  2059 +
  2060 +
  2061 + *** [5] Creating and checking the database.
  2062 +
  2063 +
  2064 +
  2065 +
  2066 +
  2067 +
  2068 +
... ...
anubis_dev/library/data_base/metaSQL_tools.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -5,31 +5,28 @@
5 5 (see metaSQL.anubis)
6 6  
7 7 read tools/basis.anubis
  8 +read tools/base64.anubis
8 9 read system/string.anubis
  10 +read data_base/metaSQL.anubis
  11 +read data_base/metaSQL_common.anubis
9 12  
10 13  
11 14 *** (1) Types.
12 15  
13   - *** (1.1) MetaSQL_Id.
  16 +
  17 + *** (1.1) 'MaybeNull($T)'.
  18 +
  19 + Defined in metaSQL_common.anubis.
  20 +
  21 +
  22 + *** (1.2) 'MetaSQL_Id'.
14 23  
15 24 A datum of this type is the identifier of a row in a table of the database.
16 25  
17 26 public type MetaSQL_Id:... (an opaque type)
18 27  
19 28  
20   - *** (1.2) MaybeNull.
21   -
22   - This type is isomorphic to 'Maybe', and is used for representing data which
23   - can have the value 'NULL'. We don't use 'Maybe' for this purpose because
24   - we want to avoid the misleading 'Maybe(Maybe(...))'. Instead, we will have
25   - sometimes 'Maybe(MaybeNull(...))', so that 'null' means 'NULL', and 'failure'
26   - means an error.
27   -
28   -public type MaybeNull($T):
29   - null,
30   - not_null($T value).
31   -
32   - *** (1.3) MetaSQL_Answer.
  29 + *** (1.3) 'MetaSQL_Answer'.
33 30  
34 31 The answer to a query as returned by a database software can be:
35 32  
... ... @@ -53,6 +50,22 @@ public type MetaSQL_Answer:
53 50 use in your Anubis program. However, you must use this type for providing the 'query function'
54 51 required by 'metaSQL.anubis'.
55 52  
  53 +
  54 +
  55 + *** [] Transactions.
  56 +
  57 + See the explanations in 'library/data_base/metaSQL.anubis'.
  58 +
  59 +public define ((String -> MetaSQL_Answer) -> Result(String,$T)) -> Result(String,$T)
  60 + metaSQL_transaction
  61 + (
  62 + String -> MetaSQL_Answer query_function
  63 + ).
  64 +
  65 +
  66 +
  67 +
  68 +
56 69  
57 70 *** (2) Conversion functions.
58 71  
... ... @@ -236,7 +249,6 @@ public define MaybeNull(String) // cannot produce an error
236 249  
237 250  
238 251  
239   -public define String unexpected_answer = "Unexpected database answer.".
240 252  
241 253  
242 254  
... ... @@ -246,16 +258,16 @@ public define String unexpected_answer = &quot;Unexpected database answer.&quot;.
246 258 --- That's all for the public part ! ------------------------------------------------
247 259  
248 260 public type MetaSQL_Id:
249   - id(Int value).
250   -
251   -
  261 + id(String value). // not an 'Int' because the database reads it as a string
  262 + // and returns it as a byte array. This is just an identifier.
  263 + // No computation is made on it.
252 264  
253 265 public define String
254 266 prepare_foreign
255 267 (
256 268 MetaSQL_Id i
257 269 ) =
258   - to_decimal(value(i)).
  270 + value(i).
259 271  
260 272 public define String
261 273 prepare_foreign_or_null
... ... @@ -265,14 +277,123 @@ public define String
265 277 if i is
266 278 {
267 279 null then "NULL",
268   - not_null(id) then to_decimal(value(id))
  280 + not_null(id) then value(id)
  281 + }.
  282 +
  283 +public define String
  284 + prepare_date
  285 + (
  286 + String d
  287 + ) = d.
  288 +
  289 +public define String
  290 + prepare_date_or_null
  291 + (
  292 + MaybeNull(String) b
  293 + ) =
  294 + if b is
  295 + {
  296 + null then "NULL",
  297 + not_null(d) then d
  298 + }.
  299 +
  300 +public define Maybe(String)
  301 + convert_to_date
  302 + (
  303 + MaybeNull(ByteArray) b
  304 + ) =
  305 + if b is
  306 + {
  307 + null then failure,
  308 + not_null(ba) then success(to_string(ba))
269 309 }.
270 310  
  311 +
  312 + In the future, if type schemes are seen as 'functors' (in the sens of category theory)
  313 + the function below could be written: MaybeNull(to_string)
  314 +
  315 +public define MaybeNull(String)
  316 + convert_to_date_or_null
  317 + (
  318 + MaybeNull(ByteArray) b
  319 + ) =
  320 + if b is
  321 + {
  322 + null then null,
  323 + not_null(ba) then not_null(to_string(ba))
  324 + }.
  325 +
  326 +public define String
  327 + prepare_boolean
  328 + (
  329 + Bool b
  330 + ) =
  331 + if b then "TRUE" else "FALSE".
  332 +
  333 +public define String
  334 + prepare_boolean_or_null
  335 + (
  336 + MaybeNull(Bool) b
  337 + ) =
  338 + if b is
  339 + {
  340 + null then "NULL",
  341 + not_null(bl) then if bl then "TRUE" else "FALSE"
  342 + }.
  343 +
  344 +
  345 +public define Maybe(Bool)
  346 + convert_to_boolean
  347 + (
  348 + MaybeNull(ByteArray) b
  349 + ) =
  350 + if b is
  351 + {
  352 + null then failure,
  353 + not_null(ba) then if nth(0,ba) is
  354 + {
  355 + failure then failure,
  356 + success(c) then if (c = 'T' | c = 't' | c = '1')
  357 + then success(true)
  358 + else if (c = 'F' | c = 'f' | c = '0')
  359 + then success(false)
  360 + else failure
  361 + }
  362 + }.
  363 +
  364 +
  365 +public define MaybeNull(Bool) convert_to_boolean_or_null (MaybeNull(ByteArray) b).
  366 +
  367 +
  368 +public define String
  369 + prepare_binary
  370 + (
  371 + ByteArray b
  372 + ) =
  373 + to_string(base64_encode(b)).
  374 +
  375 +public define String
  376 + prepare_binary_or_null
  377 + (
  378 + MaybeNull(ByteArray) b
  379 + ) =
  380 + if b is
  381 + {
  382 + null then "NULL",
  383 + not_null(ba) then to_string(base64_encode(ba))
  384 + }.
  385 +
  386 +
271 387 public define Maybe(MetaSQL_Id)
272 388 convert_to_foreign
273 389 (
274 390 MaybeNull(ByteArray) b
275   - ).
  391 + ) =
  392 + if b is
  393 + {
  394 + null then failure,
  395 + not_null(ba) then success(id(to_string(ba)))
  396 + }.
276 397  
277 398 public define MaybeNull(MetaSQL_Id)
278 399 convert_to_foreign_or_null
... ... @@ -354,3 +475,355 @@ public define Result($E,List($T))
354 475 }.
355 476  
356 477  
  478 +
  479 +public define Result(String,$T)
  480 + send_protected
  481 + (
  482 + Bool within_transaction,
  483 + (String -> MetaSQL_Answer) -> Result(String,$T) query,
  484 + String -> MetaSQL_Answer qf
  485 + ) =
  486 + protect // it is important that there is only one 'protect' for controling the communication
  487 + // with the database. If we had several 'protect' the terms they protect could be
  488 + // executed at the same time.
  489 + if within_transaction
  490 + then if qf("BEGIN;") is
  491 + {
  492 + error(msg) then error(msg),
  493 + command_complete then if query(qf) is
  494 + {
  495 + error(msg) then forget(qf("ROLLBACK;")); error(msg),
  496 + ok(res) then if qf("COMMIT;") is
  497 + {
  498 + error(msg) then error(msg),
  499 + command_complete then ok(res),
  500 + table(_) then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__)),
  501 + informations(_) then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__))
  502 + }
  503 + },
  504 + table(_) then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__)),
  505 + informations(_) then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__))
  506 + }
  507 + else query(qf).
  508 +
  509 +
  510 +public define ((String -> MetaSQL_Answer) -> Result(String,$T)) -> Result(String,$T)
  511 + transaction
  512 + (
  513 + String -> MetaSQL_Answer qf
  514 + ) =
  515 + ((String -> MetaSQL_Answer) -> Result(String,$T) query) |->
  516 + send_protected(true,query,qf).
  517 +
  518 +
  519 +
  520 + *** [] 'database_check'.
  521 +
  522 + This function proceeds as follows:
  523 +
  524 + (1) Try to recover the previous description of the database. If not possible,
  525 + this description is assumed to be empty (no table at all).
  526 + (2) Check the current description, and if correct, save it into the database.
  527 + (3) Compare the two descriptions, old and new, and
  528 + (3.1) create tables required by the new description, not present (by their
  529 + name) in the previous one,
  530 + (3.2) create columns required by the new description, not present in
  531 + the previous tables,
  532 + (3.3) update type of columns if required,
  533 + (3.4) update attributes of columns if required,
  534 + (3.5) create the indexes required by the new description and delete
  535 + any index required by the previous one but not by the new one.
  536 +
  537 +
  538 + *** [] Collecting actions to be performed.
  539 +
  540 + In order to prepare this work, it is interesting to record all operations into
  541 + a description before excuting them. This also allows to inform on what has to
  542 + be done.
  543 +
  544 +type MetaSQL_ExType: // 'extended' type (we need two more types for "id" column)
  545 + serial_32,
  546 + serial_64,
  547 + usual(MetaSQL_Type).
  548 +
  549 +type MetaSQL_DBChange:
  550 + create_special_table, // if the special table does not exist
  551 + create_table (MetaSQL_TabStruct), // table to be created
  552 + create_column (String table_name, // ordinary column to be created
  553 + MetaSQL_ColStruct),
  554 + create_pk_32_column (String table_name),
  555 + create_pk_64_column (String table_name),
  556 + set_type (String table_name, // change column type
  557 + String column_name,
  558 + MetaSQL_ExType new_type),
  559 + set_attrs (String table_name, // change column attributes
  560 + String column_name,
  561 + List(MetaSQL_Attr) old_attrs,
  562 + List(MetaSQL_Attr) new_attrs),
  563 + create_index (String table_name,
  564 + String column_name),
  565 + delete_table (String table_name),
  566 + delete_column (String table_name,
  567 + String column_name),
  568 + delete_index (String table_name,
  569 + String column_name).
  570 +
  571 +
  572 + The next function gets the name of a table and a list of tables. It extracts (if possible)
  573 + the table with the given name from the list. It returns a pair made of the extracted
  574 + table and the list of remaining tables.
  575 +
  576 +define Maybe((MetaSQL_TabStruct extracted,
  577 + List(MetaSQL_TabStruct) others))
  578 + extract_table
  579 + (
  580 + String table_name,
  581 + List(MetaSQL_TabStruct) l
  582 + ) =
  583 + if l is
  584 + {
  585 + [ ] then failure, // table not found
  586 + [h . t] then
  587 + if name(h) = table_name
  588 + then success((h,t))
  589 + else if extract_table(table_name,t) is
  590 + {
  591 + failure then failure,
  592 + success(p) then if p is (a,r) then success((a,[h . r]))
  593 + }
  594 + }.
  595 +
  596 +
  597 + The same one for a column.
  598 +
  599 +define Maybe((MetaSQL_ColStruct extracted,
  600 + List(MetaSQL_ColStruct) others))
  601 + extract_column
  602 + (
  603 + String column_name,
  604 + List(MetaSQL_ColStruct) l
  605 + ) =
  606 + if l is
  607 + {
  608 + [ ] then failure, // column not found
  609 + [h . t] then
  610 + if name(h) = column_name
  611 + then success((h,t))
  612 + else if extract_column(column_name,t) is
  613 + {
  614 + failure then failure,
  615 + success(p) then if p is (a,r) then success((a,[h . r]))
  616 + }
  617 + }.
  618 +
  619 +
  620 + The next function generates all the actions needed for deleting all the indexes
  621 + from a given table.
  622 +
  623 +define List(MetaSQL_DBChange)
  624 + make_delete_indexes
  625 + (
  626 + MetaSQL_TabStruct t
  627 + ) =
  628 + // delete all indexes from this table
  629 + if t is table(name,pk,cols) then
  630 + map_select((MetaSQL_ColStruct c) |-> if c is col(cname,ctype,attrs) then
  631 + if indexed:attrs
  632 + then success(delete_index(name,cname))
  633 + else failure,
  634 + cols).
  635 +
  636 +
  637 + The next function receives the name of a table, and two pk indicators.
  638 + It generates the actions needed.
  639 +
  640 +define List(MetaSQL_DBChange)
  641 + make_pk_changes
  642 + (
  643 + String table_name,
  644 + MetaSQL_PrimKey old_pk,
  645 + MetaSQL_PrimKey new_pk
  646 + ) =
  647 + if old_pk is
  648 + {
  649 + with_pk_32 then
  650 + if new_pk is
  651 + {
  652 + with_pk_32 then [ ],
  653 + with_pk_64 then [set_type(table_name,"id",serial_64)],
  654 + no_pk then [delete_column(table_name,"id")]
  655 + // this implies deleting refering tables, but such tables cannot
  656 + // be in the new description, hence will be deleted anyway
  657 + },
  658 + with_pk_64 then
  659 + if new_pk is
  660 + {
  661 + with_pk_32 then [set_type(table_name,"id",serial_32)],
  662 + with_pk_64 then [ ],
  663 + no_pk then [delete_column(table_name,"id")]
  664 + // same remark as above
  665 + },
  666 + no_pk then
  667 + if new_pk is
  668 + {
  669 + with_pk_32 then [create_pk_32_column(table_name)],
  670 + with_pk_64 then [create_pk_64_column(table_name)],
  671 + no_pk then [ ]
  672 + }
  673 + }.
  674 +
  675 +
  676 + The next function receives the name of a table and two descriptions of its columns
  677 + (the old one and the new one). It generates the required actions.
  678 +
  679 +define List(MetaSQL_DBChange)
  680 + make_col_changes
  681 + (
  682 + String table_name,
  683 + List(MetaSQL_ColStruct) old_cols,
  684 + List(MetaSQL_ColStruct) new_cols,
  685 + List(MetaSQL_DBChange) so_far
  686 + ) =
  687 + if old_cols is
  688 + {
  689 + [ ] then reverse(map((MetaSQL_ColStruct c) |-> create_column(table_name,c),
  690 + new_cols) + so_far),
  691 + [old1 . other_old] then if old1 is col(cname,old_type,old_attrs) then
  692 + if extract_column(cname,new_cols) is
  693 + {
  694 + failure then make_col_changes(table_name,other_old,new_cols,
  695 + [delete_column(table_name,cname) . so_far]),
  696 + success(p) then if p is (new1,other_new) then
  697 + // the column must be updated
  698 + if new1 is col(_,new_type,new_attrs) then
  699 + make_col_changes(table_name,other_old,other_new,
  700 + (if old_type = new_type then [] else [set_type(table_name,cname,usual(new_type))]) +
  701 + [set_attrs(table_name,cname,old_attrs,new_attrs)] +
  702 + so_far)
  703 + }
  704 + }.
  705 +
  706 +
  707 +
  708 + The next function takes two database descriptions (the old one and the new one),
  709 + and establishes the list of all actions to be performed.
  710 +
  711 +define List(MetaSQL_DBChange)
  712 + db_actions
  713 + (
  714 + List(MetaSQL_TabStruct) old_tables, // tables for which all operations are already collected
  715 + List(MetaSQL_TabStruct) new_tables, // are not present in these two lists
  716 + List(MetaSQL_DBChange) so_far // changes collected so far
  717 + ) =
  718 + if old_tables is
  719 + {
  720 + [ ] then
  721 + // no more old tables: we just have to create remaining new tables
  722 + reverse(map(create_table,new_tables) + so_far),
  723 + [old1 . other_old] then
  724 + if old1 is table(old_tname,old_pk,old_cols) then
  725 + if extract_table(old_tname,new_tables) is
  726 + {
  727 + failure then
  728 + // old1 not found among new tables
  729 + db_actions(other_old,new_tables,
  730 + make_delete_indexes(old1) + [delete_table(old_tname) . so_far]),
  731 + success(p) then if p is (new1,other_new) then
  732 + // here, the tables old1 and new1 have the same name, hence the
  733 + // table must be updated
  734 + if new1 is table(_,new_pk,new_cols) then
  735 + db_actions(other_old,other_new,
  736 + make_pk_changes(old_tname,old_pk,new_pk) +
  737 + make_col_changes(old_tname,old_cols,new_cols,[]) + so_far)
  738 + }
  739 + }.
  740 +
  741 +
  742 +
  743 + *** [] Formatting actions.
  744 +
  745 +
  746 + define String
  747 + format
  748 + (
  749 + MetaSQL_DBChange c
  750 + ) =
  751 + if c is
  752 + {
  753 + create_special_table then
  754 + "CREATE TABLE MetaSQL_Description (previous vartext, pwh text(260));",
  755 +
  756 + create_table(tab) then
  757 + if tab is table(name,pk,columns) then
  758 + "CREATE TABLE "+name+" ("+
  759 + if pk is
  760 + {
  761 + with_pk_32 then "id serial NOT NULL UNIQUE PRIMARY KEY, ",
  762 + with_pk_64 then "id bigserial NOT NULL UNIQUE PRIMARY KEY, ",
  763 + no_pk then ""
  764 + } + concat(map(format_column_def,columns),", ") + ");",
  765 +
  766 + create_column(table_name,c) then
  767 + if c is col(cname,ctype,attrs) then
  768 +
  769 + create_pk_32_column(table_name) then
  770 + create_pk_64_column(table_name) then
  771 + set_type(table_name,column_name,new_type) then
  772 + set_attrs(table_name,column_name,old_attrs,new_attrs) then
  773 + create_index(table_name,column_name) then
  774 + delete_table(table_name) then
  775 + delete_column(table_name,column_name) then
  776 + delete_index(table_name,column_name) then
  777 + }.
  778 +
  779 +
  780 +read tools/base64.anubis
  781 +
  782 +define Maybe(MetaSQL_DBStruct)
  783 + extract_description
  784 + (
  785 + List(List(MaybeNull(ByteArray))) rows
  786 + ) =
  787 + if rows is
  788 + {
  789 + [ ] then failure,
  790 + [h . t] then
  791 + if h is
  792 + {
  793 + [ ] then failure,
  794 + [d . _] then if d is
  795 + {
  796 + null then failure,
  797 + not_null(e) then (Maybe(MetaSQL_DBStruct))unserialize(base64_decode(e))
  798 + }
  799 + }
  800 + }.
  801 +
  802 +
  803 + public define Result(List(String),One)
  804 + database_check
  805 + (
  806 + String -> MetaSQL_Answer qf,
  807 + MetaSQL_DBStruct db
  808 + ) =
  809 + //
  810 + // get the previous description of the database
  811 + //
  812 + if qf("SELECT * FROM MetaSQL_Description;") is
  813 + {
  814 + error(msg) then error(msg),
  815 + command_complete then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__)),
  816 + table(rows) then
  817 + with previous_desc = extract_description(rows),
  818 +
  819 +
  820 +
  821 +
  822 +
  823 +
  824 +
  825 + informations(infos) then error("Unexpected answer "+__FILE__+":"+to_decimal(__LINE__))
  826 + }.
  827 +
  828 +
  829 +
... ...
anubis_dev/library/data_base/postgres_client.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -82,10 +82,12 @@ public type PG_Column_Description:
82 82  
83 83 Within a data row, each datum is just a byte array. It is up to you to transform it into
84 84 a datum of the relevant type (eventually using the description of its column).
  85 +
  86 +read data_base/metaSQL_common.anubis (contains the definition of type 'MaybeNull')
85 87  
86 88 public type PG_Answer: // (continued)
87   - table(List(PG_Column_Description) columns,
88   - List(List(Maybe(ByteArray))) data_rows),... // failure means NULL
  89 + table(List(PG_Column_Description) columns,
  90 + List(List(MaybeNull(ByteArray))) data_rows),...
89 91  
90 92 Furthermore, there is a raw alternative for holging any other situation. Each pair (Word8,ByteArray) is made
91 93 of the Postgres code of the message and the body of the message (i.e. what is comming after the length word,
... ... @@ -398,7 +400,7 @@ type Backend_Message:
398 400 String value),
399 401 ready_for_query (Word8 status),
400 402 row_description (List(PG_Column_Description) column_descriptions),
401   - data_row (List(Maybe(ByteArray)) values), // failure means NULL
  403 + data_row (List(MaybeNull(ByteArray)) values), // failure means NULL
402 404 command_complete (String tag),
403 405 notice_response (List((Word8,String))).
404 406  
... ... @@ -807,24 +809,24 @@ define Backend_Message
807 809  
808 810 Reading a single value (field) in a row.
809 811  
810   -define (Int,Maybe(ByteArray)) // new position and value. 'failure' means NULL
  812 +define (Int,MaybeNull(ByteArray)) // new position and value. 'failure' means NULL
811 813 read_row_value
812 814 (
813 815 ByteArray raw_msg,
814 816 Int i
815 817 ) =
816 818 with len = read_Word32(raw_msg,i),
817   - if len = -1 then (i+4,failure) else
  819 + if len = -1 then (i+4,null) else
818 820 with l = to_Int(len),
819 821 a = i+4,
820 822 b = a+l,
821 823 with val = extract(raw_msg,a,b), //print("Value: ["+to_string(val)+"]\n");
822   - (b,success(val)).
  824 + (b,not_null(val)).
823 825  
824 826  
825 827 Reading all values in a row.
826 828  
827   -define List(Maybe(ByteArray))
  829 +define List(MaybeNull(ByteArray))
828 830 read_row_values
829 831 (
830 832 Int num, // number of values to read
... ... @@ -1072,12 +1074,12 @@ define Result(PG_Error,One)
1072 1074 We have already read a 'T' message. We have to collect all 'D' messages until we receive
1073 1075 a 'Z' message. All other messages are ignored, except error messages.
1074 1076  
1075   -define Result(PG_Error,List(List(Maybe(ByteArray))))
  1077 +define Result(PG_Error,List(List(MaybeNull(ByteArray))))
1076 1078 read_complete_table_data
1077 1079 (
1078   - ByteArray -> One show,
1079   - Int -> Maybe(ByteArray) read,
1080   - List(List(Maybe(ByteArray))) so_far
  1080 + ByteArray -> One show,
  1081 + Int -> Maybe(ByteArray) read,
  1082 + List(List(MaybeNull(ByteArray))) so_far
1081 1083 ) =
1082 1084 if get_next_response(show,read) is (type,content,msg) then
1083 1085 if msg is ready_for_query(_) then ok(reverse(so_far)) else
... ...
anubis_dev/library/data_base/postgres_graphic_interface.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -16,6 +16,7 @@ read widgets4/check_box.anubis
16 16 read widgets4/space.anubis
17 17  
18 18 read postgres_client.anubis
  19 +read metaSQL_common.anubis
19 20  
20 21  
21 22 define String font_name = "courier_bold_r_12".
... ... @@ -244,8 +245,8 @@ define String
244 245 [ ] then "",
245 246 [d . _] then if d is
246 247 {
247   - failure then "",
248   - success(b) then to_string(b)
  248 + null then "",
  249 + not_null(b) then to_string(b)
249 250 }
250 251 }
251 252 }
... ... @@ -286,12 +287,12 @@ define List(WidgetCell)
286 287 define List(WidgetCell)
287 288 format_data_row
288 289 (
289   - SystemFont font,
290   - List(Maybe(ByteArray)) row
  290 + SystemFont font,
  291 + List(MaybeNull(ByteArray)) row
291 292 ) =
292   - map((Maybe(ByteArray) mb_b) |->
  293 + map((MaybeNull(ByteArray) mb_b) |->
293 294 cell(center,left,none,pg_text(font,
294   - if mb_b is {failure then "NULL", success(b) then to_string(b)})),
  295 + if mb_b is {null then "NULL", not_null(b) then to_string(b)})),
295 296 row).
296 297  
297 298  
... ... @@ -385,7 +386,7 @@ define Widget
385 386 table(cols,data) then table(borders(white,edge_color,0,0),1,1,
386 387 [
387 388 format_columns_row(query,font,cols)
388   - . map((List(Maybe(ByteArray)) row) |-> format_data_row(font,row),data)
  389 + . map((List(MaybeNull(ByteArray)) row) |-> format_data_row(font,row),data)
389 390 ]),
390 391 message_list(l) then table(borders(white,edge_color,1,1),1,1,
391 392 map(((Word8,ByteArray) p) |-> if p is (type,content) then
... ...
anubis_dev/library/tools/connections.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -95,7 +95,7 @@ public define ReadResult // result of reading
95 95 if (Maybe(ByteArray))read(a,n,timeout) is
96 96 {
97 97 failure then error,
98   - success((ByteArray)ba) then ok(ba)
  98 + success(ba) then ok(ba)
99 99 }
100 100 }.
101 101  
... ...
anubis_dev/library/web/generic_form.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -490,7 +490,7 @@ define HTML_Row(HTML_In_Form)
490 490 failure then same,
491 491 success(n) then same(n)
492 492 },
493   - link(button_text),
  493 + push_button(button_text),
494 494 action_name,
495 495 extra_operands))
496 496 ]
... ...
anubis_dev/library/web/making_a_web_site.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -922,6 +922,7 @@ public type HTML_Off_Form:
922 922 actioner (Actioner_Connection, Actioner_Target, Actioner_Aspect,
923 923 String action_name, List((String,String)) extra_ops,
924 924 List(Actioner_Local_Action), String form_name),
  925 + close_window_no_action(String button_text),
925 926 local_popup (Actioner_Aspect, HTML_Off_Form content,
926 927 Word32 x, Word32 y, String title, RGB color, Word32 width),
927 928 foreign_link (List(Text_Option), String url, String name),
... ... @@ -1559,7 +1560,7 @@ define One
1559 1560 Server http_server,
1560 1561 Server https_server,
1561 1562 Var(Bool) shutdown_required
1562   - ) =
  1563 + ) =
1563 1564 if *shutdown_required
1564 1565 then (shutdown(http_server); shutdown(https_server))
1565 1566 else unique;
... ... @@ -2191,6 +2192,7 @@ type HTML_Any($T):
2191 2192 List((String,String)) extra_ops,
2192 2193 List(Actioner_Local_Action),
2193 2194 Maybe(String) form_name),
  2195 + any_close_window_no_action(String button_text),
2194 2196 any_local_popup (Actioner_Aspect, $T content, Word32 x, Word32 y,
2195 2197 String title, RGB color, Word32 width),
2196 2198 any_foreign_link (List(Text_Option), String url, String name),
... ... @@ -2774,7 +2776,8 @@ define Printable_tree
2774 2776 ["<a href=\"",u,"\"", format_text_options(opt), ">"],
2775 2777 // ["<a href=\"",u,"\" style=\"",format(reverse(opt)),"\">"],
2776 2778 javascript(s,h) then
2777   - [s,"<a href=\"javascript: ",h,"\" ",format_text_options(opt),">"]
  2779 + [s,"<a href=\"javascript: ",h," ",
  2780 + format(local_actions),"\" ",format_text_options(opt),">"]
2778 2781 },
2779 2782 text,
2780 2783 "</a>"
... ... @@ -2784,11 +2787,13 @@ define Printable_tree
2784 2787 [
2785 2788 if action is
2786 2789 {
2787   - url(u) then ["<a href=\"",u,"\"", format_text_options(options), ">",
2788   - text, "</a>"
2789   - ],
  2790 + url(u) then
  2791 + //["<a href=\"",u,"\"", format_text_options(options), ">",text, "</a>"],
  2792 + ["<INPUT TYPE =\"button\" Value=\""+text+"\"", format_text_options(options),
  2793 + " onClick=\"",u," ",format(local_actions),"\">"],
2790 2794 javascript(s,h) then
2791   - [s,"<INPUT TYPE =\"button\" Value=\"",text,"\"", format_text_options(options), " onClick=\"",h,"\">"]
  2795 + [s,"<INPUT TYPE =\"button\" Value=\"",text,"\"", format_text_options(options),
  2796 + " onClick=\"",h," ",format(local_actions),"\">"]
2792 2797 }
2793 2798 ],
2794 2799  
... ... @@ -3118,6 +3123,7 @@ define Printable_tree
3118 3123  
3119 3124 Formating text options. They are formated in CSS syntax, to be used within a
3120 3125 'style=...'.
  3126 +
3121 3127 define String
3122 3128 css_style_format
3123 3129 (
... ... @@ -3137,22 +3143,22 @@ define String
3137 3143 [ ] then "",
3138 3144 [h . t] then if h is
3139 3145 {
3140   - size(n) then "font-size:"+n+"pt",
3141   - font(fn) then "font-family:"+fn,
3142   - color(c) then if c is rgb(r,g,b) then
3143   - "color:rgb("+to_decimal(r)+","+to_decimal(g)+","+to_decimal(b)+")",
3144   - italic then "font-style:italic",
3145   - oblique then "font-style:oblique",
3146   - small_capitals then "font-variant:small-caps",
3147   - bold then "font-weight:bold",
3148   - underlined then "text-decoration:underline",
3149   - left_justified then "text-align:left",
3150   - right_justified then "text-align:right",
3151   - justified then "text-align:justify",
3152   - line_through then "text-decoration:line-through",
3153   - nowrap then "white-space:nowrap",
3154   - style(s) then s,
3155   - css_class(class_name)then "class=\"" +class_name +"\""
  3146 + size(n) then "font-size:"+n+"pt",
  3147 + font(fn) then "font-family:"+fn,
  3148 + color(c) then if c is rgb(r,g,b) then
  3149 + "color:rgb("+to_decimal(r)+","+to_decimal(g)+","+to_decimal(b)+")",
  3150 + italic then "font-style:italic",
  3151 + oblique then "font-style:oblique",
  3152 + small_capitals then "font-variant:small-caps",
  3153 + bold then "font-weight:bold",
  3154 + underlined then "text-decoration:underline",
  3155 + left_justified then "text-align:left",
  3156 + right_justified then "text-align:right",
  3157 + justified then "text-align:justify",
  3158 + line_through then "text-decoration:line-through",
  3159 + nowrap then "white-space:nowrap",
  3160 + style(s) then s,
  3161 + css_class(class_name) then "class=\"" +class_name +"\""
3156 3162 } + if t is [ ] then "" else ("; "+format(t))
3157 3163 }.
3158 3164  
... ... @@ -3360,6 +3366,8 @@ define Printable_tree
3360 3366 "</object>"],
3361 3367 any_actioner(c,t,a,an,eo,ja,fn) then
3362 3368 format_actioner(cinfo,sn,c,t,a,an,eo,ja,fn,is_https),
  3369 + any_close_window_no_action(text) then
  3370 + ["<form><input type=button value=\" "+text+" \" onclick=\"window.top.close();\"></form>"],
3363 3371 any_local_popup(a,c,x,y,t,clr,w) then
3364 3372 with n = new_idnum(ic_v),
3365 3373 b = format_local_popup_button(cinfo,a,n),
... ... @@ -3577,6 +3585,8 @@ define Printable_tree
3577 3585 format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,failure),format_element,is_https),
3578 3586 actioner(c,t,a,an,eo,ja,fn) then
3579 3587 format(cinfo,sn,ic_v,any_actioner(c,t,a,an,eo,ja,success(fn)),format_element,is_https),
  3588 + close_window_no_action(t) then
  3589 + format(cinfo,sn,ic_v,any_close_window_no_action(t),format_element,is_https),
3580 3590 local_popup(a,c,x,y,t,clr,w) then
3581 3591 format(cinfo,sn,ic_v,any_local_popup(a,c,x,y,t,clr,w),format_element,is_https),
3582 3592 foreign_link(options,url,name) then
... ...
anubis_dev/library/web/multihost_http_server.anubis
  Show/Hide comments   View file @1a5d477
... ... @@ -239,7 +239,7 @@ public type Web_Site_Description:
239 239 Bool is_https) -> Printable_tree
240 240  
241 241 ('Printable_tree' is a substitute for 'String' and is defined in
242   - 'tools/basis.anubis'). This function is the 'awp handler' for the site. When the URI
  242 + 'tools/printable_tree.anubis'). This function is the 'awp handler' for the site. When the URI
243 243 ends by ".awp", this function is called, and the result (an HTML page) is sent to the
244 244 client over the connection. The last operand to this function is a boolean which is
245 245 'true' when the requests arrives through the HTTPS channel, and 'false' when it arrives
... ... @@ -502,7 +502,7 @@ read tools/connections.anubis
502 502 We use the following self-explanatory types.
503 503  
504 504 type Error:
505   - cannot_read_from_connection,
  505 + cannot_read_from_connection(String),
506 506 not_get_or_post_request(String),
507 507 end_of_line_expected,
508 508 incorrect_content_length_value,
... ... @@ -517,6 +517,27 @@ type HTTP_RequestLine:
517 517 request_line (HTTP_RequestType type,
518 518 String uri,
519 519 List(Web_arg) query_string).
  520 +
  521 + Debug tool:
  522 +
  523 +define String
  524 + format
  525 + (
  526 + HTTP_RequestLine rl
  527 + ) =
  528 + if rl is request_line(type,uri,qs) then
  529 + if type is
  530 + {
  531 + get then "GET ",
  532 + post then "POST "
  533 + }+
  534 + uri+
  535 + concat(map((Web_arg wa) |-> if wa is
  536 + {
  537 + web_arg(n,v) then n+"="+v,
  538 + upload(n,v,p) then n+"="+v+"["+p+"]"
  539 + } ,qs),"&").
  540 +
520 541  
521 542 type EncodingType:
522 543 www_url,
... ... @@ -539,8 +560,8 @@ define String
539 560 ) =
540 561 if msg is
541 562 {
542   - cannot_read_from_connection then
543   - "Cannot read from connection.\n",
  563 + cannot_read_from_connection(s) then
  564 + "Cannot read from connection: "+s+"\n",
544 565 not_get_or_post_request(s) then
545 566 "The request did not begin by 'GET' or 'POST': "+s+".\n",
546 567 end_of_line_expected then
... ... @@ -629,12 +650,12 @@ define Result(Error,Word8)
629 650 //if now > dead_line then record_dubious_connection(connection,dead_line,dos) else
630 651 if read(connection,1,600) is // the connection is closed after 10 minutes of inactivity
631 652 {
632   - error then error(cannot_read_from_connection),
  653 + error then error(cannot_read_from_connection(__FILE__+":"+__LINE__+"["+virtual_machine_id+"]")),
633 654 //record_dubious_connection(connection,dead_line,dos),
634 655 timeout then error(timeout(600)),
635 656 ok(ba) then if nth(0,ba) is
636 657 {
637   - failure then error(cannot_read_from_connection),
  658 + failure then error(cannot_read_from_connection(__FILE__+":"+__LINE__+"["+virtual_machine_id+"]")),
638 659 success(c) then ok(c)
639 660 }
640 661 }.
... ... @@ -1559,10 +1580,10 @@ define Result(Error,ByteArray)
1559 1580 Int retries // this function is called with retries = 10
1560 1581 ) =
1561 1582 if body_size = 0 then ok(constant_byte_array(0,0)) else
1562   - if retries =< 0 then error(cannot_read_from_connection) else
  1583 + if retries =< 0 then error(cannot_read_from_connection(__FILE__+":"+__LINE__+"["+virtual_machine_id+"]")) else
1563 1584 if read(connection,body_size,60) is
1564 1585 {
1565   - error then error(cannot_read_from_connection),
  1586 + error then error(cannot_read_from_connection(__FILE__+":"+__LINE__+"["+virtual_machine_id+"]")),
1566 1587 timeout then error(timeout(60)),
1567 1588 ok(new_bytes) then with
1568 1589 ba = so_far + new_bytes, // contains all the bytes read so far
... ... @@ -2768,7 +2789,7 @@ define One
2768 2789 if read_request_line(connection,start_time+*rld_v,dos) is
2769 2790 {
2770 2791 error(msg) then print(format(msg)),
2771   - ok(request_line) then
  2792 + ok(request_line) then // print(format(request_line)+"\n");
2772 2793 if read_http_headers(connection,start_time+*hd_v,dos) is
2773 2794 {
2774 2795 error(msg) then print(format(msg)),
... ... @@ -2852,7 +2873,7 @@ define One
2852 2873 List(Server) servers,
2853 2874 Int period,
2854 2875 Int next_time,
2855   - ) =
  2876 + ) =
2856 2877 if mapand(is_down,servers)
2857 2878 then unique
2858 2879 else if now > next_time
... ... @@ -2879,7 +2900,7 @@ public define One
2879 2900 List(Server) servers,
2880 2901 Int period
2881 2902 ) =
2882   - delegate http_servers_tasks(sites,servers,period,now),
  2903 + //delegate http_servers_tasks(sites,servers,period,now),
2883 2904 unique.
2884 2905  
2885 2906  
... ...