From 30705e3de7d5b87ff9b8ea51f7a9074778532bfb Mon Sep 17 00:00:00 2001 From: Kaz Kylheku Date: Sat, 1 Aug 2015 08:17:03 -0700 Subject: Big TXR Lisp documentation rearrangement. * txr.1: Numerous sections moved around. Miscellaneous formatting and wording fixes. New sections under TXR Lisp intro section. --- txr.1 | 31137 ++++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 15643 insertions(+), 15494 deletions(-) (limited to 'txr.1') diff --git a/txr.1 b/txr.1 index 1f05ea20..f45d8ea9 100644 --- a/txr.1 +++ b/txr.1 @@ -9100,7 +9100,7 @@ Much of the \*(TL syntax has been introduced in the previous sections of the manual, since directive forms are based on it. There is some additional syntax that is useful in \*(TL programming. -.SS* Symbol Tokens +.NP* Symbol Tokens The symbol tokens in \*(TL, called a @@ -9129,7 +9129,7 @@ variable syntax. Within \*(TL, regular expressions are written with a leading .codn # . -.SS* Consing Dot +.NP* Consing Dot Unlike other major Lisp dialects, \*(TL allows a consing dot with no forms preceding it. This construct simply denotes the form which follows the dot. @@ -9171,7 +9171,7 @@ or an atom as .codn "(. sym)" . -.SS* Quote and Quasiquote +.NP* Quote and Quasiquote .meIP >> ' expr @@ -9301,7 +9301,7 @@ character already has an assigned meaning, so .code * is used. -.SS* Quasiquoting non-List Objects +.NP* Quasiquoting non-List Objects Quasiquoting is supported over hash table and vector literals (see Vectors and Hashes below). A hash table or vector literal can be quoted, like any object, for instance: @@ -9386,7 +9386,7 @@ This is correct: ^#H(,hash-args ,*hash-contents)) .cble -.SS* Quasiquoting combined with Quasiliterals +.NP* Quasiquoting combined with Quasiliterals When a quasiliteral is embedded in a quasiquote, it is possible to use splicing to insert material into the quasiliteral. @@ -9398,7 +9398,7 @@ Example: -> "abc 3 3 1 2 3" .cble -.SS* Vectors +.NP* Vector Literals .coIP "#(...)" @@ -9411,7 +9411,7 @@ and and the symbol .codn a . -.SS* Hashes +.NP* Hash Literals .meIP <> #H(( hash-argument *) >> ( key << value )*) @@ -9434,7 +9434,7 @@ hash table based on the .code eql function, with no weak semantics. -.coSS The @ .. notation +.coNP The @ .. notation In \*(TL, there is a special "dotdot" notation consisting of a pair of dots. This can be written between successive atoms or compound expressions, and is a shorthand for cons. @@ -9481,7 +9481,7 @@ The notation cannot occur in the dot position; that is, the syntax is invalid. The dotdot operator can only be used between the non-dot-position elements of a list. -.SS* The DWIM Brackets +.NP* The DWIM Brackets \*(TL has a square bracket notation. The syntax .code [...] is a shorthand @@ -9514,7 +9514,7 @@ More details are given in the documentation for the .code dwim operator. -.SS* Compound Forms +.NP* Compound Forms In \*(TL, there are two types of compound forms: the Lisp-2 style compound forms, denoted by ordinary lists that are expressed with parentheses. There are Lisp-1 style compound forms denoted by the DWIM Brackets, discussed @@ -9570,7 +9570,7 @@ a function called apply (or similar) must be used for application even if the expression which gives the trailing arguments is a symbol. Moreover, applying sequences other than lists is not supported. -.SS* Regular Expressions +.NP* Regular Expression Literals In \*(TL, the .code / character can occur in symbol names, and the @@ -10176,36 +10176,75 @@ the value which existed in that place prior to deletion. .PP -.SH* TXR LISP OPERATOR AND FUNCTION LIBRARY -A compound expression with a symbol as its first element, if -intended to be evaluated, denotes either an operator invocation or a function -call. This depends on whether the symbol names an operator or a function. +.SS* Namespaces and Environments -When the form is an operator invocation, the interpretation of the meaning of -that form is under the complete control of that operator. +\*(TL is a Lisp-2 dialect: it features separate namespaces for +functions and variables. -If the compound form is a function call, the remaining forms, if any, denote -argument expressions to the function. They are evaluated in left to right -order to produce the argument values, which are passed to the function. An -exception is thrown if there are not enough arguments, or too many. Programs -can define named functions with the defun operator +.NP* Global Functions and Macros -Some operators are macros. There exist predefined macros in the library, and -macro operators can also be user-defined using the macro-defining operator -.codn defmacro . -Operators that are not macros are called special operators. +There is a global namespace for functions, +into which functions and macros can be introduced with the +.codn defun +and +.code defmacro +operators. The global function environment can be inspected and +modified using the +.code symbol-function +accessor. -Macro operators work as functions which are given the source code of the form. -They analyze the form, and translate it to another form which is substituted in -their place. This happens during a code walking phase called the expansion -phase, which is applied to Lisp code prior to evaluation. All macros are -expanded in the expansion phase, resulting in code which contains only -function calls and the executable forms of the operators. +.NP* Dynamic Variables -(Special operators can also perform code transformations during the expansion -phase, but that is not considered macroexpansion, but rather an adjustment -of the representation of the operator into an required executable form.) +There is a global namespace for variables also. +The operators +.code defun +and +.code defparm +introduce bindings into this namespace. These operators have the +side effect of marking a symbol as a special variable, +of the symbol are treated as dynamic variables, subject to +rebinding. The global variable namespace together with the special dynamic +rebinding is called the dynamic environment. +The dynamic environment can be inspected and modified using the +.code symbol-value +accessor. + +.NP* Global Symbol Macros + +Symbol macros may be defined over the global variable namespace +using +.codn defsymacro . + +.NP* Lexical Environments +In addition to global and dynamic namespaces, \*(TL provides lexically scoped +binding for functions, variables, macros, and symbol macros. +Lexical variable binding are introduced with +.codn let , +.code let* +or various binding macros derived from these. Lexical functions are bound +with +.code flet +and +.codn labels . +Lexical macros are established with +.code macrolet +and lexical symbol macros with +.codn symacrolet . + +Macros receive an environment parameter with which they may expand +forms in their correct environment, and perform some limited introspection +over that environment in order to determine the nature of bindings, +or the classification of forms in those environments. This introspection +is provided by +.codn lexical-var-p , +.codn lexical-fun-p , +and +.codn lexical-lisp1-binding . + +.SH* LISP OPERATOR, FUNCTION AND MACRO REFERENCE + +.SS* Conventions The following sections list all of the special operators, macros and functions in \*(TL. @@ -10255,1031 +10294,689 @@ Square brackets indicate optional syntax. Multiple syntactic variations allowed in one place are indicated as bar-separated items. -.SS* Control Flow and Sequencing -.coNP Operators @ progn and @ prog1 -.synb -.mets (progn << form *) -.mets (prog1 << form *) -.syne -.desc -The -.code progn -operator evaluates forms in order, and returns the value -of the last form. The return value of the form -.code (progn) -is -.codn nil . +.ie n \{\ +.coIP syntax @ -> +.\} +.el \{\ +.coIP syntax @ -> [\f[5]result\f[]] +.\} +The arrow notation is used in examples to indicate that the evaluation +of the given syntax produces a value, whose printed representation is +.metn result . -The -.code prog1 -operator evaluates forms in order, and returns the value -of the first form. The return value of the form -.code (prog1) -is -.codn nil . +.SS* Form Evaluation +A compound expression with a symbol as its first element, if +intended to be evaluated, denotes either an operator invocation or a function +call. This depends on whether the symbol names an operator or a function. -Various other operators such as -.code let -also arrange for the evaluation -of a body of forms, the value of the last of which is returned. -These operators are said to feature an implicit -.codn progn . +When the form is an operator invocation, the interpretation of the meaning of +that form is under the complete control of that operator. -.coNP Operator @ cond -.synb -.mets (cond >> {( test << form *)}*) -.syne -.desc -The -.code cond -operator provides a multi-branching conditional evaluation of -forms. Enclosed in the cond form are groups of forms expressed as lists. -Each group must be a list of at least one form. +If the compound form is a function call, the remaining forms, if any, denote +argument expressions to the function. They are evaluated in left to right +order to produce the argument values, which are passed to the function. An +exception is thrown if there are not enough arguments, or too many. Programs +can define named functions with the defun operator -The forms are processed from left to right as follows: the first form, -.metn test , -in each group is evaluated. If it evaluates true, then the remaining -forms in that group, if any, are also evaluated. Processing then terminates and -the result of the last form in the group is taken as the result of cond. -If -.meta test -is the only form in the group, then result of -.meta test -is taken -as the result of -.codn cond . +Some operators are macros. There exist predefined macros in the library, and +macro operators can also be user-defined using the macro-defining operator +.codn defmacro . +Operators that are not macros are called special operators. -If the first form of a group yields -.codn nil , -then processing continues with the -next group, if any. If all form groups yield -.codn nil , -then the cond form yields -.codn nil . -This holds in the case that the syntax is empty: -.code (cond) -yields -.codn nil . +Macro operators work as functions which are given the source code of the form. +They analyze the form, and translate it to another form which is substituted in +their place. This happens during a code walking phase called the expansion +phase, which is applied to each top-level expression prior to evaluation. All +macros occurring in a form are expanded in the expansion phase, and subsequent +evaluation takes place on a structure which is devoid of macros. All that +remains are the executable forms of special operators, function calls, +symbols denoting either variables or themselves, and atoms such as numeric +and string literals. + +Special operators can also perform code transformations during the expansion +phase, but that is not considered macroexpansion, but rather an adjustment +of the representation of the operator into an required executable form. +In effect, it is post-macro compilation phase. -.coNP Macros @, caseq @ caseql and @ casequal +Note that Lisp forms occurring in \*(TX pattern language are not individual +top-level forms. Rather, the entire \*(TX query is parsed at the same time, and +the macros occurring in its Lisp forms are expanded at that time. + +.coNP Operator @ quote .synb -.mets (caseq < test-form << normal-clause * <> [ else-clause ]) -.mets (caseql < test-form << normal-clause * <> [ else-clause ]) -.mets (casequal < test-form << normal-clause * <> [ else-clause ]) +.mets (quote << form ) .syne .desc -These three macros arrange for the evaluation of of -.metn test-form , -whose value is then compared against the key or keys in each -.meta normal-clause -in turn. -When the value matches a key, then the remaining forms of -.meta normal-clause -are evaluated, and the value of the last form is returned; subsequent -clauses are not evaluated. When the value doesn't match any of the keys -of a -.meta normal-clause -then the next -.meta normal-clause -is tested. -If all these clauses are exhausted, and there is no -.metn else-clause , -then the value nil is returned. Otherwise, the forms in the -.meta else-clause -are evaluated, and the value of the last one is returned. - -The syntax of a -.meta normal-clause -takes on these two forms: +The +.code quote +operator, when evaluated, suppresses the evaluation of +.metn form , +and instead returns +.meta form +itself as an object. For example, if +.meta form +is a symbol, then +.meta form +is not evaluated to the symbol's value; rather +the symbol itself is returned. +Note: the quote syntax .cblk -.mets >> ( key << form *) +.meti >> '
+.cble +is translated to +.cblk +.meti (quote << form ). .cble -where -.meta key -may be an atom which denotes a single key, or else a list -of keys. There is a restriction that the symbol -.code t -may not be used -as -.metn key . -The form -.code (t) -may be used as a key to match that symbol. - -The syntax of an -.meta else-clause -is: +.TP* Example: .cblk -.mets (t << form *) + ;; yields symbol a itself, not value of variable a + (quote a) -> a + + ;; yields three-element list (+ 2 2), not 4. + (quote (+ 2 2)) -> (+ 2 2) .cble -which resembles a form that is often used as the final clause -in the -.code cond -syntax. -The three forms of the case construct differ from what type of -test they apply between the value of -.meta test-form -and the keys. -The -.code caseq -macro generates code which uses the -.code eq -function's -equality. The -.code caseql -macro uses -.codn eql , +.SS* Variable Binding + +Variables are associations between symbols and storage locations +which hold values. These associations are called +.IR bindings . + +Bindings are held in a context called an +.IR environment . + +.I Lexical +environments hold local variables, and nest according to the syntactic +structure of the program. Lexical bindings are always introduced by a +some form known as a +.I "binding construct", +and the corresponding environment is instantiated during the evaluation +of that construct. There also exist bindings outside of any binding +construct, in the so-called +.I global environment . +Bindings in the global environment can be temporarily shadowed by +lexically-established binding in the +.I dynamic environment . +See the Special Variables section above. + +Certain special symbols cannot be used as variable names, namely the +symbols +.code t and -.code casequal -uses -.codn equal . +.codn nil , +and all of the keyword symbols (symbols in the keyword package), which are +denoted by a leading colon. When any of these symbols is evaluated as +a form, the resulting value is that symbol itself. It is said that these +special symbols are self-evaluating or self-quoting, similarly to all +other atom objects such as numbers or strings. + +When a form consisting of a symbol, other than the above special symbols, is +evaluated, it is treated as a variable, and yields the value of +the variable's storage location. If the variable doesn't exist, +an exception is thrown. -.TP* Example -.cblk - (let ((command-symbol (casequal command-string - (("q" "quit") 'quit) - (("a" "add") 'add) - (("d" "del" "delete") 'delete) - (t 'unknown)))) - ...) -.cble +Note: symbol forms may also denote invocations of symbol macros. (See the +operators +.code defsymacro +and +.codn symacrolet ). +All macros, including symbol macros, which occur inside +a form are fully expanded prior to the evaluation of a form, therefore +evaluation does not consider the possibility of a symbol being +a symbol macro. -.coNP Operator/function @ if +.coNP Operators @ defvar and @ defparm .synb -.mets (if < cond < t-form <> [ e-form ]) -.mets [if < cond < then <> [ else ]] +.mets (defvar < sym <> [ value ]) +.mets (defparm < sym << value ) .syne + .desc -There exist both an -.code if -operator and an -.code if -function. A list form with the symbol -.code if -in the fist position is interpreted as an invocation of the -.code if -operator. -The function can be accessed using the DWIM bracket notation and in other -ways. The -.code if -operator provides a simple two-way-selective evaluation control. -The -.meta cond -form is evaluated. If it yields true then -.meta t-form -is evaluated, and that form's return value becomes the return value of the -.codn if . -If -.meta cond -yields false, then -.meta e-form -is evaluated and its return value is taken to be that of -.codn if . -If -.meta e-form -is omitted, then the behavior is as if -.meta e-form -were specified as -.codn nil . +.code defvar +operator binds a name in the variable namespace of the global environment. +Binding a name means creating a binding: recording, in some namespace of some +environment, an association between a name and some named entity. In the +case of a variable binding, that entity is a storage location for a value. +The value of a variable is that which has most recently been written into the +storage location, and is also said to be a value of the binding, or stored +in the binding. -The -.code if -function provides no evaluation control. All of arguments -are evaluated from left to right. If the -.meta cond -argument is true, then it -returns the -.meta then -argument, otherwise it returns the value of the -.meta else -argument if present, otherwise it returns -.codn nil . +If the variable named +.meta sym +already exists in the global environment, the +form has no effect; the +.meta value +form is not evaluated, and the value of the +variable is unchanged. -.coNP Operator/function @ and -.synb -.mets (and << form *) -.mets [and << arg *] -.syne -.desc -There exist both an -.code and -operator and an -.code and -function. A list form with the -symbol -.code and -in the fist position is interpreted as an invocation of the -operator. The function can be accessed using the DWIM bracket notation and in -other ways. +If the variable does not exist, then a new binding is introduced, with a value +given by evaluating the +.meta value +form. If the form is absent, the variable is initialized +to +.codn nil . The -.code and -operator provides three functionalities in one. It computes the -logical "and" function over several forms. It controls evaluation (a.k.a. -"short-circuiting"). It also provides an idiom for the convenient substitution -of a value in place of -.code nil -when some other values are all true. +.meta value +form is evaluated in the environment +in which the +.code defvar +form occurs, not necessarily in the global environment. -The -.code and -operator evaluates as follows. First, a return value is -established and initialized to the value -.codn t . -The -.metn form s, -if any, are -evaluated from left to right. The return value is overwritten with -the result of each form. Evaluation stops when all forms are exhausted, -or when +The symbols +.code t +and .code nil -is stored in the return value. -When evaluation stops, the operator yields the return value. +may not be used as variables, and neither +can be keyword symbols: symbols denoted by a leading colon. + +In addition to creating a binding, the +.code defvar +operator also marks +.meta sym +as the name of a special variable. This changes what it means to bind +that symbol in a lexical binding construct such as the +.code let +operator, or a function parameter list. See the section "Special Variables" far +above. The -.code and -function provides no evaluation control; it receives all of its -arguments fully evaluated. If it is given no arguments, it returns -.codn t . -If it is given one or more arguments, and any of them are -.codn nil , -it returns -.codn nil . -Otherwise it returns the value of the last argument. +.code defparm +operator behaves like +.code defvar +when a variable named +.meta sym +doesn't already exist. + +If +.meta sym +already denotes a variable binding in the global namespace, +.code defparm +evaluates the +.meta value +form and assigns the resulting value to the variable. + +The following equivalence holds: -.TP* Examples: .cblk - (and) -> t - (and (> 10 5) (stringp "foo")) -> t - (and 1 2 3) -> 3 ;; short-hand for (if (and 1 2) 3). + (defparm x y) <--> (prog1 (defvar x) (set x y)) .cble -.coNP Operator/function @ or +The +.code defvar +and +.code defparm +operators return +.metn sym . + +.coNP Operators @ let and @ let* .synb -.mets (or << form *) -.mets [or << arg *] +.mets (let >> ({ sym | >> ( sym << init-form )}*) << body-form *) +.mets (let* >> ({ sym | >> ( sym << init-form )}*) << body-form *) .syne .desc -There exist both an -.code or -operator and an -.code or -function. A list form with the -symbol -.code or -in the fist position is interpreted as an invocation of the -operator. The function can be accessed using the DWIM bracket notation and in -other ways. - -The or operator provides three functionalities in one. It computes the -logical "or" function over several forms. It controls evaluation (a.k.a. -"short-circuiting"). The behavior of -.code or -also provides an idiom for the selection of the first non-nil value from a -sequence of forms. - The -.code or -operator evaluates as follows. First, a return value is -established and initialized to the value +.code let +and +.code let* +operators introduce a new scope with variables and +evaluate forms in that scope. The operator symbol, either +.code let +or +.codn let* , +is followed by a list which can contain any mixture of variable +name symbols, or +.cblk +.meti >> ( sym << init-form ) +.cble +pairs. A symbol +denotes the name of variable to be instantiated and initialized +to the value .codn nil . -The -.metn form s, -if any, -are evaluated from left to right. The return value is overwritten -with the result of each -.metn form . -Evaluation stops when all forms are -exhausted, or when a true value is stored into the return value. -When evaluation stops, the operator yields the return value. +A symbol specified with an init-form denotes +a variable which is initialized from the value of the +.metn init-form . -The -.code or -function provides no evaluation control; it receives all of its -arguments fully evaluated. If it is given no arguments, it returns -.codn nil . -If all of its arguments are -.codn nil , -it also returns -.codn nil . -Otherwise, it -returns the value of the first argument which isn't -.codn nil . +The symbols +.code t +and +.code nil +may not be used as variables, and neither +can be keyword symbols: symbols denoted by a leading colon. + +The difference between +.code let +and +.code let* +is that in +.codn let* , +later +.codn init-form s +have visibility over the variables established by earlier variables +in the same let* construct. In plain +.codn let , +the variables are not visible to any of the +.metn init-form s. + +When the variables are established, then the +.metn body-form s +are evaluated in order. The value of the last +.meta body-form +becomes the return value of the +.codn let . + +If there are no +.metn body-form s, +then the return value +.code nil +is produced. + +The list of variables may be empty. .TP* Examples: .cblk - (or) -> nil - (or 1 2) -> 1 - (or nil 2) -> 2 - (or (> 10 20) (stringp "foo")) -> t + (let ((a 1) (b 2)) (list a b)) -> (1 2) + (let* ((a 1) (b (+ a 1))) (list a b (+ a b))) -> (1 2 3) + (let ()) -> nil + (let (:a nil)) -> error, :a and nil can't be used as variables .cble -.coNP Macros @ when and @ unless +.SS* Functions +.coNP Operator @ defun .synb -.mets (when < expression << form *) -.mets (unless < expression << form *) +.mets (defun < name <> ( param * [: << opt-param *] [. << rest-param ]) +.mets \ \ << body-form ) .syne .desc -The when macro operator evaluates -.metn expression . -If -.meta expression -yields -true, and there are additional forms, then each -.meta form -is evaluated. -The value of the last form is becomes the result value of the when form. -If there are no forms, then the result is -.codn nil . - -The -.code unless -operator is similar to when, except that it reverses the -logic of the test. The forms, if any, are evaluated if, and only if -.meta expression -is false. - -.coNP Macros @ while and @ until -.synb -.mets (while < expression << form *) -.mets (until < expression << form *) -.syne -.desc -The -.code while -macro operator provides a looping construct. It evaluates -.metn expression . -If -.meta expression -yields -.codn nil , -then the evaluation of the -.code while -form -terminates, producing the value -.codn nil . -Otherwise, if there are additional forms, -then each -.meta form -is evaluated. Next, evaluation returns to -.metn expression , -repeating all of the previous steps. - The -.code until -macro operator is similar to while, except that the until form -terminates when -.meta expression -evaluates true, rather than false. +.code defun +operator introduces a new function in the global function namespace. +The function is similar to a lambda, and has the same parameter syntax +and semantics as the +.code lambda +operator. -These operators arrange for the evaluation of all their enclosed forms -in an anonymous block. Any of the -.metn form s, -or -.metn expression , -may use +Unlike in +.codn lambda , the -.code return -operator to terminate the loop, and optionally to specify -a result value for the form. +.metn body-form s +of a +.code defun +are surrounded by a block. +The name of this block is the same as the name of the function, making it +possible to terminate the function and return a value using +.cblk +.meti (return-from < name << value ). +.cble +For more information, see the definition of the block operator. -The only way these forms can yield a value other than -.code nil -is if the -.code return -operator is used to terminate the implicit anonymous block, -and is given an argument, which becomes the result value. +A function may call itself by name, allowing for recursion. -.coNP Macros @ while* and @ until* -.synb -.mets (while* < expression << form *) -.mets (until* < expression << form *) -.syne -.desc -The -.code while* -and -.code until* -macros are similar, respectively, to the macros -.code while +The special symbols +.code t and -.codn until . - -They differ in one respect: they begin by evaluating the -.metn form -s -one time unconditionally, without first evaluating -.metn expression . -After this evaluation, the subsequent behavior is -like that of -.code while -or -.codn until . +.code nil +may not be used as function names. Neither can keyword symbols. -Another way to regard the behavior is that that these forms execute -one iteration unconditionally, without evaluating the termination test prior to -the first iteration. Yet another view is that these constructs relocate the -test from the "top of the loop" to the "bottom of the loop". +.TP* "Dialect Note:" +In ANSI Common Lisp, keywords may be used as function names. +In TXR Lisp, they may not. -.coNP Macro @ whilet +.coNP Operator @ lambda .synb -.mets (whilet >> ({ sym | >> ( sym << init-form )}+) -.mets \ \ << body-form *) +.mets (lambda <> ( param * [: << opt-param *] [. << rest-param ]) +.mets \ \ << body-form ) +.mets (lambda < rest-param +.mets \ \ << body-form ) .syne .desc The -.code whilet -macro provides a construct which combines iteration with variable -binding. +.code lambda +operator produces a value which is a function. Like in most other +Lisps, functions are objects in \*(TL. They can be passed to functions as +arguments, returned from functions, aggregated into lists, stored in variables, +et cetera. -The evaluation of the form takes place as follows. First, fresh bindings are -established for -.metn sym -s -as if by the -.code let* -operator. -It is an error for the list of variable bindings to be empty. +The first argument of +.code lambda +is the list of parameters for the function. It +may be empty, and it may also be an improper list (dot notation) where the +terminating atom is a symbol other than +.codn nil . +It can also be a single symbol. -After the establishment of the bindings, the the value of the -.meta sym -is tested. If the value is -.codn nil , -then -.code whilet -terminates. Otherwise, -.metn body-form -s -are evaluated in the scope of the variable bindings, and then -.code whilet -iterates from the beginning, again establishing fresh bindings for the -.metn sym -s, -and testing the value of the last -.metn sym . +The second and subsequent arguments are the forms making up the function body. +The body may be empty. -All evaluation takes place in an anonymous block, which can be -terminated with the -.code return -operator. Doing so terminates the loop. -If the -.code whilet -loop is thus terminated by an explicit -.codn return , -a return value can be specified. Under normal termination, the return value is -.codn nil . +When a function is called, the parameters are instantiated as variables that +are visible to the body forms. The variables are initialized from the values of +the argument expressions appearing in the function call. + +The dotted notation can be used to write a function that accepts +a variable number of arguments. There are two ways write a function that +accepts only a variable argument list and no required arguments: -.TP* Examples: .cblk - ;; read lines of text from *std-input* and print them, - ;; until the end-of-stream condition: +.mets (lambda (. << rest-param ) ...) +.mets (lambda < rest-param ...) +.cble - (whilet ((line (get-line))) - (put-line line)) +(These notations are syntactically equivalent because the list notation +.code (. X) +actually denotes the object +.code X +which isn't wrapped in any list). - ;; read lines of text from *std-input* and print them, - ;; until the end-of-stream condition occurs or - ;; a line is identical to the character string "end". +The keyword symbol +.code : +(colon) can appear in the parameter list. This is +the symbol in the keyword package whose name is the empty string. This +symbol is treated specially: it serves as a separator between +required parameters and optional parameters. Furthermore, the +.code : +symbol has a role to play in function calls: it can be specified as an argument +value to an optional parameter by which the caller indicates that the +optional argument is not being specified. It will be processed exactly +that way. - (whilet ((line (get-line)) - (more (and line (not (equal line "end"))))) - (put-line line)) +An optional parameter can also be written in the form +.cblk +.meti >> ( name < expr <> [ sym ]). .cble +In this situation, if the call does not specify a value for the parameter +(or specifies a value as the keyword +.code : +(colon)) then the parameter takes on the +value of the expression +.metn expr . +If +.meta sym +is specified, then +.meta sym +will be +introduced as an additional binding with a boolean value which indicates +whether or not the optional parameter had been specified by the caller. -.coNP Macros @ iflet and @ whenlet -.synb -.mets (iflet >> ({ sym | >> ( sym << init-form )}+) -.mets \ \ < then-form <> [ else-form ]) -.mets (whenlet >> ({ sym | >> ( sym << init-form )}+) -.mets \ \ << body-form *]) -.syne -.desc -The -.code iflet -and -.code whenlet -macros combine the variable binding of -.code let* -with conditional evaluation of -.code if -and -.codn when , -respectively. +The initializer expressions are evaluated an environment in which +all of the previous parameters are visible, in addition to the surrounding +environment of the lambda. For instance: -The evaluation of these forms takes place as follows. First, fresh bindings are -established for -.metn sym -s -as if by the -.code let* -operator. -It is an error for the list of variable bindings to be empty. +.cblk + (let ((default 0)) + (lambda (str : (end (length str)) (counter default)) + (list str end counter))) +.cble -Then, the last variable's value is tested. If it is not -.code nil -then the test is true, otherwise false. +In this +.codn lambda , +the initializing expression for the optional parameter +end is +.codn (length str) , +and the +.code str +variable it refers to is the previous +argument. The initializer for the optional variable counter is +the expression default, and it refers to the binding established +by the surrounding let. This reference is captured as part of the +.codn lambda 's +lexical closure. -In the case of the -.code iflet -operator, if the test is true, the operator evaluates -.meta then-form -and yields its value. Otherwise the test is false, and if the -optional -.meta else-form -is present, that is evaluated instead and its value is returned. -If this form is missing, then -.code nil -is returned. +.TP* Examples: +.IP "Counting function:" +This function, which takes no arguments, captures the +variable +.codn counter . +Whenever this object is called, it increments +.code counter +by +.code 1 +and returns the incremented value. -In the case of the -.code whenlet -operator, if the test is true, then the -.metn body-form -s, -if any, are evaluated. The value of the last one is -returned, otherwise -.code nil -if the forms are missing. -If the test is false, then evaluation of -.metn body-form -s -is skipped, and -.code nil -is returned. +.cblk + (let ((counter 0)) + (lambda () (inc counter))) +.cble + +.IP "Function that takes two or more arguments:" +The third and subsequent arguments are aggregated into a list passed as the +single parameter +.codn z : -.TP* Examples: .cblk - ;; dispose of foo-resource if present - (whenlet ((foo-res (get-foo-resource obj))) - (foo-shutdown foo-res) - (set-foo-resource obj nil)) + (lambda (x y . z) (list 'my-arguments-are x y z)) +.cble - ;; Contrast with: above, using when and let - (let ((foo-res (get-foo-resource obj))) - (when foo-res - (foo-shutdown foo-res) - (set-foo-resource obj nil))) +.IP "Variadic function:" - ;; print frobosity value if it exceeds 150 - (whenlet ((fv (get-frobosity-value)) - (exceeds-p (> fv 150))) - (format t "frobosity value ~a exceeds 150\en" fv)) +.cblk + (lambda args (list 'my-list-of-arguments args)) .cble -.coNP Macro @ ifa +.IP "Optional arguments:" + +.cblk + [(lambda (x : y) (list x y)) 1] -> (1 nil) + [(lambda (x : y) (list x y)) 1 2] -> (1 2) +.cble + +.coNP Macros @ flet and @ labels .synb -.mets (ifa < cond < then <> [ else ]) +.mets (flet >> ({( name < param-list << function-body-form *)}*) +.mets \ \ << body-form *) + +.mets (labels >> ({( name < param-list << function-body-form *)}*) +.mets \ \ << body-form *) .syne .desc The -.code ifa -macro provides a anaphoric conditional operator resembling the -.code if -operator. Around the evaluation of the -.meta then +.code flet and -.meta else -forms, the symbol -.code it -is implicitly bound to a subexpression of -.metn cond , -providing a reference to that value, similar to the word -"it" in the English language, and similar anaphoric pronouns -in other languages. If -.code it -is bound to a place form, the binding is established -as if using the -.code placelet -operator. Otherwise, -.code it -is bound as an ordinary lexical variable to -the form's value. +.code labels +macros bind local, named functions in the lexical scope. +The difference between +.code flet +and +.code labels +is that a function defined by +.code labels +can see itself, and therefore recurse directly by name. Moreover, if multiple +functions are defined by the same labels construct, they all see each other. +By contrast, a +.codn flet -defined +function does not have itself in scope and cannot recurse. +Multiple functions in the same +.code flet +do not have each other's names in their scopes. -The -.code ifa -macro imposes several restrictions on the -.meta cond -expression. Firstly, the -.meta cond -expression must be either an atom, or a function call form: -a compound expression with a symbol in the leftmost position -which resolves to a function. Otherwise the -.code ifa -macro invocation is ill-formed. +More formally, the +.metn function-body-form -s +and +.meta param-list +of the functions defined by +.code labels +are in a scope in which all of the function +names being defined by that same +.code labels +construct are visible. -Secondly, if the -.meta cond -expression is a function call with two or more arguments, -at most one of them may be an it-candidate: an expression -viable for having its value or storage location bound to the -.code it -symbol. If there are two or more it-candidates, the -.code ifa -expression is ill-formed. +Under both +.code labels +and +.codn flet , +the local functions that are defined are +lexically visible to the main +.metn body-form -s. -If -.meta cond -is an atom, or a function call expression with no arguments, -then the -.code it -symbol is not bound. Effectively, -.code ifa -macro behaves like the ordinary -.code if -operator. +Note that +.code labels +and +.code flet +are properly scoped with regard to macros. +During macro expansion, function bindings introduced by these +macro operators shadow macros defined by +.code macrolet +and +.codn defmacro . -If -.meta cond -is an invocation of the functions -.code not -or -.codn null , +Furthermore, function bindings introduced by +.code labels +and +.code flet +also shadow symbol macros defined by +.codn symacrolet , +when those symbol macros occur as arguments of a +.code dwim +form. -If -.meta cond -is a function call with exactly one argument, then the -.code it -variable is bound to the value of that argument, except when -the function being called is -.codn not , -.code null , -or -.codn false . -That special situation is rewritten according to the following pattern: +See also: the +.code macrolet +operator. +.TP* Examples: .cblk -.mets (ifa (not << expr ) < then << else ) -> (ifa < expr < else << then ) -.cble + ;; Wastefully slow algorithm for determining evenness. + ;; Note: + ;; - mutual recursion between labels-defined functions + ;; - inner is-even bound by labels shadows the outer + ;; one bound by defun so the (is-even n) call goes + ;; to the local function. -Note the reversal of -.meta then -and -.metn else . + (defun is-even (n) + (labels ((is-even (n) + (if (zerop n) t (is-odd (- n 1)))) + (is-odd (n) + (if (zerop n) nil (is-even (- n 1))))) + (is-even n))) +.cble -If -.meta cond -is a function call with two or more arguments, then it is only -well-formed if at most one of those arguments is an it-candidates. -If there is one such argument, then the -.code it -variable is bound to it. Otherwise the variable is bound -to the leftmost argument expression, regardless of whether that -argument expression is an it-candidate. - -An it-candidate is any expression which is not a constant expression -according to the -.code constantp -function, and not a symbol. - -In all other regards, the -.code ifa -macro behaves similarly to -.codn if . - -The -.meta cond -expression is evaluated, and, if applicable, -the value of, or storage location denoted by the appropriate argument is -captured and bound to the variable -.code it -whose scope extends over the -.meta then -form, as well as over -.metn else , -if present. - -If -.meta cond -yields a true value, then -.meta then -is evaluated and the resulting value is returned, otherwise -.meta else -is evaluated if present and its value is returned. -A missing -.meta else -is treated as if it were the -.code nil -form. +.coNP Function @ call +.synb +.mets (call < function << argument *) +.syne +.desc +The +.code call +function invokes +.metn function , +passing it the given arguments, if any. .TP* Examples: -.cblk - (ifa t 1 0) -> 1 +Apply arguments +.code 1 2 +to a +.code lambda +which adds them to produce +.codn 3 : - (let ((x 6) (y 49)) - (ifa (> y (* x x)) ;; it binds to (* x x) - (list it))) - -> (36) +.cblk + (call (lambda (a b) (+ a b)) 1 2) +.cble - (ifa (evenp 4) - (list it)) - -> (4) +Useless use of +.code call +on a named function; equivalent to +.codn (list 1 2) : - (ifa (not (oddp 4)) - (list it)) - -> (4) +.cblk + (call (fun list) 1 2) .cble -.coNP Macro @ conda +.coNP Operator @ fun .synb -.mets (conda >> {( test << form *)}*) +.mets (fun << function-name ) .syne .desc The -.code conda -operator provides a multi-branching conditional evaluation of -forms, similarly to the -.code cond -operator. Enclosed in the cond form are groups of forms expressed as lists. -Each group must be a list of at least one form. +.code fun +operator retrieves the function object corresponding to a named +function in the current lexical environment. The -.code conda -operator is anaphoric: it expands into a nested structure of zero or more -.code ifa -invocations, according to these patterns: +.meta function-name +is a symbol denoting a named function: a built in +function, or one defined by +.codn defun . -.cblk - (conda) -> nil - (conda (x y ...) ...) -> (ifa x (progn y ...) (conda z ...)) -.cble +Note: the +.code fun +operator does not see macro bindings. It is possible to +retrieve a global macro expander using +.codn symbol-function . -Thus, -.code conda -inherits all the restrictions on the -.meta test -expressions from -.codn ifa , -as well as the anaphoric -.code it -variable feature. +.TP* "Dialect Note:" +A lambda expression is not a function name in \*(TL. The +syntax +.code (fun (lambda ...)) +is invalid. -.coNP Macro @ dotimes +.coNP Operator @ dwim .synb -.mets (dotimes >> ( var < count-form <> [ result-form ]) << body-form *) +.mets (dwim << argument *) +.mets <> [ argument *] +.mets (set (dwim < obj-place < index <> [ alt ]) << new-value ) +.mets (set >> [ obj-place < index <> [ alt ]] << new-value ) .syne .desc The -.code dotimes -macro implements a simple counting loop. -.meta var -is established as a variable, and initialized to zero. -.meta count-form -is evaluated one time to produce a limiting value, which should be a number. -Then, if the value of -.meta var -is less than the limiting value, the -.metn body-form -s -are evaluated, -.meta var -is incremented by one, and the process repeats with a new comparison -of -.meta var -against the limiting value possibly leading to another evaluation of -the forms. - -If -.meta var -is found to equal or exceed the limiting value, then the loop terminates. - -When the loop terminates, its return value is -.code nil -unless a -.meta result-form -is present, in which case the value of that form specifies the return value. +.code dwim +operator's name is an acronym: DWIM may be taken to mean +"Do What I Mean", or alternatively, "Dispatch, in a Way that is +Intelligent and Meaningful". -.metn body-form -s -as well as -.meta result-form -are evaluated in the scope in which the binding of -.meta var -is visible. +The notation +.code [...] +is a shorthand equivalent to +.code (dwim ...) +and is usually the preferred way for writing +.code dwim +expressions. -.coNP Operator @ unwind-protect -.synb -.mets (unwind-protect < protected-form << cleanup-form *) -.syne -.desc The -.cod unwind-protect -operator evaluates -.meta protected-form -in such a way that no matter how the execution of -.meta protected-form -terminates, the -.metn cleanup-form s -will be executed. +.code dwim +operator takes a variable number of arguments, which are +all evaluated in the same way: the first argument is not evaluated differently +from the remaining arguments. -The -.metn cleanup-form s, -however, are not protected. If a -.meta cleanup-form -terminates via -some non-local jump, the subsequent -.metn cleanup-form s -are not evaluated. +This means that the first argument isn't a function name, but an ordinary +expression which can simply compute a function object (or, more generally, +a callable object). -.metn cleanup-form s -themselves can "hijack" a non-local control transfer such -as an exception. If a -.meta cleanup-form -is evaluated during the processing of -a dynamic control transfer such as an exception, and that -.meta cleanup-form -initiates its own dynamic control transfer, the original control transfer -is aborted and replaced with the new one. +Furthermore, for those arguments of +.code dwim +which are symbols (after all +macro-expansion is performed on the arguments), the evaluation rules are +altered. For the purposes of resolving symbols to values, the function and +variable binding namespaces are considered to be merged into a single space, +creating a situation that is very similar to a Lisp-1 style dialect. -.TP* Example: -.cblk - (block foo - (unwind-protect - (progn (return-from foo 42) - (format t "not reached!\en")) - (format t "cleanup!\en"))) -.cble +This special Lisp-1 evaluation is not recursively applied. All arguments of +.code dwim +which, after macro expansion, are not symbols are evaluated using the +normal Lisp-2 evaluation rules. Thus, the DWIM operator must be used +in every expression where the Lisp-1 rules for reducing symbols to +values are desired. -In this example, the protected -.code progn -form terminates by returning from -block -.codn foo . -Therefore the form does not complete and so the -output -.str not reached! -is not produced. However, the cleanup form -executes, producing the output -.strn cleanup! . +After macro expansion, the first argument of +.code dwim +may not be an operator such +as +.codn let , +or the name of a macro. Prior to macroexpansion, any argument of +.code dwim +may be a symbol macro. -.coNP Operator @ block -.synb -.mets (block < name << body-form *) -.syne -.desc -The -.code block -operator introduces a named block around the execution of -some forms. The -.meta name -argument must be a symbol. Since a block name is not -a variable binding, keyword symbols are permitted, and so are the symbols -.code t -and -.codn nil . -A block named by the symbol nil is slightly special: it is -understood to be an anonymous block. - -Blocks in \*(TL have dynamic scope. This means that the following -situation is allowed: - -.cblk - (defun func () (return-from foo 42)) - (block foo (func)) -.cble - -The function can return from the -.code foo -block even though the -.code foo -block -does not lexically surround -.codn foo . - -Thus blocks in \*(TL provide dynamic non-local returns, as well -as returns out of lexical nesting. - -.TP* "Dialect Note:" -In Common Lisp, blocks are lexical. A separate mechanism consisting of -catch and throw operators performs non-local transfer based on symbols. -The \*(TL example: - -.cblk - (defun func () (return-from foo 42)) - (block foo (func)) -.cble - -is not allowed in Common Lisp, but can be transliterated to: - -.cblk - (defun func () (throw 'foo 42)) - (catch 'foo (func)) -.cble - -Note that foo is quoted in CL. This underscores the dynamic nature of -the construct. -.code throw -itself is a function and not an operator. - -.coNP Operators @ return and @ return-from -.synb -.mets (return <> [ value ]) -.mets (return-from < name <> [ value ]) -.syne -.desc -The -.code return -operator must be dynamically enclosed within an anonymous -block (a block named by the symbol -.codn nil ). -It immediately terminates the -evaluation of the innermost anonymous block which encloses it, causing -it to return the specified value. If the value is omitted, the anonymous -block returns -.codn nil . - -The -.code return-from -operator must be dynamically enclosed within a named block -whose name matches the -.meta name -argument. It immediately terminates the -evaluation of the innermost such block, causing it to return the specified -value. If the value is omitted, that block returns -.codn nil . - -.TP* Example: -.cblk - (block foo - (let ((a "abc\en") - (b "def\en")) - (pprint a *stdout*) - (return-from foo 42) - (pprint b *stdout*))) -.cble - -Here, the output produced is -.strn "abc" . -The value of -.code b -is not printed -because. -.code return-from -terminates block -.codn foo , -and so the second pprint form is not evaluated. - -.SS* Evaluation - -.coNP Operator @ dwim -.synb -.mets (dwim << argument *) -.mets <> [ argument *] -.mets (set (dwim < obj-place < index <> [ alt ]) << new-value ) -.mets (set >> [ obj-place < index <> [ alt ]] << new-value ) -.syne -.desc -The -.code dwim -operator's name is an acronym: DWIM may be taken to mean -"Do What I Mean", or alternatively, "Dispatch, in a Way that is -Intelligent and Meaningful". - -The notation -.code [...] -is a shorthand equivalent to -.code (dwim ...) -and is usually the preferred way for writing -.code dwim -expressions. - -The -.code dwim -operator takes a variable number of arguments, which are -all evaluated in the same way: the first argument is not evaluated differently -from the remaining arguments. - -This means that the first argument isn't a function name, but an ordinary -expression which can simply compute a function object (or, more generally, -a callable object). - -Furthermore, for those arguments of -.code dwim -which are symbols (after all -macro-expansion is performed on the arguments), the evaluation rules are -altered. For the purposes of resolving symbols to values, the function and -variable binding namespaces are considered to be merged into a single space, -creating a situation that is very similar to a Lisp-1 style dialect. - -This special Lisp-1 evaluation is not recursively applied. All arguments of -.code dwim -which, after macro expansion, are not symbols are evaluated using the -normal Lisp-2 evaluation rules. Thus, the DWIM operator must be used -in every expression where the Lisp-1 rules for reducing symbols to -values are desired. - -After macro expansion, the first argument of -.code dwim -may not be an operator such -as -.codn let , -or the name of a macro. Prior to macroexpansion, any argument of -.code dwim -may be a symbol macro. - -If a symbol has bindings both in the variable and function namespace in scope, -and is referenced by a dwim argument, this constitutes a conflict which is -resolved according to two rules. When nested scopes are concerned, then an -inner binding shadows an outer binding, regardless of their kind. An inner -variable binding for a symbol shadows an outer or global function binding, -and vice versa. +If a symbol has bindings both in the variable and function namespace in scope, +and is referenced by a dwim argument, this constitutes a conflict which is +resolved according to two rules. When nested scopes are concerned, then an +inner binding shadows an outer binding, regardless of their kind. An inner +variable binding for a symbol shadows an outer or global function binding, +and vice versa. If a symbol is bound to both a function and variable in the global namespace, then the variable binding is favored. @@ -11566,773 +11263,804 @@ argument is not given any special syntactic treatment at all) while the Lisp-2 foundation provides a traditional Lisp environment with its "natural hygiene". -.coNP Function @ identity +.SS* Sequencing, Selection and Iteration +.coNP Operators @ progn and @ prog1 .synb -.mets (identity << value ) +.mets (progn << form *) +.mets (prog1 << form *) .syne .desc The -.code identity -function returns its argument. +.code progn +operator evaluates forms in order, and returns the value +of the last form. The return value of the form +.code (progn) +is +.codn nil . -.TP* Notes: The -.code identity -function is useful as a functional argument, when a transformation -function is required, but no transformation is actually desired. +.code prog1 +operator evaluates forms in order, and returns the value +of the first form. The return value of the form +.code (prog1) +is +.codn nil . -.coNP Function @ eval +Various other operators such as +.code let +also arrange for the evaluation +of a body of forms, the value of the last of which is returned. +These operators are said to feature an implicit +.codn progn . + +.coNP Operator @ cond .synb -.mets (eval < form <> [ env ]) +.mets (cond >> {( test << form *)}*) .syne .desc The -.code eval -function treats the -.meta form -object as a Lisp expression, which is -evaluated. The side effects implied by the form are performed, and the value -which it produces is returned. The optional -.meta env -object specifies an environment for -resolving the function and variable references encountered in the expression. -If this argument is omitted -.code nil -then evaluation takes place in the global environment. +.code cond +operator provides a multi-branching conditional evaluation of +forms. Enclosed in the cond form are groups of forms expressed as lists. +Each group must be a list of at least one form. -See also: the -.code make-env -function. +The forms are processed from left to right as follows: the first form, +.metn test , +in each group is evaluated. If it evaluates true, then the remaining +forms in that group, if any, are also evaluated. Processing then terminates and +the result of the last form in the group is taken as the result of cond. +If +.meta test +is the only form in the group, then result of +.meta test +is taken +as the result of +.codn cond . -.coNP Function @ make-env +If the first form of a group yields +.codn nil , +then processing continues with the +next group, if any. If all form groups yield +.codn nil , +then the cond form yields +.codn nil . +This holds in the case that the syntax is empty: +.code (cond) +yields +.codn nil . + +.coNP Macros @, caseq @ caseql and @ casequal .synb -.mets (make-env >> [ variable-bindings >> [ function-bindings <> [ next-env ]]]) +.mets (caseq < test-form << normal-clause * <> [ else-clause ]) +.mets (caseql < test-form << normal-clause * <> [ else-clause ]) +.mets (casequal < test-form << normal-clause * <> [ else-clause ]) .syne .desc -The -.code make-env -function creates an environment object suitable as the -.code env -parameter. - -The -.meta variable-bindings -and -.meta function-bindings -parameters, if specified, -should be association lists, mapping symbols to objects. The objects in -.meta function-bindings -should be functions, or objects callable as functions. - -The -.meta next-env -argument, if specified, should be an environment. - -Note: bindings can also be added to an environment using the -.code env-vbind -and -.code env-fbind -functions. +These three macros arrange for the evaluation of of +.metn test-form , +whose value is then compared against the key or keys in each +.meta normal-clause +in turn. +When the value matches a key, then the remaining forms of +.meta normal-clause +are evaluated, and the value of the last form is returned; subsequent +clauses are not evaluated. When the value doesn't match any of the keys +of a +.meta normal-clause +then the next +.meta normal-clause +is tested. +If all these clauses are exhausted, and there is no +.metn else-clause , +then the value nil is returned. Otherwise, the forms in the +.meta else-clause +are evaluated, and the value of the last one is returned. -.coNP Functions @ env-vbind and @ env-fbind -.synb -.mets (env-vbind < env < symbol << value ) -.mets (env-fbind < env < symbol << value ) -.syne -.desc -These functions bind a symbol to a value in either the function or variable -space of environment -.codn env . +The syntax of a +.meta normal-clause +takes on these two forms: -Values established in the function space should be functions or objects that -can be used as functions such as lists, strings, arrays or hashes. +.cblk +.mets >> ( key << form *) +.cble -If -.meta symbol -already exists in the environment, in the given space, then its -value is updated with -.codn value . +where +.meta key +may be an atom which denotes a single key, or else a list +of keys. There is a restriction that the symbol +.code t +may not be used +as +.metn key . +The form +.code (t) +may be used as a key to match that symbol. -.coNP Function @ constantp +The syntax of an +.meta else-clause +is: + +.cblk +.mets (t << form *) +.cble + +which resembles a form that is often used as the final clause +in the +.code cond +syntax. + +The three forms of the case construct differ from what type of +test they apply between the value of +.meta test-form +and the keys. +The +.code caseq +macro generates code which uses the +.code eq +function's +equality. The +.code caseql +macro uses +.codn eql , +and +.code casequal +uses +.codn equal . + +.TP* Example +.cblk + (let ((command-symbol (casequal command-string + (("q" "quit") 'quit) + (("a" "add") 'add) + (("d" "del" "delete") 'delete) + (t 'unknown)))) + ...) +.cble + +.coNP Operator/function @ if .synb -.mets (constantp < form >> [ env ]) +.mets (if < cond < t-form <> [ e-form ]) +.mets [if < cond < then <> [ else ]] .syne .desc -The -.code constantp -function determines whether -.mode form -is a constant form, with respect to environment -.mode env . +There exist both an +.code if +operator and an +.code if +function. A list form with the symbol +.code if +in the fist position is interpreted as an invocation of the +.code if +operator. +The function can be accessed using the DWIM bracket notation and in other +ways. -If -.mode env -is absent, the global environment is used. The -.mode env -argument is used for macro-expanding -.modn form . - -Currently, -.code constantp -returns true for any form, which, after macro-expansion is a compound -form with the symbol -.code quote -in its first position, a non-symbolic atom, or one of the symbols -which evaluate to themselves and cannot be bound as variables. -These symbols are the keyword symbols, and the symbols -.code t -and +.code if +operator provides a simple two-way-selective evaluation control. +The +.meta cond +form is evaluated. If it yields true then +.meta t-form +is evaluated, and that form's return value becomes the return value of the +.codn if . +If +.meta cond +yields false, then +.meta e-form +is evaluated and its return value is taken to be that of +.codn if . +If +.meta e-form +is omitted, then the behavior is as if +.meta e-form +were specified as .codn nil . -In the future, -.code constantp -will be able to recognize more constant forms, such as calls to certain -functions whose arguments are constant forms. +The +.code if +function provides no evaluation control. All of arguments +are evaluated from left to right. If the +.meta cond +argument is true, then it +returns the +.meta then +argument, otherwise it returns the value of the +.meta else +argument if present, otherwise it returns +.codn nil . -.SS* Mutation of Syntactic Places -.coNP Macro @ set +.coNP Operator/function @ and .synb -.mets (set >> { place << new-value }*) +.mets (and << form *) +.mets [and << arg *] .syne .desc +There exist both an +.code and +operator and an +.code and +function. A list form with the +symbol +.code and +in the fist position is interpreted as an invocation of the +operator. The function can be accessed using the DWIM bracket notation and in +other ways. + The -.code set operator stores the values of expressions in places. It must -be given an even number of arguments. +.code and +operator provides three functionalities in one. It computes the +logical "and" function over several forms. It controls evaluation (a.k.a. +"short-circuiting"). It also provides an idiom for the convenient substitution +of a value in place of +.code nil +when some other values are all true. -If there are no arguments, then -.code set -does nothing and returns -.codn nil . +The +.code and +operator evaluates as follows. First, a return value is +established and initialized to the value +.codn t . +The +.metn form s, +if any, are +evaluated from left to right. The return value is overwritten with +the result of each form. Evaluation stops when all forms are exhausted, +or when +.code nil +is stored in the return value. +When evaluation stops, the operator yields the return value. -If there are two arguments, -.meta place -and -.metn new-value , -then -.meta place is evaluated to determine its storage location, -then -.meta new-value -is evaluated to determine the value to be stored there, -and then the value is stored in that location. Finally, -the value is also returned as the result value. +The +.code and +function provides no evaluation control; it receives all of its +arguments fully evaluated. If it is given no arguments, it returns +.codn t . +If it is given one or more arguments, and any of them are +.codn nil , +it returns +.codn nil . +Otherwise it returns the value of the last argument. -If there are more than two arguments, then -.code -set performs multiple assignments in left to right order. -Effectively, -.code (set v1 e1 v2 e2 ... vn en) -is precisely equivalent to -.codn (progn (set v1 e1) (set v2 e2) ... (set vn en)) . +.TP* Examples: +.cblk + (and) -> t + (and (> 10 5) (stringp "foo")) -> t + (and 1 2 3) -> 3 ;; short-hand for (if (and 1 2) 3). +.cble -.coNP Macro @ pset +.coNP Operator/function @ or .synb -.mets (pset >> { place << new-value }*) +.mets (or << form *) +.mets [or << arg *] .syne .desc -The syntax of -.code pset -is similar to that of -.codn set , -and the semantics is similar also in that zero or more places are -assigned zero or more values. In fact, if there are no arguments, or -if there is exactly one pair of arguments, -.code pset -is equivalent to -.codn set . +There exist both an +.code or +operator and an +.code or +function. A list form with the +symbol +.code or +in the fist position is interpreted as an invocation of the +operator. The function can be accessed using the DWIM bracket notation and in +other ways. -If there are two or more argument pairs, then all of the arguments -are evaluated first, in left-to-right order. No store takes place -until after every -.meta place -is determined, and every -.meta new-value -is calculated. During the calculation, the values to be stored -are retained in hidden, temporary locations. Finally, these values -are moved into the determined places. The rightmost value is returned -as the form's value. +The or operator provides three functionalities in one. It computes the +logical "or" function over several forms. It controls evaluation (a.k.a. +"short-circuiting"). The behavior of +.code or +also provides an idiom for the selection of the first non-nil value from a +sequence of forms. -The assignments thus appear to take place in parallel, and -.code pset -is capable of exchanging the values of a pair of places, or rotating -the values among three or more places. (However, there are more convenient -operators for this, namely -.code rotate -and -.codn swap ). +The +.code or +operator evaluates as follows. First, a return value is +established and initialized to the value +.codn nil . +The +.metn form s, +if any, +are evaluated from left to right. The return value is overwritten +with the result of each +.metn form . +Evaluation stops when all forms are +exhausted, or when a true value is stored into the return value. +When evaluation stops, the operator yields the return value. -.TP* Example: -.cblk - ;; exchange x and y - (pset x y y x) +The +.code or +function provides no evaluation control; it receives all of its +arguments fully evaluated. If it is given no arguments, it returns +.codn nil . +If all of its arguments are +.codn nil , +it also returns +.codn nil . +Otherwise, it +returns the value of the first argument which isn't +.codn nil . - ;; exchange elements 0 and 1; and 2 and 3 of vector v: - (let ((v (vec 0 10 20 30)) - (i -1)) - (pset [vec (inc i)] [vec (inc i)] - [vec (inc i)] [vec (inc i)]) - vec) - -> #(10 0 30 20) +.TP* Examples: +.cblk + (or) -> nil + (or 1 2) -> 1 + (or nil 2) -> 2 + (or (> 10 20) (stringp "foo")) -> t .cble -.coNP Macro @ zap +.coNP Macros @ when and @ unless .synb -.mets (zap < place <> [ new-value ]) +.mets (when < expression << form *) +.mets (unless < expression << form *) .syne .desc -The -.code zap -macro assigns -.meta new-value -to -.meta place -and returns the previous value of -.metn place . - +The when macro operator evaluates +.metn expression . If -.meta new-value -is missing, then -.code nil -is used. - -In more detail, first -.code place -is evaluated to determine the storage location. -Then, the location is accessed to retrieve the -previous value. Then, the -.code new-value -expression is evaluated, and that value is -placed into the storage location. -Finally, the previously retrieved value is returned. +.meta expression +yields +true, and there are additional forms, then each +.meta form +is evaluated. +The value of the last form is becomes the result value of the when form. +If there are no forms, then the result is +.codn nil . +The +.code unless +operator is similar to when, except that it reverses the +logic of the test. The forms, if any, are evaluated if, and only if +.meta expression +is false. -.coNP Macro @ flip +.coNP Macros @ while and @ until .synb -.mets (flip << place ) +.mets (while < expression << form *) +.mets (until < expression << form *) .syne .desc The -.code flip -macro toggles the boolean value stored in -.metn place . - +.code while +macro operator provides a looping construct. It evaluates +.metn expression . If -.meta place -previously held -.codn nil , -it is set to -.codn t , -and if it previously held a value other than +.meta expression +yields .codn nil , -it is set to +then the evaluation of the +.code while +form +terminates, producing the value .codn nil . +Otherwise, if there are additional forms, +then each +.meta form +is evaluated. Next, evaluation returns to +.metn expression , +repeating all of the previous steps. -.coNP Macros @ inc and @ dec -.synb -.mets (inc < place <> [ delta ]) -.mets (dec < place <> [ delta ]) -.syne -.desc The -.code inc -macro increments -.meta place -by adding -.meta delta -to its value. -If -.meta delta -is missing, the value used in its place the integer 1. +.code until +macro operator is similar to while, except that the until form +terminates when +.meta expression +evaluates true, rather than false. -First the -.meta place -argument is evaluated as a syntactic place to determine the location. -Then, the value currently stored in that location is retrieved. -Next, the -.meta delta -expression is evaluated. Its value is added to the previously retrieved -value as if by the -.code + -function. The resulting value is stored in the place, and returned. +These operators arrange for the evaluation of all their enclosed forms +in an anonymous block. Any of the +.metn form s, +or +.metn expression , +may use +the +.code return +operator to terminate the loop, and optionally to specify +a result value for the form. -The macro -.code dec -works exactly like -.code inc -except that addition is replaced by subtraction. The similarly defaulted -.meta delta -value is subtracted from the previous value of the place. +The only way these forms can yield a value other than +.code nil +is if the +.code return +operator is used to terminate the implicit anonymous block, +and is given an argument, which becomes the result value. -.coNP Macro @ swap +.coNP Macros @ while* and @ until* .synb -.mets (swap < left-place << right-place ) +.mets (while* < expression << form *) +.mets (until* < expression << form *) .syne .desc The -.code swap -macro exchanges the values of -.meta left-place +.code while* and -.meta right-place -and returns the value which is thereby transferred to -.metn right-place . - -First, -.meta left-place +.code until* +macros are similar, respectively, to the macros +.code while and -.meta right-place -are evaluated, in that order, to determine their locations. -Then the prior values are retrieved, exchanged and stored back. -The value stored in -.meta right-place -is also returned. +.codn until . -.coNP Macro @ push +They differ in one respect: they begin by evaluating the +.metn form -s +one time unconditionally, without first evaluating +.metn expression . +After this evaluation, the subsequent behavior is +like that of +.code while +or +.codn until . + +Another way to regard the behavior is that that these forms execute +one iteration unconditionally, without evaluating the termination test prior to +the first iteration. Yet another view is that these constructs relocate the +test from the "top of the loop" to the "bottom of the loop". + +.coNP Macro @ whilet .synb -.mets (push < item << place ) +.mets (whilet >> ({ sym | >> ( sym << init-form )}+) +.mets \ \ << body-form *) .syne .desc The -.code push -macro places -.meta item -at the head of the list stored in -.meta place -and returns the updated list which is stored back in -.metn place . +.code whilet +macro provides a construct which combines iteration with variable +binding. -First, the expression -.meta item -is evaluated to produce the push value. -Then, -.meta place -is evaluated to determine its storage location. -Next, the storage location is accessed to retrieve the -list value which is stored there. A new object is -produced as if by invoking -.code cons -function on the push value and list value. -This object is stored into the location, -and returned. +The evaluation of the form takes place as follows. First, fresh bindings are +established for +.metn sym -s +as if by the +.code let* +operator. +It is an error for the list of variable bindings to be empty. -.coNP Macro @ pop -.synb -.mets (pop << place ) -.syne -The -.code pop -macro removes an element from the list stored in -.meta place -and returns it. +After the establishment of the bindings, the the value of the +.meta sym +is tested. If the value is +.codn nil , +then +.code whilet +terminates. Otherwise, +.metn body-form -s +are evaluated in the scope of the variable bindings, and then +.code whilet +iterates from the beginning, again establishing fresh bindings for the +.metn sym -s, +and testing the value of the last +.metn sym . -First, -.meta place -is evaluated to determine the place. The place is accessed to -retrieve the original value. Then a new value is calculated, -as if by applying the -.code cdr -function to the old value. This new value is stored. -Finally, a return value is calculated and returned, as if by applying the -.code car -function to the original value. +All evaluation takes place in an anonymous block, which can be +terminated with the +.code return +operator. Doing so terminates the loop. +If the +.code whilet +loop is thus terminated by an explicit +.codn return , +a return value can be specified. Under normal termination, the return value is +.codn nil . -.coNP Macro @ pushnew -.synb -.mets (pushnew < item < place >> [ testfun <> [ keyfun ]]) -.syne -.desc -The -.code pushnew -macro inspects the list stored in -.metn place . -If the list already contains the item, then -it returns the list. Otherwise it creates a new list -with the item at the front and stores it back -into -.metn place , -and returns it. +.TP* Examples: +.cblk + ;; read lines of text from *std-input* and print them, + ;; until the end-of-stream condition: -First, the expression -.meta item -is evaluated to produce the push value. -Then, -.meta place -is evaluated to determine its storage location. -Next, the storage location is accessed to retrieve the -list value which is stored there. The list is -inspected to check whether it already contains the push -value, as if using the -.code member -function. If that is the case, the list -is returned and the operation finishes. -Otherwise, a new object is -produced as if by invoking -.code cons -function on the push value and list value. -This object is stored into the location -and returned. + (whilet ((line (get-line))) + (put-line line)) -.coNP Macro @ shift + ;; read lines of text from *std-input* and print them, + ;; until the end-of-stream condition occurs or + ;; a line is identical to the character string "end". + + (whilet ((line (get-line)) + (more (and line (not (equal line "end"))))) + (put-line line)) +.cble + +.coNP Macros @ iflet and @ whenlet .synb -.mets (shift << place + << shift-in-value) +.mets (iflet >> ({ sym | >> ( sym << init-form )}+) +.mets \ \ < then-form <> [ else-form ]) +.mets (whenlet >> ({ sym | >> ( sym << init-form )}+) +.mets \ \ << body-form *]) .syne .desc The -.code shift -macro treats one or more places as a "multi-place shift register". -The values of the places are shifted one place to the left. -The first (leftmost) place receives the value of the second place, -the second receives that of the third, and so on. -The last (rightmost) place receives -.meta shift-in-value -(which is not treated as a place, even if it is a syntactic place form). -The previous value of the first place is returned. +.code iflet +and +.code whenlet +macros combine the variable binding of +.code let* +with conditional evaluation of +.code if +and +.codn when , +respectively. -More precisely, all of the argument forms are evaluated left to right, in the -process of which the storage locations of the places are determined, -.meta shift-in-value -is reduced to its value. +The evaluation of these forms takes place as follows. First, fresh bindings are +established for +.metn sym -s +as if by the +.code let* +operator. +It is an error for the list of variable bindings to be empty. -The values stored in the places are sampled and saved. +Then, the last variable's value is tested. If it is not +.code nil +then the test is true, otherwise false. -Note that it is not specified whether the places are sampled in a separate -pass after the evaluation of the argument forms, or whether the -sampling is interleaved into the argument evaluation. This affects -the behavior in situations in which the evaluation of any of the -.meta place -forms, or of -.metn shift-in-value , -has the side effect of modifying later places. +In the case of the +.code iflet +operator, if the test is true, the operator evaluates +.meta then-form +and yields its value. Otherwise the test is false, and if the +optional +.meta else-form +is present, that is evaluated instead and its value is returned. +If this form is missing, then +.code nil +is returned. -Next, the places are updated by storing the saved value of the second -place into the first place, the third place into the second and so forth, -and the value of -.meta shift-in-value -into the last place. +In the case of the +.code whenlet +operator, if the test is true, then the +.metn body-form -s, +if any, are evaluated. The value of the last one is +returned, otherwise +.code nil +if the forms are missing. +If the test is false, then evaluation of +.metn body-form -s +is skipped, and +.code nil +is returned. -Finally, the saved original value of the first place is returned. +.TP* Examples: +.cblk + ;; dispose of foo-resource if present + (whenlet ((foo-res (get-foo-resource obj))) + (foo-shutdown foo-res) + (set-foo-resource obj nil)) -.coNP Macro @ rotate + ;; Contrast with: above, using when and let + (let ((foo-res (get-foo-resource obj))) + (when foo-res + (foo-shutdown foo-res) + (set-foo-resource obj nil))) + + ;; print frobosity value if it exceeds 150 + (whenlet ((fv (get-frobosity-value)) + (exceeds-p (> fv 150))) + (format t "frobosity value ~a exceeds 150\en" fv)) +.cble + +.coNP Macro @ ifa .synb -.mets (rotate << place *) +.mets (ifa < cond < then <> [ else ]) .syne .desc -Treats zero or more places as a "multi-place rotate register". -If there are no arguments, there is no effect and -.code nil -is returned. Otherwise, the last (rightmost) place receives -the value of the first (leftmost) place. The leftmost place -receives the value of the second place, and so on. -If there are two arguments, this equivalent to -.codn swap . -The prior value of the first place, which is the the value -rotated into the last place, is returned. - -More precisely, the -.meta place -arguments are evaluated left to right, -and the storage locations are thereby determined. The storage -locations are sampled, and then the sampled values are -stored back into the locations, but rotated by one place -as described above. The saved original value of the leftmost -.meta place -is returned. - -It is not specified whether the sampling of the original values -is a separate pass which takes place after the arguments -are evaluated, or whether this sampling it is interleaved into argument -evaluation. This affects -the behavior in situations in which the evaluation of any of the -.meta place -forms has the side effect of modifying the value stored in -a later -.meta place -form. +The +.code ifa +macro provides a anaphoric conditional operator resembling the +.code if +operator. Around the evaluation of the +.meta then +and +.meta else +forms, the symbol +.code it +is implicitly bound to a subexpression of +.metn cond , +providing a reference to that value, similar to the word +"it" in the English language, and similar anaphoric pronouns +in other languages. If +.code it +is bound to a place form, the binding is established +as if using the +.code placelet +operator. Otherwise, +.code it +is bound as an ordinary lexical variable to +the form's value. -.coNP Macro @ del -.synb -.mets (del << place ) -.syne -.desc The -.code del -macro requests the deletion of -.codn place . -If -.code place -doesn't support deletion, an exception is thrown. +.code ifa +macro imposes several restrictions on the +.meta cond +expression. Firstly, the +.meta cond +expression must be either an atom, or a function call form: +a compound expression with a symbol in the leftmost position +which resolves to a function. Otherwise the +.code ifa +macro invocation is ill-formed. -First -.code place -is evaluated, thereby determining its location. -Then the place is accessed to retrieve its value. -The place is then subject to deletion. Finally, the -previously retrieved value is returned. +Secondly, if the +.meta cond +expression is a function call with two or more arguments, +at most one of them may be an it-candidate: an expression +viable for having its value or storage location bound to the +.code it +symbol. If there are two or more it-candidates, the +.code ifa +expression is ill-formed. -Precisely what deletion means depends on the kind of place. -The built-in places in \*(TL have deletion semantics which are -intended to be unsurprising to the programmer familiar with the -data structure which holds the place. +If +.meta cond +is an atom, or a function call expression with no arguments, +then the +.code it +symbol is not bound. Effectively, +.code ifa +macro behaves like the ordinary +.code if +operator. -Generally, if a place denotes the element of a sequence, then deletion of the -place implies deletion of the element, and deletion of the element implies that -the gap produced by the element is closed. The deleted element is effectively -replaced by its successor, that successor by its successor and so on. If a -place denotes a value stored in a dynamic data set such as a hash table, -then deletion of that place implies deletion of the entry which holds -that value. If the entry is identified by a key, that key is also removed. +If +.meta cond +is an invocation of the functions +.code not +or +.codn null , -.SS* Binding and Iteration -.coNP Operators @ defvar and @ defparm -.synb -.mets (defvar < sym <> [ value ]) -.mets (defparm < sym << value ) -.syne +If +.meta cond +is a function call with exactly one argument, then the +.code it +variable is bound to the value of that argument, except when +the function being called is +.codn not , +.code null , +or +.codn false . +That special situation is rewritten according to the following pattern: -.desc +.cblk +.mets (ifa (not << expr ) < then << else ) -> (ifa < expr < else << then ) +.cble -The -.code defvar -operator binds a name in the variable namespace of the global environment. -Binding a name means creating a binding: recording, in some namespace of some -environment, an association between a name and some named entity. In the -case of a variable binding, that entity is a storage location for a value. -The value of a variable is that which has most recently been written into the -storage location, and is also said to be a value of the binding, or stored -in the binding. +Note the reversal of +.meta then +and +.metn else . -If the variable named -.meta sym -already exists in the global environment, the -form has no effect; the -.meta value -form is not evaluated, and the value of the -variable is unchanged. +If +.meta cond +is a function call with two or more arguments, then it is only +well-formed if at most one of those arguments is an it-candidates. +If there is one such argument, then the +.code it +variable is bound to it. Otherwise the variable is bound +to the leftmost argument expression, regardless of whether that +argument expression is an it-candidate. -If the variable does not exist, then a new binding is introduced, with a value -given by evaluating the -.meta value -form. If the form is absent, the variable is initialized -to -.codn nil . +An it-candidate is any expression which is not a constant expression +according to the +.code constantp +function, and not a symbol. + +In all other regards, the +.code ifa +macro behaves similarly to +.codn if . The -.meta value -form is evaluated in the environment -in which the -.code defvar -form occurs, not necessarily in the global environment. +.meta cond +expression is evaluated, and, if applicable, +the value of, or storage location denoted by the appropriate argument is +captured and bound to the variable +.code it +whose scope extends over the +.meta then +form, as well as over +.metn else , +if present. -The symbols -.code t -and +If +.meta cond +yields a true value, then +.meta then +is evaluated and the resulting value is returned, otherwise +.meta else +is evaluated if present and its value is returned. +A missing +.meta else +is treated as if it were the .code nil -may not be used as variables, and neither -can be keyword symbols: symbols denoted by a leading colon. - -In addition to creating a binding, the -.code defvar -operator also marks -.meta sym -as the name of a special variable. This changes what it means to bind -that symbol in a lexical binding construct such as the -.code let -operator, or a function parameter list. See the section "Special Variables" far -above. +form. -The -.code defparm -operator behaves like -.code defvar -when a variable named -.meta sym -doesn't already exist. +.TP* Examples: +.cblk + (ifa t 1 0) -> 1 -If -.meta sym -already denotes a variable binding in the global namespace, -.code defparm -evaluates the -.meta value -form and assigns the resulting value to the variable. + (let ((x 6) (y 49)) + (ifa (> y (* x x)) ;; it binds to (* x x) + (list it))) + -> (36) -The following equivalence holds: + (ifa (evenp 4) + (list it)) + -> (4) -.cblk - (defparm x y) <--> (prog1 (defvar x) (set x y)) + (ifa (not (oddp 4)) + (list it)) + -> (4) .cble -The -.code defvar -and -.code defparm -operators return -.metn sym . - -.coNP Operators @ let and @ let* +.coNP Macro @ conda .synb -.mets (let >> ({ sym | >> ( sym << init-form )}*) << body-form *) -.mets (let* >> ({ sym | >> ( sym << init-form )}*) << body-form *) +.mets (conda >> {( test << form *)}*) .syne .desc The -.code let -and -.code let* -operators introduce a new scope with variables and -evaluate forms in that scope. The operator symbol, either -.code let -or -.codn let* , -is followed by a list which can contain any mixture of variable -name symbols, or -.cblk -.meti >> ( sym << init-form ) -.cble -pairs. A symbol -denotes the name of variable to be instantiated and initialized -to the value -.codn nil . -A symbol specified with an init-form denotes -a variable which is initialized from the value of the -.metn init-form . - -The symbols -.code t -and -.code nil -may not be used as variables, and neither -can be keyword symbols: symbols denoted by a leading colon. - -The difference between -.code let -and -.code let* -is that in -.codn let* , -later -.codn init-form s -have visibility over the variables established by earlier variables -in the same let* construct. In plain -.codn let , -the variables are not visible to any of the -.metn init-form s. - -When the variables are established, then the -.metn body-form s -are evaluated in order. The value of the last -.meta body-form -becomes the return value of the -.codn let . - -If there are no -.metn body-form s, -then the return value -.code nil -is produced. +.code conda +operator provides a multi-branching conditional evaluation of +forms, similarly to the +.code cond +operator. Enclosed in the cond form are groups of forms expressed as lists. +Each group must be a list of at least one form. -The list of variables may be empty. +The +.code conda +operator is anaphoric: it expands into a nested structure of zero or more +.code ifa +invocations, according to these patterns: -.TP* Examples: .cblk - (let ((a 1) (b 2)) (list a b)) -> (1 2) - (let* ((a 1) (b (+ a 1))) (list a b (+ a b))) -> (1 2 3) - (let ()) -> nil - (let (:a nil)) -> error, :a and nil can't be used as variables + (conda) -> nil + (conda (x y ...) ...) -> (ifa x (progn y ...) (conda z ...)) .cble -.coNP Operators @ for and @ for* +Thus, +.code conda +inherits all the restrictions on the +.meta test +expressions from +.codn ifa , +as well as the anaphoric +.code it +variable feature. + +.coNP Macro @ dotimes .synb -.mets ({for | for*} >> ({ sym | >> ( sym << init-form )}*) -.mets \ \ \ \ \ \ \ \ \ \ \ \ \ >> ([ test-form << result-form *]) -.mets \ \ \ \ \ \ \ \ \ \ \ \ \ <> ( inc-form *) -.mets \ \ << body-form *) +.mets (dotimes >> ( var < count-form <> [ result-form ]) << body-form *) .syne .desc - The -.code for -and -.code for* -operators combine variable binding with loop iteration. -The first argument is a list of variables with optional initializers, -exactly the same as in the -.code let -and -.code let* -operators. Furthermore, the -difference between -.code for -and -.code for* -is like that between -.code let -and -.code let* -with regard to this list of variables. +.code dotimes +macro implements a simple counting loop. +.meta var +is established as a variable, and initialized to zero. +.meta count-form +is evaluated one time to produce a limiting value, which should be a number. +Then, if the value of +.meta var +is less than the limiting value, the +.metn body-form -s +are evaluated, +.meta var +is incremented by one, and the process repeats with a new comparison +of +.meta var +against the limiting value possibly leading to another evaluation of +the forms. -The -.code for -and -.code for* -operators execute these steps: -.RS -.IP 1. -Establish bindings for the specified variables similarly to -.code let -and -.codn let* . -The variable bindings are visible over the -.metn test-form , -each -.metn result-form , -each -.meta inc-form -and each -.metn body-form . -.IP 2. -Establish an anonymous block over the remaining forms, allowing -the -.code return -operator to be used to terminate the loop. -.IP 3. -Evaluate -.metn test-form . If -.meta test-form -yields -.codn nil , -then the loop terminates. Each +.meta var +is found to equal or exceed the limiting value, then the loop terminates. + +When the loop terminates, its return value is +.code nil +unless a .meta result-form -is evaluated, and the value of the last of these -forms is is the result value of the loop. -If there are no -.metn result-form s -then the result value is -.codn nil . -If the -.meta test-form -is omitted, then the test -is taken to be true, and the loop does not terminate. -.IP 4. -Otherwise, if -.meta test-form -yields true, then each -.meta body-form -is evaluated in turn. Then, each -.code inc-form -is evaluated in turn and processing resumes at step 2. -.RE +is present, in which case the value of that form specifies the return value. -.IP -Furthermore, the -.code for -and -.code for* -operators establish an anonymous block, -allowing the -.code return -operator to be used to terminate at any point. +.metn body-form -s +as well as +.meta result-form +are evaluated in the scope in which the binding of +.meta var +is visible. .coNP Operators @, each @, each* @, collect-each @, collect-each* @ append-each and @ append-each* .synb @@ -12449,236 +12177,319 @@ of their lists. 10 is even .cble -.SS* Function Objects and Named Functions - -.coNP Operator @ defun +.coNP Operators @ for and @ for* .synb -.mets (defun < name <> ( param * [: << opt-param *] [. << rest-param ]) -.mets \ \ << body-form ) +.mets ({for | for*} >> ({ sym | >> ( sym << init-form )}*) +.mets \ \ \ \ \ \ \ \ \ \ \ \ \ >> ([ test-form << result-form *]) +.mets \ \ \ \ \ \ \ \ \ \ \ \ \ <> ( inc-form *) +.mets \ \ << body-form *) .syne .desc -The -.code defun -operator introduces a new function in the global function namespace. -The function is similar to a lambda, and has the same parameter syntax -and semantics as the -.code lambda -operator. - -Unlike in -.codn lambda , -the -.metn body-form s -of a -.code defun -are surrounded by a block. -The name of this block is the same as the name of the function, making it -possible to terminate the function and return a value using -.cblk -.meti (return-from < name << value ). -.cble -For more information, see the definition of the block operator. - -A function may call itself by name, allowing for recursion. -.coNP Operator @ lambda -.synb -.mets (lambda <> ( param * [: << opt-param *] [. << rest-param ]) -.mets \ \ << body-form ) -.mets (lambda < rest-param -.mets \ \ << body-form ) -.syne -.desc The -.code lambda -operator produces a value which is a function. Like in most other -Lisps, functions are objects in \*(TL. They can be passed to functions as -arguments, returned from functions, aggregated into lists, stored in variables, -et cetera. - -The first argument of -.code lambda -is the list of parameters for the function. It -may be empty, and it may also be an improper list (dot notation) where the -terminating atom is a symbol other than -.codn nil . -It can also be a single symbol. - -The second and subsequent arguments are the forms making up the function body. -The body may be empty. - -When a function is called, the parameters are instantiated as variables that -are visible to the body forms. The variables are initialized from the values of -the argument expressions appearing in the function call. +.code for +and +.code for* +operators combine variable binding with loop iteration. +The first argument is a list of variables with optional initializers, +exactly the same as in the +.code let +and +.code let* +operators. Furthermore, the +difference between +.code for +and +.code for* +is like that between +.code let +and +.code let* +with regard to this list of variables. -The dotted notation can be used to write a function that accepts -a variable number of arguments. There are two ways write a function that -accepts only a variable argument list and no required arguments: - -.cblk -.mets (lambda (. << rest-param ) ...) -.mets (lambda < rest-param ...) -.cble - -(These notations are syntactically equivalent because the list notation -.code (. X) -actually denotes the object -.code X -which isn't wrapped in any list). - -The keyword symbol -.code : -(colon) can appear in the parameter list. This is -the symbol in the keyword package whose name is the empty string. This -symbol is treated specially: it serves as a separator between -required parameters and optional parameters. Furthermore, the -.code : -symbol has a role to play in function calls: it can be specified as an argument -value to an optional parameter by which the caller indicates that the -optional argument is not being specified. It will be processed exactly -that way. - -An optional parameter can also be written in the form -.cblk -.meti >> ( name < expr <> [ sym ]). -.cble -In this situation, if the call does not specify a value for the parameter -(or specifies a value as the keyword -.code : -(colon)) then the parameter takes on the -value of the expression -.metn expr . +The +.code for +and +.code for* +operators execute these steps: +.RS +.IP 1. +Establish bindings for the specified variables similarly to +.code let +and +.codn let* . +The variable bindings are visible over the +.metn test-form , +each +.metn result-form , +each +.meta inc-form +and each +.metn body-form . +.IP 2. +Establish an anonymous block over the remaining forms, allowing +the +.code return +operator to be used to terminate the loop. +.IP 3. +Evaluate +.metn test-form . If -.meta sym -is specified, then -.meta sym -will be -introduced as an additional binding with a boolean value which indicates -whether or not the optional parameter had been specified by the caller. +.meta test-form +yields +.codn nil , +then the loop terminates. Each +.meta result-form +is evaluated, and the value of the last of these +forms is is the result value of the loop. +If there are no +.metn result-form s +then the result value is +.codn nil . +If the +.meta test-form +is omitted, then the test +is taken to be true, and the loop does not terminate. +.IP 4. +Otherwise, if +.meta test-form +yields true, then each +.meta body-form +is evaluated in turn. Then, each +.code inc-form +is evaluated in turn and processing resumes at step 2. +.RE -The initializer expressions are evaluated an environment in which -all of the previous parameters are visible, in addition to the surrounding -environment of the lambda. For instance: +.IP +Furthermore, the +.code for +and +.code for* +operators establish an anonymous block, +allowing the +.code return +operator to be used to terminate at any point. -.cblk - (let ((default 0)) - (lambda (str : (end (length str)) (counter default)) - (list str end counter))) -.cble -In this -.codn lambda , -the initializing expression for the optional parameter -end is -.codn (length str) , -and the -.code str -variable it refers to is the previous -argument. The initializer for the optional variable counter is -the expression default, and it refers to the binding established -by the surrounding let. This reference is captured as part of the -.codn lambda 's -lexical closure. +.coNP Operator @ block +.synb +.mets (block < name << body-form *) +.syne +.desc +The +.code block +operator introduces a named block around the execution of +some forms. The +.meta name +argument must be a symbol. Since a block name is not +a variable binding, keyword symbols are permitted, and so are the symbols +.code t +and +.codn nil . +A block named by the symbol nil is slightly special: it is +understood to be an anonymous block. -.TP* Examples: -.IP "Counting function:" -This function, which takes no arguments, captures the -variable -.codn counter . -Whenever this object is called, it increments -.code counter -by -.code 1 -and returns the incremented value. +Blocks in \*(TL have dynamic scope. This means that the following +situation is allowed: .cblk - (let ((counter 0)) - (lambda () (inc counter))) + (defun func () (return-from foo 42)) + (block foo (func)) .cble -.IP "Function that takes two or more arguments:" -The third and subsequent arguments are aggregated into a list passed as the -single parameter -.codn z : +The function can return from the +.code foo +block even though the +.code foo +block +does not lexically surround +.codn foo . -.cblk - (lambda (x y . z) (list 'my-arguments-are x y z)) -.cble +Thus blocks in \*(TL provide dynamic non-local returns, as well +as returns out of lexical nesting. -.IP "Variadic function:" +.TP* "Dialect Note:" +In Common Lisp, blocks are lexical. A separate mechanism consisting of +catch and throw operators performs non-local transfer based on symbols. +The \*(TL example: .cblk - (lambda args (list 'my-list-of-arguments args)) + (defun func () (return-from foo 42)) + (block foo (func)) .cble -.IP "Optional arguments:" +is not allowed in Common Lisp, but can be transliterated to: .cblk - [(lambda (x : y) (list x y)) 1] -> (1 nil) - [(lambda (x : y) (list x y)) 1 2] -> (1 2) + (defun func () (throw 'foo 42)) + (catch 'foo (func)) .cble -.coNP Function @ call +Note that foo is quoted in CL. This underscores the dynamic nature of +the construct. +.code throw +itself is a function and not an operator. +.coNP Operators @ return and @ return-from .synb -.mets (call < function << argument *) +.mets (return <> [ value ]) +.mets (return-from < name <> [ value ]) .syne .desc The -.code call -function invokes -.metn function , -passing it the given arguments, if any. +.code return +operator must be dynamically enclosed within an anonymous +block (a block named by the symbol +.codn nil ). +It immediately terminates the +evaluation of the innermost anonymous block which encloses it, causing +it to return the specified value. If the value is omitted, the anonymous +block returns +.codn nil . -.TP* Examples: -Apply arguments -.code 1 2 -to a -.code lambda -which adds them to produce -.codn 3 : +The +.code return-from +operator must be dynamically enclosed within a named block +whose name matches the +.meta name +argument. It immediately terminates the +evaluation of the innermost such block, causing it to return the specified +value. If the value is omitted, that block returns +.codn nil . +.TP* Example: .cblk - (call (lambda (a b) (+ a b)) 1 2) + (block foo + (let ((a "abc\en") + (b "def\en")) + (pprint a *stdout*) + (return-from foo 42) + (pprint b *stdout*))) .cble -Useless use of -.code call -on a named function; equivalent to -.codn (list 1 2) : +Here, the output produced is +.strn "abc" . +The value of +.code b +is not printed +because. +.code return-from +terminates block +.codn foo , +and so the second pprint form is not evaluated. -.cblk - (call (fun list) 1 2) -.cble +.SS* Evaluation -.coNP Operator @ fun +.coNP Function @ eval .synb -.mets (fun << function-name ) +.mets (eval < form <> [ env ]) .syne .desc The -.code fun -operator retrieves the function object corresponding to a named -function in the current lexical environment. +.code eval +function treats the +.meta form +object as a Lisp expression, which is +evaluated. The side effects implied by the form are performed, and the value +which it produces is returned. The optional +.meta env +object specifies an environment for +resolving the function and variable references encountered in the expression. +If this argument is omitted +.code nil +then evaluation takes place in the global environment. + +See also: the +.code make-env +function. +.coNP Function @ constantp +.synb +.mets (constantp < form >> [ env ]) +.syne +.desc The -.meta function-name -is a symbol denoting a named function: a built in -function, or one defined by -.codn defun . +.code constantp +function determines whether +.mode form +is a constant form, with respect to environment +.mode env . -Note: the -.code fun -operator does not see macro bindings. It is possible to -retrieve a global macro expander using -.codn symbol-function . +If +.mode env +is absent, the global environment is used. +The +.mode env +argument is used for macro-expanding +.modn form . -.TP* "Dialect Note:" -A lambda expression is not a function name in \*(TL. The -syntax -.code (fun (lambda ...)) -is invalid. +Currently, +.code constantp +returns true for any form, which, after macro-expansion is a compound +form with the symbol +.code quote +in its first position, a non-symbolic atom, or one of the symbols +which evaluate to themselves and cannot be bound as variables. +These symbols are the keyword symbols, and the symbols +.code t +and +.codn nil . + +In the future, +.code constantp +will be able to recognize more constant forms, such as calls to certain +functions whose arguments are constant forms. + +.coNP Function @ make-env +.synb +.mets (make-env >> [ variable-bindings >> [ function-bindings <> [ next-env ]]]) +.syne +.desc +The +.code make-env +function creates an environment object suitable as the +.code env +parameter. + +The +.meta variable-bindings +and +.meta function-bindings +parameters, if specified, +should be association lists, mapping symbols to objects. The objects in +.meta function-bindings +should be functions, or objects callable as functions. + +The +.meta next-env +argument, if specified, should be an environment. + +Note: bindings can also be added to an environment using the +.code env-vbind +and +.code env-fbind +functions. + +.coNP Functions @ env-vbind and @ env-fbind +.synb +.mets (env-vbind < env < symbol << value ) +.mets (env-fbind < env < symbol << value ) +.syne +.desc +These functions bind a symbol to a value in either the function or variable +space of environment +.codn env . + +Values established in the function space should be functions or objects that +can be used as functions such as lists, strings, arrays or hashes. + +If +.meta symbol +already exists in the environment, in the given space, then its +value is updated with +.codn value . +.SS* Global Environment .coNP Accessors @ symbol-function and @ symbol-value .synb .mets (symbol-function << symbol ) @@ -12882,96 +12693,6 @@ if is an interpreted function, otherwise it returns .codn nil . -.coNP Macros @ flet and @ labels -.synb -.mets (flet >> ({( name < param-list << function-body-form *)}*) -.mets \ \ << body-form *) - -.mets (labels >> ({( name < param-list << function-body-form *)}*) -.mets \ \ << body-form *) -.syne -.desc -The -.code flet -and -.code labels -macros bind local, named functions in the lexical scope. -The difference between -.code flet -and -.code labels -is that a function defined by -.code labels -can see itself, and therefore recurse directly by name. Moreover, if multiple -functions are defined by the same labels construct, they all see each other. -By contrast, a -.codn flet -defined -function does not have itself in scope and cannot recurse. -Multiple functions in the same -.code flet -do not have each other's names in their scopes. - -More formally, the -.metn function-body-form -s -and -.meta param-list -of the functions defined by -.code labels -are in a scope in which all of the function -names being defined by that same -.code labels -construct are visible. - -Under both -.code labels -and -.codn flet , -the local functions that are defined are -lexically visible to the main -.metn body-form -s. - -Note that -.code labels -and -.code flet -are properly scoped with regard to macros. -During macro expansion, function bindings introduced by these -macro operators shadow macros defined by -.code macrolet -and -.codn defmacro . - -Furthermore, function bindings introduced by -.code labels -and -.code flet -also shadow symbol macros defined by -.codn symacrolet , -when those symbol macros occur as arguments of a -.code dwim -form. - -See also: the -.code macrolet -operator. - -.TP* Examples: -.cblk - ;; Wastefully slow algorithm for determining evenness. - ;; Note: - ;; - mutual recursion between labels-defined functions - ;; - inner is-even bound by labels shadows the outer - ;; one bound by defun so the (is-even n) call goes - ;; to the local function. - - (defun is-even (n) - (labels ((is-even (n) - (if (zerop n) t (is-odd (- n 1)))) - (is-odd (n) - (if (zerop n) nil (is-even (- n 1))))) - (is-even n))) -.cblk - .SS* Object Type And Equivalence .coNP Function @ typeof .synb @@ -13028,6 +12749,21 @@ A bignum integer: arbitrary precision integer that is heap-allocated. There are additional kinds of objects, such as streams. +.coNP Function @ identity +.synb +.mets (identity << value ) +.syne +.desc +The +.code identity +function returns its argument. + +.TP* Notes: +The +.code identity +function is useful as a functional argument, when a transformation +function is required, but no transformation is actually desired. + .coNP Functions @, null @, not and @ false .synb .mets (null << value ) @@ -13277,40 +13013,255 @@ Certain object types have a custom .code equal function. -.SS* Basic List Library -.coNP Function @ cons +.coNP Function @ less .synb -.mets (cons < car-value << cdr-value ) +.mets (less < left-obj << right-obj ) +.mets (less < obj << obj *) .syne .desc The -.code cons -function allocates, initializes and returns a single cons cell. -A cons cell has two fields called -.code car -and -.codn cdr , -which are accessed by -functions of the same name, or by the functions -.code first -and -.codn rest , -which are synonyms for these. +.code less +function, when called with two arguments, determines whether +.meta left-obj +compares less than +.meta right-obj +in a generic way which handles arguments of various types. -Lists are made up of conses. A (proper) list is either the symbol -.code nil -denoting an empty list, or a cons cell which holds the first item of -the list in its -.codn car , -and the list of the remaining items in -.codn cdr . -The expression -.code (cons 1 nil) -allocates and returns a single cons cell which denotes the one-element -list -.codn (1) . -The -.code cdr +The argument syntax of +.code less +is generalized. It can accept one argument, in which case it unconditionally +returns +.code t +regardless of that argument's value. If more than two arguments are +given, then +.code less +generalizes in a way which can be described by the following equivalence +pattern, with the understanding that each argument expression +is evaluated exactly once: + +.cblk + (less a b c) <--> (and (less a b) (less b c)) + (less a b c d) <--> (and (less a b) (less b c) (less c d)) +.cble + +The +.code less +function is used as the default for the +.meta lessfun +argument of the functions +.code sort +and +.codn merge , +as well as the +.meta testfun +argument of the +.code pos-min +and +.codn find-min . + +The +.code less +function is capable of comparing numbers, characters, symbols, strings, +as well as lists and vectors of these. + +If both arguments are the same object so that +.cblk +.meti (eq < left-obj << right-obj ) +.cble +holds true, then the function returns +.code nil +regardless of the type of +.metn left-obj , +even if the function doesn't handle comparing different instances +of that type. In other words, no object is less than itself, no matter +what it is. + +If both arguments are numbers or characters, they are compared as if using the +.code < function. + +If both arguments are strings, they are compared as if using the +.code string-lt +function. + +If both arguments are symbols, then their names are compared in +their place, as if by the +.code string-lt +function. + +If both arguments are conses, then they are compared as follows: +.RS +.IP 1. +The +.code less +function is recursively applied to the +.code car +fields of both arguments. If it yields true, then +.meta left-obj +is deemed to be less than +.metn right-obj . +.IP 2. +Otherwise, if the +.code car +fields are unequal under +the +.code equal +function, +.code less +returns +.codn nil. +.IP 3. +If the +.code car +fields are +.code equal +then +.code less +is recursively applied to the +.code cdr +fields of the arguments, and the result of that comparison is returned. +.RE + +.IP +This logic performs a lexicographic comparison on ordinary lists such +that for instance +.code (1 1) +is less than +.code (1 1 1) +but not less than +.code (1 0) +or +.codn (1) . + +Note that the empty +.code nil +list nil compared to a cons is handled by type-based precedence, described +below. + +If the arguments are vectors, they are compared lexicographically, similar +to strings. Corresponding elements, starting with element 0, of the +vectors are compared until an index position is found where the vectors +differ. If this differing position is beyond the end of one of the two vectors, +then the shorter vector is considered to be lesser. Otherwise, the result +of +.code less +is the outcome of comparing those differing elements themselves +with +.codn less . + +If the two arguments are of the above types, but of mutually different types, +then +.code less +resolves the situation based on the following precedence: numbers and +characters are less than strings, which are less than symbols, +which are less than conses, which are less than vectors. + +Note that since +.code nil +is a symbol, it is ranked lower than a cons. This interpretation ensures +correct behavior when +.code nil +is regarded as an empty list, since the empty list is lexicographically prior to +a nonempty list. + +Finally, if either of the arguments has a type other than the above discussed +types, the situation is an error. + +.coNP Function @ greater +.synb +.mets (greater < left-obj << right-obj ) +.mets (greater < obj << obj *) +.syne +.desc +The +.code +greater +function is equivalent to +.code less +with the arguments reversed. That is to say, the following +equivalences hold: + +.cblk + (greater a <--> (less a) <--> t + (greater a b) <--> (less b a) + (greater a b c ...) <--> (less ... c b a) +.cble + +The +.code greater +function is used as the default for the +.meta testfun +argument of the +.code pos-max +and +.code find-max +functions. + +.coNP Functions @ lequal and @ gequal +.synb +.mets (lequal < obj << obj *) +.mets (gequal < obj << obj *) +.syne +.desc +The functions +.code lequal +and +.code gequal +are similar to +.code less +and +.code greater +respectively, but differ in the following respect: +when called with two arguments which compare true under the +.code equal +function, the +.code lequal +and +.code gequal +functions return +.codn t . + +When called with only one argument, both functions return +.code t +and both functions generalize to three or more arguments +in the same way as do +.code less +and +.codn greater . + +.SS* List Manipulation +.coNP Function @ cons +.synb +.mets (cons < car-value << cdr-value ) +.syne +.desc +The +.code cons +function allocates, initializes and returns a single cons cell. +A cons cell has two fields called +.code car +and +.codn cdr , +which are accessed by +functions of the same name, or by the functions +.code first +and +.codn rest , +which are synonyms for these. + +Lists are made up of conses. A (proper) list is either the symbol +.code nil +denoting an empty list, or a cons cell which holds the first item of +the list in its +.codn car , +and the list of the remaining items in +.codn cdr . +The expression +.code (cons 1 nil) +allocates and returns a single cons cell which denotes the one-element +list +.codn (1) . +The +.code cdr is .codn nil , so there are no additional items. @@ -14387,31 +14338,86 @@ structure is itself lazy. (flatten '(((()) ()))) -> nil .cble -.coNP Functions @, memq @ memql and @ memqual +.coNP Function @ tree-find .synb -.mets (memq < object << list ) -.mets (memql < object << list ) -.mets (memqual < object << list ) +.mets (tree-find < obj < tree << test-function ) .syne .desc The -.codn memq , -.code memql -and -.code memqual -functions search -.meta list -for a member -which is, respectively, -.codn eq , -.code eql -or -.coe equal -to -.metn object . -(See the -.codn eq , +.code tree-find +function searches +.meta tree +for an occurrence of +.metn obj . +Tree can be +any atom, or a cons. If +.meta tree +it is a cons, it is understood to be a proper +list whose elements are also trees. + +The equivalence test is performed by +.meta test-function +which must take two +arguments, and has conventions similar to +.codn eq , +.code eql +or +.codn equal . + +.code tree-find +works as follows. If +.meta tree +is equivalent to +.meta obj +under +.metn test-function , +then +.code t +is returned to announce a successful finding. +If this test fails, and +.meta tree +is an atom, +.code nil +is returned immediately to +indicate that the find failed. Otherwise, +.meta tree +is taken to be a proper list, +and tree-find is recursively applied to each element of the list in turn, using +the same +.meta obj +and +.meta test-function +arguments, stopping at the first element +which returns a +.cod2 non- nil +value. + +.coNP Functions @, memq @ memql and @ memqual +.synb +.mets (memq < object << list ) +.mets (memql < object << list ) +.mets (memqual < object << list ) +.syne +.desc + +The +.codn memq , +.code memql +and +.code memqual +functions search +.meta list +for a member +which is, respectively, +.codn eq , +.code eql +or +.coe equal +to +.metn object . +(See the +.codn eq , .code eql and .code equal @@ -14479,16144 +14485,16357 @@ is returned, otherwise what is returned is the suffix of .meta sequence which begins with the matching element. -.coNP Functions @, remq @ remql and @ remqual +.coNP Functions @ conses and @ conses* .synb -.mets (remq < object << list ) -.mets (remql < object << list ) -.mets (remqual < object << list ) +.mets (conses << list ) +.mets (conses* << list ) .syne .desc +These functions return a list whose elements are the conses which make +up +.metn list . The -.codn remq , -.code remql -and -.code remqual -functions produce a new list based on -.metn list , -removing the items which are -.codn eq , -.code eql -or -.code equal -to -.metn object . +.code conses* +function does this in a lazy way, avoiding the +computation of the entire list: it returns a lazy list of the conses of +.metn list . +The +.code conses +function computes the entire list before returning. The input .meta list -is unmodified, but the returned list may share substructure -with it. If no items are removed, it is possible that the return value -is +may be proper or improper. + +The first cons of .meta list -itself. +is that +.meta list +itself. The second cons is the rest +of the list, or +.cblk +.meti (cdr << list ). +.cble +The third cons is +.cblk +.meti (cdr (cdr << list )) +.cble +and so on. -.coNP Functions @, remq @ remql* and @ remqual* -.synb -.mets (remq* < object << list ) -.mets (remql* < object << list ) -.mets (remqual* < object << list ) -.syne -.desc -The -.codn remq* , -.code remql* -and -.code remqual* -functions are lazy versions of -.codn remq , -.code remql -and -.codn remqual . -Rather than computing the entire new list -prior to returning, these functions return a lazy list. +.TP* Example: +.cblk + (conses '(1 2 3)) -> ((1 2 3) (2 3) (3)) +.cble -Caution: these functions can still get into infinite looping behavior. -For instance, in -.codn (remql* 0 (repeat '(0))) , -.code remql -will keep consuming -the -.code 0 -values coming out of the infinite list, looking for the first item that -does not have to be deleted, in order to instantiate the first lazy value. +.TP* "Dialect Note:" + +These functions are useful for simulating the +.code maplist +function found in other dialects like Common Lisp. + +\*(TL's +.code (conses x) +can be expressed in Common Lisp as +.codn (maplist #'identity x) . + +Conversely, the Common Lisp operation +.code (maplist function list) +can be computed in \*(TL as +.codn (mapcar function (conses list)) . + +More generally, the Common Lisp operation -.TP* Examples: .cblk - ;; Return a list of all the natural numbers, excluding 13, - ;; then take the first 100 of these. - ;; If remql is used, it will loop until memory is exhausted, - ;; because (range 1) is an infinite list. + (maplist function list0 list1 ... listn) +.cble - [(remql* 13 (range 1)) 0..100] +can be expressed as: + +.cblk + (mapcar function (conses list0) + (conses list1) ... (conses listn)) .cble -.coNP Functions @, countqual @ countql and @ countq +.SS* Association Lists + +Association lists are ordinary lists formed according to a special convention. +Firstly, any empty list is a valid association list. A non-empty association +list contains only cons cells as the key elements. These cons cells are +understood to represent key/value associations, hence the name "association +list". + +.coNP Function @ assoc .synb -.mets (countq < object << list ) -.mets (countql < object << list ) -.mets (countqual < object << list ) +.mets (assoc < key << alist ) .syne .desc The -.codn countq , -.code countql -and -.code countqual -functions count the number of objects -in -.meta list -which are -.codn eq , +.code assoc +function searches an association list +.meta alist +for a cons cell whose +car field is equivalent to +.meta key +(with equality determined by the equal +function). The first such cons is returned. If no such cons is found, +.code nil +is returned. + +.coNP Function @ assql +.synb +.mets (assql < key << alist ) +.syne +.desc +The +.code assql +function is just like +.codn assoc , +except that the equality test +is determined using the .code eql -or -.code equal -to -.metn object , -and return the count. +function rather than +.codn equal . -.SS* Applicative List Processing -.coNP Functions @, remove-if @, keep-if @ remove-if* and @ keep-if* +.coNP Function @ acons .synb -.mets (remove-if < predicate-function < list <> [ key-function ]) -.mets (keep-if < predicate-function < list <> [ key-function ]) -.mets (remove-if* < predicate-function < list <> [ key-function ]) -.mets (keep-if* < predicate-function < list <> [ key-function ]) +.mets (acons < car < cdr << alist ) .syne .desc The -.code remove-if -function produces a list whose contents are those of -.meta list -but with those elements removed which satisfy -.metn predicate-function . -Those elements which are not removed appear in the same order. -The result list may share substructure with the input list, -and may even be the same list object if no items are removed. +.code acons +function constructs a new alist by consing a new cons to the +front of +.metn alist . +The following equivalence holds: -The optional -.meta key-function -specifies how each element from the -.meta list -is transformed to an argument to -.metn predicate-function . -If this argument is omitted -then the predicate function is applied to the elements directly, a behavior -which is identical to -.meta key-function -being -.codn (fun identity) . +.cblk + (acons car cdr alist) <--> (cons (cons car cdr) alist) +.cble -The -.code keep-if -function is exactly like -.codn remove-if , -except the sense of -the predicate is inverted. The function -.code keep-if -retains those items -which -.code remove-if -will delete, and removes those that -.code remove-if -will preserve. +.coNP Function @ acons-new +.synb +.mets (acons-new < car < cdr << alist ) +.syne +.desc The -.code remove-if* -and -.code keep-if* -functions are like -.code remove-if +.code acons-new +function searches +.metn alist , +as if using the assoc function, +for an existing cell which matches the key provided by the car argument. +If such a cell exists, then its cdr field is overwritten with the +.meta cdr +argument, and then the +.meta alist +is returned. If no such cell exists, then +a new list is returned by adding a new cell to the input list consisting +of the +.meta car and -.codn keep-if , -but produce lazy lists. - -.TP* Examples: -.cblk - ;; remove any element numerically equal to 3. - (remove-if (op = 3) '(1 2 3 4 3.0 5)) -> (1 2 4 5) - - ;; remove those pairs whose first element begins with "abc" - [remove-if (op equal [@1 0..3] "abc") - '(("abcd" 4) ("defg" 5)) - car] - -> (("defg" 5)) +.meta cdr +values, as if by the +.code acons +function. - ;; equivalent, without test function - (remove-if (op equal [(car @1) 0..3] "abc") - '(("abcd" 4) ("defg" 5))) - -> (("defg" 5)) -.cble +.coNP Function @ aconsql-new +.synb +.mets (aconsql-new < car < cdr << alist ) +.syne +.desc +This function is like +.codn acons-new , +except that the +.code +eql +function is used +for equality testing. Thus, the list is searched for an existing cell +as if using the +.code assql +function rather than +.codn assoc . -.coNP Function @ count-if +.coNP Function @ alist-remove .synb -.mets (count-if < predicate-function < list <> [ key-function ]) +.mets (alist-remove < alist << keys ) .syne .desc The -.code count-if -function counts the number of elements of -.meta list -which satisfy -.meta predicate-function -and returns the count. - -The optional -.meta key-function -specifies how each element from the -.meta list -is transformed to an argument to -.metn predicate-function . -If this argument is omitted -then the predicate function is applied to the elements directly, a behavior -which is identical to -.meta key-function -being -.codn (fun identity) . +.code alist-remove +function takes association list +.meta alist +and produces a +duplicate from which cells matching the specified keys have been removed. The +.meta keys +argument is a list of the keys not to appear in the output list. -.coNP Functions @, posqual @ posql and @ posq +.coNP Function @ alist-nremove .synb -.mets (posq < object << list ) -.mets (posql < object << list ) -.mets (posqual < object << list ) +.mets (alist-nremove < alist << keys ) .syne .desc The -.codn posq , -.code posql -and -.code posqual -functions return the zero-based position of the -first item in -.meta list -which is, respectively, -.codn eq , -.code eql -or -.code equal -to -.metn object . +.code alist-nremove +function is like +.codn alist-remove , +but potentially destructive. +The input list +.meta alist +may be destroyed and its structural material re-used to +form the output list. The application should not retain references to the input +list. -.coNP Functions @ pos and @ pos-if +.coNP Function @ copy-alist .synb -.mets (pos < key < list >> [ testfun <> [ keyfun ]]) -.mets (pos-if < predfun < list <> [ keyfun ]) +.mets (copy-alist << alist ) .syne .desc - The -.code pos -and -.code pos-if -functions search through -.meta list -for an item which matches -.metn key , -or satisfies predicate function -.metn predfun , -respectively. -They return the zero-based position of the matching item. +.code copy-alist +function duplicates +.codn alist . +Unlike +.codn copy-list , +which only duplicates list structure, +.code copy-alist +also duplicates each cons +cell of the input alist. That is to say, each element of the output list +is produced as if by the +.code copy-cons +function applied to the corresponding +element of the input list. -The -.meta keyfun -argument specifies a function which is applied to the elements -of -.meta list -to produce the comparison key. If this argument is omitted, -then the untransformed elements of -.meta list -are examined. +.SS* Property Lists +.coNP Function @ prop +.synb +.mets (prop < plist << key ) +.syne +.desc +A property list a flat list of even length consisting of interleaved +pairs of property names (usually symbols) and their values (arbitrary +objects). An example property list is (:a 1 :b "two") which contains +two properties, :a having value 1, and :b having value "two". The -.code pos -function's -.meta testfun -argument specifies the test function which -is used to compare the comparison keys from -.meta list -to +.code prop +function searches property list +.meta plist +for key .metn key . -If this argument is omitted, then the -.code equal -function is used. -The position of the first element -.meta list -whose comparison key (as -retrieved by -.metn keyfun ) -matches the search (under -.metn testfun ) -is -returned. If no such element is found, +If the key is found, then the value next to it is returned. Otherwise .code nil is returned. -The -.code pos-if -function's -.meta predfun -argument specifies a predicate function -which is applied to the successive comparison keys taken from -.meta list -by applying -.meta keyfun -to successive elements. The position of -the first element for which -.meta predfun -yields true is returned. If -no such element is found, +It is ambiguous whether .code nil -is returned. +is returned due to the property not being +found, or due to the property being present with a +.code nil +value. -.coNP Functions @ pos-max and @ pos-min +.SS* List Sorting +.coNP Function @ merge .synb -.mets (pos-max < sequence >> [ testfun <> [ keyfun ]]) -.mets (pos-min < sequence >> [ testfun <> [ keyfun ]]) +.mets (merge < seq1 < seq2 >> [ lessfun <> [ keyfun ]]) .syne .desc - The -.code pos-min +.code merge +function merges two sorted sequences +.meta seq1 and -.code pos-max -functions implement exactly the same algorithm; they -differ only in their defaulting behavior with regard to the -.meta testfun -argument. If -.meta testfun -is not given, then the pos-max function defaults -.meta testfun -to the -.code greater -function, whereas -.code pos-min -defaults it to the -.code less -function. - -If -.meta sequence -is empty, both functions return -.codn nil . - -Without a -.meta testfun -argument, the -.code pos-max -function finds the zero-based -position index of the numerically maximum value occurring in -.metn sequence , -whereas -.code pos-min -without a -.meta testfun -argument finds the index of the minimum -value. - -If a -.meta testfun -argument is given, the two functions are equivalent. -The -.meta testfun -function must be callable with two arguments. -If -.meta testfun -behaves like a greater-than comparison, then -.code pos-max +.meta seq2 +into a single +sorted sequence. The semantics and defaulting behavior of the +.meta lessfun and -.code pos-min -return the index of the maximum element. If -.meta testfun -behaves like a -.code less-than -comparison, then the functions return -the index of the minimum element. - -The .meta keyfun -argument defaults to the -.code identity -function. Each element -from -.meta sequence -is passed through this one-argument function, and -the resulting value is used in its place. +arguments are the same as those of the sort function. -.coNP Function @ where +The sequence which is returned is of the same kind as +.metn seq1 . + +This function is destructive of any inputs that are lists. If the output +is a list, it is formed out of the structure of the input lists. + +.coNP Function @ multi-sort .synb -.mets (where < function << object ) +.mets (multi-sort < columns < less-funcs <> [ key-funcs ]) .syne .desc +The +.code multi-sort +function regards a list of lists to be the columns of a +database. The corresponding elements from each list constitute a record. +These records are to be sorted, producing a new list of lists. -If -.meta object -is a sequence, the -.code where -function returns -a list of the numeric indices of those of its elements which satisfy -.metn function . -The numeric indices appear in increasing order. +The +.meta columns +argument supplies the list of lists which comprise the columns of +the database. The lists should ideally be of the same length. If the lists are +of different lengths, then the shortest list is taken to be the length of the +database. Excess elements in the longer lists are ignored, and do not appear in +the sorted output. -If -.meta object -is a hash, the -.code where -function returns an unordered list -of keys which have values which satisfy -.metn function . +The +.meta less-funcs +argument supplies a list of comparison functions which are +applied to the columns. Successive functions correspond to successive +columns. If +.meta less-funcs +is an empty list, then the sorted database will +emerge in the original order. If +.meta less-funcs +contains exactly one function, +then the rows of the database is sorted according to the first column. The +remaining columns simply follow their row. If +.meta less-funcs +contains more than +one function, then additional columns are taken into consideration if the items +in the previous columns compare +.codn equal . +For instance if two elements from column +one compare +.codn equal , +then the corresponding second column elements are compared +using the second column comparison function. -.meta function -must be a function that can be called with one argument. -For each element of -.metn object , -.meta function -is called with that element -as an argument. If a -.cod2 non- nil -value is returned, then the zero-based index of -that element is added to a list. Finally, the list is returned. +The optional +.meta key-funcs +argument supplies transformation functions through +which column entries are converted to comparison keys, similarly to the single +key function used in the sort function and others. If there are more key +functions than less functions, the excess key functions are ignored. -.coNP Function @ select +.SS* Lazy Lists and Lazy Evaluation +.coNP Function @ make-lazy-cons .synb -.mets (select < object >> { index-list <> | function }) +.mets (make-lazy-cons << function ) .syne .desc +The function +.code make-lazy-cons +makes a special kind of cons cell called a lazy +cons, or lcons. Lazy conses are useful for implementing lazy lists. -The -.code select -function returns an object, of the same kind as -.metn object , -which consists of those elements of -.meta object -which are identified by -the indices in -.metn index-list , -which may be a list or a vector. +Lazy lists are lists which are not allocated all at once. Rather, +their elements materialize when they are accessed, like +magic stepping stones appearing under one's feet out of thin air. -If -.meta function -is given instead of -.metn index-list , -then +A lazy cons has +.code car +and +.code cdr +fields like a regular cons, and those +fields are initialized to +.code nil +when the lazy cons is created. A lazy cons also +has an update function, the one which is provided as the .meta function -is invoked with -.meta object -as its argument. The return value is then taken as -if it were the -.meta index-list -argument . +argument to +.codn make-lazy-cons . -If -.meta object -is a sequence, then -.meta index-list -consists of numeric -indices. The -.code select -function stops processing -.meta object -upon encountering an -index inside -.meta index-list -which is out of range. (Rationale: without -this strict behavior, -.code select -would not be able to terminate if -.meta index-list -is infinite.) +When either the +.code car +and +.code cdr +fields of a cons are accessed for the first time, +the function is automatically invoked first. That function has the opportunity +to initialize the +.code car +and +.code cdr +fields. Once the function is called, it is removed +from the lazy cons: the lazy cons no longer has an update function. -If -.meta object -is a list, then -.meta index-list -must contain monotonically increasing -numeric values, even if no value is out of range, since the -.code select -function -makes a single pass through the list based on the assumption that indices -are ordered. (Rationale: optimization.) +To continue a lazy list, the function can make another call to +.code make-lazy-cons +and install the resulting cons as the +.code cdr +of the lazy cons. -If -.meta object -is a hash, then -.meta index-list -is a list of keys. A new hash is -returned which contains those elements of -.meta object -whose keys appear -in -.metn index-list . -All of -.meta index-list -is processed, even if it contains -keys which are not in -.metn object . +.TP* Example: -.coNP Function @ in -.synb -.mets (in < sequence < key >> [ testfun <> [ keyfun ]]) -.mets (in < hash << key ) -.syne -.desc -The -.code in -function tests whether -.meta key -is found inside -.meta sequence -or -.metn hash . - -If the -.meta testfun -argument is specified, it specifies the function -which is used to comparison keys from the sequence -to -.metn key . -Otherwise the -.code equal -function is used. - -If the -.meta keyfun -argument is specified, it specifies a function which -is applied to the elements of -.meta sequence -to produce the comparison keys. Without this -argument, the elements themselves are taken -as the comparison keys. - -If the object being searched is a hash, then the -.meta keyfun -and -.meta testfun -arguments are ignored. +.cblk + ;;; lazy list of integers between min and max + (defun integer-range (min max) + (let ((counter min)) + ;; min is greater than max; just return empty list, + ;; otherwise return a lazy list + (if (> min max) + nil + (make-lazy-cons + (lambda (lcons) + ;; install next number into car + (rplaca lcons counter) + ;; now deal wit cdr field + (cond + ;; max reached, terminate list with nil! + ((eql counter max) + (rplacd lcons nil)) + ;; max not reached: increment counter + ;; and extend with another lazy cons + (t + (inc counter) + (rplacd lcons (make-lazy-cons + (lcons-fun lcons)))))))))) +.cble +.coNP Function @ lconsp +.synb +.mets (lconsp << value ) +.syne +.desc The -.code in +.code lconsp function returns .code t -if it finds -.meta key -in -.meta sequence -or -.metn hash, -otherwise -.codn nil . +if +.meta value +is a lazy cons cell. Otherwise +it returns +.codn nil , +even if +.meta value +is an ordinary cons cell. -.coNP Function @ partition +.coNP Function @ lcons-fun .synb -.mets (partition < sequence >> { index-list >> | index <> | function }) +.mets (lcons-fun << lazy-cons ) .syne .desc +The +.code lcons-fun +function retrieves the update function of a lazy cons. +Once a lazy cons has been accessed, it no longer has an update function +and +.code lcons-fun +returns +.codn nil . +While the update function of a lazy cons is +executing, it is still accessible. This allows the update function +to retrieve a reference to itself and propagate itself into +another lazy cons (as in the example under +.codn make-lazy-cons ). -If -.meta sequence -is empty, then -.code partition -returns an empty list, and the -second argument is ignored; if it is -.metn function , -it is not called. +.coNP Macro @ lcons +.synb +.mets (lcons < car-expression << cdr-expression ) +.syne +.desc +The +.code lcons +macro simplifies the construction of structures based on lazy conses. +Syntactically, it resembles the +.code cons +function. However, the arguments are expressions rather than values. +The macro generates code which, when evaluated, immediately produces +a lazy cons. The expressions +.meta car-expression +and +.meta cdr-expression +are not immediately evaluated. Rather, when either the +.code car +or +.code cdr +field of the lazy cons cell is accessed, these expressions are both +evaluated at that time, in the order that they appear in the +.code lcons +expression, and in the original lexical scope in which that +expression was evaluated. The return values of these expressions +are used, respectively, to initialize the corresponding fields +of the lazy cons. -Otherwise, -.code partition -returns a lazy list of partitions of -.metn sequence . -Partitions are consecutive, non-overlapping, non-empty sub-strings of -.metn sequence , -of the same kind as -.metn sequence , -such that if these sub-strings are catenated together in their order -of appearance, a sequence -.code equal -to the original is produced. +Note: the +.code lcons +macro may be understood in terms of the following reference +implementation, as a syntactic sugar combining the +.code make-lazy-cons +constructor with a lexical closure provided by a +.code lambda +function: -If the second argument is of the form -.metn index-list , -it shall be a sequence of -strictly non-decreasing, integers. First, any leading negative or zero values -in this sequence are dropped. The -.code partition -function then divides -.meta sequence -according to the -indices in index list. The first partition begins with the first element of -.metn sequence . -The second partition begins at the first position in -.metn index-list , -and so on. Indices beyond the length of the sequence are ignored. +.cblk + (defmacro lcons (car-form cdr-form) + (let ((lc (gensym))) + ^(make-lazy-cons (lambda (,lc) + (rplaca ,lc ,car-form) + (rplacd ,lc ,cdr-form))))) +.cble -If -.meta index-list -is empty then a one-element list containing the entire -.meta sequence -is returned. +.TP* Example: -If the second argument is a function, then this function is applied -to -.metn sequence , -and the return value of this call is then used in place of the -second argument, which must be an -.meta index -or -.metn index-list . +.cblk + ;; Given the following function ... -If the second argument is an atom other than a function, it is assumed to be -an integer index, and is turned into an -.meta index-list -of one element. + (defun fib-generator (a b) + (lcons a (fib-generator b (+ a b)))) -.TP* Examples: -.cblk - (partition '(1 2 3) 1) -> ((1) (2 3)) + ;; ... the following function call generates the Fibonacci + ;; sequence as an infinite lazy list. - ;; split the string where there is a "b" - (partition "abcbcbd" (op where (op eql #\eb))) -> ("a" "bc" - "bc" "bd") + (fib-generator 1 1) -> (1 1 2 3 5 8 13 ...) .cble -.coNP Function @ split +.coNP Functions @ lazy-stream-cons and @ get-lines .synb -.mets (split < sequence >> { index-list >> | index <> | function }) +.mets (lazy-stream-cons << stream ) +.mets (get-lines <> [ stream ]) .syne .desc +The +.code lazy-stream-cons +and +.code get-lines +functions are synonyms, except that the +.meta stream +argument is optional in +.code get-lines +and defaults to +.codn *stdin* . +Thus, the following +description of +.code lazy-stream-cons +also applies to +.codn get-lines . -If -.meta sequence -is empty, then -.code split -returns an empty list, and the -second argument is ignored; if it is -.metn function , -it is not called. - -Otherwise, -.code split -returns a lazy list of pieces of -.metn sequence : -consecutive, non-overlapping, possibly empty sub-strings of -.metn sequence , -of the same kind as -.metn sequence . -A catenation of these pieces in the order they appear would produce -a sequence that is -.code equal -to the original sequence. +The +.code lazy-stream-cons +returns a lazy cons which generates a lazy list based on +reading lines of text from input stream +.metn stream , +which form the elements of +the list. The +.code get-line +function is called on demand to add elements to the +list. -If the second argument is of the form -.metn index-list , -it shall be a sequence of increasing integers. The -.code split -function divides -.meta sequence -according to the -indices in index list. The first piece always begins with the first -element of -.metn sequence . -Each subsequent piece begins with the position indicated by -an element of -.metn index-list . -Negative indices are ignored. Repeated values give rise to empty -pieces. -If -.meta index-list -includes index zero, -then an empty first piece is generated. -If -.meta index-list -includes an index greater than or equal to the length of -.meta sequence -(equivalently, an index beyond the last element of the sequence) -then an additional empty last piece is generated. +.code lazy-stream-cons +function itself makes the first call to +.code get-line +on the stream. If this returns +.codn nil , +then the stream is closed and +.code nil +is +returned. Otherwise, a lazy cons is returned whose update function will install +that line into the +.code car +field of the lazy cons, and continue the lazy list +by making another call to +.codn lazy-stream-cons , +installing the result into the +.code cdr +field. -If -.meta index-list -is empty then a one-element list containing the entire -.meta sequence -is returned. +.code lazy-stream-cons +inspects the real-time property of a stream +as if by the +.code real-time-stream-p +function. This determines which of two +styles of lazy list are returned. For an ordinary (non-real-time) stream, +the lazy list treats the end-of-file condition accurately: an empty +file turns into the empty list +.codn nil , +a one line file into a one-element +list which contains that line and so on. This accuracy requires one +line of lookahead which is not acceptable in real-time streams, and +so a different type of lazy list is used, which generates an extra +.code nil +item after the last line. Under this type of lazy list, an empty input stream +translates to the list +.codn (nil) ; +a one-line stream translates to +.code ("line" nil) +and so forth. -If the second argument is a function, then this function is applied -to -.metn sequence , -and the return value of this call is then used in place of the -second argument, which must be an -.meta index -or -.metn index-list . +.coNP Macro @ delay +.synb +.mets (delay << expression ) +.syne +.desc +The delay operator arranges for the delayed (or "lazy") evaluation of +.metn expression . +This means that the expression is not evaluated immediately. +Rather, the delay expression produces a promise object. -If the second argument is an atom other than a function, it is assumed to be -an integer index, and is turned into an -.meta index-list -of one element. +The promise object can later be passed to the +.code force +function (described +later in this document). The force function will trigger the evaluation +of the expression and retrieve the value. + +The expression is evaluated in the original scope, no matter where +the +.code force +takes place. + +The expression is evaluated at most once, by the first call to +.codn force . +Additional calls to +.code force +only retrieve a cached value. + +.TP* Example: -.TP* Examples: .cblk - (split '(1 2 3) 1) -> ((1) (2 3)) + @(do + ;; list is popped only once: the value is computed + ;; just once when force is called on a given promise + ;; for the first time. - (split "abc" 0) -> ("" "abc") - (split "abc" 3) -> ("abc" "") - (split "abc" 1) -> ("a" "bc") - (split "abc" 0 1 2 3) -> ("" "a" "b" "c" "") - (split "abc" 1 2) -> ("a" "b" "c") + (defun get-it (promise) + (format t "*list* is ~s\en" *list*) + (format t "item is ~s\en" (force promise)) + (format t "item is ~s\en" (force promise)) + (format t "*list* is ~s\en" *list*)) - (split "abc" -1 1 2 15) -> ("a" "b" "c") + (defvar *list* '(1 2 3)) - ;; triple split at makes two additional empty pieces - (split "abc" '(1 1 1)) -> ("a" "" "" "bc") + (get-it (delay (pop *list*)))) + + Output: + + *list* is (1 2 3) + item is 1 + item is 1 + *list* is (2 3) .cble -.coNP Function @ partition* +.coNP Accessor @ force .synb -.mets (partition* < sequence >> { index-list >> | index <> | function }) +.mets (force << promise ) +.mets (set (force << promise ) << new-value ) .syne .desc -If -.meta sequence -is empty, then -.code partition* -returns an empty list, and the -second argument is ignored; if it is -.metn function , -it is not called. - -If the second argument is of the form -.metn index-list , -which is a sequence -of strictly increasing non-negative integers, then -.code partition* -produces a -lazy list of pieces taken from -.metn sequence . -The pieces are formed by -deleting from -.meta sequence -the elements at the positions given -in -.metn index-list . -The pieces are the non-empty sub-strings between -the deleted elements. - -If -.meta index-list -is empty then a one-element list containing the entire -.meta sequence -is returned. - -If the second argument is a function, then this function is applied -to -.metn sequence , -and the return value of this call is then used in place of the -second argument, which must be an -.meta index -or -.metn index-list . +The +.code force +function accepts a promise object produced by the +.code delay +macro. +The first time +.code force +is invoked, the +.meta expression +which was wrapped inside +.meta promise +by the +.code delay +macro is evaluated (in its original lexical environment, regardless of where in +the program the +.code force +call takes place). The value of +.meta expression +is +cached inside +.meta promise +and returned, becoming the return value of the +.code force +function call. If the +.code force +function is invoked additional times on +the same promise, the cached value is retrieved. -If the second argument is an atom other than a function, it is assumed to be -an integer index, and is turned into an -.meta index-list -of one element. +A +.code force +form is a syntactic place, denoting the value cache location within +.metn promise . -.TP* Examples: -.cblk - (partition* '(1 2 3 4 5) '(0 2 4)) -> ((1) (3) (5)) +Storing a value in a +.code force +place causes future accesses to the +.meta promise +to return that value. - (partition* "abcd" '(0 3)) -> "bc" +If the promise had not yet been forced, then +storing a value into it prevents that from ever happening. The +delayed +.meta expression +will never be evaluated. - (partition* "abcd" '(0 1 2 3)) -> nil -.cble +If, while a promise is being forced, the evaluation of +.meta expression +itself causes an assignment to the promise, it is not specified whether +the promise will take on the value of +.meta expression +or the assigned value. -.coNP Function @ tree-find +.coNP Macro @ mlet .synb -.mets (tree-find < obj < tree << test-function ) +.mets (mlet >> ({ sym | >> ( sym << init-form )}*) << body-form *) .syne .desc - The -.code tree-find -function searches -.meta tree -for an occurrence of -.metn obj . -Tree can be -any atom, or a cons. If -.meta tree -it is a cons, it is understood to be a proper -list whose elements are also trees. +.code mlet +macro ("magic let" or "mutual let") implements a variable binding construct +similar to +.code let +and +.codn let* . -The equivalence test is performed by -.meta test-function -which must take two -arguments, and has conventions similar to -.codn eq , -.code eql -or -.codn equal . +Under +.codn mlet , +the scope of the bindings of the +.meta sym +variables extends over the +.metn init-form -s, +as well as the +.metn body-form -s. -.code tree-find -works as follows. If -.meta tree -is equivalent to -.meta obj -under -.metn test-function , -then -.code t -is returned to announce a successful finding. -If this test fails, and -.meta tree -is an atom, -.code nil -is returned immediately to -indicate that the find failed. Otherwise, -.meta tree -is taken to be a proper list, -and tree-find is recursively applied to each element of the list in turn, using -the same -.meta obj -and -.meta test-function -arguments, stopping at the first element -which returns a -.cod2 non- nil -value. +Unlike the +.code let* +construct, each +.meta init-form +has each +.meta sym +in scope. That is to say, an +.metn init-form +can refer not only to previous variables, but also to later variables +as well as to its own variable. -.coNP Functions @ find and @ find-if -.synb -.mets (find < key < sequence >> [ testfun <> [ keyfun ]]) -.mets (find-if < predfun < sequence <> [ keyfun ]) -.syne -.desc -The -.code find -and -.code find-if -functions search through a sequence for an item which -matches a key, or satisfies a predicate function, respectively. +The variables are not initialized until their values are accessed for +the first time. Any +.meta sym +whose value is not accessed is not initialized. -The -.meta keyfun -argument specifies a function which is applied to the elements -of -.meta sequence -to produce the comparison key. If this argument is omitted, -then the untransformed elements of the -.meta sequence -are searched. +Furthermore, the evaluation of each +.meta init-form +does not take place until the time when its value is needed +to initialize the associated +.metn sym . +This evaluation takes place once. If a given +.meta sym +is not accessed during the evaluation of the +.code mlet +construct, then its +.meta init-form +is never evaluated. -The -.code find -function's -.meta testfun -argument specifies the test function which -is used to compare the comparison keys from -.meta sequence -to the search key. -If this argument is omitted, then the -.code equal -function is used. -The first element from the list whose comparison key (as retrieved by -.metn keyfun ) -matches the search (under -.metn testfun ) -is returned. If no such element is found, -.code nil -is returned. +The bound variables may be assigned. If, before initialization, a variable is +updated in such a way that its prior value is not needed, it is unspecified +whether initialization takes place, and thus whether its +.meta init-form +is evaluated. -The -.code find-if -function's -.meta predfun -argument specifies a predicate function -which is applied to the successive comparison keys pulled from the list -by applying -.meta keyfun -to successive elements. The first element -for which -.meta predfun -yields true is returned. If no such -element is found, -.code nil -is returned. +Direct circular references erroneous and are diagnosed. -.coNP Functions @ find-max and @ find-min -.synb -.mets (find-max < sequence >> [ testfun <> [ keyfun ]]) -.mets (find-min < sequence >> [ testfun <> [ keyfun ]]) -.syne -.desc -The -.code find-min -and -.code find-max -function implement exactly the same algorithm; they -differ only in their defaulting behavior with regard to the -.meta testfun -argument. If -.meta testfun -is not given, then the find-max function defaults it to -the -.code greater -function, whereas -.code find-min -defaults it to the -.code less -function. +.TP* Examples: -Without a -.meta testfun -argument, the -.code find-max -function finds the numerically -maximum value occurring in -.metn sequence , -whereas -.code pos-min -without a -.meta testfun -argument finds the minimum value. +.cblk + ;; Dependent calculations in arbitrary order + (mlet ((x (+ y 3)) + (z (+ x 1)) + (y 4)) + (+ z 4)) --> 12 -If a -.meta testfun -argument is given, the two functions are equivalent. -The -.meta testfun -function must be callable with two arguments. -If -.meta testfun -behaves like a greater-than comparison, then -.code find-max -and -.code find-min -both return the maximum element. If -.meta testfun -behaves like a less-than comparison, then the functions return -the minimum element. + ;; Error: circular reference: + ;; x depends on y, y on z, but z on x again. + (mlet ((x (+ y 1)) + (y (+ z 1)) + (z (+ x 1))) + z) -The -.meta keyfun -argument defaults to the -.code identity -function. Each element -from -.meta sequence -is passed through this one-argument function, and -the resulting value is used in its place for the purposes of the -comparison. However, the original element is returned. + ;; Okay: lazy circular reference because lcons is used + (mlet ((list (lcons 1 list))) + list) --> (1 1 1 1 1 ...) ;; circular list +.cble -.coNP Function @ set-diff +In the last example, the +.code list +variable is accessed for the first time in the body of the +.code mlet +form. This causes the evaluation of the +.code lcons +form. This form evaluates its arguments lazily, which means that it +is not a problem that +.code list +is not yet initialized. The form produces a lazy cons, which is then used +to initialize +.code list. +When the +.code car +or +.code cdr +fields of the lazy cons are accessed, the +.code list +expression in the +.code lcons +argument is accessed. By that time, the variable is initialized +and holds the lazy cons itself, which creates the circular reference, +and a circular list. + +.coNP Functions @ generate and @ giterate .synb -.mets (set-diff < seq1 < seq2 >> [ testfun <> [ keyfun ]]) +.mets (generate < while-fun << gen-fun ) +.mets (giterate < while-fun < gen-fun <> [ value ]) .syne .desc The -.code set-diff -function treats the sequences -.meta seq1 -and -.meta seq2 -as if they were sets -and computes the set difference: a sequence which contains those elements in -.meta seq1 -which do not occur in -.metn seq2 . +.code generate +function produces a lazy list which dynamically produces items +according to the following logic. -.code set-diff -returns a sequence of the same kind as -.metn seq1 . +The arguments to +.code generate +are functions which do not take any arguments. The +return value of generate is a lazy list. -Element equivalence is determined by a combination of -.meta testfun -and -.metn keyfun . -Elements are compared pairwise, and each element of a pair is passed through -.meta keyfun -function to produce a comparison value. The comparison values -are compared using -.metn testfun . -If -.meta keyfun -is omitted, then the -untransformed elements themselves are compared, and if -.meta testfun -is omitted, -then the -.code equal -function is used. +When the lazy list is accessed, for instance with the functions car and cdr, it +produces items on demand. Prior to producing each item, +.meta while-fun +is +called. If it returns a true boolean value (any value other than +.codn nil ), +then +the +.meta gen-fun +function is called, and its return value is incorporated as +the next item of the lazy list. But if +.meta while-fun +yields +.codn nil , +then the lazy list immediately terminates. +Prior to returning the lazy list, generate invokes the +.meta while-fun +one time. If -.meta seq1 -contains duplicate elements which do not occur in -.meta seq2 -(and thus are preserved in the set difference) then these duplicates appear -in the resulting -sequence. Furthermore, the order of the items from -.meta seq1 -is preserved. - -.coNP Functions @, mapcar @ mappend @ mapcar* and @ mappend* -.synb -.mets (mapcar < function << sequence *) -.mets (mappend < function << sequence *) -.mets (mapcar* < function << sequence *) -.mets (mappend* < function << sequence *) -.syne -.desc - -When given only one argument, the -.code mapcar -function returns -.codn nil . -.meta function -is never called. - -When given two arguments, the -.code mapcar -function applies -.meta function -to each elements of -.meta sequence -and returns a sequence of the resulting values -in the same order as the original values. -The returned sequence is the same kind as -.metn sequence , -if possible. If the accumulated values cannot be -elements of that type of sequence, then a list is returned. - -When additional sequences are given as arguments, this filtering behavior is -generalized in the following way: -.code mapcar -traverses the sequences in parallel, -taking a value from each sequence as an argument to the function. If there -are two lists, -.meta function -is called with two arguments and so forth. -The traversal is limited by the length of the shortest sequence. -The return values of the function are collected into a new sequence which is -returned. The returned sequence is of the same kind as the leftmost -input sequence, unless the accumulated values cannot be elements of that type of -sequence, in which case a list is returned. - -The -.code mappend -function works like -.codn mapcar , -with the following difference. -Rather than accumulating the values returned by the function into a sequence, -mappend expects the items returned by the function to be sequences which -are catenated with -.codn append , -and the resulting sequence is returned. The returned sequence is of the same -kind as the leftmost input sequence, unless the values cannot be elements -of that type of sequence, in which case a list is returned. +.code while-fun +yields +.codn nil , +then +.code generate +returns the empty list +.code nil +instead of a lazy list. Otherwise, it instantiates a lazy list, and invokes the +.code gen-func +to populate it with the first item. The -.code mapcar* -and -.code mappend* -functions work like -.code mapcar +.code giterate +function is similar to +.codn generate , +except that +.meta while-fun and -.codn mappend , -respectively. -However, they return lazy lists rather than generating the entire -output list prior to returning. - -.TP* Caveats: +.meta gen-fun +are functions of one argument rather than functions of +no arguments. The optional +.meta value +argument defaults to +.code nil +and is threaded through the function calls. That is to say, the lazy +list returned is +.cblk +.meti >> ( value >> [ gen-fun << value ] >> [ gen-fun >> [ gen-fun << value ]] ...). +.cble -Like -.codn mappend , -.code mappend* -must "consume" empty lists. For instance, -if the function being mapped puts out a sequence of -.codn nil s, -then the result must be the empty list +The lazy list terminates when a value fails to satisfy +.metn while-fun . +That is to say, prior to generating each value, the lazy list tests +the value using +.metn while-fun . +If that function returns .codn nil , -because -.code (append nil nil nil nil ...) -is -.codn nil . +then the item is not added, and the sequence terminates. -But suppose that -.code mappend* -is used on inputs which are infinite lazy -lists, such that the function returns -.code nil -values indefinitely. -For instance: +Note: +.code giterate +could be written in terms of +.code generate +like this: .cblk - ;; Danger: infinite loop!!! - (mappend* (fun identity) (repeat '(nil))) + (defun giterate (w g v) + (generate (lambda () [w v]) + (lambda () (prog1 v (set v [g v]))))) .cble -The -.code mappend* -function is caught in a loop trying to consume -and squash an infinite stream of -.codn nil s, -and so doesn't return. +.TP* Example: -.TP* Examples: .cblk - ;; multiply every element by two - (mapcar (lambda (item) (* 2 item)) '(1 2 3)) -> (4 6 8) - - ;; "zipper" two lists together - (mapcar (lambda (le ri) (list le ri)) '(1 2 3) '(a b c)) '((1 a) (2 b) (3 c))) - - ;; like append, mappend allows a lone atom or a trailing atom: - (mappend (fun identity) 3) -> (3) - (mappend (fun identity) '((1) 2)) -> (1 . 2) - - ;; take just the even numbers - (mappend (lambda (item) (if (evenp x) (list x))) '(1 2 3 4 5)) - -> (2 4) + (giterate (op > 5) (op + 1) 0) -> (0 1 2 3 4) .cble -.coNP Function @ mapdo +.coNP Function @ repeat .synb -.mets (mapdo < function << sequence *) +.mets (repeat < list <> [ count ]) .syne .desc -The -.code mapdo -function is similar to -.codn mapcar , -but always returns -.codn nil . -It is useful -when -.meta function -performs some kind of side effect, hence the "do" in the name, -which is a mnemonic for the execution of imperative actions. +If +.meta list +is empty, then repeat returns an empty list. -When only the -.meta function -argument is given, -.meta function -is never called, -and -.code nil -is returned. +If +.meta count +is omitted, the +.code repeat +function produces an infinite lazy list +formed by catenating together copies of +.metn list . -If a single -.meta sequence -argument is given, then -.code mapdo -iterates over -.metn sequence , -invoking -.meta function -on each element. +If +.meta count +is specified and is zero or negative, then an empty list is +returned. -If two or more -.meta sequence -arguments are given, then -.code mapdo -iterates over -the sequences in parallel, extracting parallel tuples of items. These -tuples are passed as arguments to -.metn function, -which must accept as many -arguments as there are sequences. +Otherwise a list is returned consisting of +.meta count +repetitions of +.meta list +catenated together. -.coNP Functions @ transpose and @ zip +.coNP Function @ pad .synb -.mets (transpose << sequence ) -.mets (zip << sequence *) +.mets (pad < sequence < object <> [ count ]) .syne .desc The -.code transpose -function performs a transposition on -.metn sequence . -This means that the +.code pad +function produces a lazy list which consists of all of the elements of .meta sequence -must be sequences. These sequences are understood to be -columns; transpose exchanges rows and columns, returning a sequence of the rows -which make up the columns. The returned sequence is of the same kind as -.metn sequence , -and the rows are also the same kind of sequence as the first column -of the original sequence. The number of rows returned is limited by the -shortest column among the sequences. +followed by repetitions of +.metn object . -All of the input sequences (the elements of -.metn sequence) -must have elements -which are compatible with the first sequence. This means that if the first -element of -.meta sequence -is a string, then the remaining sequences must be -strings, or else sequences of characters, or of strings. +If +.meta count +is omitted, then the repetition of +.meta object +is infinite. Otherwise the specified number of repetitions +occur. -The -.code zip -function takes variable arguments, and is equivalent to calling -.code transpose -on a list of the arguments. The following equivalences hold: +Note that +.meta sequence +may be a lazy list which is infinite. In that case, the repetitions +of +.meta object +will never occur. +.coNP Function @ weave .synb - (zip . x) <--> (transpose x) - - [apply zip x] <--> (transpose x) +.mets (weave <> { sequence }*) .syne +.desc +The +.code weave +function interleaves elements from the sequences given as arguments. -.TP* Examples: -.cblk - ;; transpose list of lists - (transpose '((a b c) (c d e))) -> ((a c) (b d) (c e)) +If called with no arguments, it returns the empty list. - ;; transpose vector of strings: - ;; - string columns become string rows - ;; - vector input becomes vector output - (transpose #("abc" "def" "ghij")) -> #("adg" "beh" "cfi") +If called with a single sequence, it returns the elements of that sequence +as a new lazy list. - ;; error: transpose wants to make a list of strings - ;; but 1 is not a character - (transpose #("abc" "def" '(1 2 3))) ;; error! +When called with two or more sequences, +.code +weave +returns a lazy list which draws elements from the sequences in a round-robin +fashion, repeatedly scanning the sequences from left to right, and +taking an item from each one, removing it from the sequence. +Whenever a sequence runs out of items, it is deleted; the weaving then +continues with the remaining sequences. The weaved sequence terminates +when all sequences are eliminated. (If at least one of the sequences +is an infinite lazy list, then the weaved sequence is infinite.) - ;; String elements are catenated: - (transpose #("abc" "def" ("UV" "XY" "WZ"))) -> #("adUV" "beXY" "cfWZ") +.TP* Examples: - (zip '(a b c) '(c d e)) -> ((a c) (b d) (c e)) +.cblk + ;; Weave negative integers with positive ones: + (weave (range 1) (range -1 : -1)) -> (1 -1 2 -2 3 -3 ...) + + (weave "abcd" (range 1 3) '(x x x x x x x)) + --> (#\ea 1 x #\eb 2 x #\ec 3 x #\ed x x x x) .cble -.coNP Function @ interpose +.coNP Macros @ gen and @ gun .synb -.mets (interpose < sep << sequence ) +.mets (gen < while-expression << produce-item-expression ) +.mets (gun << produce-item-expression ) .syne .desc The -.code interpose -function returns a sequence of the same type as -.metn sequence , -in which the elements from -.meta sequence -appear with the -.meta sep -value inserted -between them. +.code gen +macro operator produces a lazy list, in a manner similar to the +.code generate +function. Whereas the +.code generate +function takes functional arguments, +the +.code gen +operator takes two expressions, which is often more convenient. -If -.meta sequence -is an empty sequence or a sequence of length 1, then a -sequence identical to -.meta sequence -is returned. It may be a copy of -.meta sequence -or it may be -.meta sequence -itself. +The return value of +.code gen +is a lazy list. When the lazy list is accessed, for +instance with the functions +.code car +and +.codn cdr , +it produces items on demand. Prior to +producing each item, the +.meta while-expression +is evaluated, in its original +lexical scope. If the expression yields a +.cod2 non- nil +value, then +.meta produce-item-expression +is evaluated, and its return value is incorporated as +the next item of the lazy list. If the expression yields +.codn nil , +then the lazy list immediately terminates. -If -.meta sequence -is a character string, then the value -.meta sep -must be a character. +The +.code gen +operator itself immediately evaluates +.meta while-expression +before +producing the lazy list. If the expression yields +.codn nil , +then the operator +returns the empty list +.codn nil . +Otherwise, it instantiates the lazy list and +invokes the +.meta produce-item-expression +to force the first item. -It is permissible for -.metn sequence , -or for a suffix of -.meta sequence -to be a lazy -list, in which case interpose returns a lazy list, or a list with a lazy -suffix. +The +.code gun +macro similarly creates a lazy list according to the following +rules. Each successive item of the lazy list is obtained as a result of +evaluating +.metn produce-item-expression . +However, when +.meta produce-item-expression +yields +.codn nil , +then the list terminates (without adding that +.code nil +as an item). + +Note 1: the form +.code gun +can be implemented as a macro-expanding to +an instance of the +.code gen +operator, like this: -.TP* Examples: .cblk - (interpose #\e- "xyz") -> "x-y-z" - (interpose t nil) -> nil - (interpose t #()) -> #() - (interpose #\ea "") -> "" - (interpose t (range 0 0)) -> (0) - (interpose t (range 0 1)) -> (0 t 1) - (interpose t (range 0 2)) -> (0 t 1 t 2) + (defmacro gun (expr) + (let ((var (gensym))) + ^(let (,var) + (gen (set ,var ,expr) + ,var)))) .cble -.coNP Functions @ conses and @ conses* -.synb -.mets (conses << list ) -.mets (conses* << list ) -.syne -.desc -These functions return a list whose elements are the conses which make -up -.metn list . -The -.code conses* -function does this in a lazy way, avoiding the -computation of the entire list: it returns a lazy list of the conses of -.metn list . -The -.code conses -function computes the entire list before returning. +This exploits the fact that the +.code set +operator returns the value that is +assigned, so the set expression is tested as a condition by +.codn gen , +while having the side effect of storing the next item temporarily +in a hidden variable. -The input -.meta list -may be proper or improper. +In turn, +.code gen +can be implemented as a macro expanding to some +.code lambda +functions which are passed to the +.code generate +function: -The first cons of -.meta list -is that -.meta list -itself. The second cons is the rest -of the list, or -.cblk -.meti (cdr << list ). -.cble -The third cons is .cblk -.meti (cdr (cdr << list )) + (defmacro gen (while-expr produce-expr) + ^(generate (lambda () ,while-expr) (lambda () ,produce-expr))) .cble -and so on. + +Note 2: +.code gen +can be considered as an acronym for Generate, testing Expression +before Next item, whereas +.code gun +stands for Generate Until Null. .TP* Example: + .cblk - (conses '(1 2 3)) -> ((1 2 3) (2 3) (3)) + @(do + ;; Make a lazy list of integers up to 1000 + ;; access and print the first three. + (let* ((counter 0) + (list (gen (< counter 1000) (inc counter)))) + (format t "~s ~s ~s\en" (pop list) (pop list) (pop list)))) + + Output: + 1 2 3 .cble -.TP* "Dialect Note:" - -These functions are useful for simulating the -.code maplist -function found in other dialects like Common Lisp. - -\*(TL's -.code (conses x) -can be expressed in Common Lisp as -.codn (maplist #'identity x) . +.coNP Functions @ range and @ range* +.synb +.mets (range >> [ from >> [ to <> [ step ]]]) +.mets (range* >> [ from >> [ to <> [ step ]]]) +.syne +.desc +The +.code range +and +.code range* +functions generate a lazy sequence of integers, with a +fixed step between successive values. -Conversely, the Common Lisp operation -.code (maplist function list) -can be computed in \*(TL as -.codn (mapcar function (conses list)) . +The difference between +.code range +and +.code range* +is that +.code range* +excludes the endpoint. +For instance +.code (range 0 3) +generates the list +.codn (0 1 2 3) , +whereas +.code (range* 0 3) +generates +.codn (0 1 2) . -More generally, the Common Lisp operation +All arguments are optional. If the +.meta step +argument is omitted, then it defaults +to +.codn 1 : +each value in the sequence is greater than the previous one by +.codn 1 . +Positive or negative step sizes are allowed. There is no check for a step size +of zero, or for a step direction which cannot meet the endpoint. -.cblk - (maplist function list0 list1 ... listn) -.cble +The +.meta to +argument specifies the endpoint value, which, if it occurs in the +sequence, is excluded from it by the +.code range* +function, but included by the range +function. If +.meta to +is missing, or specified as +.codn nil , +then there is no endpoint, +and the sequence which is generated is infinite, regardless of +.metn step . -can be expressed as: +If +.meta from +is omitted, then the sequence begins at zero, otherwise +.meta from +must be an integer which specifies the initial value. -.cblk - (mapcar function (conses list0) - (conses list1) ... (conses listn)) -.cble +The sequence stops if it reaches the endpoint value (which is included in the +case of +.codn range , +and excluded in the case of +.codn range *). +However, a sequence with a stepsize greater than +.code 1 +or less than +.code -1 +might step over the endpoint value, and +therefore never attain it. In this situation, the sequence also stops, and the +excess value which surpasses the endpoint is excluded from the sequence. -.coNP Functions @ apply and @ iapply +.SS* Characters and Strings +.coNP Function @ mkstring .synb -.mets (apply < function <> [ arg * << trailing-args ]) -.mets (iapply < function <> [ arg * << trailing-args ]) +.mets (mkstring < length << char ) .syne .desc The -.code apply -function invokes -.metn function , -optionally passing to it an argument -list. The return value of the -.code apply -call is that of -.metn function . +.code mkstring +function constructs a string object of a length specified +by the +.meta length +parameter. Every position in the string is initialized +with +.metn char , +which must be a character value. -If no arguments are present after -.metn function , -then -.meta function -is invoked without arguments. +.coNP Function @ copy-str +.synb +.mets (copy-str << string ) +.syne +.desc +The +.code copy-str +function constructs a new string whose contents are identical +to +.metn string . -If one argument is present after -.metn function , -then it is interpreted as -.metn trailing-args . -If this is a sequence (a list, vector or string), -then the elements of the sequence are passed as individual arguments to -.metn function . -If -.meta trailing-args -is not a sequence, then -.meta function -is invoked -with an improper argument list, terminated by the -.meta trailing-args -atom. +.coNP Function @ upcase-str +.synb +.mets (upcase-str << string ) +.syne +.desc +The +.code upcase-str +function produces a copy of +.meta string +such that all lower-case +characters of the English alphabet are mapped to their upper case counterparts. -If two or more arguments are present after -.metn function , -then the last of these arguments is interpreted as -.metn trailing-args . -The previous arguments represent leading arguments which are applied to -.metn function , -prior to the arguments taken from -.metn trailing-args . +.coNP Function @ downcase-str +.synb +.mets (downcase-str << string ) +.syne +.desc +The +.code downcase-str +function produces a copy of +.meta string +such that +all upper case characters of the English alphabet are mapped to their +lower case counterparts. +.coNP Function @ string-extend +.synb +.mets (string-extend < string << tail ) +.syne +.desc The -.code iapply -function ("improper apply") is similar to -.codn apply , -except with regard to the treatment of -.metn trailing-args . -Firstly, under -.codn iapply , -if -.meta trailing-args -is an atom other than -.code nil -(possibly a sequence, such as a vector or string), -then it is treated as an ordinary argument: -.meta function -is invoked with a proper argument list, whose last element is -.metn trailing-args . -Secondly, if -.meta trailing-args -is a list, but an improper list, then the terminating atom of -.meta trailing-args -becomes an ordinary argument. Thus, in all possible cases, -.code iapply -treats an extra -.cod2 non- nil -atom as an argument, and never calls -.meta function -with an improper argument list. +.code string-extend +function destructively increases the length of +.metn string , +which must be an ordinary dynamic string. It is an error to invoke this +function on a literal string or a lazy string. -.TP* Examples: -.cblk - ;; '(1 2 3) becomes arguments to list, thus (list 1 2 3). - (apply (fun list) '(1 2 3)) -> (1 2 3) +The +.meta tail +argument can be a character, string or integer. If it is a string or +character, it specifies material which is to be added to the end of the string: +either a single character or a sequence of characters. If it is an integer, it +specifies the number of characters to be added to the string. - ;; this effectively invokes (list 1 2 3 4) - (apply (fun list) 1 2 '(3 4)) -> (1 2 3) +If +.meta tail +is an integer, the newly added characters have indeterminate contents. +The string appears to be the original one because of an internal terminating +null character remains in place, but the characters beyond the terminating zero +are indeterminate. - ;; this effectively invokes (list 1 2 . 3) - (apply (fun list) 1 2 3)) -> (1 2 . 3) +.coNP Function @ stringp +.synb +.mets (stringp << obj ) +.syne +.desc +The +.code stringp +function returns t if +.meta obj +is one of the several +kinds of strings. Otherwise it returns +.codn nil . - ;; "abc" is separated into characters which become arguments of list - (apply (fun list) "abc") -> (#\ea #\eb #\ec) -.cble +.coNP Function @ length-str +.synb +.mets (length-str << string ) +.syne +.desc +The +.code length-str +function returns the length +.meta string +in characters. The argument must be a string. -.TP* "Dialect Note:" -Note that some uses of this function that are necessary in other Lisp dialects -are not necessary in \*(TL. The reason is that in \*(TL, improper list -syntax is accepted as a compound form, and performs application: +.coNP Function @ search-str +.synb +.mets (search-str < haystack < needle >> [ start <> [ from-end ]]) +.syne +.desc +The +.code search-str +function finds an occurrence of the string +.meta needle +inside +the +.meta haystack +string and returns its position. If no such occurrence exists, +it returns +.codn nil . -.cblk - (foo a b . x) -.cble +If a +.meta start +argument is not specified, it defaults to zero. If it is +a non-negative integer, it specifies the starting character position for +the search. Negative values of +.meta start +indicate positions from the end of the +string, such that +.code -1 +is the last character of the string. -Here, the variables -.code a -and -.code b -supply the first two arguments for -.codn foo . -In -the dotted position, -.code x -must evaluate to a list or vector. The list or -vector's elements are pulled out and treated as additional arguments for -.codn foo . -Of course, this syntax can only be used if -.code x -is a symbolic form or an atom. It -cannot be a compound form, because -.code (foo a b . (x)) -and -.code (foo a b x) -are equivalent structures. +If the +.meta from-end +argument is specified and is not +.codn nil , +it means +that the search is conducted right-to-left. If multiple matches are possible, +it will find the rightmost one rather than the leftmost one. -.coNP Functions @ reduce-left and @ reduce-right +.coNP Function @ search-str-tree .synb -.mets (reduce-left < binary-function < list -.mets \ \ \ \ \ \ \ \ \ \ \ \ >> [ init-value <> [ key-function ]]) - -.mets (reduce-right < binary-function < list -.mets \ \ \ \ \ \ \ \ \ \ \ \ \ >> [ init-value <> [ key-function ]]) +.mets (search-str-tree < haystack < tree >> [ start <> [ from-end ]]) .syne .desc The -.code reduce-left -and -.code reduce-right -functions reduce lists of operands specified -by -.meta list -and -.meta init-value -to a single value by the repeated application of -.metn binary-function . +.code search-str-tree +function is similar to +.codn search-str , +except that instead of +searching +.meta haystack +for the occurrence of a single needle string, it searches +for the occurrence of numerous strings at the same time. These search strings +are specified, via the +.meta tree +argument, as an arbitrarily structured tree whose +leaves are strings. -An effective list of operands is formed by combining -.meta list -and -.metn init-value . -If -.meta key-function -is specified, then the items of -.meta list -are -mapped to a new values through -.metn key-function . -If -.meta init-value -is supplied, -then in the case of -.codn reduce-left , -the effective list of operands is formed by -prepending -.meta init-value -to -.metn list . -In the case of -.codn reduce-right , -the effective operand list is produced by appending -.meta init-value -to -.metn list . +The function finds the earliest possible match, in the given search direction, +from among all of the needle strings. -The production of the effective list can be expressed like this, -though this is not to be understood as the actual implementation: +If +.meta tree +is a single string, the semantics is equivalent to +.codn search-str . -.cblk - (append (if init-value-present (list init-value)) - [mapcar (or key-function identity) list])))) -.cble +.coNP Function @ match-str +.synb +.mets (match-str < bigstring < littlestring <> [ start ]) +.syne +.desc +Without the +.meta start +argument, the +.code match-str +function determines whether +.meta littlestring +is a prefix of +.metn bigstring , +returning a +.code t +or +.code nil +indication. -In the -.code reduce-right -case, the arguments to -.code append -are reversed. +If the +.meta start +argument is specified, and is a non-negative integer, then the +function tests whether +.meta littlestring +matches a prefix of that portion of +.meta bigstring +which starts at the given position. -If the effective list of operands is empty, then -.meta binary-function -is called -with no arguments at all, and its value is returned. This is the only -case in which -.meta binary-function -is called with no arguments; in all -remaining cases, it is called with two arguments. +If the +.meta start +argument is a negative integer, then +.code match-str +determines +whether +.meta littlestring +is a suffix of +.metn bigstring , +ending on that position +of bigstring, where +.code -1 +denotes the last character of +.metn bigstring , +.code -2 +the second last one and so on. -If the effective list contains one item, then that item is returned. +If +.meta start +is +.codn -1 , +then this corresponds to testing whether +.meta littlestring +is a suffix of +.metn bigstring . -Otherwise, the effective list contains two or more items, and is decimated as -follows. +.coNP Function @ match-str-tree +.synb +.mets (match-str-tree < bigstring < tree <> [ start ]) +.syne +.desc +The +.code match-str-tree +function is a generalization of match-str which matches +multiple test strings against +.meta bigstring +at the same time. The value +reported is the longest match from among any of the strings. -Note that an -.meta init-value -specified as -.code nil -is not the same as a missing -.metn init-value ; -this means that the initial value is the object -.codn nil . -Omitting -.meta init-value -is the same as specifying a value of -.code : -(the colon symbol). -It is possible to specify -.meta key-function -while omitting an -.meta init-value -argument. This is achieved by explicitly specifying -.code : -as the -.meta init-value -argument. +The strings are specified as an arbitrarily shaped tree structure which has +strings at the leaves. -Under -.codn reduce-left , -the leftmost pair of operands is removed -from the list and passed as arguments to -.metn binary-function , -in the same order -that they appear in the list, and the resulting value initializes an -accumulator. Then, for each remaining item in the list, -.meta binary-function -is invoked on two arguments: the current accumulator value, and the next element -from the list. After each call, the accumulator is updated with the return -value of -.metn binary-function . -The final value of the accumulator is returned. +If +.meta tree +is a single string atom, then the function behaves +exactly like match-str. -Under -.codn reduce-right , -the list is processed right to left. The rightmost -pair of elements in the effective list is removed, and passed as arguments to -.metn binary-function , -in the same order that they appear in the list. The -resulting value initializes an accumulator. Then, for each remaining item in -the list, -.meta binary-function -is invoked on two arguments: the -next element from the list, in right to left order, and the current -accumulator value. After each call, the accumulator is updated with the return -value of -.metn binary-function . -The final value of the accumulator is returned. - -.TP* Examples: -.cblk - ;;; effective list is (1) so 1 is returned - (reduce-left (fun +) () 1 nil) -> 1 - - ;;; computes (- (- (- 0 1) 2) 3) - (reduce-left (fun -) '(1 2 3) 0 nil) -> -6 +.coNP Function @ sub-str +.synb +.mets (sub-str < string >> [ from <> [ to ]]) +.syne +.desc +The +.code sub-str +function is like the more generic function +.codn sub , +except that it +operates only on strings. For a description of the arguments and semantics, +refer to the +.code sub +function. - ;;; computes (- 1 (- 2 (- 3 0))) - (reduce-right (fun -) '(1 2 3) 0 nil) -> 2 +.coNP Function @ replace-str +.synb +.mets (replace-str < string < item-sequence >> [ from <> [ to ]]) +.syne +.desc +The +.code replace-str +function is like the +.code replace +function, except that the first +argument must be a string. - ;;; computes (* 1 2 3) - (reduce-left (fun *) '((1) (2) (3)) nil (fun first)) -> 6 +For a description of the arguments, semantics and return value, refer to the +.code replace +function. - ;;; computes 1 because the effective list is empty - ;;; and so * is called with no arguments, which yields 1. - (reduce-left (fun *) nil) -.cble +.coNP Function @ cat-str +.synb +.mets (cat-str < string-list <> [ sep-string ]) +.syne +.desc +The +.code cat-str +function catenates a list of strings given by +.meta string-list +into a +single string. The optional +.meta sep-string +argument specifies a separator string +which is interposed between the catenated strings. -.coNP Function @, some @ all and @ none +.coNP Function @ split-str .synb -.mets (some < sequence >> [ predicate-fun <> [ key-fun ]]) -.mets (all < sequence >> [ predicate-fun <> [ key-fun ]]) -.mets (none < sequence >> [ predicate-fun <> [ key-fun ]]) +.mets (split-str < string << sep ) .syne .desc The -.codn some , -.code all -and -.code none -functions apply a predicate test function -.meta predicate-fun -over a list of elements. If the argument -.meta key-fun -is -specified, then elements of -.meta sequence -are passed into -.metn key-fun , -and -.meta predicate-fun -is -applied to the resulting values. If -.meta key-fun -is omitted, the behavior is -as if -.meta key-fun -is the identity function. If -.meta predicate-fun -is omitted, -the behavior is as if -.meta predicate-fun -is the identity function. +.code split-str +function breaks the +.meta string +into pieces, returning a list +thereof. The +.meta sep +argument must be either a string or a regular expression. +It specifies the separator character sequence within +.metn string . -These functions have short-circuiting semantics and return conventions similar -to the and and or operators. +All non-overlapping matches for +.meta sep +within +.meta string +are identified in left +to right order, and are removed from +.metn string . +The string is broken into pieces +according to the gaps left behind by the removed separators, and a list +of the remaining pieces is returned. -The some function applies -.meta predicate-fun -to successive values -produced by retrieving elements of -.meta list -and processing them through -.metn key-fun . -If the list is empty, it returns -.codn nil . -Otherwise it returns the -first -.cod2 non- nil -return value returned by a call to -.meta predicate-fun -and -stops evaluating more elements. If -.meta predicate-fun -returns -.code nil -for all -elements, it returns -.metn nil . +If +.meta sep +is the empty string, then the separator pieces removed from the +string are considered to be the empty strings between its +characters. In this case, if +.meta string +is of length one or zero, then it is considered to have no such pieces, and a +list of one element is returned containing the original string. -The -.code all -function applies -.meta predicate-fun -to successive values -produced by retrieving elements of -.meta list -and processing them through -.metn key-fun . -If the list is empty, it returns -.codn t . -Otherwise, if -.meta predicate-fun -yields -.code nil -for any value, the -.cod all -function immediately -returns without invoking -.meta predicate-fun -on any more elements. -If all the elements are processed, then the all function returns -the value which -.meta predicate-fun -yielded for the last element. +If a match for +.meta sep +is not found in the string at all, then the string is not +split at all: a list of one element is returned containing the original +string. -The -.code none -function applies -.meta predicate-fun -to successive values -produced by retrieving elements of -.meta list -and processing them through -.metn key-fun . -If the list is empty, it returns -.codn t . -Otherwise, if -.meta predicate-fun -yields -.cod2 non- nil -for any value, the none function -immediately returns nil. If -.meta predicate-fun -yields nil for all -values, the none function returns -.codn t . +If +.meta sep +matches the entire string, then a list of two empty strings is +returned, except in the case that the original string is empty, in which case a +list of one element is returned, containing the empty string. -.TP* Examples: +Whenever two adjacent matches for +.meta sep +occur, they are considered separate +cuts with an empty piece between them. -.cblk - ;; some of the integers are odd - [some '(2 4 6 9) oddp] -> t +This operation is nondestructive: +.meta string +is not modified in any way. - ;; none of the integers are even - [none '(1 3 4 7) evenp] -> t +Note: To split a string into pieces of length one such that an empty string +produces +.code nil +rather than +.codn ("") , +use the +.cblk +.meti (tok-str < string #/./) .cble +pattern. -.coNP Function @ multi +.coNP Function @ split-str-set .synb -.mets (multi < function << list *) +.mets (split-str-set < string << set ) .syne .desc The -.code multi -function distributes an arbitrary list processing function -.meta multi -over multiple lists given by the -.meta list -arguments. - -The -.meta list -arguments are first transposed into a single list of tuples. Each -successive element of this transposed list consists of a tuple of the -successive items from the lists. The length of the transposed list is that -of the shortest -.meta list -argument. +.code split-str-set +function breaks the +.meta string +into pieces, returning a list +thereof. The +.meta set +argument must be a string. It specifies a set of +characters. All occurrences of any of these characters within +.meta string +are +identified, and are removed from +.metn string . +The string is broken into pieces +according to the gaps left behind by the removed separators. -The transposed list is then passed to -.meta function -as an argument. - -The -.meta function -is expected to produce a list of tuples, which are transposed -again to produce a list of lists which is then returned. +Adjacent occurrences of characters from +.meta set +within +.meta string +are considered to +be separate gaps which come between empty strings. -Conceptually, the input lists are columns and -.meta function -is invoked on -a list of the rows formed from these columns. The output of -.meta function -is a transformed list of rows which is reconstituted into a list of columns. +This operation is nondestructive: +.meta string +is not modified in any way. -.TP* Example: +.coNP Functions @ tok-str and @ tok-where +.synb +.mets (tok-str < string < regex <> [ keep-between ]) +.mets (tok-where < string << regex ) +.syne +.desc +The +.code tok-str +function searches +.meta string +for tokens, which are defined as +substrings of +.mta string +which match the regular expression +.meta regex +in the +longest possible way, and do not overlap. These tokens are extracted from the +string and returned as a list. +Whenever +.meta regex +matches an empty string, then an empty token is returned, and +the search for another token within +.meta string +resumes after advancing by one +character position. So for instance, +.code (tok-str "abc" #/a?/) +returns the .cblk - ;; Take three lists in parallel, and remove from all of them - ;; them the element at all positions where the third list - ;; has an element of 20. - - (multi (op remove-if (op eql 20) @1 third) - '(1 2 3) - '(a b c) - '(10 20 30)) - - -> ((1 3) (a c) (10 30)) - - ;; The (2 b 20) "row" is gone from the three "columns". - - ;; Note that the (op remove if (op eql 20) @1 third) - ;; expression can be simplified using the ap operator: - ;; - ;; (op remove-if (ap eql @3 20)) +("a" "" "" ""). .cble +After the token +.str "a" +is extracted from a non-empty match +for the regex, the regex is considered to match three more times: before the +.strn "b" , +between +.str "b" +and +.strn "c" , +and after the +.strn "c" . -.SS* Association Lists +If the +.meta keep-between +argument is specified, and is not +.codn nil , +then the behavior +of +.code tok-str +changes in the following way. The pieces of +.meta string +which are +skipped by the search for tokens are included in the output. If no token is +found in +.metn string , +then a list of one element is returned, containing +.metn string . +Generally, if N tokens are found, then the returned list consists of 2N + 1 +elements. The first element of the list is the (possibly empty) substring which +had to be skipped to find the first token. Then the token follows. The next +element is the next skipped substring and so on. The last element is the +substring of +.meta string +between the last token and the end. -Association lists are ordinary lists formed according to a special convention. -Firstly, any empty list is a valid association list. A non-empty association -list contains only cons cells as the key elements. These cons cells are -understood to represent key/value associations, hence the name "association -list". +The +.code tok-where +function works similarly to +.codn tok-str , +but instead of returning +the extracted tokens themselves, it returns a list of the character position +ranges within +.meta string +where matches for +.meta regex +occur. The ranges +are pairs of numbers, represented as cons cells, where the first number +of the pair gives the starting character position, and the second number +is one position past the end of the match. If a match is empty, then the +two numbers are equal. -.coNP Function @ assoc +The tok-where function does not support the +.meta keep-between +parameter. + +.coNP Function @ list-str .synb -.mets (assoc < key << alist ) +.mets (list-str << string ) .syne .desc The -.code assoc -function searches an association list -.meta alist -for a cons cell whose -car field is equivalent to -.meta key -(with equality determined by the equal -function). The first such cons is returned. If no such cons is found, -.code nil -is returned. +.code list-str +function converts a string into a list of characters. -.coNP Function @ assql +.coNP Function @ trim-str .synb -.mets (assql < key << alist ) +.mets (trim-str << string ) .syne .desc The -.code assql -function is just like -.codn assoc , -except that the equality test -is determined using the -.code eql -function rather than -.codn equal . +.code trim-str +function produces a copy of +.meta string +from which leading and +trailing whitespace is removed. Whitespace consists of spaces, tabs, +carriage returns, linefeeds, vertical tabs and form feeds. -.coNP Function @ acons +.coNP Function @ chrp .synb -.mets (acons < car < cdr << alist ) +.mets (chrp << obj ) .syne .desc +Returns +.code t +if +.meta obj +is a character, otherwise nil. -The -.code acons -function constructs a new alist by consing a new cons to the -front of -.metn alist . -The following equivalence holds: - -.cblk - (acons car cdr alist) <--> (cons (cons car cdr) alist) -.cble - -.coNP Function @ acons-new +.coNP Function @ chr-isalnum .synb -.mets (acons-new < car < cdr << alist ) +.mets (chr-isalnum << char ) .syne .desc +Returns +.code t +if +.meta char +is an alpha-numeric character, otherwise nil. Alpha-numeric +means one of the upper or lower case letters of the English alphabet found in +ASCII, or an ASCII digit. This function is not affected by locale. -The -.code acons-new -function searches -.metn alist , -as if using the assoc function, -for an existing cell which matches the key provided by the car argument. -If such a cell exists, then its cdr field is overwritten with the -.meta cdr -argument, and then the -.meta alist -is returned. If no such cell exists, then -a new list is returned by adding a new cell to the input list consisting -of the -.meta car -and -.meta cdr -values, as if by the -.code acons -function. - -.coNP Function @ aconsql-new +.coNP Function @ chr-isalpha .synb -.mets (aconsql-new < car < cdr << alist ) +.mets (chr-isalpha << char ) .syne .desc -This function is like -.codn acons-new , -except that the -.code -eql -function is used -for equality testing. Thus, the list is searched for an existing cell -as if using the -.code assql -function rather than -.codn assoc . +Returns +.code t +if +.meta char +is an alphabetic character, otherwise +.codn nil . +Alphabetic +means one of the upper or lower case letters of the English alphabet found in +ASCII. This function is not affected by locale. -.coNP Function @ alist-remove +.coNP Function @ chr-isascii .synb -.mets (alist-remove < alist << keys ) +.mets (chr-isalpha << char ) .syne .desc -The -.code alist-remove -function takes association list -.meta alist -and produces a -duplicate from which cells matching the specified keys have been removed. The -.meta keys -argument is a list of the keys not to appear in the output list. +This function returns +.code t +if the code of character +.meta char +is in the range 0 to 127 inclusive. For characters outside of this range, it +returns +.codn nil . -.coNP Function @ alist-nremove +.coNP Function @ chr-iscntrl .synb -.mets (alist-nremove < alist << keys ) +.mets (chr-iscntrl << char ) .syne .desc -The -.code alist-nremove -function is like -.codn alist-remove , -but potentially destructive. -The input list -.meta alist -may be destroyed and its structural material re-used to -form the output list. The application should not retain references to the input -list. +This function returns +.code t +if the character +.meta char +is a character whose code +ranges from 0 to 31, or is 127. In other words, any non-printable ASCII +character. For other characters, it returns +.codn nil . -.coNP Function @ copy-alist +.coNP Function @ chr-isdigit .synb -.mets (copy-alist << alist ) +.mets (chr-isdigit << char ) .syne .desc -The -.code copy-alist -function duplicates -.codn alist . -Unlike -.codn copy-list , -which only duplicates list structure, -.code copy-alist -also duplicates each cons -cell of the input alist. That is to say, each element of the output list -is produced as if by the -.code copy-cons -function applied to the corresponding -element of the input list. +This function returns +.code t +if the character +.meta char +is is an ASCII digit. +Otherwise, it returns +.codn nil . -.SS* Property Lists -.coNP Function @ prop +.coNP Function @ chr-isgraph .synb -.mets (prop < plist << key ) +.mets (chr-isgraph << char ) .syne .desc -A property list a flat list of even length consisting of interleaved -pairs of property names (usually symbols) and their values (arbitrary -objects). An example property list is (:a 1 :b "two") which contains -two properties, :a having value 1, and :b having value "two". - -The -.code prop -function searches property list -.meta plist -for key -.metn key . -If the key is found, then the value next to it is returned. Otherwise -.code nil -is returned. +This function returns +.code t +if +.meta char +is a non-space printable ASCII character. +It returns nil if it is a space or control character. -It is ambiguous whether -.code nil -is returned due to the property not being -found, or due to the property being present with a -.code nil -value. +It also returns nil for non-ASCII characters: Unicode characters with a code +above 127. -.SS* List Sorting -.coNP Function @ merge +.coNP Function @ chr-islower .synb -.mets (merge < seq1 < seq2 >> [ lessfun <> [ keyfun ]]) +.mets (chr-islower << char ) .syne .desc -The -.code merge -function merges two sorted sequences -.meta seq1 -and -.meta seq2 -into a single -sorted sequence. The semantics and defaulting behavior of the -.meta lessfun -and -.meta keyfun -arguments are the same as those of the sort function. - -The sequence which is returned is of the same kind as -.metn seq1 . - -This function is destructive of any inputs that are lists. If the output -is a list, it is formed out of the structure of the input lists. +This function returns +.code t +if +.meta char +is an ASCII lower case letter. Otherwise it returns +.codn nil . -.coNP Function @ multi-sort +.coNP Function @ chr-isprint .synb -.mets (multi-sort < columns < less-funcs <> [ key-funcs ]) +.mets (chr-isprint << char ) .syne .desc -The -.code multi-sort -function regards a list of lists to be the columns of a -database. The corresponding elements from each list constitute a record. -These records are to be sorted, producing a new list of lists. - -The -.meta columns -argument supplies the list of lists which comprise the columns of -the database. The lists should ideally be of the same length. If the lists are -of different lengths, then the shortest list is taken to be the length of the -database. Excess elements in the longer lists are ignored, and do not appear in -the sorted output. - -The -.meta less-funcs -argument supplies a list of comparison functions which are -applied to the columns. Successive functions correspond to successive -columns. If -.meta less-funcs -is an empty list, then the sorted database will -emerge in the original order. If -.meta less-funcs -contains exactly one function, -then the rows of the database is sorted according to the first column. The -remaining columns simply follow their row. If -.meta less-funcs -contains more than -one function, then additional columns are taken into consideration if the items -in the previous columns compare -.codn equal . -For instance if two elements from column -one compare -.codn equal , -then the corresponding second column elements are compared -using the second column comparison function. - -The optional -.meta key-funcs -argument supplies transformation functions through -which column entries are converted to comparison keys, similarly to the single -key function used in the sort function and others. If there are more key -functions than less functions, the excess key functions are ignored. +This function returns +.code t +if +.meta char +is an ASCII character which is not a +control character. It also returns +.code nil +for all non-ASCII characters: Unicode +characters with a code above 127. -.SS* Lazy Lists and Lazy Evaluation -.coNP Function @ make-lazy-cons +.coNP Function @ chr-ispunct .synb -.mets (make-lazy-cons << function ) +.mets (chr-ispunct << char ) .syne .desc -The function -.code make-lazy-cons -makes a special kind of cons cell called a lazy -cons, or lcons. Lazy conses are useful for implementing lazy lists. - -Lazy lists are lists which are not allocated all at once. Rather, -their elements materialize when they are accessed, like -magic stepping stones appearing under one's feet out of thin air. - -A lazy cons has -.code car -and -.code cdr -fields like a regular cons, and those -fields are initialized to -.code nil -when the lazy cons is created. A lazy cons also -has an update function, the one which is provided as the -.meta function -argument to -.codn make-lazy-cons . - -When either the -.code car -and -.code cdr -fields of a cons are accessed for the first time, -the function is automatically invoked first. That function has the opportunity -to initialize the -.code car -and -.code cdr -fields. Once the function is called, it is removed -from the lazy cons: the lazy cons no longer has an update function. - -To continue a lazy list, the function can make another call to -.code make-lazy-cons -and install the resulting cons as the -.code cdr -of the lazy cons. - -.TP* Example: - -.cblk - ;;; lazy list of integers between min and max - (defun integer-range (min max) - (let ((counter min)) - ;; min is greater than max; just return empty list, - ;; otherwise return a lazy list - (if (> min max) - nil - (make-lazy-cons - (lambda (lcons) - ;; install next number into car - (rplaca lcons counter) - ;; now deal wit cdr field - (cond - ;; max reached, terminate list with nil! - ((eql counter max) - (rplacd lcons nil)) - ;; max not reached: increment counter - ;; and extend with another lazy cons - (t - (inc counter) - (rplacd lcons (make-lazy-cons - (lcons-fun lcons)))))))))) -.cble +This function returns +.code t +if +.meta char +is an ASCII character which is not a +control character. It also returns nil for all non-ASCII characters: Unicode +characters with a code above 127. -.coNP Function @ lconsp +.coNP Function @ chr-isspace .synb -.mets (lconsp << value ) +.mets (chr-isspace << char ) .syne .desc -The -.code lconsp -function returns +This function returns .code t if -.meta value -is a lazy cons cell. Otherwise -it returns -.codn nil , -even if -.meta value -is an ordinary cons cell. +.meta char +is an ASCII whitespace character: any of the +characters in the set +.codn #\espace , +.codn #\etab , +.codn #\elinefeed , +.codn #\enewline , +.codn #\ereturn , +.code #\evtab +and +.codn #\epage . +For all other characters, it returns +.codn nil . -.coNP Function @ lcons-fun +.coNP Function @ chr-isblank .synb -.mets (lcons-fun << lazy-cons ) +.mets (chr-isblank << char ) .syne .desc -The -.code lcons-fun -function retrieves the update function of a lazy cons. -Once a lazy cons has been accessed, it no longer has an update function -and -.code lcons-fun -returns +This function returns +.code t +if +.meta char +is a space or tab: the character +.code #\espace +or +.codn #\etab . +For all other characters, it returns .codn nil . -While the update function of a lazy cons is -executing, it is still accessible. This allows the update function -to retrieve a reference to itself and propagate itself into -another lazy cons (as in the example under -.codn make-lazy-cons ). -.coNP Macro @ lcons +.coNP Function @ chr-isunisp .synb -.mets (lcons < car-expression << cdr-expression ) +.mets (chr-isunisp << char ) .syne .desc -The -.code lcons -macro simplifies the construction of structures based on lazy conses. -Syntactically, it resembles the -.code cons -function. However, the arguments are expressions rather than values. -The macro generates code which, when evaluated, immediately produces -a lazy cons. The expressions -.meta car-expression +This function returns +.code t +if +.meta char +is a Unicode whitespace character. This the case for +all the characters for which +.code chr-isspace +returns +.codn t. +It also returns +.code t +for these additional characters: +.codn #\exa0 , +.codn #\ex1680 , +.codn #\ex180e , +.codn #\ex2000 , +.codn #\ex2001 , +.codn #\ex2002 , +.codn #\ex2003 , +.codn #\ex2004 , +.codn #\ex2005 , +.codn #\ex2006 , +.codn #\ex2007 , +.codn #\ex2008 , +.codn #\ex2009 , +.codn #\ex200a , +.codn #\ex2028 , +.codn #\ex2029 , +.codn #\ex205f , and -.meta cdr-expression -are not immediately evaluated. Rather, when either the -.code car -or -.code cdr -field of the lazy cons cell is accessed, these expressions are both -evaluated at that time, in the order that they appear in the -.code lcons -expression, and in the original lexical scope in which that -expression was evaluated. The return values of these expressions -are used, respectively, to initialize the corresponding fields -of the lazy cons. - -Note: the -.code lcons -macro may be understood in terms of the following reference -implementation, as a syntactic sugar combining the -.code make-lazy-cons -constructor with a lexical closure provided by a -.code lambda -function: +.codn #\ex3000 . +For all other characters, it returns +.codn nil . -.cblk - (defmacro lcons (car-form cdr-form) - (let ((lc (gensym))) - ^(make-lazy-cons (lambda (,lc) - (rplaca ,lc ,car-form) - (rplacd ,lc ,cdr-form))))) -.cble +.coNP Function @ chr-isupper +.synb +.mets (chr-isupper < char ) +.syne +.desc +This function returns +.code t +if +.meta char +is an ASCII upper case letter. Otherwise it returns +.codn nil . -.TP* Example: +.coNP Function @ chr-isxdigit +.synb +.mets (chr-isxdigit << char ) +.syne +.desc +This function returns +.code t +if +.meta char +is a hexadecimal digit. One of the ASCII +letters +.code A +through +.codn F , +or their lower-case equivalents, or an ASCII digit +.code 0 +through +.codn 9 . -.cblk - ;; Given the following function ... +.coNP Function @ chr-toupper +.synb +.mets (chr-toupper << char ) +.syne +.desc +If character +.meta char +is a lower case ASCII letter character, this function +returns the upper case equivalent character. If it is some other +character, then it just returns +.metn char . - (defun fib-generator (a b) - (lcons a (fib-generator b (+ a b)))) +.coNP Function @ chr-tolower +.synb +.mets (chr-tolower << char ) +.syne +.desc +If character +.meta char +is an upper case ASCII letter character, this function +returns the lower case equivalent character. If it is some other +character, then it just returns +.metn char . - ;; ... the following function call generates the Fibonacci - ;; sequence as an infinite lazy list. +.coNP Functions @ num-chr and @ chr-num +.synb +.mets (num-chr << char ) +.mets (chr-num << num ) +.syne +.desc +The argument +.meta char +must be a character. The +.code num-chr +function returns that +character's Unicode code point value as an integer. - (fib-generator 1 1) -> (1 1 2 3 5 8 13 ...) -.cble +The argument +.meta num +must be a fixnum integer in the range +.code 0 +to +.codn #\ex10FFFF . +The argument is taken to be a Unicode code point value and the +corresponding character object is returned. -.coNP Functions @ lazy-stream-cons and @ get-lines +.coNP Accessor @ chr-str .synb -.mets (lazy-stream-cons << stream ) -.mets (get-lines <> [ stream ]) +.mets (chr-str < str << idx ) +.mets (set (chr-str < str << idx ) << new-value ) .syne .desc The -.code lazy-stream-cons -and -.code get-lines -functions are synonyms, except that the -.meta stream -argument is optional in -.code get-lines -and defaults to -.codn *stdin* . -Thus, the following -description of -.code lazy-stream-cons -also applies to -.codn get-lines . +.code chr-str +function performs random access on string +.meta str +to retrieve +the character whose position is given by integer +.metn idx , +which must +be within range of the string. -The -.code lazy-stream-cons -returns a lazy cons which generates a lazy list based on -reading lines of text from input stream -.metn stream , -which form the elements of -the list. The -.code get-line -function is called on demand to add elements to the -list. +The index value 0 corresponds to the first (leftmost) character of the string +and so non-negative values up to one less than the length are possible. -The -.code lazy-stream-cons -function itself makes the first call to -.code get-line -on the stream. If this returns -.codn nil , -then the stream is closed and -.code nil -is -returned. Otherwise, a lazy cons is returned whose update function will install -that line into the -.code car -field of the lazy cons, and continue the lazy list -by making another call to -.codn lazy-stream-cons , -installing the result into the -.code cdr -field. +Negative index values are also allowed, such that -1 corresponds to the +last (rightmost) character of the string, and so negative values down to +the additive inverse of the string length are possible. -.code lazy-stream-cons -inspects the real-time property of a stream -as if by the -.code real-time-stream-p -function. This determines which of two -styles of lazy list are returned. For an ordinary (non-real-time) stream, -the lazy list treats the end-of-file condition accurately: an empty -file turns into the empty list -.codn nil , -a one line file into a one-element -list which contains that line and so on. This accuracy requires one -line of lookahead which is not acceptable in real-time streams, and -so a different type of lazy list is used, which generates an extra -.code nil -item after the last line. Under this type of lazy list, an empty input stream -translates to the list -.codn (nil) ; -a one-line stream translates to -.code ("line" nil) -and so forth. - -.coNP Macro @ delay -.synb -.mets (delay << expression ) -.syne -.desc -The delay operator arranges for the delayed (or "lazy") evaluation of -.metn expression . -This means that the expression is not evaluated immediately. -Rather, the delay expression produces a promise object. +An empty string cannot be indexed. A string of length one supports index 0 and +index -1. A string of length two is indexed left to right by the values 0 and +1, and from right to left by -1 and -2. -The promise object can later be passed to the -.code force -function (described -later in this document). The force function will trigger the evaluation -of the expression and retrieve the value. +If the element +.meta idx +of string +.meta str +exists, and the string is modifiable, then the +.code chr-str +form denotes a place. -The expression is evaluated in the original scope, no matter where -the -.code force -takes place. +A +.code chr-str +place +supports deletion. When a deletion takes place, +then the character at +.meta idx +is removed from the string. Any characters +after that position move by one position +to close the gap, and the length of the string +decreases by one. -The expression is evaluated at most once, by the first call to -.codn force . -Additional calls to -.code force -only retrieve a cached value. +.TP* Notes: -.TP* Example: +Direct use of +.code chr-str +is equivalent to the DWIM bracket notation except +that +.code str +must be a string. The following relation holds: .cblk - @(do - ;; list is popped only once: the value is computed - ;; just once when force is called on a given promise - ;; for the first time. - - (defun get-it (promise) - (format t "*list* is ~s\en" *list*) - (format t "item is ~s\en" (force promise)) - (format t "item is ~s\en" (force promise)) - (format t "*list* is ~s\en" *list*)) - - (defvar *list* '(1 2 3)) - - (get-it (delay (pop *list*)))) + (chr-str s i) --> [s i] +.cble - Output: +since +.codn [s i] <--> (ref s i) , +this also holds: - *list* is (1 2 3) - item is 1 - item is 1 - *list* is (2 3) +.cblk + (chr-str s i) --> (ref s i) .cble -.coNP Accessor @ force +However, note the following difference. When the expression +.code [s i] +is used as a place, then the subexpression +.code s +must be a place. When +.code (chr-str s i) +is used as a place, +.code s +need not be a place. + +.coNP Function @ chr-str-set .synb -.mets (force << promise ) -.mets (set (force << promise ) << new-value ) +.mets (chr-str-set < str < idx << char ) .syne .desc The -.code force -function accepts a promise object produced by the -.code delay -macro. -The first time -.code force -is invoked, the -.meta expression -which was wrapped inside -.meta promise -by the -.code delay -macro is evaluated (in its original lexical environment, regardless of where in -the program the -.code force -call takes place). The value of -.meta expression -is -cached inside -.meta promise -and returned, becoming the return value of the -.code force -function call. If the -.code force -function is invoked additional times on -the same promise, the cached value is retrieved. +.code chr-str +function performs random access on string +.meta str +to overwrite +the character whose position is given by integer +.metn idx , +which must +be within range of the string. The character at +.meta idx +is overwritten +with character +.metn char . -A -.code force -form is a syntactic place, denoting the value cache location within -.metn promise . +The +.meta idx +argument works exactly as in +.codn chr-str . -Storing a value in a -.code force -place causes future accesses to the -.meta promise -to return that value. +The +.meta str +argument must be a modifiable string. -If the promise had not yet been forced, then -storing a value into it prevents that from ever happening. The -delayed -.meta expression -will never be evaluated. +.TP* Notes: -If, while a promise is being forced, the evaluation of -.meta expression -itself causes an assignment to the promise, it is not specified whether -the promise will take on the value of -.meta expression -or the assigned value. +Direct use of +.code chr-str +is equivalent to the DWIM bracket notation except +that +.meta str +must be a string. The following relation holds: -.coNP Macro @ mlet +.cblk + (chr-str-set s i c) --> (set [s i] c) +.cble + +since +.codn (set [s i] c) <--> (refset s i c) , +this also holds: + +.cblk + (chr-str s i) --> (refset s i c) +.cble + +.coNP Function @ span-str .synb -.mets (mlet >> ({ sym | >> ( sym << init-form )}*) << body-form *) +.mets (span-str < str << set ) .syne .desc The -.code mlet -macro ("magic let" or "mutual let") implements a variable binding construct -similar to -.code let -and -.codn let* . +.code span-str +function determines the longest prefix of string +.meta str +which +consists only of the characters in string +.metn set , +in any combination. -Under -.codn mlet , -the scope of the bindings of the -.meta sym -variables extends over the -.metn init-form -s, -as well as the -.metn body-form -s. +.coNP Function @ compl-span-str +.synb +.mets (compl-span-str < str << set ) +.syne +.desc +The +.code compl-span-str +function determines the longest prefix of string +.meta str +which +consists only of the characters which do not appear in +.metn set , +in any combination. -Unlike the -.code let* -construct, each -.meta init-form -has each -.meta sym -in scope. That is to say, an -.metn init-form -can refer not only to previous variables, but also to later variables -as well as to its own variable. +.coNP Function @ break-str +.synb +.mets (break-str < str << set ) +.syne +.desc +The +.code break-str +function returns an integer which represents the position of the +first character in string +.meta str +which appears in string +.metn set . -The variables are not initialized until their values are accessed for -the first time. Any -.meta sym -whose value is not accessed is not initialized. - -Furthermore, the evaluation of each -.meta init-form -does not take place until the time when its value is needed -to initialize the associated -.metn sym . -This evaluation takes place once. If a given -.meta sym -is not accessed during the evaluation of the -.code mlet -construct, then its -.meta init-form -is never evaluated. - -The bound variables may be assigned. If, before initialization, a variable is -updated in such a way that its prior value is not needed, it is unspecified -whether initialization takes place, and thus whether its -.meta init-form -is evaluated. +If there is no such character, then +.code nil +is returned. -Direct circular references erroneous and are diagnosed. +.SS* Lazy Strings +Lazy strings are objects that were developed for the \*(TX pattern matching +language, and are exposed via \*(TL. Lazy strings behave much like strings, +and can be substituted for strings. However, unlike regular strings, which +exist in their entirety, first to last character, from the moment they are +created, lazy strings do not exist all at once, but are created on demand. If +character at index N of a lazy string is accessed, then characters 0 through N +of that string are forced into existence. However, characters at indices +beyond N need not necessarily exist. -.TP* Examples: +A lazy string dynamically grows by acquiring new text from a list of strings +which is attached to that lazy string object. When the lazy string is accessed +beyond the end of its hitherto materialized prefix, it takes enough strings +from the list in order to materialize the index. If the list doesn't have +enough material, then the access fails, just like an access beyond the end of a +regular string. A lazy string always takes whole strings from the attached +list. -.cblk - ;; Dependent calculations in arbitrary order - (mlet ((x (+ y 3)) - (z (+ x 1)) - (y 4)) - (+ z 4)) --> 12 +Lazy string growth is achieved via the +.code lazy-str-force-upto +function which +forces a string to exist up to a given character position. This function is +used internally to handle various situations. - ;; Error: circular reference: - ;; x depends on y, y on z, but z on x again. - (mlet ((x (+ y 1)) - (y (+ z 1)) - (z (+ x 1))) - z) +The +.code lazy-str-force +function forces the entire string to materialize. If the +string is connected to an infinite lazy list, this will exhaust all memory. - ;; Okay: lazy circular reference because lcons is used - (mlet ((list (lcons 1 list))) - list) --> (1 1 1 1 1 ...) ;; circular list -.cble +Lazy strings are specially recognized in many of the regular string functions, +which do the right thing with lazy strings. For instance when +.code sub-str +is invoked on a lazy string, a special version of the +.code sub-str +logic is +used which handles various lazy string cases, and can potentially return +another lazy string. Taking a +.code sub-str +of a lazy string from a given character position +to the end does not force the entire lazy string to exist, +and in fact the operation will work on a lazy string that is infinite. -In the last example, the -.code list -variable is accessed for the first time in the body of the -.code mlet -form. This causes the evaluation of the -.code lcons -form. This form evaluates its arguments lazily, which means that it -is not a problem that -.code list -is not yet initialized. The form produces a lazy cons, which is then used -to initialize -.code list. -When the -.code car -or -.code cdr -fields of the lazy cons are accessed, the -.code list -expression in the -.code lcons -argument is accessed. By that time, the variable is initialized -and holds the lazy cons itself, which creates the circular reference, -and a circular list. +Furthermore, special lazy string functions are provided which allow programs to +be written carefully to take better advantage of lazy strings. What carefully +means is code that avoids unnecessarily forcing the lazy string. For instance, +in many situations it is necessary to obtain the length of a string, only to +test it for equality or inequality with some number. But it is not necessary to +compute the length of a string in order to know that it is greater than some +value. -.coNP Functions @ generate and @ giterate +.coNP Function @ lazy-str .synb -.mets (generate < while-fun << gen-fun ) -.mets (giterate < while-fun < gen-fun <> [ value ]) +.mets (lazy-str < string-list >> [ terminator <> [ limit-count ]]) .syne .desc The -.code generate -function produces a lazy list which dynamically produces items -according to the following logic. - -The arguments to -.code generate -are functions which do not take any arguments. The -return value of generate is a lazy list. +.code lazy-str +function constructs a lazy string which draws material from +.meta string-list +which is a list of strings. -When the lazy list is accessed, for instance with the functions car and cdr, it -produces items on demand. Prior to producing each item, -.meta while-fun -is -called. If it returns a true boolean value (any value other than -.codn nil ), -then -the -.meta gen-fun -function is called, and its return value is incorporated as -the next item of the lazy list. But if -.meta while-fun -yields -.codn nil , -then the lazy list immediately terminates. +If the optional +.meta terminator +argument is given, then it specifies a string +which is appended to every string from +.metn string-list , +before that string is +incorporated into the lazy string. If +.meta terminator +is not given, +then it defaults to the string +.strn "\en" , +and so the strings from +.meta string-list +are effectively treated as lines which get terminated by newlines +as they accumulate into the growing prefix of the lazy string. +To avoid the use of a terminator string, a null string +.meta terminator +argument +must be explicitly passed. In that case, the lazy string grows simply +by catenating elements from +.metn string-list . -Prior to returning the lazy list, generate invokes the -.meta while-fun -one time. -If -.code while-fun -yields -.codn nil , -then -.code generate -returns the empty list -.code nil -instead of a lazy list. Otherwise, it instantiates a lazy list, and invokes the -.code gen-func -to populate it with the first item. +If the +.meta limit-count +argument is specified, it must be a positive integer. It +expresses a maximum limit on how many elements will be consumed from +.meta string-list +in order to feed the lazy string. Once that many elements are +drawn, the string ends, even if the list has not been exhausted. +.coNP Function @ lazy-stringp +.synb +.mets (lazy-stringp << obj ) +.syne +.desc The -.code giterate -function is similar to -.codn generate , -except that -.meta while-fun -and -.meta gen-fun -are functions of one argument rather than functions of -no arguments. The optional -.meta value -argument defaults to -.code nil -and is threaded through the function calls. That is to say, the lazy -list returned is -.cblk -.meti >> ( value >> [ gen-fun << value ] >> [ gen-fun >> [ gen-fun << value ]] ...). -.cble - -The lazy list terminates when a value fails to satisfy -.metn while-fun . -That is to say, prior to generating each value, the lazy list tests -the value using -.metn while-fun . -If that function returns -.codn nil , -then the item is not added, and the sequence terminates. - -Note: -.code giterate -could be written in terms of -.code generate -like this: - -.cblk - (defun giterate (w g v) - (generate (lambda () [w v]) - (lambda () (prog1 v (set v [g v]))))) -.cble - -.TP* Example: - -.cblk - (giterate (op > 5) (op + 1) 0) -> (0 1 2 3 4) -.cble +.code lazy-stringp +function returns +.code t +if +.meta obj +is a lazy +string. Otherwise it returns +.codn nil . -.coNP Function @ repeat +.coNP Function @ lazy-str-force-upto .synb -.mets (repeat < list <> [ count ]) +.mets (lazy-str-force-upto < lazy-str << index ) .syne .desc -If -.meta list -is empty, then repeat returns an empty list. - -If -.meta count -is omitted, the -.code repeat -function produces an infinite lazy list -formed by catenating together copies of -.metn list . - -If -.meta count -is specified and is zero or negative, then an empty list is -returned. +The +.code lazy-str-force-upto +function tries to instantiate the lazy string such that +the position given by +.meta index +materializes. The +.meta index +is a character +position, exactly as used in the +.code chr-str +function. -Otherwise a list is returned consisting of -.meta count -repetitions of -.meta list -catenated together. +Some positions beyond +.meta index +may also materialize, as a side effect. -.coNP Function @ pad +If the string is already materialized through to at least +.metn index , +or if it is +possible to materialize the string that far, then the value +.code t +is returned to indicate success. + +If there is insufficient material to force the lazy string through to the +.meta index +position, then nil is returned. + +It is an error if the +.meta lazy-str +argument isn't a lazy string. + +.coNP Function @ lazy-str-force .synb -.mets (pad < sequence < object <> [ count ]) +.mets (lazy-str-force << lazy-str ) .syne .desc The -.code pad -function produces a lazy list which consists of all of the -elements of -.meta sequence -followed by repetitions of -.metn object . - -If -.meta count -is omitted, then the repetition of -.meta object -is infinite. Otherwise the specified number of repetitions -occur. +.meta lazy-str +argument must be a lazy string. The lazy string is forced +to fully materialize. -Note that -.meta sequence -may be a lazy list which is infinite. In that case, the repetitions -of -.meta object -will never occur. +The return value is an ordinary, non-lazy string equivalent to the fully +materialized lazy string. -.coNP Function @ weave +.coNP Function @ lazy-str-get-trailing-list .synb -.mets (weave <> { sequence }*) +.mets (lazy-str-get-trailing-list < string << index ) .syne .desc The -.code weave -function interleaves elements from the sequences given as arguments. +.code lazy-str-get-trailing-list +function can be considered, in some way, an inverse operation to +the production of the lazy string from its associated list. -If called with no arguments, it returns the empty list. +First, +.meta string +is forced up through the position +.metn index . +That is the only extent to which +.meta string +is modified by this function. -If called with a single sequence, it returns the elements of that sequence -as a new lazy list. +Next, the suffix of the materialized part of the lazy string starting at +position +.metn index , +is split into pieces on occurrences of the +terminator character (which had been given as the +.meta terminator +argument in the +.code lazy-str +constructor, and defaults to newline). If the +.meta index +position is beyond the part of the string which can be materialized +(in adherence with the lazy string's +.meta limit-count +constructor parameter), then the list of pieces is considered +to be empty. -When called with two or more sequences, -.code -weave -returns a lazy list which draws elements from the sequences in a round-robin -fashion, repeatedly scanning the sequences from left to right, and -taking an item from each one, removing it from the sequence. -Whenever a sequence runs out of items, it is deleted; the weaving then -continues with the remaining sequences. The weaved sequence terminates -when all sequences are eliminated. (If at least one of the sequences -is an infinite lazy list, then the weaved sequence is infinite.) +Finally, a list is returned consisting of the pieces produced by the split, +to which is appended the remaining list of the string which has not yet been +forced to materialize. -.TP* Examples: +.coNP Functions @, length-str-> @, length-str->= @ length-str-< and @ length-str-<= +.synb +.mets (length-str-> < string << len ) +.mets (length-str->= < string << len ) +.mets (length-str-< < string << len ) +.mets (length-str-<= < string << len ) +.syne +.desc +These functions compare the lengths of two strings. The following +equivalences hold, as far as the resulting value is concerned: .cblk - ;; Weave negative integers with positive ones: - (weave (range 1) (range -1 : -1)) -> (1 -1 2 -2 3 -3 ...) - - (weave "abcd" (range 1 3) '(x x x x x x x)) - --> (#\ea 1 x #\eb 2 x #\ec 3 x #\ed x x x x) + (length-str-> s l) <--> (> (length-str s) l) + (length-str->= s l) <--> (>= (length-str s) l) + (length-str-< s l) <--> (< (length-str s) l) + (length-str-<= s l) <--> (<= (length-str s) l) .cble -.coNP Macros @ gen and @ gun +The difference between the functions and the equivalent forms is that if the +string is lazy, the +.code length-str +function will fully force it in order to +calculate and return its length. + +These functions only force a string up to position +.metn len , +so they are not +only more efficient, but on infinitely long lazy strings they are usable. + +.code length-str +cannot compute the length of a lazy string with an unbounded +length; it will exhaust all memory trying to force the string. + +These functions can be used to test such as string whether it is longer +or shorter than a given length, without forcing the string beyond +that length. + +.coNP Function @ cmp-str .synb -.mets (gen < while-expression << produce-item-expression ) -.mets (gun << produce-item-expression ) +.mets (cmp-str < left-string << right-string ) .syne .desc The -.code gen -macro operator produces a lazy list, in a manner similar to the -.code generate -function. Whereas the -.code generate -function takes functional arguments, -the -.code gen -operator takes two expressions, which is often more convenient. +.code cmp-str +function returns a negative integer if +.meta left-string +is lexicographically prior to +.metn right-string , +and a positive integer +if the reverse situation is the case. Otherwise the strings are equal +and zero is returned. -The return value of -.code gen -is a lazy list. When the lazy list is accessed, for -instance with the functions -.code car +If either or both of the strings are lazy, then they are only forced to the +minimum extent necessary for the function to reach a conclusion and return the +appropriate value, since there is no need to look beyond the first character +position in which they differ. + +The lexicographic ordering is naive, based on the character code point +values in Unicode taken as integers, without regard for locale-specific +collation orders. + +.coNP Functions @, str= @, str< @, str> @ str>= and @ str<= +.synb +.mets (str= < left-string << right-string ) +.mets (str< < left-string << right-string ) +.mets (str> < left-string << right-string ) +.mets (str<= < left-string << right-string ) +.mets (str>= < left-string << right-string ) +.syne +.desc +These functions compare +.meta left-string and -.codn cdr , -it produces items on demand. Prior to -producing each item, the -.meta while-expression -is evaluated, in its original -lexical scope. If the expression yields a -.cod2 non- nil -value, then -.meta produce-item-expression -is evaluated, and its return value is incorporated as -the next item of the lazy list. If the expression yields -.codn nil , -then the lazy list immediately terminates. +.meta right-string +lexicographically, +as if by the +.code cmp-str +function. The -.code gen -operator itself immediately evaluates -.meta while-expression -before -producing the lazy list. If the expression yields -.codn nil , -then the operator -returns the empty list +.code str= +function returns +.code t +if the two strings are exactly the same, character +for character, otherwise it returns .codn nil . -Otherwise, it instantiates the lazy list and -invokes the -.meta produce-item-expression -to force the first item. The -.code gun -macro similarly creates a lazy list according to the following -rules. Each successive item of the lazy list is obtained as a result of -evaluating -.metn produce-item-expression . -However, when -.meta produce-item-expression -yields -.codn nil , -then the list terminates (without adding that -.code nil -as an item). +.code str< +function returns +.code t +if +.meta left-string +is lexicographically before +.metn right-string , +otherwise nil. -Note 1: the form -.code gun -can be implemented as a macro-expanding to -an instance of the -.code gen -operator, like this: +The +.code str> +function returns +.code t +if +.meta left-string +is lexicographically after +.metn right-string , +otherwise +.codn nil . -.cblk - (defmacro gun (expr) - (let ((var (gensym))) - ^(let (,var) - (gen (set ,var ,expr) - ,var)))) -.cble +The +.code str< +function returns +.code t +if +.meta left-string +is lexicographically before +.metn right-string , +or if they are exactly the same, otherwise +.codn nil . -This exploits the fact that the -.code set -operator returns the value that is -assigned, so the set expression is tested as a condition by -.codn gen , -while having the side effect of storing the next item temporarily -in a hidden variable. +The +.code str< +function returns +.code t +if +.meta left-string +is lexicographically after +.metn right-string , +or if they are exactly the same, otherwise +.codn nil . -In turn, -.code gen -can be implemented as a macro expanding to some -.code lambda -functions which are passed to the -.code generate -function: +.coNP Function @ string-lt +.synb +.mets (string-lt < left-str << right-str ) +.syne +.desc +The +.code string-lt +is a deprecated alias for +.codn str< . -.cblk - (defmacro gen (while-expr produce-expr) - ^(generate (lambda () ,while-expr) (lambda () ,produce-expr))) -.cble +.SS* Vectors +.coNP Function @ vector +.synb +.mets (vector < length <> [ initval ]) +.syne +.desc +The +.code vector +function creates and returns a vector object of the specified +length. The elements of the vector are initialized to +.metn initval , +or to nil if +.meta initval +is omitted. -Note 2: -.code gen -can be considered as an acronym for Generate, testing Expression -before Next item, whereas -.code gun -stands for Generate Until Null. +.coNP Function @ vec +.synb +.mets (vec << arg *) +.syne +.desc +The +.code vec +function creates a vector out of its arguments. -.TP* Example: +.coNP Function @ vectorp +.synb +.mets (vectorp << obj ) +.syne +.desc +The +.code vectorp +function returns t if +.meta obj +is a vector, otherwise it returns +.codn nil . -.cblk - @(do - ;; Make a lazy list of integers up to 1000 - ;; access and print the first three. - (let* ((counter 0) - (list (gen (< counter 1000) (inc counter)))) - (format t "~s ~s ~s\en" (pop list) (pop list) (pop list)))) +.coNP Function @ vec-set-length +.synb +.mets (vec-set-length < vec << len ) +.syne +.desc +The +.code vec-set-length +modifies the length of +.metn vec , +making it longer or +shorter. If the vector is made longer, then the newly added elements +are initialized to nil. The +.meta len +argument must be nonnegative. - Output: - 1 2 3 -.cble +The return value is +.metn vec . -.coNP Functions @ range and @ range* +.coNP Accessor @ vecref .synb -.mets (range >> [ from >> [ to <> [ step ]]]) -.mets (range* >> [ from >> [ to <> [ step ]]]) +.mets (vecref < vec << idx ) +.mets (set (vecref < vec << idx ) << new-value ) .syne .desc The -.code range -and -.code range* -functions generate a lazy sequence of integers, with a -fixed step between successive values. +.code vecref +function performs indexing into a vector. It retrieves +an element of +.meta vec +at position +.metn idx , +counted from zero. +The +.meta idx +value must range from 0 to one less than the +length of the vector. The specified element is returned. -The difference between -.code range -and -.code range* -is that -.code range* -excludes the endpoint. -For instance -.code (range 0 3) -generates the list -.codn (0 1 2 3) , -whereas -.code (range* 0 3) -generates -.codn (0 1 2) . +If the element +.meta idx +of vector +.meta vec +exists, then the +.code vecref +form denotes a place. -All arguments are optional. If the -.meta step -argument is omitted, then it defaults -to -.codn 1 : -each value in the sequence is greater than the previous one by -.codn 1 . -Positive or negative step sizes are allowed. There is no check for a step size -of zero, or for a step direction which cannot meet the endpoint. +A +.code vecref place +supports deletion. When a deletion takes place, +then if +.meta idx +denotes the last element in the vector, the +vector's length is decreased by one, so that +the vector no longer has that element. +Otherwise, if +.meta idx +isn't the last element, then each elements +values at a higher index than +.meta idx +shifts by one one element position to the +adjacent lower index. Then, the length of the +vector is decreased by one, so that the last +element position disappears. +.coNP Function @ vec-push +.synb +.mets (vec-push < vec << elem ) +.syne +.desc The -.meta to -argument specifies the endpoint value, which, if it occurs in the -sequence, is excluded from it by the -.code range* -function, but included by the range -function. If -.meta to -is missing, or specified as -.codn nil , -then there is no endpoint, -and the sequence which is generated is infinite, regardless of -.metn step . +.code vec-push +function extends the length of a vector +.meta vec +by one element, and +sets the new element to the value +.metn elem . -If -.meta from -is omitted, then the sequence begins at zero, otherwise -.meta from -must be an integer which specifies the initial value. +The previous length of the vector (which is also the position of +.metn elem ) +is returned. -The sequence stops if it reaches the endpoint value (which is included in the -case of -.codn range , -and excluded in the case of -.codn range *). -However, a sequence with a stepsize greater than -.code 1 -or less than -.code -1 -might step over the endpoint value, and -therefore never attain it. In this situation, the sequence also stops, and the -excess value which surpasses the endpoint is excluded from the sequence. +This function performs similarly to the generic function +.codn ref , +except that the +first argument must be a vector. -.coNP Function @ perm +.coNP Function @ length-vec .synb -.mets (perm < seq <> [ len ]) +.mets (length-vec << vec ) .syne .desc The -.code rperm -function returns a lazy list which consists of all -length -.meta len -permutations of formed by items taken from -.metn seq . -The permutations do not use any element of -.meta seq -more than once. - -Argument -.metn len , -if present, must be a positive integer, and -.meta seq -must be a sequence. - -If -.meta len -is not present, then its value defaults to the length of -.metn seq : -the list of the full permutations of the entire sequence is returned. - -The permutations in the returned list are sequences of the same kind as -.codn seq . - -If -.meta len -is zero, then a list containing one permutation is returned, and that -permutations is of zero length. - -If -.meta len -exceeds the length of -.metn seq , -then an empty list is returned, -since it is impossible to make a single non-repeating permutation that -requires more items than are available. - -The permutations are lexicographically ordered. +.code length-vec +function returns the length of vector +.metn vec . +It performs +similarly to the generic +.code length +function, except that the argument must +be a vector. -.coNP Function @ rperm +.coNP Function @ size-vec .synb -.mets (rperm < seq << len ) +.mets (size-vec << vec ) .syne .desc The -.code rperm -function returns a lazy list which consists of all the repeating -permutations of length -.meta len -formed by items taken from -.metn seq . -"Repeating" means that the items from -.meta seq -can appear more than -once in the permutations. - -The permutations which are returned are sequences of the same kind as -.metn seq . - -Argument -.meta len -must be a nonnegative integer, and -.meta seq -must be a sequence. - -If -.meta len -is zero, then a single permutation is returned, of zero length. -This is true regardless of whether -.meta seq -is itself empty. - -If -.meta seq -is empty and -.meta len -is greater than zero, then no permutations are -returned, since permutations of a positive length require items, and the -sequence has no items. Thus there exist no such permutations. - -The first permutation consists of -.meta le -repetitions of the first element of -.metn seq . -The next repetition, if there is one, differs from the first -repetition in that its last element is the second element of -.metn seq . -That is to say, the permutations are lexicographically ordered. - -.TP* Examples: - -.cblk - (rperm "01" 4) -> ("000" "001" "010" "011" - "100" "101" "110" "111") +.code size-vec +function returns the number of elements for which storage +is reserved in the vector +.metn vec . - (rperm #(1) 3) -> (#(1 1 1)) +.TP* Notes: - (rperm '(0 1 2) 2) -> ((0 0) (0 1) (0 2) (1 0) - (1 1) (1 2) (2 0) (2 1) (2 2)) -.cble +The +.code length +of the vector can be extended up to this size without any memory +allocation operations having to be performed. -.coNP Function @ comb +.coNP Function @ vector-list .synb -.mets (comb < seq << len ) +.mets (vector-list << list ) .syne .desc -The -.code comb -function returns a lazy list which consists of all -length -.meta len -non-repeating combinations formed by taking items taken from -.metn seq . -"Non-repeating combinations" means that the combinations do not use any -element of -.meta seq -more than once. If -.meta seq -contains no duplicates, then -the combinations contain no duplicates. - -Argument -.meta len -must be a nonnegative integer, and -.meta seq -must be a sequence or a hash table. - -The combinations in the returned list are objects of the same kind as -.metn seq . - -If -.meta len -is zero, then a list containing one combination is returned, and that -permutations is of zero length. - -If -.meta len -exceeds the number of elements in -.metn seq , -then an empty list is returned, since it is impossible to make a single -non-repeating combination that requires more items than are available. - -If -.meta seq -is a sequence, the returned combinations are lexicographically ordered. -This requirement is not applicable when -.meta seq -is a hash table. - -.TP* Example: -.cblk - ;; powerset function, in terms of comb. - ;; Yields a lazy list of all subsets of s, - ;; expressed as sequences of the same type as s. - - (defun powerset (s) - (mappend* (op comb s) (range 0 (length s)))) -.cble +This function returns a vector which contains all of the same elements +and in the same order as list +.metn list . -.coNP Function @ rcomb +.coNP Function @ list-vector .synb -.mets (rcomb < seq << len ) +.mets (list-vector << vec ) .syne .desc The -.code comb -function returns a lazy list which consists of all -length -.meta len -repeating combinations formed by taking items taken from -.metn seq . -"Repeating combinations" means that the combinations can use -an element of -.meta seq -more than once. - -Argument -.meta len -must be a nonnegative integer, and -.meta seq -must be a sequence. - -The combinations in the returned list are sequences of the same kind as -.metn seq . - -If -.meta len -is zero, then a list containing one combination is returned, and that -permutations is of zero length. This is true even if -.meta seq -is empty. - -If -.meta seq -is empty, and -.meta len -is nonzero, then an empty list is returned. - -The combinations are lexicographically ordered. +.code list-vector +function returns a list of the elements of vector +.metn vec . -.SS* Characters and Strings -.coNP Function @ mkstring +.coNP Function @ copy-vec .synb -.mets (mkstring < length << char ) +.mets (copy-vec << vec ) .syne .desc The -.code mkstring -function constructs a string object of a length specified -by the -.meta length -parameter. Every position in the string is initialized -with -.metn char , -which must be a character value. +.code copy-vec +function returns a new vector object of the same length +as +.meta vec +and containing the same elements in the same order. -.coNP Function @ copy-str +.coNP Function @ sub-vec .synb -.mets (copy-str << string ) +.mets (sub-vec < vec >> [ from <> [ to ]]) .syne .desc The -.code copy-str -function constructs a new string whose contents are identical -to -.metn string . +.code sub-vec +function is like the more generic function +.codn sub , +except that it +operates only on vectors. -.coNP Function @ upcase-str +For a description of the arguments and semantics, refer to the +.code sub +function. + +.coNP Function @ replace-vec .synb -.mets (upcase-str << string ) +.mets (replace-vec < vec < item-sequence >> [ from <> [ to ]]) .syne .desc The -.code upcase-str -function produces a copy of -.meta string -such that all lower-case -characters of the English alphabet are mapped to their upper case counterparts. +.code replace-vec +is like the +.code replace +function, except that the first argument +must be a vector. -.coNP Function @ downcase-str +For a description of the arguments, semantics and return value, refer to the +.code replace +function. + +.coNP Function @ cat-vec .synb -.mets (downcase-str << string ) +.mets (cat-vec << vec-list ) .syne .desc The -.code downcase-str -function produces a copy of -.meta string -such that -all upper case characters of the English alphabet are mapped to their -lower case counterparts. +.meta vec-list +argument is a list of vectors. The +.code cat-vec +function +produces a catenation of the vectors listed in +.metn vec-list . +It returns +a single large vector formed by catenating those vectors together in +order. -.coNP Function @ string-extend +.SS* Sequence Manipulation +.coNP Function @ seqp .synb -.mets (string-extend < string << tail ) +.mets (seqp << object ) .syne .desc -The -.code string-extend -function destructively increases the length of -.metn string , -which must be an ordinary dynamic string. It is an error to invoke this -function on a literal string or a lazy string. - -The -.meta tail -argument can be a character, string or integer. If it is a string or -character, it specifies material which is to be added to the end of the string: -either a single character or a sequence of characters. If it is an integer, it -specifies the number of characters to be added to the string. +The function +.code seqp +returns +.code t +if +.meta object +is a sequence, otherwise +.codn nil . -If -.meta tail -is an integer, the newly added characters have indeterminate contents. -The string appears to be the original one because of an internal terminating -null character remains in place, but the characters beyond the terminating zero -are indeterminate. +A sequence is defined as a list, vector or string. The object +.code nil +denotes +the empty list and so is a sequence. -.coNP Function @ stringp +.coNP Function @ length .synb -.mets (stringp << obj ) +.mets (length << sequence ) .syne .desc The -.code stringp -function returns t if -.meta obj -is one of the several -kinds of strings. Otherwise it returns -.codn nil . +.code length +function returns the number of items in +.metn sequence , +and +returns it. +.meta sequence +may be a hash, in which case +.cblk +.meti (hash-count << sequence ) +.cble +is returned. -.coNP Function @ length-str +.coNP Function @ empty .synb -.mets (length-str << string ) +.mets (empty << sequence ) .syne .desc -The -.code length-str -function returns the length -.meta string -in characters. The argument must be a string. +Returns +.code t +if +.cblk +.meti (length << sequence ) +.cble +is zero, otherwise +.codn nil . -.coNP Function @ search-str +.coNP Function @ copy .synb -.mets (search-str < haystack < needle >> [ start <> [ from-end ]]) +.mets (copy << object ) .syne .desc The -.code search-str -function finds an occurrence of the string -.meta needle -inside -the -.meta haystack -string and returns its position. If no such occurrence exists, -it returns +.code copy +function duplicates objects of various supported types: sequences, hashes and random states. If +.meta object +is +.codn nil , +it +returns .codn nil . +If +.meta object +is a list, it returns +.cblk +.meti (copy-list << object ). +.cble +If +.meta object +is a string, it returns +.cblk +.meti (copy-str << object ). +.cble +If +.meta object +is a vector, it returns +.cblk +.meti (copy-vec << object ). +.cble +If +.meta object +is a hash, it returns +.cblk +.meti (copy-hash << object ) +.cble +Lastly, if +.meta object +is a random state, it returns +.cblk +.meti (make-random-state << object ). +.cble -If a -.meta start -argument is not specified, it defaults to zero. If it is -a non-negative integer, it specifies the starting character position for -the search. Negative values of -.meta start -indicate positions from the end of the -string, such that -.code -1 -is the last character of the string. - -If the -.meta from-end -argument is specified and is not +Except in the case when +.meta sequence +is .codn nil , -it means -that the search is conducted right-to-left. If multiple matches are possible, -it will find the rightmost one rather than the leftmost one. +.code copy +returns a value that +is distinct from (not +.code eq +to) +.metn sequence . +This is different from +the behavior of +.cblk +.meti >> [ sequence 0..t] +.cblk +or +.cblk +.meti (sub < sequence 0 t) +.cble +which recognize +that they need not make a copy of +.metn sequence , +and just return it. -.coNP Function @ search-str-tree +Note however, that the elements of the returned sequence may be +eq to elements of the original sequence. In other words, copy is +a deeper copy than just duplicating the +.code sequence +value itself, +but it is not a deep copy. + +.coNP Function @ sub .synb -.mets (search-str-tree < haystack < tree >> [ start <> [ from-end ]]) +.mets (sub < sequence >> [ from <> [ to ]]) .syne .desc The -.code search-str-tree -function is similar to -.codn search-str , -except that instead of -searching -.meta haystack -for the occurrence of a single needle string, it searches -for the occurrence of numerous strings at the same time. These search strings -are specified, via the -.meta tree -argument, as an arbitrarily structured tree whose -leaves are strings. +.code sub +function extracts a slice from input sequence +.metn sequence . +The slice is +a sequence of the same type as +.metn sequence . -The function finds the earliest possible match, in the given search direction, -from among all of the needle strings. +If the +.meta from +argument is omitted, it defaults to +.codn 0 . +If the +.meta to +parameter is +omitted, it defaults to +.codn t . +Thus +.code (sub a) +means +.codn (sub a 0 t) . -If -.meta tree -is a single string, the semantics is equivalent to -.codn search-str . +The following equivalence holds between the +.code sub +function and +the DWIM-bracket syntax: -.coNP Function @ match-str -.synb -.mets (match-str < bigstring < littlestring <> [ start ]) -.syne -.desc -Without the -.meta start -argument, the -.code match-str -function determines whether -.meta littlestring -is a prefix of -.metn bigstring , -returning a -.code t -or -.code nil -indication. - -If the -.meta start -argument is specified, and is a non-negative integer, then the -function tests whether -.meta littlestring -matches a prefix of that portion of -.meta bigstring -which starts at the given position. +.cblk + ;; from is not a list + (sub seq from to) <--> [seq from..to] +.cble -If the -.meta start -argument is a negative integer, then -.code match-str -determines -whether -.meta littlestring -is a suffix of -.metn bigstring , -ending on that position -of bigstring, where -.code -1 -denotes the last character of -.metn bigstring , -.code -2 -the second last one and so on. +The description of the +.code dwim +operator\(emin particular, the section +on Range Indexing\(emexplains the semantics of the range specification. -If -.meta start -is -.codn -1 , -then this corresponds to testing whether -.meta littlestring -is a suffix of -.metn bigstring . +If the sequence is a list, the output sequence may share substructure +with the input sequence. -.coNP Function @ match-str-tree +.coNP Function @ replace .synb -.mets (match-str-tree < bigstring < tree <> [ start ]) +.mets (replace < sequence < replacement-sequence >> [ from <> [ to ]]) +.mets (replace < sequence < replacement-sequence << index-list ) .syne .desc The -.code match-str-tree -function is a generalization of match-str which matches -multiple test strings against -.meta bigstring -at the same time. The value -reported is the longest match from among any of the strings. +.meta replace +function modifies +.meta sequence +in the ways described below. -The strings are specified as an arbitrarily shaped tree structure which has -strings at the leaves. +The operation is destructive: it may work "in place" by modifying +the original sequence. The caller should retain the return value +and stop relying on the original input sequence. -If -.meta tree -is a single string atom, then the function behaves -exactly like match-str. +The return value of +.code replace +is the modified +version of +.metn sequence . +This may be the same object as +.meta sequence +or it may be a newly allocated object. -.coNP Function @ sub-str -.synb -.mets (sub-str < string >> [ from <> [ to ]]) -.syne -.desc -The -.code sub-str -function is like the more generic function -.codn sub , -except that it -operates only on strings. For a description of the arguments and semantics, -refer to the -.code sub -function. +Note that the form: + +.cblk + (set seq (replace seq new fr to)) +.cble + +has the same effect on the variable +.code seq +as the form: + +.cblk + (set [seq fr..to] new) +.cble + +except that the former +.code set +form returns the entire modified sequence, whereas the latter +returns the value of the +.code new +argument. -.coNP Function @ replace-str -.synb -.mets (replace-str < string < item-sequence >> [ from <> [ to ]]) -.syne -.desc The -.code replace-str -function is like the .code replace -function, except that the first -argument must be a string. +function has two invocation styles, distinguished by the +type of the third argument. If the third argument is a list or vector, then it +is deemed to be the +.meta index-list +parameter of the second form. +Otherwise, if the third argument is missing, or is not a list, then +it is deemed to be the +.meta from +argument of the first form. -For a description of the arguments, semantics and return value, refer to the -.code replace -function. +The first form of the replace function replaces a contiguous subsequence of the +.meta sequence +with +.metn replacement-sequence . +The replaced subsequence may be empty, +in which case an insertion is performed. If +.meta replacement-sequence +is empty +(for example, the empty list +.codn nil ), +then a deletion is performed. -.coNP Function @ cat-str -.synb -.mets (cat-str < string-list <> [ sep-string ]) -.syne -.desc -The -.code cat-str -function catenates a list of strings given by -.meta string-list -into a -single string. The optional -.meta sep-string -argument specifies a separator string -which is interposed between the catenated strings. +If the +.meta from +and +.meta to +arguments are omitted, their values default +to +.code 0 +and +.code t +respectively. -.coNP Function @ split-str +The description of the dwim operator\(emin particular, the section +on Range Indexing\(emexplains the semantics of the range specification. + +The second form of the replace function replaces a subsequence of +elements from +.meta sequence +given by +.metn index-list , +with their counterparts +from +.metn replacement-sequence . +This form of the replace function does not insert +or delete; it simply overwrites elements. If +.meta replacement-sequence +and +.meta index-list +are of different lengths, then the shorter of the two determines +the maximum number of elements which are overwritten. +Furthermore, similar restrictions apply on +.meta index-list +as under the +select function. Namely, the replacement stops when an index value +in +.meta index-list +is encountered which is out of range for +.metn sequence . +furthermore, if +.meta sequence +is a list, then +.meta index-list +must +be monotonically increasing. + +.coNP Function @ search .synb -.mets (split-str < string << sep ) +.mets (search < haystack < needle >> [ testfun <> [ keyfun ]) .syne .desc The -.code split-str -function breaks the -.meta string -into pieces, returning a list -thereof. The -.meta sep -argument must be either a string or a regular expression. -It specifies the separator character sequence within -.metn string . - -All non-overlapping matches for -.meta sep +.code search +function determines whether the sequence +.meta needle +occurs as substring within -.meta string -are identified in left -to right order, and are removed from -.metn string . -The string is broken into pieces -according to the gaps left behind by the removed separators, and a list -of the remaining pieces is returned. - +.metn haystack , +under the given comparison function +.meta testfun +and +key function +.metn keyfun . +If this is the case, then the zero-based position of +the leftmost occurrence of +.meta key +within +.meta haystack +is returned. Otherwise +.code nil +is returned to indicate that +.meta key +does not occur within +.metn haystack . If -.meta sep -is the empty string, then the separator pieces removed from the -string are considered to be the empty strings between its -characters. In this case, if -.meta string -is of length one or zero, then it is considered to have no such pieces, and a -list of one element is returned containing the original string. +.meta key +is empty, then zero is always returned. -If a match for -.meta sep -is not found in the string at all, then the string is not -split at all: a list of one element is returned containing the original -string. +The arguments +.meta haystack +and +.meta needle +are sequences: lists, vectors +or strings, in any combination. If -.meta sep -matches the entire string, then a list of two empty strings is -returned, except in the case that the original string is empty, in which case a -list of one element is returned, containing the empty string. +.meta needle +is not empty, then occurs at some position N within +.meta haystack +if +the first element of +.meta needle +matches the element at position N of +.metn haystack , +the second element of +.meta needle +matches the element at position N+1 of +.meta haystack +and so forth, for all elements of +.metn needle . +A match between elements +is determined by passing each element through +.metn keyfun , +and then comparing the resulting values using +.metn testfun . -Whenever two adjacent matches for -.meta sep -occur, they are considered separate -cuts with an empty piece between them. +If +.meta testfun +is supplied, it must be a function which can be +called with two arguments. If it is not supplied, it defaults to +.codn eql . -This operation is nondestructive: -.meta string -is not modified in any way. +If +.meta keyfun +is supplied, it must be a function which can be called +with one argument. If it is not supplied, it defaults to +.codn identity . + +.TP* Examples: -Note: To split a string into pieces of length one such that an empty string -produces -.code nil -rather than -.codn ("") , -use the .cblk -.meti (tok-str < string #/./) + ;; fails because 3.0 doesn't match 3 + ;; under the default eql function + [search #(1.0 3.0 4.0 7.0) '(3 4)] -> nil + + ;; occurrence found at position 1: + ;; (3.0 4.0) matches (3 4) under = + [search #(1.0 3.0 4.0 7.0) '(3 4) =] -> 1 + + ;; "even odd odd odd even" pattern + ;; matches at position 2 + [search #(1 1 2 3 5 7 8) '(2 1 1 1 2) : evenp] -> 2 + + ;; Case insensitive string search + [search "abcd" "CD" : chr-toupper] -> 2 + + ;; Case insensitive string search + ;; using vector of characters as key + [search "abcd" #(#\eC #\eD) : chr-toupper] -> 2 .cble -pattern. -.coNP Function @ split-str-set +.coNP Functions @ ref and @ refset .synb -.mets (split-str-set < string << set ) +.mets (ref < seq << index ) +.mets (refset < seq < index << new-value ) .syne .desc The -.code split-str-set -function breaks the -.meta string -into pieces, returning a list -thereof. The -.meta set -argument must be a string. It specifies a set of -characters. All occurrences of any of these characters within -.meta string -are -identified, and are removed from -.metn string . -The string is broken into pieces -according to the gaps left behind by the removed separators. - -Adjacent occurrences of characters from -.meta set -within -.meta string -are considered to -be separate gaps which come between empty strings. +.code ref +and +.code refset +functions perform array-like indexing into sequences. +The +.code ref +function retrieves an element of +.metn seq , +whereas +.code refset +overwrites an +element of +.meta seq +with a new value. -This operation is nondestructive: -.meta string -is not modified in any way. +The +.meta index +argument is based from zero, and negative values are permitted, +with a special meaning as described in the Range Indexing section under the +description of the +.code dwim +operator. -.coNP Functions @ tok-str and @ tok-where -.synb -.mets (tok-str < string < regex <> [ keep-between ]) -.mets (tok-where < string << regex ) -.syne -.desc The -.code tok-str -function searches -.meta string -for tokens, which are defined as -substrings of -.mta string -which match the regular expression -.meta regex -in the -longest possible way, and do not overlap. These tokens are extracted from the -string and returned as a list. +.code refset +function returns the new value. -Whenever -.meta regex -matches an empty string, then an empty token is returned, and -the search for another token within -.meta string -resumes after advancing by one -character position. So for instance, -.code (tok-str "abc" #/a?/) -returns the -.cblk -("a" "" "" ""). -.cble -After the token -.str "a" -is extracted from a non-empty match -for the regex, the regex is considered to match three more times: before the -.strn "b" , -between -.str "b" +The following equivalences hold between +.code ref and -.strn "c" , -and after the -.strn "c" . +.codn refset , +and the DWIM bracket syntax: -If the -.meta keep-between -argument is specified, and is not -.codn nil , -then the behavior -of -.code tok-str -changes in the following way. The pieces of -.meta string -which are -skipped by the search for tokens are included in the output. If no token is -found in -.metn string , -then a list of one element is returned, containing -.metn string . -Generally, if N tokens are found, then the returned list consists of 2N + 1 -elements. The first element of the list is the (possibly empty) substring which -had to be skipped to find the first token. Then the token follows. The next -element is the next skipped substring and so on. The last element is the -substring of -.meta string -between the last token and the end. +.cblk + (ref seq idx) <--> [seq idx] -The -.code tok-where -function works similarly to -.codn tok-str , -but instead of returning -the extracted tokens themselves, it returns a list of the character position -ranges within -.meta string -where matches for -.meta regex -occur. The ranges -are pairs of numbers, represented as cons cells, where the first number -of the pair gives the starting character position, and the second number -is one position past the end of the match. If a match is empty, then the -two numbers are equal. + (refset seq idx new) <--> (set [seq idx] new) +.cble -The tok-where function does not support the -.meta keep-between -parameter. +The difference is that +.code ref +and +.code refset +are first class functions which +can be used in functional programming as higher order functions, whereas the +bracket notation is syntactic sugar, and +.code set +is an operator, not a function. +Therefore the brackets cannot replace all uses of +.code ref +and +.codn refset . -.coNP Function @ list-str +.coNP Function @ update .synb -.mets (list-str << string ) +.mets (update < sequence-or-hash << function ) .syne .desc The -.code list-str -function converts a string into a list of characters. +.code update +function replaces each elements in a sequence, or each value +in a hash table, with the value of +.meta function +applied to that element +or value. -.coNP Function @ trim-str +The sequence or hash table is returned. + +.coNP Functions @, remq @ remql and @ remqual .synb -.mets (trim-str << string ) +.mets (remq < object << list ) +.mets (remql < object << list ) +.mets (remqual < object << list ) .syne .desc The -.code trim-str -function produces a copy of -.meta string -from which leading and -trailing whitespace is removed. Whitespace consists of spaces, tabs, -carriage returns, linefeeds, vertical tabs and form feeds. +.codn remq , +.code remql +and +.code remqual +functions produce a new list based on +.metn list , +removing the items which are +.codn eq , +.code eql +or +.code equal +to +.metn object . -.coNP Function @ chrp -.synb -.mets (chrp << obj ) -.syne -.desc -Returns -.code t -if -.meta obj -is a character, otherwise nil. +The input +.meta list +is unmodified, but the returned list may share substructure +with it. If no items are removed, it is possible that the return value +is +.meta list +itself. -.coNP Function @ chr-isalnum +.coNP Functions @, remq @ remql* and @ remqual* .synb -.mets (chr-isalnum << char ) +.mets (remq* < object << list ) +.mets (remql* < object << list ) +.mets (remqual* < object << list ) .syne .desc -Returns -.code t -if -.meta char -is an alpha-numeric character, otherwise nil. Alpha-numeric -means one of the upper or lower case letters of the English alphabet found in -ASCII, or an ASCII digit. This function is not affected by locale. +The +.codn remq* , +.code remql* +and +.code remqual* +functions are lazy versions of +.codn remq , +.code remql +and +.codn remqual . +Rather than computing the entire new list +prior to returning, these functions return a lazy list. -.coNP Function @ chr-isalpha +Caution: these functions can still get into infinite looping behavior. +For instance, in +.codn (remql* 0 (repeat '(0))) , +.code remql +will keep consuming +the +.code 0 +values coming out of the infinite list, looking for the first item that +does not have to be deleted, in order to instantiate the first lazy value. + +.TP* Examples: +.cblk + ;; Return a list of all the natural numbers, excluding 13, + ;; then take the first 100 of these. + ;; If remql is used, it will loop until memory is exhausted, + ;; because (range 1) is an infinite list. + + [(remql* 13 (range 1)) 0..100] +.cble + +.coNP Functions @, remove-if @, keep-if @ remove-if* and @ keep-if* .synb -.mets (chr-isalpha << char ) +.mets (remove-if < predicate-function < list <> [ key-function ]) +.mets (keep-if < predicate-function < list <> [ key-function ]) +.mets (remove-if* < predicate-function < list <> [ key-function ]) +.mets (keep-if* < predicate-function < list <> [ key-function ]) .syne .desc -Returns -.code t -if -.meta char -is an alphabetic character, otherwise -.codn nil . -Alphabetic -means one of the upper or lower case letters of the English alphabet found in -ASCII. This function is not affected by locale. -.coNP Function @ chr-isascii +The +.code remove-if +function produces a list whose contents are those of +.meta list +but with those elements removed which satisfy +.metn predicate-function . +Those elements which are not removed appear in the same order. +The result list may share substructure with the input list, +and may even be the same list object if no items are removed. + +The optional +.meta key-function +specifies how each element from the +.meta list +is transformed to an argument to +.metn predicate-function . +If this argument is omitted +then the predicate function is applied to the elements directly, a behavior +which is identical to +.meta key-function +being +.codn (fun identity) . + +The +.code keep-if +function is exactly like +.codn remove-if , +except the sense of +the predicate is inverted. The function +.code keep-if +retains those items +which +.code remove-if +will delete, and removes those that +.code remove-if +will preserve. + +The +.code remove-if* +and +.code keep-if* +functions are like +.code remove-if +and +.codn keep-if , +but produce lazy lists. + +.TP* Examples: +.cblk + ;; remove any element numerically equal to 3. + (remove-if (op = 3) '(1 2 3 4 3.0 5)) -> (1 2 4 5) + + ;; remove those pairs whose first element begins with "abc" + [remove-if (op equal [@1 0..3] "abc") + '(("abcd" 4) ("defg" 5)) + car] + -> (("defg" 5)) + + ;; equivalent, without test function + (remove-if (op equal [(car @1) 0..3] "abc") + '(("abcd" 4) ("defg" 5))) + -> (("defg" 5)) +.cble + +.coNP Functions @, countqual @ countql and @ countq .synb -.mets (chr-isalpha << char ) +.mets (countq < object << list ) +.mets (countql < object << list ) +.mets (countqual < object << list ) .syne .desc -This function returns -.code t -if the code of character -.meta char -is in the range 0 to 127 inclusive. For characters outside of this range, it -returns -.codn nil . +The +.codn countq , +.code countql +and +.code countqual +functions count the number of objects +in +.meta list +which are +.codn eq , +.code eql +or +.code equal +to +.metn object , +and return the count. -.coNP Function @ chr-iscntrl +.coNP Function @ count-if .synb -.mets (chr-iscntrl << char ) +.mets (count-if < predicate-function < list <> [ key-function ]) .syne .desc -This function returns -.code t -if the character -.meta char -is a character whose code -ranges from 0 to 31, or is 127. In other words, any non-printable ASCII -character. For other characters, it returns -.codn nil . +The +.code count-if +function counts the number of elements of +.meta list +which satisfy +.meta predicate-function +and returns the count. -.coNP Function @ chr-isdigit +The optional +.meta key-function +specifies how each element from the +.meta list +is transformed to an argument to +.metn predicate-function . +If this argument is omitted +then the predicate function is applied to the elements directly, a behavior +which is identical to +.meta key-function +being +.codn (fun identity) . + +.coNP Functions @, posqual @ posql and @ posq .synb -.mets (chr-isdigit << char ) +.mets (posq < object << list ) +.mets (posql < object << list ) +.mets (posqual < object << list ) .syne .desc -This function returns -.code t -if the character -.meta char -is is an ASCII digit. -Otherwise, it returns -.codn nil . +The +.codn posq , +.code posql +and +.code posqual +functions return the zero-based position of the +first item in +.meta list +which is, respectively, +.codn eq , +.code eql +or +.code equal +to +.metn object . -.coNP Function @ chr-isgraph +.coNP Functions @ pos and @ pos-if .synb -.mets (chr-isgraph << char ) +.mets (pos < key < list >> [ testfun <> [ keyfun ]]) +.mets (pos-if < predfun < list <> [ keyfun ]) .syne .desc -This function returns -.code t -if -.meta char -is a non-space printable ASCII character. -It returns nil if it is a space or control character. -It also returns nil for non-ASCII characters: Unicode characters with a code -above 127. +The +.code pos +and +.code pos-if +functions search through +.meta list +for an item which matches +.metn key , +or satisfies predicate function +.metn predfun , +respectively. +They return the zero-based position of the matching item. -.coNP Function @ chr-islower -.synb -.mets (chr-islower << char ) -.syne -.desc -This function returns -.code t -if -.meta char -is an ASCII lower case letter. Otherwise it returns -.codn nil . +The +.meta keyfun +argument specifies a function which is applied to the elements +of +.meta list +to produce the comparison key. If this argument is omitted, +then the untransformed elements of +.meta list +are examined. -.coNP Function @ chr-isprint -.synb -.mets (chr-isprint << char ) -.syne -.desc -This function returns -.code t -if -.meta char -is an ASCII character which is not a -control character. It also returns +The +.code pos +function's +.meta testfun +argument specifies the test function which +is used to compare the comparison keys from +.meta list +to +.metn key . +If this argument is omitted, then the +.code equal +function is used. +The position of the first element +.meta list +whose comparison key (as +retrieved by +.metn keyfun ) +matches the search (under +.metn testfun ) +is +returned. If no such element is found, .code nil -for all non-ASCII characters: Unicode -characters with a code above 127. +is returned. -.coNP Function @ chr-ispunct -.synb -.mets (chr-ispunct << char ) -.syne -.desc -This function returns -.code t -if -.meta char -is an ASCII character which is not a -control character. It also returns nil for all non-ASCII characters: Unicode -characters with a code above 127. +The +.code pos-if +function's +.meta predfun +argument specifies a predicate function +which is applied to the successive comparison keys taken from +.meta list +by applying +.meta keyfun +to successive elements. The position of +the first element for which +.meta predfun +yields true is returned. If +no such element is found, +.code nil +is returned. -.coNP Function @ chr-isspace +.coNP Functions @ pos-max and @ pos-min .synb -.mets (chr-isspace << char ) +.mets (pos-max < sequence >> [ testfun <> [ keyfun ]]) +.mets (pos-min < sequence >> [ testfun <> [ keyfun ]]) .syne .desc -This function returns -.code t -if -.meta char -is an ASCII whitespace character: any of the -characters in the set -.codn #\espace , -.codn #\etab , -.codn #\elinefeed , -.codn #\enewline , -.codn #\ereturn , -.code #\evtab + +The +.code pos-min and -.codn #\epage . -For all other characters, it returns -.codn nil . +.code pos-max +functions implement exactly the same algorithm; they +differ only in their defaulting behavior with regard to the +.meta testfun +argument. If +.meta testfun +is not given, then the pos-max function defaults +.meta testfun +to the +.code greater +function, whereas +.code pos-min +defaults it to the +.code less +function. -.coNP Function @ chr-isblank -.synb -.mets (chr-isblank << char ) -.syne -.desc -This function returns -.code t -if -.meta char -is a space or tab: the character -.code #\espace -or -.codn #\etab . -For all other characters, it returns +If +.meta sequence +is empty, both functions return .codn nil . -.coNP Function @ chr-isunisp -.synb -.mets (chr-isunisp << char ) -.syne -.desc -This function returns -.code t -if -.meta char -is a Unicode whitespace character. This the case for -all the characters for which -.code chr-isspace -returns -.codn t. -It also returns -.code t -for these additional characters: -.codn #\exa0 , -.codn #\ex1680 , -.codn #\ex180e , -.codn #\ex2000 , -.codn #\ex2001 , -.codn #\ex2002 , -.codn #\ex2003 , -.codn #\ex2004 , -.codn #\ex2005 , -.codn #\ex2006 , -.codn #\ex2007 , -.codn #\ex2008 , -.codn #\ex2009 , -.codn #\ex200a , -.codn #\ex2028 , -.codn #\ex2029 , -.codn #\ex205f , +Without a +.meta testfun +argument, the +.code pos-max +function finds the zero-based +position index of the numerically maximum value occurring in +.metn sequence , +whereas +.code pos-min +without a +.meta testfun +argument finds the index of the minimum +value. + +If a +.meta testfun +argument is given, the two functions are equivalent. +The +.meta testfun +function must be callable with two arguments. +If +.meta testfun +behaves like a greater-than comparison, then +.code pos-max and -.codn #\ex3000 . -For all other characters, it returns -.codn nil . +.code pos-min +return the index of the maximum element. If +.meta testfun +behaves like a +.code less-than +comparison, then the functions return +the index of the minimum element. -.coNP Function @ chr-isupper -.synb -.mets (chr-isupper < char ) -.syne -.desc -This function returns -.code t -if -.meta char -is an ASCII upper case letter. Otherwise it returns -.codn nil . +The +.meta keyfun +argument defaults to the +.code identity +function. Each element +from +.meta sequence +is passed through this one-argument function, and +the resulting value is used in its place. -.coNP Function @ chr-isxdigit +.coNP Function @ where .synb -.mets (chr-isxdigit << char ) +.mets (where < function << object ) .syne .desc -This function returns -.code t -if -.meta char -is a hexadecimal digit. One of the ASCII -letters -.code A -through -.codn F , -or their lower-case equivalents, or an ASCII digit -.code 0 -through -.codn 9 . - -.coNP Function @ chr-toupper -.synb -.mets (chr-toupper << char ) -.syne -.desc -If character -.meta char -is a lower case ASCII letter character, this function -returns the upper case equivalent character. If it is some other -character, then it just returns -.metn char . -.coNP Function @ chr-tolower -.synb -.mets (chr-tolower << char ) -.syne -.desc -If character -.meta char -is an upper case ASCII letter character, this function -returns the lower case equivalent character. If it is some other -character, then it just returns -.metn char . +If +.meta object +is a sequence, the +.code where +function returns +a list of the numeric indices of those of its elements which satisfy +.metn function . +The numeric indices appear in increasing order. -.coNP Functions @ num-chr and @ chr-num -.synb -.mets (num-chr << char ) -.mets (chr-num << num ) -.syne -.desc -The argument -.meta char -must be a character. The -.code num-chr -function returns that -character's Unicode code point value as an integer. +If +.meta object +is a hash, the +.code where +function returns an unordered list +of keys which have values which satisfy +.metn function . -The argument -.meta num -must be a fixnum integer in the range -.code 0 -to -.codn #\ex10FFFF . -The argument is taken to be a Unicode code point value and the -corresponding character object is returned. +.meta function +must be a function that can be called with one argument. +For each element of +.metn object , +.meta function +is called with that element +as an argument. If a +.cod2 non- nil +value is returned, then the zero-based index of +that element is added to a list. Finally, the list is returned. -.coNP Accessor @ chr-str +.coNP Function @ select .synb -.mets (chr-str < str << idx ) -.mets (set (chr-str < str << idx ) << new-value ) +.mets (select < object >> { index-list <> | function }) .syne .desc -The -.code chr-str -function performs random access on string -.meta str -to retrieve -the character whose position is given by integer -.metn idx , -which must -be within range of the string. - -The index value 0 corresponds to the first (leftmost) character of the string -and so non-negative values up to one less than the length are possible. -Negative index values are also allowed, such that -1 corresponds to the -last (rightmost) character of the string, and so negative values down to -the additive inverse of the string length are possible. +The +.code select +function returns an object, of the same kind as +.metn object , +which consists of those elements of +.meta object +which are identified by +the indices in +.metn index-list , +which may be a list or a vector. -An empty string cannot be indexed. A string of length one supports index 0 and -index -1. A string of length two is indexed left to right by the values 0 and -1, and from right to left by -1 and -2. +If +.meta function +is given instead of +.metn index-list , +then +.meta function +is invoked with +.meta object +as its argument. The return value is then taken as +if it were the +.meta index-list +argument . -If the element -.meta idx -of string -.meta str -exists, and the string is modifiable, then the -.code chr-str -form denotes a place. +If +.meta object +is a sequence, then +.meta index-list +consists of numeric +indices. The +.code select +function stops processing +.meta object +upon encountering an +index inside +.meta index-list +which is out of range. (Rationale: without +this strict behavior, +.code select +would not be able to terminate if +.meta index-list +is infinite.) -A -.code chr-str -place -supports deletion. When a deletion takes place, -then the character at -.meta idx -is removed from the string. Any characters -after that position move by one position -to close the gap, and the length of the string -decreases by one. +If +.meta object +is a list, then +.meta index-list +must contain monotonically increasing +numeric values, even if no value is out of range, since the +.code select +function +makes a single pass through the list based on the assumption that indices +are ordered. (Rationale: optimization.) -.TP* Notes: +If +.meta object +is a hash, then +.meta index-list +is a list of keys. A new hash is +returned which contains those elements of +.meta object +whose keys appear +in +.metn index-list . +All of +.meta index-list +is processed, even if it contains +keys which are not in +.metn object . -Direct use of -.code chr-str -is equivalent to the DWIM bracket notation except -that -.code str -must be a string. The following relation holds: +.coNP Function @ in +.synb +.mets (in < sequence < key >> [ testfun <> [ keyfun ]]) +.mets (in < hash << key ) +.syne +.desc +The +.code in +function tests whether +.meta key +is found inside +.meta sequence +or +.metn hash . -.cblk - (chr-str s i) --> [s i] -.cble +If the +.meta testfun +argument is specified, it specifies the function +which is used to comparison keys from the sequence +to +.metn key . +Otherwise the +.code equal +function is used. -since -.codn [s i] <--> (ref s i) , -this also holds: +If the +.meta keyfun +argument is specified, it specifies a function which +is applied to the elements of +.meta sequence +to produce the comparison keys. Without this +argument, the elements themselves are taken +as the comparison keys. -.cblk - (chr-str s i) --> (ref s i) -.cble +If the object being searched is a hash, then the +.meta keyfun +and +.meta testfun +arguments are ignored. -However, note the following difference. When the expression -.code [s i] -is used as a place, then the subexpression -.code s -must be a place. When -.code (chr-str s i) -is used as a place, -.code s -need not be a place. +The +.code in +function returns +.code t +if it finds +.meta key +in +.meta sequence +or +.metn hash, +otherwise +.codn nil . -.coNP Function @ chr-str-set +.coNP Function @ partition .synb -.mets (chr-str-set < str < idx << char ) +.mets (partition < sequence >> { index-list >> | index <> | function }) .syne .desc -The -.code chr-str -function performs random access on string -.meta str -to overwrite -the character whose position is given by integer -.metn idx , -which must -be within range of the string. The character at -.meta idx -is overwritten -with character -.metn char . -The -.meta idx -argument works exactly as in -.codn chr-str . +If +.meta sequence +is empty, then +.code partition +returns an empty list, and the +second argument is ignored; if it is +.metn function , +it is not called. -The -.meta str -argument must be a modifiable string. +Otherwise, +.code partition +returns a lazy list of partitions of +.metn sequence . +Partitions are consecutive, non-overlapping, non-empty sub-strings of +.metn sequence , +of the same kind as +.metn sequence , +such that if these sub-strings are catenated together in their order +of appearance, a sequence +.code equal +to the original is produced. -.TP* Notes: +If the second argument is of the form +.metn index-list , +it shall be a sequence of +strictly non-decreasing, integers. First, any leading negative or zero values +in this sequence are dropped. The +.code partition +function then divides +.meta sequence +according to the +indices in index list. The first partition begins with the first element of +.metn sequence . +The second partition begins at the first position in +.metn index-list , +and so on. Indices beyond the length of the sequence are ignored. -Direct use of -.code chr-str -is equivalent to the DWIM bracket notation except -that -.meta str -must be a string. The following relation holds: +If +.meta index-list +is empty then a one-element list containing the entire +.meta sequence +is returned. -.cblk - (chr-str-set s i c) --> (set [s i] c) -.cble +If the second argument is a function, then this function is applied +to +.metn sequence , +and the return value of this call is then used in place of the +second argument, which must be an +.meta index +or +.metn index-list . -since -.codn (set [s i] c) <--> (refset s i c) , -this also holds: +If the second argument is an atom other than a function, it is assumed to be +an integer index, and is turned into an +.meta index-list +of one element. +.TP* Examples: .cblk - (chr-str s i) --> (refset s i c) + (partition '(1 2 3) 1) -> ((1) (2 3)) + + ;; split the string where there is a "b" + (partition "abcbcbd" (op where (op eql #\eb))) -> ("a" "bc" + "bc" "bd") .cble -.coNP Function @ span-str +.coNP Function @ split .synb -.mets (span-str < str << set ) +.mets (split < sequence >> { index-list >> | index <> | function }) .syne .desc -The -.code span-str -function determines the longest prefix of string -.meta str -which -consists only of the characters in string -.metn set , -in any combination. -.coNP Function @ compl-span-str -.synb -.mets (compl-span-str < str << set ) -.syne -.desc -The -.code compl-span-str -function determines the longest prefix of string -.meta str -which -consists only of the characters which do not appear in -.metn set , -in any combination. +If +.meta sequence +is empty, then +.code split +returns an empty list, and the +second argument is ignored; if it is +.metn function , +it is not called. -.coNP Function @ break-str -.synb -.mets (break-str < str << set ) -.syne -.desc +Otherwise, +.code split +returns a lazy list of pieces of +.metn sequence : +consecutive, non-overlapping, possibly empty sub-strings of +.metn sequence , +of the same kind as +.metn sequence . +A catenation of these pieces in the order they appear would produce +a sequence that is +.code equal +to the original sequence. + +If the second argument is of the form +.metn index-list , +it shall be a sequence of increasing integers. The -.code break-str -function returns an integer which represents the position of the -first character in string -.meta str -which appears in string -.metn set . +.code split +function divides +.meta sequence +according to the +indices in index list. The first piece always begins with the first +element of +.metn sequence . +Each subsequent piece begins with the position indicated by +an element of +.metn index-list . +Negative indices are ignored. Repeated values give rise to empty +pieces. +If +.meta index-list +includes index zero, +then an empty first piece is generated. +If +.meta index-list +includes an index greater than or equal to the length of +.meta sequence +(equivalently, an index beyond the last element of the sequence) +then an additional empty last piece is generated. -If there is no such character, then -.code nil +If +.meta index-list +is empty then a one-element list containing the entire +.meta sequence is returned. -.SS* Lazy Strings -Lazy strings are objects that were developed for the \*(TX pattern matching -language, and are exposed via \*(TL. Lazy strings behave much like strings, -and can be substituted for strings. However, unlike regular strings, which -exist in their entirety, first to last character, from the moment they are -created, lazy strings do not exist all at once, but are created on demand. If -character at index N of a lazy string is accessed, then characters 0 through N -of that string are forced into existence. However, characters at indices -beyond N need not necessarily exist. +If the second argument is a function, then this function is applied +to +.metn sequence , +and the return value of this call is then used in place of the +second argument, which must be an +.meta index +or +.metn index-list . -A lazy string dynamically grows by acquiring new text from a list of strings -which is attached to that lazy string object. When the lazy string is accessed -beyond the end of its hitherto materialized prefix, it takes enough strings -from the list in order to materialize the index. If the list doesn't have -enough material, then the access fails, just like an access beyond the end of a -regular string. A lazy string always takes whole strings from the attached -list. +If the second argument is an atom other than a function, it is assumed to be +an integer index, and is turned into an +.meta index-list +of one element. -Lazy string growth is achieved via the -.code lazy-str-force-upto -function which -forces a string to exist up to a given character position. This function is -used internally to handle various situations. +.TP* Examples: +.cblk + (split '(1 2 3) 1) -> ((1) (2 3)) -The -.code lazy-str-force -function forces the entire string to materialize. If the -string is connected to an infinite lazy list, this will exhaust all memory. + (split "abc" 0) -> ("" "abc") + (split "abc" 3) -> ("abc" "") + (split "abc" 1) -> ("a" "bc") + (split "abc" 0 1 2 3) -> ("" "a" "b" "c" "") + (split "abc" 1 2) -> ("a" "b" "c") -Lazy strings are specially recognized in many of the regular string functions, -which do the right thing with lazy strings. For instance when -.code sub-str -is invoked on a lazy string, a special version of the -.code sub-str -logic is -used which handles various lazy string cases, and can potentially return -another lazy string. Taking a -.code sub-str -of a lazy string from a given character position -to the end does not force the entire lazy string to exist, -and in fact the operation will work on a lazy string that is infinite. + (split "abc" -1 1 2 15) -> ("a" "b" "c") -Furthermore, special lazy string functions are provided which allow programs to -be written carefully to take better advantage of lazy strings. What carefully -means is code that avoids unnecessarily forcing the lazy string. For instance, -in many situations it is necessary to obtain the length of a string, only to -test it for equality or inequality with some number. But it is not necessary to -compute the length of a string in order to know that it is greater than some -value. + ;; triple split at makes two additional empty pieces + (split "abc" '(1 1 1)) -> ("a" "" "" "bc") +.cble -.coNP Function @ lazy-str +.coNP Function @ partition* .synb -.mets (lazy-str < string-list >> [ terminator <> [ limit-count ]]) +.mets (partition* < sequence >> { index-list >> | index <> | function }) .syne .desc -The -.code lazy-str -function constructs a lazy string which draws material from -.meta string-list -which is a list of strings. - -If the optional -.meta terminator -argument is given, then it specifies a string -which is appended to every string from -.metn string-list , -before that string is -incorporated into the lazy string. If -.meta terminator -is not given, -then it defaults to the string -.strn "\en" , -and so the strings from -.meta string-list -are effectively treated as lines which get terminated by newlines -as they accumulate into the growing prefix of the lazy string. -To avoid the use of a terminator string, a null string -.meta terminator -argument -must be explicitly passed. In that case, the lazy string grows simply -by catenating elements from -.metn string-list . +If +.meta sequence +is empty, then +.code partition* +returns an empty list, and the +second argument is ignored; if it is +.metn function , +it is not called. -If the -.meta limit-count -argument is specified, it must be a positive integer. It -expresses a maximum limit on how many elements will be consumed from -.meta string-list -in order to feed the lazy string. Once that many elements are -drawn, the string ends, even if the list has not been exhausted. +If the second argument is of the form +.metn index-list , +which is a sequence +of strictly increasing non-negative integers, then +.code partition* +produces a +lazy list of pieces taken from +.metn sequence . +The pieces are formed by +deleting from +.meta sequence +the elements at the positions given +in +.metn index-list . +The pieces are the non-empty sub-strings between +the deleted elements. -.coNP Function @ lazy-stringp -.synb -.mets (lazy-stringp << obj ) -.syne -.desc -The -.code lazy-stringp -function returns -.code t -if -.meta obj -is a lazy -string. Otherwise it returns -.codn nil . +If +.meta index-list +is empty then a one-element list containing the entire +.meta sequence +is returned. -.coNP Function @ lazy-str-force-upto -.synb -.mets (lazy-str-force-upto < lazy-str << index ) -.syne -.desc -The -.code lazy-str-force-upto -function tries to instantiate the lazy string such that -the position given by -.meta index -materializes. The +If the second argument is a function, then this function is applied +to +.metn sequence , +and the return value of this call is then used in place of the +second argument, which must be an .meta index -is a character -position, exactly as used in the -.code chr-str -function. +or +.metn index-list . -Some positions beyond -.meta index -may also materialize, as a side effect. +If the second argument is an atom other than a function, it is assumed to be +an integer index, and is turned into an +.meta index-list +of one element. -If the string is already materialized through to at least -.metn index , -or if it is -possible to materialize the string that far, then the value -.code t -is returned to indicate success. +.TP* Examples: +.cblk + (partition* '(1 2 3 4 5) '(0 2 4)) -> ((1) (3) (5)) -If there is insufficient material to force the lazy string through to the -.meta index -position, then nil is returned. + (partition* "abcd" '(0 3)) -> "bc" -It is an error if the -.meta lazy-str -argument isn't a lazy string. + (partition* "abcd" '(0 1 2 3)) -> nil +.cble -.coNP Function @ lazy-str-force +.coNP Functions @ find and @ find-if .synb -.mets (lazy-str-force << lazy-str ) +.mets (find < key < sequence >> [ testfun <> [ keyfun ]]) +.mets (find-if < predfun < sequence <> [ keyfun ]) .syne .desc The -.meta lazy-str -argument must be a lazy string. The lazy string is forced -to fully materialize. - -The return value is an ordinary, non-lazy string equivalent to the fully -materialized lazy string. +.code find +and +.code find-if +functions search through a sequence for an item which +matches a key, or satisfies a predicate function, respectively. -.coNP Function @ lazy-str-get-trailing-list -.synb -.mets (lazy-str-get-trailing-list < string << index ) -.syne -.desc The -.code lazy-str-get-trailing-list -function can be considered, in some way, an inverse operation to -the production of the lazy string from its associated list. - -First, -.meta string -is forced up through the position -.metn index . -That is the only extent to which -.meta string -is modified by this function. +.meta keyfun +argument specifies a function which is applied to the elements +of +.meta sequence +to produce the comparison key. If this argument is omitted, +then the untransformed elements of the +.meta sequence +are searched. -Next, the suffix of the materialized part of the lazy string starting at -position -.metn index , -is split into pieces on occurrences of the -terminator character (which had been given as the -.meta terminator -argument in the -.code lazy-str -constructor, and defaults to newline). If the -.meta index -position is beyond the part of the string which can be materialized -(in adherence with the lazy string's -.meta limit-count -constructor parameter), then the list of pieces is considered -to be empty. +The +.code find +function's +.meta testfun +argument specifies the test function which +is used to compare the comparison keys from +.meta sequence +to the search key. +If this argument is omitted, then the +.code equal +function is used. +The first element from the list whose comparison key (as retrieved by +.metn keyfun ) +matches the search (under +.metn testfun ) +is returned. If no such element is found, +.code nil +is returned. -Finally, a list is returned consisting of the pieces produced by the split, -to which is appended the remaining list of the string which has not yet been -forced to materialize. +The +.code find-if +function's +.meta predfun +argument specifies a predicate function +which is applied to the successive comparison keys pulled from the list +by applying +.meta keyfun +to successive elements. The first element +for which +.meta predfun +yields true is returned. If no such +element is found, +.code nil +is returned. -.coNP Functions @, length-str-> @, length-str->= @ length-str-< and @ length-str-<= +.coNP Functions @ find-max and @ find-min .synb -.mets (length-str-> < string << len ) -.mets (length-str->= < string << len ) -.mets (length-str-< < string << len ) -.mets (length-str-<= < string << len ) +.mets (find-max < sequence >> [ testfun <> [ keyfun ]]) +.mets (find-min < sequence >> [ testfun <> [ keyfun ]]) .syne .desc -These functions compare the lengths of two strings. The following -equivalences hold, as far as the resulting value is concerned: - -.cblk - (length-str-> s l) <--> (> (length-str s) l) - (length-str->= s l) <--> (>= (length-str s) l) - (length-str-< s l) <--> (< (length-str s) l) - (length-str-<= s l) <--> (<= (length-str s) l) -.cble +The +.code find-min +and +.code find-max +function implement exactly the same algorithm; they +differ only in their defaulting behavior with regard to the +.meta testfun +argument. If +.meta testfun +is not given, then the find-max function defaults it to +the +.code greater +function, whereas +.code find-min +defaults it to the +.code less +function. -The difference between the functions and the equivalent forms is that if the -string is lazy, the -.code length-str -function will fully force it in order to -calculate and return its length. +Without a +.meta testfun +argument, the +.code find-max +function finds the numerically +maximum value occurring in +.metn sequence , +whereas +.code pos-min +without a +.meta testfun +argument finds the minimum value. -These functions only force a string up to position -.metn len , -so they are not -only more efficient, but on infinitely long lazy strings they are usable. - -.code length-str -cannot compute the length of a lazy string with an unbounded -length; it will exhaust all memory trying to force the string. +If a +.meta testfun +argument is given, the two functions are equivalent. +The +.meta testfun +function must be callable with two arguments. +If +.meta testfun +behaves like a greater-than comparison, then +.code find-max +and +.code find-min +both return the maximum element. If +.meta testfun +behaves like a less-than comparison, then the functions return +the minimum element. -These functions can be used to test such as string whether it is longer -or shorter than a given length, without forcing the string beyond -that length. +The +.meta keyfun +argument defaults to the +.code identity +function. Each element +from +.meta sequence +is passed through this one-argument function, and +the resulting value is used in its place for the purposes of the +comparison. However, the original element is returned. -.coNP Function @ cmp-str +.coNP Function @ set-diff .synb -.mets (cmp-str < left-string << right-string ) +.mets (set-diff < seq1 < seq2 >> [ testfun <> [ keyfun ]]) .syne .desc The -.code cmp-str -function returns a negative integer if -.meta left-string -is lexicographically prior to -.metn right-string , -and a positive integer -if the reverse situation is the case. Otherwise the strings are equal -and zero is returned. +.code set-diff +function treats the sequences +.meta seq1 +and +.meta seq2 +as if they were sets +and computes the set difference: a sequence which contains those elements in +.meta seq1 +which do not occur in +.metn seq2 . -If either or both of the strings are lazy, then they are only forced to the -minimum extent necessary for the function to reach a conclusion and return the -appropriate value, since there is no need to look beyond the first character -position in which they differ. +.code set-diff +returns a sequence of the same kind as +.metn seq1 . -The lexicographic ordering is naive, based on the character code point -values in Unicode taken as integers, without regard for locale-specific -collation orders. +Element equivalence is determined by a combination of +.meta testfun +and +.metn keyfun . +Elements are compared pairwise, and each element of a pair is passed through +.meta keyfun +function to produce a comparison value. The comparison values +are compared using +.metn testfun . +If +.meta keyfun +is omitted, then the +untransformed elements themselves are compared, and if +.meta testfun +is omitted, +then the +.code equal +function is used. -.coNP Functions @, str= @, str< @, str> @ str>= and @ str<= +If +.meta seq1 +contains duplicate elements which do not occur in +.meta seq2 +(and thus are preserved in the set difference) then these duplicates appear +in the resulting +sequence. Furthermore, the order of the items from +.meta seq1 +is preserved. + +.coNP Functions @, mapcar @ mappend @ mapcar* and @ mappend* .synb -.mets (str= < left-string << right-string ) -.mets (str< < left-string << right-string ) -.mets (str> < left-string << right-string ) -.mets (str<= < left-string << right-string ) -.mets (str>= < left-string << right-string ) +.mets (mapcar < function << sequence *) +.mets (mappend < function << sequence *) +.mets (mapcar* < function << sequence *) +.mets (mappend* < function << sequence *) .syne .desc -These functions compare -.meta left-string -and -.meta right-string -lexicographically, -as if by the -.code cmp-str -function. -The -.code str= +When given only one argument, the +.code mapcar function returns -.code t -if the two strings are exactly the same, character -for character, otherwise it returns .codn nil . +.meta function +is never called. -The -.code str< -function returns -.code t -if -.meta left-string -is lexicographically before -.metn right-string , -otherwise nil. +When given two arguments, the +.code mapcar +function applies +.meta function +to each elements of +.meta sequence +and returns a sequence of the resulting values +in the same order as the original values. +The returned sequence is the same kind as +.metn sequence , +if possible. If the accumulated values cannot be +elements of that type of sequence, then a list is returned. -The -.code str> -function returns -.code t -if -.meta left-string -is lexicographically after -.metn right-string , -otherwise -.codn nil . +When additional sequences are given as arguments, this filtering behavior is +generalized in the following way: +.code mapcar +traverses the sequences in parallel, +taking a value from each sequence as an argument to the function. If there +are two lists, +.meta function +is called with two arguments and so forth. +The traversal is limited by the length of the shortest sequence. +The return values of the function are collected into a new sequence which is +returned. The returned sequence is of the same kind as the leftmost +input sequence, unless the accumulated values cannot be elements of that type of +sequence, in which case a list is returned. The -.code str< -function returns -.code t -if -.meta left-string -is lexicographically before -.metn right-string , -or if they are exactly the same, otherwise -.codn nil . +.code mappend +function works like +.codn mapcar , +with the following difference. +Rather than accumulating the values returned by the function into a sequence, +mappend expects the items returned by the function to be sequences which +are catenated with +.codn append , +and the resulting sequence is returned. The returned sequence is of the same +kind as the leftmost input sequence, unless the values cannot be elements +of that type of sequence, in which case a list is returned. The -.code str< -function returns -.code t -if -.meta left-string -is lexicographically after -.metn right-string , -or if they are exactly the same, otherwise +.code mapcar* +and +.code mappend* +functions work like +.code mapcar +and +.codn mappend , +respectively. +However, they return lazy lists rather than generating the entire +output list prior to returning. + +.TP* Caveats: + +Like +.codn mappend , +.code mappend* +must "consume" empty lists. For instance, +if the function being mapped puts out a sequence of +.codn nil s, +then the result must be the empty list +.codn nil , +because +.code (append nil nil nil nil ...) +is .codn nil . -.coNP Function @ string-lt -.synb -.mets (string-lt < left-str << right-str ) -.syne -.desc -The -.code string-lt -is a deprecated alias for -.codn str< . +But suppose that +.code mappend* +is used on inputs which are infinite lazy +lists, such that the function returns +.code nil +values indefinitely. +For instance: -.SS* Vectors -.coNP Function @ vector -.synb -.mets (vector < length <> [ initval ]) -.syne -.desc -The -.code vector -function creates and returns a vector object of the specified -length. The elements of the vector are initialized to -.metn initval , -or to nil if -.meta initval -is omitted. +.cblk + ;; Danger: infinite loop!!! + (mappend* (fun identity) (repeat '(nil))) +.cble -.coNP Function @ vec -.synb -.mets (vec << arg *) -.syne -.desc -The -.code vec -function creates a vector out of its arguments. - -.coNP Function @ vectorp -.synb -.mets (vectorp << obj ) -.syne -.desc -The -.code vectorp -function returns t if -.meta obj -is a vector, otherwise it returns -.codn nil . - -.coNP Function @ vec-set-length -.synb -.mets (vec-set-length < vec << len ) -.syne -.desc The -.code vec-set-length -modifies the length of -.metn vec , -making it longer or -shorter. If the vector is made longer, then the newly added elements -are initialized to nil. The -.meta len -argument must be nonnegative. +.code mappend* +function is caught in a loop trying to consume +and squash an infinite stream of +.codn nil s, +and so doesn't return. -The return value is -.metn vec . +.TP* Examples: +.cblk + ;; multiply every element by two + (mapcar (lambda (item) (* 2 item)) '(1 2 3)) -> (4 6 8) -.coNP Accessor @ vecref -.synb -.mets (vecref < vec << idx ) -.mets (set (vecref < vec << idx ) << new-value ) -.syne -.desc -The -.code vecref -function performs indexing into a vector. It retrieves -an element of -.meta vec -at position -.metn idx , -counted from zero. -The -.meta idx -value must range from 0 to one less than the -length of the vector. The specified element is returned. + ;; "zipper" two lists together + (mapcar (lambda (le ri) (list le ri)) '(1 2 3) '(a b c)) '((1 a) (2 b) (3 c))) -If the element -.meta idx -of vector -.meta vec -exists, then the -.code vecref -form denotes a place. + ;; like append, mappend allows a lone atom or a trailing atom: + (mappend (fun identity) 3) -> (3) + (mappend (fun identity) '((1) 2)) -> (1 . 2) -A -.code vecref place -supports deletion. When a deletion takes place, -then if -.meta idx -denotes the last element in the vector, the -vector's length is decreased by one, so that -the vector no longer has that element. -Otherwise, if -.meta idx -isn't the last element, then each elements -values at a higher index than -.meta idx -shifts by one one element position to the -adjacent lower index. Then, the length of the -vector is decreased by one, so that the last -element position disappears. + ;; take just the even numbers + (mappend (lambda (item) (if (evenp x) (list x))) '(1 2 3 4 5)) + -> (2 4) +.cble -.coNP Function @ vec-push +.coNP Function @ mapdo .synb -.mets (vec-push < vec << elem ) +.mets (mapdo < function << sequence *) .syne .desc The -.code vec-push -function extends the length of a vector -.meta vec -by one element, and -sets the new element to the value -.metn elem . +.code mapdo +function is similar to +.codn mapcar , +but always returns +.codn nil . +It is useful +when +.meta function +performs some kind of side effect, hence the "do" in the name, +which is a mnemonic for the execution of imperative actions. -The previous length of the vector (which is also the position of -.metn elem ) +When only the +.meta function +argument is given, +.meta function +is never called, +and +.code nil is returned. -This function performs similarly to the generic function -.codn ref , -except that the -first argument must be a vector. +If a single +.meta sequence +argument is given, then +.code mapdo +iterates over +.metn sequence , +invoking +.meta function +on each element. -.coNP Function @ length-vec -.synb -.mets (length-vec << vec ) -.syne -.desc -The -.code length-vec -function returns the length of vector -.metn vec . -It performs -similarly to the generic -.code length -function, except that the argument must -be a vector. +If two or more +.meta sequence +arguments are given, then +.code mapdo +iterates over +the sequences in parallel, extracting parallel tuples of items. These +tuples are passed as arguments to +.metn function, +which must accept as many +arguments as there are sequences. -.coNP Function @ size-vec +.coNP Functions @ transpose and @ zip .synb -.mets (size-vec << vec ) +.mets (transpose << sequence ) +.mets (zip << sequence *) .syne .desc The -.code size-vec -function returns the number of elements for which storage -is reserved in the vector -.metn vec . +.code transpose +function performs a transposition on +.metn sequence . +This means that the +elements of +.meta sequence +must be sequences. These sequences are understood to be +columns; transpose exchanges rows and columns, returning a sequence of the rows +which make up the columns. The returned sequence is of the same kind as +.metn sequence , +and the rows are also the same kind of sequence as the first column +of the original sequence. The number of rows returned is limited by the +shortest column among the sequences. -.TP* Notes: +All of the input sequences (the elements of +.metn sequence) +must have elements +which are compatible with the first sequence. This means that if the first +element of +.meta sequence +is a string, then the remaining sequences must be +strings, or else sequences of characters, or of strings. The -.code length -of the vector can be extended up to this size without any memory -allocation operations having to be performed. +.code zip +function takes variable arguments, and is equivalent to calling +.code transpose +on a list of the arguments. The following equivalences hold: -.coNP Function @ vector-list .synb -.mets (vector-list << list ) -.syne -.desc -This function returns a vector which contains all of the same elements -and in the same order as list -.metn list . + (zip . x) <--> (transpose x) -.coNP Function @ list-vector -.synb -.mets (list-vector << vec ) + [apply zip x] <--> (transpose x) .syne -.desc -The -.code list-vector -function returns a list of the elements of vector -.metn vec . -.coNP Function @ copy-vec -.synb -.mets (copy-vec << vec ) -.syne -.desc -The -.code copy-vec -function returns a new vector object of the same length -as -.meta vec -and containing the same elements in the same order. +.TP* Examples: +.cblk + ;; transpose list of lists + (transpose '((a b c) (c d e))) -> ((a c) (b d) (c e)) -.coNP Function @ sub-vec + ;; transpose vector of strings: + ;; - string columns become string rows + ;; - vector input becomes vector output + (transpose #("abc" "def" "ghij")) -> #("adg" "beh" "cfi") + + ;; error: transpose wants to make a list of strings + ;; but 1 is not a character + (transpose #("abc" "def" '(1 2 3))) ;; error! + + ;; String elements are catenated: + (transpose #("abc" "def" ("UV" "XY" "WZ"))) -> #("adUV" "beXY" "cfWZ") + + (zip '(a b c) '(c d e)) -> ((a c) (b d) (c e)) +.cble + +.coNP Function @ interpose .synb -.mets (sub-vec < vec >> [ from <> [ to ]]) +.mets (interpose < sep << sequence ) .syne .desc The -.code sub-vec -function is like the more generic function -.codn sub , -except that it -operates only on vectors. - -For a description of the arguments and semantics, refer to the -.code sub -function. +.code interpose +function returns a sequence of the same type as +.metn sequence , +in which the elements from +.meta sequence +appear with the +.meta sep +value inserted +between them. -.coNP Function @ replace-vec -.synb -.mets (replace-vec < vec < item-sequence >> [ from <> [ to ]]) -.syne -.desc -The -.code replace-vec -is like the -.code replace -function, except that the first argument -must be a vector. +If +.meta sequence +is an empty sequence or a sequence of length 1, then a +sequence identical to +.meta sequence +is returned. It may be a copy of +.meta sequence +or it may be +.meta sequence +itself. -For a description of the arguments, semantics and return value, refer to the -.code replace -function. +If +.meta sequence +is a character string, then the value +.meta sep +must be a character. -.coNP Function @ cat-vec +It is permissible for +.metn sequence , +or for a suffix of +.meta sequence +to be a lazy +list, in which case interpose returns a lazy list, or a list with a lazy +suffix. + +.TP* Examples: +.cblk + (interpose #\e- "xyz") -> "x-y-z" + (interpose t nil) -> nil + (interpose t #()) -> #() + (interpose #\ea "") -> "" + (interpose t (range 0 0)) -> (0) + (interpose t (range 0 1)) -> (0 t 1) + (interpose t (range 0 2)) -> (0 t 1 t 2) +.cble + +.coNP Functions @ apply and @ iapply .synb -.mets (cat-vec << vec-list ) +.mets (apply < function <> [ arg * << trailing-args ]) +.mets (iapply < function <> [ arg * << trailing-args ]) .syne .desc The -.meta vec-list -argument is a list of vectors. The -.code cat-vec -function -produces a catenation of the vectors listed in -.metn vec-list . -It returns -a single large vector formed by catenating those vectors together in -order. +.code apply +function invokes +.metn function , +optionally passing to it an argument +list. The return value of the +.code apply +call is that of +.metn function . -.SS* Sequence Manipulation -.coNP Function @ seqp -.synb -.mets (seqp << object ) -.syne -.desc -The function -.code seqp -returns -.code t -if -.meta object -is a sequence, otherwise -.codn nil . +If no arguments are present after +.metn function , +then +.meta function +is invoked without arguments. -A sequence is defined as a list, vector or string. The object -.code nil -denotes -the empty list and so is a sequence. +If one argument is present after +.metn function , +then it is interpreted as +.metn trailing-args . +If this is a sequence (a list, vector or string), +then the elements of the sequence are passed as individual arguments to +.metn function . +If +.meta trailing-args +is not a sequence, then +.meta function +is invoked +with an improper argument list, terminated by the +.meta trailing-args +atom. + +If two or more arguments are present after +.metn function , +then the last of these arguments is interpreted as +.metn trailing-args . +The previous arguments represent leading arguments which are applied to +.metn function , +prior to the arguments taken from +.metn trailing-args . -.coNP Function @ length -.synb -.mets (length << sequence ) -.syne -.desc The -.code length -function returns the number of items in -.metn sequence , -and -returns it. -.meta sequence -may be a hash, in which case +.code iapply +function ("improper apply") is similar to +.codn apply , +except with regard to the treatment of +.metn trailing-args . +Firstly, under +.codn iapply , +if +.meta trailing-args +is an atom other than +.code nil +(possibly a sequence, such as a vector or string), +then it is treated as an ordinary argument: +.meta function +is invoked with a proper argument list, whose last element is +.metn trailing-args . +Secondly, if +.meta trailing-args +is a list, but an improper list, then the terminating atom of +.meta trailing-args +becomes an ordinary argument. Thus, in all possible cases, +.code iapply +treats an extra +.cod2 non- nil +atom as an argument, and never calls +.meta function +with an improper argument list. + +.TP* Examples: .cblk -.meti (hash-count << sequence ) + ;; '(1 2 3) becomes arguments to list, thus (list 1 2 3). + (apply (fun list) '(1 2 3)) -> (1 2 3) + + ;; this effectively invokes (list 1 2 3 4) + (apply (fun list) 1 2 '(3 4)) -> (1 2 3) + + ;; this effectively invokes (list 1 2 . 3) + (apply (fun list) 1 2 3)) -> (1 2 . 3) + + ;; "abc" is separated into characters which become arguments of list + (apply (fun list) "abc") -> (#\ea #\eb #\ec) .cble -is returned. -.coNP Function @ empty -.synb -.mets (empty << sequence ) -.syne -.desc -Returns -.code t -if +.TP* "Dialect Note:" +Note that some uses of this function that are necessary in other Lisp dialects +are not necessary in \*(TL. The reason is that in \*(TL, improper list +syntax is accepted as a compound form, and performs application: + .cblk -.meti (length << sequence ) + (foo a b . x) .cble -is zero, otherwise -.codn nil . -.coNP Function @ copy +Here, the variables +.code a +and +.code b +supply the first two arguments for +.codn foo . +In +the dotted position, +.code x +must evaluate to a list or vector. The list or +vector's elements are pulled out and treated as additional arguments for +.codn foo . +Of course, this syntax can only be used if +.code x +is a symbolic form or an atom. It +cannot be a compound form, because +.code (foo a b . (x)) +and +.code (foo a b x) +are equivalent structures. + +.coNP Functions @ reduce-left and @ reduce-right .synb -.mets (copy << object ) +.mets (reduce-left < binary-function < list +.mets \ \ \ \ \ \ \ \ \ \ \ \ >> [ init-value <> [ key-function ]]) + +.mets (reduce-right < binary-function < list +.mets \ \ \ \ \ \ \ \ \ \ \ \ \ >> [ init-value <> [ key-function ]]) .syne .desc The -.code copy -function duplicates objects of various supported types: sequences, hashes and random states. If -.meta object -is -.codn nil , -it -returns -.codn nil . +.code reduce-left +and +.code reduce-right +functions reduce lists of operands specified +by +.meta list +and +.meta init-value +to a single value by the repeated application of +.metn binary-function . + +An effective list of operands is formed by combining +.meta list +and +.metn init-value . If -.meta object -is a list, it returns -.cblk -.meti (copy-list << object ). -.cble -If -.meta object -is a string, it returns -.cblk -.meti (copy-str << object ). -.cble -If -.meta object -is a vector, it returns -.cblk -.meti (copy-vec << object ). -.cble +.meta key-function +is specified, then the items of +.meta list +are +mapped to a new values through +.metn key-function . If -.meta object -is a hash, it returns -.cblk -.meti (copy-hash << object ) -.cble -Lastly, if -.meta object -is a random state, it returns -.cblk -.meti (make-random-state << object ). -.cble +.meta init-value +is supplied, +then in the case of +.codn reduce-left , +the effective list of operands is formed by +prepending +.meta init-value +to +.metn list . +In the case of +.codn reduce-right , +the effective operand list is produced by appending +.meta init-value +to +.metn list . + +The production of the effective list can be expressed like this, +though this is not to be understood as the actual implementation: -Except in the case when -.meta sequence -is -.codn nil , -.code copy -returns a value that -is distinct from (not -.code eq -to) -.metn sequence . -This is different from -the behavior of -.cblk -.meti >> [ sequence 0..t] -.cblk -or .cblk -.meti (sub < sequence 0 t) + (append (if init-value-present (list init-value)) + [mapcar (or key-function identity) list])))) .cble -which recognize -that they need not make a copy of -.metn sequence , -and just return it. -Note however, that the elements of the returned sequence may be -eq to elements of the original sequence. In other words, copy is -a deeper copy than just duplicating the -.code sequence -value itself, -but it is not a deep copy. +In the +.code reduce-right +case, the arguments to +.code append +are reversed. -.coNP Function @ sub -.synb -.mets (sub < sequence >> [ from <> [ to ]]) -.syne -.desc -The -.code sub -function extracts a slice from input sequence -.metn sequence . -The slice is -a sequence of the same type as -.metn sequence . +If the effective list of operands is empty, then +.meta binary-function +is called +with no arguments at all, and its value is returned. This is the only +case in which +.meta binary-function +is called with no arguments; in all +remaining cases, it is called with two arguments. -If the -.meta from -argument is omitted, it defaults to -.codn 0 . -If the -.meta to -parameter is -omitted, it defaults to -.codn t . -Thus -.code (sub a) -means -.codn (sub a 0 t) . +If the effective list contains one item, then that item is returned. -The following equivalence holds between the -.code sub -function and -the DWIM-bracket syntax: +Otherwise, the effective list contains two or more items, and is decimated as +follows. + +Note that an +.meta init-value +specified as +.code nil +is not the same as a missing +.metn init-value ; +this means that the initial value is the object +.codn nil . +Omitting +.meta init-value +is the same as specifying a value of +.code : +(the colon symbol). +It is possible to specify +.meta key-function +while omitting an +.meta init-value +argument. This is achieved by explicitly specifying +.code : +as the +.meta init-value +argument. + +Under +.codn reduce-left , +the leftmost pair of operands is removed +from the list and passed as arguments to +.metn binary-function , +in the same order +that they appear in the list, and the resulting value initializes an +accumulator. Then, for each remaining item in the list, +.meta binary-function +is invoked on two arguments: the current accumulator value, and the next element +from the list. After each call, the accumulator is updated with the return +value of +.metn binary-function . +The final value of the accumulator is returned. +Under +.codn reduce-right , +the list is processed right to left. The rightmost +pair of elements in the effective list is removed, and passed as arguments to +.metn binary-function , +in the same order that they appear in the list. The +resulting value initializes an accumulator. Then, for each remaining item in +the list, +.meta binary-function +is invoked on two arguments: the +next element from the list, in right to left order, and the current +accumulator value. After each call, the accumulator is updated with the return +value of +.metn binary-function . +The final value of the accumulator is returned. + +.TP* Examples: .cblk - ;; from is not a list - (sub seq from to) <--> [seq from..to] -.cble + ;;; effective list is (1) so 1 is returned + (reduce-left (fun +) () 1 nil) -> 1 -The description of the -.code dwim -operator\(emin particular, the section -on Range Indexing\(emexplains the semantics of the range specification. + ;;; computes (- (- (- 0 1) 2) 3) + (reduce-left (fun -) '(1 2 3) 0 nil) -> -6 -If the sequence is a list, the output sequence may share substructure -with the input sequence. + ;;; computes (- 1 (- 2 (- 3 0))) + (reduce-right (fun -) '(1 2 3) 0 nil) -> 2 -.coNP Function @ replace + ;;; computes (* 1 2 3) + (reduce-left (fun *) '((1) (2) (3)) nil (fun first)) -> 6 + + ;;; computes 1 because the effective list is empty + ;;; and so * is called with no arguments, which yields 1. + (reduce-left (fun *) nil) +.cble + +.coNP Function @, some @ all and @ none .synb -.mets (replace < sequence < replacement-sequence >> [ from <> [ to ]]) -.mets (replace < sequence < replacement-sequence << index-list ) +.mets (some < sequence >> [ predicate-fun <> [ key-fun ]]) +.mets (all < sequence >> [ predicate-fun <> [ key-fun ]]) +.mets (none < sequence >> [ predicate-fun <> [ key-fun ]]) .syne .desc The -.meta replace -function modifies -.meta sequence -in the ways described below. - -The operation is destructive: it may work "in place" by modifying -the original sequence. The caller should retain the return value -and stop relying on the original input sequence. - -The return value of -.code replace -is the modified -version of -.metn sequence . -This may be the same object as +.codn some , +.code all +and +.code none +functions apply a predicate test function +.meta predicate-fun +over a list of elements. If the argument +.meta key-fun +is +specified, then elements of .meta sequence -or it may be a newly allocated object. - -Note that the form: +are passed into +.metn key-fun , +and +.meta predicate-fun +is +applied to the resulting values. If +.meta key-fun +is omitted, the behavior is +as if +.meta key-fun +is the identity function. If +.meta predicate-fun +is omitted, +the behavior is as if +.meta predicate-fun +is the identity function. -.cblk - (set seq (replace seq new fr to)) -.cble +These functions have short-circuiting semantics and return conventions similar +to the and and or operators. -has the same effect on the variable -.code seq -as the form: - -.cblk - (set [seq fr..to] new) -.cble - -except that the former -.code set -form returns the entire modified sequence, whereas the latter -returns the value of the -.code new -argument. +The some function applies +.meta predicate-fun +to successive values +produced by retrieving elements of +.meta list +and processing them through +.metn key-fun . +If the list is empty, it returns +.codn nil . +Otherwise it returns the +first +.cod2 non- nil +return value returned by a call to +.meta predicate-fun +and +stops evaluating more elements. If +.meta predicate-fun +returns +.code nil +for all +elements, it returns +.metn nil . The -.code replace -function has two invocation styles, distinguished by the -type of the third argument. If the third argument is a list or vector, then it -is deemed to be the -.meta index-list -parameter of the second form. -Otherwise, if the third argument is missing, or is not a list, then -it is deemed to be the -.meta from -argument of the first form. +.code all +function applies +.meta predicate-fun +to successive values +produced by retrieving elements of +.meta list +and processing them through +.metn key-fun . +If the list is empty, it returns +.codn t . +Otherwise, if +.meta predicate-fun +yields +.code nil +for any value, the +.cod all +function immediately +returns without invoking +.meta predicate-fun +on any more elements. +If all the elements are processed, then the all function returns +the value which +.meta predicate-fun +yielded for the last element. -The first form of the replace function replaces a contiguous subsequence of the -.meta sequence -with -.metn replacement-sequence . -The replaced subsequence may be empty, -in which case an insertion is performed. If -.meta replacement-sequence -is empty -(for example, the empty list -.codn nil ), -then a deletion is performed. +The +.code none +function applies +.meta predicate-fun +to successive values +produced by retrieving elements of +.meta list +and processing them through +.metn key-fun . +If the list is empty, it returns +.codn t . +Otherwise, if +.meta predicate-fun +yields +.cod2 non- nil +for any value, the none function +immediately returns nil. If +.meta predicate-fun +yields nil for all +values, the none function returns +.codn t . -If the -.meta from -and -.meta to -arguments are omitted, their values default -to -.code 0 -and -.code t -respectively. +.TP* Examples: -The description of the dwim operator\(emin particular, the section -on Range Indexing\(emexplains the semantics of the range specification. +.cblk + ;; some of the integers are odd + [some '(2 4 6 9) oddp] -> t -The second form of the replace function replaces a subsequence of -elements from -.meta sequence -given by -.metn index-list , -with their counterparts -from -.metn replacement-sequence . -This form of the replace function does not insert -or delete; it simply overwrites elements. If -.meta replacement-sequence -and -.meta index-list -are of different lengths, then the shorter of the two determines -the maximum number of elements which are overwritten. -Furthermore, similar restrictions apply on -.meta index-list -as under the -select function. Namely, the replacement stops when an index value -in -.meta index-list -is encountered which is out of range for -.metn sequence . -furthermore, if -.meta sequence -is a list, then -.meta index-list -must -be monotonically increasing. + ;; none of the integers are even + [none '(1 3 4 7) evenp] -> t +.cble -.coNP Function @ search +.coNP Function @ multi .synb -.mets (search < haystack < needle >> [ testfun <> [ keyfun ]) +.mets (multi < function << sequence *) .syne .desc The -.code search -function determines whether the sequence -.meta needle -occurs as substring -within -.metn haystack , -under the given comparison function -.meta testfun -and -key function -.metn keyfun . -If this is the case, then the zero-based position of -the leftmost occurrence of -.meta key -within -.meta haystack -is returned. Otherwise -.code nil -is returned to indicate that -.meta key -does not occur within -.metn haystack . -If -.meta key -is empty, then zero is always returned. +.code multi +function distributes an arbitrary list processing function +.meta multi +over multiple sequences given by the +.meta list +arguments. -The arguments -.meta haystack -and -.meta needle -are sequences: lists, vectors -or strings, in any combination. +The +.meta sequence +arguments are first transposed into a single list of tuples. Each +successive element of this transposed list consists of a tuple of the +successive items from the lists. The length of the transposed list is that +of the shortest +.meta list +argument. -If -.meta needle -is not empty, then occurs at some position N within -.meta haystack -if -the first element of -.meta needle -matches the element at position N of -.metn haystack , -the second element of -.meta needle -matches the element at position N+1 of -.meta haystack -and so forth, for all elements of -.metn needle . -A match between elements -is determined by passing each element through -.metn keyfun , -and then comparing the resulting values using -.metn testfun . +The transposed list is then passed to +.meta function +as an argument. -If -.meta testfun -is supplied, it must be a function which can be -called with two arguments. If it is not supplied, it defaults to -.codn eql . +The +.meta function +is expected to produce a list of tuples, which are transposed +again to produce a list of lists which is then returned. -If -.meta keyfun -is supplied, it must be a function which can be called -with one argument. If it is not supplied, it defaults to -.codn identity . +Conceptually, the input sequences are columns and +.meta function +is invoked on +a list of the rows formed from these columns. The output of +.meta function +is a transformed list of rows which is reconstituted into a list of columns. -.TP* Examples: +.TP* Example: .cblk - ;; fails because 3.0 doesn't match 3 - ;; under the default eql function - [search #(1.0 3.0 4.0 7.0) '(3 4)] -> nil + ;; Take three lists in parallel, and remove from all of them + ;; them the element at all positions where the third list + ;; has an element of 20. - ;; occurrence found at position 1: - ;; (3.0 4.0) matches (3 4) under = - [search #(1.0 3.0 4.0 7.0) '(3 4) =] -> 1 + (multi (op remove-if (op eql 20) @1 third) + '(1 2 3) + '(a b c) + '(10 20 30)) - ;; "even odd odd odd even" pattern - ;; matches at position 2 - [search #(1 1 2 3 5 7 8) '(2 1 1 1 2) : evenp] -> 2 + -> ((1 3) (a c) (10 30)) - ;; Case insensitive string search - [search "abcd" "CD" : chr-toupper] -> 2 + ;; The (2 b 20) "row" is gone from the three "columns". - ;; Case insensitive string search - ;; using vector of characters as key - [search "abcd" #(#\eC #\eD) : chr-toupper] -> 2 + ;; Note that the (op remove if (op eql 20) @1 third) + ;; expression can be simplified using the ap operator: + ;; + ;; (op remove-if (ap eql @3 20)) .cble -.coNP Functions @ ref and @ refset +.coNP Function @ sort .synb -.mets (ref < seq << index ) -.mets (refset < seq < index << new-value ) +.mets (sort < sequence >> [ lessfun <> [ keyfun ]]) .syne .desc The -.code ref +.code sort +function destructively sorts +.metn sequence , +producing a sequence +which is sorted according to the +.meta lessfun and -.code refset -functions perform array-like indexing into sequences. +.meta keyfun +arguments. + The -.code ref -function retrieves an element of -.metn seq , -whereas -.code refset -overwrites an -element of -.meta seq -with a new value. +.meta keyfun +argument specifies a function which is applied to elements +of the sequence to obtain the key values which are then compared +using the lessfun. If +.meta keyfun +is omitted, the identity function is used +by default: the sequence elements themselves are their own sort keys. The -.meta index -argument is based from zero, and negative values are permitted, -with a special meaning as described in the Range Indexing section under the -description of the -.code dwim -operator. +.meta lessfun +argument specifies the comparison function which determines +the sorting order. It must be a binary function which can be invoked +on pairs of keys as produced by the key function. It must +return a +.cod2 non- nil +value if the left argument is considered to be lesser +than the right argument. For instance, if the numeric function +.code < +is used +on numeric keys, it produces an ascending sorted order. If the function +.code > +is used, then a descending sort is produced. If +.meta lessfun +is omitted, then it defaults to the generic +.code less +function. The -.code refset -function returns the new value. +.code sort +function is stable for sequences which are lists. This means that the +original order of items which are considered identical is preserved. +For strings and vectors, +.code sort +is not stable. -The following equivalences hold between -.code ref +.coNP Function @ sort-group +.synb +.mets (sort-group < sequence >> [ keyfun <> [ lessfun ]]) +.syne +.desc +The +.code sort-group +function sorts +.meta sequence +according to the +.meta keyfun and -.codn refset , -and the DWIM bracket syntax: +.meta lessfun +arguments, and then breaks the resulting sequence into groups, +based on the equivalence of the elements under +.metn keyfun . -.cblk - (ref seq idx) <--> [seq idx] +The following equivalence holds: - (refset seq idx new) <--> (set [seq idx] new) +.cblk + (sort-group sq lf kf) <--> (partition-by kf (sort (copy sq) kf lf)) .cble -The difference is that -.code ref +Note the reversed order of +.meta keyfun and -.code refset -are first class functions which -can be used in functional programming as higher order functions, whereas the -bracket notation is syntactic sugar, and -.code set -is an operator, not a function. -Therefore the brackets cannot replace all uses of -.code ref +.meta lessfun +arguments between +.code sort and -.codn refset . +.codn sort-group . -.coNP Function @ update +.coNP Function @ uniq .synb -.mets (update < sequence-or-hash << function ) +.mets (uniq << sequence ) .syne .desc The -.code update -function replaces each elements in a sequence, or each value -in a hash table, with the value of -.meta function -applied to that element -or value. +.code uniq +function returns a sequence of the same kind as +.metn sequence , +but with +duplicates removed. Elements of +.meta sequence +are considered equal under +the +.code equal +function. The first occurrence of each element is retained, +and the subsequent duplicates of that element, of any, are suppressed, +such that the order of the elements is otherwise preserved. -The sequence or hash table is returned. +The following equivalence holds between +.code uniq +and +.codn unique : -.coNP Function @ less +.cblk + (uniq s) <--> [unique s : :equal-based] +.cble + +That is, +.code uniq +is like +.code unique +with the default +.meta keyfun +argument (the +.code identity +function) and an +.codn equal -based +hash table. + +.coNP Function @ unique .synb -.mets (less < left-obj << right-obj ) -.mets (less < obj << obj *) +.mets (uniq < sequence >> [ keyfun <> { hash-arg }* ]) .syne .desc The -.code less -function, when called with two arguments, determines whether -.meta left-obj -compares less than -.meta right-obj -in a generic way which handles arguments of various types. +.code unique +function is a generalization of +.codn uniq . +It returns a sequence of the same kind as +.metn sequence , +but with duplicates removed. -The argument syntax of -.code less -is generalized. It can accept one argument, in which case it unconditionally -returns -.code t -regardless of that argument's value. If more than two arguments are -given, then -.code less -generalizes in a way which can be described by the following equivalence -pattern, with the understanding that each argument expression -is evaluated exactly once: +If neither +.meta keyfun +nor +.metn hash-arg -s +are specified, then elements of sequence are considered equal under the +.code eql +function. The first occurrence of each element is retained, +and the subsequent duplicates of that element, of any, are suppressed, +such that the order of the elements is otherwise preserved. -.cblk - (less a b c) <--> (and (less a b) (less b c)) - (less a b c d) <--> (and (less a b) (less b c) (less c d)) -.cble +If +.meta keyfun +is specified, then that function is applied to each element, +and the resulting values are compared for equality. +In other words, the behavior is as if +.meta keyfun +were the +.code identity +function. -The -.code less -function is used as the default for the -.meta lessfun -argument of the functions -.code sort -and -.codn merge , -as well as the -.meta testfun -argument of the -.code pos-min -and -.codn find-min . +If one or more +.metn hash-arg -s +are present, these specify the arguments for the construction of +the internal hash table used by +.codn unique . +The arguments are like those of the +.code hash +function. In particular, the argument +.code :equal-based +causes +.code unique +to use +.code equal +equality. +.coNP Function @ tuples +.synb +.mets (tuples < length < sequence <> [ fill-value ]) +.syne +.desc The -.code less -function is capable of comparing numbers, characters, symbols, strings, -as well as lists and vectors of these. +.code tuples +function produces a lazy list which represents a reorganization +of the elements of +.meta sequence +into tuples of +.metn length , +where +.meta length +must be a positive integer. + +The length of the sequence might not be evenly divisible by the tuple length. +In this case, if a +.meta fill-value +argument is specified, then the last tuple +is padded with enough repetitions of +.meta fill-value +to make it have +.meta length +elements. If +.meta fill-value +is not specified, then the last tuple is left +shorter than +.metn length . + +The output of the function is a list, but the tuples themselves are sequences +of the same kind as +.metn sequence . +If +.meta sequence +is any kind of list, they +are lists, and not lazy lists. + +.TP* Examples: -If both arguments are the same object so that .cblk -.meti (eq < left-obj << right-obj ) + (tuples 3 #(1 2 3 4 5 6 7 8) 0) -> (#(1 2 3) #(4 5 6) #(7 8 0)) + (tuples 3 "abc") -> ("abc") + (tuples 3 "abcd") -> ("abc" "d") + (tuples 3 "abcd" #\ez) -> ("abc" "dzz") + (tuples 3 (list 1 2) #\ez) -> ((1 2 #\ez)) .cble -holds true, then the function returns -.code nil -regardless of the type of -.metn left-obj , -even if the function doesn't handle comparing different instances -of that type. In other words, no object is less than itself, no matter -what it is. -If both arguments are numbers or characters, they are compared as if using the -.code < function. +.coNP Function @ partition-by +.synb +.mets (partition-by < function << sequence ) +.syne +.desc +If +.meta sequence +is empty, then +.code partition-by +returns an empty list, +and +.meta function +is never called. -If both arguments are strings, they are compared as if using the -.code string-lt -function. +Otherwise, +.code partition-by +returns a lazy list of partitions of the sequence +.metn sequence . +Partitions are consecutive, non-empty sub-strings of +.metn sequence , +of the same kind as +.metn sequence . -If both arguments are symbols, then their names are compared in -their place, as if by the -.code string-lt -function. +The partitioning begins with the first element of +.meta sequence +being placed into a partition. -If both arguments are conses, then they are compared as follows: -.RS -.IP 1. -The -.code less -function is recursively applied to the -.code car -fields of both arguments. If it yields true, then -.meta left-obj -is deemed to be less than -.metn right-obj . -.IP 2. -Otherwise, if the -.code car -fields are unequal under -the -.code equal -function, -.code less -returns -.codn nil. -.IP 3. -If the -.code car -fields are +The subsequent partitioning is done according to +.metn function , +which is applied +to each element of +.metn sequence . +Whenever, for the next element, the function +returns the same value as it returned for the previous element, the +element is placed into the same partition. Otherwise, the next element +is placed into, and begins, a new partition. + +The return values of the calls to +.meta function +are compared using the .code equal -then -.code less -is recursively applied to the -.code cdr -fields of the arguments, and the result of that comparison is returned. -.RE +function. -.IP -This logic performs a lexicographic comparison on ordinary lists such -that for instance -.code (1 1) -is less than -.code (1 1 1) -but not less than -.code (1 0) -or -.codn (1) . +.TP* Examples: -Note that the empty -.code nil -list nil compared to a cons is handled by type-based precedence, described -below. +.cblk + [partition-by identity '(1 2 3 3 4 4 4 5)] -> ((1) (2) (3 3) + (4 4 4) (5)) -If the arguments are vectors, they are compared lexicographically, similar -to strings. Corresponding elements, starting with element 0, of the -vectors are compared until an index position is found where the vectors -differ. If this differing position is beyond the end of one of the two vectors, -then the shorter vector is considered to be lesser. Otherwise, the result -of -.code less -is the outcome of comparing those differing elements themselves -with -.codn less . + (partition-by (op = 3) #(1 2 3 4 5 6 7)) -> (#(1 2) #(3) + #(4 5 6 7)) +.cble -If the two arguments are of the above types, but of mutually different types, +.coNP Function @ make-like +.synb +.mets (make-like < list << ref-sequence ) +.syne +.desc +The +.meta list +argument must be a list. If +.meta ref-sequence +is a sequence type, then -.code less -resolves the situation based on the following precedence: numbers and -characters are less than strings, which are less than symbols, -which are less than conses, which are less than vectors. - -Note that since -.code nil -is a symbol, it is ranked lower than a cons. This interpretation ensures -correct behavior when -.code nil -is regarded as an empty list, since the empty list is lexicographically prior to -a nonempty list. +.meta list +is converted to the same type of sequence and returned. +Otherwise the original +.meta list +is returned. -Finally, if either of the arguments has a type other than the above discussed -types, the situation is an error. +Note: the +.code make-like +function is a helper which supports the development of +unoptimized versions of a generic function that accepts any type of +sequence as input, and produces a sequence of the same type as output. +The implementation of such a function can internally accumulate a list, and +then convert the resulting list to the same type as an input value +by using +.codn make-like . -.coNP Function @ greater +.coNP Function @ nullify .synb -.mets (greater < left-obj << right-obj ) -.mets (greater < obj << obj *) +.mets (nullify << sequence ) .syne .desc The -.code -greater -function is equivalent to -.code less -with the arguments reversed. That is to say, the following -equivalences hold: - -.cblk - (greater a <--> (less a) <--> t - (greater a b) <--> (less b a) - (greater a b c ...) <--> (less ... c b a) -.cble +.code nullify +function returns +.code nil +if +.meta sequence +is an empty sequence. +Otherwise it returns +.meta sequence +itself. -The -.code greater -function is used as the default for the -.meta testfun -argument of the -.code pos-max +Note: the +.code nullify +function is a helper to support unoptimized generic +programming over sequences. Thanks to the generic behavior of +.codn cdr , +any sequence can be traversed using +.code cdr +functions, checking for the +.code nil +value as a terminator. This, however, breaks for empty sequences which are not +lists, because they are not equal to +.codn nil : +to +.code car and -.code find-max -functions. +.code cdr +they look like +a one-element sequence containing +.codn nil . +The +.code nullify +function reduces all empty +sequences to +.codn nil , +thereby correcting the behavior of code which traverses +sequences using +.codn cdr , +and tests for termination with +.codn nil . -.coNP Functions @ lequal and @ gequal +.SS* Permutations and Combinations + +.coNP Function @ perm .synb -.mets (lequal < obj << obj *) -.mets (gequal < obj << obj *) +.mets (perm < seq <> [ len ]) .syne .desc -The functions -.code lequal -and -.code gequal -are similar to -.code less -and -.code greater -respectively, but differ in the following respect: -when called with two arguments which compare true under the -.code equal -function, the -.code lequal -and -.code gequal -functions return -.codn t . +The +.code rperm +function returns a lazy list which consists of all +length +.meta len +permutations of formed by items taken from +.metn seq . +The permutations do not use any element of +.meta seq +more than once. -When called with only one argument, both functions return -.code t -and both functions generalize to three or more arguments -in the same way as do -.code less -and -.codn greater . +Argument +.metn len , +if present, must be a positive integer, and +.meta seq +must be a sequence. -.coNP Function @ sort -.synb -.mets (sort < sequence >> [ lessfun <> [ keyfun ]]) -.syne -.desc -The -.code sort -function destructively sorts -.metn sequence , -producing a sequence -which is sorted according to the -.meta lessfun -and -.meta keyfun -arguments. +If +.meta len +is not present, then its value defaults to the length of +.metn seq : +the list of the full permutations of the entire sequence is returned. -The -.meta keyfun -argument specifies a function which is applied to elements -of the sequence to obtain the key values which are then compared -using the lessfun. If -.meta keyfun -is omitted, the identity function is used -by default: the sequence elements themselves are their own sort keys. +The permutations in the returned list are sequences of the same kind as +.codn seq . -The -.meta lessfun -argument specifies the comparison function which determines -the sorting order. It must be a binary function which can be invoked -on pairs of keys as produced by the key function. It must -return a -.cod2 non- nil -value if the left argument is considered to be lesser -than the right argument. For instance, if the numeric function -.code < -is used -on numeric keys, it produces an ascending sorted order. If the function -.code > -is used, then a descending sort is produced. If -.meta lessfun -is omitted, then it defaults to the generic -.code less -function. +If +.meta len +is zero, then a list containing one permutation is returned, and that +permutations is of zero length. -The -.code sort -function is stable for sequences which are lists. This means that the -original order of items which are considered identical is preserved. -For strings and vectors, -.code sort -is not stable. +If +.meta len +exceeds the length of +.metn seq , +then an empty list is returned, +since it is impossible to make a single non-repeating permutation that +requires more items than are available. -.coNP Function @ sort-group +The permutations are lexicographically ordered. + +.coNP Function @ rperm .synb -.mets (sort-group < sequence >> [ keyfun <> [ lessfun ]]) +.mets (rperm < seq << len ) .syne .desc The -.code sort-group -function sorts -.meta sequence -according to the -.meta keyfun -and -.meta lessfun -arguments, and then breaks the resulting sequence into groups, -based on the equivalence of the elements under -.metn keyfun . +.code rperm +function returns a lazy list which consists of all the repeating +permutations of length +.meta len +formed by items taken from +.metn seq . +"Repeating" means that the items from +.meta seq +can appear more than +once in the permutations. -The following equivalence holds: +The permutations which are returned are sequences of the same kind as +.metn seq . -.cblk - (sort-group sq lf kf) <--> (partition-by kf (sort (copy sq) kf lf)) -.cble +Argument +.meta len +must be a nonnegative integer, and +.meta seq +must be a sequence. -Note the reversed order of -.meta keyfun -and -.meta lessfun -arguments between -.code sort -and -.codn sort-group . +If +.meta len +is zero, then a single permutation is returned, of zero length. +This is true regardless of whether +.meta seq +is itself empty. -.coNP Function @ uniq -.synb -.mets (uniq << sequence ) -.syne -.desc -The -.code uniq -function returns a sequence of the same kind as -.metn sequence , -but with -duplicates removed. Elements of -.meta sequence -are considered equal under -the -.code equal -function. The first occurrence of each element is retained, -and the subsequent duplicates of that element, of any, are suppressed, -such that the order of the elements is otherwise preserved. +If +.meta seq +is empty and +.meta len +is greater than zero, then no permutations are +returned, since permutations of a positive length require items, and the +sequence has no items. Thus there exist no such permutations. -The following equivalence holds between -.code uniq -and -.codn unique : +The first permutation consists of +.meta le +repetitions of the first element of +.metn seq . +The next repetition, if there is one, differs from the first +repetition in that its last element is the second element of +.metn seq . +That is to say, the permutations are lexicographically ordered. + +.TP* Examples: .cblk - (uniq s) <--> [unique s : :equal-based] -.cble + (rperm "01" 4) -> ("000" "001" "010" "011" + "100" "101" "110" "111") -That is, -.code uniq -is like -.code unique -with the default -.meta keyfun -argument (the -.code identity -function) and an -.codn equal -based -hash table. + (rperm #(1) 3) -> (#(1 1 1)) -.coNP Function @ unique + (rperm '(0 1 2) 2) -> ((0 0) (0 1) (0 2) (1 0) + (1 1) (1 2) (2 0) (2 1) (2 2)) +.cble + +.coNP Function @ comb .synb -.mets (uniq < sequence >> [ keyfun <> { hash-arg }* ]) +.mets (comb < seq << len ) .syne .desc The -.code unique -function is a generalization of -.codn uniq . -It returns a sequence of the same kind as -.metn sequence , -but with duplicates removed. +.code comb +function returns a lazy list which consists of all +length +.meta len +non-repeating combinations formed by taking items taken from +.metn seq . +"Non-repeating combinations" means that the combinations do not use any +element of +.meta seq +more than once. If +.meta seq +contains no duplicates, then +the combinations contain no duplicates. -If neither -.meta keyfun -nor -.metn hash-arg -s -are specified, then elements of sequence are considered equal under the -.code eql -function. The first occurrence of each element is retained, -and the subsequent duplicates of that element, of any, are suppressed, -such that the order of the elements is otherwise preserved. +Argument +.meta len +must be a nonnegative integer, and +.meta seq +must be a sequence or a hash table. + +The combinations in the returned list are objects of the same kind as +.metn seq . If -.meta keyfun -is specified, then that function is applied to each element, -and the resulting values are compared for equality. -In other words, the behavior is as if -.meta keyfun -were the -.code identity -function. +.meta len +is zero, then a list containing one combination is returned, and that +permutations is of zero length. -If one or more -.metn hash-arg -s -are present, these specify the arguments for the construction of -the internal hash table used by -.codn unique . -The arguments are like those of the -.code hash -function. In particular, the argument -.code :equal-based -causes -.code unique -to use -.code equal -equality. +If +.meta len +exceeds the number of elements in +.metn seq , +then an empty list is returned, since it is impossible to make a single +non-repeating combination that requires more items than are available. -.coNP Function @ tuples +If +.meta seq +is a sequence, the returned combinations are lexicographically ordered. +This requirement is not applicable when +.meta seq +is a hash table. + +.TP* Example: +.cblk + ;; powerset function, in terms of comb. + ;; Yields a lazy list of all subsets of s, + ;; expressed as sequences of the same type as s. + + (defun powerset (s) + (mappend* (op comb s) (range 0 (length s)))) +.cble + +.coNP Function @ rcomb .synb -.mets (tuples < length < sequence <> [ fill-value ]) +.mets (rcomb < seq << len ) .syne .desc The -.code tuples -function produces a lazy list which represents a reorganization -of the elements of -.meta sequence -into tuples of -.metn length , -where -.meta length -must be a positive integer. +.code comb +function returns a lazy list which consists of all +length +.meta len +repeating combinations formed by taking items taken from +.metn seq . +"Repeating combinations" means that the combinations can use +an element of +.meta seq +more than once. -The length of the sequence might not be evenly divisible by the tuple length. -In this case, if a -.meta fill-value -argument is specified, then the last tuple -is padded with enough repetitions of -.meta fill-value -to make it have -.meta length -elements. If -.meta fill-value -is not specified, then the last tuple is left -shorter than -.metn length . +Argument +.meta len +must be a nonnegative integer, and +.meta seq +must be a sequence. + +The combinations in the returned list are sequences of the same kind as +.metn seq . -The output of the function is a list, but the tuples themselves are sequences -of the same kind as -.metn sequence . If -.meta sequence -is any kind of list, they -are lists, and not lazy lists. +.meta len +is zero, then a list containing one combination is returned, and that +permutations is of zero length. This is true even if +.meta seq +is empty. -.TP* Examples: +If +.meta seq +is empty, and +.meta len +is nonzero, then an empty list is returned. -.cblk - (tuples 3 #(1 2 3 4 5 6 7 8) 0) -> (#(1 2 3) #(4 5 6) #(7 8 0)) - (tuples 3 "abc") -> ("abc") - (tuples 3 "abcd") -> ("abc" "d") - (tuples 3 "abcd" #\ez) -> ("abc" "dzz") - (tuples 3 (list 1 2) #\ez) -> ((1 2 #\ez)) -.cble +The combinations are lexicographically ordered. -.coNP Function @ partition-by -.synb -.mets (partition-by < function << sequence ) -.syne -.desc -If -.meta sequence -is empty, then -.code partition-by -returns an empty list, -and -.meta function -is never called. -Otherwise, -.code partition-by -returns a lazy list of partitions of the sequence -.metn sequence . -Partitions are consecutive, non-empty sub-strings of -.metn sequence , -of the same kind as -.metn sequence . +.SS* Macros +\*(TL supports structural macros. \*(TX's model of macroexpansion is that +\*(TL code is processed in two phases: the expansion phase and the +evaluation phase. The expansion phase is invoked on Lisp code early during the +processing of source code. For instance when a \*(TX file containing a +.code @(do ...) +directive +is loaded, expansion of the Lisp forms are its arguments takes place during the +parsing of the entire source file, and is complete before any of the code in +that file is executed. If the +.code @(do ...) +form is later executed, +the expanded forms are then evaluated. -The partitioning begins with the first element of -.meta sequence -being placed into a partition. +\*(TL also supports symbol macros, which are symbolic forms that stand +for forms, with which they are replaced at macro expansion time. -The subsequent partitioning is done according to -.metn function , -which is applied -to each element of -.metn sequence . -Whenever, for the next element, the function -returns the same value as it returned for the previous element, the -element is placed into the same partition. Otherwise, the next element -is placed into, and begins, a new partition. +When Lisp data is processed as code by the +.code eval +function, it is first expanded, +and so processed in its entirety in the expansion phase. Then it is processed +in the evaluation phase. -The return values of the calls to -.meta function -are compared using the -.code equal -function. +.coNP Macro parameter lists -.TP* Examples: +\*(TX macros support destructuring, similarly to Common Lisp macros. +This means that macro parameter lists are like function argument lists, +but support nesting. A macro parameter list can specify a nested parameter +list in every place where a function argument list allows only a parameter +name. For instance, consider this macro parameter list: .cblk - [partition-by identity '(1 2 3 3 4 4 4 5)] -> ((1) (2) (3 3) - (4 4 4) (5)) - - (partition-by (op = 3) #(1 2 3 4 5 6 7)) -> (#(1 2) #(3) - #(4 5 6 7)) + ((a (b c)) : (c frm) ((d e) frm2 de-p) . g) .cble -.coNP Function @ make-like -.synb -.mets (make-like < list << ref-sequence ) -.syne -.desc -The -.meta list -argument must be a list. If -.meta ref-sequence -is a sequence type, -then -.meta list -is converted to the same type of sequence and returned. -Otherwise the original -.meta list -is returned. - -Note: the -.code make-like -function is a helper which supports the development of -unoptimized versions of a generic function that accepts any type of -sequence as input, and produces a sequence of the same type as output. -The implementation of such a function can internally accumulate a list, and -then convert the resulting list to the same type as an input value -by using -.codn make-like . +The top level of this list has four elements: the mandatory parameter +.codn (a (b c)) , +the optional parameter +.code c +(with default init form +.codn frm ), +the optional parameter +.code (d e) +(with default init form +.code frm2 +and presence-indicating variable +.codn de-p ), +and the dot-position parameter +.code g +which captures trailing arguments. -.coNP Function @ nullify -.synb -.mets (nullify << sequence ) -.syne -.desc -The -.code nullify -function returns -.code nil -if -.meta sequence -is an empty sequence. -Otherwise it returns -.meta sequence -itself. +Note that some of the parameters are compounds: +.code (a (b c)) +and +.codn (d e) . +These compounds express nested macro parameter lists. -Note: the -.code nullify -function is a helper to support unoptimized generic -programming over sequences. Thanks to the generic behavior of -.codn cdr , -any sequence can be traversed using -.code cdr -functions, checking for the -.code nil -value as a terminator. This, however, breaks for empty sequences which are not -lists, because they are not equal to -.codn nil : +Macro parameter lists match a similar tree structure to their own. +For instance a mandatory parameter +.code (a (b c)) +matches a structure like +.codn (1 (2 3)) , +such that the parameters +.codn a , +.code b +and +.code c +will end up bound to -.code car +.codn 1 , +.code 2 and -.code cdr -they look like -a one-element sequence containing +.codn 3 , +respectively. + +The binding strictness is relaxed for optional parameters. If +.code (a (b c)) +is optional, and the argument is, say, +.codn (1) , +then +.code a +gets +.codn 1 , +and +.code b +and +.code c +receive .codn nil . + +Macro parameter lists also support two special keywords, namely +.code :env +and +.codn :whole . + +The parameter list +.code (:whole x :env y) +will bind parameter +.code x +to the entire +macro form, and bind parameter +.code y +to the macro environment. The -.code nullify -function reduces all empty -sequences to -.codn nil , -thereby correcting the behavior of code which traverses -sequences using -.codn cdr , -and tests for termination with -.codn nil . +.code :whole +and +.code :env +notation can occur anywhere in a macro parameter list. -.SS* Math Library -.coNP Functions @ + and @ - +.coNP Operator @ macro-time .synb -.mets (+ << number *) -.mets (- < number << number *) -.mets (* << number *) +.mets (macro-time << form *) .syne .desc The -.codn + , -.code - -and -.code * -functions perform addition, subtraction and multiplication, -respectively. Additionally, the -.code - -function performs additive inverse. +.code macro-time +operator has a syntax similar to the +.code progn +operator. Each +.meta form +is evaluated from left to right, and the resulting value is that of the last +form. -The -.code + -function requires zero or more arguments. When called with no -arguments, it produces 0 (the identity element for addition), otherwise it -produces the sum over all of the arguments. +The special behavior of +.code macro-time +is that the evaluation takes place during +the expansion phase, rather than during the evaluation phase. -Similarly, the -.code * -function requires zero or more arguments. When called -with no arguments, it produces 1 (the identity element for multiplication). -Otherwise it produces the product of all the arguments. +During the expansion phase, all +.code macro-time +expressions which occur in a context +that calls for evaluation are evaluated, and replaced by their quoted values. +For instance +.code (macro-time (list 1 2 3)) +evaluates +.code (list 1 2 3) +to the object +.code (1 2 3) +and the entire +.code macro-time +form is replaced by that value, quoted: +.codn '(1 2 3) . +If the form is evaluated again at evaluation-time, the resulting value will be +that of the quote, in this case +.codn (1 2 3) . -The semantics of -.code - -changes from subtraction to additive inverse -when there is only one argument. The argument is treated as a subtrahend, -against an implicit minuend of zero. When there are two or more -argument, the first one is the minuend, and the remaining are subtrahends. +.code macro-time +forms do not see the surrounding lexical environment; the see only +global function and variable bindings and macros. -When there are three or more operands, these operations are performed as if by -binary operations, in a left-associative way. That is to say, -.code (+ a b c) -means -.codn (+ (+ a b) c) . -The sum of -.code a +Note 1: +.code macro-time +is intended for defining helper functions and variables that +are used by macros. A macro cannot "see" a +.code defun +function or +.code defvar +variable +because +.code defun and -.code b -is computed first, and then this is added to -.codn c . -Similarly -.code (- a b c) -means -.codn (- (- a b) c) . -First, -.code b -is subtracted from -.codn a , -and then -.code c -is subtracted from that result. +.code defvar +forms are evaluated at evaluation time, which occurs +after expansion time. The macro-time operator hoists the evaluation of these +forms to macro-expansion time. -The arithmetic inverse is performed as if it were subtraction from integer 0. -That is, -.code (- x) -means the same thing as -.codn (- 0 x) . +Note 2: +.code defmacro +forms are implicitly in macro-time; they do not have to +be wrapped with the +.code macro-time +operator. The +.code macro-time +operator doesn't have +to be used in programs whose macros do not make references to variables +or functions. -The operands of -.codn + , -.code - +.coNP Operator @ defmacro +.synb +.mets (defmacro < name <> ( param * [: << opt-param * ] [. < rest-param ]) +.mets \ \ << body-form *) +.syne +.desc +The +.code defmacro +operator is evaluated at expansion time. It defines a +macro-expander function under the name +.metn name , +effectively creating a new operator. + +Note that the parameter list is a macro parameter list, and not a +function parameter list. This means that each +.meta param and -.code * -can be characters, integers (fixnum and bignum), and -floats, in nearly any combination. +.meta opt-param +can be not only a symbol, but it can itself be a parameter list. +The corresponding argument is then treated as a structure which +matches that parameter list. This nesting of parameter lists +can be carried to an arbitrary depth. -If two operands have different types, then one of them is converted to the -type of the one with the higher rank, according to this ranking: -character < integer < float. For instance if one operand is integer, and the -other float, the integer is converted to a float. +A macro is called like any other operator, and resembles a function. Unlike in +a function call, the macro receives the argument expressions themselves, rather +than their values. Therefore it operates on syntax rather than on values. +Also, unlike a function call, a macro call occurs in the expansion phase, +rather than the evaluation phase. -.TP* Restrictions: +The return value of the macro is the macro expansion. It is substituted in +place of the entire macro call form. That form is then expanded again; +it may itself be another macro call, or contain more macro calls. -Characters are not considered numbers, and participate in these operations in -limited ways. Subtraction can be used to computed the displacement between the -Unicode values of characters, and an integer displacement can be added to a -character, or subtracted from a character. For instance -.codn (- #\e9 #\e0) is 9 . -The Unicode value of a character -.code C -can be found using -.codn (- C #\ex0) : -the displacement from the NUL character. +.TP* Example: -The rules can be stated as a set of restrictions: -.RS -.IP 1 -Two characters may not be added together. -.IP 2 -A character may not be subtracted from an integer (which also rules out -the possibility of computing the additive inverse of a character). -.IP 3 -A character operand may not be opposite to a floating point operand -in any operation. -.IP 4 -A character may not be an operand of multiplication. -.RE +.cblk + ;; dolist macro similar to Common Lisp's: + ;; + ;; The following will print 1, 2 and 3 + ;; on separate lines: + ;; and return 42. + ;; + ;; (dolist (x '(1 2 3) 42) + ;; (format t "~s\en")) -.PP + (defmacro dolist ((var list : result) . body) + (let ((i (my-gensym))) + ^(for ((i ,list)) (i ,result) ((set i (cdr i))) + (let ((,var (car i))) + ,*body)))) +.cble -.coNP Functions @, / @ trunc, @ mod and @ trunc-rem +.coNP Operator @ macrolet .synb -.mets (/ <> [ dividend ] << divisor ) -.mets (trunc < dividend << divisor ) -.mets (mod < dividend << divisor ) -.mets (trunc-rem < dividend << divisor ) +.mets (macrolet >> ({( name < macro-style-params << macro-body-form *)}*) +.mets \ \ << body-form *) .syne .desc -The arguments to these functions are numbers. Characters are not permitted. - The -.code / -function performs floating-point division. Each operands is first -converted to floating-point type, if necessary. If -.meta dividend -is omitted, -then it is taken to be -.code 1.0 -and the function calculates the reciprocal. +.code macrolet +binding operator extends the macro-time lexical environment +by making zero or more new local macros visible. The -.code trunc -function performs a division of -.meta dividend -by -.meta divisor -whose result -is truncated to integer toward zero. If both operands are integers, then an -integer division is performed and the result is an integer. If either operand -is a floating point value, a floating point division occurs, and the result is -truncated toward zero to a floating-point integral value. +.code macrolet +symbol is followed by a list of macro definitions. +Each definition is a form which begins with a +.metn name , +followed by +.meta macro-style-params +which is a macro parameter list, and zero or more +.metn macro-body-form s. +These macro definitions are similar +to those globally defined by the +.code defmacro +operator, except that they +are in a local environment. -The -.code mod -function performs a modulus operation. Firstly, the absolute value -of -.meta divisor -is taken to be a modulus. Then a residue of -.meta dividend -with respect to -.meta modulus -is calculated. The residue's sign follows -that of the sign of -.metn divisor . -That is, it is the smallest magnitude -(closest to zero) residue of -.meta dividend -with respect to the absolute -value of -.metn divisor , -having the same sign as -.metn divisor . -If the operands are integer, the result is an integer. If either operand -is of type float, then the result is a float. The modulus operation is -then generalized into the floating point domain. For instance the expression -.code (mod 0.75 0.5) -yields a residue of 0.25 because 0.5 "goes into" 0.75 only -once, with a "remainder" of 0.25. +The macro definitions are followed by optional +.metn body-forms . +The macros specified in the definitions are visible to these +forms. -The -.code trunc-rem -function returns a list of two values: a -.meta quotient -and a -.metn remainder . -The -.meta quotient -is exactly the same value as what -.code trunc -would return for the same inputs. -The -.meta remainder -obeys the following identity: +Forms inside the macro definitions such as the +.metn macro-body-form s, +and initializer forms appearing in the +.meta macro-style-params +are subject +to macro-expansion in a scope in which none of the new macros being +defined are yet visible. Once the macro definitions are themselves +macro-expanded, they are placed into a new macro environment, which +is then used for macro expanding the +.metn body-form s. -.cblk -.mets (eql < remainder (- < dividend >> (* divisor << quotient ))) -.cble +A +.code macrolet +form is fully processed in the expansion phase of a form, and is +effectively replaced by +.code progn +form which contains expanded versions of +.metn body-form s. +This expanded structure shows no evidence that any +macrolet forms ever existed in it. Therefore, it is impossible for the code +evaluated in the bodies and parameter lists of +.code macrolet +macros to have any visibility to any surrounding lexical variable bindings, +which are only instantiated in the evaluation phase, after expansion is done +and macros no longer exist. -.coNP Functions @ wrap and @ wrap* +.coNP Function @ macro-form-p .synb -.mets (wrap < start < end << number ) -.mets (wrap* < start < end << number ) +.mets (macro-form-p < obj <> [ env ]) .syne .desc The -.code wrap -and -.code wrap* -functions reduce -.meta number -into the range specified by -.meta start -and -.metn end . +.code macro-form-p +function returns +.code t +if +.meta obj +represents the syntax of +a form which is a macro form: either a compound macro or a symbol macro. +Otherwise it returns +.codn nil . -Under -.code wrap -the range is inclusive of the -.meta end -value, whereas under -.code wrap* -it is exclusive. +A macro form is one that will transform under +.code macroexpand-1 +or +.codn macroexpand ; +an object which isn't a macro form will not undergo expansion. -The following equivalence holds +The optional +.meta env +parameter is a macroexpansion environment. +A macroexpansion environment is passed down to macros and can be received +via their special +.code :env +parameter. -.cblk - (wrap a b c) <--> (wrap* a (succ b) c) -.cble +.meta env +is used by +.code macro-form-p +to determine whether +.meta obj +is a macro in a lexical macro environment. -The expression -.code (wrap* x0 x1 x) -performs the following calculation: +If +.meta env +is not specified or is +.codn nil , +then +.code macro-form-p +only recognizes global macros. + +.TP* Example: .cblk -.mets (+ (mod (- x x0) (- x1 x0)) x0) -.cble + ;; macro which translates to 'yes if its + ;; argument is a macro from, or otherwise + ;; transforms to the form 'no. -In other words, first -.meta start -is subtracted from -.metn number . -Then the result is reduced modulo the displacement -between -.code start -and -.codn end . -Finally, -.meta start -is added back to that result, which is returned. + (defmacro ismacro (:env menv form) + (if (macro-form-p form menv) + ''yes ''no)) -.TP* Example: + (macrolet ((local ())) + (ismacro (local))) ;; yields yes -.cblk - ;; perform ROT13 on the string "nop" - [mapcar (opip (+ 13) (wrap #\ea #\ez)) "nop"] -> "abc" -.cble + (ismacro (local)) ;; yields no -.coNP Functions @ gcd and @ lcm -.synb -.mets (gcd << number *) -.mets (lcm << number *) -.syne -.desc -The -.code gcd -function computes the greatest common divisor: the largest positive -integer which divides each -.metn number . + (ismacro (ismacro foo)) ;; yields yes +.cble -The -.code lcm -function computes the lowest common multiple: the smallest positive -integer which is a multiple of -each -.metn number . +During macro expansion, the global macro +.code ismacro +is handed the macro-expansion environment +via +.codn :env menv . -Each -.meta number -must be an integer. +When the macro is invoked within the macrolet, +this environment includes the macro-time lexical scope in which the +.code local +macro is defined. So when global checks whether the argument form +.code (local) +is a macro, the conclusion is yes: the (local) form is a macro +call in that environment: +.code macro-form-p +yields +.codn t . -Negative integers are replaced by their absolute values, so -.code (lcm -3 -4) -is -.code 12 -and -.code (gcd -12 -9) +When +.code (global (local)) +is invoked outside of the macrolet, no local macro is visible is +there, and so +.code macro-form-p yields -.codn 3 . +.codn nil . -The value of -.code (gcd) -is -.code 0 -and that of -.code (lcm) -is 1 . - -The value of -.code (gcd x) -and -.code (lcm x) -is -.codn (abs x) . - -Any arguments of -.code gcd -which are zero are effectively ignored so that -.code (gcd 0) -and -.code (gcd 0 0 0) -are both the same as -.code (gcd) -and -.code (gcd 1 0 2 0 3) -is the same as -.codn (gcd 1 2 3) . - -If -.code lcm -has any argument which is zero, it yields zero. - -.coNP Function @ abs +.coNP Functions @ macroexpand-1 and @ macroexpand .synb -.mets (abs << number ) +.mets (macroexpand-1 < obj <> [ env ]) +.mets (macroexpand < obj <> [ env ]) .syne .desc -The -.code abs -function computes the absolute value of -.metn number . If -.meta number -is positive, it is returned. If -.meta number -is negative, its additive inverse is -returned: a positive number of the same type with exactly the same magnitude. +.meta obj +is a macro form (an object for which +.code macro-form-p +returns +.codn t ), +these functions expand the macro form and return the expanded form. +Otherwise, they return +.metn obj . -.coNP Functions @ floor and @ ceil -.synb -.mets (floor << number ) -.mets (ceil << number ) -.syne -.desc -The -.code floor -function returns the highest integer which does not exceed -the value of -.metn number . -The ceiling function returns the lowest integer which -does not exceed the value of -.metn number . +.code macroexpand-1 +performs a single expansion, expanding just the macro +that is referenced by the symbol in the first position of +.metn obj , +and returns the expansion. That expansion may itself be a macro form. -If -.meta number -an integer, it is simply returned. +.code macroexpand +performs an expansion similar to +.codn macroexpand-1 . +If the result is +a macro form, then it expands that form, and keeps repeating this process +until the expansion yields a non-macro-form. That non-macro-form is then +returned. -If the argument is a float, then the value returned is a float. -For instance -.code (floor 1.1) -returns 1.0 rather than 1. +The optional +.meta env +parameter is a macroexpansion environment. +A macroexpansion environment is passed down to macros and can be received +via their special +.code :env +parameter. The environment they receive is their +lexically apparent macro-time environment in which local macros may be +visible. A macro can use this environment to "manually" expand some +form in the context of that environment. -.coNP Functions @, sin @, cos @, tan @, asin @, acos @ atan and @ atan2 -.synb -.mets (sin << radians ) -.mets (cos << radians ) -.mets (tan << radians ) -.mets (atan << slope ) -.mets (atan2 < y << x ) -.mets (asin << num ) -.mets (acos << num ) -.syne -.desc -These trigonometric functions convert their argument to floating point and -return a float result. The -.codn sin , -.code cos +.TP* Example: + +.cblk + ;; (foo x) expands x, and if x begins with a number, + ;; it removes the number and returns the resulting + ;; form. Otherwise, it returns the entire form. + + (defmacro rem-num (:env menv some-form) + (let ((expanded (macroexpand some-form menv))) + (if (numberp (car expanded)) + (cdr expanded) + some-form))) + + ;; The following yields (42 a). + + (macrolet ((foo () '(1 list 42)) + (bar () '(list 'a))) + (list (rem-num (foo)) (rem-num (bar))))) +.cble + +The +.code rem-num +macro is able to expand the +.code (foo) and -.code tan -functions compute the sine and -cosine and tangent of the -.meta radians -argument which represents an angle -expressed in radians. The -.codn atan , -.code acos +.code (bar) +forms it receives as +the +.code some-form +argument, even though these forms use local macro that are only +visible in their local scope. This is thanks to the macro +environment passed to +.codn rem-num . +It is correctly able to work with the +expansions +.code (1 list 42) and -.code asin -are their respective inverse -functions. The -.meta num -argument to -.code asin +.code (list 'a) +to produce +.code (list 42) and -.code acos -must be in the -range -1.0 to 1.0. The -.code atan2 -function converts the rectilinear coordinates -.meta x +.code (list 'a) +which evaluate to +.code 42 and -.meta y -to an angle in polar coordinates in the range [0, 2\(*p). +.code a +respectively. -.coNP Functions @, exp @, log @ log10 and @ log2 +.coNP Functions @ lexical-var-p and @ lexical-fun-p .synb -.mets (exp << arg ) -.mets (log << arg ) -.mets (log10 << arg ) -.mets (log2 << arg ) +.mets (lexical-var-p < env << form ) +.mets (lexical-fun-p < env << form ) .syne .desc -The -.code exp -function calculates the value of the transcendental number e raised to -the exponent -.metn arg . +These two functions are useful to macro writers. They are intended +to be called from the bodies of macro expanders, such as the bodies of +.code defmacro +or +.code macrolet +forms. The +.meta env +argument is a macro-time environment, which is available to macros +via the special +.code :env +parameter. Using these functions, a macro can enquire whether +a given +.meta form +is a symbol which has a variable binding or a function binding +in the lexical environment. +This information is known during macro expansion. The macro expander +recognizes lexical function and variable bindings, because these +bindings can shadow macros. -The -.code log -function calculates the base e logarithm of -.metn arg , -which must be a positive value. +.TP* Example: -The -.code log10 -function calculates the base 10 logarithm of -.metn arg , -which must be a positive value. +.cblk + ;; + ;; this macro replaces itself with :lexical-var if its + ;; argument is a lexical variable, :lexical-fun if + ;; its argument is a lexical function, or with + ;; :not-lex-fun-var if neither is the case. + ;; + (defmacro classify (sym :env e) + (cond + ((lexical-var-p e expr) :lexical-var) + ((lexical-fun-p e expr) :lexical-fun) + (t :not-lex-fun-var))) -The -.code log2 -function calculates the base 2 logarithm of -.metn arg , -which must be a positive value. + ;; + ;; This returns: + ;; + ;; (:lexical-var :not-lex-fun-var :lexical-fun) + ;; + (let ((x 1) (y 2)) + (symacrolet ((y x)) + (flet ((f () (+ 2 2))) + (list (classify x) (classify y) (classify z))))) +.cble -.coNP Functions @, expt @ sqrt and @ isqrt -.synb -.mets (expt < base << exponent *) -.mets (sqrt << arg ) -.mets (isqrt << arg ) -.syne -.desc -The -.code expt -function raises -.meta base -to zero or more exponents given -by the -.meta exponent -arguments. -.code (expt x) -is equivalent to -.codn (expt x 1) , -and yields -.code x -for all -.codn x . -For three or more arguments, the operation is right-associative. -That is to say, -.code (expt x y z) -is equivalent to -.codn (expt x (expt y z)) , -similarly to the way nested exponents work in standard algebraic -notation. - -Exponentiation is done pairwise using a binary operation. -If both operands to this binary operation are integers, then the -result is an integer. If either operand is a float, then the other -operand is converted to a float, and a floating point exponentiation -is performed. Exponentiation that would produce a complex number is -not supported. - -The -.code sqrt -function produces a floating-point square root of -.metn arg , -which is converted from integer to floating-point if necessary. Negative -operands are not supported. +.TP* Note: -The -.code isqrt -function computes the integer square root of -.metn arg , -which must be an integer. -The integer square root is a value which is the -greatest integer that is no greater than the real square root of -.metn arg . -The input value must be an integer. +These functions do not call +.code macroexpand +on the form. In most cases, it is necessary for the macro writers +to do so. Not that in the above example, symbol +.code y +is classified as neither a lexical function nor variable. +However, it can be macro-expanded to +.code x +which is a lexical variable. -.coNP Function @ exptmod +.coNP Function @ lexical-lisp1-binding .synb -.mets (exptmod < base < exponent << modulus ) +.mets (lexical-lisp1-binding < env << symbol ) .syne .desc The -.code exptmod -function performs modular exponentiation and accepts only integer -arguments. Furthermore, -.meta exponent -must be a non-negative and -.meta modulus -must be positive. +.code lexical-lisp1-binding +function inspects the macro-time environment +.meta env +to determine what kind of binding, if any, does +.meta symbol +have in that environment, from a Lisp-1 perspective. -The return value is -.meta base -raised to -.metn exponent , -and reduced to the -least positive residue modulo -.metn modulus . +That is to say, it considers function bindings, variable bindings +and symbol macro bindings to be in a single name space and finds +the innermost binding of one of these types for +.metn symbol . -.coNP Function @ cum-norm-dist +If such a binding is found, then the function returns one of +the three keyword symbols +.codn :var , +.codn :fun , +or +.codn :symacro . + +If no such lexical binding is found, then the function +returns +.codn nil . + +Note that a +.code nil +return doesn't mean that the symbol doesn't have a lexical binding. It could +have an operator macro lexical binding (a macro binding in the function +namespace established by +.codn macrolet ). + +.coNP Operator @ defsymacro .synb -.mets (cum-norm-dist << argument ) +.mets (defsymacro < sym << form ) .syne .desc -The -.code cum-norm-dist -function calculates an approximation to the cumulative normal -distribution function: the integral, of the normal distribution function, from -negative infinity to the -.metn argument . -.coNP Functions @ n-choose-k and @ n-perm-k +A +.code defsymacro +form introduces a symbol macro: a symbol macro is a parameterless +substitution keyed to a symbol. In contexts where a symbol macro +definition of +.meta sym +is visible, if the form +.meta sym +appears such that its evaluation is called for, it is subject +to replacement by +.metn form . +After replacement takes place, +.meta form +itself is then processed for further replacement of macros and +symbol macros. + +Symbol macros are also recognized in contexts +where +.meta sym +denotes a place which is the target of an assignment operation +like +.code set +and similar. + +A +.code defsymacro +form is implicitly executed at expansion time, and thus need +not be wrapped in a +.code macro-time +form, just like +.codn defmacro . + +Note: if a symbol macro expands to itself directly, expansion stops. However, +if a symbol macro expands to itself through a chain of expansions, +an infinite expansion time loop results. + +.coNP Operator @ symacrolet .synb -.mets (n-choose-k < n << k ) -.mets (n-perm-k < n << k ) +.mets (symacrolet >> ({( sym << form )}*) << body-form *) .syne .desc The -.code n-choose-k -function computes the binomial coefficient nCk which -expresses the number of combinations of -.meta k -items that can be chosen from -a set of -.metn n , -where combinations are subsets. +.code symacrolet +operator binds local, lexically scoped macros that are +similar to the global symbol macros introduced by +.codn defsymacro . -The -.code n-perm-k -function computes nPk: the number of permutations of size -.meta k -that can be drawn from a set of -.metn n , -where permutations are sequences, -whose order is significant. +Each +.meta sym +in the bindings list is bound to its corresponding form, creating a +new extension of the expansion-time lexical macro environment. -The calculations only make sense when -.meta n -and -.meta k -are nonnegative integers, and -.meta k -does not exceed -.metn n . -The behavior is not specified if these conditions -are not met. +Each +.meta body-form +is subsequently macro-expanded in this new environment +in which the new symbol macros are visible. -.coNP Functions @, fixnump @, bignump @, integerp @ floatp and @ numberp +Note: ordinary lexical bindings such as those introduced by let or by +function parameters lists shadow symbol macros. If a symbol +.code x +is bound by nested instances of +.code macrolet +and a +.codn let , +then the scope enclosed by both +constructs will see whichever of the two bindings is more inner, +even though the bindings are active in completely separate phases of +processing. + +From the perspective of the arguments of a +.code dwim +form, lexical function bindings also shadow symbol macros. +This is consistent with the Lisp-1-style name resolution which +applies inside a +.code dwim +form. Of course, lexical operator macros do not shadow +symbol macros under any circumstances. + +.coNP Macros @ placelet and @ placelet* .synb -.mets (fixnump << object ) -.mets (bignump << object ) -.mets (integerp << object ) -.mets (floatp << object ) -.mets (numberp << object ) -.syne -.desc -These functions test the type of -.metn object , -returning -.code t -if it is an object -of the implied type, -.code nil -otherwise. The -.codn fixnump , -.code bignump -and -.code floatp -functions return -.code t -if the object is of the basic type -.codn fixnum , -.code bignum -or -.codn float . -The function -.code integerp -returns true of -.meta object -is either a -.code fixnum -or -a -.codn bignum . -The function -.code numberp -returns -.code t -if -.meta object -is either -a -.codn fixnum , -.code bignum -or -.codn float . - -.coNP Function @ zerop -.synb -.mets (zerop << number ) +.mets (placelet >> ({( sym << place )}*) << body-form *) +.mets (placelet* >> ({( sym << place )}*) << body-form *) .syne .desc The -.code zerop -function tests -.meta number -for equivalence to zero. The argument must be -a number or character. It returns -.code t -for the integer value -.code 0 -and for the floating-point -value -.codn 0.0 . -For other numbers, it returns -.codn nil . -It returns -.code t -for the null character -.code #\enul -and -.code nil -for all other characters. +.code placelet +macro binds lexically scoped symbol macros in such +a way that they behave as aliases for places +denoted by place forms. -.coNP Functions @ plusp and @ minusp -.synb -.mets (plusp << number ) -.mets (minusp << number ) -.syne -.desc -These functions test whether a number is positive or negative, -returning -.code t -or -.codn nil , -as the case may be. +Each +.meta place +must be an expression denoting a syntactic place. The +corresponding +.meta sym +is established as an alias for the storage location which that place denotes, +over the scope of the +.metn body-form -s. -The argument may also be a character. All characters other than -the null character -.code #\enul -are positive. No character is negative. +This binding takes place in such a way that each +.meta place +is evaluated exactly once, only in order to determine its +storage location. The corresponding +.meta sym +then serves as an alias for that location, over the +scope of the +.metn body-form -s. +This means that whenever +.meta sym +is evaluated, it stands for the value of the storage +location, and whenever a value is apparently stored into +.metn sym , +it is actually the storage location which receives it. -.coNP Functions @ evenp and @ oddp -.synb -.mets (evenp << integer ) -.mets (oddp << integer ) -.syne -.desc The -.code evenp -and -.code oddp -functions require integer arguments. -.code evenp -returns -.code t -if -.meta integer -is even (divisible by two), otherwise it returns -.codn nil . -.code oddp -returns -.code t -if -.meta integer -is not divisible by two (odd), otherwise -it returns -.codn nil . +.code placelet* +variant implements an alternative scoping rule, which allows a later +.meta place +form to refer to a +.meta sym +bound to an earlier +.meta place +form. In other words, a given +.meta sym +binding is visible not only to the +.metn body-form -s +but also to +.meta place +forms which occur later. -.coNP Functions @, succ @, ssucc @, sssucc @, pred @, ppred @ and pppred +Note: certain kinds of places, notably +.cblk +.meti (force << promise ) +.cble +expressions, must be accessed before they can be stored, +and this restriction continues to hold when those +places are accessed through +.code placelet +aliases. + +Note: +.code placelet +differs from +.code symacrolet +in that the forms themselves are not aliased, but the storage +locations which they denote. +.code (symacrolet ((x y)) z) +performs the syntactic substitution of symbol +.code x +by form +.codn y , +wherever +.code x +appears inside +.code z +as an evaluated form, and is not shadowed by any inner binding. +Whereas +.code (placelet ((x y)) z) +generates code which arranges for +.code y +to be evaluated to a storage location, and syntactically replaces occurrences +of +.code x +with a form which directly denotes that storage location, +wherever +.code x +appears inside +.code z +as an evaluated form, and is not shadowed by any inner binding. +Also, +.code x +is not necessarily substituted by a single, fixed form, +as in the case of +.codn symacrolet . +Rather it may be substituted by one kind of form when it +is treated as a pure value, and another kind of form +when it is treated as a place. + +.TP* "Example:" + +Implementation of +.code inc +using +.codn placelet : + +.cblk + (defmacro inc (place : (delta 1)) + (with-gensyms (p) + ^(placelet ((,p ,place)) + (set ,p (+ ,p ,delta))))) +.cble + +The gensym +.code p +is used to avoid accidental capture of references +emanating from the +.code delta +form. + +.coNP Operators @ tree-bind and @ mac-param-bind .synb -.mets (succ << number ) -.mets (ssucc << number ) -.mets (sssucc << number ) -.mets (pred << number ) -.mets (ppred << number ) -.mets (pppred << number ) +.mets (tree-bind < macro-style-params < expr << form *) +.mets (mac-param-bind < context-expr < macro-style-params < expr << form *) .syne .desc The -.code succ -function adds 1 to its argument and returns the resulting value. -If the argument is an integer, then the return value is the successor -of that integer, and if it is a character, then the return value -is the successor of that character according to Unicode. +.code tree-bind +operator evaluates +.codn expr , +and then uses the +resulting value as a counterpart to a macro-style parameter list. +If the value has a tree structure which matches the parameters, +then those parameters are established as bindings, and the +.metn form s, +if any, are evaluated in the scope of those bindings. The value +of the last +.meta form +is returned. If there are no forms, +.code nil +is returned. -The -.code pred -function subtracts 1 from its argument, and under similar considerations -as above, the result represents the predecessor. +Note: this operator throws an exception if there is a +structural mismatch between the parameters and the value of +.codn expr . + +One way to avoid this exception is to use +.codn tree-case . The -.code ssucc -and -.code sssucc -functions add 2 and 3, respectively. Similarly, -.code ppred -and -.code pppred -subtract 2 and 3 from their argument. +.code mac-param-bind +operator is similar to +.code tree-bind +except that it takes an extra argument, +.metn context-expr. +This argument is an expression which is evaluated. It is expected to +evaluate to a compound form. If an error occurs during binding, the error +diagnostic message is based on information obtained from this form. +By contrast, the +.code tree-bind +operator's error diagnostic refers to the +.code tree-bind +form, which is cryptic if the binding is used for the implementation +of some other construct, hidden from the user of that construct. -.coNP Functions @, > @, < @, >= @ <= and @ = +.coNP Operator @ tree-case .synb -.mets (> < number << number *) -.mets (> < number << number *) -.mets (>= < number << number *) -.mets (<= < number << number *) -.mets (= < number << number *) +.mets (tree-case < expr >> {( macro-style-params << form *)}*) .syne .desc -These relational functions compare characters and numbers for numeric equality -or inequality. The arguments must be one or more numbers or characters. +The +.code tree-case +operator evaluates +.meta expr +and matches it against a succession +of zero or more cases. Each case defines a pattern match, expressed as a macro +style parameter list +.metn macro-style-params . -If just one argument is given, then these functions all return -.codn t . +If the object produced by +.meta expr +matches +.metn macro-style-params , +then the parameters are bound, becoming local variables, and the +.metn form s, +if any, are evaluated in order in the environment in which those variables are +visible. If there are forms, the value of the last +.meta form +becomes the result +value of the case, otherwise the result value of the case is nil. -If two arguments are given then, they are compared as follows. -First, if the numbers do not have the same type, then the one -which has the lower ranking type is converted to the type of -the other, according to this ranking: character < integer < float. -For instance if a character and integer are compared, the character -is converted to integer. Then a straightforward numeric comparison -is applied. +If the result value of a case is the object +.code : +(the colon symbol), then processing continues with the next case. Otherwise the +evaluation of +.code tree-case +terminates, returning the result value. + +If the value of +.meta expr +does not match the +.meta macro-style-params +parameter list of a case, processing continues with the next case. + +If no cases match, then +.code tree-case +terminates, returning +.codn nil . + +.TP* Example: -Three or more arguments may be given, in which case the comparison proceeds -pairwise from left to right. For instance in -.codn (< a b c) , -the comparison -.code (< a b) -is performed in isolation. If the comparison is false, then -.code nil -is returned, otherwise -the comparison -.code (< b c) -is performed in isolation, and if that is false, -.code nil -is returned, otherwise -.code t -is returned. Note that it is possible for -.code b -to -undergo two different conversions. For instance in the .cblk -.meti (< < float < character << integer ) + ;; reverse function implemented using tree-case + + (defun tb-reverse (obj) + (tree-case obj + (() ()) ;; the empty list is just returned + ((a) obj) ;; one-element list returned (unnecessary case) + ((a . b) ^(,*(tb-reverse b) ,a)) ;; car/cdr recursion + (a a))) ;; atom is just returned .cble -comparison, -.meta character -will first convert to a floating-point representation -of its Unicode value so that it can be compared to -.metn float , -and if that comparison succeeds, then in the second comparison, -.meta character -will be converted to integer so that it can be compared to -.metn integer . -.coNP Function @ /= +Note that in this example, the atom case is placed last, because an +argument list which consists of a symbol is a "catch all" match +that matches any object. We know that it matches an atom, because +the previous +.code (a . b) +case matches conses. In general, the order of the cases in +.code tree-case +is important: even more so than the order of cases in a +.code cond +or +.codn caseql . + +.coNP Macro @ tb .synb -.mets (/= << number *) +.mets (tb < macro-style-params << form *) .syne .desc -The arguments to -.code /= -may be numbers or characters. The -.code /= -function returns -.code t -if no two of its arguments are numerically equal. That is to say, if there -exist some -.code a -and -.code b -which are distinct arguments such that -.code (= a b) -is true, then -the function returns +The +.code tb +macro is similar to the +.code lambda +operator but its argument binding is based on a macro-style parameter list. +The name is an abbreviation of +.codn tree-bind . + +A +.code tb +form evaluates to a function which takes a variable number of +arguments. + +When that function is called, those arguments are taken as a list object which +is matched against +.meta macro-style-params +as if by +.metn tree-bind . +If the match is successful, then the parameters are bound to the +corresponding elements from the argument structure and each successive +.meta form +is evaluated an environment in which those bindings are visible. +The value of the last +.meta form +is the return value of the function. If there are no forms, +the function's return value is .codn nil . -Otherwise it returns -.codn t . -.coNP Functions @ max and @ min +The following equivalence holds, where +.code args +should be understood to be a globally unique symbol: + +.cblk + (tb pattern body ...) <--> (lambda (. args) + (tree-bind pattern args body ...)) +.cble + +.coNP Macro @ tc .synb -.mets (max < first-arg << arg *) -.mets (min < first-arg << args *) +.mets (tc >> {( macro-style-params << form *)}*) .syne .desc The -.code max -and -.code min -functions determine and return the highest or lowest -value from among their arguments. +.code tc +macro produces an anonymous function whose behavior is closely +based on the +.code tree-case +operator. Its name is an abbreviation of +.codn tree-case . -If only -.meta first-arg -is given, that value is returned. +The anonymous function takes a variable number of arguments. +Its argument list is taken to be the value macro is tested +against the multiple pattern clauses of an implicit +.codn tree-bind . +The return value of the function is that of the implied +.codn tree-bind . -These functions are type generic, since they compare arguments -using the same semantics as the -.code less -function. +The following equivalence holds, where +.code args +should be understood to be a globally unique symbol: -If two or more arguments are given, then -.code (max a b) -is equivalent to -.codn (if (less a b) b a) , -and -.code (min a b) -is equivalent to -.codn (if (less a b) a b) . -If the operands do not -have the same type, then one of them is converted to the type of the other; -however, the original unconverted values are returned. For instance -.code (max 4 3.0) -yields the integer -.codn 4 , -not -.codn 4.0 . +.cblk + (tc clause1 clause2 ...) <--> (lambda (. args) + (tree-bind args + clause1 clause2 ...)) +.cble -If three or more arguments are given, -.code max -and -.code min -reduce the arguments in a left-associative manner. -Thus -.code (max a b c) -means -.codn (max (max a b) c) . -.coNP Functions @, int-str @ flo-str and @ num-str +.SS* Mutation of Syntactic Places +.coNP Macro @ set .synb -.mets (int-str < string <> [ radix ]) -.mets (flo-str << string ) -.mets (num-str << string ) +.mets (set >> { place << new-value }*) .syne .desc -These functions extract numeric values from character string -.metn string . -Leading whitespace in -.metn string , -if any, is skipped. If no digits can be successfully extracted, then -.code nil -is returned. Trailing material which does not contribute to the number is -ignored. - The -.code int-str -function converts a string of digits in the specified -radix to an integer value. If the radix isn't specified, it defaults to 10. -Otherwise it must be an integer in the range 2 to 36. -For radices above 10, letters of the alphabet -are used for digits: -.code A -represent a digit whose value is 10, -.code B -represents 11 and -so forth until -.codn Z . -For values of radix above 36, the returned value is -unspecified. Upper and lower case letters are recognized. -Any character which is not a digit of the specified radix is regarded -as the start of trailing junk at which the extraction of the digits stops. +.code set operator stores the values of expressions in places. It must +be given an even number of arguments. -The -.code flo-str -function converts a floating-point decimal notation to a nearby -floating point value. The material which contributes to the value -is the longest match for optional leading space, followed by a -mantissa which consists of an optional sign followed by a mixture of at least -one digit, and at most one decimal point, optionally followed by an exponent -part denoted by the letter -.code E -or -.codn e , -an optional sign and one or more optional exponent digits. +If there are no arguments, then +.code set +does nothing and returns +.codn nil . -The -.code num-str -function converts a decimal notation to either an integer as if by -a radix 10 application of -.codn int-str , -or to a floating point value as if by -.codn flo-str . -The floating point interpretation is chosen if the possibly empty -initial sequence of digits (following any whitespace and optional sign) is -followed by a period, or by -.code e -or -.codn E . +If there are two arguments, +.meta place +and +.metn new-value , +then +.meta place is evaluated to determine its storage location, +then +.meta new-value +is evaluated to determine the value to be stored there, +and then the value is stored in that location. Finally, +the value is also returned as the result value. -.coNP Functions @ int-flo and @ flo-int +If there are more than two arguments, then +.code +set performs multiple assignments in left to right order. +Effectively, +.code (set v1 e1 v2 e2 ... vn en) +is precisely equivalent to +.codn (progn (set v1 e1) (set v2 e2) ... (set vn en)) . + +.coNP Macro @ pset .synb -.mets (int-flo << float ) -.mets (flo-int << integer ) +.mets (pset >> { place << new-value }*) .syne .desc -These functions perform numeric conversion between integer and floating point -type. The -.code int-flo -function returns an integer by truncating toward zero. -The -.code flo-int -function returns an exact floating point value corresponding to -.metn integer , -if possible, otherwise an approximation using a nearby -floating point value. +The syntax of +.code pset +is similar to that of +.codn set , +and the semantics is similar also in that zero or more places are +assigned zero or more values. In fact, if there are no arguments, or +if there is exactly one pair of arguments, +.code pset +is equivalent to +.codn set . -.coNP Functions @ tofloat and @ toint +If there are two or more argument pairs, then all of the arguments +are evaluated first, in left-to-right order. No store takes place +until after every +.meta place +is determined, and every +.meta new-value +is calculated. During the calculation, the values to be stored +are retained in hidden, temporary locations. Finally, these values +are moved into the determined places. The rightmost value is returned +as the form's value. + +The assignments thus appear to take place in parallel, and +.code pset +is capable of exchanging the values of a pair of places, or rotating +the values among three or more places. (However, there are more convenient +operators for this, namely +.code rotate +and +.codn swap ). + +.TP* Example: +.cblk + ;; exchange x and y + (pset x y y x) + + ;; exchange elements 0 and 1; and 2 and 3 of vector v: + (let ((v (vec 0 10 20 30)) + (i -1)) + (pset [vec (inc i)] [vec (inc i)] + [vec (inc i)] [vec (inc i)]) + vec) + -> #(10 0 30 20) +.cble + +.coNP Macro @ zap .synb -.mets (tofloat << value ) -.mets (toint < value <> [ radix ]) +.mets (zap < place <> [ new-value ]) .syne .desc -These convenience functions convert -.meta value -to floating-point or integer, respectively. - -If a floating-point value is passed into tofloat, or an integer value into -toint, then the value is simply returned. +The +.code zap +macro assigns +.meta new-value +to +.meta place +and returns the previous value of +.metn place . If -.meta value -is a character, then it is treated as a string of length one -containing that character. +.meta new-value +is missing, then +.code nil +is used. -If -.meta value -is a string, then it is converted by -.code tofloat -as if by the function -.metn flo-str , -, and by -.code toint -as if by the function -.codn int-str . +In more detail, first +.code place +is evaluated to determine the storage location. +Then, the location is accessed to retrieve the +previous value. Then, the +.code new-value +expression is evaluated, and that value is +placed into the storage location. +Finally, the previously retrieved value is returned. -If -.meta value -is an integer, then it is converted by -.code tofloat -as if by the function -.codn flo-int . + +.coNP Macro @ flip +.synb +.mets (flip << place ) +.syne +.desc +The +.code flip +macro toggles the boolean value stored in +.metn place . If -.meta value -is a floating-point number, then it is converted by -.code toint -as if by the function -.codn int-flo . +.meta place +previously held +.codn nil , +it is set to +.codn t , +and if it previously held a value other than +.codn nil , +it is set to +.codn nil . -.coNP Variables @, *flo-min* @, *flo-max* and @ *flo-epsilon* +.coNP Macros @ inc and @ dec +.synb +.mets (inc < place <> [ delta ]) +.mets (dec < place <> [ delta ]) +.syne .desc -These variables hold, respectively: the smallest positive floating-point -value; the largest positive floating-point value; and the difference -between 1.0 and the smallest representable value greater than 1.0. +The +.code inc +macro increments +.meta place +by adding +.meta delta +to its value. +If +.meta delta +is missing, the value used in its place the integer 1. -.code *flo-min* -and -.code *flo-max* -define the floating-point range, which consists -of three regions: values from -.code (- *flo-max*) -to -.codn (- *flo-min*) ; -the value 0.0, and values from -.code *flo-min* -to -.codn *flo-max* . +First the +.meta place +argument is evaluated as a syntactic place to determine the location. +Then, the value currently stored in that location is retrieved. +Next, the +.meta delta +expression is evaluated. Its value is added to the previously retrieved +value as if by the +.code + +function. The resulting value is stored in the place, and returned. -.coNP Variable @ *flo-dig* -.desc -This variable holds an integer representing the number of decimal digits -in a decimal floating-point number such that this number can be converted -to a \*(TX floating-point number, and back to decimal, without a change in any of -the digits. This holds regardless of the value of the number, provided that it -does not exceed the floating-point range. +The macro +.code dec +works exactly like +.code inc +except that addition is replaced by subtraction. The similarly defaulted +.meta delta +value is subtracted from the previous value of the place. -.coNP Variables @ *pi* and @ *e* +.coNP Macro @ swap +.synb +.mets (swap < left-place << right-place ) +.syne .desc -These variables hold an approximation of the mathematical constants \(*p and e. -To four digits of precision, \(*p is 3.142 and e is 2.718. The -.code *pi* +The +.code swap +macro exchanges the values of +.meta left-place and -.code *e* -approximations are accurate to -.code *flo-dig* -decimal digits. +.meta right-place +and returns the value which is thereby transferred to +.metn right-place . -.SS* Bit Operations -In \*(TL, similarly to Common Lisp, bit operations on integers are based -on a concept that might be called "infinite two's-complement". -Under infinite two's complement, a positive number is regarded as having -a binary representation prefixed by an infinite stream of zero digits (for -example -.code 1 -is -.codn ...00001 ). -A negative number -in infinite two's complement is the bitwise negation of its positive counterpart, -plus one: it carries an infinite prefix of 1 digits. So for instance the number -.code -1 -is represented by -.codn ...11111111 : -an infinite sequence of -1 -bits. There -is no specific sign bit; any operation which produces such an infinite sequence -of 1 digits on the left gives rise to a negative number. For instance, consider the -operation of computing the bitwise complement of the number -.codn 1 . -Since the -number -.code 1 -is represented as -.codn ...0000001 , -its complement is -.codn ...11111110 . -Each one of the -.cod 0 -digits in the infinite sequence is replaced by -.codn 1 , -And this leading sequence means that the number -is negative, in fact corresponding to the two's-complement representation of -the value -.codn -2 . -Hence, the infinite digit concept corresponds to an arithmetic -interpretation. - -In fact \*(TL's bignum integers do not use a two's complement -representation internally. Numbers are represented as an array which holds a -pure binary number. A separate field indicates the sign: negative, -or non-negative. That negative numbers appear as two's-complement under the -bit operations is merely a carefully maintained illusion (which makes bit -operations on negative numbers more expensive). - -The -.code logtrunc -function, as well as a feature of the -.code lognot -function, allow bit -manipulation code to be written which works with positive numbers only, even if -complements are required. The trade off is that the application has to manage a -limit on the number of bits. +First, +.meta left-place +and +.meta right-place +are evaluated, in that order, to determine their locations. +Then the prior values are retrieved, exchanged and stored back. +The value stored in +.meta right-place +is also returned. -.coNP Functions @, logand @, logior and @ logxor +.coNP Macro @ push .synb -.mets (logand << integer *) -.mets (logior << integer *) -.mets (logxor < int1 << int2 ) +.mets (push < item << place ) .syne .desc -These operations perform the familiar bitwise and, inclusive or, and exclusive -or operations, respectively. Positive values inputs are treated as -pure binary numbers. Negative inputs are treated as infinite-bit -two's-complement. +The +.code push +macro places +.meta item +at the head of the list stored in +.meta place +and returns the updated list which is stored back in +.metn place . -For example -.code (logand -2 7) -produces -.codn 6 . -This is because -.code -2 -is -.code ...111110 -in infinite-bit two's-complement. And-ing this value with -.code 7 -(or -.codn ...000111 ) -produces -.codn 110 . +First, the expression +.meta item +is evaluated to produce the push value. +Then, +.meta place +is evaluated to determine its storage location. +Next, the storage location is accessed to retrieve the +list value which is stored there. A new object is +produced as if by invoking +.code cons +function on the push value and list value. +This object is stored into the location, +and returned. +.coNP Macro @ pop +.synb +.mets (pop << place ) +.syne The -.code logand -and -.code logior -functions are variadic, and may be called with zero, one, -two, or more input values. If -.code logand -is called with no arguments, it produces -the value -1 (all bits 1). If -.code logior -is called with no arguments it produces -zero. In the one-argument case, the functions just return their argument value. +.code pop +macro removes an element from the list stored in +.meta place +and returns it. -.coNP Function @ logtest +First, +.meta place +is evaluated to determine the place. The place is accessed to +retrieve the original value. Then a new value is calculated, +as if by applying the +.code cdr +function to the old value. This new value is stored. +Finally, a return value is calculated and returned, as if by applying the +.code car +function to the original value. + +.coNP Macro @ pushnew .synb -.mets (logtest < int1 << int2 ) +.mets (pushnew < item < place >> [ testfun <> [ keyfun ]]) .syne .desc The -.code logtest -function returns true if -.meta int1 -and -.meta int2 -have bits in -common. The following equivalence holds: +.code pushnew +macro inspects the list stored in +.metn place . +If the list already contains the item, then +it returns the list. Otherwise it creates a new list +with the item at the front and stores it back +into +.metn place , +and returns it. -.cblk - (logtest a b) <--> (not (zerop (logand a b))) -.cble +First, the expression +.meta item +is evaluated to produce the push value. +Then, +.meta place +is evaluated to determine its storage location. +Next, the storage location is accessed to retrieve the +list value which is stored there. The list is +inspected to check whether it already contains the push +value, as if using the +.code member +function. If that is the case, the list +is returned and the operation finishes. +Otherwise, a new object is +produced as if by invoking +.code cons +function on the push value and list value. +This object is stored into the location +and returned. -.coNP Functions @ lognot and @ logtrunc +.coNP Macro @ shift .synb -.mets (lognot < value <> [ bits ]) -.mets (logtrunc < value << bits ) +.mets (shift << place + << shift-in-value) .syne .desc The -.code lognot -function performs a bitwise complement of -.metn value . -When the one-argument form of lognot is used, then if -.meta value -is nonnegative, -then the result is negative, and vice versa, according to the infinite-bit -two's complement representation. For instance -.code (lognot -2) -is -.codn 1 , -and -.code (lognot 1) -is -.codn -2 . +.code shift +macro treats one or more places as a "multi-place shift register". +The values of the places are shifted one place to the left. +The first (leftmost) place receives the value of the second place, +the second receives that of the third, and so on. +The last (rightmost) place receives +.meta shift-in-value +(which is not treated as a place, even if it is a syntactic place form). +The previous value of the first place is returned. -The two-argument form of -.code lognot -produces a truncated complement. Conceptually, -a bitwise complement is first calculated, and then the resulting number is -truncated to the number of bits given by -.metn bits , -which must be a nonnegative integer. The following equivalence holds: +More precisely, all of the argument forms are evaluated left to right, in the +process of which the storage locations of the places are determined, +.meta shift-in-value +is reduced to its value. -.cblk - (lognot a b) <--> (logtrunc (lognot a) b) -.cble +The values stored in the places are sampled and saved. -The -.code logtrunc -function truncates the integer -.meta value -to the specified number -of bits. If -.meta value -is negative, then the two's-complement representation -is truncated. The return value of -.code logtrunc -is always a non-negative integer. +Note that it is not specified whether the places are sampled in a separate +pass after the evaluation of the argument forms, or whether the +sampling is interleaved into the argument evaluation. This affects +the behavior in situations in which the evaluation of any of the +.meta place +forms, or of +.metn shift-in-value , +has the side effect of modifying later places. -.coNP Function @ sign-extend +Next, the places are updated by storing the saved value of the second +place into the first place, the third place into the second and so forth, +and the value of +.meta shift-in-value +into the last place. + +Finally, the saved original value of the first place is returned. + +.coNP Macro @ rotate .synb -.mets (sign-extend < value << bits ) +.mets (rotate << place *) .syne .desc -The -.code sign-extend -function first truncates the infinite-bit two's complement representation of -the integer -.meta value -to the specified number of bits, similarly to the -.code logtrunc -function. Then, this truncated value is regarded as a -.meta bits -wide two's complement integer. The value of this integer is -calculated and returned. - -.TP* Examples: - -.cblk - (sign-extend 127 8) -> 127 - (sign-extend 128 8) -> -128 - (sign-extend 129 8) -> -127 - (sign-extend 255 8) -> -1 - (sign-extend 256 8) -> 0 - (sign-extend -1 8) -> -1 - (sign-extend -255 8) -> 0 -.cble - -.coNP Function @ ash -.synb -.mets (ash < value << bits ) -.syne -.desc -The -.code ash -function shifts -.meta value -by the specified number of -.meta bits -producing a -new value. If -.meta bits -is positive, then a left shift takes place. If -.meta bits -is negative, then a right shift takes place. If -.meta bit -is zero, then -.meta value -is returned unaltered. For positive numbers, a left shift by n bits is -equivalent to a multiplication by two to the power of n, or -.codn (expt 2 n) . -A right shift by n bits of a positive integer is equivalent to integer -division by -.codn (expt 2 n) , -with truncation toward zero. -For negative numbers, the bit shift is performed as if on the two's-complement -representation. Under the infinite two's-complement representation, -a right shift does not exhaust the infinite sequence of -.code 1 -digits which -extends to the left. Thus if -.code -4 -is shifted right it becomes -.code -2 -because -the bitwise representations of these values are -.code ...111100 -and -.codn ...11110 . - -.coNP Function @ bit -.synb -.mets (bit < value << bit ) -.syne -.desc -The -.code bit -function tests whether the integer -.meta value -has a 1 in bit position -.metn bit . -The -.meta bit -argument must be a non-negative integer. A value of zero of -.meta bit -indicates the least significant bit position of -.metn value . +Treats zero or more places as a "multi-place rotate register". +If there are no arguments, there is no effect and +.code nil +is returned. Otherwise, the last (rightmost) place receives +the value of the first (leftmost) place. The leftmost place +receives the value of the second place, and so on. +If there are two arguments, this equivalent to +.codn swap . +The prior value of the first place, which is the the value +rotated into the last place, is returned. -The -.code bit -function has a boolean result, returning the symbol -.code t -if bit -.meta bit -of -.meta value -is set, otherwise -.codn nil . +More precisely, the +.meta place +arguments are evaluated left to right, +and the storage locations are thereby determined. The storage +locations are sampled, and then the sampled values are +stored back into the locations, but rotated by one place +as described above. The saved original value of the leftmost +.meta place +is returned. -If -.meta value -is negative, it is treated as if it had an infinite-bit two's -complement representation. For instance, if value is -.codn -2 , -then the bit -function returns -.code nil -for a -.meta bit -value of zero, and -.code t -for all other values, -since the infinite bit two's complement representation of -.code -2 -is -.codn ...11110 . +It is not specified whether the sampling of the original values +is a separate pass which takes place after the arguments +are evaluated, or whether this sampling it is interleaved into argument +evaluation. This affects +the behavior in situations in which the evaluation of any of the +.meta place +forms has the side effect of modifying the value stored in +a later +.meta place +form. -.coNP Function @ mask +.coNP Macro @ del .synb -.mets (mask << integer *) +.mets (del << place ) .syne .desc The -.code mask -function takes zero or more integer arguments, and produces an integer -value which corresponds a bitmask made up of the bit positions specified by the -integer values. - +.code del +macro requests the deletion of +.codn place . If -.code mask -is called with no arguments, then the return value is zero. +.code place +doesn't support deletion, an exception is thrown. -If -.code mask -is called with a single argument -.meta integer -then the return value is the same as -that of the expression -.codn (ash 1 ) : -the value 1 shifted left by -.meta integer -bit positions. If -.meta integer -is zero, then the result is -.codn 1 ; -if -.meta integer -is -.codn 1 , -the -result is -.code 2 -and so forth. If -.meta value -is negative, then the result is zero. +First +.code place +is evaluated, thereby determining its location. +Then the place is accessed to retrieve its value. +The place is then subject to deletion. Finally, the +previously retrieved value is returned. -If -.code mask -is called with two or more arguments, then the result is a bitwise of -the masks individually computed for each of the values. +Precisely what deletion means depends on the kind of place. +The built-in places in \*(TL have deletion semantics which are +intended to be unsurprising to the programmer familiar with the +data structure which holds the place. -In other words, the following equivalences hold: +Generally, if a place denotes the element of a sequence, then deletion of the +place implies deletion of the element, and deletion of the element implies that +the gap produced by the element is closed. The deleted element is effectively +replaced by its successor, that successor by its successor and so on. If a +place denotes a value stored in a dynamic data set such as a hash table, +then deletion of that place implies deletion of the entry which holds +that value. If the entry is identified by a key, that key is also removed. -.cblk - (mask) <--> 0 - (mask a) <--> (ash 1 a) - (mask a b c ...) <--> (logior (mask a) (mask b) (mask c) ...) -.cble +.SS* User-Defined Places and Place Operators +\*(TL provides a number of place-modifying operators such as +.codn set , +.codn push , +and +.codn inc . +It also provides a variety of kinds of syntactic places +which may be used with these operators. -.coNP Function @ width -.synb -.mets (width << integer *) -.syne -.desc -A two's complement representation of an integer consists of a sign bit and a -mantissa field. -The -.code width -function computes the minimum number of bits required for the mantissa portion -of the two's complement representation of the -.meta integer -argument. +Both of these categories are open-ended: \*(TL programs may extend +the set of place-modifying operators, as well as the vocabulary of +forms which are recognized as syntactic places. -For a nonnegative argument, the width also corresponds to the number of bits -required for a natural binary representation of that value. +Regarding place operators, it might seem obvious that new place operators can +be developed, since they are macros, and macros can expand to uses +of existing place operators. As an example, it may seem that +.code inc +operator could be written as a macro which uses +.codn set : -Two integer values have a width of zero, namely 0 and -1. This means that these -two values can be represented in a one-bit two's complement, consisting of only -a sign bit: the one-bit two's complement bitfield 1 denotes -1, and 0 denotes -0. +.cblk + (defmacro new-inc (place : (delta 1)) + ^(set ,place (+ ,place ,delta))) +.cble -Similarly, two integer values have a width of 1: 1 and -2. The two-bit -two's complement bitfield 01 denotes 1, and 10 denotes -2. +However, the above +.code new-inc +macro has a problem: the +.code place +argument form is inserted into two places in the expansion, which +leads to two evaluations. This is visibly incorrect if the place +form contains any side effects. It is also potentially inefficient. -The argument may be a character. +\*(TL provides a framework for writing place update macros which +evaluate their argument forms once, even if they have to access +and update the same places. +The framework also supports the development of new kinds of place forms +as capsules of code which introduce the right kind of material into +the lexical environment of the body of an update macro, to enable +this special evaluation. -.SS* Exceptions -.coNP Functions @, throw @ throwf and @ error -.synb -.mets (throw < symbol << arg *) -.mets (throwf < symbol < format-string << format-arg *) -.mets (error < format-string << format-arg *) -.syne -.desc -These functions generate an exception. The -.code throw -and -.code throwf -functions generate -an exception identified by -.metn symbol , -whereas -.code error -throws an exception of -type -.codn error . -The call -.code (error ...) -can be regarded as a shorthand for -.codn (throwf 'error ...) . +.NP* Place-Expander Functions +The central design concept in \*(TL syntactic places are +.IR "place-expander functions" . +Each compound place is defined by up to three place-expander functions, +which are associated with the place via the leftmost operator +symbol of the place form. One place-expander, the +.IR "update expander" , +is mandatory. Optionally, a place may also provide a +.I "clobber expander" +as well as a +.IR "delete expander" . +An update expander provides the expertise for evaluating a place form once +in its proper run-time context to determine its actual run-time storage +location, and to access and modify the storage location. +A clobber expander provides an optimized mechanism for uses that perform +a one-time store to a place without requiring its prior value. +If a place definition does not supply a clobber expander, then the syntactic +places framework uses the update expander to achieve the functionality. +A delete expander provides the expertise for determining the actual run-time +storage location corresponding to a place, and obliterating it, +returning its prior value. If a place does not supply a delete expander, then +the place does not support deletion. Operators which require deletion, such as +.code del +will raise an error when applied to that place. -The -.code throw -function takes zero or more additional arguments. These arguments -become the arguments of a -.code catch -handler which takes the exception. The -handler will have to be capable of accepting that number of arguments. +The expanders operate independently, and it is expected that place-modifying +operators choose one of the three, and use only that expander. For example, +accessing a place with an update expander and then overwriting its value +with a clobber expander may result in incorrect code which contains multiple +evaluations of the place form. -The -.code throwf +The programmer who implements a new place does not write expanders directly, +but rather defines them via the +.code defplace +macro. + +The programmer who implements a new place update macro likewise does not +call the expanders directly. Usually, they are invoked via the macros +.codn with-update-expander , +.codn with-clobber-expander and -.code error -functions generate an exception which has a single -argument: a character string created by a formatted print to a string stream -using the -.code format -string and additional arguments. +.codn with-delete-expander . +These are sufficient for most kind of macros. +In certain complicated cases, expanders may be invoked using the wrapper +functions +.codn call-update-expander , +.codn call-clobber-expander +and +.codn call-delete-expander . +These convenience macros and functions perform certain common chores, like +macro-expanding the place in the correct environment, and choosing the +appropriate function. -.coNP Operator @ catch +The expanders are described in the following sections. + +.NP* The Update Expander .synb -.mets (catch < try-expression -.mets \ \ >> {( symbol <> ( arg *) << body-form *)}*) +.mets (lambda >> ( getter-sym < setter-sym < place-form +.mets \ \ \ \ \ \ \ \ << body-form ) ...) .syne .desc -The -.code catch -operator establishes an exception catching block around -the -.metan try-expression . -The -.meta try-expression -is followed by zero or more -catch clauses. Each catch clause consists of a symbol which denotes -an exception type, an argument list, and zero or more body forms. - -If -.meta try-expression -terminates normally, then the catch clauses -are ignored. The catch itself terminates, and its return value is -that of the -.metn try-expression . - -If -.meta try-expression -throws an exception which is a subtype of one or more of -the type symbols given in the exception clauses, then the first (leftmost) such -clause becomes the exit point where the exception is handled. -The exception is converted into arguments for the clause, and the clause -body is executed. When the clause body terminates, the catch terminates, -and the return value of the catch is that of the clause body. - -If -.meta try-expression -throws an exception which is not a subtype of any of -the symbols given in the clauses, then the search for an exit point for -the exception continues through the enclosing forms. The catch clauses -are not involved in the handling of that exception. - -When a clause catches an exception, the number of arguments in the catch must -match the number of elements in the exception. A catch argument list -resembles a function or lambda argument list, and may be dotted. For instance -the clause -.code (foo (a . b)) -catches an exception subtyped from -.codn foo , -with one or -more elements. The first element binds to parameter -.codn a , -and the rest, if any, -bind to parameter -.codn b . -If there is only one element, -.code b -takes on the value -.codn nil . +The update expander is a code-writer. It takes a +.meta body-form +argument, representing code, and returns a larger form which surrounds +this code with additional code. -Also see: the -.code unwind-protect -operator, and the functions -.codn throw , -.code throwf +This larger form returned by the update expander can be regarded as having two +abstract actions, when it is substituted and evaluated in the context where +.meta place-form +occurs. The first abstract action is to evaluate +.meta place-form +exactly one time, in order to determine the actual run-time location to which +that form refers. +The second abstract action is to evaluate the caller's +.metn body-form -s, +in a lexical environment in which bindings exist for some lexical +functions or (more usually) lexical macros. These lexical macros +are explicitly referenced by the +.metn body-form ; +the update expander just provides their definition, under the names +it is given via the +.meta getter-sym and -.codn error . +.meta setter-sym +arguments. -.coNP Macro @ ignerr -.synb -.mets (ignerr << form *) -.syne -.desc -The -.code ignerr -macro operator evaluates each -.meta form -similarly to the -.code progn -operator. If no forms are present, it returns -.codn nil . -Otherwise it evaluates each -.meta form -in turn, yielding the value of the last one. +The update expander writes local functions or macros under these names: a +getter function and a setter function. Usually, update expanders write +macros rather than functions, possibly in combination with some lexical +anonymous variables which hold temporary objects. Therefore the getter +and setter are henceforth referred to as macros. -If the evaluation of any -.meta form -is abandoned due to an exception of type -.codn error , -the code generated by the -.code ignerr -macro catches this exception. In this situation, -the execution of the -.code ignerr -form terminates without evaluating the remaining -forms, and yields -.codn nil . +The code being generated is with regard to some concrete instance of +.metn place-form . +This argument is the actual form which occurs in a program. For +instance, the update expander for the +.code car +place might be called with an arbitrary variant of the +.meta place-form +might look like +.codn (car (inc (third some-list))) . -.coNP Variable @ *unhandled-hook* +In the abstract semantics, upfront code wrapped around the +.meta body-form +by the update expander provides the logic to evaluate this place to +a location, which is retained in some hidden local context. +The getter local macro named by +.meta getter-sym +must provide the logic for retrieving the value of this place. +The getter macro takes no arguments. The -.code *unhandled-hook* -variable is initialized with -.code nil -by default. +.meta body-form +makes free use of the getter function; they may call it multiple times, +which must not trigger multiple evaluations of the original place +form. -It may instead be assigned a function which is capable of taking -three arguments. +The setter local macro named by +.meta setter-sym +must generate the logic for storing a new value into the once-evaluated +version of +.metn place-form . +The setter function takes exactly one argument, whose +value specifies the value to be stored into the place. +It is the caller's responsibility to ensure that the +argument form which produces the value to be stored via the setter is evaluated +only once, and in the correct order. The setter does not concern itself with +this form. Multiple calls to the setter can be expected to result in multiple +evaluations of its argument. Thus, if necessary, the caller must supply the code +to evaluate the new value form to a temporary variable, and then pass the +temporary variable to the setter. This code can be embedded in +the +.meta body-form +or can be added to the code returned by a call to the update expander. -When an exception occurs which has no handler, this function is called, -with the following arguments: the exception type symbol, the exception object, -and a third value which is either -.code nil -or else the form which was being evaluated the exception was thrown. +The setter local macro or function must return the new value which is stored. +That is to say, when +.meta body-form +invokes this local macro or function, it may rely on it yielding the +new value which was stored, as part of achieving its own semantics. -Otherwise, if the variable is -.code nil -some informational messages are printed about the exception, and the process -exits with a failed termination status. -In the same situation, if the variable contains an object which is not a -function, the process terminates abnormally as if by a call to the -.code abort -function. - -Prior to the function being called, the -.code *unhandled-hook* -variable is reset to -.codn nil . - -If the function registered in -.code *unhandled-hook* -returns, the process exits with a failed termination status. - -Note: the functions -.code source-loc -or -.code source-loc-str -may be applied to the third argument of the -.code *unhandled-hook* -function to obtain more information about the form. - -.SS* Regular Expression Library -.coNP Functions @ search-regex and @ range-regex -.synb -.mets (search-regex < string < regex >> [ start <> [ from-end ]]) -.mets (range-regex < string < regex >> [ start <> [ from-end ]]) -.mets (range-regst < string < regex >> [ start <> [ from-end ]]) -.syne -.desc -The -.code search-regex -function searches through -.meta string -starting -at position -.meta start -for a match for -.metn regex . -If -.meta start -is omitted, the search starts at position 0. If -.meta from-end -is specified and has a -.cod2 non- nil -value, the search -proceeds in reverse, from the last position in the string, toward -.metn start . -This function returns -.code nil -if no match is found, otherwise it returns -a cons, whose -.code car -indicates the position of the match, and whose -.code cdr -indicates the length of the match. - -The -.code range-regex -function is similar to -.codn search-regex , -except that when -a match is found, it returns a position range, rather than a position -and length. A cons is returned whose +The update expander does not macro-expand +.codn place-form . +It is assumed that the expander is invoked in such a way that the +place has been expanded in the correct environment. In other words, the +form matches the type of place which the expander handles. +If the expander had to macro-expand the place form, it would sometimes have +to come to the conclusion that the place form must be handled by a different +expander. No such consideration is the case: when an expander is called on +a form, that is final; it is certain that it is the correct expander, which +matches the symbol in the .code car -indicates the position -of the match, and whose -.code cdr -indicates the position one element past the -last character of the match. If the match is empty, the two integers -are equal. - -The -.code search-regst -differs from -.code search-regex -in the representation of the return value in the matching case. -Rather than returning the position and length of the match, -it returns the matching substring of -.metn string . - -.coNP Functions @ match-regex and @ match-regst -.synb -.mets (match-regex < string < regex <> [ position ]) -.mets (match-regst < string < regex <> [ position ]) -.syne -.desc -The -.code match-regex -function tests whether -.meta regex -matches at -.meta position -in -.metn string . -If -.meta position -is not specified, it is taken to be zero. -If the regex matches, then the length of the match is returned. -If it does not match, then -.code nil -is returned. - -The -.code match-regst -differs from -.code match-regex -in the representation of the return value in the matching case. -Rather than returning the length of the match, it returns -matching substring of -.metn string . - -.coNP Functions @ match-regex-right and @ match-regst-right -.synb -.mets (match-regex-right < string < regex <> [ end-position ]) -.mets (match-regst-right < string < regex <> [ end-position ]) -.syne -.desc -The -.code match-regex -function tests whether -.meta string -contains a match which ends -precisely on the character just before -.metn end-position . -If -.meta end-position -is not specified, it defaults to the length of the string, and the function -performs a right-anchored regex match. - -If a match is found, then the length of the match is returned. - -The match must terminate just before -.meta end-position -in the sense that -additional characters at -.meta end-position -and beyond can no longer satisfy the -regular expression. More formally, the function searches, starting from -position zero, for positions where there occurs a match for the regular -expression, taking the longest possible match. The length of first such a match -which terminates on the character just before -.meta end-position -is returned. -If no such a match is found, then -.code nil -is returned. - -The -.code match-regst-right -differs from -.code match-regst-right -in the representation of the return value in the matching case. -Rather than returning the length of the match, it returns -matching substring of -.metn string . - -.TP* Examples: +position of the form, which is not a macro in the context where it occurs. +An update expander is free to assume that any place which is stored +(the setter local macro is invoked on it) is accessed at least once by +an invocation of the getter. A place update macro which relies on an update +expander, but uses only the store macro, might not work properly. +An example of an update expander which relies on this assumption is the +expander for the .cblk - ;; Return matching portion rather than length thereof. - - (defun match-regex-right-substring (str reg : end-pos) - (set end-pos (or end-pos (length str))) - (let ((len (match-regex-right str reg end-pos))) - (if len - [str (- end-pos len)..end-pos] - nil))) +.meti (force << promise ) +.cble +place type. If +.meta promise +has not yet been forced, and only the setter is used, then +.meta promise +might remain unforced as its internal value location is updated. +A subsequent access to the place will incorrectly trigger a force, +which will overwrite the value. The expected behavior is that storing +a value in an unforced +.code force +place changes the place to forced state, preempting the evaluation of +the delayed form. Afterward, the promise exhibits the value which was +thus assigned. - (match-regex-right-substring "abc" #/c/) -> "" +The update expander is not responsible for all issues of evaluation order. A +place update macro may consist of numerous places, as well as numerous +value-producing forms which are not places. Each of the places can provide its +registered update expander which provides code for evaluating just that place, +and a means of accessing and storing the values. The place update macro must +call the place expanders in the correct order, and generate any additional code +in the correct order, so that the macro achieves its required documented +evaluation order. - (match-regex-right-substring "acc" #/c*/) -> "cc" +.TP* "Example Update Expander Call:" - ;; Regex matches starting at multiple positions, but all - ;; the matches extend past the limit. - (match-regex-right-substring "acc" #/c*/ 2) -> nil +.cblk + ;; First, capture the update expander + ;; function for (car ...) places + ;; in a variable, for clarity. - ;; If the above behavior is not wanted, then - ;; we can extract the string up to the limiting - ;; position and do the match on that. - (match-regex-right-substring ["acc" 0..2] #/c*/) -> "c" + (defvar car-update-expander [*place-update-expander* 'car]) - ;; Equivalent of above call - (match-regex-right-substring "ac" #/c*/) -> "c" -.cble + ;; Next, call it for the place (car [a 0]). + ;; The body form specifies logic for + ;; incrementing the place by one and + ;; returning the new value. -.coNP Function @ regsub -.synb -.mets (regsub < regex < replacement << string ) -.syne -.desc -The -.code regsub -function searches -.meta -string -for multiple occurrences of -non-overlapping matches for -.metn regex . -A new string is constructed -similar to -.meta string -but in which each matching region is replaced -with using -.meta replacement -as follows. + (call car-update-expander 'getit 'setit '(car [a 0]) + '(setit (+ (getit) 1))) -The -.meta replacement -object may be a character or a string, in which -case it is simply taken to be the replacement for each match -of the regular expression. + --> ;; Resulting code: -The -.meta replacement -object may be a function of one argument, in -which case for every match which is found, this function is invoked, -with the matching piece of text as an argument. The function's -return value is then taken to be the replacement text. - -.TP* Examples: - -.cblk - ;; match every lower case e or o, and replace by filtering - ;; through the upcase-str function: - - [regsub #/[eo]/ upcase-str "Hello world!"] -> "HEllO wOrld!" - - ;; Replace Hello with Goodbye: - (regsub #/Hello/ "Goodbye" "Hello world!") -> "Goodbye world!" -.cble - -.coNP Function @ regexp -.synb -.mets (regexp << obj ) -.syne -.desc -The -.code regexp -function returns -.code t -if -.meta obj -is a compiled regular expression -object. For any other object type, it returns -.codn nil . - -.coNP Function @ regex-compile -.synb -.mets (regex-compile < form-or-string <> [ error-stream ]) -.syne -.desc -The -.code regex-compile -function takes the source code of a regular expression, -expressed as a Lisp data structure representing an abstract syntax tree, or -else a regular expression specified as a character string, and compiles it to a -regular expression object. - -If -.meta form-or-string -is a character string, it is parsed to an -abstract syntax tree first, if by the -.code regex-parse -function. -If the parse is successful (the result is not -.codn nil ) -then -the resulting tree structure is compiled by a recursive call to -.codn regex-compile . - -The optional -.meta error-stream -argument is passed down to -.code regex-parse -as well as in the recursive call to -.codn regex-compile , -if that call takes place. - -If -.meta error-stream -is specified, it must be a stream. Any error diagnostics are sent to that -stream. - -.TP* Examples: - -.cblk - ;; the equivalent of #/[a-zA-Z0-9_/ - (regex-compile '(set (#\ea . #\ez) (#\eA . #\eZ) - (#\e0 . #\e9) #\e_)) - - ;; the equivalent of #/.*/ and #/.+/ - (regex-compile '(0+ wild)) - (regex-compile '(1+ wild)) - - ;; #/a|b|c/ - (regex-compile '(or (or #\ea #\eb) #\ec)) - - ;; string - (regex-compile "a|b|c") -.cble - -.coNP Function @ regex-parse -.synb -.mets (regex-parse < string <> [ error-stream ]) -.syne -.desc -The -.code regex-parse -function parses a character string which contains a regular expression -(without any surrounding / characters) and turns it into a Lisp data structure -(the abstract syntax tree representation of the regular expression). - -The regular expression syntax -.code #/RE/ -produces the same structure, but as a -literal which is processed at the time \*(TX source code is read; the -.code regex-parse -function performs this parsing at run-time. - -If there are parse errors, the function returns -.codn nil . - -The optional -.meta error-stream -argument specifies a stream to which error messages -are sent from the parser. By default, diagnostic output goes to the -.code *stdnull* -stream, which discards it. If -.meta error-stream -is specified as -.codn t , -then the diagnostic output goes to the -.code *stdout* -stream. - -If -.code regex-parse -returns a -.cod2 non- nil -value, that structure is then something -which is suitable as input to -.codn regex-compile . - -.SS* Hashing Library -.coNP Functions @, make-hash and @ hash -.synb -.mets (make-hash < weak-keys < weak-vals << equal-based ) -.mets (hash { :weak-keys | :weak-vals | :equal-based }*) -.syne -.desc -These functions construct a new hash table. - -A hash table is an object which retains an association between pairs of -objects. Each pair consists of a key and value. Given an object which is -similar to a key in the hash table, it is possible to retrieve the -corresponding value. Entries in a hash table are not ordered in any way, and -lookup is facilitated by hashing: quickly mapping a key object to a numeric -value which is then used to index into one of many buckets where the matching -key will be found (if such a key is present in the hash table). - -.code make-hash -takes three mandatory boolean arguments. The -.meta weak-keys -argument specifies whether the hash table shall have weak keys. The -.meta weak-vals -argument specifies whether it shall have weak values, and -.meta equal-based -specifies whether it is -.codn equal- based. -The hash function defaults -all three of these properties to false, and allows them to be overridden to -true by the presence of keyword arguments. - -It is an error to attempt to construct an -.codn equal -based -hash table which has weak keys. - -The hash function provides an alternative interface. It accepts optional -arguments which are keyword symbols. Any combination of the three symbols -.codn :weak-keys , -.code :weak-vals -and -.code :equal-based -can be specified in any order -to turn on the corresponding properties in the newly constructed hash table. -If any of the keywords is not specified, the corresponding property defaults to -.codn nil . - -If a hash table has weak keys, this means that from the point of view -of garbage collection, that table holds only weak references to the keys -stored in it. Similarly, if a hash table has weak values, it means that it -holds a weak reference to each value stored. A weak reference is one -which does not prevent the reclamation of an object by the garbage -collector. That is to say, when the garbage collector discovers that the only -references to some object are weak references, then that object is considered -garbage, just as if it had no references to it. The object is reclaimed, and -the weak references "lapse" in some way, which depends on what kind they are. -Hash table weak references lapse by entry removal: if either a key or a value -object is reclaimed, then the corresponding key-value entry is erased from the -hash table. - -Important to the operation of a hash table is the criterion by which keys are -considered same. By default, this similarity follows the eql function. A hash -table will search for a stored key which is -.code eql -to the given search key. -A hash table constructed with the -.codn equal -based -property compares keys using -the -.code equal -function instead. - -In addition to storing key-value pairs, a hash table can have a piece of -information associated with it, called the user data. - -A hash table can be traversed to visit all of the keys and data. The order of -traversal bears no relation to the order of insertion, or to any properties of -the key type. - -During an open traversal, new keys can be inserted into a hash table or deleted -from it while a a traversal is in progress. Insertion of a new key during -traversal will not cause any existing key to be visited twice or to be skipped; -however, it is not specified whether the new key will be traversed. Similarly, -if a key is deleted during traversal, and that key has not yet been visited, it -is not specified whether it will be visited during the remainder of the -traversal. - -An open traversal of a hash table is performed by the -.code maphash -function and the -.code dohash -operator. The traversal is open because code supplied by the program -is evaluated for each entry. - -The functions -.codn hash-keys , -.codn hash-values , -.codn hash-pairs , -and -.code hash-alist also perform an open traversal, because they return -lazy lists. The traversal isn't complete until the returned lazy list -is fully instantiated. In the meanwhile, the -\*(TX program can mutate the hash table from which the lazy list -is being generated. - -.coNP Functions @ hash-construct and @ hash-from-pairs -.synb -.mets (hash-construct < hash-args << key-val-pairs ) -.mets (hash-from-pairs < key-val-pairs << hash-arg *) -.syne -.desc -The -.code hash-construct -function constructs a populated hash in one step. The -.meta hash-args -argument specifies a list suitable as an argument list in a call to the hash -function. The -.meta key-val-pairs -is a sequence of pairs, which are two-element -lists representing key-value pairs. - -A hash is constructed as if by a call to -.cblk -.meti (apply hash << hash-args ), -.cble -then populated -with the specified pairs, and returned. - -The -.code hash-from-pairs -function is an alternative interface to the same semantics. The -.meta key-val-pairs -argument is first, and the -.meta hash-args -are passed as trailing variadic arguments, rather than a single list argument. - -.coNP Function @ hash-list -.synb -.mets (hash-list < key-list << hash-arg *) -.syne -.desc -The -.code hash-list -function constructs a hash as if by a call to -.cblk -.meti (apply hash << hash-args ), -.cble -where -.meta hash-args -is a list of the individual -.meta hash-arg -variadic arguments. - -The hash is then populated with keys taken from -.meta key-list -and returned. - -The value associated with each key is that key itself. - -.coNP Function @ hash-update -.synb -.mets (hash-update < hash << function ) -.syne -.desc -The -.code hash-update -function replaces each values in -.metn hash , -with the value of -.meta function -applied to that value. - -The return value is -.metn hash . - -.coNP Function @ hash-update-1 -.synb -.mets (hash-update-1 < hash < key < function <> [ init ]) -.syne -.desc -The -.code hash-update-1 -function operates on a single entry in the hash table. - -If -.meta key -exists in the hash table, then its corresponding value is passed -into -.metn function , -and the return value of -.meta function -is then installed -in place of the key's value. The value is then returned. - -If -.meta key -does not exist in the hash table, and no -.meta init -argument is given, -then -.code hash-update-1 -does nothing and returns -.codn nil . - -If -.meta key -does not exist in the hash table, and an -.meta init -argument is given, -then -.meta function -is applied to -.metn init , -and then -.meta key -is inserted into -.meta hash -with the value returned by -.meta function -as the datum. This value -is also returned. - -.coNP Function @ group-by -.synb -.mets (group-by < func < sequence << option *) -.syne -.desc -The -.code group-by -function produces a hash table from -.metn sequence , -which is a -list or vector. Entries of the hash table are not elements of -.metn sequence , -but lists of elements of -.metn sequence . -The function -.meta func -is applied to -each element of -.meta sequence -to compute a key. That key is used to determine -which list the item is added to in the hash table. - -The trailing arguments -.cblk -.meti << option * -.cble -if any, consist of the same keywords -that are understood by the hash function, and determine the properties -of the hash. - -.TP* Example: -Group the integers from 0 to 10 into three buckets keyed on 0, 1 and 2 -according to the modulo 3 congruence: - -.cblk - (group-by (op mod @1 3) (range 0 10))) - - -> #H(() (0 (0 3 6 9)) (1 (1 4 7 10)) (2 (2 5 8))) -.cble - -.coNP Functions @ make-similar-hash and @ copy-hash -.synb -.mets (make-similar-hash << hash ) -.mets (copy-hash << hash ) -.syne -.desc -The -.code make-similar-hash -and copy-hash functions create a new hash object based on -the existing -.meta hash -object. - -.code make-similar-hash -produces an empty hash table which inherits all of the -attributes of -.metn hash . -It uses the same kind of key equality, the -same configuration of weak keys and values, and has the same user data (see -the -.code set-hash-userdata -function). - -The -.code copy-hash -function is like -.codn make-similar-hash , -except that instead of -producing an empty hash table, it produces one which has all the same elements -as -.metn hash : -it contains the same key and value objects. - -.coNP Function @ inhash -.synb -.mets (inhash < hash < key <> [ init ]) -.syne -.desc -The -.code inhash -function searches hash table -.meta hash -for -.metn key . -If -.meta key -is found, then it return the hash table's cons cell which -represents the association between -.meta hash -and -.metn key . -Otherwise, it returns -.codn nil . - -If argument -.meta init -is specified, then the function will create -an entry for -.meta key -in -.meta hash -whose value is that of -.metn init . -The cons cell representing that association is returned. - -Note: for as long as the -.meta key -continues to exist inside -.metn hash . -modifying the -.code car -field of the returned cons has ramifications for the logical integrity of -the hash. Modifying the -.code cdr -field has the effect of updating the association with a new value. - -.coNP Accessor @ gethash -.synb -.mets (gethash < hash < key <> [ alt ]) -.mets (set (gethash < hash < key <> [ alt ]) << new-value ) -.syne -.desc -The -.code gethash -function searches hash table -.meta hash -for key -.metn key . -If the -key is found then the associated value is returned. Otherwise, if -the -.meta alt -argument was specified, it is returned. If the -.meta alt -argument -was not specified, -.code nil -is returned. - -A valid -.code gethash -form serves as a place. It denotes either an existing value in a hash -table or a value that would be created by the evaluation of the form. -The -.meta alt -argument is meaningful when -.code gethash -is used as a place, and, if present, is always evaluated whenever the place is -evaluated. -In place update operations, it provides the initial value, which defaults -to -.code nil -if the argument is not specified. For example -.code (inc (gethash h k d)) -will increment the value stored under key -.code k -in hash table -.code h -by one. If the key does not exist in the hash table, then -the value -.code (+ 1 d) -is inserted into the table under that key. -The expression -.code d -is always evaluated, whether or not its value is needed. - -If a -.code gethash -place is subject to a deletion, but doesn't exist, it is not an error. -The operation does nothing, and -.code nil -is considered the prior value of the place yielded -by the deletion. - -.coNP Function @ sethash -.synb -.mets (sethash < hash < key << value ) -.syne -.desc -The -.code sethash -function places a value into -.meta hash -table under the given -.metn key . -If a similar key already exists in the hash table, then that key's -value is replaced by -.metn value . -Otherwise, the -.meta key -and -.meta value -pair is -newly inserted into -.metn hash . - -The -.code sethash -function returns the -.metn value -argument. - -.coNP Function @ pushhash -.synb -.mets (pushhash < hash < key << element ) -.syne -.desc -The -.code pushhash -function is useful when the values stored in a hash table -are lists. If the given -.meta key -does not already exist in -.metn hash , -then a list of -length one is made which contains -.metn element , -and stored in -.meta hash -table under -.metn key . -If the -.meta key -already exists in the hash table, then the corresponding -value must be a list. The -.meta element -value is added to the front of that list, -and the extended list then becomes the new value under -.metn key . - -The return value is boolean. If true, indicates that the hash table entry was -newly created. If false, it indicates that the push took place on an existing -entry. - -.coNP Function @ remhash -.synb -.mets (remhash < hash << key ) -.syne -.desc -The -.code remhash -function searches -.meta hash -for a key similar to the -.metn key . -If that key is found, then that key and its corresponding value are -removed from the hash table. - -If the key is found and removal takes place, then the associated value -is returned. Otherwise -.code nil -is returned. - -.coNP Function @ hash-count -.synb -.mets (hash-count << hash ) -.syne -.desc -The -.code hash-count -function returns an integer representing the number of -key-value pairs stored in -.metn hash . - -.coNP Function @ get-hash-userdata -.synb -.mets (get-hash-userdata << hash ) -.syne -.desc -This function retrieves the user data object associated with -.metn hash . -The user data object of a newly-created hash table is initialized to -.codn nil . - -.coNP Function @ set-hash-userdata -.synb -.mets (set-hash-userdata < hash << object ) -.syne -.desc -The -.code set-hash-userdata -replaces, with the -.metn object , -the user data object -associated with -.metn hash . - -.coNP Function @ hashp -.synb -.mets (hashp << object ) -.syne -.desc -The -.code hashp -function returns -.code t -if the -.meta object -is a hash table, -otherwise it returns -.codn nil . - -.coNP Function @ maphash -.synb -.mets (maphash < hash << binary-function ) -.syne -.desc -The -.code maphash -function successively invokes -.meta binary-function -for each entry stored in -.metn hash . -Each entry's key and value are passed as arguments -to -.codn binary-function . - -The function returns -.codn nil . - -.coNP Functions @ hash-eql and @ hash-equal -.synb -.mets (hash-eql << object ) -.mets (hash-equal << object ) -.syne -.desc -These functions each compute an integer hash value from the internal -representation of -.metn object , -which satisfies the following properties. -If two objects -.code A -and -.code B -are the same under the -.code eql -function, then -.code (hash-eql A) -and -.code (hash-eql B) -produce the same integer hash value. Similarly, -if two objects -.code A -and -.code B -are the same under the -.code equal -function, then -.code (hash-equal A) -and -.code (hash-equal B) -each produce the same integer hash value. In all other -circumstances, the hash values of two distinct objects are unrelated, and -may or may not be the same. - -.coNP Functions @, hash_keys @, hash_values @ hash_pairs and @ hash_alist -.synb -.mets (hash-keys << hash ) -.mets (hash-values << hash ) -.mets (hash-pairs << hash ) -.mets (hash-alist << hash ) -.syne -.desc -These functions retrieve the bulk key-value data of hash table -.meta hash -in various ways. -.code hash-keys -retrieves a list of the keys. -.code hash-values -retrieves a list of the values. -.code hash-pairs -retrieves a list of pairs, -which are two-element lists consisting of the key, followed by the value. -Finally, -.code hash-alist -retrieves the key-value pairs as a Lisp association list: -a list of cons cells whose -.code car -fields are keys, and whose -.code cdr -fields are the values. Note that -.code hash-alist -returns the actual entries from the hash table, which are -conses. Modifying the -.code cdr -fields of these conses constitutes modifying the hash values -in the original hash table. Modifying the -.code car -fields interferes with the integrity of the hash table. - -These functions all retrieve the keys and values in the -same order. For example, if the keys are retrieved with -.codn hash-keys , -and the values with -.codn hash-values , -then the corresponding entries from -each list pairwise correspond to the pairs in -.metn hash . - -The list returned by each of these functions is lazy, and hence constitutes -an open traversal of the hash table. - -.coNP Operator @ dohash -.synb -.mets (dohash >> ( key-var < value-var < hash-form <> [ result-form ]) -.mets \ \ << body-form *) -.syne -.desc -The -.code dohash -operator iterates over a hash table. The -.meta hash-form -expression must -evaluate to an object of hash table type. The -.meta key-var -and -.meta value-var -arguments must be symbols suitable for use as variable names. -Bindings are established for these variables over the scope of the -.metn body-form s -and the optional -.metn result-form . + (rlet ((#:g0032 [a 0])) + (macrolet ((getit nil + (append (list 'car) (list '#:g0032))) + (setit (val) + (append (list 'sys:rplaca) + (list '#:g0032) (list val)))) + (setit (+ (getit) 1)))) -For each element in the hash table, the -.meta key-var -and -.meta value-var -variables are set to the key and value of that entry, respectively, -and each -.metn body-form , -if there are any, is evaluated. + ;; Same expander call as above, with a sys:expand to show the + ;; fully expanded version of the returned code, in which the + ;; setit and getit calls have disappeared, replaced by their + ;; macro-expansions. -When all of the entries of the table are thus processed, the -.meta result-form -is evaluated, and its return value becomes the return value of the dohash form. -If there is no -.metn result-form , -the return value is -.codn nil . + (sys:expand + (call car-update-expander 'getit 'setit '(car [a 0]) + '(setit (+ (getit) 1)))) + + --> + + (let ((#:g0032 [a 0])) + (sys:rplaca #:g0032 (+ (car #:g0032) 1))) -The -.meta result-form -and -.metn body-form s -are in the scope of an implicit anonymous -block, which means that it is possible to terminate the execution of -dohash early using -.cblk -.meti (return << value ) .cble -or -.codn (return) . +The main noteworthy points about the generated code are: +.RS +.IP - +the +.code (car [a 0]) +place is evaluated by evaluating the embedded form +.code [a 0] +and storing storing the resulting object into a hidden local variable. +That's as close a reference as we can make to the +.code car +field. +.IP - +the getter macro expands to code which simply calls the +.code car +function on the cell. +.IP - +the setter uses a system function called +.codn sys:rplaca , +which differs from +.code rplaca +in that it returns the stored value, rather than the cell. +.RE -.coNP Functions @, hash-uni @ hash-diff and @ hash-isec +.NP* The Clobber Expander .synb -.mets (hash-uni < hash1 < hash2 <> [ join-func ]) -.mets (hash-diff < hash1 << hash2 ) -.mets (hash-isec < hash1 < hash2 <> [ join-func ]) +.mets (lambda >> ( simple-setter-sym < place-form +.mets \ \ \ \ \ \ \ \ << body-form ) ...) .syne .desc -These functions perform basic set operations on hash tables in a nondestructive -way, returning a new hash table without altering the inputs. The arguments -.meta hash1 -and -.meta hash2 -must be compatible hash tables. This means that their keys -must use the same kind of equality. +The clobber expander is a code-writer similar to the update expander. +It takes a +.meta body-form +argument, and returns a larger form which surrounds this form +with additional program code. -The resulting hash table inherits attributes from -.metn hash1 , -as if created by the -.code make-similar-hash -function. If -.meta hash1 -has userdata, the resulting hash table -has the same userdata. If -.meta hash1 -has weak keys, the resulting table has weak -keys, and so forth. +The returned block of code has one main abstract action. +It must arrange for the evaluation of +.meta body-form +in a lexical environment in which a lexical macro or lexical function +exists which has the name requested by the +.meta simple-setter-sym +argument. -The -.code hash-uni -function performs a set union. The resulting hash contains all of -the keys from -.meta hash1 -and all of the keys from -.metn hash2 , -and their corresponding -values. If a key occurs both in -.meta hash1 -and -.metn hash2 , -then it occurs only once -in the resulting hash. In this case, if the -.meta join-func -argument is not given, -the value associated with this key is the one from -.metn hash1 . -If -.meta join-func -is specified then it is called with two arguments: the respective -data items from -.meta hash1 -and -.metn hash2 . -The return value of this function is used -as the value in the union hash. +The simple setter local macro written by the clobber expander is similar to the +local setter written by the update expander. It has exactly the +same interface, performs the same action of storing a value into +the place, and returns the new value. -The -.code hash-diff -function performs a set difference. First, a copy of -.meta hash1 -is made as if by the -.code copy-hash -function. Then from this copy, all keys which occur -in -.code hash2 -are deleted. +The difference is that its logic may be considerably simplified by the +assumption that the place is being subject to exactly one store, +and no access. -The -.code hash-isec -function performs a set intersection. The resulting hash contains -only those keys which occur both in -.meta hash1 -and -.metn hash2 . -If -.meta join-func -is not -specified, the values selected for these common keys are those from -.metn hash1 . -If -.meta join-func -is specified, then for each key which occurs in both -.meta hash1 -and -.metn hash2 , -it is called with two arguments: the respective data items. The return -value is then used as the data item in the intersection hash. +A place update macro which uses a clobber expander, and calls it more than +once, break the assumption; doing so may result in multiple evaluations +of the +.metn place-form . -.coNP Functions @ hash-subset and @ hash-proper-subset +.NP* The Delete Expander .synb -.mets (hash-subset < hash1 << hash2 ) -.mets (hash-proper-subset < hash1 << hash2 ) +.mets (lambda >> ( deleter-sym < place-form +.mets \ \ \ \ \ \ \ \ << body-form ) ...) .syne .desc -The -.code hash-subset -function returns -.code t -if the keys in -.meta hash1 -are a subset of the keys in -.metn hash2 . +The delete expander is a code-writer similar to clobber expander. +It takes a +.meta body-form +arguments, and returns a larger form which surrounds this form +with additional program code. -The -.code hash-proper-subset -function returns -.code t -if the keys in -.meta hash1 -are a proper subset of the keys in -.metn hash2 . -This means that -.meta hash2 -has all the keys which are in -.meta hash1 -and at least one which isn't. +The returned block of code has one main abstract action. +It must arrange for the evaluation of +.meta body-form +in a lexical environment in which a lexical macro or lexical function +exists which has the name requested by the +.meta deleter-sym +argument. -Note: the return value may not be mathematically meaningful if -.meta hash1 -and -.meta hash2 -use different equality. In any case, the actual behavior -may be understood as follows. The implementation of -.code hash-subset -tests whether each of the keys in -.meta hash1 -occurs in -.meta hash2 -using their respective equalities. -The implementation of -.code hash-proper-subset -applies -.code hash-subset -first, as above. If that is true, and the two hashes have the same number of -elements, the result is falsified. +The deleter macro written by the clobber expander takes no arguments. +It may be called at most once. It returns the previous value of the +place, and arranges for its obliteration, whatever that means for +that particular kind of place. -.SS* Partial Evaluation and Combinators -.coNP Macros @ op and @ do +.coNP Macro @ with-update-expander .synb -.mets (op << form +) -.mets (do << form +) +.mets (with-update-expander >> ( getter << setter ) < place < env +.mets \ << body-form ) .syne .desc The -.code op -and -.code do -macro operators are similar. +.code with-update-expander +macro evaluates the +.meta body-form +argument, whose result is expected to be a Lisp form. +The macro adds additional code around this code, and the result is returned. +This additional code is called the +.IR "place-access code" . + +The +.meta getter +and +.meta setter +arguments must be symbols. Over the evaluation of the +.metn body-form , +these symbols are bound to the names of local functions which +are provided in the place-access code. + +The +.meta place +argument is a form which evaluates to a syntactic place. The generated +place-access code is based on this place. + +The +.meta env +argument is a form which evaluates to a macro-expansion-time environment. +The +.code with-update-expander +macro uses this environment to perform macro-expansion on the value of the +.meta place +form, to obtain the correct update expander function for the fully +macro-expanded place. -Like the lambda operator, the -.code op -operator creates an anonymous function based on its syntax. -The difference is that the arguments of the function are implicit, or -optionally specified within the function body, rather than as a formal -parameter list before the body. +The place-access code is generated by calling the update expander +for the expanded version of +.codn place . -Also, the -.meta form -arguments of -.code op -are implicitly turned into a DWIM expression, -which means that argument evaluation follows Lisp-1 rules. (See the -.code dwim -operator). +.TP* "Example:" -The -.code do -operator is like the -.code op -operator with the following difference: -the -.meta form -arguments of -.code op -are not implicitly treated as a DWIM expression, -but as an ordinary expression. In particular, this means that operator -syntax is permitted. Note that the syntax -.code (op @1) -makes sense, since -the argument can be a function, which will be invoked, but -.code (do @1) -doesn't -make sense because it will produce a Lisp-2 form like -.code (#:arg1 ...) -referring -to nonexistent function -.codn #:arg1 . -Because it accepts operators, -.code do -can be used with imperative constructs -which are not functions, like set: like set: for instance -.code (do set x) -produces -an anonymous function which, if called with one argument, stores that argument -into -.codn x . +The following is an implementation of the +.code swap +macro, which exchanges the contents of two places. + +Two places are involved, and, correspondingly, the +.code with-update-expander +macro is used twice, to add two instances of place-update code +to the macro's body. -The argument forms are arbitrary expressions, within which a special -convention is permitted: -.RS -.meIP >> @ num -A number preceded by a -.code @ -is a metanumber. This is a special syntax -which denotes an argument. For instance -.code @2 -means that the second argument of -the anonymous function is to be substituted in place of the -.codn @2 . -.code op -generates a function which has a number of required arguments equal to the -highest value of -.meta num -appearing in a .cblk -.mati >> @ num + (defmacro swap (place-0 place-1 :env env) + (with-gensyms (tmp) + (with-update-expander (getter-0 setter-0) place-0 env + (with-update-expander (getter-1 setter-1) place-1 env + ^(let ((,tmp (,getter-0))) + (,setter-0 (,getter-1)) + (,setter-1 ,tmp)))))) .cble -construct in the body. For instance -.code (op car @3) -generates a three-argument function (which passes its third -argument to -.codn car , -returning the result, and ignores its first two arguments). -There is no way to use -.code op -to generate functions which have optional arguments. -.meIP < @rest -If the meta-symbol -.meta @rest -appears in the -.code op -syntax, it explicitly denotes the list of trailing arguments, -allowing them to be placed anywhere in the expression. -.RE - -.IP -Functions generated by -.code op -are always variadic; they always take additional arguments after -any required ones, whether or not the -.meta @rest -syntax is used. - -If the body does not contain -any -.meta @num -or -.meta @rest -syntax, then -.code @rest -is implicitly inserted. What this means is that, for example, since -the form -.code (op foo) -does not contain any numeric positional arguments like -.codn @1 , -and does not contain -.codn @rest , -it is actually a shorthand for -.codn (op foo . @rest) : -a function which applies all of its arguments to -.codn foo . -The actions of -.code op -be understood by these examples, which show -how -.code op -is rewritten to lambda. However, note that the real translator -uses generated symbols for the arguments, which are not equal to any -symbols in the program. +The basic logic for swapping two places is contained in the code template: .cblk - (op) -> invalid + ^(let ((,tmp (,getter-0))) + (,setter-0 (,getter-1)) + (,setter-1 ,tmp)) +.cble - (op +) -> (lambda rest [+ . rest]) +The temporary variable named by the +.code gensym +symbol +.code tmp +is initialized by calling the getter function for +.metn place-0 . +Then the setter function of +.meta place-0 +is called in order to store the value of +.meta place-1 +into +.metn place-0 . +Finally, the setter for +.meta place-1 +is invoked to store the previously saved temporary value into +that place. - (op + foo) -> (lambda rest [+ foo . rest]) +The name for the temporary variable is provided by the +.code with-gensyms +macro, but establishing the variable is the caller's responsibility; +this is seen as an explicit +.code let +binding in the code template. - (op @1 @2) -> (lambda (arg1 arg2 . rest) [arg1 arg2]) +The names of the getter and setter functions are similarly provided +by the +.code with-update-expander +macros. However, binding those functions is the responsibility of that +macro. To achieve this, it adds the place-access code to the code generated by +the +.code ^(let ...) +backquote template. In the following example macro-expansion, the additional +code added around the template is seen. It takes the form of two +.code macrolet +binding blocks, each added by an invocation of +.codn with-update-expander : - (op @1 . @rest) -> (lambda (arg1 . rest) [arg1 . @rest]) +.cblk + (macroexpand '(swap a b)) - (op @1 @rest) -> (lambda (arg1 . rest) [arg1 @rest]) + --> - (op @1 @2) -> (lambda (arg1 arg2 . rest) [arg1 arg2]) + (macrolet ((#:g0036 () 'a) ;; getter macro for a + (#:g0037 (val-expr) ;; setter macro for a + (append (list 'sys:setq) (list 'a) + (list val-expr)))) + (macrolet ((#:g0038 () 'b) ;; getter macro for b + (#:g0039 (val-expr) ;; setter macro for b + (append (list 'sys:setq) (list 'b) + (list val-expr)))) + (let ((#:g0035 (#:g0036))) ;; temp <- a + (#:g0037 (#:g0038)) ;; a <- b + (#:g0039 #:g0035)))) ;; b <- temp +.cble - (op foo @1 (@2) (bar @3)) -> (lambda (arg1 arg2 arg3 . rest) - [foo arg1 (arg2) (bar arg3)]) +In this expansion, for example +.code #:g0036 +is the generated symbol which forms the value of the +.code getter-0 +variable in the +.code swap +macro. The getter is a macro which simply expands to a +.codn a : +straightforward access to the variable a. +Of course, +.code #:g0035 +is nothing but the value of the +.code tmp +variable. Thus the swap macro's +.cblk +^(let ((,tmp (,getter-0))) ...) +.cble +has turned into +.cblk +^(let ((#:g0035 (#:g0036))) ...) +.cble - (op foo @rest @1) -> (lambda (arg1 . rest) [foo rest arg1]) +A full expansion, with the +.code macrolet +local macros expanded out: - (do + foo) -> (lambda rest (+ foo . rest)) +.cblk + (sys:expand '(swap a b)) - (do @1 @2) -> (lambda (arg1 arg2 . rest) (arg1 arg2)) + --> - (do foo @rest @1) -> (lambda (arg1 . rest) (foo rest arg1)) + (let ((#:g0035 a)) + (sys:setq a b) + (sys:setq b #:g0035)) .cble -Note that if argument -.meta @n -appears, it is not necessary -for arguments -.meta @1 -through -.meta @n-1 -to appear. The function will have -.code n -arguments: +In other words, the original syntax +.cblk +(,getter-0) +.cble +became +.cblk +(#:g0036) +.cble +and finally just +.codn a . +Similarly, .cblk - (op @3) -> (lambda (arg1 arg2 arg3 . rest) [arg3]) +(,setter-0 (,getter-1)) +.cble +became the +.code macrolet +invocations +.cblk +(#:g0037 (#:g0038)) .cble +which finally turned into: +.codn "(sys:setq a b)" . +.coNP Macro @ with-clobber-expander +.synb +.mets (with-clobber-expander <> ( simple-setter ) < place < env +.mets \ << body-form ) +.syne +.desc The -.code op -and -.code do -operators can be nested, in any combination. This raises the -question: if a metanumber like -.code @1 -or -.code @rest -occurs in an -.code op -that is nested -within an -.codn op , -what is the meaning? +.code with-clobber-expander +macro evaluates +.metn body-form , +whose result is expected to be a Lisp form. The macro adds additional code +around this form, and the result is returned. This additional code is called +the +.IR "place-access code" . -A metanumber always belongs with the inner-most op or do operator. So for -instance -.code (op (op @1)) -means that an -.code (op @1) -expression is nested -within an -.code op -expression which itself contains no meta-syntax. The -.code @1 -belongs with the inner op. +.meta simple-setter +argument must be a symbol. Over the evaluation of the +.metn body-form , +this symbol is bound to the name of a functions which +are provided in the place-access code. -There is a way for an inner -.code op -to refer to an outer op metanumber argument. This is -expressed by adding an extra -.code @ -prefix for every level of escape. For example in -.code (op (op @@1)) -the -.code @@1 -belongs to the outer -.codn op : -it is the same as -.code @1 -appearing in the outer -.codn op . -That is to say, -in the expression -.codn (op @1 (op @@1)) , -the -.code @1 -and -.code @@1 -are the same thing: -both are parameter 1 of the lambda function generated by the outer -.codn op . -By contrast, in the expression -.code (op @1 (op @1)) -there are two different parameters: -the first -.code @1 -is argument of the outer function, and the second -.code @1 -is the first argument of the inner function. Of course, if there -are three levels of nesting, then three -.code @ -meta-prefixes are needed to insert -a parameter from the outermost -.code op -into the innermost -.codn op . +The +.meta place +argument is a form which evaluates to a syntactic place. The generated +place-access code is based on this place. -.TP* Examples: +The +.meta env +argument is a form which evaluates to a macro-expansion-time environment. +The +.code with-clobber-expander +macro uses this environment to perform macro-expansion on the value of the +.meta place +form, to obtain the correct update expander function for the fully +macro-expanded place. -.cblk - ;; Take a list of pairs and produce a list in which those pairs - ;; are reversed. +The place-access code is generated by calling the update expander +for the expanded version of +.codn place . - (mapcar (op list @2 @1) '((1 2) (a b))) -> ((2 1) (b a)) -.cble +.TP* "Example:" -The -.code op -syntax interacts with quasiliterals which are nested within it. -The metanumber notation as well as -.code @rest -are recognized without requiring an additional -.code @ -escape: +The following implements a simple assignment statement, similar to +.code set +except that it only handles exactly two arguments: .cblk - (apply (op list `@1-@rest`) '(1 2 3)) -> "1-2 3" - - (apply (op list `@@1-@@rest`) '(1 2 3)) -> "1-2 3" + (defmacro assign (place new-value :env env) + (with-clobber-expander (setter) place env + ^(,setter ,new-value))) .cble -This is because the -.code op -macro traverses the code structure produced by the literal without recognizing -it specially, and there imposes its own meaning on these elements. +Note that the correct evaluation order of +.code place +and +.code new-value +is taken care of, because +.code with-clobber-expander +generates the code which performs all the necessary evaluations of +.codn place . +This evaluation occurs before the code which is generated by +.cblk +^(,setter ,new-value) +.cble +part is evaluated, and that code is what evaluates +.codn new-value . -Though they produce the same result, the above two examples differ in that -.code @rest -embeds a metasymbol into the quasiliteral structure, whereas -.code @@rest -embeds the Lisp expression -.code @rest -into the quasiliteral. In ordinary circumstances, the former refers to the -variable -.codn rest . -Contrast the previous example with: +Suppose that a macro were desired which allows assignment to be notated in a right to left +style, as in: .cblk - (let ((rest "0")) - `rest: @rest`) -> "rest: 0" + (assign 42 a) ;; store 42 in variable a +.cble + +Now, the new value must be evaluated prior to the place, if left to right +evaluation order is to be maintained. The standard +.code push +macro has this property: the push value is on the left, and the place +is on the right. + +Now, the code has to explicitly take care of the order, like this: - (let ((rest "0")) - `rest: @@rest`) -> ;; error: no such function or operator: sys:var .cblk + ;; WRONG! We can't just swap the parameters; + ;; place is still evaluated first, then new-value: -Under the -.code op -macro and its relatives, occurrences of -.code @rest -are replaced with syntax which refers to the trailing arguments -of the anonymous function. This happens before the interior of the -.code op -syntax undergoes expansion. Therefore the quasiliteral expander never -sees the -.codn @rest . + (defmacro assign (new-value place :env env) + (with-clobber-expander (setter) place env + ^(,setter ,new-value))) -This convenient omission of the -.codn @ -character isn't supported for reaching the arguments of an outer -.code op -from a quasiliteral within a nested -.codn op : + ;; Correct: arrange for evaluation of new-value first, + ;; then place: -.cblk - ;; To reach @@1, @@@1 must be written. - ;; @@1 Lisp expression introduced by @. - (op ... (op ... `@@@1`)) + (defmacro assign (new-value place :env env) + (with-gensym (tmp) + ^(let ((,tmp ,new-value)) + ,(with-clobber-expander (setter) place env + ^(,setter ,tmp))))) .cble -.coNP Macros @, ap @, ip @ ado and @ ido. +.coNP Macro @ with-delete-expander .synb -.mets (ap << form +) -.mets (ip << form +) -.mets (ado << form +) -.mets (ido << form +) +.mets (with-delete-expander <> ( deleter ) < place < env +.mets \ << body-form ) .syne .desc The -.code ap -macro is based on the -.code op -macro and has identical argument -conventions. +.code with-delete-expander +macro evaluates +.metn body-form , +whose result is expected to be a Lisp form. +The macro adds additional code +around this code, and the resulting code is returned. This additional code is +called the +.IR "place-access code" . The -.code ap -macro analyzes its arguments and produces a function -.metn f , -in exactly the same same way as the -.code op -macro. However, instead of returning -.metn f , -directly, it returns a different function -.metn g , -which is a one-argument function which accepts a list, -and then applies the list as arguments to -.metn f . - -In other words, the following equivalence holds: - -.cblk - (ap form ...) <--> (apf (op form ...)) -.cble +.meta deleter +argument must be a symbol. Over the evaluation of the +.metn body-form , +this symbol is bound to the name of a functions which +are provided in the place-access code. The -.code ap -macro nests properly with -.code op -and -.codn do , -in any combination, in regard to the -.meta ...@@n -notation. +.meta place +argument is a form which evaluates to a syntactic place. The generated +place-access code is based on this place. The -.ode ip -macro is very similar to the -.code ap -macro, except that it is based -on the semantics of the function -.code iapply -rather than -.codn apply , -according -to the following equivalence: +.meta env +argument is a form which evaluates to a macro-expansion-time environment. +The +.code with-delete-expander +macro uses this environment to perform macro-expansion on the value of the +.meta place +form, to obtain the correct update expander function for the fully +macro-expanded place. -.cblk - (ip form ...) <--> (ipf (op form ...)) -.cble +The place-access code is generated by calling the update expander +for the expanded version of +.codn place . -The -.code ado -and -.code ido -macros are related to do macro in the same way that -.code ap -and -.code ip -are related to -.codn op . -They produce a one-argument function which works -as if by applying its arguments to the function generated by do, -according to the following equivalence: +.TP* "Example:" -.cblk - (ado form ...) <--> (apf (do form ...)) +The following implements the +.code del +macro: - (ido form ...) <--> (ipf (do form ...)) .cblk + (defmacro del (place :env env) + (with-delete-expander (deleter) place env + ^(,deleter))) +.cble -See also: the -.code apf -and -.code ipf -functions. - -.coNP Macros @ opip and @ oand +.coNP Function @ call-update-expander .synb -.mets (opip << clause *) -.mets (oand << clause *) +.mets (call-update-expander < getter < setter < place < env << body-form ) .syne .desc The -.code opip +.code call-update-expander +function provides an alternative interface for making use of an update +expander, complementary to +.codn with-update-expander . + +Arguments +.meta getter and -.code oand -operators make it possible to chain together functions which are expressed -using the -.code op -syntax. (See the -.code op -operator for more information). +.meta setter +are symbols, provided by the caller. These are passed to the update +expander function, and are used for naming local functions in the +generated code which the update expander adds to +.metn body-form . -Both macros perform the same transformation except that -.code opip -translates its arguments to a call to the -.code chain -function, whereas -.code oand -translates its arguments in the same way to a call to the -.code chand -function. +The +.meta place +argument is a place which has not been subject to macro-expansion. +The +.code call-update-expander +function takes on the responsibility for macro-expanding the place. -More precisely, these macros perform the following rewrites: +The +.meta env +parameter is the macro-expansion environment object required to +correctly expand +.code place +in its original environment. -.cblk - (opip arg1 arg2 ... argn) -> [chain {arg1} {arg2} ... {argn}] - (oand arg1 arg2 ... argn) -> [chand {arg1} {arg2} ... {argn}] -.cble +The +.meta body-form +argument represents the source code of a place update operation. +This code makes references to the local functions whose names +are given by +.meta getter +and +.metn setter . +Those arguments allow the update expander to write these functions +with the matching names expected by +.metn body-form . -where the above -.code {arg} -notation denotes the following transformation applied to each argument: +The return value is an object representing source code which incorporates +the +.metn body-form , +augmenting it with additional code which evaluates +.code place +to determine its location, and provides place accessor local functions +expected by the +.metn body-form . + +.TP* "Example:" + +The following shows how to implement a +.code with-update-expander +macro using +.codn call-update-expander : .cblk - (function ...) -> (op function ...) - (operator ...) -> (do operator ...) - (macro ...) -> (do macro ...) - (dwim ...) -> (dwim ...) - [...] -> [...] - atom -> atom + (defmacro with-update-expander ((getter setter) + unex-place env body) + ^(with-gensyms (,getter ,setter) + (call-update-expander ,getter ,setter + ,unex-place ,env ,body))) .cble -In other words, compound forms whose leftmost symbol is a macro or operator -are translated to the -.code do -notation. Compound forms denoting function calls are translated to the -.code op -notation. Compound forms which are -.code dwim -invocations, either explicit or via the DWIM brackets notation, are -preserved, as are any forms which are atoms. +Essentially, all that +.code with-update-expander +does is to choose the names for the local functions, and bind them +to the local variable names it is given as arguments. Then it +calls +.codn call-update-expander . -Note: the -.code opip -and -.code oand -macros use their macro environment in determining whether a form is a -macro call, thereby respecting lexical scoping. +.TP* "Example:" -.TP* Example: -Take each element from the list -.code (1 2 3 4) -and multiply it by three, then add 1. -If the result is odd, collect that into the resulting list: +Implement the swap macro using +.codn call-update-expander : .cblk -(mappend (opip (* 3) - (+ 1) - [iff oddp list]) - (range 1 4)) + (defmacro swap (place-0 place-1 :env env) + (with-gensyms (tmp getter-0 setter-0 getter-1 setter-1) + (call-update-expander getter-0 setter-0 place-0 env + (call-update-expander getter-1 setter-1 place-1 env + ^(let ((,tmp (,getter-0))) + (,setter-0 (,getter-1)) + (,setter-1 ,tmp)))))) .cble -The above is equivalent to: +.coNP Function @ call-clobber-expander +.synb +.mets (call-clobber-expander < simple-setter < place < env << body-form ) +.syne +.desc +The +.code call-clobber-expander +function provides an alternative interface for making use of a clobber +expander, complementary to +.codn with-clobber-expander . -.cblk -(mappend (chain (op * 3) - (op + 1) - [iff oddp list]) - (range 1 4)) -.cble +Argument +.meta simple-setter +is a symbol, provided by the caller. It is passed to the clobber +expander function, and is used for naming a local function in the +generated code which the update expander adds to +.metn body-form . The -.code (* 3) -and -.code (+ 1) -terms are rewritten to -.code (op * 3) -and -.codn (op + 1) , -respectively, whereas -.code [iff oddp list] -is passed through untransformed. +.meta place +argument is a place which has not been subject to macro-expansion. +The +.code call-clobber-expander +function takes on the responsibility for macro-expanding the place. + +The +.meta env +parameter is the macro-expansion environment object required to +correctly expand +.code place +in its original environment. + +The +.metn body-form +argument represents the source code of a place update operation. +This code makes references to the local function whose name +is given by +.metn simple-setter . +That argument allows the update expander to write this function +with the matching name expected by +.metn body-form . -.coNP Macro @ ret +The return value is an object representing source code which incorporates +the +.metn body-form , +augmenting it with additional code which evaluates +.code place +to determine its location, and provides the clobber local function +to the +.metn body-form . + +.coNP Function @ call-delete-expander .synb -.mets (ret << form ) +.mets (call-delete-expander < deleter < place < env << body-form ) .syne .desc The -.code ret -macro's -.meta form -argument is treated similarly to the second and subsequent arguments of the -.code op -operator. +.code call-delete-expander +function provides an alternative interface for making use of a delete +expander, complementary to +.codn with-delete-expander . -The -.code ret -macro produces a function which takes any number of arguments, -and returns the value specified by -.metn form . +Argument +.meta deleter +is a symbol, provided by the caller. It is passed to the delete +expander function, and is used for naming a local function in the +generated code which the update expander adds to +.metn body-form . -.meta form -can contain -.code op -meta syntax like -.code @n -and -.codn @rest . +The +.meta place +argument is a place which has not been subject to macro-expansion. +The +.code call-delete-expander +function takes on the responsibility for macro-expanding the place. -The following equivalence holds: +The +.meta env +parameter is the macro-expansion environment object required to +correctly expand +.code place +in its original environment. -.cblk - (ret x) <--> (op identity x)) -.cble +The +.meta body-form +argument represents the source code of a place delete operation. +This code makes references to the local function whose name +is given by +.metn deleter . +That argument allows the update expander to write this function +with the matching name expected by +.metn body-form . -Thus the expression -.code (ret @2) -returns a function similar to -.codn (lambda (x y . z) y) , -and the expression -.code (ret 42) -returns a function similar to -.codn (lambda (. rest) 42) . +The return value is an object representing source code which incorporates +the +.metn body-form , +augmenting it with additional code which evaluates +.code place +to determine its location, and provides the delete local function +to the +.metn body-form . -.coNP Macro @ aret +.coNP Macro @ define-modify-macro .synb -.mets (aret << form ) +.mets (define-modify-macro < name < parameter-list << function-name ) .syne .desc The -.code ret -macro's -.meta form -argument is treated similarly to the second and subsequent arguments of the -.code op -operator. +.code define-modify-macro +macro provides a simplified way to write certain kinds of place update +macros. Specifically, it provides a way to write place update macros +which modify a place by retrieving the previous value, pass it through +a function (perhaps together with some additional arguments), and then store +the resulting value back into the place and return it. The -.code aret -macro produces a function which takes any number of arguments, -and returns the value specified by -.metn form . +.meta name +parameter specifies the name for the place update macro to be written. -.meta form -can contain -.code ap -meta syntax like -.meta @n -and -.codn @rest . +The +.meta function-name +parameter must specify a symbol: the name of the update function. -The following equivalence holds: +The update macro and update function both take at least one parameter: +the place to be updated, and its value, respectively. + +The +.meta parameter-list +specifies the additional parameters for update function, which will also +become additional parameters of the macro. Because it is a +function parameter list, it cannot use the special destructuring features of +macro parameter lists, or the +.code :env +or +.code :whole +special parameters. It can use optional parameters. Of course, it may be empty. + +The +.code define-modify-macro +macro writes a macro called +.metn name . +The leftmost parameter of this macro is a place, followed by the additional arguments +specified by +.metn parameter-list . +The macro will arrange for the evaluation of the place argument to determine +the place location. It will then retrieve and save the prior value of the +place, and evaluate the remaining arguments. The prior value of the +place, and the values of the additional arguments, are all passed to +.meta function +and the resulting value is then stored back into the location previously +determined for +.metn place . + +.TP* "Example:" + +Some standard place update macros are implementable using +.codn define-modify-macro , +such as +.codn inc . + +The +.code inc +macro reads the old value of the place, then passes it through the +.code + +(plus) function, along with an extra argument: the delta value, which +defaults to one. The +.code inc +macro could be written using +.code define-modify-macro +as follows: .cblk - (aret x) <--> (ap identity x)) + (define-modify-macro inc (: (delta 1)) +) .cble -Thus the expression -.code (aret @2) -returns a function similar to -.codn (lambda (. rest) (second rest)) , -and the expression -.code (aret 42) -returns a function similar to -.codn (lambda (. rest) 42) . +Note that the argument list +.code (: (delta 1)) +doesn't specify the place, because the place is the implicit leftmost +argument of the macro which isn't given a name. With the above definition +in place, when +.code (inc (car a)) +is invoked, then +.code (car a) +is first reduced to a location, and that location's value is retrieved and +saved. Then the +.code delta +parameter s evaluated to its value, which has defaulted to 1, since +the argument was omitted. +Then these two values are passed to the +.code + +function, and so 1 is added to the value previously retrieved from +.codn (car a) . +The resulting sum is then stored back +.code (car a) +without, of course, evaluating +.code (car a) +again. -.coNP Function @ dup +.coNP Macro @ defplace .synb -.mets (dup << func ) +.mets (defplace < place-destructuring-args < body-sym +.mets \ \ \ \ \ \ \ \ \ >> ( getter-sym < setter-sym << update-body ) +.mets \ \ \ \ \ \ \ \ \ >> [( ssetter-sym << clobber-body ) +.mets \ \ \ \ \ \ \ \ \ \ >> [( deleter-sym << delete-body )]]) .syne .desc The -.code dup -function returns a one-argument function which calls the two-argument -function -.metn func -by duplicating its argument. +.code defplace +macro is used to introduce a new kind of syntactic place. +It writes the update expander, and optionally clobber and delete +expander functions, from a simpler, more compact specification, +and automatically registers the resulting functions. The compact specification +of a +.code defplace +call contains only code fragments for the expander functions. -.TP* Example: +The name and syntax of the place is determined by the +.meta place-destructuring-args +argument, which is macro-style parameter list whose structure +mimics that of the the place. In particular, its leftmost symbol +gives the name under which the place is registered. +The +.code defplace +macro provides automatic destructuring of the syntactic place, +so that the expander code fragments can refer to the components +of a place by name. -.cblk - ;; square the elements of a list - (mapcar [dup *] '(1 2 3)) -> (1 4 9) -.cble +The +.meta body-sym +parameter must be be a symbol. This symbol will capture the +.meta body-forms +parameter which is passed to the update expander, clobber +expander or delete expander. The code fragments then have +access to the the body forms via this name. + +The +.metn getter-sym , +.metn setter-sym , +and +.metn update-body +parenthesized triplet specify the update expander fragment. +The +.code defplace +macro will bind +.meta getter-sym +and +.meta setter-sym +to symbols. The +.meta update-body +must then specify a template of code which evaluates the syntactic place to +determine its storage location, and provides a pair of local functions, using +these two symbols as their name. The template must also insert the +.meta body-sym +forms into the scope of these local functions, and the place determining code. -.coNP Function @ flipargs -.synb -.mets (flipargs << func ) -.syne -.desc The -.code flipargs -function returns a two-argument function which calls the two-argument -function -.metn func -with reversed arguments. +.meta setter-sym +and +.meta clobber-body +arguments similarly specify an optional clobber expander fragment, +as a single optional argument. If specified, the +.meta clobber-body +must generate a local function named using +.meta setter-sym +wrapped around +.meta body-sym +forms. -.coNP Functions @ chain and @ chand -.synb -.mets (chain << func *) -.mets (chand << func *) -.syne -.desc The -.code chain -function accepts zero or more functions as arguments, and returns -a single function, called the chained function, which represents the chained -application of those functions, in left to right order. +.meta deleter-sym +and +.meta deleter-body +likewise specify a delete expander fragment. If this is omitted, +then the place shall not support deletion. -If -.code chain -is given no arguments, then it returns a variadic function which -ignores all of its arguments and returns -.codn nil . +.TP* "Example:" -Otherwise, the first function may accept any number of arguments. The second -and subsequent functions, if any, must accept one argument. +Implementation of the place denoting the +.code car +field of +.code cons +cells: -The chained function can be called with an argument list which is acceptable -to the first function. Those arguments are in fact passed to the first -function. The return value of that call is then passed to the second -function, and the return value of that call is passed to the third function -and so on. The final return value is returned to the caller. +.cblk + (defplace (car cell) body -The -.code chand -function is similar, except that it combines the functionality of -.code andf -into chaining. The difference between -.code chain -and -.code chand -is that -.code chand -immediately terminates and returns -.code nil -whenever any of the functions returns -.codn nil , -without calling the remaining functions. + ;; the update expander fragment + (getter setter + (with-gensyms (cell-sym) ;; temporary symbol for cell + ^(let ((,cell-sym ,cell)) ;; evaluate place to cell + ;; getter and setter access cell via temp var + (macrolet ((,getter () + ^(car ,',cell-sym)) + (,setter (val) + ^(sys:rplaca ,',cell-sym ,val))) -.TP* Example: + ;; insert body form from place update macro + ,body)))) -.cblk - (call [chain + (op * 2)] 3 4) -> 14 + ;; clobber expander fragment: simpler: no need + ;; to evaluate cell to temporary variable. + (ssetter + ^(macrolet ((,ssetter (val) + ^(sys:rplaca ,',cell ,val))) + ,body)) + + ;; deleter: delegate to pop semantics: + ;; (del (car a)) == (pop a). + (deleter + ^(macrolet ((,deleter () ^(pop ,',cell))) + ,body))) .cble -In this example, a two-element chain is formed from the -.code + -function -and the function produced by -.code (op * 2) -which is a one-argument -function that returns the value of its argument multiplied by two. -(See the definition of the -.code op -operator). +.coNP Macro @ define-place-macro +.synb +.mets (define-place-macro < name < macro-style-params +.mets \ \ << body-form *) +.syne +.desc +In some situations, an equivalence exists between two forms, only one +of which is recognized as a place. The +.code define-place-macro +macro can be used to establish a form as a place in terms of a translation to +an equivalent form which is already a place. -The chained function is invoked using the -.code call -function, with the arguments -.code 3 -and -.codn 4 . -The chained evaluation begins by passing -.code 3 -and -.code 4 -to -.codn + , -which yields -.codn 7 . -This -.code 7 -is then passed to the -.code (op * 2) -doubling function, resulting in -.codn 14 . +The +.code define-place-macro +has the same syntax as +.codn defmacro . +It specifies a macro transformation for a compound form which has the +.meta name +symbol in its leftmost position. +This macro expansion is applied when such a form is used as a place. +It is applied after all other expansions, and no other macro-expansions +are applied afterward. -A way to write the above example without the use of the DWIM brackets and the -op operator is this: +.TP* "Example:" + +Implementation of +.code first +in terms of +.codn car : .cblk - (call (chain (fun +) (lambda (x) (* 2 x))) 3 4) + (define-place-macro first (obj) + ^(car ,obj)) .cble -.coNP Function @ juxt +.coNP Macro @ rlet .synb -.mets (juxt << func *) +.mets (rlet >> ({( sym << init-form )}*) << body-form *) .syne .desc +The macro +.code rlet +is similar to the +.code let +operator. It establishes bindings for one or more +.metn sym -s, +which are initialized using the values of +.metn init-form -s. + +Note that the simplified syntax for a variable which initializes to +.code nil +by default is not supported by +.codn rlet ; +that is to say, the syntax +.meta sym +cannot be used in place of the +.meti >> ( sym << init-form ) +syntax when +.meta sym +is to be initialized to +.codn nil . + The -.code juxt -function accepts a variable number of arguments which are functions. It -combines these into a single function which, when invoked, passes its arguments -to each of these functions, and collects the results into a list. +.code rlet +macro differs from +.code let +in that +.code rlet +assumes that those +.metn sym -s +which have constant +.metn init-form -s +(according to the +.code constantp +function) may be safely implemented as a symbol macro rather than a lexical +variable. -Note: the juxt function can be understood in terms of the following reference -implementation: +Therefore +.code rlet +is suitable in situations in which simpler code is desired from the output +of certain kinds of machine-generated code, which binds local symbols: +code with fewer temporary variables. -.cblk - (defun juxt (funcs) - (lambda (. args) - (mapcar (lambda (fun) - (apply fun args)) - funcs))) -.cble +On the other hand, +.code rlet +is not suitable in situations when true variables are required, which +are assignable, and provide temporary storage. -.TP* Example: +.TP* "Example:" .cblk - ;; separate list (1 2 3 4 5 6) into lists of evens and odds, - ;; which end up juxtaposed in the output list: + ;; WRONG! Exchange two variables, a and b: + (rlet ((temp a)) + (set a b) + (set b temp)) - [(op [juxt keep-if remove-if] evenp) - '(1 2 3 4 5 6)] -> ((2 4 6) (1 3 5)) + ;; Demonstration of constant-propagation + (let ((a 42)) + (rlet ((x 1) + (y a)) + (+ x y))) --> 43 - ;; call several functions on 1, collecting their results: - [[juxt (op + 1) (op - 1) evenp sin cos] 1]' - -> (2 0 nil 0.841470984807897 0.54030230586814) + (sys:expand + '(let ((a 42)) + (rlet ((x 1) + (y a)) + (+ x y)))) --> (let ((a 42)) + (let ((y a)) + (+ 1 y))) .cble -.coNP Functions @ andf and @ orf +The last example shows that the +.code x +variable has disappeared in the expansion. The +.code rlet +macro turned it into into a +.code symacrolet +denoting the constant 1, which then propagated to the use site, +turning the expression +.code (+ x y) +into +.codn (+ 1 y) . + +.coNP Macro @ with-gensyms .synb -.mets (andf << func *) -.mets (orf << func *) +.mets (with-gensyms <> ( sym *) << body-form *) .syne .desc The -.code andf -and -.code orf -functions are the functional equivalent of the -.code and -and -.code or -operators. These functions accept multiple functions and return a new function -which represents the logical combination of those functions. +.code with-gensyms +evaluates the +.metn body-form -s +in an environment in which each variable name symbol +.meta sym +is bound to a new uninterned symbol ("gensym"). -The input functions should have the same arity. Failing that, there should -exist some common argument arity with which each of these can be invoked. The -resulting combined function is then callable with that many arguments. +.TP* "Example:" -The -.code andf -function returns a function which combines the input functions with -a short-circuiting logical conjunction. The resulting function passes its -arguments to the functions successively, in left to right order. As soon as any -of the functions returns -.codn nil , -then nil is returned immediately, and the -remaining functions are not called. Otherwise, if none of the functions return -.codn nil , -then the value returned by the last function is returned. If the list of -functions is empty, then -.code t -is returned. That is, -.code (andf) -returns a function -which accepts any arguments, and returns -.codn t . +The code: -The -.code orf -function combines the input functions with a short-circuiting logical -disjunction. The function produced by -.code orf -passes its arguments down to the -functions successively, in left to right order. As soon as any function -returns a -.cod2 non- nil -value, that value is returned and the remaining functions are -not called. If all functions return -.codn nil , -then -.code nil -is returned. The expression -.code (orf) -returns a function which accepts any arguments and returns -.codn nil . +.cblk + (let ((x (gensym)) + (y (gensym)) + (z (gensym))) + ^(,x ,y ,z)) +.cble -.coNP Function @ notf -.synb -.mets (notf << function ) -.syne -.desc -The -.code notf -function returns a function which is the boolean negation -of -.metn function . +may be expressed more conveniently using the +.code with-gensyms +shorthand: -The returned function takes a variable number of arguments. When -invoked, it passes all of these arguments to -.meta function -and then inverts the result as if by application of the -.codn not . +.cblk + (with-gensyms (x y z) + ^(,x ,y ,z)) +.cble -.coNP Functions @ iff and @ iffi +.SS* Quasiquote Operator Syntax +.coNP Macro @ qquote .synb -.mets (iff < cond-func >> [ then-func <> [ else-func ]]) -.mets (iffi < cond-func < then-func <> [ else-func ]) +.mets (qquote << form ) .syne .desc The -.code iff -function is the functional equivalent of the -.code if -operator. It accepts -functional arguments and returns a function. +.code qquote +(quasi-quote) macro operator implements a notation for convenient +list construction. If +.meta form +is an atom, or a list structure which +does not contain any +.code unquote +or +.code splice +operators, then +.cblk +.meti (qquote << form ) +.cble +is equivalent to +.cblk +.meti (qquote << form ). +.cble -The resulting function takes its arguments, if any, and applies them to -.metn cond-func . If -.meta cond-func -yields true, then the arguments are passed to -.meta then-func +.metn form , +however, is a list structure which contains +.code unquote +or +.code splice +operators, then the substitutions implied by those operators are performed +on +.metn form , and the -resulting value is returned. Otherwise the arguments are passed to -.meta else-func -and the resulting value is returned. +.code qquote +operator returns the resulting structure. -If -.meta then-func -is omitted then -.code identity -is used as default. This omission is not permitted by -.codn iffi , -only -.codn iff . +Note: how the qquote operator actually works is that it is compiled into +code. It becomes a Lisp expression which, when evaluated, computes the +resulting structure. -If -.meta else-func -needs to be called, but is omitted, then -.code nil -is returned. +A +.code qquote +can contain another +.codn qquote . +If an +.code unquote +or +.code splice +operator occurs +within a nested +.codn qquote , +it belongs to that +.codn qquote , +and not to the outer one. -The -.code iffi -function differs from -.code iff -only in the defaulting behavior with respect -to the -.meta else-func -argument. If -.meta else-func -is omitted in a call to -.code iffi -then the default function is -.codn identity . -This is useful in situations when one value is to be -replaced with another one when the condition is true, otherwise -preserved. +However, an unquote operator which occurs inside another one belongs one level +higher. For instance in -The following equivalences hold between -.code iffi +.cblk + (qquote (qquote (unquote (unquote x)))) +.cble + +the leftmost +.code qquote +belongs with the rightmost unquote, and the inner +.code qquote and -.codn iff : +.code unquote +belong together. When the outer +.code qquote +is evaluated, +it will insert the value of +.codn x , +resulting in the object +.codn (qquote (unquote [value-of-x])) . +If this resulting qquote value is evaluated again as Lisp syntax, then it will +yield +.codn [value-of-value-of-x] , +the value of +.code [value-of-x] +when treated as a Lisp expression and evaluated. + +.TP* Examples: .cblk - (iffi a b c) <--> (iff a b c) + (qquote a) -> a - (iffi a b) <--> (iff a b identity) + (qquote (a b c)) -> (a b c) - (iffi a b false) <--> (iff a b) + (qquote (1 2 3 (unquote (+ 2 2)) (+ 2 3))) -> (1 2 3 4 (+ 2 3)) - (iffi a identity false) <--> (iff a) + (qquote (unquote (+ 2 2))) -> 4 +.cble + +In the second-to-last example, the +.code 1 2 3 +and the +.code (+ 2 3) +are quoted verbatim. +Whereas the +.code (unquote (+ 2 2)) +operator caused the evaluation of +.code (+ 2 2) +and the substitution of the resulting value. + +The last example shows that +.meta form +can itself (the entire argument of +.codn qquote ) +can be an unquote operator. +However, note: +.code (quote (splice form)) +is not valid. + +Note: a way to understand the nesting behavior is a via a possible model of +quasi-quote expansion which recursively compiles any nested quasi quotes first, +and then treats the result of their expansion. For instance, in the processing +of + +.cblk + (qquote (qquote (unquote (unquote x)))) .cble -The following equivalence illustrates -.code iff -with both optional arguments omitted: +the +.code qquote +operator first encounters the +embedded +.code (qquote ...) +and compiles it to code. During that recursive +compilation, the syntax +.code (unquote (unquote x)) +is encountered. The inner quote +processes the outer unquote which belongs to it, and the inner +.code (unquote x) +becomes material that is embedded verbatim in the compilation, which will then +be found when the recursion pops back to the outer quasiquote, which will +then traverse the result of the inner compilation and find the +.codn (unquote x) . + +.TP* "Dialect note:" -.cblk - (iff a) <---> (iff a identity false) -.cble +In Lisp dialects which have a published quasiquoting operator syntax, there is +the expectation that the quasiquote read syntax corresponds to it. That is to +say, that for instance the read syntax +.code ^(a b ,c) +is expected translated to +.codn (qquote b (unquote c)) . -.coNP Functions @ tf and @ nilf -.synb -.mets (tf << arg *) -.mets (nilf << arg *) -.syne -.desc -The -.code tf +In \*(TL, this is not true! Although +.code ^(b b ,c) +is translated to a +quasiquoting macro, it is an internal one, not based on the public +.codn qquote , +.code unquote and -.code nilf -functions take zero or more arguments, and ignore them. -The -.code tf -function returns -.codn t , -and the -.code nilf -function returns -.codn nil . +.code splice +symbols being documented here. -Note: the following equivalences hold between these functions and the -.code ret -operator, and -.code retf -function. +This idea exists for hygiene. The quasiquote read syntax is not confused +by the presence of the symbols +.codn qquote , +.code unquote +or +.code splice +in the template, since it doesn't treat them specially. + +This also allows programmers to use the quasiquote read syntax to construct +quasiquote macros. For instance .cblk - (fun tf) <--> (ret t) <--> (retf t) - (fun nilf) <--> (ret nil) <--> (ret) <--> (retf nil) + ^(qquote (unquote ,x)) ;; does not mean ^^,x .cble -In Lisp-1-style code, -.code tf -and -.code nilf -behave like constants which can replace uses of -.code (ret t) +To the quasiquote reader, the +.code qquote and -.codn (ret nil) : - -.cblk - [mapcar (ret nil) list] <--> [mapcar nilf list] -.cble +.code unquote +symbols mean nothing special, +and so this syntax simply means that if the value of +.code x +is +.codn foo , +the result will be +.codn (qquote (unquote foo)) . -.TP* Example: +The form's expansion is actually this: .cblk - ;; tf and nilf are useful when functions are chained together. - ;; test whether (trunc n 2) is odd. - - (defun trunc-n-2-odd (n) - [[chain (op trunc @1 2) [iff oddp tf nilf]] n]) + (sys:qquote (qquote (unquote (sys:unquote x)))) .cble -In this example, two functions are chained together, and -.code n -is passed -through the chain such that it is first divided by two via the -function denoted by -.code (op trunc @1 2) -and then the result is passed into the -function denoted by -.codn [iff oddp tf nilf] . -The -.code iff -function passes its argument into -.codn oddp , -and if -.code oddp -yields true, it passes the same argument to -.codn tf . -Here -.code tf -proves its utility by ignoring that value and returning -.codn t . -If the argument (the divided value) passed into -.code iff -is even, then iff passes it into the -.code nilf -function, which ignores the value and returns -.codn nil . +the +.code sys:qquote +macro recognizes +.code sys:unquote +embedded in the form, and +the other symbols not in the +.code sys: +package are just static template material. -.coNP Function @ retf -.synb -.mets (retf << value ) -.syne -.desc The -.code retf -function returns a function. That function can take zero or -more arguments. When called, it ignores its arguments and returns -.metn value . - -See also: the -.code ret -macro. - -.TP* Example: - -.cblk - ;; the function returned by (retf 42) - ;; ignores 1 2 3 and returns 42. - (call (retf 42) 1 2 3) -> 42 -.cble +.code sys:quote +macro and its associated +.code sys:unquote +and +.code sys:splice +operators work exactly like their ordinary counterparts. So in effect, \*(TX has +two nearly identical, independent quasi-quote implementations, one of which is +tied to the read syntax, and one of which isn't. This is useful for writing +quasiquotes which write quasiquotes. -.coNP Functions @ apf and @ ipf +.coNP Operator @ unquote .synb -.mets (apf << function ) -.mets (ipf << function ) +.mets (qquote (... (unquote << form ) ...)) +.mets (qquote (unquote << form )) .syne .desc The -.code apf -function returns a one-argument function which accepts -a list. When the function is called, it treats the list as -arguments which are applied to -.meta function -as if by apply. It returns whatever -.meta function -returns. - +.code unquote +operator is not an operator +.I per +.IR se . The -.code ipf -function is similar to -.codn apf , -except that the returned -function applies arguments as if by -.code iapply -rather than -.codn apply . - -See also: the -.code ap -macro. - -.TP* Example: +.code unquote +symbol has no +binding in the global environment. It is a special syntax that is recognized +within a +.code qquote +form, to indicate forms within the quasiquote which are to be +evaluated and inserted into the resulting structure. +The syntax .cblk - ;; Function returned by [apf +] accepts the - ;; (1 2 3) list and applies it to +, as - ;; if (+ 1 2 3) were called. - - (call [apf +] '(1 2 3)) -> 6 -.cble +.meti (qquote (unquote << form )) +.cblk +is equivalent to +.metn form : +the +.code qquote +and +.code unquote +"cancel out". -.coNP Function @ callf +.coNP Operator @ splice .synb -.mets (callf < main-function << arg-function *) +.mets (qquote (... (splice << form ) ...)) .syne .desc The -.code callf -function returns a function which applies its arguments to each -.metn arg-function , -juxtaposing the return values of these calls to form arguments -which are then passed to -.metn main-function . -The return value of -.meta main-function -is returned. - -The following equivalence holds, except for the order of evaluation of -arguments: +.code splice +operator is not an operator +.I per +.IR se . +The +.code splice +symbol has no +binding in the global environment. It is a special syntax that is recognized +within a +.code qquote +form, to indicate forms within the quasiquote which are to be +evaluated and inserted into the resulting structure. +The syntax .cblk - (callf fm f0 f1 f2 ...) <--> (chain (juxt f0 f1 f2 ...) (apf fm)) +.meti (qquote (splice << form )) .cble +is not permitted and raises an exception if evaluated. The +.code splice +syntax must occur within a list, and not in the dotted position. -.TP* Example: - -.cblk - ;; Keep those pairs which are two of a kind - - (keep-if [callf eql first second] '((1 1) (2 3) (4 4) (5 6))) - -> ((1 1) (4 4)) +The +.code splice +form differs from unquote in that +.cblk +.meti (splice << form ) .cble +requires that +.meta form +must evaluate to a list. That list is +integrated into the surrounding list. -.coNP Function @ mapf +.SS* Math Library +.coNP Functions @ + and @ - .synb -.mets (mapf < main-function << arg-function *) +.mets (+ << number *) +.mets (- < number << number *) +.mets (* << number *) .syne .desc The -.code mapf -function returns a function which distributes its arguments -into the -.metn arg-function -s. -That is to say, each successive argument of the returned -function is associated with a successive -.metn arg-function . +.codn + , +.code - +and +.code * +functions perform addition, subtraction and multiplication, +respectively. Additionally, the +.code - +function performs additive inverse. -Each -.metn arg-function -is called, passed the corresponding argument. The return -values of these functions are then passed as arguments -to -.meta main function -and the resulting value is returned. +The +.code + +function requires zero or more arguments. When called with no +arguments, it produces 0 (the identity element for addition), otherwise it +produces the sum over all of the arguments. -If the returned function is called with fewer arguments than there -are -.metn arg-function -s, -then only that many functions are used. Conversely, if the function is -called with more arguments than there are -.metn arg-function -s, then those arguments are ignored. +Similarly, the +.code * +function requires zero or more arguments. When called +with no arguments, it produces 1 (the identity element for multiplication). +Otherwise it produces the product of all the arguments. -The following equivalence holds: +The semantics of +.code - +changes from subtraction to additive inverse +when there is only one argument. The argument is treated as a subtrahend, +against an implicit minuend of zero. When there are two or more +argument, the first one is the minuend, and the remaining are subtrahends. -.cblk - (mapf fm f0 f1 ...) <--> (lambda (. rest) - [apply fm [mapcar call (list f0 f1 ...) rest]]) -.cble +When there are three or more operands, these operations are performed as if by +binary operations, in a left-associative way. That is to say, +.code (+ a b c) +means +.codn (+ (+ a b) c) . +The sum of +.code a +and +.code b +is computed first, and then this is added to +.codn c . +Similarly +.code (- a b c) +means +.codn (- (- a b) c) . +First, +.code b +is subtracted from +.codn a , +and then +.code c +is subtracted from that result. -.TP* Example: +The arithmetic inverse is performed as if it were subtraction from integer 0. +That is, +.code (- x) +means the same thing as +.codn (- 0 x) . -.cblk - ;; Add the squares of 2 and 3 - [[mapf + [dup *] [dup *]] 2 3] -> 13 +The operands of +.codn + , +.code - +and +.code * +can be characters, integers (fixnum and bignum), and +floats, in nearly any combination. -.cble +If two operands have different types, then one of them is converted to the +type of the one with the higher rank, according to this ranking: +character < integer < float. For instance if one operand is integer, and the +other float, the integer is converted to a float. -.SS* Input and Output (Streams) -\*(TL supports input and output streams of various kinds, with -generic operations that work across the stream types. +.TP* Restrictions: -In general, I/O errors are usually turned into exceptions. When the description -of error reporting is omitted from the description of a function, it can be -assumed that it throws an error. +Characters are not considered numbers, and participate in these operations in +limited ways. Subtraction can be used to computed the displacement between the +Unicode values of characters, and an integer displacement can be added to a +character, or subtracted from a character. For instance +.codn (- #\e9 #\e0) is 9 . +The Unicode value of a character +.code C +can be found using +.codn (- C #\ex0) : +the displacement from the NUL character. -.coNP Special variables @, *stdout* @, *stddebug* @, *stdin* @ *stderr* and @ *stdnull* +The rules can be stated as a set of restrictions: +.RS +.IP 1 +Two characters may not be added together. +.IP 2 +A character may not be subtracted from an integer (which also rules out +the possibility of computing the additive inverse of a character). +.IP 3 +A character operand may not be opposite to a floating point operand +in any operation. +.IP 4 +A character may not be an operand of multiplication. +.RE + +.PP + +.coNP Functions @, / @ trunc, @ mod and @ trunc-rem +.synb +.mets (/ <> [ dividend ] << divisor ) +.mets (trunc < dividend << divisor ) +.mets (mod < dividend << divisor ) +.mets (trunc-rem < dividend << divisor ) +.syne .desc -These variables hold predefined stream objects. The -.codn *stdin* , -.code *stdout* -and -.code *stderr* -streams closely correspond to the underlying operating system streams. -Various I/O functions require stream objects as arguments. +The arguments to these functions are numbers. Characters are not permitted. The -.code *stddebug* -stream goes to the same destination as -.codn *stdout* , -but is a separate object which can be redirected independently, allowing -debugging output to be separated from normal output. +.code / +function performs floating-point division. Each operands is first +converted to floating-point type, if necessary. If +.meta dividend +is omitted, +then it is taken to be +.code 1.0 +and the function calculates the reciprocal. The -.code *stdnull* -stream is a special kind of stream called a null stream. -This stream is not connected to any device or file. It is similar to -the -.code /dev/null -device on Unix, but does not involve the operating system. +.code trunc +function performs a division of +.meta dividend +by +.meta divisor +whose result +is truncated to integer toward zero. If both operands are integers, then an +integer division is performed and the result is an integer. If either operand +is a floating point value, a floating point division occurs, and the result is +truncated toward zero to a floating-point integral value. -.coNP Function @ format -.synb -.mets (format < stream-designator < format-string << format-arg *) -.syne -.desc The -.code format -function performs output to a stream given by -.metn stream-designator , -by interpreting the actions implicit in a -.metn format-string , -incorporating material pulled from additional arguments given by -.cblk -.meti << format-arg *. -.cble -Though the function is simple to invoke, there is complexity in format string -language, which is documented below. +.code mod +function performs a modulus operation. Firstly, the absolute value +of +.meta divisor +is taken to be a modulus. Then a residue of +.meta dividend +with respect to +.meta modulus +is calculated. The residue's sign follows +that of the sign of +.metn divisor . +That is, it is the smallest magnitude +(closest to zero) residue of +.meta dividend +with respect to the absolute +value of +.metn divisor , +having the same sign as +.metn divisor . +If the operands are integer, the result is an integer. If either operand +is of type float, then the result is a float. The modulus operation is +then generalized into the floating point domain. For instance the expression +.code (mod 0.75 0.5) +yields a residue of 0.25 because 0.5 "goes into" 0.75 only +once, with a "remainder" of 0.25. The -.meta stream-designator -argument can be a stream object, or one of the values -.code t -or -.codn nil . -The value -.code t -serves as a shorthand for -.codn *stdout* . -The value -.code nil -means that the function will send output into a newly instantiated string -output stream, and then return the resulting string. +.code trunc-rem +function returns a list of two values: a +.meta quotient +and a +.metn remainder . +The +.meta quotient +is exactly the same value as what +.code trunc +would return for the same inputs. +The +.meta remainder +obeys the following identity: + +.cblk +.mets (eql < remainder (- < dividend >> (* divisor << quotient ))) +.cble -.TP* "Format string syntax:" +.coNP Functions @ wrap and @ wrap* +.synb +.mets (wrap < start < end << number ) +.mets (wrap* < start < end << number ) +.syne +.desc +The +.code wrap +and +.code wrap* +functions reduce +.meta number +into the range specified by +.meta start +and +.metn end . -Within -.metn format-string , -most characters represent themselves. Those -characters are simply output. The character -.code ~ -(tilde) introduces formatting -directives, which are denoted by a single character, usually a letter. +Under +.code wrap +the range is inclusive of the +.meta end +value, whereas under +.code wrap* +it is exclusive. -The special sequence -.code ~~ -(tilde-tilde) encodes a single tilde. Nothing is -permitted between the two tildes. +The following equivalence holds -The syntax of a directive is generally as follows: +.cblk + (wrap a b c) <--> (wrap* a (succ b) c) +.cble + +The expression +.code (wrap* x0 x1 x) +performs the following calculation: .cblk -.mets <> ~[ width ] <> [, precision ] < letter +.mets (+ (mod (- x x0) (- x1 x0)) x0) .cble -In other words, the -.code ~ -(tilde) character, followed by a -.meta width -specifier, a -.meta precision -specifier introduced by a comma, -and a -.metn letter , -such that -.meta width +In other words, first +.meta start +is subtracted from +.metn number . +Then the result is reduced modulo the displacement +between +.code start and -.meta precision -are independently optional: either or both may be omitted. -No whitespace is allowed between these elements. +.codn end . +Finally, +.meta start +is added back to that result, which is returned. -The -.meta letter -is a single alphabetic character which determines the -general action of the directive. The optional width and precision -are specified as follows: +.TP* Example: -.RS -.meIP < width -The width specifier consists of an optional -.code < -(left angle bracket) character or -.code ^ -(caret) -character followed by an optional width specification. +.cblk + ;; perform ROT13 on the string "nop" + [mapcar (opip (+ 13) (wrap #\ea #\ez)) "nop"] -> "abc" +.cble -If the leading -.code < -character is present, then the printing will be left-adjusted within -this field. If the -.code ^ -character is present, the printing will be centered within the field. -Otherwise it will be right-adjusted by default. +.coNP Functions @ gcd and @ lcm +.synb +.mets (gcd << number *) +.mets (lcm << number *) +.syne +.desc +The +.code gcd +function computes the greatest common divisor: the largest positive +integer which divides each +.metn number . -The width can be specified as a decimal integer, or as the character -.codn * . The -.code * -notation means that instead of digits, the value of the next argument is -consumed, and expected to be an integer which specifies the width. If that -integer value is negative, then the field will be left-adjusted. -If the value is positive, but either the -.code < -or -.code ^ -prefix character is present in the width -specifier, then the field is adjusted according to that character. +.code lcm +function computes the lowest common multiple: the smallest positive +integer which is a multiple of +each +.metn number . -.meIP < precision -The precision specifier is introduced by a leading comma. If this comma appears -immediately after the directive's -.code ~ -character, then it means that -.meta width -is being omitted; there is only a precision field. +Each +.meta number +must be an integer. -The precision specifier may begin with these optional characters: -.RS -.coIP 0 -(the "leading zero flag"), -.coIP + -(print a sign for positive values") -.IP space -(print a space in place of a positive sign). -.RE +Negative integers are replaced by their absolute values, so +.code (lcm -3 -4) +is +.code 12 +and +.code (gcd -12 -9) +yields +.codn 3 . -The precision specifier itself is either a decimal integer that does not -begin with a zero digit, or the -.code * -character. +The value of +.code (gcd) +is +.code 0 +and that of +.code (lcm) +is 1 . -The precision field's components have a meaning which depends on the type of -object printed and the conversion specifier. +The value of +.code (gcd x) +and +.code (lcm x) +is +.codn (abs x) . -For integer arguments, the precision value specifies the minimum number of digits -to print. If the precision field has a leading zero flag, then the integer is -padded with zeros to the required number of digits, otherwise the number is -padded with spaces instead of zeros. If zero or space padding is present, and -a leading positive or negative sign must be printed, then it is placed before -leading zeros, or after leading spaces, as the case may be. +Any arguments of +.code gcd +which are zero are effectively ignored so that +.code (gcd 0) +and +.code (gcd 0 0 0) +are both the same as +.code (gcd) +and +.code (gcd 1 0 2 0 3) +is the same as +.codn (gcd 1 2 3) . -For floating-point values, the meaning of the precision value depends on which -specific conversion specifier -.cod1 ( f , -.codn e , -.code a -or -.codn s ) -is used. The details are -documented in the description of each of these, below. The leading zero flag is -ignored for floating-point values regardless of the conversion specifier. +If +.code lcm +has any argument which is zero, it yields zero. -For integer or floating-point arguments, if the precision specifier has a -.code + -sign -among the special characters, then a -.code + -sign is printed for positive numbers. If -the precision specifier has a leading space instead of a -.code + -sign, then the -.ocde + -sign is rendered as a space for positive numbers. If there is no leading space -or -.codn + , -then a sign character is omitted for positive numbers. Negative -numbers are unconditionally prefixed with a -.code - -sign. +.coNP Function @ abs +.synb +.mets (abs << number ) +.syne +.desc +The +.code abs +function computes the absolute value of +.metn number . +If +.meta number +is positive, it is returned. If +.meta number +is negative, its additive inverse is +returned: a positive number of the same type with exactly the same magnitude. -For all other objects, the precision specifies the maximum number of characters -to print. The object's printed representation is crudely truncated at that -number of characters. -.RE +.coNP Functions @ floor and @ ceil +.synb +.mets (floor << number ) +.mets (ceil << number ) +.syne +.desc +The +.code floor +function returns the highest integer which does not exceed +the value of +.metn number . +The ceiling function returns the lowest integer which +does not exceed the value of +.metn number . + +If +.meta number +an integer, it is simply returned. + +If the argument is a float, then the value returned is a float. +For instance +.code (floor 1.1) +returns 1.0 rather than 1. -.TP* "Format directives:" -.RS -Format directives are case sensitive, so that for example -.code ~x +.coNP Functions @, sin @, cos @, tan @, asin @, acos @ atan and @ atan2 +.synb +.mets (sin << radians ) +.mets (cos << radians ) +.mets (tan << radians ) +.mets (atan << slope ) +.mets (atan2 < y << x ) +.mets (asin << num ) +.mets (acos << num ) +.syne +.desc +These trigonometric functions convert their argument to floating point and +return a float result. The +.codn sin , +.code cos and -.code ~X -have a -different effect, and -.code ~A -doesn't exist whereas -.code ~a -does. They are: +.code tan +functions compute the sine and +cosine and tangent of the +.meta radians +argument which represents an angle +expressed in radians. The +.codn atan , +.code acos +and +.code asin +are their respective inverse +functions. The +.meta num +argument to +.code asin +and +.code acos +must be in the +range -1.0 to 1.0. The +.code atan2 +function converts the rectilinear coordinates +.meta x +and +.meta y +to an angle in polar coordinates in the range [0, 2\(*p). -.coIP a -Prints any object in an aesthetic way, as if by the -.code pprint -function. -The aesthetic notation violates read-print consistency: this notation -is not necessarily readable if it is implanted in \*(TX source code. -The field width specifier is honored, including the left-right adjustment -semantics. +.coNP Functions @, exp @, log @ log10 and @ log2 +.synb +.mets (exp << arg ) +.mets (log << arg ) +.mets (log10 << arg ) +.mets (log2 << arg ) +.syne +.desc +The +.code exp +function calculates the value of the transcendental number e raised to +the exponent +.metn arg . -When this specifier is used for floating-point values, the precision specifies -the maximum number of total significant figures, which do not include any -digits in the exponent, if one is printed. Numbers are printed in exponential -notation if their magnitude is small, or else if their exponent exceeds their -precision. (If the precision is not specified, then it defaults to the -system-dependent number of digits in a floating point value, derived from the C -language -.code DBL_DIG -constant.) Floating point values which are integers are -printed without a trailing -.code .0 -(point zero). +The +.code log +function calculates the base e logarithm of +.metn arg , +which must be a positive value. -.coIP s -Prints any object in a standard way, as if by the print function. Objects for -which read-print consistency is possible are printed in a way such that -if their notation is implanted in \*(TX source, they are readable. -The field width specifier is honored, including the left-right adjustment -semantics. The precision field is treated very similarly to the -.code ~a -format directive, except that non-exponentiated floating point numbers that -would be mistaken for integers include a trailing -.code .0 -for the sake of read-print -consistency. Objects truncated by precision may not have read-print -consistency. For instance, if a string object is truncated, it loses its -trailing closing quote, so that the resulting representation is no longer -a properly formed string object. +The +.code log10 +function calculates the base 10 logarithm of +.metn arg , +which must be a positive value. -.coIP x -Requires an argument of character or integer type. The integer value or -character code is printed in hexadecimal, using lower-case letters -for the digits -.code a -through -.codn f . -Width and precision semantics -are as described for the -.code a -format directive, for integers. +The +.code log2 +function calculates the base 2 logarithm of +.metn arg , +which must be a positive value. -.coIP X -Like the +.coNP Functions @, expt @ sqrt and @ isqrt +.synb +.mets (expt < base << exponent *) +.mets (sqrt << arg ) +.mets (isqrt << arg ) +.syne +.desc +The +.code expt +function raises +.meta base +to zero or more exponents given +by the +.meta exponent +arguments. +.code (expt x) +is equivalent to +.codn (expt x 1) , +and yields .code x -directive, but the hexadecimal digits -.code a -through -.code f -are rendered in upper case. +for all +.codn x . +For three or more arguments, the operation is right-associative. +That is to say, +.code (expt x y z) +is equivalent to +.codn (expt x (expt y z)) , +similarly to the way nested exponents work in standard algebraic +notation. -.coIP o -Like the -.code x -directive, but octal is used instead of hexadecimal. +Exponentiation is done pairwise using a binary operation. +If both operands to this binary operation are integers, then the +result is an integer. If either operand is a float, then the other +operand is converted to a float, and a floating point exponentiation +is performed. Exponentiation that would produce a complex number is +not supported. -.coIP f The -.code f -directive prints numbers in a fixed point decimal notation, with -a fixed number of digits after the decimal point. It requires a numeric -argument. (Unlike -.codn x , -.code X -and -.codn o , -it does not allow an argument of character type). -The precision specifier gives the number of digits past the decimal point. -The number is rounded off to the specified precision, if necessary. -Furthermore, that many digits are always printed, regardless of the actual -precision of the number or its type. If it is omitted, then the default value -is three: three digits past the decimal point. A precision of zero means no -digits pas the decimal point, and in this case the decimal point is suppressed -(regardless of whether the numeric argument is floating-point or integer). +.code sqrt +function produces a floating-point square root of +.metn arg , +which is converted from integer to floating-point if necessary. Negative +operands are not supported. -.coIP e The -.code e -directive prints numbers in exponential notation. It requires -a numeric argument. (Unlike -.codn x , -.code X -and -.codn o , -it does not allow an argument of character type). -The precision specifier gives the number of digits past the decimal point -printed in the exponential notation, not counting the digits in the exponent. -Exactly that many digits are printed, regardless of the precision of the -number. If the precision is omitted, then the number of digits after the -decimal point is three. If the precision is zero, then a decimal portion is -truncated off entirely, including the decimal point. +.code isqrt +function computes the integer square root of +.metn arg , +which must be an integer. +The integer square root is a value which is the +greatest integer that is no greater than the real square root of +.metn arg . +The input value must be an integer. -.coIP p +.coNP Function @ exptmod +.synb +.mets (exptmod < base < exponent << modulus ) +.syne +.desc The -.code p -directive prints a numeric representation in hexadecimal of the bit pattern -of the object, which is meaningful to someone familiar with the internals -of \*(TX. If the object is a pointer to heaped data, that value -has a correspondence to its address. -.RE +.code exptmod +function performs modular exponentiation and accepts only integer +arguments. Furthermore, +.meta exponent +must be a non-negative and +.meta modulus +must be positive. -.PP +The return value is +.meta base +raised to +.metn exponent , +and reduced to the +least positive residue modulo +.metn modulus . + +.coNP Function @ cum-norm-dist +.synb +.mets (cum-norm-dist << argument ) +.syne +.desc +The +.code cum-norm-dist +function calculates an approximation to the cumulative normal +distribution function: the integral, of the normal distribution function, from +negative infinity to the +.metn argument . -.coNP Functions @, print @, pprint @, prinl @, pprinl @ tostring and @ tostringp +.coNP Functions @ n-choose-k and @ n-perm-k .synb -.mets (print < obj <> [ stream ]) -.mets (pprint < obj <> [ stream ]) -.mets (prinl < obj <> [ stream ]) -.mets (pprinl < obj <> [ stream ]) -.mets (tostring << obj ) -.mets (tostringp << obj ) +.mets (n-choose-k < n << k ) +.mets (n-perm-k < n << k ) .syne .desc The -.code print -and -.code pprint -functions render a printed character representation of the -.meta obj -argument into -.metn stream . -If a stream argument is not supplied, then -the destination is the stream currently stored in the -.code *stdout* -variable. The -.code print -function renders in a way which strives for read-print -consistency: an object is printed in a notation which is recognized as -a similar object of the same kind when it appears in \*(TX source code. -The -.code pprint -function ("pretty print") does not strive for read-print consistency. -For instance it prints a string object simply by dumping its characters, rather -than by adding the surrounding quotes and rendering escape syntax for -special characters. Both functions return -.metn obj . +.code n-choose-k +function computes the binomial coefficient nCk which +expresses the number of combinations of +.meta k +items that can be chosen from +a set of +.metn n , +where combinations are subsets. The -.code prinl +.code n-perm-k +function computes nPk: the number of permutations of size +.meta k +that can be drawn from a set of +.metn n , +where permutations are sequences, +whose order is significant. + +The calculations only make sense when +.meta n and -.code pprinl -functions are like -.code print +.meta k +are nonnegative integers, and +.meta k +does not exceed +.metn n . +The behavior is not specified if these conditions +are not met. + +.coNP Functions @, fixnump @, bignump @, integerp @ floatp and @ numberp +.synb +.mets (fixnump << object ) +.mets (bignump << object ) +.mets (integerp << object ) +.mets (floatp << object ) +.mets (numberp << object ) +.syne +.desc +These functions test the type of +.metn object , +returning +.code t +if it is an object +of the implied type, +.code nil +otherwise. The +.codn fixnump , +.code bignump and -.codn pprint , -except that they issue a newline character after printing the object. -These functions also return -.metn obj . +.code floatp +functions return +.code t +if the object is of the basic type +.codn fixnum , +.code bignum +or +.codn float . +The function +.code integerp +returns true of +.meta object +is either a +.code fixnum +or +a +.codn bignum . +The function +.code numberp +returns +.code t +if +.meta object +is either +a +.codn fixnum , +.code bignum +or +.codn float . +.coNP Function @ zerop +.synb +.mets (zerop << number ) +.syne +.desc The -.code tostring -and -.code tostringp -functions are like -.code print +.code zerop +function tests +.meta number +for equivalence to zero. The argument must be +a number or character. It returns +.code t +for the integer value +.code 0 +and for the floating-point +value +.codn 0.0 . +For other numbers, it returns +.codn nil . +It returns +.code t +for the null character +.code #\enul and -.codn pprint , -but they do not accept a stream argument. Instead they print to a freshly -instantiated string stream, and return the resulting string. +.code nil +for all other characters. -The following equivalences hold between calls to the -.code format -function and calls to the above functions: +.coNP Functions @ plusp and @ minusp +.synb +.mets (plusp << number ) +.mets (minusp << number ) +.syne +.desc +These functions test whether a number is positive or negative, +returning +.code t +or +.codn nil , +as the case may be. -.cblk - (format stream "~s" obj) <--> (print obj stream) - (format t "~s" obj) <--> (print obj) - (format t "~s\en" obj) <--> (prinl obj) - (format nil "~s" obj) <--> (tostring obj) -.cble +The argument may also be a character. All characters other than +the null character +.code #\enul +are positive. No character is negative. -For -.codn pprint , -.code tostringp +.coNP Functions @ evenp and @ oddp +.synb +.mets (evenp << integer ) +.mets (oddp << integer ) +.syne +.desc +The +.code evenp and -.codn pprinl , -the equivalence is produced by using -.code ~a -in format rather than -.codn ~s . +.code oddp +functions require integer arguments. +.code evenp +returns +.code t +if +.meta integer +is even (divisible by two), otherwise it returns +.codn nil . +.code oddp +returns +.code t +if +.meta integer +is not divisible by two (odd), otherwise +it returns +.codn nil . -Note: for characters, the print function behaves as follows: most control -characters in the Unicode -.code C0 -and -.code C1 -range are rendered using the -.code #\ex -notation, -using two hex digits. Codes in the range -.code D800 -to -.codn DFFF , -and the codes -.code FFFE -and -.code FFFF -are printed in the -.code #\exNNNN -with four hexadecimal digits, and -character above this range are printed using the same notation, but with six -hexadecimal digits. Certain characters in the -.code C0 -range are printed using -their names such as -.code #\enul -and -.codn #\ereturn , -which are documented -in the Character Literals section. +.coNP Functions @, succ @, ssucc @, sssucc @, pred @, ppred @ and pppred +.synb +.mets (succ << number ) +.mets (ssucc << number ) +.mets (sssucc << number ) +.mets (pred << number ) +.mets (ppred << number ) +.mets (pppred << number ) +.syne +.desc The -.code DC00 -character is printed as -.codn #\epnul . -All other characters are printed as -.cblk -.meti >> #\e char -.cble -where -.meta char -is the actual character. +.code succ +function adds 1 to its argument and returns the resulting value. +If the argument is an integer, then the return value is the successor +of that integer, and if it is a character, then the return value +is the successor of that character according to Unicode. -Caution: read-print consistency is affected by trailing material. If additional -digits are printed immediately after a number without intervening whitespace, -they extend that number. If hex digits are printed after the character -.codn x , -which is rendered as -.codn #\ex , -they look like a hex character code. +The +.code pred +function subtracts 1 from its argument, and under similar considerations +as above, the result represents the predecessor. -.coNP Function @ tprint +The +.code ssucc +and +.code sssucc +functions add 2 and 3, respectively. Similarly, +.code ppred +and +.code pppred +subtract 2 and 3 from their argument. + +.coNP Functions @, > @, < @, >= @ <= and @ = .synb -.mets (tprint < obj <> [ stream ]) +.mets (> < number << number *) +.mets (> < number << number *) +.mets (>= < number << number *) +.mets (<= < number << number *) +.mets (= < number << number *) .syne .desc +These relational functions compare characters and numbers for numeric equality +or inequality. The arguments must be one or more numbers or characters. -The -.codn tprint -function prints a representation of -.meta obj -on -.metn stream . - -If the stream argument is not supplied, then -the destination is the stream currently stored in the -.code *stdout* -variable. +If just one argument is given, then these functions all return +.codn t . -For all object types except lists and vectors, -.code tprint -behaves like -.codn pprinl . +If two arguments are given then, they are compared as follows. +First, if the numbers do not have the same type, then the one +which has the lower ranking type is converted to the type of +the other, according to this ranking: character < integer < float. +For instance if a character and integer are compared, the character +is converted to integer. Then a straightforward numeric comparison +is applied. -If -.code obj -is a list or vector, then -.code tprint -recurses: the -.code tprint -function is applied to each element. An empty list or vector -results in no output at all. This effectively means that an arbitrarily nested -structure of lists and vectors is printed flattened, with one element on each -line. +Three or more arguments may be given, in which case the comparison proceeds +pairwise from left to right. For instance in +.codn (< a b c) , +the comparison +.code (< a b) +is performed in isolation. If the comparison is false, then +.code nil +is returned, otherwise +the comparison +.code (< b c) +is performed in isolation, and if that is false, +.code nil +is returned, otherwise +.code t +is returned. Note that it is possible for +.code b +to +undergo two different conversions. For instance in the +.cblk +.meti (< < float < character << integer ) +.cble +comparison, +.meta character +will first convert to a floating-point representation +of its Unicode value so that it can be compared to +.metn float , +and if that comparison succeeds, then in the second comparison, +.meta character +will be converted to integer so that it can be compared to +.metn integer . -.coNP Function @ streamp +.coNP Function @ /= .synb -.mets (streamp << obj ) +.mets (/= << number *) .syne .desc -The -.code streamp +The arguments to +.code /= +may be numbers or characters. The +.code /= function returns .code t -if -.meta obj -is any type of stream. Otherwise it returns +if no two of its arguments are numerically equal. That is to say, if there +exist some +.code a +and +.code b +which are distinct arguments such that +.code (= a b) +is true, then +the function returns .codn nil . +Otherwise it returns +.codn t . -.coNP Function @ real-time-stream-p +.coNP Functions @ max and @ min .synb -.mets (real-time-stream-p << obj ) +.mets (max < first-arg << arg *) +.mets (min < first-arg << args *) .syne .desc The -.code real-time-streamp-p -function returns -.code t -if -.meta obj -is a stream marked as -"real-time". If -.meta obj -is not a stream, or not a stream marked as "real-time", -then it returns -.codn nil . +.code max +and +.code min +functions determine and return the highest or lowest +value from among their arguments. -Only certain kinds of streams accept the real-time attribute: file streams and -tail streams. This attribute controls the semantics of the application of -.code lazy-stream-cons -to the stream. For a real-time stream, -.code lazy-stream-cons -returns a stream with "naive" semantics which returns data as soon as it is -available, at the cost of generating spurious -.code nil -item when the stream -terminates. The application has to recognize and discard that +If only +.meta first-arg +is given, that value is returned. + +These functions are type generic, since they compare arguments +using the same semantics as the +.code less +function. + +If two or more arguments are given, then +.code (max a b) +is equivalent to +.codn (if (less a b) b a) , +and +.code (min a b) +is equivalent to +.codn (if (less a b) a b) . +If the operands do not +have the same type, then one of them is converted to the type of the other; +however, the original unconverted values are returned. For instance +.code (max 4 3.0) +yields the integer +.codn 4 , +not +.codn 4.0 . + +If three or more arguments are given, +.code max +and +.code min +reduce the arguments in a left-associative manner. +Thus +.code (max a b c) +means +.codn (max (max a b) c) . + +.coNP Functions @, int-str @ flo-str and @ num-str +.synb +.mets (int-str < string <> [ radix ]) +.mets (flo-str << string ) +.mets (num-str << string ) +.syne +.desc +These functions extract numeric values from character string +.metn string . +Leading whitespace in +.metn string , +if any, is skipped. If no digits can be successfully extracted, then .code nil -item. -The ordinary lazy streams read ahead by one line and suppress this extra -item, so their representation is more accurate. +is returned. Trailing material which does not contribute to the number is +ignored. -When \*(TX starts up, it automatically marks the -.code *std-input* -stream as real-time, if it is connected to a TTY device (a device for which -the POSIX function -.code isatty -reports true). This is only supported on platforms that have this function. -The behavior is overridden by the -.code -n -command line option. +The +.code int-str +function converts a string of digits in the specified +radix to an integer value. If the radix isn't specified, it defaults to 10. +Otherwise it must be an integer in the range 2 to 36. +For radices above 10, letters of the alphabet +are used for digits: +.code A +represent a digit whose value is 10, +.code B +represents 11 and +so forth until +.codn Z . +For values of radix above 36, the returned value is +unspecified. Upper and lower case letters are recognized. +Any character which is not a digit of the specified radix is regarded +as the start of trailing junk at which the extraction of the digits stops. + +The +.code flo-str +function converts a floating-point decimal notation to a nearby +floating point value. The material which contributes to the value +is the longest match for optional leading space, followed by a +mantissa which consists of an optional sign followed by a mixture of at least +one digit, and at most one decimal point, optionally followed by an exponent +part denoted by the letter +.code E +or +.codn e , +an optional sign and one or more optional exponent digits. + +The +.code num-str +function converts a decimal notation to either an integer as if by +a radix 10 application of +.codn int-str , +or to a floating point value as if by +.codn flo-str . +The floating point interpretation is chosen if the possibly empty +initial sequence of digits (following any whitespace and optional sign) is +followed by a period, or by +.code e +or +.codn E . -.coNP Function @ make-string-input-stream +.coNP Functions @ int-flo and @ flo-int .synb -.mets (make-string-input-stream << string ) +.mets (int-flo << float ) +.mets (flo-int << integer ) .syne .desc -This function produces an input stream object. Character read operations on the -stream object read successive characters from -.metn string . -Output operations and byte operations are not supported. +These functions perform numeric conversion between integer and floating point +type. The +.code int-flo +function returns an integer by truncating toward zero. +The +.code flo-int +function returns an exact floating point value corresponding to +.metn integer , +if possible, otherwise an approximation using a nearby +floating point value. -.coNP Function @ make-string-byte-input-stream +.coNP Functions @ tofloat and @ toint .synb -.mets (make-string-byte-input-stream << string ) +.mets (tofloat << value ) +.mets (toint < value <> [ radix ]) .syne .desc -This function produces an input stream object. Byte read operations on -this stream object read successive byte values obtained by encoding -.meta string -into UTF-8. Character read operations are not supported, and neither -are output operations. +These convenience functions convert +.meta value +to floating-point or integer, respectively. -.coNP Function @ make-string-output-stream -.synb - (make-string-output-stream) -.syne -.desc -This function, which takes no arguments, creates a string output stream. -Data sent to this stream is accumulated into a string object. -String output streams supports both character and byte output operations. -Bytes are assumed to represent a UTF-8 encoding, and are decoded in order -to form characters which are stored into the string. +If a floating-point value is passed into tofloat, or an integer value into +toint, then the value is simply returned. -If an incomplete UTF-8 code is output, and a character output operation then -takes place, that code is assumed to be terminated and is decoded as invalid -bytes. The UTF-8 decoding machine is reset and ready for the start of a new -code. +If +.meta value +is a character, then it is treated as a string of length one +containing that character. -The -.code get-string-from-stream -function is used to retrieve the accumulated string. +If +.meta value +is a string, then it is converted by +.code tofloat +as if by the function +.metn flo-str , +, and by +.code toint +as if by the function +.codn int-str . -If the null character is written to a string output stream, either via -a character output operation or as a byte operation, the resulting string -will appear to be prematurely terminated. \*(TX strings cannot contain null -bytes. +If +.meta value +is an integer, then it is converted by +.code tofloat +as if by the function +.codn flo-int . -.coNP Function @ get-string-from-stream -.synb -.mets (get-string-from-stream << stream ) -.syne +If +.meta value +is a floating-point number, then it is converted by +.code toint +as if by the function +.codn int-flo . + +.coNP Variables @, *flo-min* @, *flo-max* and @ *flo-epsilon* .desc -The -.meta stream -argument must be a string output stream. This function finalizes -the data sent to the stream and retrieves the accumulated character string. +These variables hold, respectively: the smallest positive floating-point +value; the largest positive floating-point value; and the difference +between 1.0 and the smallest representable value greater than 1.0. -If a partial UTF-8 code has been written to -.metn stream , -and then this -function is called, the byte stream is considered complete and the partial -code is decoded as invalid bytes. +.code *flo-min* +and +.code *flo-max* +define the floating-point range, which consists +of three regions: values from +.code (- *flo-max*) +to +.codn (- *flo-min*) ; +the value 0.0, and values from +.code *flo-min* +to +.codn *flo-max* . -After this function is called, further output on the stream is not possible. +.coNP Variable @ *flo-dig* +.desc +This variable holds an integer representing the number of decimal digits +in a decimal floating-point number such that this number can be converted +to a \*(TX floating-point number, and back to decimal, without a change in any of +the digits. This holds regardless of the value of the number, provided that it +does not exceed the floating-point range. -.coNP Function @ make-strlist-output-stream +.coNP Variables @ *pi* and @ *e* +.desc +These variables hold an approximation of the mathematical constants \(*p and e. +To four digits of precision, \(*p is 3.142 and e is 2.718. The +.code *pi* +and +.code *e* +approximations are accurate to +.code *flo-dig* +decimal digits. + +.SS* Bit Operations +In \*(TL, similarly to Common Lisp, bit operations on integers are based +on a concept that might be called "infinite two's-complement". +Under infinite two's complement, a positive number is regarded as having +a binary representation prefixed by an infinite stream of zero digits (for +example +.code 1 +is +.codn ...00001 ). +A negative number +in infinite two's complement is the bitwise negation of its positive counterpart, +plus one: it carries an infinite prefix of 1 digits. So for instance the number +.code -1 +is represented by +.codn ...11111111 : +an infinite sequence of +1 +bits. There +is no specific sign bit; any operation which produces such an infinite sequence +of 1 digits on the left gives rise to a negative number. For instance, consider the +operation of computing the bitwise complement of the number +.codn 1 . +Since the +number +.code 1 +is represented as +.codn ...0000001 , +its complement is +.codn ...11111110 . +Each one of the +.cod 0 +digits in the infinite sequence is replaced by +.codn 1 , +And this leading sequence means that the number +is negative, in fact corresponding to the two's-complement representation of +the value +.codn -2 . +Hence, the infinite digit concept corresponds to an arithmetic +interpretation. + +In fact \*(TL's bignum integers do not use a two's complement +representation internally. Numbers are represented as an array which holds a +pure binary number. A separate field indicates the sign: negative, +or non-negative. That negative numbers appear as two's-complement under the +bit operations is merely a carefully maintained illusion (which makes bit +operations on negative numbers more expensive). + +The +.code logtrunc +function, as well as a feature of the +.code lognot +function, allow bit +manipulation code to be written which works with positive numbers only, even if +complements are required. The trade off is that the application has to manage a +limit on the number of bits. + +.coNP Functions @, logand @, logior and @ logxor .synb - (make-strlist-output-stream) +.mets (logand << integer *) +.mets (logior << integer *) +.mets (logxor < int1 << int2 ) .syne .desc -This function is very similar to -.codn make-string-output-stream . -However, the stream object produced by this function does not produce a string, -but a list of strings. The data is broken into multiple strings by newline -characters written to the stream. Newline characters do not appear in the -string list. Also, byte output operations are not supported. +These operations perform the familiar bitwise and, inclusive or, and exclusive +or operations, respectively. Positive values inputs are treated as +pure binary numbers. Negative inputs are treated as infinite-bit +two's-complement. -.coNP Function @ get-list-from-stream +For example +.code (logand -2 7) +produces +.codn 6 . +This is because +.code -2 +is +.code ...111110 +in infinite-bit two's-complement. And-ing this value with +.code 7 +(or +.codn ...000111 ) +produces +.codn 110 . + +The +.code logand +and +.code logior +functions are variadic, and may be called with zero, one, +two, or more input values. If +.code logand +is called with no arguments, it produces +the value -1 (all bits 1). If +.code logior +is called with no arguments it produces +zero. In the one-argument case, the functions just return their argument value. + +.coNP Function @ logtest .synb -.mets (get-list-from-stream << stream ) +.mets (logtest < int1 << int2 ) .syne .desc -This function returns the string list which has accumulated inside -a string output stream given by -.metn stream . -The string output stream is -finalized, so that further output is no longer possible. +The +.code logtest +function returns true if +.meta int1 +and +.meta int2 +have bits in +common. The following equivalence holds: -.coNP Function @ close-stream +.cblk + (logtest a b) <--> (not (zerop (logand a b))) +.cble + +.coNP Functions @ lognot and @ logtrunc .synb -.mets (close-stream < stream <> [ throw-on-error-p ]) +.mets (lognot < value <> [ bits ]) +.mets (logtrunc < value << bits ) .syne .desc The -.code close-stream -function performs a close operation on -.metn stream , -whose meaning is depends on the type of the stream. For some types of streams, -such as string streams, it does nothing. For streams which are connected -to operating system files or devices, will perform a close of the underlying -file descriptor, and dissociate that descriptor from the stream. Any buffered -data is flushed first. - -.code close-stream -returns a boolean true value if the close has occurred without -errors, otherwise -.codn nil . +.code lognot +function performs a bitwise complement of +.metn value . +When the one-argument form of lognot is used, then if +.meta value +is nonnegative, +then the result is negative, and vice versa, according to the infinite-bit +two's complement representation. For instance +.code (lognot -2) +is +.codn 1 , +and +.code (lognot 1) +is +.codn -2 . -For most streams, "without errors" means that any buffered output data is -flushed successfully. +The two-argument form of +.code lognot +produces a truncated complement. Conceptually, +a bitwise complement is first calculated, and then the resulting number is +truncated to the number of bits given by +.metn bits , +which must be a nonnegative integer. The following equivalence holds: -For command and process pipes (see open-command and open-process), success also -means that the process terminates normally, with a successful error code, or an -unsuccessful one. An abnormal termination is considered an error, as -as is the inability to retrieve the termination status, as well as the situation -that the process continues running in spite of the close attempt. -Detecting these situations is platform specific. +.cblk + (lognot a b) <--> (logtrunc (lognot a) b) +.cble -If the -.meta throw-on-error-p -argument is specified, and isn't -.codn nil , -then the -function throws an exception if an error occurs during the close operation -instead of returning -.codn nil . +The +.code logtrunc +function truncates the integer +.meta value +to the specified number +of bits. If +.meta value +is negative, then the two's-complement representation +is truncated. The return value of +.code logtrunc +is always a non-negative integer. -.coNP Functions @, get-error @ get-error-str and @ clear-error +.coNP Function @ sign-extend .synb -.mets (get-error << stream ) -.mets (get-error-str << stream ) -.mets (clear-error << stream ) +.mets (sign-extend < value << bits ) .syne .desc -When a stream operation fails, the -.code get-error -and -.code get-error-str -functions may be used to inquire about a more detailed cause of the error. - -Not all streams support these functions to the same extent. For instance, -string input streams have no persistent state. The only error which occurs -is the condition when the string has no more data. - The -.code get-error -inquires -.meta stream -about its error condition. - -The function returns -.code nil -to indicate there is no error condition, -.code t -to indicate an end-of-data condition, -or else a value which is specific to the stream type indicating the -specific error type. +.code sign-extend +function first truncates the infinite-bit two's complement representation of +the integer +.meta value +to the specified number of bits, similarly to the +.code logtrunc +function. Then, this truncated value is regarded as a +.meta bits +wide two's complement integer. The value of this integer is +calculated and returned. -Note: for some streams, it is possible for the -.code t -value to be returned even though no operation has failed; that is to say, the -streams "know" they are at the end of the data even though no read operation -has failed. Code which depends on this will not work with streams which -do not thus indicate the end-of-data -.I a priori, -but by means of a read operation which fails. +.TP* Examples: -The -.code get-error-str -function returns a text representation of the error code. The -.code nil -error code is represented as the string -.codn "no error" ; -the -.code t -error code as -.code "eof" -and other codes have a stream-specific representation. +.cblk + (sign-extend 127 8) -> 127 + (sign-extend 128 8) -> -128 + (sign-extend 129 8) -> -127 + (sign-extend 255 8) -> -1 + (sign-extend 256 8) -> 0 + (sign-extend -1 8) -> -1 + (sign-extend -255 8) -> 0 +.cble +.coNP Function @ ash +.synb +.mets (ash < value << bits ) +.syne +.desc The -.code clear-error -function removes the error situation from a stream. On some streams, it does -nothing. If an error has occurred on a stream, this function should be called -prior to re-trying any I/O or positioning operations. -The return value is the previous error code, or -.code nil -if there was no error, or the operation is not supported on the stream. +.code ash +function shifts +.meta value +by the specified number of +.meta bits +producing a +new value. If +.meta bits +is positive, then a left shift takes place. If +.meta bits +is negative, then a right shift takes place. If +.meta bit +is zero, then +.meta value +is returned unaltered. For positive numbers, a left shift by n bits is +equivalent to a multiplication by two to the power of n, or +.codn (expt 2 n) . +A right shift by n bits of a positive integer is equivalent to integer +division by +.codn (expt 2 n) , +with truncation toward zero. +For negative numbers, the bit shift is performed as if on the two's-complement +representation. Under the infinite two's-complement representation, +a right shift does not exhaust the infinite sequence of +.code 1 +digits which +extends to the left. Thus if +.code -4 +is shifted right it becomes +.code -2 +because +the bitwise representations of these values are +.code ...111100 +and +.codn ...11110 . -.coNP Functions @, get-line @ get-char and @ get-byte +.coNP Function @ bit .synb -.mets (get-line <> [ stream ]) -.mets (get-char <> [ stream ]) -.mets (get-byte <> [ stream ]) +.mets (bit < value << bit ) .syne .desc -These fundamental stream functions perform input. The -.meta stream -argument -is optional. If it is specified, it should be an input stream which supports -the given operation. If it is not specified, then the -.code *stdin* -stream is used. - The -.code get-char -function pulls a character from a stream which supports character -input. Streams which support character input also support the -.code get-line -function which extracts a line of text delimited by the end of the stream or a -newline character and returns it as a string. (The newline character does not -appear in the string which is returned). - -Character input from streams based on bytes requires UTF-8 decoding, so that -get-char actually may read several bytes from the underlying low level -operating system stream. - +.code bit +function tests whether the integer +.meta value +has a 1 in bit position +.metn bit . The -.code get-byte -function bypasses UTF-8 decoding and reads raw bytes from -any stream which supports byte input. Bytes are represented as integer -values in the range 0 to 255. +.meta bit +argument must be a non-negative integer. A value of zero of +.meta bit +indicates the least significant bit position of +.metn value . -Note that if a stream supports both byte input and character input, then mixing -the two operations will interfere with the UTF-8 decoding. +The +.code bit +function has a boolean result, returning the symbol +.code t +if bit +.meta bit +of +.meta value +is set, otherwise +.codn nil . -These functions return +If +.meta value +is negative, it is treated as if it had an infinite-bit two's +complement representation. For instance, if value is +.codn -2 , +then the bit +function returns .code nil -when the end of data is reached. Errors are -represented as exceptions. - -See also: -.code get-lines +for a +.meta bit +value of zero, and +.code t +for all other values, +since the infinite bit two's complement representation of +.code -2 +is +.codn ...11110 . -.coNP Function @ get-string +.coNP Function @ mask .synb -.mets (get-string >> [ stream >> [ count <> [ close-after-p ]]]) +.mets (mask << integer *) .syne .desc The -.code get-string -function reads characters from a stream, and assembles them into -a string, which is returned. If the -.meta stream -argument is omitted, then the -.code *stdin* -stream is used. +.code mask +function takes zero or more integer arguments, and produces an integer +value which corresponds a bitmask made up of the bit positions specified by the +integer values. -The stream is closed after extracting the data, unless -.meta close-after-p -is specified as -.codn nil . -The default value of this argument is -.codn t . +If +.code mask +is called with no arguments, then the return value is zero. -If the -.meta count -argument is missing, then all of the characters from the -stream are read and assembled into a string. +If +.code mask +is called with a single argument +.meta integer +then the return value is the same as +that of the expression +.codn (ash 1 ) : +the value 1 shifted left by +.meta integer +bit positions. If +.meta integer +is zero, then the result is +.codn 1 ; +if +.meta integer +is +.codn 1 , +the +result is +.code 2 +and so forth. If +.meta value +is negative, then the result is zero. -If present, the -.meta count -argument should be a positive integer indicating -a limit on how many characters to read. The returned string will be no -longer than -.metn count , -but may be shorter. +If +.code mask +is called with two or more arguments, then the result is a bitwise of +the masks individually computed for each of the values. -.coNP Functions @ unget-char and @ unget-byte +In other words, the following equivalences hold: + +.cblk + (mask) <--> 0 + (mask a) <--> (ash 1 a) + (mask a b c ...) <--> (logior (mask a) (mask b) (mask c) ...) +.cble + +.coNP Function @ width .synb -.mets (unget-char < char <> [ stream ]) -.mets (unget-byte < byte <> [ stream ]) +.mets (width << integer *) .syne .desc -These functions put back, into a stream, a character or byte which was -previously read. The character or byte must match the one which was most -recently read. If the -.meta stream -argument is omitted, then the -.code *stdin* -stream is used. - -If the operation succeeds, the byte or character value is returned. -A -.code nil -return indicates that the operation is unsupported. +A two's complement representation of an integer consists of a sign bit and a +mantissa field. +The +.code width +function computes the minimum number of bits required for the mantissa portion +of the two's complement representation of the +.meta integer +argument. -Some streams do not support these operations; some support -only one of them. In general, if a stream supports -.codn get-char , -it supports -.codn unget-char , -and likewise for -.code get-byte -and -.codn unget-byte . +For a nonnegative argument, the width also corresponds to the number of bits +required for a natural binary representation of that value. -Space is available for only one character or byte of pushback. +Two integer values have a width of zero, namely 0 and -1. This means that these +two values can be represented in a one-bit two's complement, consisting of only +a sign bit: the one-bit two's complement bitfield 1 denotes -1, and 0 denotes +0. -Pushing both a byte and a character, in either order, is also unsupported. -Pushing a byte and then reading a character, or pushing a character and -reading a byte, are unsupported mixtures of operations. +Similarly, two integer values have a width of 1: 1 and -2. The two-bit +two's complement bitfield 01 denotes 1, and 10 denotes -2. -If the stream is binary, then pushing back a byte decrements its position, -except if the position is already zero. At that point, the position becomes -indeterminate. +The argument may be a character. -.coNP Functions @, put-string @, put-line @ put-char and @ put-byte +.SS* Exceptions +.coNP Functions @, throw @ throwf and @ error .synb -.mets (put-string < string <> [ stream ]) -.mets (put-line >> [ string <> [ stream ]]) -.mets (put-char < char <> [ stream ]) -.mets (put-byte < byte <> [ stream ]) +.mets (throw < symbol << arg *) +.mets (throwf < symbol < format-string << format-arg *) +.mets (error < format-string << format-arg *) .syne .desc -These functions perform output on an output stream. The -.meta stream -argument -must be an output stream which supports the given operation. If it is omitted, -then -.code *stdout* -is used. - -The -.code put-char -function writes a character given by -.code char -to a stream. If the -stream is based on bytes, then the character is encoded into UTF-8 and multiple -bytes are written. Streams which support -.code put-char -also support put-line, and -.codn put-string . - -The -.code put-string -function writes the characters of a string out to -the stream as if by multiple calls to put-char. The -.meta string -argument -may be a symbol, in which case its name is used as the string. +These functions generate an exception. The +.code throw +and +.code throwf +functions generate +an exception identified by +.metn symbol , +whereas +.code error +throws an exception of +type +.codn error . +The call +.code (error ...) +can be regarded as a shorthand for +.codn (throwf 'error ...) . The -.code put-line -function is like -.codn put-string , -but also writes an additional newline -character. The string is optional in -.codn put-line , -and defaults to the empty string. +.code throw +function takes zero or more additional arguments. These arguments +become the arguments of a +.code catch +handler which takes the exception. The +handler will have to be capable of accepting that number of arguments. The -.code put-byte -function writes a raw byte given by the -.meta byte -argument -to -.metn stream , -if -.meta stream -supports a byte write operation. The byte -value is specified as an integer value in the range 0 to 255. +.code throwf +and +.code error +functions generate an exception which has a single +argument: a character string created by a formatted print to a string stream +using the +.code format +string and additional arguments. -.coNP Functions @ put-strings and @ put-lines +.coNP Operator @ catch .synb -.mets (put-strings < sequence <> [ stream ]]) -.mets (put-lines < sequence <> [ stream ]]) +.mets (catch < try-expression +.mets \ \ >> {( symbol <> ( arg *) << body-form *)}*) .syne .desc -These functions assume -.meta sequence -to be a sequence of strings, or of -symbols, or a mixture thereof. These strings are sent to the stream. The -.meta stream -argument must be an output stream. If it is omitted, then -.code *stdout* -is used. - The -.code put-strings -function iterates over -.meta sequence -and writes each element -to the stream as if using the -.code put-string -function. - +.code catch +operator establishes an exception catching block around +the +.metan try-expression . The -.code put-lines -function iterates over -.code sequence -and writes each element -to the stream as if using the -.code put-line -function. +.meta try-expression +is followed by zero or more +catch clauses. Each catch clause consists of a symbol which denotes +an exception type, an argument list, and zero or more body forms. + +If +.meta try-expression +terminates normally, then the catch clauses +are ignored. The catch itself terminates, and its return value is +that of the +.metn try-expression . + +If +.meta try-expression +throws an exception which is a subtype of one or more of +the type symbols given in the exception clauses, then the first (leftmost) such +clause becomes the exit point where the exception is handled. +The exception is converted into arguments for the clause, and the clause +body is executed. When the clause body terminates, the catch terminates, +and the return value of the catch is that of the clause body. -Both functions return -.codn t . +If +.meta try-expression +throws an exception which is not a subtype of any of +the symbols given in the clauses, then the search for an exit point for +the exception continues through the enclosing forms. The catch clauses +are not involved in the handling of that exception. -.coNP Function @ flush-stream -.synb -.mets (flush-stream << stream ) -.syne -.desc -This function is meaningful for output streams which accumulate data -which is passed on to the operating system in larger transfer units. -Calling -.code flush-stream -causes all accumulated data inside -.meta stream -to be passed -to the operating system. If called on streams for which this function is not -meaningful, it does nothing, and returns +When a clause catches an exception, the number of arguments in the catch must +match the number of elements in the exception. A catch argument list +resembles a function or lambda argument list, and may be dotted. For instance +the clause +.code (foo (a . b)) +catches an exception subtyped from +.codn foo , +with one or +more elements. The first element binds to parameter +.codn a , +and the rest, if any, +bind to parameter +.codn b . +If there is only one element, +.code b +takes on the value .codn nil . -.coNP Function @ seek-stream +Also see: the +.code unwind-protect +operator, and the functions +.codn throw , +.code throwf +and +.codn error . + +.coNP Operator @ unwind-protect .synb -.mets (seek-stream < stream < offset << whence ) +.mets (unwind-protect < protected-form << cleanup-form *) .syne .desc The -.code seek-stream -function is meaningful for file streams. It changes the -current read/write position within -.metn stream . -It can also be used to determine the current position: see the notes about the -return value below. +.cod unwind-protect +operator evaluates +.meta protected-form +in such a way that no matter how the execution of +.meta protected-form +terminates, the +.metn cleanup-form s +will be executed. The -.meta offset -argument is a positive or negative integer which gives a -displacement that is measured from the point identified by the -.meta whence -argument. - -Note that for text files, there isn't necessarily a 1:1 correspondence between -characters and positions due to line-ending conversions and conversions -to and from UTF-8. +.metn cleanup-form s, +however, are not protected. If a +.meta cleanup-form +terminates via +some non-local jump, the subsequent +.metn cleanup-form s +are not evaluated. -The -.meta whence -argument is one of three keywords: :from-start, :from-current -and :from-end. These denote the start of the file, the current position -and the end of the file. +.metn cleanup-form s +themselves can "hijack" a non-local control transfer such +as an exception. If a +.meta cleanup-form +is evaluated during the processing of +a dynamic control transfer such as an exception, and that +.meta cleanup-form +initiates its own dynamic control transfer, the original control transfer +is aborted and replaced with the new one. -If -.meta offset -is zero, and -.meta whence -is -.codn :from-current , -then -.code seek-stream -returns the current absolute position within the -stream, if it can successfully obtain it. Otherwise, it -returns -.code t -if it is successful. +.TP* Example: +.cblk + (block foo + (unwind-protect + (progn (return-from foo 42) + (format t "not reached!\en")) + (format t "cleanup!\en"))) +.cble -If a character has been successfully put back into a text stream with -.code unget-char -and is still pending, then the position value is unspecified. If a -byte has been put back into a binary stream with -.codn unget-byte , -and the previous position wasn't zero, then the position is decremented by one. +In this example, the protected +.code progn +form terminates by returning from +block +.codn foo . +Therefore the form does not complete and so the +output +.str not reached! +is not produced. However, the cleanup form +executes, producing the output +.strn cleanup! . -On failure, it throws an exception of type -.codn stream-error . -.coNP Functions @ stream-get-prop and @ stream-set-prop +.coNP Macro @ ignerr .synb -.mets (stream-get-prop < stream << indicator ) -.mets (stream-set-prop < stream < indicator << value ) +.mets (ignerr << form *) .syne .desc -These functions get and set properties on a stream. Only certain properties -are meaningful with certain kinds of streams, and the meaning depends on -the stream. If two or more stream types support a property of the same name, it -is expected that the property has the same or very similar meaning for both -streams to the maximum extent that similarity is possible. - The -.code stream-set-prop -function sets a property on a stream. The -.meta indicator -argument is a symbol, usually a keyword symbol, denoting the property, -and -.meta value -is the property value. If the stream understands and accepts the -property, the function returns -.codn t . -Otherwise it returns +.code ignerr +macro operator evaluates each +.meta form +similarly to the +.code progn +operator. If no forms are present, it returns .codn nil . +Otherwise it evaluates each +.meta form +in turn, yielding the value of the last one. -The -.code stream-get-prop -function inquires about the value of a property on a -stream. If the stream understands the property, then it returns its current -value. If the stream does not understand a property, nil is returned, which is -also returned if the property exists, but its value happens to be +If the evaluation of any +.meta form +is abandoned due to an exception of type +.codn error , +the code generated by the +.code ignerr +macro catches this exception. In this situation, +the execution of the +.code ignerr +form terminates without evaluating the remaining +forms, and yields .codn nil . -Properties are currently used for marking certain streams as "real-time" (see -the -.code real-time-stream-p -function above), and also for setting the priority at -which messages are reported to syslog by the -.code *stdlog* -stream (see -.code *stdlog* -in the UNIX SYSLOG section). - -If -.meta stream -is a catenated stream (see the function -.codn make-catenated-stream ) -then these functions transparently operate on the current head stream of the -catenation. - -.coNP Functions @ make-catenated-stream and @ cat-streams +.coNP Macro @ with-resources .synb -.mets (make-catenated-stream << stream *) -.mets (cat-streams << stream-list ) +.mets (with-resources >> ({ sym >> [ init-form <> [ cleanup-form ])}*) +.mets \ \ << body-form *) .syne .desc The -.code make-catenated-stream -function takes zero or more arguments which -are input streams of the same type, and combines -them into a single virtual stream called a catenated stream. +.code with-resources +macro provides a sequential binding construct similar to +.codn let* . +Every +.meta sym +is established as a variable which is visible to the +.metn init-form -s +of subsequent variables, to all subsequent +.metn cleanup-form -s +including that of the same variable, +and to the +.metn body-form -s. + +If no +.meta init-form +is supplied, then +.meta sym +is bound to the value +.codn nil . + +If an +.meta init-form +is supplied, but no +.metn cleanup-form , +then +.meta sym +is bound to the value of the +.metn init-form . + +If a +.meta cleanup-form +is supplied in addition to +.metn init-form , +it specifies code to be executed upon the termination of the +entire +.code with-resources +construct. + +When an instance of +.code with-resources +terminates, all of the +.metn cleanup-form -s +specified in its binding clauses are evaluated, in reverse (right-to-left) +order. The value of the last +.meta body-form +is returned, or else +.code nil +if no +.metn body-form -s +are present. -The -.code cat-streams -function takes a single list of input streams of -the same type, and similarly combines them into a catenated stream. +.TP* "Example:" -A catenated stream does not support seeking operations or output, -regardless of the capabilities of the streams in the list. +The following opens a text file and reads a line from it, returning that line, +while ensuring that the stream is closed immediately: -If the stream list is not empty, then the leftmost element of the -list is called the head stream. +.cblk +(with-resources ((f (open-file "/etc/motd") (close-stream f))) + (whilet ((l (get-line f))) + (put-line l))) +.cble + + +.coNP Variable @ *unhandled-hook* The -.codn get-char , -.codn get-byte , -.codn get-line , -.code unget-char -and -.code unget-byte -functions delegate -to the corresponding operations on the head stream, if it exists. -If the stream list is empty, they return +.code *unhandled-hook* +variable is initialized with .code nil -to the caller. +by default. -If the -.codn get-char , -.code get-byte -or -.code get-line -operation on the head stream yields -.codn nil , -and there are more lists in the stream, then the stream is closed, removed from -the list, and the next stream, if any, becomes the head list. The operation is -then tried again. If any of these operations fail on the last list, it is not -removed from the list, so that a stream remains in place which can take the -.code unget-char -or -.code unget-byte -operations. +It may instead be assigned a function which is capable of taking +three arguments. -In this manner, the catenated streams appear to be a single stream. +When an exception occurs which has no handler, this function is called, +with the following arguments: the exception type symbol, the exception object, +and a third value which is either +.code nil +or else the form which was being evaluated the exception was thrown. -Note that the operations can fail due to being unsupported. It is -the caller's responsibility to make sure all of the streams in the list -are compatible with the intended operations. +Otherwise, if the variable is +.code nil +some informational messages are printed about the exception, and the process +exits with a failed termination status. +In the same situation, if the variable contains an object which is not a +function, the process terminates abnormally as if by a call to the +.code abort +function. -If the stream list is empty then an empty catenated stream is produced. -Input operations on this stream yield -.codn nil , -and the -.code unget-char -and -.code unget-byte -operations throw an exception. +Prior to the function being called, the +.code *unhandled-hook* +variable is reset to +.codn nil . -.coNP Function @ catenated-stream-p +If the function registered in +.code *unhandled-hook* +returns, the process exits with a failed termination status. + +Note: the functions +.code source-loc +or +.code source-loc-str +may be applied to the third argument of the +.code *unhandled-hook* +function to obtain more information about the form. + +.SS* Regular Expression Library +.coNP Functions @ search-regex and @ range-regex .synb -.mets (catenated-stream-p << obj ) +.mets (search-regex < string < regex >> [ start <> [ from-end ]]) +.mets (range-regex < string < regex >> [ start <> [ from-end ]]) +.mets (range-regst < string < regex >> [ start <> [ from-end ]]) .syne .desc The -.code catenated-stream-p -function returns -.code t -if -.meta obj -is a catenated stream. Otherwise it returns -.codn nil . +.code search-regex +function searches through +.meta string +starting +at position +.meta start +for a match for +.metn regex . +If +.meta start +is omitted, the search starts at position 0. If +.meta from-end +is specified and has a +.cod2 non- nil +value, the search +proceeds in reverse, from the last position in the string, toward +.metn start . +This function returns +.code nil +if no match is found, otherwise it returns +a cons, whose +.code car +indicates the position of the match, and whose +.code cdr +indicates the length of the match. -.coNP Function @ catenated-stream-push +The +.code range-regex +function is similar to +.codn search-regex , +except that when +a match is found, it returns a position range, rather than a position +and length. A cons is returned whose +.code car +indicates the position +of the match, and whose +.code cdr +indicates the position one element past the +last character of the match. If the match is empty, the two integers +are equal. + +The +.code search-regst +differs from +.code search-regex +in the representation of the return value in the matching case. +Rather than returning the position and length of the match, +it returns the matching substring of +.metn string . + +.coNP Functions @ match-regex and @ match-regst .synb -.mets (catenated-stream-push < new-stream << cat-stream ) +.mets (match-regex < string < regex <> [ position ]) +.mets (match-regst < string < regex <> [ position ]) .syne .desc The -.code catenated-stream-push -function pushes -.meta new-stream -to the front of the stream list inside -.metn cat-stream . +.code match-regex +function tests whether +.meta regex +matches at +.meta position +in +.metn string . +If +.meta position +is not specified, it is taken to be zero. +If the regex matches, then the length of the match is returned. +If it does not match, then +.code nil +is returned. -If an -.code unget-byte -or -.code unget-char -operation was successfully performed on -.meta cat-stream -previously to a call to -.codn catenated-stream-push , -those operations were forwarded to the front stream. -If those bytes or characters are still pending, -they are pending inside that stream, and thus -are logically preceded by the contents -of -.metn new-stream . +The +.code match-regst +differs from +.code match-regex +in the representation of the return value in the matching case. +Rather than returning the length of the match, it returns +matching substring of +.metn string . -.coNP Functions @ open-files and @ open-files* +.coNP Functions @ match-regex-right and @ match-regst-right .synb -.mets (open-files < path-list <> [ alternative-stream ]) -.mets (open-files* < path-list <> [ alternative-stream ]) +.mets (match-regex-right < string < regex <> [ end-position ]) +.mets (match-regst-right < string < regex <> [ end-position ]) .syne .desc The -.code open-files -and -.code open-files* -functions create a list of streams by invoking -the open-file function on each element of -.metn path-list . -These streams are turned -into a catenated stream as if applied as arguments to -.codn make-catenated-stream . - -The effect is that multiple files appear to be catenated together into a single -input stream. +.code match-regex +function tests whether +.meta string +contains a match which ends +precisely on the character just before +.metn end-position . +If +.meta end-position +is not specified, it defaults to the length of the string, and the function +performs a right-anchored regex match. -If the optional -.meta alternative-stream -argument is supplied, then if -.meta path-list -is empty, -.meta alternative-stream -is returned instead of an empty catenated stream. +If a match is found, then the length of the match is returned. -The difference between -.code open-files -and -.code open-files* -is that -.code open-files -creates all of the -streams up-front. So if any of the paths cannot be opened, the operation throws. -The -.code open-files* -variant is lazy: it creates a lazy list of streams out of the -path list. The streams are opened as needed: before the second stream is opened, -the program has to read the first stream to the end, and so on. +The match must terminate just before +.meta end-position +in the sense that +additional characters at +.meta end-position +and beyond can no longer satisfy the +regular expression. More formally, the function searches, starting from +position zero, for positions where there occurs a match for the regular +expression, taking the longest possible match. The length of first such a match +which terminates on the character just before +.meta end-position +is returned. +If no such a match is found, then +.code nil +is returned. -.TP* Example: +The +.code match-regst-right +differs from +.code match-regst-right +in the representation of the return value in the matching case. +Rather than returning the length of the match, it returns +matching substring of +.metn string . -Collect lines from all files that are given as arguments on the command line. If -there are no files, then read from standard input: +.TP* Examples: .cblk - @(next @(open-files *args* *stdin*)) - @(collect) - @line - @(end) -.cble + ;; Return matching portion rather than length thereof. -.coNP Function @ abs-path-p -.synb -.mets (abs-path-p << path ) -.syne -.desc -The -.code abs-path-function -tests whether the argument -.meta path -is an absolute path, returning a -.code t -or -.code nil -indication. + (defun match-regex-right-substring (str reg : end-pos) + (set end-pos (or end-pos (length str))) + (let ((len (match-regex-right str reg end-pos))) + (if len + [str (- end-pos len)..end-pos] + nil))) -An absolute path is a string which either begins with a slash or backslash -character, or which begins with an alphanumeric word, followed by a colon, -followed by a slash or backslash. + (match-regex-right-substring "abc" #/c/) -> "" -Examples of absolute paths: + (match-regex-right-substring "acc" #/c*/) -> "cc" -.cblk - /etc - c:/tmp - ftp://user@server - disk0:/home - Z:\eUsers -.cble + ;; Regex matches starting at multiple positions, but all + ;; the matches extend past the limit. + (match-regex-right-substring "acc" #/c*/ 2) -> nil -Examples of strings which are not absolute paths. + ;; If the above behavior is not wanted, then + ;; we can extract the string up to the limiting + ;; position and do the match on that. + (match-regex-right-substring ["acc" 0..2] #/c*/) -> "c" -.cblk -.mets < (the < empty < string) - . - abc - foo:bar/x - $:\eabc + ;; Equivalent of above call + (match-regex-right-substring "ac" #/c*/) -> "c" .cble -.coNP Function @ read +.coNP Function @ regsub .synb -.mets (read >> [ source >> [ error-stream >> [ error-return-value <> [ name ]]]]) +.mets (regsub < regex < replacement << string ) .syne .desc The -.code read -function converts text denoting \*(TL structure, into the -corresponding data structure. The -.meta source -argument may be either a character -string, or a stream. If it is omitted, then -.code *stdin* -is used as the stream. +.code regsub +function searches +.meta +string +for multiple occurrences of +non-overlapping matches for +.metn regex . +A new string is constructed +similar to +.meta string +but in which each matching region is replaced +with using +.meta replacement +as follows. -The source must provide the text representation of one complete \*(TL object. +The +.meta replacement +object may be a character or a string, in which +case it is simply taken to be the replacement for each match +of the regular expression. -Multiple calls to read on the same stream will extract successive objects -from the stream. To parse successive objects from a string, it is necessary -to convert it to a string stream. +The +.meta replacement +object may be a function of one argument, in +which case for every match which is found, this function is invoked, +with the matching piece of text as an argument. The function's +return value is then taken to be the replacement text. -The optional -.meta error-stream -argument can be used to specify a stream to which -parse errors diagnostics are sent. If absent, the diagnostics are suppressed. +.TP* Examples: -The optional -.meta name -argument can be used to specify the file name which is used for reporting -errors. If this argument is missing, the name is taken from the name -property of the -.meta source -argument if it is a stream, or else the word -.code string -is used as the name if -.meta source -is a string. +.cblk + ;; match every lower case e or o, and replace by filtering + ;; through the upcase-str function: -If there are no parse errors, the function returns the parsed data -structure. If there are parse errors, and the -.meta error-return-value -parameter is -present, its value is returned. If the -.meta error-return-value -parameter -is not present, then an exception of type -.code syntax-error -is thrown. + [regsub #/[eo]/ upcase-str "Hello world!"] -> "HEllO wOrld!" -.SS* Filesystem Access + ;; Replace Hello with Goodbye: + (regsub #/Hello/ "Goodbye" "Hello world!") -> "Goodbye world!" +.cble -.coNP Function @ open-directory +.coNP Function @ regexp .synb -.mets (open-directory << path ) +.mets (regexp << obj ) .syne .desc The -.code open-directory -function tries to create a stream which reads the -directory given by the string argument -.metn path . -If a filesystem object exists -under the path, is accessible, and is a directory, then the function -returns a stream. Otherwise, a file error exception is thrown. - -The resulting stream supports the get-line operation. Each call to the -.code get-line -operation retrieves a string representing the next directory -entry. The value -.code nil -is returned when there are no more directory entries. -The -.code . -and -.code .. -entries in Unix filesystems are not skipped. +.code regexp +function returns +.code t +if +.meta obj +is a compiled regular expression +object. For any other object type, it returns +.codn nil . -.coNP Function @ open-file +.coNP Function @ regex-compile .synb -.mets (open-file < path <> [ mode-string ]) +.mets (regex-compile < form-or-string <> [ error-stream ]) .syne .desc The -.code open-file -function creates a stream connected to the file -which is located at the given -.metn path , -which is a string. +.code regex-compile +function takes the source code of a regular expression, +expressed as a Lisp data structure representing an abstract syntax tree, or +else a regular expression specified as a character string, and compiles it to a +regular expression object. -The -.meta mode-string -argument is a string which uses the same -conventions as the mode argument of the C language -.code fopen -function, with some extensions. -The mode string determines whether the stream is an input stream -or output stream. Note that the -.str b -mode is passed through to the C library, but has no special meaning to \*(TX. -Whether a stream is text or binary depends on which operations -are invoked on it. +If +.meta form-or-string +is a character string, it is parsed to an +abstract syntax tree first, if by the +.code regex-parse +function. +If the parse is successful (the result is not +.codn nil ) +then +the resulting tree structure is compiled by a recursive call to +.codn regex-compile . -If the -.meta mode-string -argument is omitted, mode -.str r -is used. +The optional +.meta error-stream +argument is passed down to +.code regex-parse +as well as in the recursive call to +.codn regex-compile , +if that call takes place. -The option letter -.str i -is supported. If present, it will create a stream which has the real-time -property set. +If +.meta error-stream +is specified, it must be a stream. Any error diagnostics are sent to that +stream. + +.TP* Examples: + +.cblk + ;; the equivalent of #/[a-zA-Z0-9_/ + (regex-compile '(set (#\ea . #\ez) (#\eA . #\eZ) + (#\e0 . #\e9) #\e_)) + + ;; the equivalent of #/.*/ and #/.+/ + (regex-compile '(0+ wild)) + (regex-compile '(1+ wild)) + + ;; #/a|b|c/ + (regex-compile '(or (or #\ea #\eb) #\ec)) + + ;; string + (regex-compile "a|b|c") +.cble -.coNP Function @ open-tail +.coNP Function @ regex-parse .synb -.mets (open-tail < path >> [ mode-string <> [ seek-to-end-p ]]) +.mets (regex-parse < string <> [ error-stream ]) .syne .desc The -.code open-tail -function creates a tail stream connected to the file which is -located at the given -.metn path . -The -.meta mode-string -argument is a string which uses -the same conventions as the mode argument of the C language -.code fopen -function. If this argument is omitted, then -.str r -is used. -See the -.code open-file -function for a discussion of modes. +.code regex-parse +function parses a character string which contains a regular expression +(without any surrounding / characters) and turns it into a Lisp data structure +(the abstract syntax tree representation of the regular expression). -The -.code seek-to-end-p -argument is a boolean which determines whether the initial -read/write position is at the start of the file, or just past the end. -It defaults to +The regular expression syntax +.code #/RE/ +produces the same structure, but as a +literal which is processed at the time \*(TX source code is read; the +.code regex-parse +function performs this parsing at run-time. + +If there are parse errors, the function returns .codn nil . -This argument only makes a difference if the file exists -at the time -.code open-tail -is called. If the file does not exist, and is later -created, then the tail stream will follow that file from the beginning. In -other words, -.meta seek-to-end-p -controls whether the tail stream reads all the -existing data in the file, if any, or whether it reads only newly added data -from approximately the time the stream is created. -A tail stream has special semantics with regard to reading at the end -of file. A tail stream never reports an end-of-file condition; instead -it polls the file until more data is added. Furthermore, if the file -is truncated, or replaced with a smaller file, the tail stream follows -this change: it automatically opens the smaller file and starts reading from -the beginning (the -.meta seek-to-end-p -flag only applies to the initial open). -In this manner, a tail stream can dynamically growing rotating log files. +The optional +.meta error-stream +argument specifies a stream to which error messages +are sent from the parser. By default, diagnostic output goes to the +.code *stdnull* +stream, which discards it. If +.meta error-stream +is specified as +.codn t , +then the diagnostic output goes to the +.code *stdout* +stream. -Caveat: since a tail stream can re-open a new file which has the same -name as the original file, it behave incorrectly if the program -changes the current working directory, and the path name is relative. +If +.code regex-parse +returns a +.cod2 non- nil +value, that structure is then something +which is suitable as input to +.codn regex-compile . -.coNP Function @ remove-path +.SS* Hashing Library +.coNP Functions @, make-hash and @ hash .synb -.mets (remove-path << path ) +.mets (make-hash < weak-keys < weak-vals << equal-based ) +.mets (hash { :weak-keys | :weak-vals | :equal-based }*) .syne .desc -The -.code remove-path -function tries to remove the filesystem object named -by -.metn path , -which may be a file, directory or something else. +These functions construct a new hash table. -If successful, it returns -.codn t . +A hash table is an object which retains an association between pairs of +objects. Each pair consists of a key and value. Given an object which is +similar to a key in the hash table, it is possible to retrieve the +corresponding value. Entries in a hash table are not ordered in any way, and +lookup is facilitated by hashing: quickly mapping a key object to a numeric +value which is then used to index into one of many buckets where the matching +key will be found (if such a key is present in the hash table). -A failure to remove the object results in an exception of type -.codn file-error . +.code make-hash +takes three mandatory boolean arguments. The +.meta weak-keys +argument specifies whether the hash table shall have weak keys. The +.meta weak-vals +argument specifies whether it shall have weak values, and +.meta equal-based +specifies whether it is +.codn equal- based. +The hash function defaults +all three of these properties to false, and allows them to be overridden to +true by the presence of keyword arguments. -.coNP Function @ rename-path +It is an error to attempt to construct an +.codn equal -based +hash table which has weak keys. + +The hash function provides an alternative interface. It accepts optional +arguments which are keyword symbols. Any combination of the three symbols +.codn :weak-keys , +.code :weak-vals +and +.code :equal-based +can be specified in any order +to turn on the corresponding properties in the newly constructed hash table. +If any of the keywords is not specified, the corresponding property defaults to +.codn nil . + +If a hash table has weak keys, this means that from the point of view +of garbage collection, that table holds only weak references to the keys +stored in it. Similarly, if a hash table has weak values, it means that it +holds a weak reference to each value stored. A weak reference is one +which does not prevent the reclamation of an object by the garbage +collector. That is to say, when the garbage collector discovers that the only +references to some object are weak references, then that object is considered +garbage, just as if it had no references to it. The object is reclaimed, and +the weak references "lapse" in some way, which depends on what kind they are. +Hash table weak references lapse by entry removal: if either a key or a value +object is reclaimed, then the corresponding key-value entry is erased from the +hash table. + +Important to the operation of a hash table is the criterion by which keys are +considered same. By default, this similarity follows the eql function. A hash +table will search for a stored key which is +.code eql +to the given search key. +A hash table constructed with the +.codn equal -based +property compares keys using +the +.code equal +function instead. + +In addition to storing key-value pairs, a hash table can have a piece of +information associated with it, called the user data. + +A hash table can be traversed to visit all of the keys and data. The order of +traversal bears no relation to the order of insertion, or to any properties of +the key type. + +During an open traversal, new keys can be inserted into a hash table or deleted +from it while a a traversal is in progress. Insertion of a new key during +traversal will not cause any existing key to be visited twice or to be skipped; +however, it is not specified whether the new key will be traversed. Similarly, +if a key is deleted during traversal, and that key has not yet been visited, it +is not specified whether it will be visited during the remainder of the +traversal. + +An open traversal of a hash table is performed by the +.code maphash +function and the +.code dohash +operator. The traversal is open because code supplied by the program +is evaluated for each entry. + +The functions +.codn hash-keys , +.codn hash-values , +.codn hash-pairs , +and +.code hash-alist also perform an open traversal, because they return +lazy lists. The traversal isn't complete until the returned lazy list +is fully instantiated. In the meanwhile, the +\*(TX program can mutate the hash table from which the lazy list +is being generated. + +.coNP Functions @ hash-construct and @ hash-from-pairs .synb -.mets (rename-path < from-path << to-path ) +.mets (hash-construct < hash-args << key-val-pairs ) +.mets (hash-from-pairs < key-val-pairs << hash-arg *) .syne .desc The -.code remove-path -function tries to rename filesystem path -.metn from-path , -which may refer to a file, directory or something else, to the path -.metn to-path . +.code hash-construct +function constructs a populated hash in one step. The +.meta hash-args +argument specifies a list suitable as an argument list in a call to the hash +function. The +.meta key-val-pairs +is a sequence of pairs, which are two-element +lists representing key-value pairs. -If successful, it returns -.codn t . +A hash is constructed as if by a call to +.cblk +.meti (apply hash << hash-args ), +.cble +then populated +with the specified pairs, and returned. -A failure results in an exception of type -.codn file-error . +The +.code hash-from-pairs +function is an alternative interface to the same semantics. The +.meta key-val-pairs +argument is first, and the +.meta hash-args +are passed as trailing variadic arguments, rather than a single list argument. -.SS* Coprocesses -.coNP Functions @ open-command and @ open-process +.coNP Function @ hash-list .synb -.mets (open-command < system-command <> [ mode-string ]) -.mets (open-process < program < mode-string <> [ argument-list ]) +.mets (hash-list < key-list << hash-arg *) .syne .desc -These functions spawn external programs which execute concurrently -with the \*(TX program. Both functions return a unidirectional stream for -communicating with these programs: either an output stream, or an input -stream, depending on the contents of -.metn mode-string . +The +.code hash-list +function constructs a hash as if by a call to +.cblk +.meti (apply hash << hash-args ), +.cble +where +.meta hash-args +is a list of the individual +.meta hash-arg +variadic arguments. -In -.codn open-command , -the -.meta mode-string -argument is optional, defaulting to -the value -.str r -if it is missing. See the -.code open-file -function for a discussion of modes. +The hash is then populated with keys taken from +.meta key-list +and returned. -The -.code open-command -function accepts, via the -.meta system-command -string parameter, a -system command, which is in a system-dependent syntax. On a POSIX system, this -would be in the POSIX Shell Command Language. +The value associated with each key is that key itself. +.coNP Function @ hash-update +.synb +.mets (hash-update < hash << function ) +.syne +.desc The -.code open-process -function specifies a program to invoke via the -.meta command -argument. This is subject to the operating system's search strategy. -On POSIX systems, if it is an absolute or relative path, it is treated as -such, but if it is a simple base name, then it is subject to searching -via the components of the PATH environment variable. If open-process -is not able to find -.metn program , -or is otherwise unable to execute -the program, the child process will exit, using the value of the C variable -.code errno -as its exit status. This value can be retrieved via -.codn close-stream . +.code hash-update +function replaces each values in +.metn hash , +with the value of +.meta function +applied to that value. -The -.meta mode-string -argument follows the convention used by the POSIX -.code popen -function. +The return value is +.metn hash . +.coNP Function @ hash-update-1 +.synb +.mets (hash-update-1 < hash < key < function <> [ init ]) +.syne +.desc The -.meta argument-list -argument is a list of strings which specifies additional -optional arguments to be passed passed to the program. The -.meta program -argument -becomes the first argument, and -.meta argument-string -become the second and -subsequent arguments. If -.meta argument-strings -is omitted, it defaults to empty. +.code hash-update-1 +function operates on a single entry in the hash table. -If a coprocess is open for writing -.cblk -.meti >> ( mode-string -.cble -is specified as -.strn w ), +If +.meta key +exists in the hash table, then its corresponding value is passed +into +.metn function , +and the return value of +.meta function +is then installed +in place of the key's value. The value is then returned. + +If +.meta key +does not exist in the hash table, and no +.meta init +argument is given, then -writing on the returned stream feeds input to that program's standard input -file descriptor. Indicating the end of input is performed by closing the -stream. +.code hash-update-1 +does nothing and returns +.codn nil . -If a coprocess is open for reading -.cblk -.meti >> ( mode-string -.cble -is specified as -.strn r ), +If +.meta key +does not exist in the hash table, and an +.meta init +argument is given, then -the program's output can be gathered by reading from the returned stream. -When the program finishes output, it will close the stream, which can be -detected as normal end of data. +.meta function +is applied to +.metn init , +and then +.meta key +is inserted into +.meta hash +with the value returned by +.meta function +as the datum. This value +is also returned. -If a coprocess terminates abnormally or unsuccessfully, an exception is raised. +.coNP Function @ group-by +.synb +.mets (group-by < func < sequence << option *) +.syne +.desc +The +.code group-by +function produces a hash table from +.metn sequence , +which is a +list or vector. Entries of the hash table are not elements of +.metn sequence , +but lists of elements of +.metn sequence . +The function +.meta func +is applied to +each element of +.meta sequence +to compute a key. That key is used to determine +which list the item is added to in the hash table. -.SS* Symbols and Packages -A package is an object which serves as a container of symbols. +The trailing arguments +.cblk +.meti << option * +.cble +if any, consist of the same keywords +that are understood by the hash function, and determine the properties +of the hash. -A symbol which exists inside a package is said to be interned in that package. -A symbol can be interned in at most one package at a time. +.TP* Example: +Group the integers from 0 to 10 into three buckets keyed on 0, 1 and 2 +according to the modulo 3 congruence: -Each symbol has a name, which is a string. It is not necessarily unique: -two distinct symbols can have the same name. However, a symbol name is unique -within a package, because it serves as the key which associates the -symbol with the package. Two symbols cannot be in the same package if they -have the same name. Moreover, a symbol cannot exist in more than one package -at at time, although it can be relocated from one package to another. A -symbols exist which is not entered into any package: such a symbol is -called "uninterned". +.cblk + (group-by (op mod @1 3) (range 0 10))) -Packages are held in a global list which can be used to search for a package by -name. The -.code find-package -function performs this lookup. A package may be -deleted from the list with the -.code delete-package -function, but it continues -to exist until the program loses the last reference to that package. + -> #H(() (0 (0 3 6 9)) (1 (1 4 7 10)) (2 (2 5 8))) +.cble -.coNP Special variables @, *user-package* @, *keyword-package* and @ *system-package* +.coNP Functions @ make-similar-hash and @ copy-hash +.synb +.mets (make-similar-hash << hash ) +.mets (copy-hash << hash ) +.syne .desc -These variables hold predefined packages. The -.code *user-package* -is the one -in which symbols are read when a \*(TX program is being scanned. The -.code *keyword-package* -holds keyword symbols, which are printed with -a leading colon. The -.code *system-package* -is for internal symbols, helping -the implementation avoid name clashes with user code in some situations. +.code make-similar-hash +and copy-hash functions create a new hash object based on +the existing +.meta hash +object. -.coNP Function @ make-sym +.code make-similar-hash +produces an empty hash table which inherits all of the +attributes of +.metn hash . +It uses the same kind of key equality, the +same configuration of weak keys and values, and has the same user data (see +the +.code set-hash-userdata +function). + +The +.code copy-hash +function is like +.codn make-similar-hash , +except that instead of +producing an empty hash table, it produces one which has all the same elements +as +.metn hash : +it contains the same key and value objects. + +.coNP Function @ inhash .synb -.mets (make-sym << name ) +.mets (inhash < hash < key <> [ init ]) .syne .desc The -.code make-sym -function creates and returns a new symbol object. The argument -.metn name , -which must be a string, specifies the name of the symbol. The symbol -does not belong to any package (it is said to be "uninterned"). +.code inhash +function searches hash table +.meta hash +for +.metn key . +If +.meta key +is found, then it return the hash table's cons cell which +represents the association between +.meta hash +and +.metn key . +Otherwise, it returns +.codn nil . -Note: an uninterned symbol can be interned into a package with the -.code rehome-sym -function. Also see the -.code intern -function. +If argument +.meta init +is specified, then the function will create +an entry for +.meta key +in +.meta hash +whose value is that of +.metn init . +The cons cell representing that association is returned. + +Note: for as long as the +.meta key +continues to exist inside +.metn hash . +modifying the +.code car +field of the returned cons has ramifications for the logical integrity of +the hash. Modifying the +.code cdr +field has the effect of updating the association with a new value. -.coNP Function @ gensym +.coNP Accessor @ gethash .synb -.mets (gensym <> [ prefix ]) +.mets (gethash < hash < key <> [ alt ]) +.mets (set (gethash < hash < key <> [ alt ]) << new-value ) .syne .desc The -.code gensym -function is similar to make-sym. It creates and returns a new -symbol object. If the -.meta prefix -argument is omitted, it defaults to -.strn g . -Otherwise it must be a string. - -The difference between -.code gensym -and -.code make-sym -is that -.code gensym -creates the name -by combining the prefix with a numeric suffix. - -The numeric suffix is a decimal digit string, taken from the value of -the variable -.codn *gensym-counter* , -after incrementing it. +.code gethash +function searches hash table +.meta hash +for key +.metn key . +If the +key is found then the associated value is returned. Otherwise, if +the +.meta alt +argument was specified, it is returned. If the +.meta alt +argument +was not specified, +.code nil +is returned. -Note: the variation in name is not the basis of the uniqueness assurance -offered by -.code make-sym -and -.codn gensym ; -the basis is that the returned symbol is a freshly instantiated object. -.code make-sym -still returns unique symbols even if repeatedly called with the same -string. +A valid +.code gethash +form serves as a place. It denotes either an existing value in a hash +table or a value that would be created by the evaluation of the form. +The +.meta alt +argument is meaningful when +.code gethash +is used as a place, and, if present, is always evaluated whenever the place is +evaluated. +In place update operations, it provides the initial value, which defaults +to +.code nil +if the argument is not specified. For example +.code (inc (gethash h k d)) +will increment the value stored under key +.code k +in hash table +.code h +by one. If the key does not exist in the hash table, then +the value +.code (+ 1 d) +is inserted into the table under that key. +The expression +.code d +is always evaluated, whether or not its value is needed. -.coNP Special variable @ *gensym-counter* -.desc -This variable is initialized to 0. Each time the -.code gensym -function is called, -it is incremented. The incremented value forms the basis of the numeric -suffix which -.code gensym -uses to form the name of the new symbol. +If a +.code gethash +place is subject to a deletion, but doesn't exist, it is not an error. +The operation does nothing, and +.code nil +is considered the prior value of the place yielded +by the deletion. -.coNP Function @ make-package +.coNP Function @ sethash .synb -.mets (make-package << name ) +.mets (sethash < hash < key << value ) .syne .desc The -.code make-package -function creates and returns a package named -.metn name , -where -.meta name -is a string. It is an error if a package by that name exists already. +.code sethash +function places a value into +.meta hash +table under the given +.metn key . +If a similar key already exists in the hash table, then that key's +value is replaced by +.metn value . +Otherwise, the +.meta key +and +.meta value +pair is +newly inserted into +.metn hash . -.coNP Function @ packagep +The +.code sethash +function returns the +.metn value +argument. + +.coNP Function @ pushhash .synb -.mets (packagep << obj ) +.mets (pushhash < hash < key << element ) .syne .desc The -.code packagep -function returns -.codet -if -.meta obj -is a package, otherwise it returns -.codn nil . +.code pushhash +function is useful when the values stored in a hash table +are lists. If the given +.meta key +does not already exist in +.metn hash , +then a list of +length one is made which contains +.metn element , +and stored in +.meta hash +table under +.metn key . +If the +.meta key +already exists in the hash table, then the corresponding +value must be a list. The +.meta element +value is added to the front of that list, +and the extended list then becomes the new value under +.metn key . -.coNP Function @ find-package +The return value is boolean. If true, indicates that the hash table entry was +newly created. If false, it indicates that the push took place on an existing +entry. + +.coNP Function @ remhash .synb -.mets (find-package << name ) +.mets (remhash < hash << key ) .syne .desc -The argument -.meta name -should be a string. If a package called -.meta name -exists, -then it is returned. Otherwise +The +.code remhash +function searches +.meta hash +for a key similar to the +.metn key . +If that key is found, then that key and its corresponding value are +removed from the hash table. + +If the key is found and removal takes place, then the associated value +is returned. Otherwise .code nil is returned. -.coNP Function @ intern +.coNP Function @ hash-count .synb -.mets (intern < name <> [ package ]) +.mets (hash-count << hash ) .syne .desc -The argument -.meta name -should be a symbol. The optional argument -.meta package -should be a package. If -.meta package -is not supplied, then the value -taken is that of -.codn *user-package* . - The -.code intern -function searches -.meta package -for a symbol called -.metn name . -If that symbol is found, it is returned. If that symbol is not found, -then a new symbol called -.meta name -is created and inserted into -.metn package , -and that symbol is returned. In this case, the package becomes the -symbol's home package. +.code hash-count +function returns an integer representing the number of +key-value pairs stored in +.metn hash . -.coNP Function @ rehome-sym +.coNP Function @ get-hash-userdata .synb -.mets (rehome-sym < symbol <> [ package ]) +.mets (get-hash-userdata << hash ) .syne .desc -The arguments -.meta symbol -and -.meta package -must be a symbol and package object, -respectively. If -.meta package -is not given, then it defaults to the value of -.codn *user-package* . +This function retrieves the user data object associated with +.metn hash . +The user data object of a newly-created hash table is initialized to +.codn nil . +.coNP Function @ set-hash-userdata +.synb +.mets (set-hash-userdata < hash << object ) +.syne +.desc The -.code rehome-sym -function moves -.meta symbol -into -.metn package . -If -.meta symbol -is already in a package, it is first removed from that package. -If a symbol of the same name exists in -.meta package -that symbol is first removed -from -.metn package . +.code set-hash-userdata +replaces, with the +.metn object , +the user data object +associated with +.metn hash . -.coNP Function @ symbolp +.coNP Function @ hashp .synb -.mets (symbolp << obj ) +.mets (hashp << object ) .syne .desc The -.code symbolp +.code hashp function returns .code t -if -.meta obj -is a symbol, otherwise it returns +if the +.meta object +is a hash table, +otherwise it returns .codn nil . -.coNP Function @ symbol-name +.coNP Function @ maphash .synb -.mets (symbol-name << symbol ) +.mets (maphash < hash << binary-function ) .syne .desc The -.code symbol-name -function returns the name of -.metn symbol . +.code maphash +function successively invokes +.meta binary-function +for each entry stored in +.metn hash . +Each entry's key and value are passed as arguments +to +.codn binary-function . -.coNP Function @ symbol-package +The function returns +.codn nil . + +.coNP Functions @ hash-eql and @ hash-equal .synb -.mets (symbol-package << symbol ) +.mets (hash-eql << object ) +.mets (hash-equal << object ) .syne .desc -The -.code symbol-package -function returns the home package of -.metn symbol . +These functions each compute an integer hash value from the internal +representation of +.metn object , +which satisfies the following properties. +If two objects +.code A +and +.code B +are the same under the +.code eql +function, then +.code (hash-eql A) +and +.code (hash-eql B) +produce the same integer hash value. Similarly, +if two objects +.code A +and +.code B +are the same under the +.code equal +function, then +.code (hash-equal A) +and +.code (hash-equal B) +each produce the same integer hash value. In all other +circumstances, the hash values of two distinct objects are unrelated, and +may or may not be the same. -.coNP Function @ packagep +.coNP Functions @, hash_keys @, hash_values @ hash_pairs and @ hash_alist .synb -.mets (packagep << obj ) +.mets (hash-keys << hash ) +.mets (hash-values << hash ) +.mets (hash-pairs << hash ) +.mets (hash-alist << hash ) +.syne +.desc +These functions retrieve the bulk key-value data of hash table +.meta hash +in various ways. +.code hash-keys +retrieves a list of the keys. +.code hash-values +retrieves a list of the values. +.code hash-pairs +retrieves a list of pairs, +which are two-element lists consisting of the key, followed by the value. +Finally, +.code hash-alist +retrieves the key-value pairs as a Lisp association list: +a list of cons cells whose +.code car +fields are keys, and whose +.code cdr +fields are the values. Note that +.code hash-alist +returns the actual entries from the hash table, which are +conses. Modifying the +.code cdr +fields of these conses constitutes modifying the hash values +in the original hash table. Modifying the +.code car +fields interferes with the integrity of the hash table. + +These functions all retrieve the keys and values in the +same order. For example, if the keys are retrieved with +.codn hash-keys , +and the values with +.codn hash-values , +then the corresponding entries from +each list pairwise correspond to the pairs in +.metn hash . + +The list returned by each of these functions is lazy, and hence constitutes +an open traversal of the hash table. + +.coNP Operator @ dohash +.synb +.mets (dohash >> ( key-var < value-var < hash-form <> [ result-form ]) +.mets \ \ << body-form *) .syne .desc The -.code packagep -function returns -.code t -if -.meta obj -is a package, otherwise it returns +.code dohash +operator iterates over a hash table. The +.meta hash-form +expression must +evaluate to an object of hash table type. The +.meta key-var +and +.meta value-var +arguments must be symbols suitable for use as variable names. +Bindings are established for these variables over the scope of the +.metn body-form s +and the optional +.metn result-form . + +For each element in the hash table, the +.meta key-var +and +.meta value-var +variables are set to the key and value of that entry, respectively, +and each +.metn body-form , +if there are any, is evaluated. + +When all of the entries of the table are thus processed, the +.meta result-form +is evaluated, and its return value becomes the return value of the dohash form. +If there is no +.metn result-form , +the return value is .codn nil . -.coNP Function @ keywordp +The +.meta result-form +and +.metn body-form s +are in the scope of an implicit anonymous +block, which means that it is possible to terminate the execution of +dohash early using +.cblk +.meti (return << value ) +.cble +or +.codn (return) . + +.coNP Functions @, hash-uni @ hash-diff and @ hash-isec .synb -.mets (keywordp << obj ) +.mets (hash-uni < hash1 < hash2 <> [ join-func ]) +.mets (hash-diff < hash1 << hash2 ) +.mets (hash-isec < hash1 < hash2 <> [ join-func ]) .syne .desc +These functions perform basic set operations on hash tables in a nondestructive +way, returning a new hash table without altering the inputs. The arguments +.meta hash1 +and +.meta hash2 +must be compatible hash tables. This means that their keys +must use the same kind of equality. + +The resulting hash table inherits attributes from +.metn hash1 , +as if created by the +.code make-similar-hash +function. If +.meta hash1 +has userdata, the resulting hash table +has the same userdata. If +.meta hash1 +has weak keys, the resulting table has weak +keys, and so forth. + The -.code keywordp -function returns -.code t -if -.meta obj -is a keyword symbol, otherwise it -returns -.codn nil . +.code hash-uni +function performs a set union. The resulting hash contains all of +the keys from +.meta hash1 +and all of the keys from +.metn hash2 , +and their corresponding +values. If a key occurs both in +.meta hash1 +and +.metn hash2 , +then it occurs only once +in the resulting hash. In this case, if the +.meta join-func +argument is not given, +the value associated with this key is the one from +.metn hash1 . +If +.meta join-func +is specified then it is called with two arguments: the respective +data items from +.meta hash1 +and +.metn hash2 . +The return value of this function is used +as the value in the union hash. + +The +.code hash-diff +function performs a set difference. First, a copy of +.meta hash1 +is made as if by the +.code copy-hash +function. Then from this copy, all keys which occur +in +.code hash2 +are deleted. + +The +.code hash-isec +function performs a set intersection. The resulting hash contains +only those keys which occur both in +.meta hash1 +and +.metn hash2 . +If +.meta join-func +is not +specified, the values selected for these common keys are those from +.metn hash1 . +If +.meta join-func +is specified, then for each key which occurs in both +.meta hash1 +and +.metn hash2 , +it is called with two arguments: the respective data items. The return +value is then used as the data item in the intersection hash. -.coNP Function @ bindable +.coNP Functions @ hash-subset and @ hash-proper-subset .synb -.mets (bindable << obj ) +.mets (hash-subset < hash1 << hash2 ) +.mets (hash-proper-subset < hash1 << hash2 ) .syne .desc The -.code bindable +.code hash-subset function returns .code t -if -.meta obj -is a bindable symbol, otherwise it returns -.codn nil . +if the keys in +.meta hash1 +are a subset of the keys in +.metn hash2 . -All symbols are bindable, except for keyword symbols, and the -special symbols +The +.code hash-proper-subset +function returns .code t +if the keys in +.meta hash1 +are a proper subset of the keys in +.metn hash2 . +This means that +.meta hash2 +has all the keys which are in +.meta hash1 +and at least one which isn't. + +Note: the return value may not be mathematically meaningful if +.meta hash1 and -.codn nil. +.meta hash2 +use different equality. In any case, the actual behavior +may be understood as follows. The implementation of +.code hash-subset +tests whether each of the keys in +.meta hash1 +occurs in +.meta hash2 +using their respective equalities. +The implementation of +.code hash-proper-subset +applies +.code hash-subset +first, as above. If that is true, and the two hashes have the same number of +elements, the result is falsified. -.SS* Pseudo-random Numbers -.coNP Special variable @ *random-state* +.SS* Partial Evaluation and Combinators +.coNP Macros @ op and @ do +.synb +.mets (op << form +) +.mets (do << form +) +.syne .desc The -.code *random-state* -variable holds an object which encapsulates the state -of a pseudo-random number generator. This variable is the default argument -value for the -.code random-fixnum +.code op and -.codn random functions , -for the convenience of writing programs which are not concerned about the -management of random state. +.code do +macro operators are similar. -On the other hand, programs can create and manage random states, making it -possible to obtain repeatable sequences of pseudo-random numbers which do not -interfere with each other. For instance objects or modules in a program can -have their own independent streams of random numbers which are repeatable, -independently of other modules making calls to the random number functions. +Like the lambda operator, the +.code op +operator creates an anonymous function based on its syntax. +The difference is that the arguments of the function are implicit, or +optionally specified within the function body, rather than as a formal +parameter list before the body. -When \*(TX starts up, the -.code *random-state* -variable is initialized with -a newly created random state object, which is produced as if by -the call -.codn (make-random-state 42) . +Also, the +.meta form +arguments of +.code op +are implicitly turned into a DWIM expression, +which means that argument evaluation follows Lisp-1 rules. (See the +.code dwim +operator). -.coNP Function @ make-random-state -.synb -.mets (make-random-state <> [ seed ]) -.syne -.desc The -.code make-random-state -function creates and returns a new random state, -an object of the same kind as what is stored in the -.code *random-state* -variable. +.code do +operator is like the +.code op +operator with the following difference: +the +.meta form +arguments of +.code op +are not implicitly treated as a DWIM expression, +but as an ordinary expression. In particular, this means that operator +syntax is permitted. Note that the syntax +.code (op @1) +makes sense, since +the argument can be a function, which will be invoked, but +.code (do @1) +doesn't +make sense because it will produce a Lisp-2 form like +.code (#:arg1 ...) +referring +to nonexistent function +.codn #:arg1 . +Because it accepts operators, +.code do +can be used with imperative constructs +which are not functions, like set: like set: for instance +.code (do set x) +produces +an anonymous function which, if called with one argument, stores that argument +into +.codn x . -The seed, if specified, must be either an integer value, or an -existing random state object. +The argument forms are arbitrary expressions, within which a special +convention is permitted: +.RS +.meIP >> @ num +A number preceded by a +.code @ +is a metanumber. This is a special syntax +which denotes an argument. For instance +.code @2 +means that the second argument of +the anonymous function is to be substituted in place of the +.codn @2 . +.code op +generates a function which has a number of required arguments equal to the +highest value of +.meta num +appearing in a +.cblk +.mati >> @ num +.cble +construct in the body. For instance +.code (op car @3) +generates a three-argument function (which passes its third +argument to +.codn car , +returning the result, and ignores its first two arguments). +There is no way to use +.code op +to generate functions which have optional arguments. +.meIP < @rest +If the meta-symbol +.meta @rest +appears in the +.code op +syntax, it explicitly denotes the list of trailing arguments, +allowing them to be placed anywhere in the expression. +.RE -Note that the sign of the seed is ignored, so that negative seed -values are equivalent to their additive inverses. +.IP +Functions generated by +.code op +are always variadic; they always take additional arguments after +any required ones, whether or not the +.meta @rest +syntax is used. -If the seed is not specified, then -.code make-random-state -produces a seed based -on some information in the process environment, such as current -time of day. It is not guaranteed that two calls to -.code (make-random-state) -that are separated by less than some minimum increment of real time produce -different seeds. The minimum time increment depends on the platform. +If the body does not contain +any +.meta @num +or +.meta @rest +syntax, then +.code @rest +is implicitly inserted. What this means is that, for example, since +the form +.code (op foo) +does not contain any numeric positional arguments like +.codn @1 , +and does not contain +.codn @rest , +it is actually a shorthand for +.codn (op foo . @rest) : +a function which applies all of its arguments to +.codn foo . -On a platform with a millisecond-resolution real-time clock, the minimum -time increment is a millisecond. Calls to make-random-state less than -a millisecond apart may predictably produce the same seed. +The actions of +.code op +be understood by these examples, which show +how +.code op +is rewritten to lambda. However, note that the real translator +uses generated symbols for the arguments, which are not equal to any +symbols in the program. + +.cblk + (op) -> invalid + + (op +) -> (lambda rest [+ . rest]) + + (op + foo) -> (lambda rest [+ foo . rest]) + + (op @1 @2) -> (lambda (arg1 arg2 . rest) [arg1 arg2]) + + (op @1 . @rest) -> (lambda (arg1 . rest) [arg1 . @rest]) + + (op @1 @rest) -> (lambda (arg1 . rest) [arg1 @rest]) + + (op @1 @2) -> (lambda (arg1 arg2 . rest) [arg1 arg2]) + + (op foo @1 (@2) (bar @3)) -> (lambda (arg1 arg2 arg3 . rest) + [foo arg1 (arg2) (bar arg3)]) + + (op foo @rest @1) -> (lambda (arg1 . rest) [foo rest arg1]) + + (do + foo) -> (lambda rest (+ foo . rest)) + + (do @1 @2) -> (lambda (arg1 arg2 . rest) (arg1 arg2)) + + (do foo @rest @1) -> (lambda (arg1 . rest) (foo rest arg1)) +.cble + +Note that if argument +.meta @n +appears, it is not necessary +for arguments +.meta @1 +through +.meta @n-1 +to appear. The function will have +.code n +arguments: -If an integer seed is specified, then the integer value is mapped to a -pseudo-random sequence, in a platform-independent way. +.cblk + (op @3) -> (lambda (arg1 arg2 arg3 . rest) [arg3]) +.cble -If a random state is specified as a seed, then it is duplicated. The -returned random state object is a distinct object which is in the same -state as the input object. It will produce the same remaining pseudo-random -number sequence, as will the input object. +The +.code op +and +.code do +operators can be nested, in any combination. This raises the +question: if a metanumber like +.code @1 +or +.code @rest +occurs in an +.code op +that is nested +within an +.codn op , +what is the meaning? -.coNP Function @ random-state-p -.synb -.mets (random-state-p << obj ) -.syne -.desc +A metanumber always belongs with the inner-most op or do operator. So for +instance +.code (op (op @1)) +means that an +.code (op @1) +expression is nested +within an +.code op +expression which itself contains no meta-syntax. The -.code random-state-p -function returns -.code t -if -.meta obj -is a random state, otherwise it -returns -.codn nil . +.code @1 +belongs with the inner op. -.coNP Functions @, random-fixnum @ random and @ rand -.synb -.mets (random-fixnum <> [ random-state ]) -.mets (random < random-state << modulus ) -.mets (rand < modulus <> [ random-state ]) -.syne -.desc -All three functions produce pseudo-random numbers, which are positive integers. +There is a way for an inner +.code op +to refer to an outer op metanumber argument. This is +expressed by adding an extra +.code @ +prefix for every level of escape. For example in +.code (op (op @@1)) +the +.code @@1 +belongs to the outer +.codn op : +it is the same as +.code @1 +appearing in the outer +.codn op . +That is to say, +in the expression +.codn (op @1 (op @@1)) , +the +.code @1 +and +.code @@1 +are the same thing: +both are parameter 1 of the lambda function generated by the outer +.codn op . +By contrast, in the expression +.code (op @1 (op @1)) +there are two different parameters: +the first +.code @1 +is argument of the outer function, and the second +.code @1 +is the first argument of the inner function. Of course, if there +are three levels of nesting, then three +.code @ +meta-prefixes are needed to insert +a parameter from the outermost +.code op +into the innermost +.codn op . -The numbers are obtained from a WELLS 512 pseudo-random number generator, whose -state is stored in the random state object. +.TP* Examples: -The -.code random-fixnum -function produces a random fixnum integer: a reduced range -integer which fits into a value that does not have to be heap-allocated. +.cblk + ;; Take a list of pairs and produce a list in which those pairs + ;; are reversed. -The -.code random -and -.code rand -functions produce a value in the range [0, -.metn modulus ). -They differ only in the order of arguments. In the -.code rand -function, the random state -object is the second argument and is optional. If it is omitted, the global -.code *random-state* -object is used. + (mapcar (op list @2 @1) '((1 2) (a b))) -> ((2 1) (b a)) +.cble -.SS* Time -.coNP Functions @ time and @ time-usec -.synb - (time) - (time-usec) -.syne -.desc The -.code time -function returns the number of seconds that have elapsed since -midnight, January 1, 1970, in the UTC timezone. +.code op +syntax interacts with quasiliterals which are nested within it. +The metanumber notation as well as +.code @rest +are recognized without requiring an additional +.code @ +escape: -The -.code time-usec -function returns a cons cell whose -.code car -field holds the seconds measured in the same way, and whose -.code cdr -field extends the precision by giving -number of microseconds as an integer value between 0 and 999999. +.cblk + (apply (op list `@1-@rest`) '(1 2 3)) -> "1-2 3" -.coNP Functions @ time-string-local and @ time-string-utc -.synb -.mets (time-string-local < time << format ) -.mets (time-string-utc < time << format ) -.syne -.desc -These functions take the numeric time returned by the -.code time -function, and convert it to a textual representation in a flexible way, -according to the contents of the -.meta format -string. + (apply (op list `@@1-@@rest`) '(1 2 3)) -> "1-2 3" +.cble -The -.code time-string-local -function converts the time to the local timezone of -the host system. The -.code time-string-utc -function produces time in UTC. +This is because the +.code op +macro traverses the code structure produced by the literal without recognizing +it specially, and there imposes its own meaning on these elements. -The -.meta format -argument is a string, and follows exactly the same conventions as -the format string of the C library function -.codn strftime . +Though they produce the same result, the above two examples differ in that +.code @rest +embeds a metasymbol into the quasiliteral structure, whereas +.code @@rest +embeds the Lisp expression +.code @rest +into the quasiliteral. In ordinary circumstances, the former refers to the +variable +.codn rest . +Contrast the previous example with: -The -.meta time -argument is an integer representing seconds obtained from the -time function or from the -.code car -field of the cons returned by the -.code time-usec -function. +.cblk + (let ((rest "0")) + `rest: @rest`) -> "rest: 0" -.coNP Functions @ time-fields-local and @ time-fields-utc -.synb -.mets (time-fields-local << time ) -.mets (time-fields-utc << time ) -.syne -.desc -These functions take the numeric time returned by the time function, -and convert it to a list of seven fields. + (let ((rest "0")) + `rest: @@rest`) -> ;; error: no such function or operator: sys:var +.cblk -The -.code time-string-local -function converts the time to the local timezone of -the host system. The -.code time-string-utc -function produces time in UTC. +Under the +.code op +macro and its relatives, occurrences of +.code @rest +are replaced with syntax which refers to the trailing arguments +of the anonymous function. This happens before the interior of the +.code op +syntax undergoes expansion. Therefore the quasiliteral expander never +sees the +.codn @rest . -The fields returned as a list consist of six integers, and a boolean value. -The six integers represent the year, month, day, hour, minute and second. -The boolean value indicates whether daylight savings time is in effect -(always -.code nil -in the case of -.codn time-fields-utc ). +This convenient omission of the +.codn @ +character isn't supported for reaching the arguments of an outer +.code op +from a quasiliteral within a nested +.codn op : -The -.meta time -argument is an integer representing seconds obtained from the -.code time -function or from the -.code time-usec -function. +.cblk + ;; To reach @@1, @@@1 must be written. + ;; @@1 Lisp expression introduced by @. + (op ... (op ... `@@@1`)) +.cble -.coNP Functions @ make-time and @ make-time-utc +.coNP Macros @, ap @, ip @ ado and @ ido. .synb -.mets (make-time < year < month < day < hour < minute < second << dst-advice ) -.mets (make-time-utc < year < month < day < hour < minute < second << dst-advice ) +.mets (ap << form +) +.mets (ip << form +) +.mets (ado << form +) +.mets (ido << form +) .syne .desc The -.code make-time -function returns a time value, similar to the one returned by the -.code time -function. The -.code time -value is constructed not from the system clock, but -from a date and time specified as arguments. The -.meta year -argument is a calendar year, like 2014. -The -.meta month -argument ranges from 1 to 12. -The -.meta hour -argument is a 24-hour time, ranging from 0 to 23. -These arguments represent a local time, in the current time zone. +.code ap +macro is based on the +.code op +macro and has identical argument +conventions. The -.meta dst-advice -argument specifies whether the time is expressed in -daylight savings time (DST). It takes on three possible values: -.codn nil , -the keyword -.codn :auto , -or else the symbol -.codn t . -Any other value has the same interpretation as -.codn t . +.code ap +macro analyzes its arguments and produces a function +.metn f , +in exactly the same same way as the +.code op +macro. However, instead of returning +.metn f , +directly, it returns a different function +.metn g , +which is a one-argument function which accepts a list, +and then applies the list as arguments to +.metn f . -If -.meta dst-advice -is -.codn t , -then the time is assumed to be expressed in DST. -If the argument is -.codn nil , -then the time is assumed not to be in DST. -If -.meta dst-advice -is -.codn :auto , -then the function tries to determine whether -DST is in effect in the current time zone for the specified date and time. +In other words, the following equivalence holds: + +.cblk + (ap form ...) <--> (apf (op form ...)) +.cble The -.code make-time-utc -function is similar to -.codn make-time , -except that -it treats the time as UTC rather than in the local time zone. -The -.meta dst-advice -argument is supported by -.code make-time-utc -for function -call compatibility with -.codn make-time . -It may or may not have any effect -on the output (since the UTC zone by definition doesn't have daylight -savings time). +.code ap +macro nests properly with +.code op +and +.codn do , +in any combination, in regard to the +.meta ...@@n +notation. -.SS* Environment Variables and Command Line +The +.ode ip +macro is very similar to the +.code ap +macro, except that it is based +on the semantics of the function +.code iapply +rather than +.codn apply , +according +to the following equivalence: -Note that environment variable names, their values, and command line -arguments are all regarded as being externally encoded in UTF-8. \*(TX performs -the encoding and decoding automatically. +.cblk + (ip form ...) <--> (ipf (op form ...)) +.cble -.coNP Special variables @ *args* and @ *args-full* -.desc The -.code *args* -variable holds a list of strings representing the remaining -arguments which follow any options processed by the \*(TX executable, -and the script name. +.code ado +and +.code ido +macros are related to do macro in the same way that +.code ap +and +.code ip +are related to +.codn op . +They produce a one-argument function which works +as if by applying its arguments to the function generated by do, +according to the following equivalence: -The -.code *args-full* -variable holds the original, complete list of arguments passed -from the operating system. +.cblk + (ado form ...) <--> (apf (do form ...)) -Note: the -.code *args* -variable is -.code nil -during the processing of the command line, -so \*(TL expressions invoked using the -.code -p -or -.code -e -option cannot use it. + (ido form ...) <--> (ipf (do form ...)) +.cblk -.coNP Function @ env +See also: the +.code apf +and +.code ipf +functions. + +.coNP Macros @ opip and @ oand .synb - (env) +.mets (opip << clause *) +.mets (oand << clause *) .syne .desc The -.code env -function retrieves the list of environment variables. Each -variable is represented by a single entry in the list: a string which -contains an -.code = -(equal) character somewhere, separating the variable name -from its value. +.code opip +and +.code oand +operators make it possible to chain together functions which are expressed +using the +.code op +syntax. (See the +.code op +operator for more information). -See also: the -.code env-hash +Both macros perform the same transformation except that +.code opip +translates its arguments to a call to the +.code chain +function, whereas +.code oand +translates its arguments in the same way to a call to the +.code chand function. -.coNP Function @ env-hash -.synb - (env-hash) -.syne -.desc -The -.code env-hash -function constructs and returns an -.code :equal-based -hash. The hash is -populated with the environment variables, represented as key-value pairs. +More precisely, these macros perform the following rewrites: -.coNP Functions @, getenv @, setenv and @ unsetenv -.synb -.mets (getenv << name ) -.mets (setenv < name < value <> [ overwrite-p ]) -.mets (unsetenv << name ) -.syne -.desc -These functions provide access to, as well as manipulation of, environment -variables. Of these three, -.code setenv -and -.code unsetenv -might not be available on some platforms, or -.code unsetenv -might be be present in a simulated form which sets the variable -.meta name -to the empty string rather than deleting it. +.cblk + (opip arg1 arg2 ... argn) -> [chain {arg1} {arg2} ... {argn}] + (oand arg1 arg2 ... argn) -> [chand {arg1} {arg2} ... {argn}] +.cble -The -.code getenv -function searches the environment for the environment variable whose name -is -.metn name . -If the variable is found, its value is returned. Otherwise -.code nil -is returned. +where the above +.code {arg} +notation denotes the following transformation applied to each argument: -The -.code setenv -function creates or modifies the environment variable indicated by -.metn name . -The -.meta value -string argument specifies the new value for the variable. +.cblk + (function ...) -> (op function ...) + (operator ...) -> (do operator ...) + (macro ...) -> (do macro ...) + (dwim ...) -> (dwim ...) + [...] -> [...] + atom -> atom +.cble -If the -.meta overwrite-p -argument is specified, and is true, -then the variable is overwritten if it already exists. -If the argument is false, then the variable is not modified if it -already exists. If the argument is not specified, it defaults -to the value -.metn t, -effectively giving rise to a two-argument form of -.code setenv -which creates or overwrites environment variables. +In other words, compound forms whose leftmost symbol is a macro or operator +are translated to the +.code do +notation. Compound forms denoting function calls are translated to the +.code op +notation. Compound forms which are +.code dwim +invocations, either explicit or via the DWIM brackets notation, are +preserved, as are any forms which are atoms. -The -.code setenv -function unconditionally returns -.meta value -regardless of whether or not it overwrites an existing variable. +Note: the +.code opip +and +.code oand +macros use their macro environment in determining whether a form is a +macro call, thereby respecting lexical scoping. -The -.code unsetenv -function removes the environment variable -specified by -.metn name , -if it exists. On some platforms, it instead sets the environment variable -to the empty string. +.TP* Example: +Take each element from the list +.code (1 2 3 4) +and multiply it by three, then add 1. +If the result is odd, collect that into the resulting list: -.SS* System Programming -.coNP Accessor @ errno -.synb -.mets (errno <> [ new-errno ]) -.mets (set (errno) << new-value ) -.syne -.desc -The -.code errno -function retrieves the current value of the C library error variable -.codn errno . -If the argument -.meta new-errno -is present and is not -.codn nil , -then it -specifies a value which is stored into -.codn errno . -The value returned is the prior value. +.cblk +(mappend (opip (* 3) + (+ 1) + [iff oddp list]) + (range 1 4)) +.cble -The place form of -.code errno -does not take an argument. +The above is equivalent to: + +.cblk +(mappend (chain (op * 3) + (op + 1) + [iff oddp list]) + (range 1 4)) +.cble -.coNP Function @ exit -.synb -.mets (exit << status ) -.syne -.desc The -.code exit -function terminates the entire process (running \*(TX image), specifying -the termination status to the operating system. Values of -.meta status -may be -.codn nil , -.codn t , -or an integer value. The value -.code nil -corresponds to the C constant -.codn EXIT_FAILURE , +.code (* 3) and -.code t -corresponds to -.codn EXIT_SUCCESS . -These are platform-independent -indicators of failed or successful termination. The numeric value 0 also -indicates success. +.code (+ 1) +terms are rewritten to +.code (op * 3) +and +.codn (op + 1) , +respectively, whereas +.code [iff oddp list] +is passed through untransformed. -.coNP Function @ abort +.coNP Macro @ ret .synb -.mets (abort) +.mets (ret << form ) .syne .desc The -.code abort -function terminates the entire process (running \*(TX image), specifying -an abnormal termination status to the process. - -Note: -.code abort -calls the C library function -.code abort -which works by raising the -.code SIG_ABRT -signal, known in \*(TX as the -.code sig-abrt -variable. Abnormal termination of the process is this signal's -default action. +.code ret +macro's +.meta form +argument is treated similarly to the second and subsequent arguments of the +.code op +operator. -.coNP Function @ usleep -.synb -.mets (usleep << usec ) -.syne -.desc The -.code usleep -function suspends the execution of the program for at least -.meta usec -microseconds. +.code ret +macro produces a function which takes any number of arguments, +and returns the value specified by +.metn form . -The return value is -.code t -if the sleep was successfully executed. A -.code nil -value indicates premature wakeup or complete failure. +.meta form +can contain +.code op +meta syntax like +.code @n +and +.codn @rest . -Note: the actual sleep resolution is not guaranteed, and depends on granularity -of the system timer. Actual sleep times may be rounded up to the nearest 10 -millisecond multiple on a system where timed suspensions are triggered by a 100 -Hz tick. +The following equivalence holds: -.coNP Functions @ mkdir and @ ensure-dir +.cblk + (ret x) <--> (op identity x)) +.cble + +Thus the expression +.code (ret @2) +returns a function similar to +.codn (lambda (x y . z) y) , +and the expression +.code (ret 42) +returns a function similar to +.codn (lambda (. rest) 42) . + +.coNP Macro @ aret .synb -.mets (mkdir < path <> [ mode ]) -.mets (ensure-dir < path <> [ mode ]) +.mets (aret << form ) .syne .desc -.code mkdir -tries to create the directory named -.meta path -using the POSIX -.code mkdir -function. -An exception of type -.code file-error -is thrown if the function fails. Returns -.code t -on success. +The +.code ret +macro's +.meta form +argument is treated similarly to the second and subsequent arguments of the +.code op +operator. The -.meta mode -argument specifies the request numeric permissions -for the newly created directory. If omitted, the requested permissions are -.code #o777 -(511): readable and writable to everyone. The requested permissions -are subject to the system -.codn umask . +.code aret +macro produces a function which takes any number of arguments, +and returns the value specified by +.metn form . -The function -.code ensure-dir -is similar to -.code mkdir -except that it attempts to create all the missing parent directories -as necessary, and does not throw an error if the directory exists. +.meta form +can contain +.code ap +meta syntax like +.meta @n +and +.codn @rest . -.coNP Function @ chdir -.synb -.mets (chdir << path ) -.syne -.desc -.code chdir -changes the current working directory to -.metn path , -and returns -.metn t , -or else throws an exception of type -.codn file-error . +The following equivalence holds: -.coNP Function @ pwd +.cblk + (aret x) <--> (ap identity x)) +.cble + +Thus the expression +.code (aret @2) +returns a function similar to +.codn (lambda (. rest) (second rest)) , +and the expression +.code (aret 42) +returns a function similar to +.codn (lambda (. rest) 42) . + +.coNP Function @ dup .synb - (pwd) +.mets (dup << func ) .syne .desc The -.code pwd -function retrieves the current working directory. -If the underlying -.code getcwd -C library function fails with an -.code errno -other than -.codn ERANGE , -an exception will be thrown. +.code dup +function returns a one-argument function which calls the two-argument +function +.metn func +by duplicating its argument. + +.TP* Example: + +.cblk + ;; square the elements of a list + (mapcar [dup *] '(1 2 3)) -> (1 4 9) +.cble -.coNP Functions @ sh and @ run +.coNP Function @ flipargs .synb -.mets (sh << system-command ) -.mets (run < program <> [ argument-list ]) +.mets (flipargs << func ) .syne .desc The -.code sh -function executes -.meta system-command -using the system command interpreter. -The run function spawns a -.metn program , -searching for it using the -system PATH. Using either method, the executed process receives environment -variables from the parent. +.code flipargs +function returns a two-argument function which calls the two-argument +function +.metn func +with reversed arguments. -\*(TX blocks until the process finishes executing. If the program terminates -normally, then its integer exit status is returned. The value zero indicates -successful termination. +.coNP Functions @ chain and @ chand +.synb +.mets (chain << func *) +.mets (chand << func *) +.syne +.desc +The +.code chain +function accepts zero or more functions as arguments, and returns +a single function, called the chained function, which represents the chained +application of those functions, in left to right order. -The return value +If +.code chain +is given no arguments, then it returns a variadic function which +ignores all of its arguments and returns +.codn nil . + +Otherwise, the first function may accept any number of arguments. The second +and subsequent functions, if any, must accept one argument. + +The chained function can be called with an argument list which is acceptable +to the first function. Those arguments are in fact passed to the first +function. The return value of that call is then passed to the second +function, and the return value of that call is passed to the third function +and so on. The final return value is returned to the caller. + +The +.code chand +function is similar, except that it combines the functionality of +.code andf +into chaining. The difference between +.code chain +and +.code chand +is that +.code chand +immediately terminates and returns .code nil -indicates an abnormal termination, or the inability -to run the process at all. +whenever any of the functions returns +.codn nil , +without calling the remaining functions. -In the case of the -.code run -function, if the child process is created successfully -but the program cannot be executed, then the exit status will be an -.code errno -value from the failed -.code exec -attempt. +.TP* Example: -.SS* Unix Filesystem Manipulation +.cblk + (call [chain + (op * 2)] 3 4) -> 14 +.cble -.coNP Function @ stat +In this example, a two-element chain is formed from the +.code + +function +and the function produced by +.code (op * 2) +which is a one-argument +function that returns the value of its argument multiplied by two. +(See the definition of the +.code op +operator). + +The chained function is invoked using the +.code call +function, with the arguments +.code 3 +and +.codn 4 . +The chained evaluation begins by passing +.code 3 +and +.code 4 +to +.codn + , +which yields +.codn 7 . +This +.code 7 +is then passed to the +.code (op * 2) +doubling function, resulting in +.codn 14 . + +A way to write the above example without the use of the DWIM brackets and the +op operator is this: + +.cblk + (call (chain (fun +) (lambda (x) (* 2 x))) 3 4) +.cble + +.coNP Function @ juxt .synb -.mets (stat << path ) +.mets (juxt << func *) .syne .desc The -.code stat -function inquires the filesystem about the existence of an object -denoted by the string -.metn path . -If the object is not found or cannot be -accessed, an exception is thrown. +.code juxt +function accepts a variable number of arguments which are functions. It +combines these into a single function which, when invoked, passes its arguments +to each of these functions, and collects the results into a list. -Otherwise, information is retrieved about the object. The information takes the -form of a property list in which keyword symbols denote numerous properties. -An example such property list is: +Note: the juxt function can be understood in terms of the following reference +implementation: .cblk - (:dev 2049 :ino 669944 :mode 16832 :nlink 23 - :uid 500 :gid 500 :rdev 0 - :size 12288 :blksize 4096 :blocks 24 - :atime 1347933533 :mtime 1347933534 :ctime 1347933534) + (defun juxt (funcs) + (lambda (. args) + (mapcar (lambda (fun) + (apply fun args)) + funcs))) .cble -These properties correspond to the similarly-named entries of the -.code struct stat -structure in POSIX. For instance, the -.code :dev -property has the same value as the -.code st_dev -field. +.TP* Example: -.coNP Special variables @, s-ifmt @, s-iflnk @, s-ifreg @, s-ifblk ... , @ s-ixoth +.cblk + ;; separate list (1 2 3 4 5 6) into lists of evens and odds, + ;; which end up juxtaposed in the output list: -The following variables exist, having integer values. These are bitmasks -which can be applied against the value given by the -.code :mode -property -in the property list returned by the function stat: -.codn s-ifmt , -.codn s-ifsock , -.codn s-iflnk , -.codn s-ifreg , -.codn s-ifblk , -.codn s-ifdir , -.codn s-ifchr , -.codn s-ififo , -.codn s-isuid , -.codn s-isgid , -.codn s-isvtx , -.codn s-irwxu , -.codn s-irusr , -.codn s-iwusr , -.codn s-ixusr , -.codn s-irwxg , -.codn s-irgrp , -.codn s-iwgrp , -.codn s-ixgrp , -.codn s-irwxo , -.codn s-iroth , -.code s-iwoth + [(op [juxt keep-if remove-if] evenp) + '(1 2 3 4 5 6)] -> ((2 4 6) (1 3 5)) + + ;; call several functions on 1, collecting their results: + [[juxt (op + 1) (op - 1) evenp sin cos] 1]' + -> (2 0 nil 0.841470984807897 0.54030230586814) +.cble + +.coNP Functions @ andf and @ orf +.synb +.mets (andf << func *) +.mets (orf << func *) +.syne +.desc +The +.code andf and -.codn s-ixoth . +.code orf +functions are the functional equivalent of the +.code and +and +.code or +operators. These functions accept multiple functions and return a new function +which represents the logical combination of those functions. -These variables correspond to the C language constants from POSIX: -.codn S_IFMT , -.codn S_IFLNK , -.code S_IFREG -and so forth. +The input functions should have the same arity. Failing that, there should +exist some common argument arity with which each of these can be invoked. The +resulting combined function is then callable with that many arguments. The -.code logtest -function can be used to test these against values of mode. -For example -.code (logtest mode s-irgrp) -tests for the group read permission. +.code andf +function returns a function which combines the input functions with +a short-circuiting logical conjunction. The resulting function passes its +arguments to the functions successively, in left to right order. As soon as any +of the functions returns +.codn nil , +then nil is returned immediately, and the +remaining functions are not called. Otherwise, if none of the functions return +.codn nil , +then the value returned by the last function is returned. If the list of +functions is empty, then +.code t +is returned. That is, +.code (andf) +returns a function +which accepts any arguments, and returns +.codn t . + +The +.code orf +function combines the input functions with a short-circuiting logical +disjunction. The function produced by +.code orf +passes its arguments down to the +functions successively, in left to right order. As soon as any function +returns a +.cod2 non- nil +value, that value is returned and the remaining functions are +not called. If all functions return +.codn nil , +then +.code nil +is returned. The expression +.code (orf) +returns a function which accepts any arguments and returns +.codn nil . -.coNP Functions @, makedev @ minor and @ major +.coNP Function @ notf .synb -.mets (makedev < minor << major ) -.mets (minor << dev ) -.mets (major << dev ) +.mets (notf << function ) .syne .desc -The parameters -.metn minor , -.meta major -and -.meta dev -are all integers. The -.code makedev -function constructs a combined device number from a minor and major pair (by -calling the Unix -.code makedev -function). This device number is suitable as an -argument to the -.codee mknod -function (see below). Device numbers also appear the -.code :dev -property returned by the -.code stat -function. - The -.code minor -and -.code major -functions extract the minor and major device number -from a combined device number. +.code notf +function returns a function which is the boolean negation +of +.metn function . -.coNP Function @ mknod +The returned function takes a variable number of arguments. When +invoked, it passes all of these arguments to +.meta function +and then inverts the result as if by application of the +.codn not . + +.coNP Functions @ iff and @ iffi .synb -.mets (chmod < path << mode ) +.mets (iff < cond-func >> [ then-func <> [ else-func ]]) +.mets (iffi < cond-func < then-func <> [ else-func ]) .syne .desc The -.code chmod -function changes the permissions of the filesystem objects -specified by -.metn path . -It is a direct wrapper for the POSIX C library function of the same name. +.code iff +function is the functional equivalent of the +.code if +operator. It accepts +functional arguments and returns a function. -The permissions are specified by -.metn mode , -an integer argument. +The resulting function takes its arguments, if any, and applies them to +.metn cond-func . +If +.meta cond-func +yields true, then the arguments are passed to +.meta then-func +and the +resulting value is returned. Otherwise the arguments are passed to +.meta else-func +and the resulting value is returned. -The existing permissions may be obtained using the -.code stat -function. +If +.meta then-func +is omitted then +.code identity +is used as default. This omission is not permitted by +.codn iffi , +only +.codn iff . -The function throws a -.code file-error -exception if an error occurs, otherwise it returns -.codn t. +If +.meta else-func +needs to be called, but is omitted, then +.code nil +is returned. + +The +.code iffi +function differs from +.code iff +only in the defaulting behavior with respect +to the +.meta else-func +argument. If +.meta else-func +is omitted in a call to +.code iffi +then the default function is +.codn identity . +This is useful in situations when one value is to be +replaced with another one when the condition is true, otherwise +preserved. + +The following equivalences hold between +.code iffi +and +.codn iff : -.TP* Example: .cblk - ;; Set permissions of foo.txt to "rw-r--r--" - ;; (owner can read and write; group owner - ;; and other users can only read). + (iffi a b c) <--> (iff a b c) - ;; numerically: - (chmod "foo.txt" #o644) + (iffi a b) <--> (iff a b identity) - ;; symbolically: - (chmod "foo.txt" (logior s-irusr s-iwusr - s-irgrp - s-iroth)) + (iffi a b false) <--> (iff a b) + + (iffi a identity false) <--> (iff a) .cble -.coNP Function @ mknod +The following equivalence illustrates +.code iff +with both optional arguments omitted: + +.cblk + (iff a) <---> (iff a identity false) +.cble + +.coNP Functions @ tf and @ nilf .synb -.mets (mknod < path < mode <> [ dev ]) +.mets (tf << arg *) +.mets (nilf << arg *) .syne .desc The -.code mknod -function tries to create an entry in the filesystem: a file, -FIFO, or a device special file, under the name -.metn path . -If it is successful, -it returns -.codn t , -otherwise it throws an exception of type -.codn file-error . - +.code tf +and +.code nilf +functions take zero or more arguments, and ignore them. The -.meta mode -argument is a bitwise or combination of the requested permissions, -and the type of object to create: one of the constants -.codn s-ifreg , -.codn s-ififo , -.codn s-ifchr , -.code s-ifblk -or -.codn s-ifsock . -The permissions are subject to the system -.codn umask . +.code tf +function returns +.codn t , +and the +.code nilf +function returns +.codn nil . -If a block or character special device -.cod2 ( s-ifchr -or -.codn s-ifblk ) -is being -created, then the -.meta dev -argument specifies the major and minor numbers -of the device. A suitable value can be constructed from a major and minor -pair using the -.code makedev +Note: the following equivalences hold between these functions and the +.code ret +operator, and +.code retf function. +.cblk + (fun tf) <--> (ret t) <--> (retf t) + (fun nilf) <--> (ret nil) <--> (ret) <--> (retf nil) +.cble + +In Lisp-1-style code, +.code tf +and +.code nilf +behave like constants which can replace uses of +.code (ret t) +and +.codn (ret nil) : + +.cblk + [mapcar (ret nil) list] <--> [mapcar nilf list] +.cble + .TP* Example: .cblk - ;; make a character device (8, 3) called /dev/foo - ;; requesting rwx------ permissions + ;; tf and nilf are useful when functions are chained together. + ;; test whether (trunc n 2) is odd. + + (defun trunc-n-2-odd (n) + [[chain (op trunc @1 2) [iff oddp tf nilf]] n]) +.cble + +In this example, two functions are chained together, and +.code n +is passed +through the chain such that it is first divided by two via the +function denoted by +.code (op trunc @1 2) +and then the result is passed into the +function denoted by +.codn [iff oddp tf nilf] . +The +.code iff +function passes its argument into +.codn oddp , +and if +.code oddp +yields true, it passes the same argument to +.codn tf . +Here +.code tf +proves its utility by ignoring that value and returning +.codn t . +If the argument (the divided value) passed into +.code iff +is even, then iff passes it into the +.code nilf +function, which ignores the value and returns +.codn nil . + +.coNP Function @ retf +.synb +.mets (retf << value ) +.syne +.desc +The +.code retf +function returns a function. That function can take zero or +more arguments. When called, it ignores its arguments and returns +.metn value . + +See also: the +.code ret +macro. + +.TP* Example: - (mknod "dev/foo" (logior #o700 s-ifchr) (makedev 8 3)) +.cblk + ;; the function returned by (retf 42) + ;; ignores 1 2 3 and returns 42. + (call (retf 42) 1 2 3) -> 42 .cble -.coNP Functions @ symlink and @ link +.coNP Functions @ apf and @ ipf .synb -.mets (symlink < target << path ) -.mets (link < target << path ) +.mets (apf << function ) +.mets (ipf << function ) .syne .desc The -.code symlink -function creates a symbolic link called -.meta path -whose contents -are the absolute or relative path -.metn target . -.meta target -does not actually have to exist. +.code apf +function returns a one-argument function which accepts +a list. When the function is called, it treats the list as +arguments which are applied to +.meta function +as if by apply. It returns whatever +.meta function +returns. -The link function creates a hard link. The object at -.meta target -is installed -into the filesystem at -.meta path -also. +The +.code ipf +function is similar to +.codn apf , +except that the returned +function applies arguments as if by +.code iapply +rather than +.codn apply . -If these functions succeed, they return -.codn t . -Otherwise they throw an exception -of type -.codn file-error . +See also: the +.code ap +macro. -.coNP Function @ readlink +.TP* Example: + +.cblk + ;; Function returned by [apf +] accepts the + ;; (1 2 3) list and applies it to +, as + ;; if (+ 1 2 3) were called. + + (call [apf +] '(1 2 3)) -> 6 +.cble + +.coNP Function @ callf .synb -.mets (readlink << path ) +.mets (callf < main-function << arg-function *) .syne .desc -If -.meta path -names a filesystem object which is a symbolic link, the -.code readlink -function reads the contents of that symbolic link and returns it -as a string. Otherwise, it fails by throwing an exception of type -.codn file-error . - -.SS* Unix Signal Handling - -On platforms where certain advanced features of POSIX signal handling are -available at the C API level, \*(TX exposes signal-handling functionality. +The +.code callf +function returns a function which applies its arguments to each +.metn arg-function , +juxtaposing the return values of these calls to form arguments +which are then passed to +.metn main-function . +The return value of +.meta main-function +is returned. -A \*(TX program can install a \*(TL function (such as an anonymous. -.codn lambda , -or the function object associated with a named function) as the handler for -a signal. +The following equivalence holds, except for the order of evaluation of +arguments: -When that signal is delivered, \*(TX will intercept it with its own safe, -internal handler, mark the signal as deferred (in a \*(TX sense) and then -dispatch the registered function at a convenient time. +.cblk + (callf fm f0 f1 f2 ...) <--> (chain (juxt f0 f1 f2 ...) (apf fm)) +.cble -Handlers currently are not permitted to interrupt the execution of most -\*(TX internal code. Immediate, asynchronous execution of handlers is -currently enabled only while \*(TX is blocked on I/O operations or sleeping. -Additionally, the -.code sig-check -function can be used to dispatch and clear deferred -signals. These handlers are then safely called if they were subroutines of -.codn sig-check , -and not asynchronous interrupts. +.TP* Example: -.coNP Special variables @, sig-hup @, sig-int @, sig-quit @, sig-ill @, sig-trap @, sig-abrt @, sig-bus @, sig-fpe @, sig-kill @, sig-usr1 @, sig-segv @, sig-usr2 @, sig-pipe @, sig-alrm @, sig-term @, sig-chld @, sig-cont @, sig-stop @, sig-tstp @, sig-ttin @, sig-ttou @, sig-urg @, sig-xcpu @, sig-xfsz @, sig-vtalrm @, sig-prof @, sig-poll @, sig-sys @, sig-winch @, sig-iot @, sig-stkflt @, sig-io @ sig-lost and @ sig-pwr -.desc -These variables correspond to the C signal constants -.codn SIGHUP , -.code SIGINT -and so forth. -The variables -.codn sig-winch , -.codn sig-iot , -.codn sig-stk flt, -.codn sig-io , -.code sig-lost -and -.code sig-pwr -may not be available since a system may lack the corresponding signal -constants. See notes for the function -.codn log-authpriv . +.cblk + ;; Keep those pairs which are two of a kind -The highest signal number is 31. + (keep-if [callf eql first second] '((1 1) (2 3) (4 4) (5 6))) + -> ((1 1) (4 4)) +.cble -.coNP Functions @ set-sig-handler and @ get-sig-handler +.coNP Function @ mapf .synb -.mets (set-sig-handler < signal-number << handling-spec ) -.mets (get-sig-handler << signal-number ) +.mets (mapf < main-function << arg-function *) .syne .desc The -.code set-sig-handler -function is used to specify the handling for a signal, such -as the installation of a handler function. It updates the signal handling for -a signal whose number is -.meta signal-number -(usually one of the constants like -.codn sig-hup , -.code sig-int -and so forth), and returns the previous value. The -.code get-sig-handler -function returns the current value. - -The -.meta signal-number -must be an integer the range 1 to 31. - -Initially, all 31 signal handling specifications are set to the value -.codn t . - -The -.meta handling-spec -parameter may be a function. If a function is specified, -then the signal is enabled and connected to that function until another -call to -.code set-sig-handler -changes the handling for that signal. - -If -.meta handling-spec -is the symbol -.codn nil , -then the function previously associated -with the signal, if any, is removed, and the signal is disabled. For a signal -to be disabled means that the signal is set to the -.code SIG_IGN -disposition (refer to the C API). +.code mapf +function returns a function which distributes its arguments +into the +.metn arg-function -s. +That is to say, each successive argument of the returned +function is associated with a successive +.metn arg-function . -If -.meta handling-spec -is the symbol -.codn t , -then the function previously associated -with the signal, if any, is removed, and the signal is set to its default -disposition. This means that it is set to -.code SIG_DFL -(refer to the C API). -Some signals terminate the process if they are generated while the -handling is configured to the default disposition. +Each +.metn arg-function +is called, passed the corresponding argument. The return +values of these functions are then passed as arguments +to +.meta main function +and the resulting value is returned. -Note that the certain signals like -.code sig-quit -and -.code sig-kill -cannot be ignored or handled. -Please observe the signal documentation in the IEEE POSIX standard, and your -platform. +If the returned function is called with fewer arguments than there +are +.metn arg-function -s, +then only that many functions are used. Conversely, if the function is +called with more arguments than there are +.metn arg-function -s, then those arguments are ignored. -A signal handling function must take two arguments. It is of the form: +The following equivalence holds: .cblk -.mets (lambda >> ( signal << async-p ) ...) + (mapf fm f0 f1 ...) <--> (lambda (. rest) + [apply fm [mapcar call (list f0 f1 ...) rest]]) .cble -The -.meta signal -argument is an integer indicating the signal number for which the -handler is being invoked. The -.meta asyncp-p -argument is a boolean value. -If it is -.codn t , -it indicates that the handler is being invoked -asynchronously\(emdirectly in a signal handling context. If it is -.codn nil , -then it -is a deferred call. Handlers may do more things in a deferred call, such -as terminate by throwing exceptions, and perform I/O. +.TP* Example: -The return value of a handler is normally ignored. However if it invoked -asynchronously (the -.meta async-p -argument is true), then if the handler returns -a -.cod2 non- nil -value, it is understood that the handler -requesting that it be deferred. This means that the signal will be marked -as deferred, and the handler will be called again at some later -time in a deferred context, whereby -.meta async-p -is -.codn nil . -This is not guaranteed, however; -it's possible that another signal will arrive before that happens, -possibly resulting in another async call, so the handler must -be prepared to deal with an async call at any time. +.cblk + ;; Add the squares of 2 and 3 + [[mapf + [dup *] [dup *]] 2 3] -> 13 -If a handler is invoked synchronously, then its return value is ignored. +.cble + +.SS* Input and Output (Streams) +\*(TL supports input and output streams of various kinds, with +generic operations that work across the stream types. -In the current implementation, signals do not queue. If a signal is delivered -to the process again, while it is marked as deferred, it simply stays deferred; -there is no counter associated with a signal, only a boolean flag. +In general, I/O errors are usually turned into exceptions. When the description +of error reporting is omitted from the description of a function, it can be +assumed that it throws an error. -.coNP Function @ sig-check -.synb - (sig-check) -.syne +.coNP Special variables @, *stdout* @, *stddebug* @, *stdin* @ *stderr* and @ *stdnull* .desc +These variables hold predefined stream objects. The +.codn *stdin* , +.code *stdout* +and +.code *stderr* +streams closely correspond to the underlying operating system streams. +Various I/O functions require stream objects as arguments. + The -.code sig-check -function tests whether any signals are deferred, and for each -deferred signal in turn, it executes the corresponding handler. For a signal to -be deferred means that the signal was caught by an internal handler in -\*(TX and the event was recorded by a flag. If a handler function is removed -while a signal is deferred, the deferred flag is cleared for that signal. +.code *stddebug* +stream goes to the same destination as +.codn *stdout* , +but is a separate object which can be redirected independently, allowing +debugging output to be separated from normal output. -Calls to the -.code sig-check -function may be inserted into CPU-intensive code that -has no opportunity to be interrupted by signals, because it doesn't invoke any -I/O functions. +The +.code *stdnull* +stream is a special kind of stream called a null stream. +This stream is not connected to any device or file. It is similar to +the +.code /dev/null +device on Unix, but does not involve the operating system. -.coNP Function @ kill +.coNP Function @ format .synb -.mets (kill < process-id <> [ signal ]) +.mets (format < stream-designator < format-string << format-arg *) .syne .desc The -.code kill -function is used for sending a signal to a process group or process. -It is a wrapper for the POSIX -.code kill -function. +.code format +function performs output to a stream given by +.metn stream-designator , +by interpreting the actions implicit in a +.metn format-string , +incorporating material pulled from additional arguments given by +.cblk +.meti << format-arg *. +.cble +Though the function is simple to invoke, there is complexity in format string +language, which is documented below. -If the -.meta signal -argument is omitted, it defaults to the same value as -.codn sig-term . +The +.meta stream-designator +argument can be a stream object, or one of the values +.code t +or +.codn nil . +The value +.code t +serves as a shorthand for +.codn *stdout* . +The value +.code nil +means that the function will send output into a newly instantiated string +output stream, and then return the resulting string. -.SS* Unix Processes +.TP* "Format string syntax:" -.coNP Functions @ fork and @ wait -.synb -.mets (fork) -.mets (wait >> [ pid <> [ flags ]]) -.syne -.desc -The -.code fork -and -.code wait -functions are interfaces to the Unix functions -.code fork +Within +.metn format-string , +most characters represent themselves. Those +characters are simply output. The character +.code ~ +(tilde) introduces formatting +directives, which are denoted by a single character, usually a letter. + +The special sequence +.code ~~ +(tilde-tilde) encodes a single tilde. Nothing is +permitted between the two tildes. + +The syntax of a directive is generally as follows: + +.cblk +.mets <> ~[ width ] <> [, precision ] < letter +.cble + +In other words, the +.code ~ +(tilde) character, followed by a +.meta width +specifier, a +.meta precision +specifier introduced by a comma, +and a +.metn letter , +such that +.meta width and -.codn waitpid . +.meta precision +are independently optional: either or both may be omitted. +No whitespace is allowed between these elements. The -.code fork -function creates a child process which is a replica of the parent. Both -processes return from the function. In the child process, the return value is -zero. In the parent, it is an integer representing the process ID of the child. -If the function fails to create a child, it returns -.code nil -rather than an integer. In this case, the -.code errno -function can be used to inquire about the cause. +.meta letter +is a single alphabetic character which determines the +general action of the directive. The optional width and precision +are specified as follows: -The -.code wait -function, if successful, returns a cons cell consisting of a pair of integers. -The -.code car -of the cons is the process ID of the process or group which was successfully -waited on, and the -.code cdr -is the status. If -.code wait -fails, it returns -.codn nil . -The -.code errno -function can be used to inquire about the cause. +.RS +.meIP < width +The width specifier consists of an optional +.code < +(left angle bracket) character or +.code ^ +(caret) +character followed by an optional width specification. -The -.meta process-id -argument, if not supplied, defaults to -1, which means that -.code wait -waits for any process, rather than a specific process. Certain other -values have special meaning, as documented in the POSIX standard -for the -.code waitpid -function. +If the leading +.code < +character is present, then the printing will be left-adjusted within +this field. If the +.code ^ +character is present, the printing will be centered within the field. +Otherwise it will be right-adjusted by default. +The width can be specified as a decimal integer, or as the character +.codn * . The -.meta flags -argument defaults to zero. If it is specified as nonzero, it should be -a bitwise combination (via the -.code logior -function) of the constants -.codn w-nohang , -.codn w-untraced -and -.codn w-continued . -If -.code w-nohang -is used, then -.code wait -returns a cons cell whose -.code car -specifies a process ID value of zero in the situation that at least -one of the processes designated by -.code process-id -exist and are children of the calling process, but have not changed state. -In this case, the status value in the -.code cdr -is unspecified. +.code * +notation means that instead of digits, the value of the next argument is +consumed, and expected to be an integer which specifies the width. If that +integer value is negative, then the field will be left-adjusted. +If the value is positive, but either the +.code < +or +.code ^ +prefix character is present in the width +specifier, then the field is adjusted according to that character. -Status values may be inspected with the functions -.codn w-ifexited , -.codn w-exitstatus , -.codn w-ifsignaled , -.codn w-termsig , -.codn w-coredump , -.codn w-ifstopped , -.code w-stopsig -and -.codn w-ifcontinued . +.meIP < precision +The precision specifier is introduced by a leading comma. If this comma appears +immediately after the directive's +.code ~ +character, then it means that +.meta width +is being omitted; there is only a precision field. -.coNP Functions @, w-ifexited @, w-exitstatus @, w-ifsignaled @, w-termsig @, w-coredump @ w-ifstopped and @ w-stopsig -.synb -.mets (w-ifexited << status ) -.mets (w-exitstatus << status ) -.mets (w-ifsignaled << status ) -.mets (w-termsig << status ) -.mets (w-coredump << status ) -.mets (w-ifstopped << status ) -.mets (w-stopsig << status ) -.mets (w-ifcontinued << status ) -.syne -.desc -These functions analyze process exit values produced by the -.code wait -function. +The precision specifier may begin with these optional characters: +.RS +.coIP 0 +(the "leading zero flag"), +.coIP + +(print a sign for positive values") +.IP space +(print a space in place of a positive sign). +.RE -They are closely based on the -POSIX macros -.codn WIFEXITED , -.code WEXITSTATUS , -and so on. +The precision specifier itself is either a decimal integer that does not +begin with a zero digit, or the +.code * +character. -The -.meta status -value is either an integer, or a cons cell. In this case, the cons -cell is expected to have an integer in its -.code cdr -which is used as the status. +The precision field's components have a meaning which depends on the type of +object printed and the conversion specifier. -The -.codn w-ifexited , -.codn w-ifsignaled , -.codn w-coredump , -.code w-ifstopped -and -.code w-ifcontinued -functions have Lisp boolean return semantics, unlike their C language -counterparts: they return -.code t -or -.codn nil , -rather than zero or nonzero. The others return integer values. +For integer arguments, the precision value specifies the minimum number of digits +to print. If the precision field has a leading zero flag, then the integer is +padded with zeros to the required number of digits, otherwise the number is +padded with spaces instead of zeros. If zero or space padding is present, and +a leading positive or negative sign must be printed, then it is placed before +leading zeros, or after leading spaces, as the case may be. -.coNP Function @ exec -.synb -.mets (exec < file <> [ args ]) -.syne -.desc -The exec function replaces the process image with the executable specified -by string argument -.metn file . -The executable is found by searching the system path. +For floating-point values, the meaning of the precision value depends on which +specific conversion specifier +.cod1 ( f , +.codn e , +.code a +or +.codn s ) +is used. The details are +documented in the description of each of these, below. The leading zero flag is +ignored for floating-point values regardless of the conversion specifier. -The -.meta file -argument becomes the first argument of the executable, argument zero. +For integer or floating-point arguments, if the precision specifier has a +.code + +sign +among the special characters, then a +.code + +sign is printed for positive numbers. If +the precision specifier has a leading space instead of a +.code + +sign, then the +.ocde + +sign is rendered as a space for positive numbers. If there is no leading space +or +.codn + , +then a sign character is omitted for positive numbers. Negative +numbers are unconditionally prefixed with a +.code - +sign. -If -.meta args -is specified, it is a list of strings. These are passed as the additional -arguments of the executable. +For all other objects, the precision specifies the maximum number of characters +to print. The object's printed representation is crudely truncated at that +number of characters. +.RE -If -.code exec -fails, an exception of type -.code file-error -is thrown. +.TP* "Format directives:" +.RS +Format directives are case sensitive, so that for example +.code ~x +and +.code ~X +have a +different effect, and +.code ~A +doesn't exist whereas +.code ~a +does. They are: -.coNP Function @ exit* -.synb -.mets (exit* << status ) -.syne -.desc -The -.code exit* -function terminates the entire process (running \*(TX image), specifying -the termination status to the operating system. The -.meta status -argument is treated exactly like that of the -.code exit -function. Unlike that function, this one exits the process immediately, -cleaning up only low-level operating system resources such as closing file -descriptors and releasing memory mappings, without performing user-space -cleanup. +.coIP a +Prints any object in an aesthetic way, as if by the +.code pprint +function. +The aesthetic notation violates read-print consistency: this notation +is not necessarily readable if it is implanted in \*(TX source code. +The field width specifier is honored, including the left-right adjustment +semantics. -.code exit* -is implemented using a call to the POSIX function -.codn _exit . +When this specifier is used for floating-point values, the precision specifies +the maximum number of total significant figures, which do not include any +digits in the exponent, if one is printed. Numbers are printed in exponential +notation if their magnitude is small, or else if their exponent exceeds their +precision. (If the precision is not specified, then it defaults to the +system-dependent number of digits in a floating point value, derived from the C +language +.code DBL_DIG +constant.) Floating point values which are integers are +printed without a trailing +.code .0 +(point zero). -.coNP Functions @ getpid and @ getppid -.synb - (getpid) - (getppid) -.syne -.desc -These functions retrieve the current process ID and the parent process ID -respectively. They are wrappers for the POSIX functions -.code getpid -and -.codn getppid . +.coIP s +Prints any object in a standard way, as if by the print function. Objects for +which read-print consistency is possible are printed in a way such that +if their notation is implanted in \*(TX source, they are readable. +The field width specifier is honored, including the left-right adjustment +semantics. The precision field is treated very similarly to the +.code ~a +format directive, except that non-exponentiated floating point numbers that +would be mistaken for integers include a trailing +.code .0 +for the sake of read-print +consistency. Objects truncated by precision may not have read-print +consistency. For instance, if a string object is truncated, it loses its +trailing closing quote, so that the resulting representation is no longer +a properly formed string object. -.coNP Function @ daemon -.synb -.mets (daemon < nochdir-p << noclose-p ) -.syne -.desc -This is a wrapper for the function -.code daemon -which originated in BSD Unix. +.coIP x +Requires an argument of character or integer type. The integer value or +character code is printed in hexadecimal, using lower-case letters +for the digits +.code a +through +.codn f . +Width and precision semantics +are as described for the +.code a +format directive, for integers. -It returns -.code t -if successful, -.code nil -otherwise, and the -.code errno -variable is set in that case. +.coIP X +Like the +.code x +directive, but the hexadecimal digits +.code a +through +.code f +are rendered in upper case. -.SS* Unix File Descriptors +.coIP o +Like the +.code x +directive, but octal is used instead of hexadecimal. -.coNP Function @ open-fileno -.synb -.mets (open-fileno < file-descriptor <> [ mode-string ]) -.syne +.coIP f The -.code open-fileno -function creates a \*(TX stream over a file descriptor. The -.meta file-descriptor -argument must be an integer denoting a valid file descriptor. - -For a description of -.metn mode-string , -see the -.code open-file -function. +.code f +directive prints numbers in a fixed point decimal notation, with +a fixed number of digits after the decimal point. It requires a numeric +argument. (Unlike +.codn x , +.code X +and +.codn o , +it does not allow an argument of character type). +The precision specifier gives the number of digits past the decimal point. +The number is rounded off to the specified precision, if necessary. +Furthermore, that many digits are always printed, regardless of the actual +precision of the number or its type. If it is omitted, then the default value +is three: three digits past the decimal point. A precision of zero means no +digits pas the decimal point, and in this case the decimal point is suppressed +(regardless of whether the numeric argument is floating-point or integer). -.coNP Function @ fileno -.synb -.mets (fileno << stream ) -.syne -.desc +.coIP e +The +.code e +directive prints numbers in exponential notation. It requires +a numeric argument. (Unlike +.codn x , +.code X +and +.codn o , +it does not allow an argument of character type). +The precision specifier gives the number of digits past the decimal point +printed in the exponential notation, not counting the digits in the exponent. +Exactly that many digits are printed, regardless of the precision of the +number. If the precision is omitted, then the number of digits after the +decimal point is three. If the precision is zero, then a decimal portion is +truncated off entirely, including the decimal point. + +.coIP p The -.code fileno -function returns the underlying file descriptor of -.metn stream , -if it has one. Otherwise, it returns -.codn nil. +.code p +directive prints a numeric representation in hexadecimal of the bit pattern +of the object, which is meaningful to someone familiar with the internals +of \*(TX. If the object is a pointer to heaped data, that value +has a correspondence to its address. +.RE -This is equivalent to querying the stream using -.code stream-get-prop -for the -.code :fd -property. +.PP -.coNP Function @ dupfd +.coNP Functions @, print @, pprint @, prinl @, pprinl @ tostring and @ tostringp .synb -.mets (dupfd < old-fileno <> [ new-fileno ]) +.mets (print < obj <> [ stream ]) +.mets (pprint < obj <> [ stream ]) +.mets (prinl < obj <> [ stream ]) +.mets (pprinl < obj <> [ stream ]) +.mets (tostring << obj ) +.mets (tostringp << obj ) .syne .desc The -.code dupfd -function provides an interface to the POSIX functions -.code dup -or -.codn dup2 , -when called with one or two arguments, respectively. +.code print +and +.code pprint +functions render a printed character representation of the +.meta obj +argument into +.metn stream . +If a stream argument is not supplied, then +the destination is the stream currently stored in the +.code *stdout* +variable. The +.code print +function renders in a way which strives for read-print +consistency: an object is printed in a notation which is recognized as +a similar object of the same kind when it appears in \*(TX source code. +The +.code pprint +function ("pretty print") does not strive for read-print consistency. +For instance it prints a string object simply by dumping its characters, rather +than by adding the surrounding quotes and rendering escape syntax for +special characters. Both functions return +.metn obj . -.coNP Function @ pipe -.synb - (pipe) -.syne -.desc The -.code pipe -function, if successful, returns a pair of integer file descriptors -as a cons cell pair. The descriptor in the -.code car -field of the pair is the read end of the pipe. +.code prinl +and +.code pprinl +functions are like +.code print +and +.codn pprint , +except that they issue a newline character after printing the object. +These functions also return +.metn obj . + The -.code cdr -holds the write end. +.code tostring +and +.code tostringp +functions are like +.code print +and +.codn pprint , +but they do not accept a stream argument. Instead they print to a freshly +instantiated string stream, and return the resulting string. -If the function fails, it throws an exception of type -.codn file-error . +The following equivalences hold between calls to the +.code format +function and calls to the above functions: -.coNP Function @ poll +.cblk + (format stream "~s" obj) <--> (print obj stream) + (format t "~s" obj) <--> (print obj) + (format t "~s\en" obj) <--> (prinl obj) + (format nil "~s" obj) <--> (tostring obj) +.cble + +For +.codn pprint , +.code tostringp +and +.codn pprinl , +the equivalence is produced by using +.code ~a +in format rather than +.codn ~s . + +Note: for characters, the print function behaves as follows: most control +characters in the Unicode +.code C0 +and +.code C1 +range are rendered using the +.code #\ex +notation, +using two hex digits. Codes in the range +.code D800 +to +.codn DFFF , +and the codes +.code FFFE +and +.code FFFF +are printed in the +.code #\exNNNN +with four hexadecimal digits, and +character above this range are printed using the same notation, but with six +hexadecimal digits. Certain characters in the +.code C0 +range are printed using +their names such as +.code #\enul +and +.codn #\ereturn , +which are documented +in the Character Literals section. +The +.code DC00 +character is printed as +.codn #\epnul . +All other characters are printed as +.cblk +.meti >> #\e char +.cble +where +.meta char +is the actual character. + +Caution: read-print consistency is affected by trailing material. If additional +digits are printed immediately after a number without intervening whitespace, +they extend that number. If hex digits are printed after the character +.codn x , +which is rendered as +.codn #\ex , +they look like a hex character code. + +.coNP Function @ tprint .synb -.mets (poll < poll-list <> [ timeout ]) +.mets (tprint < obj <> [ stream ]) .syne .desc -The -.code poll -function suspends execution while monitoring one or more file descriptors -for specified events. It is a wrapper for the same-named POSIX function. - -The -.meta poll-list -argument is a list of -.code cons -pairs. The -.code car -of each pair is either an integer file descriptor, or else a stream -object which has a file descriptor (the -.code fileno -function can be applied to that stream to retrieve a descriptor). -The -.code cdr -of each pair is an integer bit mask specifying the events, whose -occurrence the file descriptor is to be monitored for. The variables -.codn poll-in , -.codn poll-out , -.code poll-err -and several others are available which hold bitmask values corresponding -to the constants -.codn POLLIN , -.codn POLLOUT , -.code POLLERR -used with the C language -.code poll -function. The -.meta timeout -argument, if absent, defaults to the value -1, which specifies an indefinite -wait. A nonnegative value specifies a wait with a timeout, measured in -milliseconds. +.codn tprint +function prints a representation of +.meta obj +on +.metn stream . -The function returns a list of pairs representing the descriptors or streams -which were successfully polled. If the function times out, it returns an -empty list. If an error occurs, an exception is thrown. +If the stream argument is not supplied, then +the destination is the stream currently stored in the +.code *stdout* +variable. -The returned list is similar in structure to the input list. However, it holds -only entries which polled positive. The -.code cdr -of every pair now holds a bitmask of the events which were to have occurred. +For all object types except lists and vectors, +.code tprint +behaves like +.codn pprinl . -.SS* Unix Itimers -Itimers ("interval timers") can be used in combination with signal handling to -execute asynchronous actions. Itimers deliver delayed, one-time signals, -and also periodically recurring signals. For more information, consult the -POSIX specification. +If +.code obj +is a list or vector, then +.code tprint +recurses: the +.code tprint +function is applied to each element. An empty list or vector +results in no output at all. This effectively means that an arbitrarily nested +structure of lists and vectors is printed flattened, with one element on each +line. -.coNP Variables @, itimer-real @, itimer-virtual and @ itimer-prof +.coNP Function @ streamp +.synb +.mets (streamp << obj ) +.syne .desc -These variables correspond to the POSIX constants -.codn ITIMER_REAL , -.code ITIMER_VIRTUAL -and -.codn ITIMER_PROF . -Their values are suitable as the -.meta timer -argument of the -.code getitimer -and -.code setitimer -functions. +The +.code streamp +function returns +.code t +if +.meta obj +is any type of stream. Otherwise it returns +.codn nil . -.coNP Functions @ getitimer and @ setitimer +.coNP Function @ real-time-stream-p .synb -.mets (getitimer << timer ) -.mets (setitimer < timer < interval << value ) +.mets (real-time-stream-p << obj ) .syne .desc The -.code getitimer -function returns the current value of the specified timer, -which must be -.codn itimer-real , -.code itimer-virtual -or -.codn itimer-prof . - -The current value consists of a list of two integer values, which -represents microseconds. The first value is the timer interval, -and the second value is the timer's current value. - -Like -.codn getitimer , -the -.code setitimer -function also retrieves the specified timer. -In addition, it stores a new value in the timer, -which is given by the two arguments, expressed in microseconds. +.code real-time-streamp-p +function returns +.code t +if +.meta obj +is a stream marked as +"real-time". If +.meta obj +is not a stream, or not a stream marked as "real-time", +then it returns +.codn nil . -.SS* Unix Syslog +Only certain kinds of streams accept the real-time attribute: file streams and +tail streams. This attribute controls the semantics of the application of +.code lazy-stream-cons +to the stream. For a real-time stream, +.code lazy-stream-cons +returns a stream with "naive" semantics which returns data as soon as it is +available, at the cost of generating spurious +.code nil +item when the stream +terminates. The application has to recognize and discard that +.code nil +item. +The ordinary lazy streams read ahead by one line and suppress this extra +item, so their representation is more accurate. -On platforms where a Unix-like syslog API is available, \*(TX exports this -interface. \*(TX programs can configure logging via the -.code openlog -function, -control the logging mask via -.code setlogmask -and generate logs via -.codn syslog , -or using special syslog streams. +When \*(TX starts up, it automatically marks the +.code *std-input* +stream as real-time, if it is connected to a TTY device (a device for which +the POSIX function +.code isatty +reports true). This is only supported on platforms that have this function. +The behavior is overridden by the +.code -n +command line option. -.coNP Special variables @, log-pid @, log-cons @, log-ndelay @, log-odelay @ log-nowait and @ log-perror +.coNP Function @ make-string-input-stream +.synb +.mets (make-string-input-stream << string ) +.syne .desc -These variables take on the values of the corresponding C preprocessor -constants from the -.code -header: -.codn LOG_PID , -.codn LOG_CON S, etc. -These integer values represent logging options used in the option argument to -the -.code openlog -function. +This function produces an input stream object. Character read operations on the +stream object read successive characters from +.metn string . +Output operations and byte operations are not supported. -Note: -.code LOG_PERROR -is not in POSIX, and so -.code log-perror -might not be available. -See notes about -.code LOG_AUTHPRIV -in the documentation for -.codn log-authpriv . +.coNP Function @ make-string-byte-input-stream +.synb +.mets (make-string-byte-input-stream << string ) +.syne +.desc +This function produces an input stream object. Byte read operations on +this stream object read successive byte values obtained by encoding +.meta string +into UTF-8. Character read operations are not supported, and neither +are output operations. -.coNP Special variables @, log-user @, log-daemon @ log-auth and @ log-authpriv +.coNP Function @ make-string-output-stream +.synb + (make-string-output-stream) +.syne .desc -These variables take on the values of the corresponding C preprocessor -constants from the -.code -header: -.codn LOG_USER , -.codn LOG_DAEMON , -.code LOG_AUTH -and -.codn LOG_AUTHPRIV . -These are the integer facility codes specified in the -.code openlog -function. +This function, which takes no arguments, creates a string output stream. +Data sent to this stream is accumulated into a string object. +String output streams supports both character and byte output operations. +Bytes are assumed to represent a UTF-8 encoding, and are decoded in order +to form characters which are stored into the string. -Note: -.code LOG_AUTHPRIV -is not in POSIX, and so -.code log-authpriv -might not be available. -For portability use code like -.code (or (symbol-value 'log-authpriv) 0) -to evaluate to 0 if -.code log-authpriv -doesn't exist, or else check for its existence -using -.codn (boundp 'log-authpriv) . +If an incomplete UTF-8 code is output, and a character output operation then +takes place, that code is assumed to be terminated and is decoded as invalid +bytes. The UTF-8 decoding machine is reset and ready for the start of a new +code. -.coNP Special variables @, log-emerg @, log-alert @, log-crit @, log-err @, log-warning @, log-notice @ log-info and @ log-debug +The +.code get-string-from-stream +function is used to retrieve the accumulated string. -These variables take on the values of the corresponding C preprocessor -constants from the -.code -header: -.codn LOG_EMERG , -.codn LOG_ALERT , -etc. -These are the integer priority codes specified in the -.code syslog -function. +If the null character is written to a string output stream, either via +a character output operation or as a byte operation, the resulting string +will appear to be prematurely terminated. \*(TX strings cannot contain null +bytes. -.coNP The @ *stdlog* special variable +.coNP Function @ get-string-from-stream +.synb +.mets (get-string-from-stream << stream ) +.syne .desc The -.code *stdlog* -variable holds a special kind of stream: a syslog stream. Each -newline-terminated line of text sent to this stream becomes a log message. - -The stream internally maintains a priority value that is applied -when it generates messages. By default, this value is that of -.codn log-info . -The stream holds the priority as the value of the -.code :prio -stream property, which may be changed with the -.code stream-set-prop -function. +.meta stream +argument must be a string output stream. This function finalizes +the data sent to the stream and retrieves the accumulated character string. -The latest priority value which has been configured on the stream is used -at the time the newline character is processed and the log message -is generated, not necessarily the value which was in effect at the time the -accumulation of a line began to take place. +If a partial UTF-8 code has been written to +.metn stream , +and then this +function is called, the byte stream is considered complete and the partial +code is decoded as invalid bytes. -Messages sent to -.code *stdlog* -are delimited by newline characters. That is to say, each line of -text written to the stream is a new log. +After this function is called, further output on the stream is not possible. -.coNP Function @ openlog +.coNP Function @ make-strlist-output-stream .synb -.mets (openlog < id-string >> [ options <> [ facility ]]) + (make-strlist-output-stream) .syne .desc -The -.code openlog -function is a wrapper for the -.code openlog -C function, and the -arguments have the same semantics. It is not necessary to use -.code openlog -in order -to call the -.code syslog -function or to write data to -.codn *stdlog* . -The call is necessary in order to override the default identifying string, to -set options, such as having the PID (process ID) recorded in log messages, and -to specify the facility. - -The -.meta id-string -argument is mandatory. - -The -.meta option -argument is a bitwise mask (see the logior function) of option -values such as -.code log-pid -and -.codn log-cons . -If it is missing, then a value of 0 is -used, specifying the absence of any options. +This function is very similar to +.codn make-string-output-stream . +However, the stream object produced by this function does not produce a string, +but a list of strings. The data is broken into multiple strings by newline +characters written to the stream. Newline characters do not appear in the +string list. Also, byte output operations are not supported. -The -.meta facility -argument is one of the values -.codn log-user , -.code log-daemon -or -.codn log-auth . -If it is missing, then -.code log-user -is assumed. +.coNP Function @ get-list-from-stream +.synb +.mets (get-list-from-stream << stream ) +.syne +.desc +This function returns the string list which has accumulated inside +a string output stream given by +.metn stream . +The string output stream is +finalized, so that further output is no longer possible. -.coNP Function @ closelog +.coNP Function @ close-stream .synb - (closelog) +.mets (close-stream < stream <> [ throw-on-error-p ]) .syne .desc The -.code closelog -function is a wrapper for the C function -.codn closelog . +.code close-stream +function performs a close operation on +.metn stream , +whose meaning is depends on the type of the stream. For some types of streams, +such as string streams, it does nothing. For streams which are connected +to operating system files or devices, will perform a close of the underlying +file descriptor, and dissociate that descriptor from the stream. Any buffered +data is flushed first. -.coNP Function @ setlogmask -.synb -.mets (setlogmask << bitmask-integer ) -.syne -.desc -The -.code setlogmask -function interfaces to the corresponding C function, and has the -same argument and return value semantics. The -.meta bitmask-integer -argument is a mask of priority -values to enable. The return value is the prior value. Note that if the -argument is zero, then the function doesn't set the mask to zero; it only -returns the current value of the mask. +.code close-stream +returns a boolean true value if the close has occurred without +errors, otherwise +.codn nil . -Note that the priority values like -.code log-emerg -and -.code log-debug -are integer -enumerations, not bitmasks. These values cannot be combined directly to create -a bitmask. Rather, the -.code mask -function should be used on these values. +For most streams, "without errors" means that any buffered output data is +flushed successfully. -.TP* Example: +For command and process pipes (see open-command and open-process), success also +means that the process terminates normally, with a successful error code, or an +unsuccessful one. An abnormal termination is considered an error, as +as is the inability to retrieve the termination status, as well as the situation +that the process continues running in spite of the close attempt. +Detecting these situations is platform specific. -.cblk - ;; Enable LOG_EMERG and LOG_ALERT messages, - ;; suppressing all others - (setlogmask (mask log-emerg log-alert)) -.cble +If the +.meta throw-on-error-p +argument is specified, and isn't +.codn nil , +then the +function throws an exception if an error occurs during the close operation +instead of returning +.codn nil . -.coNP Function @ syslog +.coNP Functions @, get-error @ get-error-str and @ clear-error .synb -.mets (syslog < priority < format << format-arg *) +.mets (get-error << stream ) +.mets (get-error-str << stream ) +.mets (clear-error << stream ) .syne .desc -This function is the interface to the -.code syslog -C function. The -.code printf -formatting capabilities of the function are not used; -the -.meta format -argument follows the conventions of the \*(TL -.code format -function instead. Note in particular that -the -.code %m -convention for interpolating the value of strerror(errno) which is -available in some versions of the -.code syslog -C function is currently not supported. - -Note that syslog messages are not newline-terminated. +When a stream operation fails, the +.code get-error +and +.code get-error-str +functions may be used to inquire about a more detailed cause of the error. -.SS* Unix Path Globbing +Not all streams support these functions to the same extent. For instance, +string input streams have no persistent state. The only error which occurs +is the condition when the string has no more data. -On platforms where the POSIX -.code glob -function is available \*(TX provides this functionality in -the form of a like-named function, and some numeric constants. +The +.code get-error +inquires +.meta stream +about its error condition. -.coNP Special variables @, glob-err @, glob-mark @, glob-nosort @, glob-nocheck @, glob-noescape @, glob-period @, glob-altdirfunc @, glob-brace @, glob-nomagic @, glob-tilde @ glob-tilde-check and @ glob-onlydir +The function returns +.code nil +to indicate there is no error condition, +.code t +to indicate an end-of-data condition, +or else a value which is specific to the stream type indicating the +specific error type. -These variables take on the values of the corresponding C preprocessor -constants from the -.code -header: -.codn GLOB_ERR , -.codn GLOB_MARK , -.codn GLOB_NOSORT , -etc. +Note: for some streams, it is possible for the +.code t +value to be returned even though no operation has failed; that is to say, the +streams "know" they are at the end of the data even though no read operation +has failed. Code which depends on this will not work with streams which +do not thus indicate the end-of-data +.I a priori, +but by means of a read operation which fails. -These values are passed as the optional second argument of the -.code glob -function. They are bitmasks and so multiple values can be combined -using the -.code logior -function. +The +.code get-error-str +function returns a text representation of the error code. The +.code nil +error code is represented as the string +.codn "no error" ; +the +.code t +error code as +.code "eof" +and other codes have a stream-specific representation. -Note that the -.codn glob-period , -.codn glob-altdirfunc , -.codn glob-brace , -.codn glob-nomagic , -.codn glob-tilde , -.code glob-tilde-check -and -.code glob-onlydir -variables may not be available. They are extensions in the GNU C library -implementation of -.codn glob . +The +.code clear-error +function removes the error situation from a stream. On some streams, it does +nothing. If an error has occurred on a stream, this function should be called +prior to re-trying any I/O or positioning operations. +The return value is the previous error code, or +.code nil +if there was no error, or the operation is not supported on the stream. -.coNP Function @ glob +.coNP Functions @, get-line @ get-char and @ get-byte .synb -.mets (glob < pattern >> [ flags <> [ error-func ]]) +.mets (get-line <> [ stream ]) +.mets (get-char <> [ stream ]) +.mets (get-byte <> [ stream ]) .syne .desc +These fundamental stream functions perform input. The +.meta stream +argument +is optional. If it is specified, it should be an input stream which supports +the given operation. If it is not specified, then the +.code *stdin* +stream is used. + The -.code glob -function is a interface to the Unix function of the same name. +.code get-char +function pulls a character from a stream which supports character +input. Streams which support character input also support the +.code get-line +function which extracts a line of text delimited by the end of the stream or a +newline character and returns it as a string. (The newline character does not +appear in the string which is returned). + +Character input from streams based on bytes requires UTF-8 decoding, so that +get-char actually may read several bytes from the underlying low level +operating system stream. + The -.meta pattern -argument must be a string, which holds a glob pattern: a pattern which -matches zero or more path names, similar to a regular expression. -The function tries to expand the pattern and return a list of strings -representing the matching path names in the file system. +.code get-byte +function bypasses UTF-8 decoding and reads raw bytes from +any stream which supports byte input. Bytes are represented as integer +values in the range 0 to 255. -If there are no matches, then an empty list is returned. +Note that if a stream supports both byte input and character input, then mixing +the two operations will interfere with the UTF-8 decoding. -The optional -.meta flags -argument defaults to zero. If given, it may be a bitwise combination of the -values of the variables -.codn glob-err , -.codn glob-mark , -.code glob-nosort -and others. +These functions return +.code nil +when the end of data is reached. Errors are +represented as exceptions. -If the -.meta error-func -argument is specified, it gives a callback function which is invoked -when -.code glob -encounters errors accessing paths. The function takes two arguments: -the pathname and the -.code errno -value which occurred for that pathname. The function's return value is -boolean. If the function returns true, then -.code glob -will terminate. +See also: +.code get-lines -Details of the semantics of the -.code glob -function, and the meaning of all the -.meta flags -arguments are given in the documentation for the C function. +.coNP Function @ get-string +.synb +.mets (get-string >> [ stream >> [ count <> [ close-after-p ]]]) +.syne +.desc +The +.code get-string +function reads characters from a stream, and assembles them into +a string, which is returned. If the +.meta stream +argument is omitted, then the +.code *stdin* +stream is used. + +The stream is closed after extracting the data, unless +.meta close-after-p +is specified as +.codn nil . +The default value of this argument is +.codn t . -.SS* Web Programming Support +If the +.meta count +argument is missing, then all of the characters from the +stream are read and assembled into a string. -.coNP Functions @ url-encode and @ url-decode +If present, the +.meta count +argument should be a positive integer indicating +a limit on how many characters to read. The returned string will be no +longer than +.metn count , +but may be shorter. + +.coNP Functions @ unget-char and @ unget-byte .synb -.mets (url-encode < string <> [ space-plus-p ]) -.mets (url-decode < string <> [ space-plus-p ]) +.mets (unget-char < char <> [ stream ]) +.mets (unget-byte < byte <> [ stream ]) .syne .desc -These functions convert character strings to and from a form which is suitable -for embedding into the request portions of URL syntax. - -Encoding a string for URL use means identifying in it certain characters that -might have a special meaning in the URL syntax and representing it using -"percent encoding": the percent character, followed by the ASCII value of the -character. Spaces and control characters are also encoded, as are all byte -values greater than or equal to 127 (7F hex). The printable ASCII characters -which are percent-encoded consist of this set: - -.cblk - :/?#[]@!$&'()*+,;=% -.cble +These functions put back, into a stream, a character or byte which was +previously read. The character or byte must match the one which was most +recently read. If the +.meta stream +argument is omitted, then the +.code *stdin* +stream is used. -More generally, strings can consists of Unicode characters, but the URL -encoding consists only of printable ASCII characters. Unicode characters in the -original string are encoded by expanding into UTF-8, and applying -percent-encoding the UTF-8 bytes, which are all in the range -.codn \exx80-\exxFF . +If the operation succeeds, the byte or character value is returned. +A +.code nil +return indicates that the operation is unsupported. -Decoding is the reverse process: reconstituting the UTF-8 byte sequence -specified by the URL-encoding, and then decoding the UTF-8 sequence into the -string of Unicode characters. +Some streams do not support these operations; some support +only one of them. In general, if a stream supports +.codn get-char , +it supports +.codn unget-char , +and likewise for +.code get-byte +and +.codn unget-byte . -There is an additional complication: whether or not to encode spaces as plus, -and to decode plus characters to spaces. In encoding, if spaces are not encoded -to the plus character, then they are encoded as -.codn %20 , -since spaces are reserved -characters that must be encoded. In decoding, if plus characters are not -decoded to spaces, then they are left alone: they become plus characters in the -decoded string. +Space is available for only one character or byte of pushback. -The -.code url-encode -function performs the encoding process. If the -.code space-plus-p -argument is omitted or specified as -.codn nil , -then spaces are encoded as -.codn %20 . -If the argument is a value other than -.codn nil , -then spaces are encoded as the -character -.code + -.codn (plus) . +Pushing both a byte and a character, in either order, is also unsupported. +Pushing a byte and then reading a character, or pushing a character and +reading a byte, are unsupported mixtures of operations. -The -.code url-decode -function performs the decoding process. If the -.code space-plus-p -argument is omitted or specified as -.codn nil , -then -.code + -.code (plus) -characters in the -encoded data are retained as -.code + -characters in the decoded strings. Otherwise, -plus characters are converted to spaces. +If the stream is binary, then pushing back a byte decrements its position, +except if the position is already zero. At that point, the position becomes +indeterminate. -.coNP Functions @ html-encode and @ html-decode +.coNP Functions @, put-string @, put-line @ put-char and @ put-byte .synb -.mets (html-encode << text-string ) -.mets (html-decode << html-string ) +.mets (put-string < string <> [ stream ]) +.mets (put-line >> [ string <> [ stream ]]) +.mets (put-char < char <> [ stream ]) +.mets (put-byte < byte <> [ stream ]) .syne .desc -The -.code html-encode -and -.code html-decode -functions convert between an HTML and raw -representation of of text. +These functions perform output on an output stream. The +.meta stream +argument +must be an output stream which supports the given operation. If it is omitted, +then +.code *stdout* +is used. The -.code html-encode -function returns a string which is based on the content of -.metn text-string , -but in which all characters which have special meaning in HTML -have been replaced by HTML codes for representing those characters literally. -The returned string is the HTML-encoded verbatim representation of -.metn text-string . +.code put-char +function writes a character given by +.code char +to a stream. If the +stream is based on bytes, then the character is encoded into UTF-8 and multiple +bytes are written. Streams which support +.code put-char +also support put-line, and +.codn put-string . The -.code html-decode -function converts -.metn html-string , -which may contain HTML -character encodings, into a string which contains the actual characters -represented by those encodings. - -The function composition -.code (html-decode (html-encode text)) -returns a string which is equal to -.codn text . - -The reverse composition -.code (html-encode (html-decode html)) -does not necessarily return a string equal to -.codn html . - -For instance if html is the string -.strn "

Hello, world!

" , -then -.code html-decode -produces -.strn "

Hello, world!

" . -From this, -.code html-encode -produces -.strn "<p>Hello, world!</p>" . +.code put-string +function writes the characters of a string out to +the stream as if by multiple calls to put-char. The +.meta string +argument +may be a symbol, in which case its name is used as the string. -.SS* Filter Module -The filter module provides a trie (pronounced "try") data structure, -which is suitable for representing dictionaries for efficient filtering. -Dictionaries are unordered collections of keys, which are strings, which -have associated values, which are also strings. A trie can be used to filter -text, such that keys appearing in the text are replaced by the corresponding -values. A trie supports this filtering operation by providing an efficient -prefix-based lookup method which only looks at each input character ones, and -which does not require knowledge of the length of the key in advance. +The +.code put-line +function is like +.codn put-string , +but also writes an additional newline +character. The string is optional in +.codn put-line , +and defaults to the empty string. -.coNP Function @ make-trie -.synb - (make-trie) -.syne -.desc The -.code make-trie -function creates an empty trie. There is no special data type for -a trie; a trie is some existing type such as a hash table. +.code put-byte +function writes a raw byte given by the +.meta byte +argument +to +.metn stream , +if +.meta stream +supports a byte write operation. The byte +value is specified as an integer value in the range 0 to 255. -.coNP Function @ trie-add +.coNP Functions @ put-strings and @ put-lines .synb -.mets (trie-add < trie < key << value ) +.mets (put-strings < sequence <> [ stream ]]) +.mets (put-lines < sequence <> [ stream ]]) .syne .desc -The -.code trie-add -function adds the string -.meta key -to the trie, associating -it with -.metn value . -If -.meta key -already exists in -.metn trie , -then the value is updated with -.metn value . - -The -.meta trie -must not have been compressed with -.metn trie-compress . - -A trie can contain keys which are prefixes of other keys. For instance -it can contain -.str dog -and -.strn dogma . -When a trie is used for matching -and substitution, the longest match is used. If the input presents -the text -.strn doggy , -then the match is -.strn dog . -If the input is -.strn dogmatic , -then -.str dogma -matches. +These functions assume +.meta sequence +to be a sequence of strings, or of +symbols, or a mixture thereof. These strings are sent to the stream. The +.meta stream +argument must be an output stream. If it is omitted, then +.code *stdout* +is used. -.coNP Function @ trie-compress -.synb -.mets (trie-compress << trie ) -.syne -.desc The -.code trie-compress -function changes the representation of -.meta trie -to a representation which occupies less space and supports faster lookups. -The new representation is returned. +.code put-strings +function iterates over +.meta sequence +and writes each element +to the stream as if using the +.code put-string +function. -The compressed representation of a trie does not support the -.code trie-add +The +.code put-lines +function iterates over +.code sequence +and writes each element +to the stream as if using the +.code put-line function. -This function destructively manipulates -.metn trie , -and may return an object -that is the same object as -.codn trie , -or it may return a different object, -while at the same time still modifying the internals of -.metn trie . -Consequently, the program should not retain the input object -.codn trie , -but use the returned object in its place. +Both functions return +.codn t . -.coNP Function @ trie-lookup-begin +.coNP Function @ flush-stream .synb -.mets (trie-lookup-begin << trie ) +.mets (flush-stream << stream ) .syne .desc -The -.code trie-lookup-begin -function returns a context object for performing -an open-coded lookup traversal of a trie. The -.meta tri -argument -is expected to be a trie that was created by the -.code make-trie -function. +This function is meaningful for output streams which accumulate data +which is passed on to the operating system in larger transfer units. +Calling +.code flush-stream +causes all accumulated data inside +.meta stream +to be passed +to the operating system. If called on streams for which this function is not +meaningful, it does nothing, and returns +.codn nil . -.coNP Function @ trie-lookup-feed-char +.coNP Function @ seek-stream .synb -.mets (trie-lookup-feed-char < trie-context << char ) +.mets (seek-stream < stream < offset << whence ) .syne .desc The -.code trie-lookup-feed-char -function performs a one character step in a trie -lookup. The -.meta trie-context -argument must be a trie context returned -by -.metn trie-lookup-begin , -or by some previous call to -.codn trie-lookup-feed-char . +.code seek-stream +function is meaningful for file streams. It changes the +current read/write position within +.metn stream . +It can also be used to determine the current position: see the notes about the +return value below. + The -.meta char -argument is the next character to match. +.meta offset +argument is a positive or negative integer which gives a +displacement that is measured from the point identified by the +.meta whence +argument. -If the lookup is successful (the match through the trie can continue -with the given character) then a new trie context object is returned. -The old trie context remains valid. +Note that for text files, there isn't necessarily a 1:1 correspondence between +characters and positions due to line-ending conversions and conversions +to and from UTF-8. -If the lookup is unsuccessful, -.code nil -is returned. +The +.meta whence +argument is one of three keywords: :from-start, :from-current +and :from-end. These denote the start of the file, the current position +and the end of the file. -Note: determining whether a given string is stored in a trie can be -performed looking up every character of the string successively -with -.codn trie-lookup-feed-char , -using the newly returned context -for each successive operation. If every character is found, it means -that either that exact string is found in the trie, or a prefix. -The ambiguity can be resolved by testing whether the trie has a value -at the last node using -.codn tree-value-at . -For instance, if -.str catalog -is inserted into an empty trie with value -.strn foo , -then -.str cat -will look up successfully, being a prefix of -.strn catalog ; -however, the value at -.str cat +If +.meta offset +is zero, and +.meta whence is -.codn nil , -indicating that -.str cat -is only a prefix of one or more entries in the trie. +.codn :from-current , +then +.code seek-stream +returns the current absolute position within the +stream, if it can successfully obtain it. Otherwise, it +returns +.code t +if it is successful. -.coNP Function @ tree-value-at -.synb -.mets (trie-value-at << trie-context ) -.syne -.desc -The -.code trie-value-at -function returns the value stored at the node in -in the trie given by -.metn trie-context . -Nodes which have not been given -a value hold the value -.codn nil . +If a character has been successfully put back into a text stream with +.code unget-char +and is still pending, then the position value is unspecified. If a +byte has been put back into a binary stream with +.codn unget-byte , +and the previous position wasn't zero, then the position is decremented by one. -.coNP Function @ filter-string-tree +On failure, it throws an exception of type +.codn stream-error . + +.coNP Functions @ stream-get-prop and @ stream-set-prop .synb -.mets (filter-string-tree < filter << obj ) +.mets (stream-get-prop < stream << indicator ) +.mets (stream-set-prop < stream < indicator << value ) .syne .desc -The -.code filter-string-tree -a tree structure similar to -.metn obj , -in which all of the -string atoms have been filtered through -.metn filter . +These functions get and set properties on a stream. Only certain properties +are meaningful with certain kinds of streams, and the meaning depends on +the stream. If two or more stream types support a property of the same name, it +is expected that the property has the same or very similar meaning for both +streams to the maximum extent that similarity is possible. The -.meta obj -argument is a string tree structure: either the symbol -.codn nil , -denoting an empty structure; a string; or a list of tree structures. If -.meta obj -is -.codn nil , -then -.code filter-string-tree -returns +.code stream-set-prop +function sets a property on a stream. The +.meta indicator +argument is a symbol, usually a keyword symbol, denoting the property, +and +.meta value +is the property value. If the stream understands and accepts the +property, the function returns +.codn t . +Otherwise it returns .codn nil . The -.meta filter -argument is a filter: it is either a trie, a function, or nil. -If -.meta filter -is -.codn nil , -then -.code filter-string-trie -just returns -.metn obj . +.code stream-get-prop +function inquires about the value of a property on a +stream. If the stream understands the property, then it returns its current +value. If the stream does not understand a property, nil is returned, which is +also returned if the property exists, but its value happens to be +.codn nil . -If -.meta filter -is a function, it must be a function that can be called -with one argument. The strings of the string tree are filtered by passing -each one into the function and substituting the return value into the -corresponding place in the returned structure. +Properties are currently used for marking certain streams as "real-time" (see +the +.code real-time-stream-p +function above), and also for setting the priority at +which messages are reported to syslog by the +.code *stdlog* +stream (see +.code *stdlog* +in the UNIX SYSLOG section). -Otherwise if -.meta filter -is a trie, then this trie is used for filtering, -the string elements similarly to a function. For each string, a new -string is returned in which occurrences of the keys in the trie are -replaced by the values in the trie. +If +.meta stream +is a catenated stream (see the function +.codn make-catenated-stream ) +then these functions transparently operate on the current head stream of the +catenation. -.coNP Function @ filter-equal +.coNP Functions @ make-catenated-stream and @ cat-streams .synb -.mets (filter-equal < filter-1 < filter-2 < obj-1 << obj-2 ) +.mets (make-catenated-stream << stream *) +.mets (cat-streams << stream-list ) .syne .desc The -.code filter-equal -function tests whether two string trees are equal -under the given filters. +.code make-catenated-stream +function takes zero or more arguments which +are input streams of the same type, and combines +them into a single virtual stream called a catenated stream. + +The +.code cat-streams +function takes a single list of input streams of +the same type, and similarly combines them into a catenated stream. -The precise semantics can be given by this expression: +A catenated stream does not support seeking operations or output, +regardless of the capabilities of the streams in the list. -.cblk -.mets (equal (filter-string-tree < filter-1 << obj-1 ) -.mets \ \ \ \ \ \ (filter-string-tree < filter-2 << obj-2 )) -.cble +If the stream list is not empty, then the leftmost element of the +list is called the head stream. -The string tree -.meta obj-1 -is filtered through -.metn filter-1 , -as if by the -.code filter-string-tree -function, and similarly, -.meta obj-2 -is -filtered through -.metn filter-2 . -The resulting structures are compared -using -.codn equal , -and the result of that is returned. +The +.codn get-char , +.codn get-byte , +.codn get-line , +.code unget-char +and +.code unget-byte +functions delegate +to the corresponding operations on the head stream, if it exists. +If the stream list is empty, they return +.code nil +to the caller. -.SS* Access To TXR Pattern Language From Lisp +If the +.codn get-char , +.code get-byte +or +.code get-line +operation on the head stream yields +.codn nil , +and there are more lists in the stream, then the stream is closed, removed from +the list, and the next stream, if any, becomes the head list. The operation is +then tried again. If any of these operations fail on the last list, it is not +removed from the list, so that a stream remains in place which can take the +.code unget-char +or +.code unget-byte +operations. -It is useful to be able to invoke the abilities of the \*(TX pattern Language -from \*(TL. An interface for doing this provided in the form of the -.code match-fun -function, which is used for invoking a \*(TX pattern function. +In this manner, the catenated streams appear to be a single stream. -The -.code match-fun -function has a cumbersome interface which requires the \*(TL program to -explicitly deal with the variable bindings emerging from the pattern match -in the form of an association list. +Note that the operations can fail due to being unsupported. It is +the caller's responsibility to make sure all of the streams in the list +are compatible with the intended operations. -To make it the interface easier to use, \*(TX provides -the macros -.codn txr-if , -.codn txr-when +If the stream list is empty then an empty catenated stream is produced. +Input operations on this stream yield +.codn nil , +and the +.code unget-char and -.codn txr-case . +.code unget-byte +operations throw an exception. -.coNP Function @ match-fun +.coNP Function @ catenated-stream-p .synb -.mets (match-fun < name < args < input << files ) +.mets (catenated-stream-p << obj ) .syne .desc The -.code match-fun -function invokes a \*(TX pattern function whose name is -given by -.metn name , -which must be a symbol. - -The -.meta args -argument is a list of expressions. The expressions may be symbols -which will be interpreted as pattern variables, and may be bound or unbound. -If they are not symbols, then they are treated as expressions (of the -pattern language, not \*(TL) and evaluated accordingly. +.code catenated-stream-p +function returns +.code t +if +.meta obj +is a catenated stream. Otherwise it returns +.codn nil . +.coNP Function @ catenated-stream-push +.synb +.mets (catenated-stream-push < new-stream << cat-stream ) +.syne +.desc The -.meta input -argument is a list of strings, which may be lazy. It represents the -lines of the text stream to be processed. +.code catenated-stream-push +function pushes +.meta new-stream +to the front of the stream list inside +.metn cat-stream . -The -.meta file -argument is a list of filename specifications, which follow -the same conventions as files given on the \*(TX command line. If the pattern -function uses the -.code @(next) -directive, it can process these additional files. +If an +.code unget-byte +or +.code unget-char +operation was successfully performed on +.meta cat-stream +previously to a call to +.codn catenated-stream-push , +those operations were forwarded to the front stream. +If those bytes or characters are still pending, +they are pending inside that stream, and thus +are logically preceded by the contents +of +.metn new-stream . +.coNP Functions @ open-files and @ open-files* +.synb +.mets (open-files < path-list <> [ alternative-stream ]) +.mets (open-files* < path-list <> [ alternative-stream ]) +.syne +.desc The -.code match-fun -function's return value falls into three cases. If there is a -match failure, it returns -.codn nil . -Otherwise it returns a cons cell. The -.code car -field -of the cons cell holds the list of captured bindings. The -.code cdr -of the cons cell is one of two values. If the entire input was processed, the -cdr field holds the symbol -.codn t . -Otherwise it holds another cons cell whose -.code car -is the remainder of the list of lines which were not matched, and whose -.code cdr -is the line number. - -.TP* Example: - -.cblk - @(define foo (x y)) - @x:@y - @line - @(end) - @(do - (format t "~s\en" - (match-fun 'foo '(a b) - '("alpha:beta" "gamma" "omega") nil))) +.code open-files +and +.code open-files* +functions create a list of streams by invoking +the open-file function on each element of +.metn path-list . +These streams are turned +into a catenated stream as if applied as arguments to +.codn make-catenated-stream . - Output: - (((a . "alpha") (b . "beta")) ("omega") . 3) -.cble +The effect is that multiple files appear to be catenated together into a single +input stream. -In the above example, the pattern function -.code foo -is called with arguments -.codn (a b) . -These are unbound variables, so they correspond to parameters -.code x -and -.code y -of the function. If -.code x -and -.code y -get bound, those values propagate to -.code a -and -.codn b . -The data being matched consists of the lines -.strn alpha:beta , -.str gamma -and -.strn omega . -Inside -.codn foo, -.code x -and -.code y -bind to -.str alpha -and -.strn beta , -and then the line variable binds to -.strn gamma . -The input stream is left with -.strn omega . +If the optional +.meta alternative-stream +argument is supplied, then if +.meta path-list +is empty, +.meta alternative-stream +is returned instead of an empty catenated stream. -Hence, the return value consists of the bindings of -.code x -and -.code y -transferred to -.code a +The difference between +.code open-files and -.codn b , -and the second cons cell which gives information about the rest of the -stream: it is the part starting at -.strn omega , -which is line 3. Note that the binding for the -.code line -variable does not propagate -out of the pattern function -.codn foo ; -it is local inside it. +.code open-files* +is that +.code open-files +creates all of the +streams up-front. So if any of the paths cannot be opened, the operation throws. +The +.code open-files* +variant is lazy: it creates a lazy list of streams out of the +path list. The streams are opened as needed: before the second stream is opened, +the program has to read the first stream to the end, and so on. -.coNP Macro @ txr-if +.TP* Example: + +Collect lines from all files that are given as arguments on the command line. If +there are no files, then read from standard input: + +.cblk + @(next @(open-files *args* *stdin*)) + @(collect) + @line + @(end) +.cble + +.coNP Function @ abs-path-p .synb -.mets (txr-if < name <> ( argument *) < input < then-expr <> [ else-expr ]) +.mets (abs-path-p << path ) .syne .desc The -.code txr-if -macro invokes the \*(TX pattern matching function -.metn name -on some input given by the -.meta input -parameter, which is a list of strings, or a single string. +.code abs-path-function +tests whether the argument +.meta path +is an absolute path, returning a +.code t +or +.code nil +indication. -If -.meta name -succeeds, then -.meta then-expr -is evaluated, and if it fails, -.meta else-expr -is evaluated instead. +An absolute path is a string which either begins with a slash or backslash +character, or which begins with an alphanumeric word, followed by a colon, +followed by a slash or backslash. -In the successful case, -.meta then-expr -is evaluated in a scope in which the bindings emerging from the -.meta name -function are turned into \*(TL variables. -The result of -.code txr-if -is that of -.metn then-expr . +Examples of absolute paths: -In the failed case, -.meta else-expr -is evaluated in a scope which does not have any new bindings. -The result of -.code txr-if -is that of -.metn else-expr . -If -.meta else-expr -is missing, the result is -.codn nil . +.cblk + /etc + c:/tmp + ftp://user@server + disk0:/home + Z:\eUsers +.cble + +Examples of strings which are not absolute paths. + +.cblk +.mets < (the < empty < string) + . + abc + foo:bar/x + $:\eabc +.cble +.coNP Function @ read +.synb +.mets (read >> [ source >> [ error-stream >> [ error-return-value <> [ name ]]]]) +.syne +.desc The -.meta argument -forms supply arguments to the pattern function -.metn name . -There must be as many of these arguments as the function -has parameters. +.code read +function converts text denoting \*(TL structure, into the +corresponding data structure. The +.meta source +argument may be either a character +string, or a stream. If it is omitted, then +.code *stdin* +is used as the stream. -Any argument which is a symbol is treated, for the purposes -of calling the pattern function, as an unbound pattern variable. -The function may or may not produce a binding for that variable. -Also, every argument which is a symbol also denotes a local variable -that is established around -.meta then-expr -if the function succeeds. For any such pattern variable for which the function -produces a binding, the corresponding local variable will be initialized -with the value of that pattern variable. For any such pattern variable -which is left unbound by the function, the corresponding local variable -will be set to -.codn nil . +The source must provide the text representation of one complete \*(TL object. -Any -.meta argument -can be a form other than a symbol. In this situation, the argument is -evaluated, and will be passed to the pattern function as the value of -the binding for the corresponding argument. +Multiple calls to read on the same stream will extract successive objects +from the stream. To parse successive objects from a string, it is necessary +to convert it to a string stream. -.TP* Example: +The optional +.meta error-stream +argument can be used to specify a stream to which +parse errors diagnostics are sent. If absent, the diagnostics are suppressed. -.cblk - @(define date (year month day)) - @{year /\ed\ed\ed\ed/}-@{month /\ed\ed/}-@{day /\ed\ed/} - @(end) - @(do - (each ((date '("09-10-20" "2009-10-20" "July-15-2014" "foo"))) - (txr-if date (y m d) date - (put-line `match: year @y, month @m, day @d`) - (put-line `no match for @date`)))) +The optional +.meta name +argument can be used to specify the file name which is used for reporting +errors. If this argument is missing, the name is taken from the name +property of the +.meta source +argument if it is a stream, or else the word +.code string +is used as the name if +.meta source +is a string. - Output: +If there are no parse errors, the function returns the parsed data +structure. If there are parse errors, and the +.meta error-return-value +parameter is +present, its value is returned. If the +.meta error-return-value +parameter +is not present, then an exception of type +.code syntax-error +is thrown. - no match for 09-10-20 - match: year 2009, month 10, day 20 - no match for July-15-2014 - no match for foo -.cble +.SS* Filesystem Access -.coNP Macro @ txr-when +.coNP Function @ open-directory .synb -.mets (txr-when < name <> ( argument *) < input << form *) +.mets (open-directory << path ) .syne .desc The -.code txr-when -macro is based on -.codn txr-if . -It is equivalent to -.code +.code open-directory +function tries to create a stream which reads the +directory given by the string argument +.metn path . +If a filesystem object exists +under the path, is accessible, and is a directory, then the function +returns a stream. Otherwise, a file error exception is thrown. -.cblk -.meti \ \ (txr-if < name <> ( argument *) < input (progn << form *)) -.cble +The resulting stream supports the get-line operation. Each call to the +.code get-line +operation retrieves a string representing the next directory +entry. The value +.code nil +is returned when there are no more directory entries. +The +.code . +and +.code .. +entries in Unix filesystems are not skipped. -If the pattern function -.meta name -produces a match, then each -.meta form -is evaluated in the scope of the variables established by the -.meta argument -expressions. The result of the -.code txr-when -form is that of the last -.metn form . +.coNP Function @ open-file +.synb +.mets (open-file < path <> [ mode-string ]) +.syne +.desc +The +.code open-file +function creates a stream connected to the file +which is located at the given +.metn path , +which is a string. + +The +.meta mode-string +argument is a string which uses the same +conventions as the mode argument of the C language +.code fopen +function, with some extensions. +The mode string determines whether the stream is an input stream +or output stream. Note that the +.str b +mode is passed through to the C library, but has no special meaning to \*(TX. +Whether a stream is text or binary depends on which operations +are invoked on it. + +If the +.meta mode-string +argument is omitted, mode +.str r +is used. + +The option letter +.str i +is supported. If present, it will create a stream which has the real-time +property set. + +.coNP Function @ open-tail +.synb +.mets (open-tail < path >> [ mode-string <> [ seek-to-end-p ]]) +.syne +.desc +The +.code open-tail +function creates a tail stream connected to the file which is +located at the given +.metn path . +The +.meta mode-string +argument is a string which uses +the same conventions as the mode argument of the C language +.code fopen +function. If this argument is omitted, then +.str r +is used. +See the +.code open-file +function for a discussion of modes. -If the pattern function fails then the forms are not evaluated, -and the result value is +The +.code seek-to-end-p +argument is a boolean which determines whether the initial +read/write position is at the start of the file, or just past the end. +It defaults to .codn nil . +This argument only makes a difference if the file exists +at the time +.code open-tail +is called. If the file does not exist, and is later +created, then the tail stream will follow that file from the beginning. In +other words, +.meta seek-to-end-p +controls whether the tail stream reads all the +existing data in the file, if any, or whether it reads only newly added data +from approximately the time the stream is created. -.coNP Macro @ txr-case +A tail stream has special semantics with regard to reading at the end +of file. A tail stream never reports an end-of-file condition; instead +it polls the file until more data is added. Furthermore, if the file +is truncated, or replaced with a smaller file, the tail stream follows +this change: it automatically opens the smaller file and starts reading from +the beginning (the +.meta seek-to-end-p +flag only applies to the initial open). +In this manner, a tail stream can dynamically growing rotating log files. + +Caveat: since a tail stream can re-open a new file which has the same +name as the original file, it behave incorrectly if the program +changes the current working directory, and the path name is relative. + +.coNP Function @ remove-path .synb -.mets (txr-case < input-form -.mets \ \ >> {( name <> ( argument *) << form *)}* -.mets \ \ >> [( t << form *)]) +.mets (remove-path << path ) .syne .desc The -.code txr-case -macro evaluates -.meta input-form -and then uses the value as an input to zero or more test clauses. -Each test clause invokes the pattern function named by that clause's -.meta name -argument. - -If the function succeeds, then each -.meta form -is evaluated, and the value of the last -.meta form -is taken to be the result value of -.codn txr-case , -which terminates. If there are no forms, then -.code txr-case -terminates with a -.code nil -result. - -The forms are evaluated in an environment in which variables are bound -based on the -.meta argument -forms, with values depending on the result of the -invocation of the -.meta name -pattern function, in the same manner as documented in detail for the -.code txr-if -macro. +.code remove-path +function tries to remove the filesystem object named +by +.metn path , +which may be a file, directory or something else. -If the function fails, then the forms are not evaluated, and control passes to -the next clause. +If successful, it returns +.codn t . -A clause which begins with the symbol -.code t -executes unconditionally and causes -.code txr-case -to terminate. If it has no forms, then -.code txr-case -yields -.codn nil , -otherwise the forms are evaluated in order and the value of the last -one specifies the result of -.codn txr-case . +A failure to remove the object results in an exception of type +.codn file-error . -.SS* Quote/Quasiquote Operator Syntax -.coNP Operator @ quote +.coNP Function @ rename-path .synb -.mets (quote << form ) +.mets (rename-path < from-path << to-path ) .syne .desc The -.code quote -operator, when evaluated, suppresses the evaluation of -.metn form , -and instead returns -.meta form -itself as an object. For example, if -.meta form -is a symbol, then -.meta form -is not evaluated to the symbol's value; rather -the symbol itself is returned. - -Note: the quote syntax -.cblk -.meti >> ' -.cble -is translated to -.cblk -.meti (quote << form ). -.cble - -.TP* Example: +.code remove-path +function tries to rename filesystem path +.metn from-path , +which may refer to a file, directory or something else, to the path +.metn to-path . -.cblk - (quote a) ;; yields a +If successful, it returns +.codn t . - (quote (+ 2 2)) ;; yields (+ 2 2), not 4. -.cble +A failure results in an exception of type +.codn file-error . -.coNP Macro @ qquote +.SS* Coprocesses +.coNP Functions @ open-command and @ open-process .synb -.mets (qquote << form ) +.mets (open-command < system-command <> [ mode-string ]) +.mets (open-process < program < mode-string <> [ argument-list ]) .syne .desc -The -.code qquote -(quasi-quote) macro operator implements a notation for convenient -list construction. If -.meta form -is an atom, or a list structure which -does not contain any -.code unquote -or -.code splice -operators, then -.cblk -.meti (qquote << form ) -.cble -is equivalent to -.cblk -.meti (qquote << form ). -.cble - -If -.metn form , -however, is a list structure which contains -.code unquote -or -.code splice -operators, then the substitutions implied by those operators are performed -on -.metn form , -and the -.code qquote -operator returns the resulting structure. - -Note: how the qquote operator actually works is that it is compiled into -code. It becomes a Lisp expression which, when evaluated, computes the -resulting structure. +These functions spawn external programs which execute concurrently +with the \*(TX program. Both functions return a unidirectional stream for +communicating with these programs: either an output stream, or an input +stream, depending on the contents of +.metn mode-string . -A -.code qquote -can contain another -.codn qquote . -If an -.code unquote -or -.code splice -operator occurs -within a nested -.codn qquote , -it belongs to that -.codn qquote , -and not to the outer one. +In +.codn open-command , +the +.meta mode-string +argument is optional, defaulting to +the value +.str r +if it is missing. See the +.code open-file +function for a discussion of modes. -However, an unquote operator which occurs inside another one belongs one level -higher. For instance in +The +.code open-command +function accepts, via the +.meta system-command +string parameter, a +system command, which is in a system-dependent syntax. On a POSIX system, this +would be in the POSIX Shell Command Language. -.cblk - (qquote (qquote (unquote (unquote x)))) -.cble +The +.code open-process +function specifies a program to invoke via the +.meta command +argument. This is subject to the operating system's search strategy. +On POSIX systems, if it is an absolute or relative path, it is treated as +such, but if it is a simple base name, then it is subject to searching +via the components of the PATH environment variable. If open-process +is not able to find +.metn program , +or is otherwise unable to execute +the program, the child process will exit, using the value of the C variable +.code errno +as its exit status. This value can be retrieved via +.codn close-stream . -the leftmost -.code qquote -belongs with the rightmost unquote, and the inner -.code qquote -and -.code unquote -belong together. When the outer -.code qquote -is evaluated, -it will insert the value of -.codn x , -resulting in the object -.codn (qquote (unquote [value-of-x])) . -If this resulting qquote value is evaluated again as Lisp syntax, then it will -yield -.codn [value-of-value-of-x] , -the value of -.code [value-of-x] -when treated as a Lisp expression and evaluated. +The +.meta mode-string +argument follows the convention used by the POSIX +.code popen +function. -.TP* Examples: +The +.meta argument-list +argument is a list of strings which specifies additional +optional arguments to be passed passed to the program. The +.meta program +argument +becomes the first argument, and +.meta argument-string +become the second and +subsequent arguments. If +.meta argument-strings +is omitted, it defaults to empty. +If a coprocess is open for writing .cblk - (qquote a) -> a - - (qquote (a b c)) -> (a b c) - - (qquote (1 2 3 (unquote (+ 2 2)) (+ 2 3))) -> (1 2 3 4 (+ 2 3)) +.meti >> ( mode-string +.cble +is specified as +.strn w ), +then +writing on the returned stream feeds input to that program's standard input +file descriptor. Indicating the end of input is performed by closing the +stream. - (qquote (unquote (+ 2 2))) -> 4 +If a coprocess is open for reading +.cblk +.meti >> ( mode-string .cble +is specified as +.strn r ), +then +the program's output can be gathered by reading from the returned stream. +When the program finishes output, it will close the stream, which can be +detected as normal end of data. -In the second-to-last example, the -.code 1 2 3 -and the -.code (+ 2 3) -are quoted verbatim. -Whereas the -.code (unquote (+ 2 2)) -operator caused the evaluation of -.code (+ 2 2) -and the substitution of the resulting value. +If a coprocess terminates abnormally or unsuccessfully, an exception is raised. -The last example shows that -.meta form -can itself (the entire argument of -.codn qquote ) -can be an unquote operator. -However, note: -.code (quote (splice form)) -is not valid. +.SS* Symbols and Packages +A package is an object which serves as a container of symbols. -Note: a way to understand the nesting behavior is a via a possible model of -quasi-quote expansion which recursively compiles any nested quasi quotes first, -and then treats the result of their expansion. For instance, in the processing -of +A symbol which exists inside a package is said to be interned in that package. +A symbol can be interned in at most one package at a time. -.cblk - (qquote (qquote (unquote (unquote x)))) -.cble +Each symbol has a name, which is a string. It is not necessarily unique: +two distinct symbols can have the same name. However, a symbol name is unique +within a package, because it serves as the key which associates the +symbol with the package. Two symbols cannot be in the same package if they +have the same name. Moreover, a symbol cannot exist in more than one package +at at time, although it can be relocated from one package to another. A +symbols exist which is not entered into any package: such a symbol is +called "uninterned". -the -.code qquote -operator first encounters the -embedded -.code (qquote ...) -and compiles it to code. During that recursive -compilation, the syntax -.code (unquote (unquote x)) -is encountered. The inner quote -processes the outer unquote which belongs to it, and the inner -.code (unquote x) -becomes material that is embedded verbatim in the compilation, which will then -be found when the recursion pops back to the outer quasiquote, which will -then traverse the result of the inner compilation and find the -.codn (unquote x) . +Packages are held in a global list which can be used to search for a package by +name. The +.code find-package +function performs this lookup. A package may be +deleted from the list with the +.code delete-package +function, but it continues +to exist until the program loses the last reference to that package. -.TP* "Dialect note:" +.coNP Special variables @, *user-package* @, *keyword-package* and @ *system-package* +.desc +These variables hold predefined packages. The +.code *user-package* +is the one +in which symbols are read when a \*(TX program is being scanned. +The +.code *keyword-package* +holds keyword symbols, which are printed with +a leading colon. The +.code *system-package* +is for internal symbols, helping +the implementation avoid name clashes with user code in some situations. -In Lisp dialects which have a published quasiquoting operator syntax, there is -the expectation that the quasiquote read syntax corresponds to it. That is to -say, that for instance the read syntax -.code ^(a b ,c) -is expected translated to -.codn (qquote b (unquote c)) . +.coNP Function @ make-sym +.synb +.mets (make-sym << name ) +.syne +.desc +The +.code make-sym +function creates and returns a new symbol object. The argument +.metn name , +which must be a string, specifies the name of the symbol. The symbol +does not belong to any package (it is said to be "uninterned"). -In \*(TL, this is not true! Although -.code ^(b b ,c) -is translated to a -quasiquoting macro, it is an internal one, not based on the public -.codn qquote , -.code unquote -and -.code splice -symbols being documented here. +Note: an uninterned symbol can be interned into a package with the +.code rehome-sym +function. Also see the +.code intern +function. -This idea exists for hygiene. The quasiquote read syntax is not confused -by the presence of the symbols -.codn qquote , -.code unquote -or -.code splice -in the template, since it doesn't treat them specially. +.coNP Function @ gensym +.synb +.mets (gensym <> [ prefix ]) +.syne +.desc +The +.code gensym +function is similar to make-sym. It creates and returns a new +symbol object. If the +.meta prefix +argument is omitted, it defaults to +.strn g . +Otherwise it must be a string. -This also allows programmers to use the quasiquote read syntax to construct -quasiquote macros. For instance +The difference between +.code gensym +and +.code make-sym +is that +.code gensym +creates the name +by combining the prefix with a numeric suffix. -.cblk - ^(qquote (unquote ,x)) ;; does not mean ^^,x -.cble +The numeric suffix is a decimal digit string, taken from the value of +the variable +.codn *gensym-counter* , +after incrementing it. -To the quasiquote reader, the -.code qquote +Note: the variation in name is not the basis of the uniqueness assurance +offered by +.code make-sym and -.code unquote -symbols mean nothing special, -and so this syntax simply means that if the value of -.code x -is -.codn foo , -the result will be -.codn (qquote (unquote foo)) . - -The form's expansion is actually this: +.codn gensym ; +the basis is that the returned symbol is a freshly instantiated object. +.code make-sym +still returns unique symbols even if repeatedly called with the same +string. -.cblk - (sys:qquote (qquote (unquote (sys:unquote x)))) -.cble +.coNP Special variable @ *gensym-counter* +.desc +This variable is initialized to 0. Each time the +.code gensym +function is called, +it is incremented. The incremented value forms the basis of the numeric +suffix which +.code gensym +uses to form the name of the new symbol. -the -.code sys:qquote -macro recognizes -.code sys:unquote -embedded in the form, and -the other symbols not in the -.code sys: -package are just static template material. +.coNP Function @ make-package +.synb +.mets (make-package << name ) +.syne +.desc +The +.code make-package +function creates and returns a package named +.metn name , +where +.meta name +is a string. It is an error if a package by that name exists already. +.coNP Function @ packagep +.synb +.mets (packagep << obj ) +.syne +.desc The -.code sys:quote -macro and its associated -.code sys:unquote -and -.code sys:splice -operators work exactly like their ordinary counterparts. So in effect, \*(TX has -two nearly identical, independent quasi-quote implementations, one of which is -tied to the read syntax, and one of which isn't. This is useful for writing -quasiquotes which write quasiquotes. +.code packagep +function returns +.codet +if +.meta obj +is a package, otherwise it returns +.codn nil . + +.coNP Function @ find-package +.synb +.mets (find-package << name ) +.syne +.desc +The argument +.meta name +should be a string. If a package called +.meta name +exists, +then it is returned. Otherwise +.code nil +is returned. -.coNP Operator @ unquote +.coNP Function @ intern .synb -.mets (qquote (... (unquote << form ) ...)) -.mets (qquote (unquote << form )) +.mets (intern < name <> [ package ]) .syne .desc +The argument +.meta name +should be a symbol. The optional argument +.meta package +should be a package. If +.meta package +is not supplied, then the value +taken is that of +.codn *user-package* . + The -.code unquote -operator is not an operator -.I per -.IR se . -The -.code unquote -symbol has no -binding in the global environment. It is a special syntax that is recognized -within a -.code qquote -form, to indicate forms within the quasiquote which are to be -evaluated and inserted into the resulting structure. +.code intern +function searches +.meta package +for a symbol called +.metn name . +If that symbol is found, it is returned. If that symbol is not found, +then a new symbol called +.meta name +is created and inserted into +.metn package , +and that symbol is returned. In this case, the package becomes the +symbol's home package. -The syntax -.cblk -.meti (qquote (unquote << form )) -.cblk -is equivalent to -.metn form : -the -.code qquote +.coNP Function @ rehome-sym +.synb +.mets (rehome-sym < symbol <> [ package ]) +.syne +.desc +The arguments +.meta symbol and -.code unquote -"cancel out". +.meta package +must be a symbol and package object, +respectively. If +.meta package +is not given, then it defaults to the value of +.codn *user-package* . -.coNP Operator @ splice +The +.code rehome-sym +function moves +.meta symbol +into +.metn package . +If +.meta symbol +is already in a package, it is first removed from that package. +If a symbol of the same name exists in +.meta package +that symbol is first removed +from +.metn package . + +.coNP Function @ symbolp .synb -.mets (qquote (... (splice << form ) ...)) +.mets (symbolp << obj ) .syne .desc The -.code splice -operator is not an operator -.I per -.IR se . -The -.code splice -symbol has no -binding in the global environment. It is a special syntax that is recognized -within a -.code qquote -form, to indicate forms within the quasiquote which are to be -evaluated and inserted into the resulting structure. - -The syntax -.cblk -.meti (qquote (splice << form )) -.cble -is not permitted and raises an exception if evaluated. The -.code splice -syntax must occur within a list, and not in the dotted position. +.code symbolp +function returns +.code t +if +.meta obj +is a symbol, otherwise it returns +.codn nil . +.coNP Function @ symbol-name +.synb +.mets (symbol-name << symbol ) +.syne +.desc The -.code splice -form differs from unquote in that -.cblk -.meti (splice << form ) -.cble -requires that -.meta form -must evaluate to a list. That list is -integrated into the surrounding list. - -.SS* Macros -\*(TL supports structural macros. \*(TX's model of macroexpansion is that -\*(TL code is processed in two phases: the expansion phase and the -evaluation phase. The expansion phase is invoked on Lisp code early during the -processing of source code. For instance when a \*(TX file containing a -.code @(do ...) -directive -is loaded, expansion of the Lisp forms are its arguments takes place during the -parsing of the entire source file, and is complete before any of the code in -that file is executed. If the -.code @(do ...) -form is later executed, -the expanded forms are then evaluated. - -\*(TL also supports symbol macros, which are symbolic forms that stand -for forms, with which they are replaced at macro expansion time. - -When Lisp data is processed as code by the -.code eval -function, it is first expanded, -and so processed in its entirety in the expansion phase. Then it is processed -in the evaluation phase. - -.coNP Macro parameter lists - -\*(TX macros support destructuring, similarly to Common Lisp macros. -This means that macro parameter lists are like function argument lists, -but support nesting. A macro parameter list can specify a nested parameter -list in every place where a function argument list allows only a parameter -name. For instance, consider this macro parameter list: - -.cblk - ((a (b c)) : (c frm) ((d e) frm2 de-p) . g) -.cble - -The top level of this list has four elements: the mandatory parameter -.codn (a (b c)) , -the optional parameter -.code c -(with default init form -.codn frm ), -the optional parameter -.code (d e) -(with default init form -.code frm2 -and presence-indicating variable -.codn de-p ), -and the dot-position parameter -.code g -which captures trailing arguments. - -Note that some of the parameters are compounds: -.code (a (b c)) -and -.codn (d e) . -These compounds express nested macro parameter lists. +.code symbol-name +function returns the name of +.metn symbol . -Macro parameter lists match a similar tree structure to their own. -For instance a mandatory parameter -.code (a (b c)) -matches a structure like -.codn (1 (2 3)) , -such that the parameters -.codn a , -.code b -and -.code c -will end up bound -to -.codn 1 , -.code 2 -and -.codn 3 , -respectively. +.coNP Function @ symbol-package +.synb +.mets (symbol-package << symbol ) +.syne +.desc +The +.code symbol-package +function returns the home package of +.metn symbol . -The binding strictness is relaxed for optional parameters. If -.code (a (b c)) -is optional, and the argument is, say, -.codn (1) , -then -.code a -gets -.codn 1 , -and -.code b -and -.code c -receive +.coNP Function @ packagep +.synb +.mets (packagep << obj ) +.syne +.desc +The +.code packagep +function returns +.code t +if +.meta obj +is a package, otherwise it returns .codn nil . -Macro parameter lists also support two special keywords, namely -.code :env -and -.codn :whole . - -The parameter list -.code (:whole x :env y) -will bind parameter -.code x -to the entire -macro form, and bind parameter -.code y -to the macro environment. +.coNP Function @ keywordp +.synb +.mets (keywordp << obj ) +.syne +.desc The -.code :whole -and -.code :env -notation can occur anywhere in a macro parameter list. +.code keywordp +function returns +.code t +if +.meta obj +is a keyword symbol, otherwise it +returns +.codn nil . -.coNP Operator @ macro-time +.coNP Function @ bindable .synb -.mets (macro-time << form *) +.mets (bindable << obj ) .syne .desc The -.code macro-time -operator has a syntax similar to the -.code progn -operator. Each -.meta form -is evaluated from left to right, and the resulting value is that of the last -form. - -The special behavior of -.code macro-time -is that the evaluation takes place during -the expansion phase, rather than during the evaluation phase. - -During the expansion phase, all -.code macro-time -expressions which occur in a context -that calls for evaluation are evaluated, and replaced by their quoted values. -For instance -.code (macro-time (list 1 2 3)) -evaluates -.code (list 1 2 3) -to the object -.code (1 2 3) -and the entire -.code macro-time -form is replaced by that value, quoted: -.codn '(1 2 3) . -If the form is evaluated again at evaluation-time, the resulting value will be -that of the quote, in this case -.codn (1 2 3) . +.code bindable +function returns +.code t +if +.meta obj +is a bindable symbol, otherwise it returns +.codn nil . -.code macro-time -forms do not see the surrounding lexical environment; the see only -global function and variable bindings and macros. +All symbols are bindable, except for keyword symbols, and the +special symbols +.code t +and +.codn nil. -Note 1: -.code macro-time -is intended for defining helper functions and variables that -are used by macros. A macro cannot "see" a -.code defun -function or -.code defvar -variable -because -.code defun +.SS* Pseudo-random Numbers +.coNP Special variable @ *random-state* +.desc +The +.code *random-state* +variable holds an object which encapsulates the state +of a pseudo-random number generator. This variable is the default argument +value for the +.code random-fixnum and -.code defvar -forms are evaluated at evaluation time, which occurs -after expansion time. The macro-time operator hoists the evaluation of these -forms to macro-expansion time. +.codn random functions , +for the convenience of writing programs which are not concerned about the +management of random state. -Note 2: -.code defmacro -forms are implicitly in macro-time; they do not have to -be wrapped with the -.code macro-time -operator. The -.code macro-time -operator doesn't have -to be used in programs whose macros do not make references to variables -or functions. +On the other hand, programs can create and manage random states, making it +possible to obtain repeatable sequences of pseudo-random numbers which do not +interfere with each other. For instance objects or modules in a program can +have their own independent streams of random numbers which are repeatable, +independently of other modules making calls to the random number functions. -.coNP Operator @ defmacro +When \*(TX starts up, the +.code *random-state* +variable is initialized with +a newly created random state object, which is produced as if by +the call +.codn (make-random-state 42) . + +.coNP Function @ make-random-state .synb -.mets (defmacro < name <> ( param * [: << opt-param * ] [. < rest-param ]) -.mets \ \ << body-form *) +.mets (make-random-state <> [ seed ]) .syne .desc The -.code defmacro -operator is evaluated at expansion time. It defines a -macro-expander function under the name -.metn name , -effectively creating a new operator. +.code make-random-state +function creates and returns a new random state, +an object of the same kind as what is stored in the +.code *random-state* +variable. -Note that the parameter list is a macro parameter list, and not a -function parameter list. This means that each -.meta param -and -.meta opt-param -can be not only a symbol, but it can itself be a parameter list. -The corresponding argument is then treated as a structure which -matches that parameter list. This nesting of parameter lists -can be carried to an arbitrary depth. +The seed, if specified, must be either an integer value, or an +existing random state object. -A macro is called like any other operator, and resembles a function. Unlike in -a function call, the macro receives the argument expressions themselves, rather -than their values. Therefore it operates on syntax rather than on values. -Also, unlike a function call, a macro call occurs in the expansion phase, -rather than the evaluation phase. +Note that the sign of the seed is ignored, so that negative seed +values are equivalent to their additive inverses. -The return value of the macro is the macro expansion. It is substituted in -place of the entire macro call form. That form is then expanded again; -it may itself be another macro call, or contain more macro calls. +If the seed is not specified, then +.code make-random-state +produces a seed based +on some information in the process environment, such as current +time of day. It is not guaranteed that two calls to +.code (make-random-state) +that are separated by less than some minimum increment of real time produce +different seeds. The minimum time increment depends on the platform. -.TP* Example: +On a platform with a millisecond-resolution real-time clock, the minimum +time increment is a millisecond. Calls to make-random-state less than +a millisecond apart may predictably produce the same seed. -.cblk - ;; dolist macro similar to Common Lisp's: - ;; - ;; The following will print 1, 2 and 3 - ;; on separate lines: - ;; and return 42. - ;; - ;; (dolist (x '(1 2 3) 42) - ;; (format t "~s\en")) +If an integer seed is specified, then the integer value is mapped to a +pseudo-random sequence, in a platform-independent way. - (defmacro dolist ((var list : result) . body) - (let ((i (my-gensym))) - ^(for ((i ,list)) (i ,result) ((set i (cdr i))) - (let ((,var (car i))) - ,*body)))) -.cble +If a random state is specified as a seed, then it is duplicated. The +returned random state object is a distinct object which is in the same +state as the input object. It will produce the same remaining pseudo-random +number sequence, as will the input object. -.coNP Operator @ macrolet +.coNP Function @ random-state-p .synb -.mets (macrolet >> ({( name < macro-style-params << macro-body-form *)}*) -.mets \ \ << body-form *) +.mets (random-state-p << obj ) .syne .desc The -.code macrolet -binding operator extends the macro-time lexical environment -by making zero or more new local macros visible. +.code random-state-p +function returns +.code t +if +.meta obj +is a random state, otherwise it +returns +.codn nil . -The -.code macrolet -symbol is followed by a list of macro definitions. -Each definition is a form which begins with a -.metn name , -followed by -.meta macro-style-params -which is a macro parameter list, and zero or more -.metn macro-body-form s. -These macro definitions are similar -to those globally defined by the -.code defmacro -operator, except that they -are in a local environment. +.coNP Functions @, random-fixnum @ random and @ rand +.synb +.mets (random-fixnum <> [ random-state ]) +.mets (random < random-state << modulus ) +.mets (rand < modulus <> [ random-state ]) +.syne +.desc +All three functions produce pseudo-random numbers, which are positive integers. -The macro definitions are followed by optional -.metn body-forms . -The macros specified in the definitions are visible to these -forms. +The numbers are obtained from a WELLS 512 pseudo-random number generator, whose +state is stored in the random state object. -Forms inside the macro definitions such as the -.metn macro-body-form s, -and initializer forms appearing in the -.meta macro-style-params -are subject -to macro-expansion in a scope in which none of the new macros being -defined are yet visible. Once the macro definitions are themselves -macro-expanded, they are placed into a new macro environment, which -is then used for macro expanding the -.metn body-form s. +The +.code random-fixnum +function produces a random fixnum integer: a reduced range +integer which fits into a value that does not have to be heap-allocated. -A -.code macrolet -form is fully processed in the expansion phase of a form, and is -effectively replaced by -.code progn -form which contains expanded versions of -.metn body-form s. -This expanded structure shows no evidence that any -macrolet forms ever existed in it. Therefore, it is impossible for the code -evaluated in the bodies and parameter lists of -.code macrolet -macros to have any visibility to any surrounding lexical variable bindings, -which are only instantiated in the evaluation phase, after expansion is done -and macros no longer exist. +The +.code random +and +.code rand +functions produce a value in the range [0, +.metn modulus ). +They differ only in the order of arguments. In the +.code rand +function, the random state +object is the second argument and is optional. If it is omitted, the global +.code *random-state* +object is used. -.coNP Function @ macro-form-p +.SS* Time +.coNP Functions @ time and @ time-usec .synb -.mets (macro-form-p < obj <> [ env ]) + (time) + (time-usec) .syne .desc The -.code macro-form-p -function returns -.code t -if -.meta obj -represents the syntax of -a form which is a macro form: either a compound macro or a symbol macro. -Otherwise it returns -.codn nil . - -A macro form is one that will transform under -.code macroexpand-1 -or -.codn macroexpand ; -an object which isn't a macro form will not undergo expansion. - -The optional -.meta env -parameter is a macroexpansion environment. -A macroexpansion environment is passed down to macros and can be received -via their special -.code :env -parameter. - -.meta env -is used by -.code macro-form-p -to determine whether -.meta obj -is a macro in a lexical macro environment. - -If -.meta env -is not specified or is -.codn nil , -then -.code macro-form-p -only recognizes global macros. +.code time +function returns the number of seconds that have elapsed since +midnight, January 1, 1970, in the UTC timezone. -.TP* Example: +The +.code time-usec +function returns a cons cell whose +.code car +field holds the seconds measured in the same way, and whose +.code cdr +field extends the precision by giving +number of microseconds as an integer value between 0 and 999999. -.cblk - ;; macro which translates to 'yes if its - ;; argument is a macro from, or otherwise - ;; transforms to the form 'no. +.coNP Functions @ time-string-local and @ time-string-utc +.synb +.mets (time-string-local < time << format ) +.mets (time-string-utc < time << format ) +.syne +.desc +These functions take the numeric time returned by the +.code time +function, and convert it to a textual representation in a flexible way, +according to the contents of the +.meta format +string. - (defmacro ismacro (:env menv form) - (if (macro-form-p form menv) - ''yes ''no)) +The +.code time-string-local +function converts the time to the local timezone of +the host system. The +.code time-string-utc +function produces time in UTC. - (macrolet ((local ())) - (ismacro (local))) ;; yields yes +The +.meta format +argument is a string, and follows exactly the same conventions as +the format string of the C library function +.codn strftime . - (ismacro (local)) ;; yields no +The +.meta time +argument is an integer representing seconds obtained from the +time function or from the +.code car +field of the cons returned by the +.code time-usec +function. - (ismacro (ismacro foo)) ;; yields yes -.cble +.coNP Functions @ time-fields-local and @ time-fields-utc +.synb +.mets (time-fields-local << time ) +.mets (time-fields-utc << time ) +.syne +.desc +These functions take the numeric time returned by the time function, +and convert it to a list of seven fields. -During macro expansion, the global macro -.code ismacro -is handed the macro-expansion environment -via -.codn :env menv . +The +.code time-string-local +function converts the time to the local timezone of +the host system. The +.code time-string-utc +function produces time in UTC. -When the macro is invoked within the macrolet, -this environment includes the macro-time lexical scope in which the -.code local -macro is defined. So when global checks whether the argument form -.code (local) -is a macro, the conclusion is yes: the (local) form is a macro -call in that environment: -.code macro-form-p -yields -.codn t . +The fields returned as a list consist of six integers, and a boolean value. +The six integers represent the year, month, day, hour, minute and second. +The boolean value indicates whether daylight savings time is in effect +(always +.code nil +in the case of +.codn time-fields-utc ). -When -.code (global (local)) -is invoked outside of the macrolet, no local macro is visible is -there, and so -.code macro-form-p -yields -.codn nil . +The +.meta time +argument is an integer representing seconds obtained from the +.code time +function or from the +.code time-usec +function. -.coNP Functions @ macroexpand-1 and @ macroexpand +.coNP Functions @ make-time and @ make-time-utc .synb -.mets (macroexpand-1 < obj <> [ env ]) -.mets (macroexpand < obj <> [ env ]) +.mets (make-time < year < month < day < hour < minute < second << dst-advice ) +.mets (make-time-utc < year < month < day < hour < minute < second << dst-advice ) .syne .desc +The +.code make-time +function returns a time value, similar to the one returned by the +.code time +function. The +.code time +value is constructed not from the system clock, but +from a date and time specified as arguments. The +.meta year +argument is a calendar year, like 2014. +The +.meta month +argument ranges from 1 to 12. +The +.meta hour +argument is a 24-hour time, ranging from 0 to 23. +These arguments represent a local time, in the current time zone. + +The +.meta dst-advice +argument specifies whether the time is expressed in +daylight savings time (DST). It takes on three possible values: +.codn nil , +the keyword +.codn :auto , +or else the symbol +.codn t . +Any other value has the same interpretation as +.codn t . + If -.meta obj -is a macro form (an object for which -.code macro-form-p -returns -.codn t ), -these functions expand the macro form and return the expanded form. -Otherwise, they return -.metn obj . +.meta dst-advice +is +.codn t , +then the time is assumed to be expressed in DST. +If the argument is +.codn nil , +then the time is assumed not to be in DST. +If +.meta dst-advice +is +.codn :auto , +then the function tries to determine whether +DST is in effect in the current time zone for the specified date and time. -.code macroexpand-1 -performs a single expansion, expanding just the macro -that is referenced by the symbol in the first position of -.metn obj , -and returns the expansion. That expansion may itself be a macro form. +The +.code make-time-utc +function is similar to +.codn make-time , +except that +it treats the time as UTC rather than in the local time zone. +The +.meta dst-advice +argument is supported by +.code make-time-utc +for function +call compatibility with +.codn make-time . +It may or may not have any effect +on the output (since the UTC zone by definition doesn't have daylight +savings time). -.code macroexpand -performs an expansion similar to -.codn macroexpand-1 . -If the result is -a macro form, then it expands that form, and keeps repeating this process -until the expansion yields a non-macro-form. That non-macro-form is then -returned. +.SS* Environment Variables and Command Line -The optional -.meta env -parameter is a macroexpansion environment. -A macroexpansion environment is passed down to macros and can be received -via their special -.code :env -parameter. The environment they receive is their -lexically apparent macro-time environment in which local macros may be -visible. A macro can use this environment to "manually" expand some -form in the context of that environment. +Note that environment variable names, their values, and command line +arguments are all regarded as being externally encoded in UTF-8. \*(TX performs +the encoding and decoding automatically. -.TP* Example: +.coNP Special variables @ *args* and @ *args-full* +.desc +The +.code *args* +variable holds a list of strings representing the remaining +arguments which follow any options processed by the \*(TX executable, +and the script name. -.cblk - ;; (foo x) expands x, and if x begins with a number, - ;; it removes the number and returns the resulting - ;; form. Otherwise, it returns the entire form. +The +.code *args-full* +variable holds the original, complete list of arguments passed +from the operating system. - (defmacro rem-num (:env menv some-form) - (let ((expanded (macroexpand some-form menv))) - (if (numberp (car expanded)) - (cdr expanded) - some-form))) +Note: the +.code *args* +variable is +.code nil +during the processing of the command line, +so \*(TL expressions invoked using the +.code -p +or +.code -e +option cannot use it. - ;; The following yields (42 a). +.coNP Function @ env +.synb + (env) +.syne +.desc +The +.code env +function retrieves the list of environment variables. Each +variable is represented by a single entry in the list: a string which +contains an +.code = +(equal) character somewhere, separating the variable name +from its value. - (macrolet ((foo () '(1 list 42)) - (bar () '(list 'a))) - (list (rem-num (foo)) (rem-num (bar))))) -.cble +See also: the +.code env-hash +function. +.coNP Function @ env-hash +.synb + (env-hash) +.syne +.desc The -.code rem-num -macro is able to expand the -.code (foo) -and -.code (bar) -forms it receives as -the -.code some-form -argument, even though these forms use local macro that are only -visible in their local scope. This is thanks to the macro -environment passed to -.codn rem-num . -It is correctly able to work with the -expansions -.code (1 list 42) -and -.code (list 'a) -to produce -.code (list 42) -and -.code (list 'a) -which evaluate to -.code 42 -and -.code a -respectively. +.code env-hash +function constructs and returns an +.code :equal-based +hash. The hash is +populated with the environment variables, represented as key-value pairs. -.coNP Functions @ lexical-var-p and @ lexical-fun-p +.coNP Functions @, getenv @, setenv and @ unsetenv .synb -.mets (lexical-var-p < env << form ) -.mets (lexical-fun-p < env << form ) +.mets (getenv << name ) +.mets (setenv < name < value <> [ overwrite-p ]) +.mets (unsetenv << name ) .syne .desc -These two functions are useful to macro writers. They are intended -to be called from the bodies of macro expanders, such as the bodies of -.code defmacro -or -.code macrolet -forms. The -.meta env -argument is a macro-time environment, which is available to macros -via the special -.code :env -parameter. Using these functions, a macro can enquire whether -a given -.meta form -is a symbol which has a variable binding or a function binding -in the lexical environment. -This information is known during macro expansion. The macro expander -recognizes lexical function and variable bindings, because these -bindings can shadow macros. +These functions provide access to, as well as manipulation of, environment +variables. Of these three, +.code setenv +and +.code unsetenv +might not be available on some platforms, or +.code unsetenv +might be be present in a simulated form which sets the variable +.meta name +to the empty string rather than deleting it. -.TP* Example: +The +.code getenv +function searches the environment for the environment variable whose name +is +.metn name . +If the variable is found, its value is returned. Otherwise +.code nil +is returned. -.cblk - ;; - ;; this macro replaces itself with :lexical-var if its - ;; argument is a lexical variable, :lexical-fun if - ;; its argument is a lexical function, or with - ;; :not-lex-fun-var if neither is the case. - ;; - (defmacro classify (sym :env e) - (cond - ((lexical-var-p e expr) :lexical-var) - ((lexical-fun-p e expr) :lexical-fun) - (t :not-lex-fun-var))) +The +.code setenv +function creates or modifies the environment variable indicated by +.metn name . +The +.meta value +string argument specifies the new value for the variable. - ;; - ;; This returns: - ;; - ;; (:lexical-var :not-lex-fun-var :lexical-fun) - ;; - (let ((x 1) (y 2)) - (symacrolet ((y x)) - (flet ((f () (+ 2 2))) - (list (classify x) (classify y) (classify z))))) -.cble +If the +.meta overwrite-p +argument is specified, and is true, +then the variable is overwritten if it already exists. +If the argument is false, then the variable is not modified if it +already exists. If the argument is not specified, it defaults +to the value +.metn t, +effectively giving rise to a two-argument form of +.code setenv +which creates or overwrites environment variables. -.TP* Note: +The +.code setenv +function unconditionally returns +.meta value +regardless of whether or not it overwrites an existing variable. -These functions do not call -.code macroexpand -on the form. In most cases, it is necessary for the macro writers -to do so. Not that in the above example, symbol -.code y -is classified as neither a lexical function nor variable. -However, it can be macro-expanded to -.code x -which is a lexical variable. +The +.code unsetenv +function removes the environment variable +specified by +.metn name , +if it exists. On some platforms, it instead sets the environment variable +to the empty string. -.coNP Function @ lexical-lisp1-binding +.SS* System Programming +.coNP Accessor @ errno .synb -.mets (lexical-lisp1-binding < env << symbol ) +.mets (errno <> [ new-errno ]) +.mets (set (errno) << new-value ) .syne .desc The -.code lexical-lisp1-binding -function inspects the macro-time environment -.meta env -to determine what kind of binding, if any, does -.meta symbol -have in that environment, from a Lisp-1 perspective. - -That is to say, it considers function bindings, variable bindings -and symbol macro bindings to be in a single name space and finds -the innermost binding of one of these types for -.metn symbol . - -If such a binding is found, then the function returns one of -the three keyword symbols -.codn :var , -.codn :fun , -or -.codn :symacro . +.code errno +function retrieves the current value of the C library error variable +.codn errno . +If the argument +.meta new-errno +is present and is not +.codn nil , +then it +specifies a value which is stored into +.codn errno . +The value returned is the prior value. -If no such lexical binding is found, then the function -returns -.codn nil . +The place form of +.code errno +does not take an argument. -Note that a +.coNP Function @ exit +.synb +.mets (exit << status ) +.syne +.desc +The +.code exit +function terminates the entire process (running \*(TX image), specifying +the termination status to the operating system. Values of +.meta status +may be +.codn nil , +.codn t , +or an integer value. The value .code nil -return doesn't mean that the symbol doesn't have a lexical binding. It could -have an operator macro lexical binding (a macro binding in the function -namespace established by -.codn macrolet ). +corresponds to the C constant +.codn EXIT_FAILURE , +and +.code t +corresponds to +.codn EXIT_SUCCESS . +These are platform-independent +indicators of failed or successful termination. The numeric value 0 also +indicates success. -.coNP Operator @ defsymacro +.coNP Function @ abort .synb -.mets (defsymacro < sym << form ) +.mets (abort) .syne .desc +The +.code abort +function terminates the entire process (running \*(TX image), specifying +an abnormal termination status to the process. -A -.code defsymacro -form introduces a symbol macro: a symbol macro is a parameterless -substitution keyed to a symbol. In contexts where a symbol macro -definition of -.meta sym -is visible, if the form -.meta sym -appears such that its evaluation is called for, it is subject -to replacement by -.metn form . -After replacement takes place, -.meta form -itself is then processed for further replacement of macros and -symbol macros. - -Symbol macros are also recognized in contexts -where -.meta sym -denotes a place which is the target of an assignment operation -like -.code set -and similar. - -A -.code defsymacro -form is implicitly executed at expansion time, and thus need -not be wrapped in a -.code macro-time -form, just like -.codn defmacro . - -Note: if a symbol macro expands to itself directly, expansion stops. However, -if a symbol macro expands to itself through a chain of expansions, -an infinite expansion time loop results. +Note: +.code abort +calls the C library function +.code abort +which works by raising the +.code SIG_ABRT +signal, known in \*(TX as the +.code sig-abrt +variable. Abnormal termination of the process is this signal's +default action. -.coNP Operator @ symacrolet +.coNP Function @ usleep .synb -.mets (symacrolet >> ({( sym << form )}*) << body-form *) +.mets (usleep << usec ) .syne .desc The -.code symacrolet -operator binds local, lexically scoped macros that are -similar to the global symbol macros introduced by -.codn defsymacro . - -Each -.meta sym -in the bindings list is bound to its corresponding form, creating a -new extension of the expansion-time lexical macro environment. - -Each -.meta body-form -is subsequently macro-expanded in this new environment -in which the new symbol macros are visible. +.code usleep +function suspends the execution of the program for at least +.meta usec +microseconds. -Note: ordinary lexical bindings such as those introduced by let or by -function parameters lists shadow symbol macros. If a symbol -.code x -is bound by nested instances of -.code macrolet -and a -.codn let , -then the scope enclosed by both -constructs will see whichever of the two bindings is more inner, -even though the bindings are active in completely separate phases of -processing. +The return value is +.code t +if the sleep was successfully executed. A +.code nil +value indicates premature wakeup or complete failure. -From the perspective of the arguments of a -.code dwim -form, lexical function bindings also shadow symbol macros. -This is consistent with the Lisp-1-style name resolution which -applies inside a -.code dwim -form. Of course, lexical operator macros do not shadow -symbol macros under any circumstances. +Note: the actual sleep resolution is not guaranteed, and depends on granularity +of the system timer. Actual sleep times may be rounded up to the nearest 10 +millisecond multiple on a system where timed suspensions are triggered by a 100 +Hz tick. -.coNP Macros @ placelet and @ placelet* +.coNP Functions @ mkdir and @ ensure-dir .synb -.mets (placelet >> ({( sym << place )}*) << body-form *) -.mets (placelet* >> ({( sym << place )}*) << body-form *) +.mets (mkdir < path <> [ mode ]) +.mets (ensure-dir < path <> [ mode ]) .syne .desc +.code mkdir +tries to create the directory named +.meta path +using the POSIX +.code mkdir +function. +An exception of type +.code file-error +is thrown if the function fails. Returns +.code t +on success. + The -.code placelet -macro binds lexically scoped symbol macros in such -a way that they behave as aliases for places -denoted by place forms. +.meta mode +argument specifies the request numeric permissions +for the newly created directory. If omitted, the requested permissions are +.code #o777 +(511): readable and writable to everyone. The requested permissions +are subject to the system +.codn umask . -Each -.meta place -must be an expression denoting a syntactic place. The -corresponding -.meta sym -is established as an alias for the storage location which that place denotes, -over the scope of the -.metn body-form -s. +The function +.code ensure-dir +is similar to +.code mkdir +except that it attempts to create all the missing parent directories +as necessary, and does not throw an error if the directory exists. -This binding takes place in such a way that each -.meta place -is evaluated exactly once, only in order to determine its -storage location. The corresponding -.meta sym -then serves as an alias for that location, over the -scope of the -.metn body-form -s. -This means that whenever -.meta sym -is evaluated, it stands for the value of the storage -location, and whenever a value is apparently stored into -.metn sym , -it is actually the storage location which receives it. +.coNP Function @ chdir +.synb +.mets (chdir << path ) +.syne +.desc +.code chdir +changes the current working directory to +.metn path , +and returns +.metn t , +or else throws an exception of type +.codn file-error . +.coNP Function @ pwd +.synb + (pwd) +.syne +.desc The -.code placelet* -variant implements an alternative scoping rule, which allows a later -.meta place -form to refer to a -.meta sym -bound to an earlier -.meta place -form. In other words, a given -.meta sym -binding is visible not only to the -.metn body-form -s -but also to -.meta place -forms which occur later. - -Note: certain kinds of places, notably -.cblk -.meti (force << promise ) -.cble -expressions, must be accessed before they can be stored, -and this restriction continues to hold when those -places are accessed through -.code placelet -aliases. +.code pwd +function retrieves the current working directory. +If the underlying +.code getcwd +C library function fails with an +.code errno +other than +.codn ERANGE , +an exception will be thrown. -Note: -.code placelet -differs from -.code symacrolet -in that the forms themselves are not aliased, but the storage -locations which they denote. -.code (symacrolet ((x y)) z) -performs the syntactic substitution of symbol -.code x -by form -.codn y , -wherever -.code x -appears inside -.code z -as an evaluated form, and is not shadowed by any inner binding. -Whereas -.code (placelet ((x y)) z) -generates code which arranges for -.code y -to be evaluated to a storage location, and syntactically replaces occurrences -of -.code x -with a form which directly denotes that storage location, -wherever -.code x -appears inside -.code z -as an evaluated form, and is not shadowed by any inner binding. -Also, -.code x -is not necessarily substituted by a single, fixed form, -as in the case of -.codn symacrolet . -Rather it may be substituted by one kind of form when it -is treated as a pure value, and another kind of form -when it is treated as a place. +.coNP Functions @ sh and @ run +.synb +.mets (sh << system-command ) +.mets (run < program <> [ argument-list ]) +.syne +.desc +The +.code sh +function executes +.meta system-command +using the system command interpreter. +The run function spawns a +.metn program , +searching for it using the +system PATH. Using either method, the executed process receives environment +variables from the parent. -.TP* "Example:" +\*(TX blocks until the process finishes executing. If the program terminates +normally, then its integer exit status is returned. The value zero indicates +successful termination. -Implementation of -.code inc -using -.codn placelet : +The return value +.code nil +indicates an abnormal termination, or the inability +to run the process at all. -.cblk - (defmacro inc (place : (delta 1)) - (with-gensyms (p) - ^(placelet ((,p ,place)) - (set ,p (+ ,p ,delta))))) -.cble +In the case of the +.code run +function, if the child process is created successfully +but the program cannot be executed, then the exit status will be an +.code errno +value from the failed +.code exec +attempt. -The gensym -.code p -is used to avoid accidental capture of references -emanating from the -.code delta -form. +.SS* Unix Filesystem Manipulation -.coNP Operators @ tree-bind and @ mac-param-bind +.coNP Function @ stat .synb -.mets (tree-bind < macro-style-params < expr << form *) -.mets (mac-param-bind < context-expr < macro-style-params < expr << form *) +.mets (stat << path ) .syne .desc The -.code tree-bind -operator evaluates -.codn expr , -and then uses the -resulting value as a counterpart to a macro-style parameter list. -If the value has a tree structure which matches the parameters, -then those parameters are established as bindings, and the -.metn form s, -if any, are evaluated in the scope of those bindings. The value -of the last -.meta form -is returned. If there are no forms, -.code nil -is returned. +.code stat +function inquires the filesystem about the existence of an object +denoted by the string +.metn path . +If the object is not found or cannot be +accessed, an exception is thrown. -Note: this operator throws an exception if there is a -structural mismatch between the parameters and the value of -.codn expr . +Otherwise, information is retrieved about the object. The information takes the +form of a property list in which keyword symbols denote numerous properties. +An example such property list is: -One way to avoid this exception is to use -.codn tree-case . +.cblk + (:dev 2049 :ino 669944 :mode 16832 :nlink 23 + :uid 500 :gid 500 :rdev 0 + :size 12288 :blksize 4096 :blocks 24 + :atime 1347933533 :mtime 1347933534 :ctime 1347933534) +.cble + +These properties correspond to the similarly-named entries of the +.code struct stat +structure in POSIX. For instance, the +.code :dev +property has the same value as the +.code st_dev +field. + +.coNP Special variables @, s-ifmt @, s-iflnk @, s-ifreg @, s-ifblk ... , @ s-ixoth + +The following variables exist, having integer values. These are bitmasks +which can be applied against the value given by the +.code :mode +property +in the property list returned by the function stat: +.codn s-ifmt , +.codn s-ifsock , +.codn s-iflnk , +.codn s-ifreg , +.codn s-ifblk , +.codn s-ifdir , +.codn s-ifchr , +.codn s-ififo , +.codn s-isuid , +.codn s-isgid , +.codn s-isvtx , +.codn s-irwxu , +.codn s-irusr , +.codn s-iwusr , +.codn s-ixusr , +.codn s-irwxg , +.codn s-irgrp , +.codn s-iwgrp , +.codn s-ixgrp , +.codn s-irwxo , +.codn s-iroth , +.code s-iwoth +and +.codn s-ixoth . + +These variables correspond to the C language constants from POSIX: +.codn S_IFMT , +.codn S_IFLNK , +.code S_IFREG +and so forth. The -.code mac-param-bind -operator is similar to -.code tree-bind -except that it takes an extra argument, -.metn context-expr. -This argument is an expression which is evaluated. It is expected to -evaluate to a compound form. If an error occurs during binding, the error -diagnostic message is based on information obtained from this form. -By contrast, the -.code tree-bind -operator's error diagnostic refers to the -.code tree-bind -form, which is cryptic if the binding is used for the implementation -of some other construct, hidden from the user of that construct. +.code logtest +function can be used to test these against values of mode. +For example +.code (logtest mode s-irgrp) +tests for the group read permission. -.coNP Operator @ tree-case +.coNP Functions @, makedev @ minor and @ major .synb -.mets (tree-case < expr >> {( macro-style-params << form *)}*) +.mets (makedev < minor << major ) +.mets (minor << dev ) +.mets (major << dev ) .syne .desc +The parameters +.metn minor , +.meta major +and +.meta dev +are all integers. The +.code makedev +function constructs a combined device number from a minor and major pair (by +calling the Unix +.code makedev +function). This device number is suitable as an +argument to the +.codee mknod +function (see below). Device numbers also appear the +.code :dev +property returned by the +.code stat +function. + The -.code tree-case -operator evaluates -.meta expr -and matches it against a succession -of zero or more cases. Each case defines a pattern match, expressed as a macro -style parameter list -.metn macro-style-params . +.code minor +and +.code major +functions extract the minor and major device number +from a combined device number. -If the object produced by -.meta expr -matches -.metn macro-style-params , -then the parameters are bound, becoming local variables, and the -.metn form s, -if any, are evaluated in order in the environment in which those variables are -visible. If there are forms, the value of the last -.meta form -becomes the result -value of the case, otherwise the result value of the case is nil. +.coNP Function @ mknod +.synb +.mets (chmod < path << mode ) +.syne +.desc +The +.code chmod +function changes the permissions of the filesystem objects +specified by +.metn path . +It is a direct wrapper for the POSIX C library function of the same name. -If the result value of a case is the object -.code : -(the colon symbol), then processing continues with the next case. Otherwise the -evaluation of -.code tree-case -terminates, returning the result value. +The permissions are specified by +.metn mode , +an integer argument. -If the value of -.meta expr -does not match the -.meta macro-style-params -parameter list of a case, processing continues with the next case. +The existing permissions may be obtained using the +.code stat +function. + +The function throws a +.code file-error +exception if an error occurs, otherwise it returns +.codn t. + +.TP* Example: +.cblk + ;; Set permissions of foo.txt to "rw-r--r--" + ;; (owner can read and write; group owner + ;; and other users can only read). + + ;; numerically: + (chmod "foo.txt" #o644) + + ;; symbolically: + (chmod "foo.txt" (logior s-irusr s-iwusr + s-irgrp + s-iroth)) +.cble + +.coNP Function @ mknod +.synb +.mets (mknod < path < mode <> [ dev ]) +.syne +.desc +The +.code mknod +function tries to create an entry in the filesystem: a file, +FIFO, or a device special file, under the name +.metn path . +If it is successful, +it returns +.codn t , +otherwise it throws an exception of type +.codn file-error . -If no cases match, then -.code tree-case -terminates, returning -.codn nil . +The +.meta mode +argument is a bitwise or combination of the requested permissions, +and the type of object to create: one of the constants +.codn s-ifreg , +.codn s-ififo , +.codn s-ifchr , +.code s-ifblk +or +.codn s-ifsock . +The permissions are subject to the system +.codn umask . + +If a block or character special device +.cod2 ( s-ifchr +or +.codn s-ifblk ) +is being +created, then the +.meta dev +argument specifies the major and minor numbers +of the device. A suitable value can be constructed from a major and minor +pair using the +.code makedev +function. .TP* Example: .cblk - ;; reverse function implemented using tree-case + ;; make a character device (8, 3) called /dev/foo + ;; requesting rwx------ permissions - (defun tb-reverse (obj) - (tree-case obj - (() ()) ;; the empty list is just returned - ((a) obj) ;; one-element list returned (unnecessary case) - ((a . b) ^(,*(tb-reverse b) ,a)) ;; car/cdr recursion - (a a))) ;; atom is just returned + (mknod "dev/foo" (logior #o700 s-ifchr) (makedev 8 3)) .cble -Note that in this example, the atom case is placed last, because an -argument list which consists of a symbol is a "catch all" match -that matches any object. We know that it matches an atom, because -the previous -.code (a . b) -case matches conses. In general, the order of the cases in -.code tree-case -is important: even more so than the order of cases in a -.code cond -or -.codn caseql . - -.coNP Macro @ tb +.coNP Functions @ symlink and @ link .synb -.mets (tb < macro-style-params << form *) +.mets (symlink < target << path ) +.mets (link < target << path ) .syne .desc The -.code tb -macro is similar to the -.code lambda -operator but its argument binding is based on a macro-style parameter list. -The name is an abbreviation of -.codn tree-bind . - -A -.code tb -form evaluates to a function which takes a variable number of -arguments. - -When that function is called, those arguments are taken as a list object which -is matched against -.meta macro-style-params -as if by -.metn tree-bind . -If the match is successful, then the parameters are bound to the -corresponding elements from the argument structure and each successive -.meta form -is evaluated an environment in which those bindings are visible. -The value of the last -.meta form -is the return value of the function. If there are no forms, -the function's return value is -.codn nil . +.code symlink +function creates a symbolic link called +.meta path +whose contents +are the absolute or relative path +.metn target . +.meta target +does not actually have to exist. -The following equivalence holds, where -.code args -should be understood to be a globally unique symbol: +The link function creates a hard link. The object at +.meta target +is installed +into the filesystem at +.meta path +also. -.cblk - (tb pattern body ...) <--> (lambda (. args) - (tree-bind pattern args body ...)) -.cble +If these functions succeed, they return +.codn t . +Otherwise they throw an exception +of type +.codn file-error . -.coNP Macro @ tc +.coNP Function @ readlink .synb -.mets (tc >> {( macro-style-params << form *)}*) +.mets (readlink << path ) .syne .desc -The -.code tc -macro produces an anonymous function whose behavior is closely -based on the -.code tree-case -operator. Its name is an abbreviation of -.codn tree-case . +If +.meta path +names a filesystem object which is a symbolic link, the +.code readlink +function reads the contents of that symbolic link and returns it +as a string. Otherwise, it fails by throwing an exception of type +.codn file-error . -The anonymous function takes a variable number of arguments. -Its argument list is taken to be the value macro is tested -against the multiple pattern clauses of an implicit -.codn tree-bind . -The return value of the function is that of the implied -.codn tree-bind . +.SS* Unix Signal Handling -The following equivalence holds, where -.code args -should be understood to be a globally unique symbol: +On platforms where certain advanced features of POSIX signal handling are +available at the C API level, \*(TX exposes signal-handling functionality. -.cblk - (tc clause1 clause2 ...) <--> (lambda (. args) - (tree-bind args - clause1 clause2 ...)) -.cble +A \*(TX program can install a \*(TL function (such as an anonymous. +.codn lambda , +or the function object associated with a named function) as the handler for +a signal. -.SS* User-Defined Places and Place Operators +When that signal is delivered, \*(TX will intercept it with its own safe, +internal handler, mark the signal as deferred (in a \*(TX sense) and then +dispatch the registered function at a convenient time. -\*(TL provides a number of place-modifying operators such as -.codn set , -.codn push , +Handlers currently are not permitted to interrupt the execution of most +\*(TX internal code. Immediate, asynchronous execution of handlers is +currently enabled only while \*(TX is blocked on I/O operations or sleeping. +Additionally, the +.code sig-check +function can be used to dispatch and clear deferred +signals. These handlers are then safely called if they were subroutines of +.codn sig-check , +and not asynchronous interrupts. + +.coNP Special variables @, sig-hup @, sig-int @, sig-quit @, sig-ill @, sig-trap @, sig-abrt @, sig-bus @, sig-fpe @, sig-kill @, sig-usr1 @, sig-segv @, sig-usr2 @, sig-pipe @, sig-alrm @, sig-term @, sig-chld @, sig-cont @, sig-stop @, sig-tstp @, sig-ttin @, sig-ttou @, sig-urg @, sig-xcpu @, sig-xfsz @, sig-vtalrm @, sig-prof @, sig-poll @, sig-sys @, sig-winch @, sig-iot @, sig-stkflt @, sig-io @ sig-lost and @ sig-pwr +.desc +These variables correspond to the C signal constants +.codn SIGHUP , +.code SIGINT +and so forth. +The variables +.codn sig-winch , +.codn sig-iot , +.codn sig-stk flt, +.codn sig-io , +.code sig-lost and -.codn inc . -It also provides a variety of kinds of syntactic places -which may be used with these operators. +.code sig-pwr +may not be available since a system may lack the corresponding signal +constants. See notes for the function +.codn log-authpriv . -Both of these categories are open-ended: \*(TL programs may extend -the set of place-modifying operators, as well as the vocabulary of -forms which are recognized as syntactic places. +The highest signal number is 31. -Regarding place operators, it might seem obvious that new place operators can -be developed, since they are macros, and macros can expand to uses -of existing place operators. As an example, it may seem that -.code inc -operator could be written as a macro which uses -.codn set : +.coNP Functions @ set-sig-handler and @ get-sig-handler +.synb +.mets (set-sig-handler < signal-number << handling-spec ) +.mets (get-sig-handler << signal-number ) +.syne +.desc +The +.code set-sig-handler +function is used to specify the handling for a signal, such +as the installation of a handler function. It updates the signal handling for +a signal whose number is +.meta signal-number +(usually one of the constants like +.codn sig-hup , +.code sig-int +and so forth), and returns the previous value. The +.code get-sig-handler +function returns the current value. -.cblk - (defmacro new-inc (place : (delta 1)) - ^(set ,place (+ ,place ,delta))) -.cble +The +.meta signal-number +must be an integer the range 1 to 31. -However, the above -.code new-inc -macro has a problem: the -.code place -argument form is inserted into two places in the expansion, which -leads to two evaluations. This is visibly incorrect if the place -form contains any side effects. It is also potentially inefficient. +Initially, all 31 signal handling specifications are set to the value +.codn t . -\*(TL provides a framework for writing place update macros which -evaluate their argument forms once, even if they have to access -and update the same places. +The +.meta handling-spec +parameter may be a function. If a function is specified, +then the signal is enabled and connected to that function until another +call to +.code set-sig-handler +changes the handling for that signal. + +If +.meta handling-spec +is the symbol +.codn nil , +then the function previously associated +with the signal, if any, is removed, and the signal is disabled. For a signal +to be disabled means that the signal is set to the +.code SIG_IGN +disposition (refer to the C API). + +If +.meta handling-spec +is the symbol +.codn t , +then the function previously associated +with the signal, if any, is removed, and the signal is set to its default +disposition. This means that it is set to +.code SIG_DFL +(refer to the C API). +Some signals terminate the process if they are generated while the +handling is configured to the default disposition. -The framework also supports the development of new kinds of place forms -as capsules of code which introduce the right kind of material into -the lexical environment of the body of an update macro, to enable -this special evaluation. +Note that the certain signals like +.code sig-quit +and +.code sig-kill +cannot be ignored or handled. +Please observe the signal documentation in the IEEE POSIX standard, and your +platform. -.NP* Place-Expander Functions +A signal handling function must take two arguments. It is of the form: -The central design concept in \*(TL syntactic places are -.IR "place-expander functions" . -Each compound place is defined by up to three place-expander functions, -which are associated with the place via the leftmost operator -symbol of the place form. One place-expander, the -.IR "update expander" , -is mandatory. Optionally, a place may also provide a -.I "clobber expander" -as well as a -.IR "delete expander" . -An update expander provides the expertise for evaluating a place form once -in its proper run-time context to determine its actual run-time storage -location, and to access and modify the storage location. -A clobber expander provides an optimized mechanism for uses that perform -a one-time store to a place without requiring its prior value. -If a place definition does not supply a clobber expander, then the syntactic -places framework uses the update expander to achieve the functionality. -A delete expander provides the expertise for determining the actual run-time -storage location corresponding to a place, and obliterating it, -returning its prior value. If a place does not supply a delete expander, then -the place does not support deletion. Operators which require deletion, such as -.code del -will raise an error when applied to that place. +.cblk +.mets (lambda >> ( signal << async-p ) ...) +.cble -The expanders operate independently, and it is expected that place-modifying -operators choose one of the three, and use only that expander. For example, -accessing a place with an update expander and then overwriting its value -with a clobber expander may result in incorrect code which contains multiple -evaluations of the place form. +The +.meta signal +argument is an integer indicating the signal number for which the +handler is being invoked. The +.meta asyncp-p +argument is a boolean value. +If it is +.codn t , +it indicates that the handler is being invoked +asynchronously\(emdirectly in a signal handling context. If it is +.codn nil , +then it +is a deferred call. Handlers may do more things in a deferred call, such +as terminate by throwing exceptions, and perform I/O. -The programmer who implements a new place does not write expanders directly, -but rather defines them via the -.code defplace -macro. +The return value of a handler is normally ignored. However if it invoked +asynchronously (the +.meta async-p +argument is true), then if the handler returns +a +.cod2 non- nil +value, it is understood that the handler +requesting that it be deferred. This means that the signal will be marked +as deferred, and the handler will be called again at some later +time in a deferred context, whereby +.meta async-p +is +.codn nil . +This is not guaranteed, however; +it's possible that another signal will arrive before that happens, +possibly resulting in another async call, so the handler must +be prepared to deal with an async call at any time. -The programmer who implements a new place update macro likewise does not -call the expanders directly. Usually, they are invoked via the macros -.codn with-update-expander , -.codn with-clobber-expander -and -.codn with-delete-expander . -These are sufficient for most kind of macros. -In certain complicated cases, expanders may be invoked using the wrapper -functions -.codn call-update-expander , -.codn call-clobber-expander -and -.codn call-delete-expander . -These convenience macros and functions perform certain common chores, like -macro-expanding the place in the correct environment, and choosing the -appropriate function. +If a handler is invoked synchronously, then its return value is ignored. -The expanders are described in the following sections. +In the current implementation, signals do not queue. If a signal is delivered +to the process again, while it is marked as deferred, it simply stays deferred; +there is no counter associated with a signal, only a boolean flag. -.NP* The Update Expander +.coNP Function @ sig-check .synb -.mets (lambda >> ( getter-sym < setter-sym < place-form -.mets \ \ \ \ \ \ \ \ << body-form ) ...) + (sig-check) .syne .desc -The update expander is a code-writer. It takes a -.meta body-form -argument, representing code, and returns a larger form which surrounds -this code with additional code. +The +.code sig-check +function tests whether any signals are deferred, and for each +deferred signal in turn, it executes the corresponding handler. For a signal to +be deferred means that the signal was caught by an internal handler in +\*(TX and the event was recorded by a flag. If a handler function is removed +while a signal is deferred, the deferred flag is cleared for that signal. -This larger form returned by the update expander can be regarded as having two -abstract actions, when it is substituted and evaluated in the context where -.meta place-form -occurs. The first abstract action is to evaluate -.meta place-form -exactly one time, in order to determine the actual run-time location to which -that form refers. -The second abstract action is to evaluate the caller's -.metn body-form -s, -in a lexical environment in which bindings exist for some lexical -functions or (more usually) lexical macros. These lexical macros -are explicitly referenced by the -.metn body-form ; -the update expander just provides their definition, under the names -it is given via the -.meta getter-sym -and -.meta setter-sym -arguments. +Calls to the +.code sig-check +function may be inserted into CPU-intensive code that +has no opportunity to be interrupted by signals, because it doesn't invoke any +I/O functions. -The update expander writes local functions or macros under these names: a -getter function and a setter function. Usually, update expanders write -macros rather than functions, possibly in combination with some lexical -anonymous variables which hold temporary objects. Therefore the getter -and setter are henceforth referred to as macros. +.coNP Function @ kill +.synb +.mets (kill < process-id <> [ signal ]) +.syne +.desc +The +.code kill +function is used for sending a signal to a process group or process. +It is a wrapper for the POSIX +.code kill +function. -The code being generated is with regard to some concrete instance of -.metn place-form . -This argument is the actual form which occurs in a program. For -instance, the update expander for the -.code car -place might be called with an arbitrary variant of the -.meta place-form -might look like -.codn (car (inc (third some-list))) . +If the +.meta signal +argument is omitted, it defaults to the same value as +.codn sig-term . -In the abstract semantics, upfront code wrapped around the -.meta body-form -by the update expander provides the logic to evaluate this place to -a location, which is retained in some hidden local context. +.SS* Unix Processes -The getter local macro named by -.meta getter-sym -must provide the logic for retrieving the value of this place. -The getter macro takes no arguments. +.coNP Functions @ fork and @ wait +.synb +.mets (fork) +.mets (wait >> [ pid <> [ flags ]]) +.syne +.desc The -.meta body-form -makes free use of the getter function; they may call it multiple times, -which must not trigger multiple evaluations of the original place -form. - -The setter local macro named by -.meta setter-sym -must generate the logic for storing a new value into the once-evaluated -version of -.metn place-form . -The setter function takes exactly one argument, whose -value specifies the value to be stored into the place. -It is the caller's responsibility to ensure that the -argument form which produces the value to be stored via the setter is evaluated -only once, and in the correct order. The setter does not concern itself with -this form. Multiple calls to the setter can be expected to result in multiple -evaluations of its argument. Thus, if necessary, the caller must supply the code -to evaluate the new value form to a temporary variable, and then pass the -temporary variable to the setter. This code can be embedded in -the -.meta body-form -or can be added to the code returned by a call to the update expander. +.code fork +and +.code wait +functions are interfaces to the Unix functions +.code fork +and +.codn waitpid . -The setter local macro or function must return the new value which is stored. -That is to say, when -.meta body-form -invokes this local macro or function, it may rely on it yielding the -new value which was stored, as part of achieving its own semantics. +The +.code fork +function creates a child process which is a replica of the parent. Both +processes return from the function. In the child process, the return value is +zero. In the parent, it is an integer representing the process ID of the child. +If the function fails to create a child, it returns +.code nil +rather than an integer. In this case, the +.code errno +function can be used to inquire about the cause. -The update expander does not macro-expand -.codn place-form . -It is assumed that the expander is invoked in such a way that the -place has been expanded in the correct environment. In other words, the -form matches the type of place which the expander handles. -If the expander had to macro-expand the place form, it would sometimes have -to come to the conclusion that the place form must be handled by a different -expander. No such consideration is the case: when an expander is called on -a form, that is final; it is certain that it is the correct expander, which -matches the symbol in the +The +.code wait +function, if successful, returns a cons cell consisting of a pair of integers. +The .code car -position of the form, which is not a macro in the context where it occurs. +of the cons is the process ID of the process or group which was successfully +waited on, and the +.code cdr +is the status. If +.code wait +fails, it returns +.codn nil . +The +.code errno +function can be used to inquire about the cause. -An update expander is free to assume that any place which is stored -(the setter local macro is invoked on it) is accessed at least once by -an invocation of the getter. A place update macro which relies on an update -expander, but uses only the store macro, might not work properly. -An example of an update expander which relies on this assumption is the -expander for the -.cblk -.meti (force << promise ) -.cble -place type. If -.meta promise -has not yet been forced, and only the setter is used, then -.meta promise -might remain unforced as its internal value location is updated. -A subsequent access to the place will incorrectly trigger a force, -which will overwrite the value. The expected behavior is that storing -a value in an unforced -.code force -place changes the place to forced state, preempting the evaluation of -the delayed form. Afterward, the promise exhibits the value which was -thus assigned. +The +.meta process-id +argument, if not supplied, defaults to -1, which means that +.code wait +waits for any process, rather than a specific process. Certain other +values have special meaning, as documented in the POSIX standard +for the +.code waitpid +function. -The update expander is not responsible for all issues of evaluation order. A -place update macro may consist of numerous places, as well as numerous -value-producing forms which are not places. Each of the places can provide its -registered update expander which provides code for evaluating just that place, -and a means of accessing and storing the values. The place update macro must -call the place expanders in the correct order, and generate any additional code -in the correct order, so that the macro achieves its required documented -evaluation order. +The +.meta flags +argument defaults to zero. If it is specified as nonzero, it should be +a bitwise combination (via the +.code logior +function) of the constants +.codn w-nohang , +.codn w-untraced +and +.codn w-continued . +If +.code w-nohang +is used, then +.code wait +returns a cons cell whose +.code car +specifies a process ID value of zero in the situation that at least +one of the processes designated by +.code process-id +exist and are children of the calling process, but have not changed state. +In this case, the status value in the +.code cdr +is unspecified. -.TP* "Example Update Expander Call:" +Status values may be inspected with the functions +.codn w-ifexited , +.codn w-exitstatus , +.codn w-ifsignaled , +.codn w-termsig , +.codn w-coredump , +.codn w-ifstopped , +.code w-stopsig +and +.codn w-ifcontinued . -.cblk - ;; First, capture the update expander - ;; function for (car ...) places - ;; in a variable, for clarity. +.coNP Functions @, w-ifexited @, w-exitstatus @, w-ifsignaled @, w-termsig @, w-coredump @ w-ifstopped and @ w-stopsig +.synb +.mets (w-ifexited << status ) +.mets (w-exitstatus << status ) +.mets (w-ifsignaled << status ) +.mets (w-termsig << status ) +.mets (w-coredump << status ) +.mets (w-ifstopped << status ) +.mets (w-stopsig << status ) +.mets (w-ifcontinued << status ) +.syne +.desc +These functions analyze process exit values produced by the +.code wait +function. - (defvar car-update-expander [*place-update-expander* 'car]) +They are closely based on the +POSIX macros +.codn WIFEXITED , +.code WEXITSTATUS , +and so on. - ;; Next, call it for the place (car [a 0]). - ;; The body form specifies logic for - ;; incrementing the place by one and - ;; returning the new value. +The +.meta status +value is either an integer, or a cons cell. In this case, the cons +cell is expected to have an integer in its +.code cdr +which is used as the status. - (call car-update-expander 'getit 'setit '(car [a 0]) - '(setit (+ (getit) 1))) +The +.codn w-ifexited , +.codn w-ifsignaled , +.codn w-coredump , +.code w-ifstopped +and +.code w-ifcontinued +functions have Lisp boolean return semantics, unlike their C language +counterparts: they return +.code t +or +.codn nil , +rather than zero or nonzero. The others return integer values. - --> ;; Resulting code: +.coNP Function @ exec +.synb +.mets (exec < file <> [ args ]) +.syne +.desc +The exec function replaces the process image with the executable specified +by string argument +.metn file . +The executable is found by searching the system path. - (rlet ((#:g0032 [a 0])) - (macrolet ((getit nil - (append (list 'car) (list '#:g0032))) - (setit (val) - (append (list 'sys:rplaca) - (list '#:g0032) (list val)))) - (setit (+ (getit) 1)))) +The +.meta file +argument becomes the first argument of the executable, argument zero. - ;; Same expander call as above, with a sys:expand to show the - ;; fully expanded version of the returned code, in which the - ;; setit and getit calls have disappeared, replaced by their - ;; macro-expansions. +If +.meta args +is specified, it is a list of strings. These are passed as the additional +arguments of the executable. - (sys:expand - (call car-update-expander 'getit 'setit '(car [a 0]) - '(setit (+ (getit) 1)))) +If +.code exec +fails, an exception of type +.code file-error +is thrown. - --> +.coNP Function @ exit* +.synb +.mets (exit* << status ) +.syne +.desc +The +.code exit* +function terminates the entire process (running \*(TX image), specifying +the termination status to the operating system. The +.meta status +argument is treated exactly like that of the +.code exit +function. Unlike that function, this one exits the process immediately, +cleaning up only low-level operating system resources such as closing file +descriptors and releasing memory mappings, without performing user-space +cleanup. - (let ((#:g0032 [a 0])) - (sys:rplaca #:g0032 (+ (car #:g0032) 1))) +.code exit* +is implemented using a call to the POSIX function +.codn _exit . -.cble -The main noteworthy points about the generated code are: -.RS -.IP - -the -.code (car [a 0]) -place is evaluated by evaluating the embedded form -.code [a 0] -and storing storing the resulting object into a hidden local variable. -That's as close a reference as we can make to the -.code car -field. -.IP - -the getter macro expands to code which simply calls the -.code car -function on the cell. -.IP - -the setter uses a system function called -.codn sys:rplaca , -which differs from -.code rplaca -in that it returns the stored value, rather than the cell. -.RE +.coNP Functions @ getpid and @ getppid +.synb + (getpid) + (getppid) +.syne +.desc +These functions retrieve the current process ID and the parent process ID +respectively. They are wrappers for the POSIX functions +.code getpid +and +.codn getppid . -.NP* The Clobber Expander +.coNP Function @ daemon .synb -.mets (lambda >> ( simple-setter-sym < place-form -.mets \ \ \ \ \ \ \ \ << body-form ) ...) +.mets (daemon < nochdir-p << noclose-p ) .syne .desc -The clobber expander is a code-writer similar to the update expander. -It takes a -.meta body-form -argument, and returns a larger form which surrounds this form -with additional program code. +This is a wrapper for the function +.code daemon +which originated in BSD Unix. -The returned block of code has one main abstract action. -It must arrange for the evaluation of -.meta body-form -in a lexical environment in which a lexical macro or lexical function -exists which has the name requested by the -.meta simple-setter-sym -argument. +It returns +.code t +if successful, +.code nil +otherwise, and the +.code errno +variable is set in that case. -The simple setter local macro written by the clobber expander is similar to the -local setter written by the update expander. It has exactly the -same interface, performs the same action of storing a value into -the place, and returns the new value. +.SS* Unix File Descriptors -The difference is that its logic may be considerably simplified by the -assumption that the place is being subject to exactly one store, -and no access. +.coNP Function @ open-fileno +.synb +.mets (open-fileno < file-descriptor <> [ mode-string ]) +.syne +The +.code open-fileno +function creates a \*(TX stream over a file descriptor. The +.meta file-descriptor +argument must be an integer denoting a valid file descriptor. -A place update macro which uses a clobber expander, and calls it more than -once, break the assumption; doing so may result in multiple evaluations -of the -.metn place-form . +For a description of +.metn mode-string , +see the +.code open-file +function. -.NP* The Delete Expander +.coNP Function @ fileno .synb -.mets (lambda >> ( deleter-sym < place-form -.mets \ \ \ \ \ \ \ \ << body-form ) ...) +.mets (fileno << stream ) .syne .desc -The delete expander is a code-writer similar to clobber expander. -It takes a -.meta body-form -arguments, and returns a larger form which surrounds this form -with additional program code. - -The returned block of code has one main abstract action. -It must arrange for the evaluation of -.meta body-form -in a lexical environment in which a lexical macro or lexical function -exists which has the name requested by the -.meta deleter-sym -argument. +The +.code fileno +function returns the underlying file descriptor of +.metn stream , +if it has one. Otherwise, it returns +.codn nil. -The deleter macro written by the clobber expander takes no arguments. -It may be called at most once. It returns the previous value of the -place, and arranges for its obliteration, whatever that means for -that particular kind of place. +This is equivalent to querying the stream using +.code stream-get-prop +for the +.code :fd +property. -.coNP Macro @ with-update-expander +.coNP Function @ dupfd .synb -.mets (with-update-expander >> ( getter << setter ) < place < env -.mets \ << body-form ) +.mets (dupfd < old-fileno <> [ new-fileno ]) .syne .desc The -.code with-update-expander -macro evaluates the -.meta body-form -argument, whose result is expected to be a Lisp form. -The macro adds additional code around this code, and the result is returned. -This additional code is called the -.IR "place-access code" . +.code dupfd +function provides an interface to the POSIX functions +.code dup +or +.codn dup2 , +when called with one or two arguments, respectively. +.coNP Function @ pipe +.synb + (pipe) +.syne +.desc The -.meta getter -and -.meta setter -arguments must be symbols. Over the evaluation of the -.metn body-form , -these symbols are bound to the names of local functions which -are provided in the place-access code. +.code pipe +function, if successful, returns a pair of integer file descriptors +as a cons cell pair. The descriptor in the +.code car +field of the pair is the read end of the pipe. +The +.code cdr +holds the write end. + +If the function fails, it throws an exception of type +.codn file-error . +.coNP Function @ poll +.synb +.mets (poll < poll-list <> [ timeout ]) +.syne +.desc The -.meta place -argument is a form which evaluates to a syntactic place. The generated -place-access code is based on this place. +.code poll +function suspends execution while monitoring one or more file descriptors +for specified events. It is a wrapper for the same-named POSIX function. The -.meta env -argument is a form which evaluates to a macro-expansion-time environment. +.meta poll-list +argument is a list of +.code cons +pairs. The +.code car +of each pair is either an integer file descriptor, or else a stream +object which has a file descriptor (the +.code fileno +function can be applied to that stream to retrieve a descriptor). The -.code with-update-expander -macro uses this environment to perform macro-expansion on the value of the -.meta place -form, to obtain the correct update expander function for the fully -macro-expanded place. +.code cdr +of each pair is an integer bit mask specifying the events, whose +occurrence the file descriptor is to be monitored for. The variables +.codn poll-in , +.codn poll-out , +.code poll-err +and several others are available which hold bitmask values corresponding +to the constants +.codn POLLIN , +.codn POLLOUT , +.code POLLERR +used with the C language +.code poll +function. -The place-access code is generated by calling the update expander -for the expanded version of -.codn place . +The +.meta timeout +argument, if absent, defaults to the value -1, which specifies an indefinite +wait. A nonnegative value specifies a wait with a timeout, measured in +milliseconds. -.TP* "Example:" +The function returns a list of pairs representing the descriptors or streams +which were successfully polled. If the function times out, it returns an +empty list. If an error occurs, an exception is thrown. -The following is an implementation of the -.code swap -macro, which exchanges the contents of two places. +The returned list is similar in structure to the input list. However, it holds +only entries which polled positive. The +.code cdr +of every pair now holds a bitmask of the events which were to have occurred. -Two places are involved, and, correspondingly, the -.code with-update-expander -macro is used twice, to add two instances of place-update code -to the macro's body. +.SS* Unix Itimers +Itimers ("interval timers") can be used in combination with signal handling to +execute asynchronous actions. Itimers deliver delayed, one-time signals, +and also periodically recurring signals. For more information, consult the +POSIX specification. -.cblk - (defmacro swap (place-0 place-1 :env env) - (with-gensyms (tmp) - (with-update-expander (getter-0 setter-0) place-0 env - (with-update-expander (getter-1 setter-1) place-1 env - ^(let ((,tmp (,getter-0))) - (,setter-0 (,getter-1)) - (,setter-1 ,tmp)))))) -.cble +.coNP Variables @, itimer-real @, itimer-virtual and @ itimer-prof +.desc +These variables correspond to the POSIX constants +.codn ITIMER_REAL , +.code ITIMER_VIRTUAL +and +.codn ITIMER_PROF . +Their values are suitable as the +.meta timer +argument of the +.code getitimer +and +.code setitimer +functions. -The basic logic for swapping two places is contained in the code template: +.coNP Functions @ getitimer and @ setitimer +.synb +.mets (getitimer << timer ) +.mets (setitimer < timer < interval << value ) +.syne +.desc +The +.code getitimer +function returns the current value of the specified timer, +which must be +.codn itimer-real , +.code itimer-virtual +or +.codn itimer-prof . -.cblk - ^(let ((,tmp (,getter-0))) - (,setter-0 (,getter-1)) - (,setter-1 ,tmp)) -.cble +The current value consists of a list of two integer values, which +represents microseconds. The first value is the timer interval, +and the second value is the timer's current value. -The temporary variable named by the -.code gensym -symbol -.code tmp -is initialized by calling the getter function for -.metn place-0 . -Then the setter function of -.meta place-0 -is called in order to store the value of -.meta place-1 -into -.metn place-0 . -Finally, the setter for -.meta place-1 -is invoked to store the previously saved temporary value into -that place. +Like +.codn getitimer , +the +.code setitimer +function also retrieves the specified timer. +In addition, it stores a new value in the timer, +which is given by the two arguments, expressed in microseconds. -The name for the temporary variable is provided by the -.code with-gensyms -macro, but establishing the variable is the caller's responsibility; -this is seen as an explicit -.code let -binding in the code template. +.SS* Unix Syslog -The names of the getter and setter functions are similarly provided -by the -.code with-update-expander -macros. However, binding those functions is the responsibility of that -macro. To achieve this, it adds the place-access code to the code generated by -the -.code ^(let ...) -backquote template. In the following example macro-expansion, the additional -code added around the template is seen. It takes the form of two -.code macrolet -binding blocks, each added by an invocation of -.codn with-update-expander : +On platforms where a Unix-like syslog API is available, \*(TX exports this +interface. \*(TX programs can configure logging via the +.code openlog +function, +control the logging mask via +.code setlogmask +and generate logs via +.codn syslog , +or using special syslog streams. -.cblk - (macroexpand '(swap a b)) +.coNP Special variables @, log-pid @, log-cons @, log-ndelay @, log-odelay @ log-nowait and @ log-perror +.desc +These variables take on the values of the corresponding C preprocessor +constants from the +.code +header: +.codn LOG_PID , +.codn LOG_CON S, etc. +These integer values represent logging options used in the option argument to +the +.code openlog +function. - --> +Note: +.code LOG_PERROR +is not in POSIX, and so +.code log-perror +might not be available. +See notes about +.code LOG_AUTHPRIV +in the documentation for +.codn log-authpriv . - (macrolet ((#:g0036 () 'a) ;; getter macro for a - (#:g0037 (val-expr) ;; setter macro for a - (append (list 'sys:setq) (list 'a) - (list val-expr)))) - (macrolet ((#:g0038 () 'b) ;; getter macro for b - (#:g0039 (val-expr) ;; setter macro for b - (append (list 'sys:setq) (list 'b) - (list val-expr)))) - (let ((#:g0035 (#:g0036))) ;; temp <- a - (#:g0037 (#:g0038)) ;; a <- b - (#:g0039 #:g0035)))) ;; b <- temp -.cble +.coNP Special variables @, log-user @, log-daemon @ log-auth and @ log-authpriv +.desc +These variables take on the values of the corresponding C preprocessor +constants from the +.code +header: +.codn LOG_USER , +.codn LOG_DAEMON , +.code LOG_AUTH +and +.codn LOG_AUTHPRIV . +These are the integer facility codes specified in the +.code openlog +function. -In this expansion, for example -.code #:g0036 -is the generated symbol which forms the value of the -.code getter-0 -variable in the -.code swap -macro. The getter is a macro which simply expands to a -.codn a : -straightforward access to the variable a. -Of course, -.code #:g0035 -is nothing but the value of the -.code tmp -variable. Thus the swap macro's -.cblk -^(let ((,tmp (,getter-0))) ...) -.cble -has turned into -.cblk -^(let ((#:g0035 (#:g0036))) ...) -.cble +Note: +.code LOG_AUTHPRIV +is not in POSIX, and so +.code log-authpriv +might not be available. +For portability use code like +.code (or (symbol-value 'log-authpriv) 0) +to evaluate to 0 if +.code log-authpriv +doesn't exist, or else check for its existence +using +.codn (boundp 'log-authpriv) . -A full expansion, with the -.code macrolet -local macros expanded out: +.coNP Special variables @, log-emerg @, log-alert @, log-crit @, log-err @, log-warning @, log-notice @ log-info and @ log-debug -.cblk - (sys:expand '(swap a b)) +These variables take on the values of the corresponding C preprocessor +constants from the +.code +header: +.codn LOG_EMERG , +.codn LOG_ALERT , +etc. +These are the integer priority codes specified in the +.code syslog +function. - --> +.coNP The @ *stdlog* special variable +.desc +The +.code *stdlog* +variable holds a special kind of stream: a syslog stream. Each +newline-terminated line of text sent to this stream becomes a log message. - (let ((#:g0035 a)) - (sys:setq a b) - (sys:setq b #:g0035)) -.cble +The stream internally maintains a priority value that is applied +when it generates messages. By default, this value is that of +.codn log-info . +The stream holds the priority as the value of the +.code :prio +stream property, which may be changed with the +.code stream-set-prop +function. -In other words, the original syntax -.cblk -(,getter-0) -.cble -became -.cblk -(#:g0036) -.cble -and finally just -.codn a . +The latest priority value which has been configured on the stream is used +at the time the newline character is processed and the log message +is generated, not necessarily the value which was in effect at the time the +accumulation of a line began to take place. -Similarly, -.cblk -(,setter-0 (,getter-1)) -.cble -became the -.code macrolet -invocations -.cblk -(#:g0037 (#:g0038)) -.cble -which finally turned into: -.codn "(sys:setq a b)" . +Messages sent to +.code *stdlog* +are delimited by newline characters. That is to say, each line of +text written to the stream is a new log. -.coNP Macro @ with-clobber-expander +.coNP Function @ openlog .synb -.mets (with-clobber-expander <> ( simple-setter ) < place < env -.mets \ << body-form ) +.mets (openlog < id-string >> [ options <> [ facility ]]) .syne .desc The -.code with-clobber-expander -macro evaluates -.metn body-form , -whose result is expected to be a Lisp form. The macro adds additional code -around this form, and the result is returned. This additional code is called -the -.IR "place-access code" . +.code openlog +function is a wrapper for the +.code openlog +C function, and the +arguments have the same semantics. It is not necessary to use +.code openlog +in order +to call the +.code syslog +function or to write data to +.codn *stdlog* . +The call is necessary in order to override the default identifying string, to +set options, such as having the PID (process ID) recorded in log messages, and +to specify the facility. The -.meta simple-setter -argument must be a symbol. Over the evaluation of the -.metn body-form , -this symbol is bound to the name of a functions which -are provided in the place-access code. +.meta id-string +argument is mandatory. The -.meta place -argument is a form which evaluates to a syntactic place. The generated -place-access code is based on this place. +.meta option +argument is a bitwise mask (see the logior function) of option +values such as +.code log-pid +and +.codn log-cons . +If it is missing, then a value of 0 is +used, specifying the absence of any options. The -.meta env -argument is a form which evaluates to a macro-expansion-time environment. -The -.code with-clobber-expander -macro uses this environment to perform macro-expansion on the value of the -.meta place -form, to obtain the correct update expander function for the fully -macro-expanded place. - -The place-access code is generated by calling the update expander -for the expanded version of -.codn place . - -.TP* "Example:" +.meta facility +argument is one of the values +.codn log-user , +.code log-daemon +or +.codn log-auth . +If it is missing, then +.code log-user +is assumed. -The following implements a simple assignment statement, similar to -.code set -except that it only handles exactly two arguments: +.coNP Function @ closelog +.synb + (closelog) +.syne +.desc +The +.code closelog +function is a wrapper for the C function +.codn closelog . -.cblk - (defmacro assign (place new-value :env env) - (with-clobber-expander (setter) place env - ^(,setter ,new-value))) -.cble +.coNP Function @ setlogmask +.synb +.mets (setlogmask << bitmask-integer ) +.syne +.desc +The +.code setlogmask +function interfaces to the corresponding C function, and has the +same argument and return value semantics. The +.meta bitmask-integer +argument is a mask of priority +values to enable. The return value is the prior value. Note that if the +argument is zero, then the function doesn't set the mask to zero; it only +returns the current value of the mask. -Note that the correct evaluation order of -.code place -and -.code new-value -is taken care of, because -.code with-clobber-expander -generates the code which performs all the necessary evaluations of -.codn place . -This evaluation occurs before the code which is generated by -.cblk -^(,setter ,new-value) -.cble -part is evaluated, and that code is what evaluates -.codn new-value . +Note that the priority values like +.code log-emerg +and +.code log-debug +are integer +enumerations, not bitmasks. These values cannot be combined directly to create +a bitmask. Rather, the +.code mask +function should be used on these values. -Suppose that a macro were desired which allows assignment to be notated in a right to left -style, as in: +.TP* Example: .cblk - (assign 42 a) ;; store 42 in variable a + ;; Enable LOG_EMERG and LOG_ALERT messages, + ;; suppressing all others + (setlogmask (mask log-emerg log-alert)) .cble -Now, the new value must be evaluated prior to the place, if left to right -evaluation order is to be maintained. The standard -.code push -macro has this property: the push value is on the left, and the place -is on the right. +.coNP Function @ syslog +.synb +.mets (syslog < priority < format << format-arg *) +.syne +.desc +This function is the interface to the +.code syslog +C function. The +.code printf +formatting capabilities of the function are not used; +the +.meta format +argument follows the conventions of the \*(TL +.code format +function instead. Note in particular that +the +.code %m +convention for interpolating the value of strerror(errno) which is +available in some versions of the +.code syslog +C function is currently not supported. -Now, the code has to explicitly take care of the order, like this: +Note that syslog messages are not newline-terminated. -.cblk - ;; WRONG! We can't just swap the parameters; - ;; place is still evaluated first, then new-value: +.SS* Unix Path Globbing - (defmacro assign (new-value place :env env) - (with-clobber-expander (setter) place env - ^(,setter ,new-value))) +On platforms where the POSIX +.code glob +function is available \*(TX provides this functionality in +the form of a like-named function, and some numeric constants. - ;; Correct: arrange for evaluation of new-value first, - ;; then place: +.coNP Special variables @, glob-err @, glob-mark @, glob-nosort @, glob-nocheck @, glob-noescape @, glob-period @, glob-altdirfunc @, glob-brace @, glob-nomagic @, glob-tilde @ glob-tilde-check and @ glob-onlydir - (defmacro assign (new-value place :env env) - (with-gensym (tmp) - ^(let ((,tmp ,new-value)) - ,(with-clobber-expander (setter) place env - ^(,setter ,tmp))))) -.cble +These variables take on the values of the corresponding C preprocessor +constants from the +.code +header: +.codn GLOB_ERR , +.codn GLOB_MARK , +.codn GLOB_NOSORT , +etc. -.coNP Macro @ with-delete-expander +These values are passed as the optional second argument of the +.code glob +function. They are bitmasks and so multiple values can be combined +using the +.code logior +function. + +Note that the +.codn glob-period , +.codn glob-altdirfunc , +.codn glob-brace , +.codn glob-nomagic , +.codn glob-tilde , +.code glob-tilde-check +and +.code glob-onlydir +variables may not be available. They are extensions in the GNU C library +implementation of +.codn glob . + +.coNP Function @ glob .synb -.mets (with-delete-expander <> ( deleter ) < place < env -.mets \ << body-form ) +.mets (glob < pattern >> [ flags <> [ error-func ]]) .syne .desc The -.code with-delete-expander -macro evaluates -.metn body-form , -whose result is expected to be a Lisp form. -The macro adds additional code -around this code, and the resulting code is returned. This additional code is -called the -.IR "place-access code" . - -The -.meta deleter -argument must be a symbol. Over the evaluation of the -.metn body-form , -this symbol is bound to the name of a functions which -are provided in the place-access code. - +.code glob +function is a interface to the Unix function of the same name. The -.meta place -argument is a form which evaluates to a syntactic place. The generated -place-access code is based on this place. +.meta pattern +argument must be a string, which holds a glob pattern: a pattern which +matches zero or more path names, similar to a regular expression. +The function tries to expand the pattern and return a list of strings +representing the matching path names in the file system. -The -.meta env -argument is a form which evaluates to a macro-expansion-time environment. -The -.code with-delete-expander -macro uses this environment to perform macro-expansion on the value of the -.meta place -form, to obtain the correct update expander function for the fully -macro-expanded place. +If there are no matches, then an empty list is returned. -The place-access code is generated by calling the update expander -for the expanded version of -.codn place . +The optional +.meta flags +argument defaults to zero. If given, it may be a bitwise combination of the +values of the variables +.codn glob-err , +.codn glob-mark , +.code glob-nosort +and others. -.TP* "Example:" +If the +.meta error-func +argument is specified, it gives a callback function which is invoked +when +.code glob +encounters errors accessing paths. The function takes two arguments: +the pathname and the +.code errno +value which occurred for that pathname. The function's return value is +boolean. If the function returns true, then +.code glob +will terminate. -The following implements the -.code del -macro: +Details of the semantics of the +.code glob +function, and the meaning of all the +.meta flags +arguments are given in the documentation for the C function. -.cblk - (defmacro del (place :env env) - (with-delete-expander (deleter) place env - ^(,deleter))) -.cble +.SS* Web Programming Support -.coNP Function @ call-update-expander +.coNP Functions @ url-encode and @ url-decode .synb -.mets (call-update-expander < getter < setter < place < env << body-form ) +.mets (url-encode < string <> [ space-plus-p ]) +.mets (url-decode < string <> [ space-plus-p ]) .syne .desc -The -.code call-update-expander -function provides an alternative interface for making use of an update -expander, complementary to -.codn with-update-expander . +These functions convert character strings to and from a form which is suitable +for embedding into the request portions of URL syntax. -Arguments -.meta getter -and -.meta setter -are symbols, provided by the caller. These are passed to the update -expander function, and are used for naming local functions in the -generated code which the update expander adds to -.metn body-form . +Encoding a string for URL use means identifying in it certain characters that +might have a special meaning in the URL syntax and representing it using +"percent encoding": the percent character, followed by the ASCII value of the +character. Spaces and control characters are also encoded, as are all byte +values greater than or equal to 127 (7F hex). The printable ASCII characters +which are percent-encoded consist of this set: + +.cblk + :/?#[]@!$&'()*+,;=% +.cble + +More generally, strings can consists of Unicode characters, but the URL +encoding consists only of printable ASCII characters. Unicode characters in the +original string are encoded by expanding into UTF-8, and applying +percent-encoding the UTF-8 bytes, which are all in the range +.codn \exx80-\exxFF . + +Decoding is the reverse process: reconstituting the UTF-8 byte sequence +specified by the URL-encoding, and then decoding the UTF-8 sequence into the +string of Unicode characters. + +There is an additional complication: whether or not to encode spaces as plus, +and to decode plus characters to spaces. In encoding, if spaces are not encoded +to the plus character, then they are encoded as +.codn %20 , +since spaces are reserved +characters that must be encoded. In decoding, if plus characters are not +decoded to spaces, then they are left alone: they become plus characters in the +decoded string. The -.meta place -argument is a place which has not been subject to macro-expansion. -The -.code call-update-expander -function takes on the responsibility for macro-expanding the place. +.code url-encode +function performs the encoding process. If the +.code space-plus-p +argument is omitted or specified as +.codn nil , +then spaces are encoded as +.codn %20 . +If the argument is a value other than +.codn nil , +then spaces are encoded as the +character +.code + +.codn (plus) . The -.meta env -parameter is the macro-expansion environment object required to -correctly expand -.code place -in its original environment. +.code url-decode +function performs the decoding process. If the +.code space-plus-p +argument is omitted or specified as +.codn nil , +then +.code + +.code (plus) +characters in the +encoded data are retained as +.code + +characters in the decoded strings. Otherwise, +plus characters are converted to spaces. +.coNP Functions @ html-encode and @ html-decode +.synb +.mets (html-encode << text-string ) +.mets (html-decode << html-string ) +.syne +.desc The -.meta body-form -argument represents the source code of a place update operation. -This code makes references to the local functions whose names -are given by -.meta getter +.code html-encode and -.metn setter . -Those arguments allow the update expander to write these functions -with the matching names expected by -.metn body-form . - -The return value is an object representing source code which incorporates -the -.metn body-form , -augmenting it with additional code which evaluates -.code place -to determine its location, and provides place accessor local functions -expected by the -.metn body-form . - -.TP* "Example:" +.code html-decode +functions convert between an HTML and raw +representation of of text. -The following shows how to implement a -.code with-update-expander -macro using -.codn call-update-expander : +The +.code html-encode +function returns a string which is based on the content of +.metn text-string , +but in which all characters which have special meaning in HTML +have been replaced by HTML codes for representing those characters literally. +The returned string is the HTML-encoded verbatim representation of +.metn text-string . -.cblk - (defmacro with-update-expander ((getter setter) - unex-place env body) - ^(with-gensyms (,getter ,setter) - (call-update-expander ,getter ,setter - ,unex-place ,env ,body))) -.cble +The +.code html-decode +function converts +.metn html-string , +which may contain HTML +character encodings, into a string which contains the actual characters +represented by those encodings. -Essentially, all that -.code with-update-expander -does is to choose the names for the local functions, and bind them -to the local variable names it is given as arguments. Then it -calls -.codn call-update-expander . +The function composition +.code (html-decode (html-encode text)) +returns a string which is equal to +.codn text . -.TP* "Example:" +The reverse composition +.code (html-encode (html-decode html)) +does not necessarily return a string equal to +.codn html . -Implement the swap macro using -.codn call-update-expander : +For instance if html is the string +.strn "

Hello, world!

" , +then +.code html-decode +produces +.strn "

Hello, world!

" . +From this, +.code html-encode +produces +.strn "<p>Hello, world!</p>" . -.cblk - (defmacro swap (place-0 place-1 :env env) - (with-gensyms (tmp getter-0 setter-0 getter-1 setter-1) - (call-update-expander getter-0 setter-0 place-0 env - (call-update-expander getter-1 setter-1 place-1 env - ^(let ((,tmp (,getter-0))) - (,setter-0 (,getter-1)) - (,setter-1 ,tmp)))))) -.cble +.SS* Filter Module +The filter module provides a trie (pronounced "try") data structure, +which is suitable for representing dictionaries for efficient filtering. +Dictionaries are unordered collections of keys, which are strings, which +have associated values, which are also strings. A trie can be used to filter +text, such that keys appearing in the text are replaced by the corresponding +values. A trie supports this filtering operation by providing an efficient +prefix-based lookup method which only looks at each input character ones, and +which does not require knowledge of the length of the key in advance. -.coNP Function @ call-clobber-expander +.coNP Function @ make-trie .synb -.mets (call-clobber-expander < simple-setter < place < env << body-form ) + (make-trie) .syne .desc The -.code call-clobber-expander -function provides an alternative interface for making use of a clobber -expander, complementary to -.codn with-clobber-expander . - -Argument -.meta simple-setter -is a symbol, provided by the caller. It is passed to the clobber -expander function, and is used for naming a local function in the -generated code which the update expander adds to -.metn body-form . - -The -.meta place -argument is a place which has not been subject to macro-expansion. -The -.code call-clobber-expander -function takes on the responsibility for macro-expanding the place. +.code make-trie +function creates an empty trie. There is no special data type for +a trie; a trie is some existing type such as a hash table. +.coNP Function @ trie-add +.synb +.mets (trie-add < trie < key << value ) +.syne +.desc The -.meta env -parameter is the macro-expansion environment object required to -correctly expand -.code place -in its original environment. +.code trie-add +function adds the string +.meta key +to the trie, associating +it with +.metn value . +If +.meta key +already exists in +.metn trie , +then the value is updated with +.metn value . The -.metn body-form -argument represents the source code of a place update operation. -This code makes references to the local function whose name -is given by -.metn simple-setter . -That argument allows the update expander to write this function -with the matching name expected by -.metn body-form . +.meta trie +must not have been compressed with +.metn trie-compress . -The return value is an object representing source code which incorporates -the -.metn body-form , -augmenting it with additional code which evaluates -.code place -to determine its location, and provides the clobber local function -to the -.metn body-form . +A trie can contain keys which are prefixes of other keys. For instance +it can contain +.str dog +and +.strn dogma . +When a trie is used for matching +and substitution, the longest match is used. If the input presents +the text +.strn doggy , +then the match is +.strn dog . +If the input is +.strn dogmatic , +then +.str dogma +matches. -.coNP Function @ call-delete-expander +.coNP Function @ trie-compress .synb -.mets (call-delete-expander < deleter < place < env << body-form ) +.mets (trie-compress << trie ) .syne .desc The -.code call-delete-expander -function provides an alternative interface for making use of a delete -expander, complementary to -.codn with-delete-expander . - -Argument -.meta deleter -is a symbol, provided by the caller. It is passed to the delete -expander function, and is used for naming a local function in the -generated code which the update expander adds to -.metn body-form . +.code trie-compress +function changes the representation of +.meta trie +to a representation which occupies less space and supports faster lookups. +The new representation is returned. -The -.meta place -argument is a place which has not been subject to macro-expansion. -The -.code call-delete-expander -function takes on the responsibility for macro-expanding the place. +The compressed representation of a trie does not support the +.code trie-add +function. -The -.meta env -parameter is the macro-expansion environment object required to -correctly expand -.code place -in its original environment. +This function destructively manipulates +.metn trie , +and may return an object +that is the same object as +.codn trie , +or it may return a different object, +while at the same time still modifying the internals of +.metn trie . +Consequently, the program should not retain the input object +.codn trie , +but use the returned object in its place. +.coNP Function @ trie-lookup-begin +.synb +.mets (trie-lookup-begin << trie ) +.syne +.desc The -.meta body-form -argument represents the source code of a place delete operation. -This code makes references to the local function whose name -is given by -.metn deleter . -That argument allows the update expander to write this function -with the matching name expected by -.metn body-form . - -The return value is an object representing source code which incorporates -the -.metn body-form , -augmenting it with additional code which evaluates -.code place -to determine its location, and provides the delete local function -to the -.metn body-form . +.code trie-lookup-begin +function returns a context object for performing +an open-coded lookup traversal of a trie. The +.meta tri +argument +is expected to be a trie that was created by the +.code make-trie +function. -.coNP Macro @ define-modify-macro +.coNP Function @ trie-lookup-feed-char .synb -.mets (define-modify-macro < name < parameter-list << function-name ) +.mets (trie-lookup-feed-char < trie-context << char ) .syne .desc The -.code define-modify-macro -macro provides a simplified way to write certain kinds of place update -macros. Specifically, it provides a way to write place update macros -which modify a place by retrieving the previous value, pass it through -a function (perhaps together with some additional arguments), and then store -the resulting value back into the place and return it. - -The -.meta name -parameter specifies the name for the place update macro to be written. - -The -.meta function-name -parameter must specify a symbol: the name of the update function. - -The update macro and update function both take at least one parameter: -the place to be updated, and its value, respectively. - +.code trie-lookup-feed-char +function performs a one character step in a trie +lookup. The +.meta trie-context +argument must be a trie context returned +by +.metn trie-lookup-begin , +or by some previous call to +.codn trie-lookup-feed-char . The -.meta parameter-list -specifies the additional parameters for update function, which will also -become additional parameters of the macro. Because it is a -function parameter list, it cannot use the special destructuring features of -macro parameter lists, or the -.code :env -or -.code :whole -special parameters. It can use optional parameters. Of course, it may be empty. +.meta char +argument is the next character to match. -The -.code define-modify-macro -macro writes a macro called -.metn name . -The leftmost parameter of this macro is a place, followed by the additional arguments -specified by -.metn parameter-list . -The macro will arrange for the evaluation of the place argument to determine -the place location. It will then retrieve and save the prior value of the -place, and evaluate the remaining arguments. The prior value of the -place, and the values of the additional arguments, are all passed to -.meta function -and the resulting value is then stored back into the location previously -determined for -.metn place . +If the lookup is successful (the match through the trie can continue +with the given character) then a new trie context object is returned. +The old trie context remains valid. -.TP* "Example:" +If the lookup is unsuccessful, +.code nil +is returned. -Some standard place update macros are implementable using -.codn define-modify-macro , -such as -.codn inc . +Note: determining whether a given string is stored in a trie can be +performed looking up every character of the string successively +with +.codn trie-lookup-feed-char , +using the newly returned context +for each successive operation. If every character is found, it means +that either that exact string is found in the trie, or a prefix. +The ambiguity can be resolved by testing whether the trie has a value +at the last node using +.codn tree-value-at . +For instance, if +.str catalog +is inserted into an empty trie with value +.strn foo , +then +.str cat +will look up successfully, being a prefix of +.strn catalog ; +however, the value at +.str cat +is +.codn nil , +indicating that +.str cat +is only a prefix of one or more entries in the trie. +.coNP Function @ tree-value-at +.synb +.mets (trie-value-at << trie-context ) +.syne +.desc The -.code inc -macro reads the old value of the place, then passes it through the -.code + -(plus) function, along with an extra argument: the delta value, which -defaults to one. The -.code inc -macro could be written using -.code define-modify-macro -as follows: - -.cblk - (define-modify-macro inc (: (delta 1)) +) -.cble - -Note that the argument list -.code (: (delta 1)) -doesn't specify the place, because the place is the implicit leftmost -argument of the macro which isn't given a name. With the above definition -in place, when -.code (inc (car a)) -is invoked, then -.code (car a) -is first reduced to a location, and that location's value is retrieved and -saved. Then the -.code delta -parameter s evaluated to its value, which has defaulted to 1, since -the argument was omitted. -Then these two values are passed to the -.code + -function, and so 1 is added to the value previously retrieved from -.codn (car a) . -The resulting sum is then stored back -.code (car a) -without, of course, evaluating -.code (car a) -again. +.code trie-value-at +function returns the value stored at the node in +in the trie given by +.metn trie-context . +Nodes which have not been given +a value hold the value +.codn nil . -.coNP Macro @ defplace +.coNP Function @ filter-string-tree .synb -.mets (defplace < place-destructuring-args < body-sym -.mets \ \ \ \ \ \ \ \ \ >> ( getter-sym < setter-sym << update-body ) -.mets \ \ \ \ \ \ \ \ \ >> [( ssetter-sym << clobber-body ) -.mets \ \ \ \ \ \ \ \ \ \ >> [( deleter-sym << delete-body )]]) +.mets (filter-string-tree < filter << obj ) .syne .desc The -.code defplace -macro is used to introduce a new kind of syntactic place. -It writes the update expander, and optionally clobber and delete -expander functions, from a simpler, more compact specification, -and automatically registers the resulting functions. The compact specification -of a -.code defplace -call contains only code fragments for the expander functions. +.code filter-string-tree +a tree structure similar to +.metn obj , +in which all of the +string atoms have been filtered through +.metn filter . -The name and syntax of the place is determined by the -.meta place-destructuring-args -argument, which is macro-style parameter list whose structure -mimics that of the the place. In particular, its leftmost symbol -gives the name under which the place is registered. The -.code defplace -macro provides automatic destructuring of the syntactic place, -so that the expander code fragments can refer to the components -of a place by name. +.meta obj +argument is a string tree structure: either the symbol +.codn nil , +denoting an empty structure; a string; or a list of tree structures. If +.meta obj +is +.codn nil , +then +.code filter-string-tree +returns +.codn nil . The -.meta body-sym -parameter must be be a symbol. This symbol will capture the -.meta body-forms -parameter which is passed to the update expander, clobber -expander or delete expander. The code fragments then have -access to the the body forms via this name. +.meta filter +argument is a filter: it is either a trie, a function, or nil. +If +.meta filter +is +.codn nil , +then +.code filter-string-trie +just returns +.metn obj . -The -.metn getter-sym , -.metn setter-sym , -and -.metn update-body -parenthesized triplet specify the update expander fragment. -The -.code defplace -macro will bind -.meta getter-sym -and -.meta setter-sym -to symbols. The -.meta update-body -must then specify a template of code which evaluates the syntactic place to -determine its storage location, and provides a pair of local functions, using -these two symbols as their name. The template must also insert the -.meta body-sym -forms into the scope of these local functions, and the place determining code. +If +.meta filter +is a function, it must be a function that can be called +with one argument. The strings of the string tree are filtered by passing +each one into the function and substituting the return value into the +corresponding place in the returned structure. -The -.meta setter-sym -and -.meta clobber-body -arguments similarly specify an optional clobber expander fragment, -as a single optional argument. If specified, the -.meta clobber-body -must generate a local function named using -.meta setter-sym -wrapped around -.meta body-sym -forms. +Otherwise if +.meta filter +is a trie, then this trie is used for filtering, +the string elements similarly to a function. For each string, a new +string is returned in which occurrences of the keys in the trie are +replaced by the values in the trie. +.coNP Function @ filter-equal +.synb +.mets (filter-equal < filter-1 < filter-2 < obj-1 << obj-2 ) +.syne +.desc The -.meta deleter-sym -and -.meta deleter-body -likewise specify a delete expander fragment. If this is omitted, -then the place shall not support deletion. - -.TP* "Example:" +.code filter-equal +function tests whether two string trees are equal +under the given filters. -Implementation of the place denoting the -.code car -field of -.code cons -cells: +The precise semantics can be given by this expression: .cblk - (defplace (car cell) body +.mets (equal (filter-string-tree < filter-1 << obj-1 ) +.mets \ \ \ \ \ \ (filter-string-tree < filter-2 << obj-2 )) +.cble - ;; the update expander fragment - (getter setter - (with-gensyms (cell-sym) ;; temporary symbol for cell - ^(let ((,cell-sym ,cell)) ;; evaluate place to cell - ;; getter and setter access cell via temp var - (macrolet ((,getter () - ^(car ,',cell-sym)) - (,setter (val) - ^(sys:rplaca ,',cell-sym ,val))) +The string tree +.meta obj-1 +is filtered through +.metn filter-1 , +as if by the +.code filter-string-tree +function, and similarly, +.meta obj-2 +is +filtered through +.metn filter-2 . +The resulting structures are compared +using +.codn equal , +and the result of that is returned. - ;; insert body form from place update macro - ,body)))) +.SS* Access To TXR Pattern Language From Lisp - ;; clobber expander fragment: simpler: no need - ;; to evaluate cell to temporary variable. - (ssetter - ^(macrolet ((,ssetter (val) - ^(sys:rplaca ,',cell ,val))) - ,body)) +It is useful to be able to invoke the abilities of the \*(TX pattern Language +from \*(TL. An interface for doing this provided in the form of the +.code match-fun +function, which is used for invoking a \*(TX pattern function. - ;; deleter: delegate to pop semantics: - ;; (del (car a)) == (pop a). - (deleter - ^(macrolet ((,deleter () ^(pop ,',cell))) - ,body))) -.cble +The +.code match-fun +function has a cumbersome interface which requires the \*(TL program to +explicitly deal with the variable bindings emerging from the pattern match +in the form of an association list. -.coNP Macro @ define-place-macro +To make it the interface easier to use, \*(TX provides +the macros +.codn txr-if , +.codn txr-when +and +.codn txr-case . + +.coNP Function @ match-fun .synb -.mets (define-place-macro < name < macro-style-params -.mets \ \ << body-form *) +.mets (match-fun < name < args < input << files ) .syne .desc -In some situations, an equivalence exists between two forms, only one -of which is recognized as a place. The -.code define-place-macro -macro can be used to establish a form as a place in terms of a translation to -an equivalent form which is already a place. +The +.code match-fun +function invokes a \*(TX pattern function whose name is +given by +.metn name , +which must be a symbol. The -.code define-place-macro -has the same syntax as -.codn defmacro . -It specifies a macro transformation for a compound form which has the -.meta name -symbol in its leftmost position. -This macro expansion is applied when such a form is used as a place. -It is applied after all other expansions, and no other macro-expansions -are applied afterward. +.meta args +argument is a list of expressions. The expressions may be symbols +which will be interpreted as pattern variables, and may be bound or unbound. +If they are not symbols, then they are treated as expressions (of the +pattern language, not \*(TL) and evaluated accordingly. -.TP* "Example:" +The +.meta input +argument is a list of strings, which may be lazy. It represents the +lines of the text stream to be processed. -Implementation of -.code first -in terms of -.codn car : +The +.meta file +argument is a list of filename specifications, which follow +the same conventions as files given on the \*(TX command line. If the pattern +function uses the +.code @(next) +directive, it can process these additional files. + +The +.code match-fun +function's return value falls into three cases. If there is a +match failure, it returns +.codn nil . +Otherwise it returns a cons cell. The +.code car +field +of the cons cell holds the list of captured bindings. The +.code cdr +of the cons cell is one of two values. If the entire input was processed, the +cdr field holds the symbol +.codn t . +Otherwise it holds another cons cell whose +.code car +is the remainder of the list of lines which were not matched, and whose +.code cdr +is the line number. + +.TP* Example: .cblk - (define-place-macro first (obj) - ^(car ,obj)) + @(define foo (x y)) + @x:@y + @line + @(end) + @(do + (format t "~s\en" + (match-fun 'foo '(a b) + '("alpha:beta" "gamma" "omega") nil))) + + Output: + (((a . "alpha") (b . "beta")) ("omega") . 3) .cble -.coNP Macro @ rlet +In the above example, the pattern function +.code foo +is called with arguments +.codn (a b) . +These are unbound variables, so they correspond to parameters +.code x +and +.code y +of the function. If +.code x +and +.code y +get bound, those values propagate to +.code a +and +.codn b . +The data being matched consists of the lines +.strn alpha:beta , +.str gamma +and +.strn omega . +Inside +.codn foo, +.code x +and +.code y +bind to +.str alpha +and +.strn beta , +and then the line variable binds to +.strn gamma . +The input stream is left with +.strn omega . + +Hence, the return value consists of the bindings of +.code x +and +.code y +transferred to +.code a +and +.codn b , +and the second cons cell which gives information about the rest of the +stream: it is the part starting at +.strn omega , +which is line 3. Note that the binding for the +.code line +variable does not propagate +out of the pattern function +.codn foo ; +it is local inside it. + +.coNP Macro @ txr-if .synb -.mets (rlet >> ({( sym << init-form )}*) << body-form *) +.mets (txr-if < name <> ( argument *) < input < then-expr <> [ else-expr ]) .syne .desc -The macro -.code rlet -is similar to the -.code let -operator. It establishes bindings for one or more -.metn sym -s, -which are initialized using the values of -.metn init-form -s. +The +.code txr-if +macro invokes the \*(TX pattern matching function +.metn name +on some input given by the +.meta input +parameter, which is a list of strings, or a single string. -Note that the simplified syntax for a variable which initializes to -.code nil -by default is not supported by -.codn rlet ; -that is to say, the syntax -.meta sym -cannot be used in place of the -.meti >> ( sym << init-form ) -syntax when -.meta sym -is to be initialized to +If +.meta name +succeeds, then +.meta then-expr +is evaluated, and if it fails, +.meta else-expr +is evaluated instead. + +In the successful case, +.meta then-expr +is evaluated in a scope in which the bindings emerging from the +.meta name +function are turned into \*(TL variables. +The result of +.code txr-if +is that of +.metn then-expr . + +In the failed case, +.meta else-expr +is evaluated in a scope which does not have any new bindings. +The result of +.code txr-if +is that of +.metn else-expr . +If +.meta else-expr +is missing, the result is .codn nil . The -.code rlet -macro differs from -.code let -in that -.code rlet -assumes that those -.metn sym -s -which have constant -.metn init-form -s -(according to the -.code constantp -function) may be safely implemented as a symbol macro rather than a lexical -variable. +.meta argument +forms supply arguments to the pattern function +.metn name . +There must be as many of these arguments as the function +has parameters. -Therefore -.code rlet -is suitable in situations in which simpler code is desired from the output -of certain kinds of machine-generated code, which binds local symbols: -code with fewer temporary variables. +Any argument which is a symbol is treated, for the purposes +of calling the pattern function, as an unbound pattern variable. +The function may or may not produce a binding for that variable. +Also, every argument which is a symbol also denotes a local variable +that is established around +.meta then-expr +if the function succeeds. For any such pattern variable for which the function +produces a binding, the corresponding local variable will be initialized +with the value of that pattern variable. For any such pattern variable +which is left unbound by the function, the corresponding local variable +will be set to +.codn nil . -On the other hand, -.code rlet -is not suitable in situations when true variables are required, which -are assignable, and provide temporary storage. +Any +.meta argument +can be a form other than a symbol. In this situation, the argument is +evaluated, and will be passed to the pattern function as the value of +the binding for the corresponding argument. -.TP* "Example:" +.TP* Example: .cblk - ;; WRONG! Exchange two variables, a and b: - (rlet ((temp a)) - (set a b) - (set b temp)) + @(define date (year month day)) + @{year /\ed\ed\ed\ed/}-@{month /\ed\ed/}-@{day /\ed\ed/} + @(end) + @(do + (each ((date '("09-10-20" "2009-10-20" "July-15-2014" "foo"))) + (txr-if date (y m d) date + (put-line `match: year @y, month @m, day @d`) + (put-line `no match for @date`)))) - ;; Demonstration of constant-propagation - (let ((a 42)) - (rlet ((x 1) - (y a)) - (+ x y))) --> 43 + Output: - (sys:expand - '(let ((a 42)) - (rlet ((x 1) - (y a)) - (+ x y)))) --> (let ((a 42)) - (let ((y a)) - (+ 1 y))) + no match for 09-10-20 + match: year 2009, month 10, day 20 + no match for July-15-2014 + no match for foo .cble -The last example shows that the -.code x -variable has disappeared in the expansion. The -.code rlet -macro turned it into into a -.code symacrolet -denoting the constant 1, which then propagated to the use site, -turning the expression -.code (+ x y) -into -.codn (+ 1 y) . - -.coNP Macro @ with-gensyms +.coNP Macro @ txr-when .synb -.mets (with-gensyms <> ( sym *) << body-form *) +.mets (txr-when < name <> ( argument *) < input << form *) .syne .desc The -.code with-gensyms -evaluates the -.metn body-form -s -in an environment in which each variable name symbol -.meta sym -is bound to a new uninterned symbol ("gensym"). - -.TP* "Example:" - -The code: +.code txr-when +macro is based on +.codn txr-if . +It is equivalent to +.code .cblk - (let ((x (gensym)) - (y (gensym)) - (z (gensym))) - ^(,x ,y ,z)) +.meti \ \ (txr-if < name <> ( argument *) < input (progn << form *)) .cble -may be expressed more conveniently using the -.code with-gensyms -shorthand: +If the pattern function +.meta name +produces a match, then each +.meta form +is evaluated in the scope of the variables established by the +.meta argument +expressions. The result of the +.code txr-when +form is that of the last +.metn form . -.cblk - (with-gensyms (x y z) - ^(,x ,y ,z)) -.cble +If the pattern function fails then the forms are not evaluated, +and the result value is +.codn nil . + +.coNP Macro @ txr-case +.synb +.mets (txr-case < input-form +.mets \ \ >> {( name <> ( argument *) << form *)}* +.mets \ \ >> [( t << form *)]) +.syne +.desc +The +.code txr-case +macro evaluates +.meta input-form +and then uses the value as an input to zero or more test clauses. +Each test clause invokes the pattern function named by that clause's +.meta name +argument. + +If the function succeeds, then each +.meta form +is evaluated, and the value of the last +.meta form +is taken to be the result value of +.codn txr-case , +which terminates. If there are no forms, then +.code txr-case +terminates with a +.code nil +result. + +The forms are evaluated in an environment in which variables are bound +based on the +.meta argument +forms, with values depending on the result of the +invocation of the +.meta name +pattern function, in the same manner as documented in detail for the +.code txr-if +macro. + +If the function fails, then the forms are not evaluated, and control passes to +the next clause. + +A clause which begins with the symbol +.code t +executes unconditionally and causes +.code txr-case +to terminate. If it has no forms, then +.code txr-case +yields +.codn nil , +otherwise the forms are evaluated in order and the value of the last +one specifies the result of +.codn txr-case . .SS* Debugging Functions -.coNP Functions source-loc and source-loc-str +.coNP Functions @ source-loc and @ source-loc-str .synb .mets (source-loc << form ) .mets (source-loc-str << form ) @@ -30939,76 +31158,6 @@ Parser error messages are directed to the .code *stderr* stream. -.SS* Scoped Resource Management -.coNP Macro @ with-resources -.synb -.mets (with-resources >> ({ sym >> [ init-form <> [ cleanup-form ])}*) -.mets \ \ << body-form *) -.syne -.desc -The -.code with-resources -macro provides a sequential binding construct similar to -.codn let* . -Every -.meta sym -is established as a variable which is visible to the -.metn init-form -s -of subsequent variables, to all subsequent -.metn cleanup-form -s -including that of the same variable, -and to the -.metn body-form -s. - -If no -.meta init-form -is supplied, then -.meta sym -is bound to the value -.codn nil . - -If an -.meta init-form -is supplied, but no -.metn cleanup-form , -then -.meta sym -is bound to the value of the -.metn init-form . - -If a -.meta cleanup-form -is supplied in addition to -.metn init-form , -it specifies code to be executed upon the termination of the -entire -.code with-resources -construct. - -When an instance of -.code with-resources -terminates, all of the -.metn cleanup-form -s -specified in its binding clauses are evaluated, in reverse (right-to-left) -order. The value of the last -.meta body-form -is returned, or else -.code nil -if no -.metn body-form -s -are present. - -.TP* "Example:" - -The following opens a text file and reads a line from it, returning that line, -while ensuring that the stream is closed immediately: - -.cblk -(with-resources ((f (open-file "/etc/motd") (close-stream f))) - (whilet ((l (get-line f))) - (put-line l))) -.cble - .SS* Debugger \*(TX has a simple, crude, built-in debugger. The debugger is invoked by adding the -- cgit v1.2.3