summaryrefslogtreecommitdiffstats
path: root/parser.l
Commit message (Collapse)AuthorAgeFilesLines
* Bumping copyrights to 2014 and expressing them as year ranges.Kaz Kylheku2013-12-101-1/+1
| | | | Fixing some errors in copyright comments.
* * parser.l (end_of_regex): Fix start condition stack underflowKaz Kylheku2013-12-061-3/+5
| | | | during regex_parse.
* * parser.l (regex_parse): Match the simplified grammar nowKaz Kylheku2013-12-061-3/+5
| | | | | expected in the parser, with no slashes around the regex. Change prefix for diagnostic message.
* * eval.c (eval_init): Registered regex_parse as newKaz Kylheku2013-12-051-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | intrinsic function and std_null as new variable. * parser.h (yylex_destroy): Existing function declared. * parser.l (regex_parse): New function. New lexical syntax added which returns SECRET_ESCAPE_R. * parser.y (SECRET_ESCAPE_R): New token. (spec): Added syntactic variant which lets us smuggle a regex into the parser easily. * stream.c:x (std_null): New global variable. (null_stream_print): New static function. (null_ops): New static structure. (make_null_stream): New function. (stream_init): Protect and initialize std_null. * stream.h (std_null, make_null_stream): Declared. * txr.1: New features documented: regex-parse, *stdnull*. * txr.c (txr_main): Call yylex_destroy after parsing the program now that I know about this function; this can free up some memory.
* * Changelog: missing entries reconstructed.Kaz Kylheku2013-11-271-1/+3
| | | | | | | | | | | | | * Makefile (conftest.clean): Use @ to suppress output. * configure (have_unistd): New variable. Set true by every successful test that compiles something that contains #include <unistd.h>. HAVE_UNISTD_H is conditionally generated in config.h based on this variable. Minor cleanup. * parser.l: Inclusion of <unistd.h> wrapped in #if/#endif. * stream.c: Conditional inclusion of <unistd.h> based on new HAVE_UNISTD_H symbol.
* * configure: Added check to detect POSIX sleep function.Kaz Kylheku2013-11-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * eval.c (eval_init): Register new open_tail function as intrinsic. * match.c (complex_snarf, complex_stream): Update calls to make_stdio_stream and make_pipe_stream to take fewer arguments. (match_files): Support a stream object as a data source specification in place of a string. * parser.l (parse_reset): Update call to make_stdio_stream to take fewer arguments. * stream.c: Inclusion of <unistd.h> made properly conditional. (struct stdio_handle): pid member defined as pid_t only if we have fork functionality, otherwise defined as int. (tail_get_line, tail_get_char, tail_get_byte): New static functions. (tail_ops): New static structure. (make_stdio_stream_common): New static structure. (make_stdio_stream, make_pipe_stream): These functions lose the input and output parameters, which ended up never used. Reimplemented in terms of new common function. (make_tail_stream): New function. (make_pipevp_stream): Reimplemented in terms of new common function. (open_file, open_command): Simplified by removal of useless local variables and their computation, which used to be extra arguments to make_stdio_stream and make_pipe_stream. (open_tail): New function. (stream_init): Calls to make_stdio_stream updated. * stream.h (make_stdio_stream, make_pipe_stream): Declarations updated. (make_tail_stream, open_tail): Declared. * txr.c (txr_main): Calls to make_stdio_stream updated.
* * parser.l: Diagnose unrecognized escape sequences in stringKaz Kylheku2013-07-131-1/+9
| | | | | | literals and quasiliterals and after @\. Support "\ " (backslash space) escape in string literals and quasiliterals.
* * parser.l: Implemented hexadecimal integer constants.Kaz Kylheku2012-09-161-2/+25
| | | | | | | These will be very useful since bit operations are about to be implemented. * txr.1: Documented.
* First cut at implementing \s, \d, \w, \S, \D and \W regex tokens.Kaz Kylheku2012-04-191-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * lib.c (init): Call regex_init. * parser.l: return new REGTOKEN kind. * parser.y (REGTOKEN): New token type. (REGTERM): Translate REGTERM to keyword. (regclass): Restructured to handle inherited nodes as lists. (regclassterm): Produce $$ as list. Add handling for REGTOKEN occurring inside character class by expanding it. This might not be the best approach. (yybadtoken): Handle REGTOKEN in switch. * regex.c (struct any_char_set, struct small_char_set, struct displaced_char_set, struct large_char_set, struct xlarge_char_set): New bitfield member, stat. (char_set_create): New parameter for indicating static char set. (char_set_destroy): Do not free a static char set. (char_set_compile): Pass zero to new parameter of char_set_create. (spaces): New static array. (space_cs, digit_cs, word_cs, cspace_cs, cdigit_cs, cword_cs): New static pointers to char_set_t. (init_special_char_sets, nfa_compile_given_set): New static function. (nfa_compile_regex, dv_compile_regex): Handle new character set token keywords. (space_k, digit_k, word_char_k, cspace_k, cdigit_k, cword_char_k, regex_space_chars): New variables. (regex_init): New function. * regex.h (space_k, digit_k, word_char_k, cspace_k, cdigit_k, cword_char_k, regex_space_chars, regex_init): Declared.
* * parser.l: Bugfix: was not allowing e-notation floatsKaz Kylheku2012-03-221-1/+1
| | | | | | | | | | | | | | | | with no decimal point like 1E1. * stream.c: (vformat): Keep track of whether or not precision was given in precision_p local variable. When printing #<bad-float> pass a precision of 0 to vformat_str, not precision, since precision does not apply. In ~f and ~e, if the precision was not given, default it to 3. Restructured float printing in ~a and ~s. It now just uses sprintf's %g with a precision. If user does not specify precision, it defaults to DBL_DIG to print the number with reasonable accuracy. A .0 is added if it sprintf produces an integer, and the conversion is ~s rather than ~a.
* * arith.c (neg): Floating-point support.Kaz Kylheku2012-03-211-2/+12
| | | | | | | | | | | | | * parser.l: FLO and FLODOT cases had to be reordered because the lex trailing context counts as part of the match length, causing 3.0 to be matched as three characters with 0 as the trailing context. The cases are split up to eliminate a flex warning. * stream.c (vformat): Support bignum in floating point conversion. Bugfixes: floating point conversion was accessing obj->fl.n instead of using n. Changed some if/else ladders to switches.
* Regression fix: 1..3 scans incorrectly into 1. .3 tokens.Kaz Kylheku2012-03-201-3/+8
| | | | | | | | | | | * parser.l (SGN, EXP, DIG): New regex definitions. (FLO): Do not recognize numbers of the form 123. Decimal point must be followed either by exponent, or digits (which may then be followed by an exponent). (FLODOT): New token type, recognizes 123. (grammar): Recognize FLODOT as a floating point number, only if it not trailed by another dot, and recognize FLO unconditionally.
* * parser.l (FLO): Adjusted syntax of floating point numbersKaz Kylheku2012-03-201-1/+1
| | | | to allow leading or trailing decimal.
* * configure (uintptr): New variable. Indicates whether unsignedKaz Kylheku2012-03-191-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | version of intptr_t is available and should be generated in config.h as uintptr_t. * eval.c (eval_init): New intrinsic functions floatp, integerp, flo-str. * gc.c (finalize): Handle FLNUM case. Rearranged cases so that all trivially returning cases are together. (mark): Handle FLNUM case. * hash.c (hash_double): New function. (equal_hash): Handle FLNUM via hash_double. (eql_hash): Likewise. * lib.c: <math.h> is included. (float_s): New symbol variable. (code2type, equal): Handle FLNUM case in switch. (integerp): New function; does the same thing as integerp before. (numberp): Returns t for floats. (flo, floatp, flo_str): New functions. (obj_init): Initialize new float_s variable. (obj_print, obj_pprint): Handle FLNUM case in switch. Printing does not work yet; needs work in stream.c. * lib.h (enum type): New enumeration FLNUM. (struct flonum): New struct type. (union obj): New member, fl. (float_s, flo, floatp, integerp, flo_str): Declared. * parser.l (FLO): New token pattern definition. Scans to a NUMBER token. Corrected uses of yylval.num to yylval.val. * parser.y (%union): Removed num member from yystype.
* Implementing #H((...) ...) read/print syntax for hash tables.Kaz Kylheku2012-03-141-0/+5
| | | | | | | | | | | | | | | | * hash.c (print_key_val, hash_print_op): New static functions. (hash_ops): hash_print_op wired in in place of cobj_print_op. * parser.l (HASH_H): New token recognized. * parser.y (HASH_H): New terminal symbol. (hash): New nonterminal symbol. (expr): Acquires hash as a constituent. (hash_from_notation): New static function. * txr.1: Hash syntax described. * txr.vim: Updated.
* Change: @(block) requires @(end) from now on.Kaz Kylheku2012-03-131-0/+6
| | | | | | | | | | | | | | | | | | | Blocks no longer extend to the end of the surrounding scope. * match.c (v_block): Rewrite for new syntax. * parser.l (BLOCK): New token type handled. * parser.y (BLOCK): New token. (block_clause): New nonterminal grammar symbol. (clause): Collateral fix: replaced a bunch of list(X, nao) forms with cons(X, nil). Introduced block_clause as a constituent of clause. * txr.1: Revamped documentation of block, and wrote about using blocks for reducing nested skips and reducing backtracking in general.
* * parser.l (ID_END): Bugfix: ID_END was defined incorrectlyKaz Kylheku2012-03-131-1/+1
| | | | | | for the current way in which an identifier token is recognized. As a result @(collect-ing) was being interpreted as @(collect -ing). It should be the complement of NSCHR.
* Bug #35625Kaz Kylheku2012-02-261-464/+490
| | | | | | | | | | | | | | | | | | | | | | * parser.l (BSCHR, BSYM, BTOK): New lexical definitions. (BRACED): New state. (grammar): Refactored so that braced variables are now handled in the BRACED state, allowing for lexical differences between braced variables and Lisp. This allows us to have the /regex/ syntax in braces, but /regex/ is just a symbol in the Lisp. The new #/ token is recognized and returned as HASH_SLASH. All rules reformatted to a more easily maintainble convention. * parser.y (HASH_SLASH): New token. (modifiers, lisp_regex): New nonterminals. (var): Grammar changed to use modifiers nonterminal instead of exprs. (var_op): Rule moved closer to var. (expr): Produces lisp_regex rather than regex. (yybadtoken): Handle HASH_SLASH in the switch statement. Bugfix: HASH_BACKSLASH was not handled. * txr.1: Documented #/regex/ syntax.
* * parser.l (num_esc): Skip octal indicator 'o' if present.Kaz Kylheku2012-02-251-1/+3
| | | | | | | This is needed for character constants. (CHRLIT): Fix broken hex constants, being treated as octal. * txr.1: Document octal character constants.
* * parser.l (source_loc_str): Missing function re-written.Kaz Kylheku2012-02-251-2/+14
| | | | | | | | (Accidentally originally written in lex.yy.c file!) (parse_reset): If file can't be opened, try adding .txr suffix to it. * txr.1: Suffix-handling behavior documented.
* * debug.c (debug): Use new way of getting line number.Kaz Kylheku2012-02-241-1/+1
| | | | | | | | | | | | | | * eval.c (eval_error): Use source_loc_str to get source location. * match.c (debuglf, sem_err, file_err): Likewise. * parser.h (source_loc_str): Declared. * parser.l (parse_init): form_to_ln_hash must be equal based now. * parser.y (rl): Store new form of read-time source location info. * txr.1: Documented load.
* First cut at @(load) directive. Incomplete: debug location infoKaz Kylheku2012-02-241-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | needs to record file name, not only line number; absolute paths not handled, etc. * match.c (load_s): New symbol variable. (v_load): New static function. (syms_init): load_s initialized. (dir_tables_init): Load directive registered. * match.h (load_s): Declared. * parser.h (parse_reset): New function declared. * parser.l (spec_file_str): Global variable moved from txr.c. (parse_reset): New function. * parser.y (clause): Special handling for @(load ...) directive. parent file path inserted into the syntax at parse time, so when the load directive executes, it can load the file from the same directory as the parent file. * txr.c (spec_file_str): Global variable moved to parser.l.
* Introducing optional arguments.Kaz Kylheku2012-02-211-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * debug.c (help, show_bindings): put_string arguments reversed. * eval.c (bind_args): Support colon notation in interpreted function lambda lists for optional arguments. Improved error checking. (apply): Allow optional arguments to be left out. (dwim_loc): Reversed arguments to replace_str, replace_vec, replace_list. (eval_init): Numerous intrinsics now have arguments that are optional. New function rand introduced which reverses arguments relative to random. New intrinsic function hash introduced for alternative construction of hashes. * gc.c (sweep): Reversed arguments to put_char. * hash.c (weak_keys_k, weak_vals_k, equal_based_k): New keyword symbol variables. (hashv): New function. (hash_init): Intern new symbols. * hash.h (weak_keys_k, weak_vals_k, equal_based_k, hashv): Declared. * lib.c (colon_k): New keyword symbol variable. (replace_list, replace_str, replace_vec): Arguments rearranged. (tree_find): testfun becomes optional argument. (int_str): base becomes optional argument. (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, func_n1, func_n2, func_n3, func_n4, func_f0v, func_f1v, func_f2v, func_f3v, func_f4v, func_n0v, func_n1v, func_n2v, func_n3v, func_n4v, func_interp): Initialize optargs to zero. (func_n0o, func_n1o, func_n2o, func_n3o, func_n4o): New functions. (cobj_print_op): Reversed arguments to put_string. (find): testfun and keyfun become optional arguments. (replace): Parameters rearranged and arguments rearranged in calls to replace_list, replace_str and replace_vec. (obj_init): colon_k initialized. (obj_print, obj_pprint): Arguments reversed, and stream defaults to std_output. Arguments reversed in calls to put_char and put_string. (dump): Arguments reversed in call to put_char. * lib.h (struct func): sizes of minparam, fixparam bitfields adjusted. New bitfield optargs. New unnamed bitfield added so the previous ones add up to 16 bits. (colon_k): Declared. (func_n0o, func_n1o, func_n2o, func_n3o, func_n4o): Declared. (replace_list, replace_str, replace_vec, replace): Declarations updated. * match.c (debuglf, dump_shell_string, dump_byte_string, dump_var, do_output_line, extract): Reversed arguments to put_char and
* * eval.c (subst_vars): Handle lone symbols (not wrappedKaz Kylheku2012-02-131-1/+1
| | | | | | | | | | | | in sys::var form) as variables. This allows interpretation of metanums in quasiliterals, because these are replaced by variables in the op expander. * parser.l: Recognize METANUM in QSILIT state. * parser.y (quasi_item): METANUM parsed in quasiliteral. * txr.vim: Metanums highlighted properly.
* * lib.c (obj_print): Print control characters in string andKaz Kylheku2012-02-121-1/+1
| | | | | | | | | | | | | | | | | character literals as hex escapes, followed by semicolon if necessary. Don't use iswprint function since it is locale-specific and concludes that non-ASCII characters are unprintable. Changed print syntax for lazy strings. (obj_pprint): Changed print syntax for lazy strings. * parser.l: Bugfix in hex/octal character constant. num_esc(yytext) was called rather than num_esc(yytext+1). * parser.y (chrlit): Bugfix: missing case for hex and octal constants which are given by a LITCHAR token. * stream.c (vformat): Bugfix: strings were being printed as if using ~a even under ~s.
* * parser.l: Allow semicolon terminator on hex and octalKaz Kylheku2012-02-051-2/+2
| | | | | | | | | escapes in a regex. Removed o escape character from octal constants in strings and quasiliterals: no such thing is documented. Octal constants can be semicolon-terminated like hex ones. * txr.1: Documented semicolons after octal constants in string literals and after hex and octal constants in regexes.
* * eval.c (rest_s, op_s): New variables.Kaz Kylheku2012-02-031-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (do_eval_args): Allow calls specified by improper lists like (x y . z) where the z expression must evaluate to a list that turns into addition arguments to be applied. (transform_op, expand_op): New static functions. (expand): Call expand_op. (eval_init): Initialize rest_s and op_s. Use rest_s to register rest function. * lib.c (gensym): New function based on gensymv. (gensymv): Now calls gensym. * lib.h (gensym): Declared. * parser.l: Parse @ followed by digits as a new kind of token, METANUM. * parser.y (METANUM): New token. (meta_expr, exprs): Missing rlcp's added. (expr): METANUM variant introduced. (yybadtoken): Handle METANUM. * txr.1: Documented one-symbol argument list of lambda. Documented op. Closed some unbalanced parentheses. * txr.vim: Highlight op.
* * parser.l: Support hex and octal escapes in string and quasiliterals,Kaz Kylheku2012-01-271-0/+5
| | | | | | | as the documentation says. Also support an optional trailing ; delimiter in hex escapes. * txr.1: Documented.
* * eval.c (eval_init): New functions registered.Kaz Kylheku2012-01-251-0/+5
| | | | | | | | | | | | | | | | | | * lib.c (sub_list, replace_list, vectorp): New functions. (sub_vec): Allow negative indices from end of array. (replace_vec): New function. * lib.h (sub_list, replace_list, vectorp, replace_vec): Declared. * parser.l (DOTDOT): Scan .. as new token. * parser.y (DOTDOT): New token. (expr): New syntax with DOTDOT. (yybadtoken): Handle DOTDOT. * txr.vim: Added new functions. Also missing append* and dwim. * txr.1: Updated.
* * eval.c (dwim_s): New symbol variable.Kaz Kylheku2012-01-251-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (dwim_loc, op_dwim): New static functions. (op_modplace): Support assignment to dwim forms with the help of dwim_loc. (expand_place): Handle dwim places. (eval_init): Initialize dwim_s. Register dwim operator in op_table. * eval.h (dwim_s): Declared. * lib.c (chr_str, chr_str_set): Allow negative indices to index backwards from end of string. (vecref, vecref_l): Allow negative indices to index from rear of array. (obj_print, obj_pprint): Render (dwim ...) forms as [...]. * parser.l: Peoduce new METABKT token type for @[, and '[', ']' tokens. * parser.y (METABKT): New token. %type declaration for '['. (list): Support square-bracket style of list, translated into dwim form. (meta_expr): Support @[...] variant. (yybadtoken): Handle METABKT in switch. * txr.1: Documented [...] syntax and dwim operator. * txr.vim: Updated.
* * match.c (counter_k): New keyword symbol variable.Kaz Kylheku2011-12-301-4/+4
| | | | | | | | | | | | | | | | | | | | (do_output_line): Process new :counter argument of rep. (do_output): Ditto, for repeat. (syms_init): Intern new keyword symbol. * match.h (counter_k): Declared. * parser.l (REPEAT, REP): Lexical syntax changed to allow arguments. * parser.y (repeat_rep_helper): Takes extra argument, representing the repeat/rep args. This is inserted into the second position of the output list. (repeat_clause, rep_elem): Extract repeat/rep arguments and pass to repeat_rep_helper. (yybadtoken): Do not put quotes around the word "number". * txr.1: Updated.
* New functionality: mod and modlast directives in repeat and rep.Kaz Kylheku2011-12-291-0/+12
| | | | | | | | | | | | | | | | | | | | | | * eval.c (eval_init): Use new symbol variable mod_s instead of calling intern. * match.c (mod_s, modlast_s): Symbol variables defined. (do_output_line): mod and modlast directives implemented under rep. (do_output): likewise under repeat. (syms_init): Initialize new symbol variables. * match.h (mod_s, modlast_s): Declared. * parser.l (MOD, MODLAST): Parse new token types. * parser.y (MOD, MODLAST): New tokens. (repeat_parts_opt, rep_parts_opt): New syntax. (repeat_rep_helper): Handle mod and modlast syntax. * txr.1: Updated. * txr.vim: Updated.
* @# comments are becoming obsolescent. @# commentsKaz Kylheku2011-12-141-4/+7
| | | | | | | | | | | can now be used. Within nested forms, Lisp-compatible ; comments are suported. * parser.l: Support @# and ; comments. * txr.1: Documentation updated. * txr.vim: Updated.
* Bignum support, here we go!Kaz Kylheku2011-12-091-23/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bignums, based on Michael Fromberger's MPI library, are integrated into the input syntax, stream output, equality testing, the garbage collector, and hashing. The plus operation handles transitions between fixnums and bignums. Other operations are still fixnum only. * Makefile (CFLAGS): Add mpi directory to include file search. (OBJS): Include new arith.o module and all of MPI_OBJS. (MPI_OBJS, MPI_OBJS_BASE): New variables. * configure (mpi_version, have_quilt, have_patch): New variables. Script detects whether patch and quilt are available. Unpacks mpi library, applies patches. Detects 128 bit integer type. Records more information in config.h about the sizes of types. * dep.mk: Updated. * depend.txr: Make work with paths that have directory components. * eval.c (eval_init): Rename of nump to fixnump. * gc.c (finalize, mark_obj): Handle BGNUM case. * hash.c: (hash_c_str): Changed to return unsigned long instead of long. (equal_hash): Handle BGNUM case. (eql_hash): Handle bignums with equal-hash, but other objects as eq. * lib.c (num_s): Variable renamed to fixnum_s. (bignum_s): New symbol variable. (code2type): Follow rename of num_s. Handle BGNUM case. (typeof): Follow rename of num_s. (eql): Handle bignums using equal, and other types using eq. (equal): Handle BGNUM case. (chk_calloc): New function. (c_num): Wording change in error message: is not a fixnum. (nump): Renamed to fixnump. (bignump): New function. (plus): Function removed, reimplemented in arith.c. (int_str): Handle integers which are too large for wcstol using bignum conversion. Base 0 is no longer passed to wcstol but converted to 10 because the special semantics for 0 would be inconsistent for bignums. (obj_init): Follow rename of num_s. Initialize bignum_s.
* * parser.l: In the CHRLIT state, return a nonblank character as anKaz Kylheku2011-12-071-0/+5
| | | | IDENT token. This allows for character literals like #\$.
* * eval.c (op_unwind_protect): Fixed uninitialized variableKaz Kylheku2011-12-061-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | warning. (eval_init): New functions registered: typeof and vector functions, as well as length_list. * lib.c (length): Function renamed to length_list, because it is list specific. (length_vec, size_vec, vector_list): New functions. (length): New function, generic over lists, vectors and strings. * lib.h (length_list, length_vec, size_vec, vector_list): Declared. * match.c (h_var, h_fun, robust_length, v_deffilter, v_fun): Use length_list instead of length. * parser.l: Introduced # token. * parser.y (vector): New nonterminal. (expr): vector is a kind of expr. (chrlist): Bugfix: single-character syntax was not working; for instance #\x to denote the charcter x. (lit_char_helper): Use length_list instead of length. * stream.c (string_in_get_line): Bugfix: this was using the wrong length function: length was being applied to a string. The genericity of length makes that correct now, but changing to length_str anyway. * txr.1: Blank sections created for functions. Vector syntax documented.
* Removing useless hash table.Kaz Kylheku2011-11-301-3/+1
| | | | | | | | | | | | * parser.h (ln_to_forms_hash): Declaration removed. * parser.l (ln_to_forms_hash): Variable removed. (parse_init): Initialization and protection of ln_to_forms_hash removed. * parser.y (rl): Update of ln_to_forms_hash removed. * txr.1:
* Task #11436Kaz Kylheku2011-11-261-7/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lisp interpreter added. * gc.c (finalize, mark_obj): Handle ENV objects. * hash.c (struct hash): acons_new_l_fun function pointer order of arguments change. (equal_hash): Handle ENV. (make_hash, gethash_l): Use cobj_handle for type safety. Follow change in acons_new_l. (gethash, gethash_f, remhash, hash_count, hash_get_userdata, hash_set_userdata, hash_next): Use cobj_handle. (gethash_n): New function. * hash.h (gethash_n): Declared. * lib.c (env_s): New symbol variable. (code2type, equal): Handle ENV. (plusv, minusv, mul, mulv, trunc, mod, gtv, ltv, gev, lev, maxv, minv, int_str): New functions. (rehome_sym): New static function. (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, func_n1, func_n2, func_n3, func_n4): Initialize new fields of struct func. (func_f0v, func_f1v, func_f2v, func_f3v, func_f4v, func_n0v, func_n1v, func_n2v, func_n3v, func_n4v, func_interp): New functions. (apply): Function removed: sanely re-implemented in new eval.c file. (funcall, funcall1, funcall2, funcall3, funcall4): Handle variadic and interpreted functions. (acons, acons_new, acons_new_l, aconsq_new, aconsq_new_l): Reordered arguments for compatibility with Common Lisp acons. (obj_init): Special hack to prepare hash_s symbol, which is needed for type checking inside the hash table funtions invoked by make_package, at a time when the symbol is not yet interned. Initialize new env_s variable. (obj_print, obj_pprint): Handle ENV. Fix confusing rendering of of function type. (init): Call new function eval_init. * lib.h (enum type): New enumeration member ENV. (struct func): functype member changed to bitfield. New bitfied members minparam and variadic. New members in f union: f0v, f1v, f2v, f3v, f4v, n0v, n1v, n2v, n3v, n4v. (struct env): New type. (union obj): New member e of type struct env. (env_s): Variable declared. (plusv, minusv, mul, mulv, trunc, mod, gtv, ltv, gev, lev, maxv, minv,
* Task #11598.Kaz Kylheku2011-11-171-2/+2
| | | | | | | | | | | | | * match.c (resolve_k): New keyword symbol variable. (h_parallel, v_parallel): Implement :resolve keyword in @(some) directive. (syms_init): New symbol variable initialized. * parser.l: Allow (some) to have argument material. * parser.y (some_clause, elem): SOME syntax adjusted. * txr.1: Documented new :resolve keyword in @(some).
* Adding quote and unquote read syntax to list forms, resemblingKaz Kylheku2011-11-171-0/+10
| | | | | | | | | | | | | | | | Lisp. The difference is that splice is spelled ,* because @ already means something, and that there is only one quote operator. None of this does anything; it is only syntax. * lib.c (quote_s, qquote_s, unquote_s, splice_s): New variables. (obj_init): New variables initialized. * lib.h (quote_s, qquote_s, unquote_s, splice_s): Declared. * parser.l: Added recognition rules. * parser.y (SPLICE): New symbolic token. (list): Added new syntax for quote and splicing.
* Changing read syntax for character literals, because we are going toKaz Kylheku2011-11-151-15/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | need the single quote in the Lisp way for suppressing evaluation, eventually. I'm going with a Scheme-compatible syntax for character literals. It has a richer repertoire of standard character names than Common Lisp, and has a x convention for coding characters in hex. * lib.c (obj_print): Print characters in a Scheme-like way. * parser.h (end_of_char): New function declared. * parser.l (grammar): Implement rules for #\ syntax, with involving new HASH_BACKSLASH token. (end_of_regex): Enhancement: added check that end_of_regex is called in correct state, like the one in end_of_char. (end_of_char): New function. * parser.y (repeat_rep_helper, o_elems_transform, define_transform, lit_char_helper): Functions changed to static. (rl): Function moved down, past the grammar section. (HASH_BACKSLASH): New terminal symbol. (chrlit): Grammar redesigned. (char_from_name): New function. * txr.1: Character syntax documented.
* Eliminate line numbers from the abstract syntaxKaz Kylheku2011-11-121-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tree representation of the TXR query. * match.c (debuglf, sem_error, file_err, eval_form): Line number argument replaced with the form to which the situation pertains. Location information is pulled from the hash table entry associated with the form. (dest_set, dest_bind, eval_form, vars_to_bindings): Context argument renamed since it isn't a line number. (struct match_line_ctx): spec_lineno member removed. (ml_all, ml_bindings_specline): lineno parameter removed. (LOG_MISMATCH, LOG_MATCH, h_var, h_skip, h_coll, h_parallel, match_line): Pass elem to debuglf instead of line number. as context. (h_trailer, h_eol): define elem for LOG_MISMATCH and LOG_MATCH macros. (h_fun): Pass elem variable to debuglf instead of line number. Body stored as a simple cons cell once again (no line number). (do_output_line): Line number parameter removed. Pass specline to sem_error instead of line number. (do_output): Adjusted for one less parameter in do_output_line. (mf_from_ml): Pass one less parameter to ml_all. Conversion of specline to spec is just a wrapping into a nested list, with no line number. (spec_bind): Linenumber variable parameter removed from macro. Definition simplified. (v_skip): Pass specline to debuglf instead of spec_linenum, which is no longer computed. (v_trailer): Use new definition of specline. Pass first_spec to sem_error instead of spec_linenum. Computation of ff_specline no longer has to skip line number. (v_freeform, v_block, v_accept_fail, v_next, v_parallel, v_gather, v_collect, v_merge, v_bind, hv_trampoline, v_cat, v_output, v_try, v_defex, v_throw, v_deffilter, v_filter, match_funcall): Use new definition of specline. Pass first_spec to sem_error instead of spec_linenum. (v_forget_local): Specline computed differently since there is no linenumber to skip. (h_define): Back to implified representation of function with no extra cell for line number. (v_define, v_fun): Pass first_spec to sem_error instead of spec_linenum. Back to implified representation of function with no extra cell for line number. (match_files): first_spec_item computed differently. Pass first_spec to sem_error instead of spec_linenum. * parser.h (source_loc): Declared. * parser.l (source_loc): New function. * parser.y:x (grammar): Removed line numbers from abstract sytnax tree. A few more places needed the annotation of forms with location info, and a couple of cases of the need to propagate the info was
* Infrastructure for storing line number informationKaz Kylheku2011-11-121-1/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | outside of the code, in hash tables. * filter.c (make_trie, trie_add): Update to three-argument make_hash. * hash.c (struct hash): New members, hash_fun, assoc_fun acons_new_l_fun. (ll_hash): Renamed to equal_hash. (eql_hash): New static function. (cobj_hash_op): Follows ll_hash rename. (hash_grow): Use new function indirection to call hashing function. (make_hash): New argument to specify type of hashing. Initialize new members of struct hash. (gethash_l, gethash, remhash): Use function indirection for hashing and chain search and update. (pushhash): New function. * hash.h (make_hash): Declaration updated with new parameter. (pushhash): Declared. * lib.c (eql_f): New global variable. (eql, assq, aconsq_new, aconsq_new_l): New functions. (make_package): Updated to new three-argument make_hash. (obj_init): gc-protect and initialize new variable eql_f. * lib.h (eql, assq, aconsq_new, aconsq_new_l): Declared. * match.c (dir_tables_init): Updated to there-argument make_hash. * parser.h (form_to_ln_hash, ln_to_forms_hash): Global variables declared. * parser.l (form_to_ln_hash, ln_to_forms_hash): New global variables. (grammar): Set yylval.lineno for tokens that are classified to that type in parser.y. (parse_init): Initialize and gc-protect new global variables. * parser.y (rl): New static helper function. (%union): New member, lineno. (ALL, SOME, NONE, MAYBE, CASES, CHOOSE, GATHER, AND, OR, END, COLLECT, UNTIL, COLL, OUTPUT, REPEAT, REP, SINGLE, FIRST, LAST, EMPTY, DEFINE, TRY, CATCH, FINALLY, ERRTOK, '('): Reclassified as lineno type. In the grammar, these keywords can thus provide a stable line number from the lexer. (grammar): Numerous rules updated to add constructs to the line number hash tables via the rl helper. * dep.mk: Updated. * Makefile (depend): Use the installed, stable txr in the system path to update dependencies rather than locally built ./txr, to prevent the problem that txr is broken because out out-of-date dependencies, and thus cannot regenerate dependencies.
* * match.c (opt_nobindings, opt_arraydims): GlobalKaz Kylheku2011-11-101-2/+0
| | | | | | | | | | | | | | | | | variables moved from parser.l. (opt_lisp_bindings): New variable. (dump_bindings): Dump Lisp syntax bindings on standard output if opt_lisp_bindings is set. (v_cat): Do not complain about trailing material; this is not compatible with horizontal cat. * parser.l (opt_nobindings, opt_arraydims): Moved to match.c. * txr.c (txr_main): New options, --lisp-bindings and the equivalent -l. * txr.h: opt_lisp_bindings declared.
* * parser.l: Fixed wrong error message.Kaz Kylheku2011-11-101-1/+1
|
* Task #11581Kaz Kylheku2011-11-061-0/+4
| | | | | | | | | | | | | | | | | * match.c (gather_s): New keyword variable. (v_gather): New function. (syms_init): gather_s initialized. (dir_tables_init): v_gather entered into table. * match.h (gather_s): Declared. * parser.l: GATHER token scanning added. * parser.y: GATHER token added. gather_clause nonterminal added. * txr.1: New directive documented. * txr.vim: gather keyword introduced.
* Bugfix: prepared_error_message variable needs to be gc-protected.Kaz Kylheku2011-10-261-0/+5
| | | | | | | | | * parser.h (parse_init): Declared. * parser.l (parse_init): New function. * txr.c (main): Call parse_init. (txr_main): No need to gc-protect yyin_stream since parse_init does it.
* Parse error handling improvements.Kaz Kylheku2011-10-261-62/+32
| | | | | | | | | | | | | | | | | | | | | | | | * parser.l (prepared_error_message): New static variable. (yyerror): Emit and clear prepared error message. (yyerrprepf): New static function. (yybadtoken): Function moved into parser.y. (grammar): For irrecoverable lexical errors, stash error message with yyerrprepf and return the special error token ERRTOK to generate a syntax error. I could find no other interface to the parser to make it cleanly exit. * parser.y (ERRTOK): New terminal symbol, does not appear anywhere in the grammar. (spec): Bail after 8 errors, recover to nearest newline, and use yyerrok to clear error situation. (YYEOF): Provided by Bison, conditionally defined for other yacc-s. (yybadtoken): Function moved from parser.l. Checks for the next token being YYEMPTY or YYEOF, and also handles ERRTOK. * stream.c (vformat_to_string): New function. (format): If stream is nil, format to string and return it. * stream.h (vformat_to_string): Declared.
* * Makefile (%.ok: %.txr): Use unified diff for showingKaz Kylheku2011-10-131-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | differences between expected and actual test output. * parser.l (yybadtoken): Handle new terminal symbol, SPACE. New rule for producing SPACE token out of an extent of tabs and spaces. * parser.y (SPACE): New terminal symbol. (o_var): New nonterminal. I noticed that the var rule was being used for output elements, and the var rule refers to elem rather than o_elem. A new o_var rule is a simplified duplicate of var. (elem): Handle SPACE token. Transform to regex if it is a single space, otherwise to literal text. (o_elem): Handle SPACE token in output. * tests/001/query-2.txr: This query depends on matching single spaces and so needs to use escapes. * tests/001/query-4.txr, test/001/query-4.expected: New test case, based on query-2.txr. It produces the same output, but is simpler thanks to the new semantics of space. * txr.1: Documented.
* Ported to Cygwin.Kaz Kylheku2011-10-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TODO: there should be some type safety with the new wli macro so that if it is forgotten, there will be a diagnostic. * configure (lit_align): New configuration variable and configuration test. Generates LIT_ALIGN in config.h. Fixed the integer-holds-pointer test for the different output from the nm program on Cygwin. The arrays become common symbols marked C which do not show an offset attribute, only size: one less column. * filter.c (to_html_table, from_html_table): wrap wide string literals with the wli macro. This must be done from now on for all literals and initializes of arrays that are going to be directly converted to type tagged val-s. * lib.h (wli): New macro. (auto_str, static_str, litptr, lit_noex): Handle wide literals on platforms where they are aligned to only two bytes, such that we don't have two bits in the pointer. We can still add our 11 bit type tag, but then when recovering the pointer to the data, we have may have to fix up the pointer. * parser.l: Another portability issue here. Flex generates a scanner which has #include <unistd.h> in the middle, after the source file's own #includes which can introduce macros. On Cygwin, there is some hygiene problem whereby our "noreturn" macro causes the <unistd.h> header to generate bad syntax and fail to compile. Stupid Cygwin and even stupider flex! The workaround is to include <unistd.h> at the top in the flex source. * stream.c (string_out_put_char): This is one more place where the string literal handling hack spreads. * txr.c (version): Wrap string in wli.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-10 12:37:36 +0000