summaryrefslogtreecommitdiffstats
path: root/parser.y
Commit message (Collapse)AuthorAgeFilesLines
...
* parser: more efficient treatment of string literals.Kaz Kylheku2017-08-171-27/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this patch we switch the string literal parser from right recursion to left recursion, so that it doesn't require a Yacc stack depth proportional to the number of characters in the literal. Secondly, we build the string directly in a syntax-directed way, rather than building a list of characters and then walking the list to build a string. This was discovered as a byacc regression, though the fix is not for the sake of byacc but basic efficiency. The Sep 29, 2015 commit 111650e235ab2e529fa1529b1c9a23688a11cd1f "Implementation of static slots for structures." extended the string literal in the struct.tl test case in such a way that if the parser is generated by byacc rather than GNU Bison, the test case fails with a "yacc stack overflow". I haven't done any regression testing with byacc in over two years so I didn't notice this. Quasiliterals could use this treatment also. Word list literals benefit from this change, but they still use a Yacc stack depth proportional to the number of words, since the accumulation of words is right recursive. * parser.y (lit_char_helper): Static function removed. (restlitchar): New grammar nonterminal symbol. (strlit, quasi_item, wordslit): No need to call lit_char_helper. (litchars): A litchars is now either a single LITCHAR, or else a LITCHARS followed by a sequence of more. This sequence is a separate production rule called restlitchar, which is purely left recursive. (If litchars is made directly left recursive without this helper rule, intractable reduce/reduce and shift/reduce conflicts arise.)
* Bugfix: (sys:expr . atom) bad syntax out of parser.Kaz Kylheku2017-08-021-1/+1
| | | | | | | | | | | | | * parser.y (expand_meta): Fix incorrect conversion of (sys:var x) when x is a non-bindable term to (sys:expr . x). Should be (sys:expr x). This doesn't have that much of an impact, I don't think. It prevent certain degenerate forms from working like @(bind x @"str"). The bad thing is that this particular one has a silent problem: @"str" wrongly evaluates to #\s. Neverheless, this doesn't seem worth the addition of a compat flag test; the odds of someone depending on @"str" producing #\s in some pattern language code see vanishingly low.
* Continuing implementation of buffers.Kaz Kylheku2017-04-211-2/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | * Makefile (OBJS): New objects itypes.o and buf.o. * buf.c, buf.h: New files. * itypes.c, itypes.h: New files. * lib.c (obj_print_impl): Handle BUF via buf_print and buf_pprint. (init): Call itypes_init and buf_init. * parser.h (end_of_buflit): Declared. * parser.l (BUFLIT): New exclusive state. (grammar): New rules for recognizing start of buffer literal and its interior. (end_of_buflit): New function. * parser.y (HASH_B_QUOTE): New token. (buflit, buflit_items, buflit_item): New nonterminals and corresponding grammar rules. (i_expr, n_expr): These symbols now generate a buflit; a buffer literal is a kind of expression. (yybadtoken): Handle HASH_B_QUOTE case.
* parser: add some error cases to hash notations.Kaz Kylheku2017-04-081-1/+9
| | | | | | | | Produce better diagnostics for expressions like #[... or #Habc. * parser.y (vector, hash, struct, range): Add error productions.
* parser: refactor grammar to banish #[] etc.Kaz Kylheku2017-04-071-16/+26
| | | | | | | | | | | | | | | | | | | | Turns out we have an over-eager parser whcih recognizes invalid notion such as #[...], #S[...] and others. This is because the grammar nonterminal list is overloaded with phrase structures. The syntax of a vector literal, for instance, is '#' list, but a list can be '[' ... and other expressions. * parser.y (dwim, meta, compound): New non-terminal symbols. Dwim derives the square bracketed "dwim" expressons that were previously overloaded into list. Meta derives the @ exprs. compound is what list used to be. (list): Handle only (...) list expressions. (o_elem, modifiers): Derives compound rather than list, thus preserving existing behavior. (i_expr, n_expr): Likewise. All other uses references to the list nonterminal stay, thereby trimming the grammar of dubious expressions.
* parser: fix a...b syntax error.Kaz Kylheku2017-04-021-0/+6
| | | | | | | | | | | | | This issue has implications mainly for read/print consistency. The (rcons a .b) expression prints a...b, but that doesn't read back. The reason is that the . on .b isn't preceded by whitespace, and so isn't the UREFDOT token recognized in a n_expr. It's just the '.' token which is a syntax error in that situation. * parser.y (n_expr): New special case rule to handle the phrase pattern n_expr DOTDOT '.' n_expr which is now a syntax error.
* parser: support uref dot as top-level expr.Kaz Kylheku2017-03-151-0/+8
| | | | | * parser.y (hash_semi_oor_n_expr, hash_semi_or_i_expr): add grammar rules for leading dot.
* parser: factor repeated uref-related code.Kaz Kylheku2017-03-151-32/+20
| | | | | | | * parser.y (uref_helper): New static function. (list, i_dot_expr, n_expr, n_dot_expr): Replace most action code releated to unbound ref dot syntax with call to uref_helper.
* Add in-package directive.Kaz Kylheku2017-03-131-0/+4
| | | | | | | | | | | | * match.c (in_package_s): New symbol variable. (syms_init): Initialize in_package_s. * match.h (in_package_s): Declared. * parser.y (check_parse_time_action): Add case for in-package. Evaluate just with eval, as a case of the in-package macro. * txr.1: Documented.
* New directive: mdo.Kaz Kylheku2017-03-121-5/+11
| | | | | | | | | | | | | | | * eval.h (progn_s): Declarationa added. * match.c (mdo_s): New symbol variable. (syms_init): Initialize mdo_s. * match.h (mdo_s): Declared. * parser.y (check_for_include): Renamed to check_parse_time_action and implements mdo, not only include. (clauses_rev): Follow rename of function. * txr.1: Documented.
* uref: the a.b.c syntax extended to .a.b.cKaz Kylheku2017-03-061-13/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now it is possible to use a leading dot on the referencing dot syntax. This is the is the "unbound reference dot". It expands to the uref macro, which denotes an unbound-reference: it produces a function which takes an object as the argument, and curries the reference implied by the remaining arguments. * eval.c (uref_s): New global symbol variable. (eval_init): Intern uref symbol and init uref_s. * eval.h (uref_s): Declared. * lib.c (simple_qref_args_p): A qref expression is now also not simple if it contains an embedded uref, meaning that it cannot be rendered into the dot notation without ambiguity. (obj_print_impl): Support printing (uref a b c) as .a.b.c. * lisplib.c (struct_set_entries): Add uref to the list of autoload triggers for struct.tl. * parser.l (DOTDOT): Consume any leading whitespace as part of recognizing the DOTDOT token. Otherwise the new rule for UREFDOT, which matches (mandatory) leading space will take precedence, causing " .." to be scanned wrong. (UREFDOT): Rule for new kind of dot token, which is preceded by mandatory whitespace, and isn't consing dot (which has mandatory trailing whitespace too, matched by an earlier rule). * parser.y (UREFDOT): New token type. (i_dot_expr, n_dot_expr): New grammar rules. (list): Handle a leading dot on the first element of a list as a special case. Things are done this way because trying to work a UREFDOT into the grammar otherwise causes intractable conflicts. (i_expr): The ^, ' and , punctuators are now followed by an i_dot_expr, so that the expression can be an unbound dot. (n_expr): Same change as in i_expr, but using n_dot_expr. Plus new UREFDOT n_expr production. * share/txr/stdlib/struct.tl (uref): New macro. * txr.1: Documented.
* Harmonize code with previous commit.Kaz Kylheku2017-03-041-2/+3
| | | | | | | * parser.y (expand_repeat_rep_args): Use a sym local variable to avoid evaluating first(arg) twice, like the previous commit does in another case of this function.
* bugfix: :vars in output repeat not registered.Kaz Kylheku2017-03-041-3/+7
| | | | | | | | | | | | | | | | Test case: @(output) @ (repeat :vars (x (y 42)) @ (list x y) @ (end) @(end) x and y are spuriously reported as unbound variables in the (list x y) form. * parser.y (expand_repeat_rep_args): Do the missing calls to match_reg_var when processing :vars list.
* Support horizontal @(block), phase 1.Kaz Kylheku2017-02-151-0/+5
| | | | | | | | | | | | Unresolved issue: horizontal @(accept) terminating in a vertical @(block) or horizontal @(block) in a different line, or vertical @(accept) caught in horizontal context. * match.c (h_block, h_accept_fail): New functions. (dir_tables_init): Register horizontal @(block), @(accept) and @(fail). * parser.y (elem): Support BLOCK syntax.
* bugfix: :filter not handled right in output var.Kaz Kylheku2017-01-261-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This issue was fixed in quasiliterals only. Because of the implementation duplicity between output vars and quasiliteral vars, we have to fix it in two places. When the parser handles quasiliterals, it builds vars without expanding the contents. The quasiliteral expander takes care of recognzing (sys:var ...) forms and properly handles them and their attributes, avoiding expanding the argument of a :filter keyword. When the parser handles an o_var that is a braced variable, it calls expand on its contents right there, then builds the (sys:var ...) form from the expanded contents. Why don't we just call expand_quasi in the o_var rule to have a single (sys:var ...) form expanded exactly how it is done in quasiliterals. * eval.c (expand_quasi): Change static function to external. * eval.c (expand_quasi): Declared. * parser.y (o_var): Construct an unexpanded (sys:var ...) form, and then wrap it in a one-element list. This is a de-facto quasi-items list, which can be expanded by expand_quasi. Then we pull the car of the expansion to get our expanded var.
* bugfix: catch arguments not registered properly.Kaz Kylheku2017-01-231-1/+1
| | | | | | | | Symptom: variables appearing in a @(catch) are reported as unbound variables anyway. * parser.y (process_catch_exprs); The parameters are the second element of the catch form, not its rest.
* Bump copyright year to 2017.Kaz Kylheku2017-01-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | * LICENSE, LICENSE-CYG, METALICENSE, Makefile, args.c, args.h, arith.c, arith.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, struct.c, struct.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/except.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/package.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl: Add 2017 to all copyright headers and strings.
* parser bugfix: expand used instead of expand_forms.Kaz Kylheku2017-01-221-1/+1
| | | | | * parser.y (o_var): fix expand wrongly being called on a list of forms.
* Enable unbound warnings when expanding TXR code.Kaz Kylheku2017-01-221-12/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this change, Lisp expansion-time warnings are no longer suppressed during the parsing of the TXR pattern language. Embedded Lisp expressions can refer to TXR pattern variables, which generates spurious warnings that must be suppressed. Since TXR pattern variables are dynamically introduced in a very flexible way, it's hard to do an exact job of this. We take the crude approach that warnings are suppressed for all pattern variables that appear anywhere in the TXR code. To do that, we identify, at parse time, all directives which can bind new variables, and register those variables as if they were tentative global defs, purging all pending warnings for them. * match.c (binding_directive_table): New static hash table. (match_reg_var, match_reg_params, match_reg_elem): New functions. (match_reg_var_rec): New static function. (dir_tables_init): gc-protect binding_directive_table, and populate its entries. * match.h (into_k, named_k): Declared. (match_reg_var, match_reg_params, match_reg_elem): Declared. * parser.y (process_catch_exprs): New static function. (elem): Call match_reg_elem for each basic directive, to process the variables in that directive according to its operator symbol. Do this for each compound form elem and variable elem. Te horizontal @(define) eleme has its own grammar production here, and we handle its parameter list in that rule. (define_clause): Handle the parameters of a vertical @(define). It binds pattern variables, and so we must suppress unbound warnings for those. (catch_clauses_opt): Process the parameters bound by @(catch) clauses. (output_clause): Suppress warnings for the variables nominated by any :into or :named argument. (expand_repeat_rep_args): Suppress warnings for :counter variable, and for :vars variables. (parse_once): Remove the warning-muffling handler frame set up around the yyparse call. * txr.c (txr_main): Suppress warnings for TXR variables defined using -D syntax on the command line. Dump deferred warnings after parsing a .txr file.
* Improve accuracy of expansion of repeat/rep args.Kaz Kylheku2017-01-221-8/+10
| | | | | | | * parser.y (expand_repeat_rep_args): Correctly handle situation when :counter or :vars appears as an argument to another keyword. (A warning might be generated here, since this situation is wrong.)
* bugfix: expand macros in a number of directives.Kaz Kylheku2017-01-211-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | This is the last round of changes on this topic, bringing proper macro expansion to the arguments to @(skip), @(fuzz), @(next), @(call), @(cat), @(load) and @(close). * match.c (match_expand_keyword_args): Only process the keyword arguments if they are followed by an argument. Process @(next) arguments here too: :list and :string take a Lisp expression, but :tlist and :var take an argument which is not a Lisp expression and must be handled properly. Also, expand any non-keyword expression. This handles the <source> argument of @(next). (match_expand_elem): New function. * match.h (match_expand_elem): Declared. * parser.h (expand_meta): Declared. * parser.y (expand_meta): Static function becomes external. (elem): Expand elem other than require or do using match_expand_elem. We don't fold require and do into this because match_expand_elem has a backward compat switch in it that doesn't apply to these.
* bugfix: expand dest arg of @(output).Kaz Kylheku2017-01-211-1/+12
| | | | | * parser.y (expand_form_ver): New inline function. (output_clause): If exprs are present, expand first one.
* Expand lisp forms in @(mod) and @(modlast) args.Kaz Kylheku2017-01-191-4/+15
| | | | | | * parser.y (expand_forms_ver): New function. (repeat_parts_opt, rep_parts_opt): Expand the exprs_opt that follow MOD or MODLAST.
* Bugfix: expand macros in collect, coll, gather.Kaz Kylheku2017-01-191-16/+24
| | | | | | | | | | | | | | | In the argument lists of @(collect)/@(repeat), @(coll)/@(rep) and @(gather), Lisp expressions can appear as arguments to keywords or for supplying default values for variables. These are not being macro-expanded. * match.c (match_expand_vars): New static function. (match_expand_keyword_args): New function. * match.h (match_expand_keyword_args): Declared. * parser.y (gather_clause, collect_clause, elem): Use new function in match.c to expand the argument lists.
* Eliminate rejection of empty clauses.Kaz Kylheku2017-01-081-87/+27
| | | | | | * parser.y (grammar): Remove all checks which raise a syntax error if a clause is empty. These reject some correct situations, getting in the programmer's way.
* Bugfix: incorrect quasi-quoting over #R syntax.Kaz Kylheku2016-12-251-4/+0
| | | | | | | | | | | | | | | | | | | The issue is that ^#R(,(+ 2 2) ,(+ 3 3)) produces 4..6 rather than #R(4..6). 4..6 is, of course, the syntax (rcons 4 6) which evaluates to a range. Here we want the range to which it evaluates, not the syntax. * eval.c (expand_qquote_rec): Handle the case when the qquoted_form is a range atom: expand the from and to parts, and generate a rcons expression. Though this seems to be opposite to the previous paragraph, it's the right thing. * parser.y (range): Drop the unquotes_occurs case which produces rcons syntax. Produce a range object, always. This is the source of the problem: a (rcons ...) expression was produced here which was just traversed by the qquote expander as list. It's the expander that must produce the rcons expression.
* Tweak terminology in some parser error messages.Kaz Kylheku2016-12-231-3/+3
| | | | | | | | | | | | Eliminate references to a "list expression". This term specifically denotes (list ...) and not any compound expression. * parser.y (define_clause): Change "unterminated list expression" to "unterminated define directive". (output_clause): Change "unterminated list expression" to "unterminated output directive". (list): Change "list expression" to "expression".
* parser: fix problems at EOF involving #; syntax.Kaz Kylheku2016-12-061-24/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch addresses a problem whereby if a TXR Lisp file ends with an erased object notation such as #;(a b c), there is a syntax error. The strategy is to simplify the grammar so that a single yyparse primed with SECRET_ESCAPE_E or SECRET_ESCAPE_I will read either an object, or just one instance of the #; notation. If #;OBJ is read, then the parse tree is returned as the nao value. The caller knows that #;OBJ must have occurred because there are no errors and the parser isn't at EOF, yet there is no parse tree. Then in lisp_parse we can loop on this situation, and make adjustments elsewhere also. So that iread continues to work, we must separate the parser_eof condition from the lookahead token. Under iread, we were clearing the token in prime_parser_post, but that was having the side effect of making the parser look like it is in EOF. We now preserve the EOF indication in a flag, so we can manipulate the token. * parser.h (struct parser): new member, eof. * parser.c (parser_common_init): Initialize new eof flag in parser structure to zero. (prime_parser_post): Set the eof flag if the parser's most recent token is zero. (lisp_parse_impl): Call the parser repeatedly while there are no errors, and no EOF, yet no object has been produced. This indicates that a #; erasure has been processed. (read_eval_stream): Restructure the logic here for clarity. Do not break the loop if error_val was returned from the parser, but there are no errors, and the parser isn't at EOF. This is behavior is probably redundant with respect to the loop in lisp_parse_impl. (read_eval_ret_last): Bugfixes here. Pass an error indicating value down to lisp_parse, like in read_eval_stream and make the logic similar. (parser_eof): Just return an indication based no the eof flag. * parser.y (hash_semis_n_expr, hash_semis_i_expr, ignored_i_exprs, ignored_n_exprs): Grammar rules removed. (hash_semi_or_n_expr, hash_semi_or_i_expr): New grammar rules. (spec): Retarget SECRET_ESCAPE_E and SECRET_ESCAPE_I cases to new rules. (parse): Clear eof flag to zero.
* Eliminate duplicated warning-suppressing function.Kaz Kylheku2016-11-281-6/+2
| | | | | | | | | | | | | | * eval.c (warning_continue): Static function removed. (no_warn_expand): Use uw_muffle_warning instead of removed function. * parser.y (warning_continue): Static function removed. (parse_once): Use uw_muffle_warning instead of removed function. * unwind.c (uw_muffle_warning): New function. * unwind.h (uw_muffle_warning): Declared.
* Expander warns about unbound variables.Kaz Kylheku2016-11-261-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * eval.c (eval_exception): New static function. (eval_error): Reduced to wrapper around eval_exception. (eval_warn): New function. (me_op): Bind the rest symbol in a shadowing env to suppress watnings about unbound rest. (do_expand): Throw a warning when a bindable symbol is traversed that has no binding. (expand): Don't install atoms as last_form_expanded. * lib.c (warning_s, restart_s, continue_s): New symbol variables. (obj_init): Initialize new symbol variables. * lib.h (warning_s, restart_s, continue_s): Declared. * lisplib.c (except_set_entries): New entries for ignwarn and macro-time-ignwarn. * parser.c (repl_warning): New static function. (repl): Use repl_warning function as a handler for warning exceptions: to print their message and then continue by throwing a continue exception. * parser.y (warning_continue): New static function. (parse_once): Use warning_continue to ignore warnings. In other words, we suppress warnings from Lisp that is mixed into TXR pattern language code, because this produces too many false positives. * share/txr/stdlib/except.tl (ignwarn, macro-time-ignwarn): New macros. * share/txr/stdlib/place.tl (call-update-expander, call-clobber-expander, call-delete-expander): Ignore warnings around calls to sys:expand, because of some gensym-related false positives (we expand code into which we inserted some gensyms, without having inserted the constructs which bind them. * tests/011/macros-2.txr: Suppress unbound variable warnings from a test case. * tests/012/ifa.tl: Bind unbound x y variables in one test case. * tests/012/struct.tl: Suppress unbound variable warnings in some test cases. * uwind.c (uw_throw): If a warning is unhandled, then print its message with a "warning" prefix and then throw a continue exception. (uw_register_subtype): Eliminate the check for sub already being a subtype of sup. This allows us to officially register new types against t. (uw_late_init): Register continue exception type as a subtype of the restart type. Formally register warning type. * txr.1: Documented ignwarn.
* bugfix: quasilit read/print consistency, part 2.Kaz Kylheku2016-11-261-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this patch commit I'm addressing the issue introduced in part 1 that expressions in @(output) blocks are still using (sys:expr ...) wrapping, but are passed down to an evaluator which now expects unwrapped expressions now. As part of this change, I'm changing the representation of @expr from (sys:expr . expr) to (sys:expr expr). * eval.c (format_field): Adjust access to sys:expr expression based on new representation. (transform_op): Likewise. * lib.c (obj_print_impl): Likewise. * match.c (dest_bind): Likewise. (do_txeval): Likewise. (do_output_line): Likewise, in some compat code. Here is the fix for the issue: when calling tx_subst_vars, we pass a list of one element containing the expression, not wrapped in sys:expr. Previously, we passed a one-element list containing the sys:expr. * parser.y (o_elem): If a list occurs in the syntax, represent it as (sys:expr list) rather than (sys:expr . list). (list): Do the same for @ n_expr syntax. (expand_meta, make_expr): Harmonize with the representation change.
* bugfix: quasilit read/print consistency, part 1.Kaz Kylheku2016-11-261-20/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bug is that `@@@a` prints as `@@a` which reads as a different object. In this patch we simplify how quasiliterals are represented. Embedded expressions are no longer (sys:expr E), just E. Meta-numbers N and variables V are still (sys:var N). However `@@a` and `@a` remain equivalent. * eval.c (subst_vars): No need to look for expr_s; just evaluate a compound form. The recursive nested case is unnecessary and is removed. (expand_quasi): Do nothandle expr_s; it is not part of the quasi syntax any more. * lib.c (out_quasi_str): Do not look for expr_s in the quasi syntax; just print any expression with a @ the fallback case. * match.c (tx_subst_vars): Analogous changes to those done in subst_vars in eval.c. * parser.y (quasi_meta_helper): Static function removed. This was responsible for the issue due to stripping a level of meta from expressions already having a meta on them. (quasi_item): In the `@` n_expr syntax case, no longer call quasi_meta_helper. The remaining logic is simple enough to put in line. Symbols and integers get wrapped with (sys:var ...); other expressions are integrated into the syntax as-is.
* Completion of fallback list implementation.Kaz Kylheku2016-11-161-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | * lib.c (find_symbol): New function. (symbol_present): Search the fallback list also to determine whether the symbol is visible. * lib.h (find_symbol): Declared. * parser.y (sym_helper): Implement a new behavior for qualified symbols. Interning new symbols is only allowed for packages that have an empty fallback list. * parser.c (get_visible_syms): New static function. (find_matching_syms): Use get_visible_syms to get the list of eligible symbols. This way the fallback list of the package is included if it is the current package. * share/txr/stdlib/package.tl (defpackage): Do not insert a default (:use usr) if there is no :usr clause. Since defpackage is very new, no need for backward compatibility; the amount of code depending on this is likely zero. * txr.1: Documented fallback list feature.
* Start of fallback package list implementation.Kaz Kylheku2016-11-161-10/+10
| | | | | | | | | | | | | | | | | | | | * eval.c (eval_init): Register package-fallback-list and set-package-fallback-list intrinsics. * lib.c (package_fallback_list, set_package_fallback_list, intern_fallback): New functions * lib.h (package_fallback_list, set_package_fallback_list, intern_fallback): Declared. * parser.y (sym_helper): Slightly restructure function so that the symbol interning is done separately in the various cases. In the unqualified symbol case, use intern_fallback to search the fallback list of the current package. * share/txr/stdlib/package.tl (defpackage): Implement :fallback clause.
* Implement *package* special var; package overhaul.Kaz Kylheku2016-11-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * eval.c (load): Rebind *package* in the local dynamic environment already established for the sake of *load-path*. By doing this we cause *package* to be restored to its prior value, which allows the loaded file to alter it. Common Lisp works this way. (eval_init): Register *package* variable, with the user package as its default value. * lib.c (package_s): New symbol variable. (intern, rehome_sym): Default the package argument to the current package, not to user_package. (get_user_package, get_system_package, get_keyword_package): Functions removed. (get_current_package): New function. (obj_print_impl): Revise symbol printing. Keyword and uninterned symbols are printed with : and #: prefixes. The remainder are printed with a package prefix if their home package isn't the current package. * lib.h (keyword_package, user_package, system_package): These macros are just straight aliases for the global variables, not going through the lookup mechanism, which was pointless. (cur_package): New macro. (package_s): Declared. (get_current_package): Declared. * lisplib.c (lisplib_try_load): Establish a local dynamic environment, and bind the *package* variable to the user package which the library modules expect. * parser.c (find_matching_syms, provide_completions): Treat unqualified symbols in the current package rather than user package. * parser.y (sym_helper): Intern unqualified symbols in the current package, not user package. * txr.1: Document that the variables user-package, system-package and keyword-package should not be modified. Document the *package* special variable, and that intern and rehome-sym default their package argument to its value. (Here we get rid of wrong references to the undocumented variable *user-package*).
* Support #: reading for uninterned symbols.Kaz Kylheku2016-11-071-6/+10
| | | | | | | | | | | | | | * parser.l (BTKEY, NTKEY): Renamed to BTKWUN and NTKWUN ("keyword and uninterned") respectively. Include an optional match for the # character. (BTOK, NTOK): Refer to BTKEY and NTKEY respectively * parser.y (sym_helper): Implement uninterned symbols by detecting when the package name string is "#" and handling specially. * txr.1: Documented package prefixes and uninterned symbols.
* New #; syntax for erasing following object.Kaz Kylheku2016-11-071-4/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parser.c (parser_circ_ref): Don't generate the circular reference if circular suppression is in effect. * parser.h (struct parser): New member, circ_suppress. We use this for suppressing the generation of circular #n# references in erased objects. * parser.l (grammar): Scan #; producing HASH_SEMI token. * parser.y (HASH_SEMI): New token. (hash_semis_n_expr, hash_semis_i_expr, ignored_i_exprs, ignored_n_exprs): New nonterminals, needed for supporting the use of #; in front of top-level forms. (spec): Use hash_semis_n_expr and hash_semis_i_expr instead of n_expr and i_expr. (r_expr): Support object erasure within nested syntax. (yybadtoken): Handle H_SEMI token. (parse): Initialize new circ_suppress member of parser struct to zero. * txr.1: Documented. * genvim.txr (txr_ign_par, txr_ign_bkt, txr_ign_par_interior, txr_ign_bkt_interior): New regions for colorizing erased objects (partial support). (txr_list, txr_bracket, txr_mlist, txr_mbrackets): Include erased objects by including regions txr_ign_par and txr_ign_bkt. * txr.vim, tl.vim: Regenerated.
* parser: missing case for #R in yybadtoken.Kaz Kylheku2016-11-071-0/+1
| | | | | * parser.y (yybadtoken): HASH_R token not handled; must be mapped to "#R" string.
* Fix struct lit problem exposed by circular syntax.Kaz Kylheku2016-10-201-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The semantics of how struct literals come to life is poorly designed: namely, the slot-value pairs in the struct literal are used as the plist argument in a call to make-struct. This is wrong because the implied initializations are then clobbered by the structure type's :init and :postinit handlers, resulting in an object with slot values that don't match what is in the literal. When you add circular syntax to the mix, things get worse. Slots may be initialized with (sys:circ-ref ...) expressions corresponding to #<n># syntax. These expressions then get clobbered by the constructor actions before the circ_backpatch processes the syntax. * parser.y (struct): Use make_struct_lit rather than make_struct to instantiate struct object. * struct.tl (sys:struct-lit): Expand to a form which calls sys:make-struct-lit, rather than make-struct. * struct.c (struct_init): Register new make_struct_lit function as sys:make-struct-lit intrinsic. (make_struct_lit): New function. * struct.h (make_struct_lit): Declared. * tests/012/struct.tl: struct literal expansion test case updated. * txr.1: Updated documentation of struct literals. Added compat notes.
* Adding notation for cycles and shared structure.Kaz Kylheku2016-10-181-1/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit implements the parse-side support for handling a notation that exists in ANSI Common Lisp for specifying objects with cycles and shared substructure. * parser.h (struct parser): New members, circ_ref_hash and circ_count. (circref_s, parser_resolve_circ, parser_circ_def, parser_circ_ref): Declared. * parser.c (circref_s): New symbol variable. (parser_mark): Visit the new circ_ref_hash member of the parser structure. (parser_common_init): Initialize new members circ_ref_hash and circ_count of parser structure. (patch_ref, circ_backpatch): New static functions. (parser_resolve_circ, parser_circ_def, parser_circ_ref): New functions. (circref): New static function. (parse_init): Initialize circref_s as sys:circref symbol. Register sys:circref function. * parser.l (grammar): Scan #<num>= and #<num># notation as tokens, extracting their numeric value. * parser.y (HASH_N_EQUALS, HASH_N_HASH): New token types. (i_expr, n_expr): Adding phrases for hash-equalsign and hash-hash syntax. (yybadtoken): Handle new token types in switch. (parse_once): Call parser_resolve_circ after parsing to rewrite any remaining #<num># references in the structure to the objects they denote. (parse): Reset new struct parse members to initial state. Call parser_resolve_circ after parsing to rewrite any remaining #<num># references.
* Detect cycles in rlcp_tree.Kaz Kylheku2016-10-181-3/+16
| | | | | | | | | | | | | This will be required when the parser becomes capable of creating object graphs with cycles. * parser.c (parser_callgraph_circ_check): New function. * parser.h (struct circ_stack): New struct. (parser_callgraph_circ_check): Declared. * parser.y (rlcp_tree_rec): New static function. (rlcp_tree): Reduced to wrapper for rlcp_tree_rec.
* Synchronize license comments with LICENSE.Kaz Kylheku2016-10-011-16/+17
| | | | | | | | | | | | | | | | | | | | * Makefile, args.c, args.h, arith.c, arith.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/except.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Revert to verbatim 2-Clause BSD.
* Regression: @(rep) wrongly diagnoses empty clause.Kaz Kylheku2016-09-061-2/+2
| | | | | | | Introduced on 2016-04-27 in 7afbcc19. * parser.y (elem): Check $4 phrase position for empty clauses, rather than $2. That's where they are.
* Fix emulation of TXR 135 @(if) semantics.Kaz Kylheku2016-08-291-2/+4
| | | | | | | | | * parser.y (if_clause, elif_clauses_opt): The previous commit changes the emulation of old @(if) behavior, since expressions obtained via the n_exprs_opt grammar phrase are not subject to expand_meta. We can counteract this by calling expand_meta in the compatibility code.
* Fix broken expansion in @(if) and output @(repeat).Kaz Kylheku2016-08-291-5/+5
| | | | | | | | | * parser.y (if_clause, elif_clauses_opt, repeat_clause, rep_elem): Recognize argument expressions as n_exprs_opt rather than exprs_opt, so that expand_meta is not applied. They are Lisp expressions, which are broken by expand_meta. A failing test case is op syntax with @num metanum variables, e.g. @(if (foo (op bar @1.slot))).
* Allow empty @(catch) and @(finally).Kaz Kylheku2016-06-051-12/+3
| | | | | | | * parser.y (catch_clauses_opt): Don't diagnose empty catch and finally. There is no benefit in doing so; moreover, it contravenes the documentation, which explicitly says these may be empty. I.e. this fixes a regression.
* Diagnose empty clauses better in parallel constructs.Kaz Kylheku2016-04-271-86/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this change we allow the grammar, as such, to express empty clauses in the parallel constructs like @(some), @(cases), @(gather) and so forth. However, we raise an error when these occur. This results in a cleaner diagnostic behavior. In the future, empty clauses may be allowed; the semantics has to be worked out. An empty clause should neither succeed nor fail; the behavior should be as if it is not there. The general strategy in this patch is to eliminate the use of the clauses terminal symbol (completely) and use clauses_opt everywhere, and then diagnose when that produces a nil. Some elems are similarly changed to elems_opt in the horizontal versions of directives. * parser.y (clauses): Nonterminal symbol completely removed. (spec): Empty production rule removed, by using clauses_opt instead of clauses. (all_clause, some_clause, none_clause, maybe_clause, cases_clause, choose_clause, gather_clause): Instead of diagnosing empty clauses via a dedicated grammar rule, diagnose by looking for nil subtree emerging from from a clauses phrase. (gather_parts): Use clauses_opt. Yield nil instead of a cons if this clauses_opt is nil. (additional_gather_parts): Simplify grammar and actions by recursing back to gather_parts. If gather_parts produces nil, diagnose the empty and/or subclause situation. (collect_clause): Use clauses_opt and diagnose empty. Simplify error production, which doesn't have to look at the lookahead token yychar any more to implement the empty diagnostic. (clause_parts): Use clauses_opt instad of clauses. Yield nil instead of a cons if clause_opt yields nil. (additional_parts): Grammar and actions simplified by recursing back to clause_parts. If the recursive clause_parts produces nil, generate empty and/or subclause diagnostic. (elem): Use elems_opt for the various horizontal clauses and implement empty checks for them. Explicit empty productions are eliminated. (clause_parts_h): Changed in way analogous to clause_parts. (additional_parts_h): Changed analogously to additional_parts. (try_clause): Use clauses_opt and catch empties. Eliminate the second error production. (catch_clauses_opt): Diagnose empty catch and finally, unless in compatibility mode with 139 or less. The grammar already uses clauses_opt here; empty catches and finally had been allowed. Fixed messages in error productions for catch and finally to refer to catch and finally rather than try.
* Strengthen against resource leaks upon exceptions.Kaz Kylheku2016-04-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | * glob.c (glob_wrap): Perform argument conversions that might throw before allocating UTF-8 string. * parser.y (text): In the action for SPACE, the lexeme is not needed so free($1) right away. If regex_compile were to throw an exception, that lexeme will leak. * socket.c (getaddrinfo_wrap): Harden against leakage of node_u8 and service_u8 strings with an unwind block. For instance, the hints structure could contain bad values which cause addrinfo_in to throw. * stream.c (make_string_byte_input_stream): Perform possibly throwing argument conversions before allocating resources. * sysif.c (mkdir_wrap, mknod_wrap, chmod_wrap, symlink_wrap, link_wrap, setenv_wrap, crypt_wrap): Likewise. * syslog.c (openlog_wrap, syslog_wrapv): Likewise.
* Bugfix: @(if expr) not macro-expanding expr.Kaz Kylheku2016-04-181-2/+2
| | | | | * parser.y (if_clause, elif_clauses_opt): Add missing expand calls.
* Bugfix: @(output) not expanding some Lisp exprs.Kaz Kylheku2016-04-171-1/+1
| | | | | | | * parser.y (make_expr): This function is part of a hack for converting some hard-coded syntax like @(if ...) or @(and ...) in an @(output) block to a to a Lisp expression. Alas, the crucial step of expanding the Lisp form was neglected.