summaryrefslogtreecommitdiffstats
path: root/match.c
Commit message (Collapse)AuthorAgeFilesLines
* Converting cast expressions to macros that are retargettedKaz Kylheku2014-10-171-71/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to C++ style casts when compiling as C++. * lib.h (strip_qual, convert, coerce): New casting macros. (TAG_MASK, tag, type, wli_noex, auto_str, static_str, litptr, num_fast, chr, lit_noex, nil, nao): Use cast macros. * arith.c (mul, isqrt_fixnum, bit): Use cast macros. * configure (INT_PTR_MAX): Define using cast macro. * debug.c (debug_init): Use cast macro. * eval.c (do_eval, expand_macro, reg_op, reg_mac, eval_init): Use cast macros. * filter.c (filter_init): Use cast macro. * gc.c (more, mark_obj, in_heap, mark, sweep_one, unmark): Use cast macros. * hash.c (hash_double, equal_hash, eql_hash, hash_equal_op, hash_hash_op, hash_print_op, hash_mark, make_hash, make_similar_hash, copy_hash, gethash_c, gethash, gethash_f, gethash_n, remhash, hash_count, get_hash_userdata, set_hash_userdata, hash_iter_destroy, hash_iter_mark, hash_begin, hash_uni, hash_diff, hash_isec): Use cast macros. * lib.c (code2type, chk_malloc, chk_malloc_gc_more, chk_calloc, chk_realloc, chk_strdup, num, c_num, string, mkstring, mkustring, upcase_str, downcase_str, string_extend, sub_str, cat_str, trim_str, c_chr, vector, vec_set_length, copy_vec, sub_vec, cat_vec, cobj_print_op, obj_init): Likewise. * match.c (do_match_line, hv_trampoline, match_files, dir_tables_init): Likewise. * parser.l (grammar): Likewise. * parser.y (parse): Likewise. * rand.c (make_state, make_random_state, random_fixnum, random): Likewise. * regex.c (CHAR_SET_L2_LO, CHAR_SET_L2_HI, CHAR_SET_L1_LO, CHAR_SET_L1_HI, CHAR_SET_L0_LO, CHAR_SET_L0_HI, L0_full, L0_fill_range, L1_full, L1_fill_range, L1_contains, L1_free, L2_full, L2_fill_range, L2_contains, L2_free, L3_fill_range, L3_contains, L3_free, char_set_create, char_set_cobj_destroy, nfa_state_accept, nfa_state_empty, nfa_state_single, nfa_state_wild, nfa_state_set,
* * match.c (dest_bind): More detailed log message for variableKaz Kylheku2014-10-161-1/+2
| | | | mismatch.
* New @(line) and @(chr) directives.Kaz Kylheku2014-10-161-1/+48
| | | | | | | | | | | | | * match.c (line_s): New variable. (h_chr, v_line): New static functions. (syms_init): line_s initialized. (dir_tables_init): Register v_line and h_chr. * match.h (line_s): Declared. * txr.1: Document @(line) and @(chr) directives. * txr.vim: Regenerated.
* * match.c (subst_vars): Fix buggy rendering of TXR Lisp expressionsKaz Kylheku2014-10-151-5/+24
| | | | | | | | | | | | | | | | | that evaluate to lists. For instance `@(list)` renders to the string "nil", and `@(list 1 2)` renders as "(1 2)". The desired behavior is "" and "1 2", respectively. (do_output_line): In output directives, there is a similar problem. A @(list) in the middle of an output block turns to nil, and a @(list 1 2) renders in parentheses as (1 2). Furthermore, there is the additional problem that no filtering is applied to the interpolated value. These behaviors are subject to the compatibility option, since they change the externally visible behavior of TXR programs. * txr.1: Document that empty lists in @(output) variable substitutions turn into nothing. Document value of 100 for -C option, describing the above issue.
* Eliminating the extra list wrapping applied to regularKaz Kylheku2014-10-031-28/+30
| | | | | | | | | | | | | | | | | | | expression objects in the syntax tree. The parser just puts out a #<regex ...> instead of (#<regex ...> regex-syntax). * eval.c (do_eval): We no longer need the hack of treating (#<regex> ...) as a special form which evaluates to #<regex>. (expand): We no longer have to skip over regex syntax, so the case is removed. * match.c (h_var, do_txeval, do_match_line): regexp cases are no longer subcases of consp but stand on their own. In do_match_line, we introduce a COBJ case into the type switch for regexes. * parser.y: regexes are now compiled in the regex and lisp_regex grammar rules instead of the dependent rules, and are not wrapped in extra syntax.
* * match.c (h_var): Fix regression introduced in 2014-08-11Kaz Kylheku2014-10-031-4/+3
| | | | | | | | | | | commit. The incompleteness of that change broke the case of an unbound variable followed by a bound variable. The value of the second variable was still being wrapped in the old complicated representation before being pushed to the front of the spec. * txr.1: Replace bogus text which says that variables are not bound to regexes, and so regex matches from variable substitutions do not arise. This works fine after this change.
* * match.c (v_load): Fix regression introduced in 94: broken @(load).Kaz Kylheku2014-08-291-1/+1
|
* Uprooting stupidities in handling of output variables.Kaz Kylheku2014-08-131-5/+1
| | | | | | | | | | | | | | | | | | | | * parser.y (o_elems_transform): Remove useless function which was only unwrapping the strange parse of output vars. (o_elems_opt, rep_elem, quasilit, wordsqlit): Eliminate o_elems_transform call. (o_var, q_var): Eliminate the phrase structure rules which match an extra o_elem or quasi_item, and which incorporate them into the var syntax tree element. Place the modifiers into the third position, not fourth. * eval.c (subst_vars): Eliminate handling of "pat" element. Actually that was not even there thanks to o_elems_transform being applied: dead code. Pull modifiers from the third element of the var form now, not fourth. * match.c (subst_vars): Similar changes as in the match.c subst_vars function. Here the pat variable is even more obviously useless; if it is not nil, it is just punted back to the spec.
* Fix regression in previous change: we must match a compound textKaz Kylheku2014-08-131-8/+13
| | | | | | | | | | | | element whole, and not break it up. * match.c (search_match): Take a spec argument. (h_var): Turn a text element into a one-element spec and process with search_match. * txr.1: Updated text about matching of variables followed by a directive or function, and about consecutive variables via directive.
* When a variable is delimited by some form other thanKaz Kylheku2014-08-121-95/+110
| | | | | | | | | | | | | | | | | | | | the contents of a variable, fixed string or regex, we now use the entire tail of the specline to find the match. So for instance @var@(trailer)foo works as intuition might expect. * match.c (search_form): Static function removed. (search_match): New static function based on search_form. Does not handle regexes, and does not update c->bindings. (h_var): Renamed local variable pat to next. Added a few missing rlcp's. Combined the cases when pat is a cons to one block so consp isn't repeatedly tested. Function now handles a var followed by (sys:text ...) elements specially; the first element of the text block is pulled out and matched. Implemented "var delimiting spec" general case which matches the entire tail of the spec at successive character positions until a match is found, and the skipped text goes into the variable.
* First cut at restructuring how variable matching works in the patternKaz Kylheku2014-08-111-22/+15
| | | | | | | | | | | | | | | | | language. The goal is to remove the strict behavior of using only one element of context after a variable. variable form at parse time: we unravel that first. * parser.y (grammar): Simplifying the phrase structure rule for the var element. All the variants that have a trailing elem are removed. The abstract syntax changes; the modifier moves to the third position in the list. * match.c (h_var): Matching change: the element which follows a variable is now pulled from the specline rather than the variable syntax, which is how it should have been done in the first place. The modifiers are pulled from a different spot in the variable syntax.
* Big switch to reentrant lexing and parsing.Kaz Kylheku2014-08-021-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parser.l (YY_INPUT): Stop relying on removed yyin_stream; refer to stream via yyextra. (yyin_stream, lineno, errors, spec_file_str, prepared_error_message): Global variables removed. (yyget_column, yyset_column): Missing prototypes not generated by flex in bison bridge mode have to be added by us to avoid warning. (yyerror): Takes parser and scanner as parameters. Prepared error message is now in the parser context. Calls to other error handling functions receive scanner context. (yyerr): New function. (yyerrorf, yyerrprepf): Takes scanner argument, chases extra data to get to parser, and refers to parser variables instead of globals. (num_esc): Scanner argument added. (%option reentrant, %option bison-bridge, %option extra-type): New flex options. (grammar): yyscanner added everywhere. (end_of_char): Takes scanner argument. (parse_init): Removed references to yyin_stream and prepared_error_message. (parse_reset): Function renamed to open_txr_file. Returns results via pointers instead of setting global variables. (regex_parse, lisp_parse): Use reentrant parser interface. * parser.y (yyerror): Prototype removed. (yylex): Prototype moved after grammar, with new arguments. (sym_helper, define_transform): Take scanner argument. (make_expr): Takes parser argument. (rlrec): New static function. (rl): Function turned into macro. (mkexp, symhlpr): New macros. (%purse-parser, %parse-param, %lex-param): New Yacc options. (grammar): Actions re-worked for reentrance. Parser and scanner contexts are passed down to helper functions, in some cases via the three new macros. The result of the parse is stored in the syntax_tree member of the parser_t structure instead of a global. The yylex function receives the scanner instance. (get_spec): Function removed. (parse): New function. * parser.h (lineno, errors, yyin_stream, spec_file_str): Declarations removed. (parser_t): New struct. (yyerr): New function declared. (yyparse, yyerror, yyerrorf, end_of_regex, end_of_char, yylex, yylex_destroy): Declarations updated.
* * Makefile, arith.c, arith.h, combi.c, combi.h, configure, debug.c,Kaz Kylheku2014-07-231-16/+16
| | | | | | | | debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, hash.c, hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, syslog.c, syslog.h, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Synchronize license header with LICENSE.
* * match.c (subst_vars): Bugfix: I neglected to apply theKaz Kylheku2014-07-221-1/+1
| | | | | filter which is in effect to the result of interpolating a TXR Lisp expression, oops!
* * match.c (v_do, v_require): Set up and tear down environment frame,Kaz Kylheku2014-07-151-2/+11
| | | | | | | like other situations that evaluate TXR Lisp from the pattern language. Otherwise obscure things will go wrong. (h_do): Same as above, and additionally, add the forgotten call to install the bindings into the match context.
* * match.c (h_eol): Fix broken horizontal @(eol).Kaz Kylheku2014-07-151-1/+1
| | | | | It should be returning next_spec_k, rather than bindings, which indicate a complete match.
* Optimization: add missing tail updates to some listKaz Kylheku2014-06-201-1/+1
| | | | | | | | | | | collecting loops. * lib.c (tuples_func, where, sel): Catch return value of list_collect and update tail variable. * match.c (do_txeval): Likewise. * parser.y (expand_meta): Likewise for list_collect_nconc.
* * Makefile: Install share/txr/stdlib/*.txr material.Kaz Kylheku2014-06-121-1/+2
| | | | | | | | | | | * match.c (do_txeval): If a variable is not in the bindings, fall back on treating it as a TXR Lisp dynamic variable. This allows us to refer to the stdlib variable from a quasistring in a @(load ...) directive. * txr.c (sysroot_init): Register new variable, *txr-version*. * share/txr/stdlib/ver.txr: New file.
* * match.c (v_load): use the abs_path_p function instead ofKaz Kylheku2014-06-121-1/+1
| | | | | | | | | | | | | checking for leading slash. * stream.c (abs_path_p): New function. (stream_init): Register abs_path_p as abs-path-p. * stream.h (abs_path_p): Declared. * txr.1: Documented abs-path-p. * dep.mk: Updated.
* The dumping of bindings and printing of false must nowKaz Kylheku2014-06-091-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | be explicitly requested by the -B option. * match.c (opt_nobindings): Variable removed. (opt_print_bindings): New variable. (extract): Print bindings or "false" if opt_print_bindings is true. * stream.c (output_produced): Variable removed. (stdio_put_string, stdio_put_char, stdio_put_byte): Remove update of output_produced. * stream.h (output_produced): Declaration removed. * txr.1: Documentation updated. * txr.c (txr_main): Option 'b' does nothing. 'B', 'l', 'a', and '--lisp-bindings' set opt_print_bindings to 1. * txr.h (opt_nobindings): Declaration removed. (opt_print_bindings): Declared. * unwind.c (uw_throw): When exiting due to a query error or file error, print false when opt_print_bindings is true.
* String type related bugfixes: neglecting to handle all three kinds inKaz Kylheku2014-05-101-0/+4
| | | | | | | | | | | | | | | some places. In particular, the test case echo : | ./txr -c '@a:@a' - breaks because of neglected LIT in do_match_line. * arith.c (tofloat, toint): Handle LIT type in switch. * lib.c (ref, refset, replace, update): Handle LSTR type. * match.c (do_match_line, do_output_line): Handle LSTR and LIT objects in switch.
* Change to how locations are passed around, for the sake of generationalKaz Kylheku2014-03-291-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GC. The issue being solved here is the accuracy of the gc_set function. The existing impelmentation is too conservative. It has no generation information about the memory location being stored, and so it assumes the worst: that it is a location in the middle of a gen 1 object. This is sub-optimal, creating unacceptable pressure against the checkobj array and, worse, as a consequence causing unreachable gen 0 objects to be tenured into gen 1. To solve this problem, we replace "val *" pointers with a structure of type "loc" which keeps track of the object too, which lets us discover the generation. I tried another approach: using just a pointer with a bitfield indicating the generation. This turned out to have a serious issue: such a bitfield goes stale when the object is moved to a different generation. The object holding the memory location is in gen 1, but the annotated pointer still indicates gen 0. The gc_set function then makes the wrong decision, and premature reclamation takes place. * combi.c (perm_init_common, comb_gen_fun_common, rcomb_gen_fun_common, rcomb_list_gen_fun): Update to new interfaces for managing mutation. * debug.c (debug): Update to new interfaces for managing mutation. Avoid loc variable name. * eval.c (env_fbind, env_fbind): Update to new interfaces for managing mutation. (lookup_var_l, dwim_loc): Return loc type and update to new interfaces. (apply_frob_args, op_modplace, op_dohash, transform_op, mapcarv, mappendv, repeat_infinite_func, repeat_times_func): Update to new interfaces for managing mutation. * eval.h (lookup_var_l): Declaration updated. * filter.c (trie_add, trie_compress, trie_compress_intrinsic, * build_filter, built_filter_from_list, filter_init): Update to new * interfaces. * gc.c (gc_set): Rewritten to use loc type which provides the exact generation. We do not need the in_malloc_range hack any more, since we have the backpointer to the object. (gc_push): Take loc rather than raw pointer. * gc.h (gc_set, gc_push): Declarations updated. * hash.c (struct hash): The acons* functions use loc instead of val * now. (hash_equal_op, copy_hash, gethash_c, inhash, gethash_n, pushhash, Change to how locations are passed around, for the sake of generational GC. The issue being solved here is the accuracy of the gc_set function. The existing impelmentation is too conservative. It has no generation information about the memory location being stored, and so it assumes the worst: that it is a location in the middle of a gen 1 object. This is sub-optimal, creating unacceptable pressure against the checkobj array and, worse, as a consequence causing unreachable gen 0 objects to be tenured into gen 1. To solve this problem, we replace "val *" pointers with a structure of type "loc" which keeps track of the object too, which lets us discover the generation. I tried another approach: using just a pointer with a bitfield indicating the generation. This turned out to have a serious issue: such a bitfield goes stale when the object is moved to a different generation. The object holding the memory location is in gen 1, but the annotated pointer still indicates gen 0. The gc_set function then makes the wrong decision, and premature reclamation takes place. * combi.c (perm_init_common, comb_gen_fun_common, rcomb_gen_fun_common, rcomb_list_gen_fun): Update to new interfaces for managing mutation. * debug.c (debug): Update to new interfaces for managing mutation. Avoid loc variable name. * eval.c (env_fbind, env_fbind): Update to new interfaces for managing mutation. (lookup_var_l, dwim_loc): Return loc type and update to new interfaces. (apply_frob_args, op_modplace, op_dohash, transform_op, mapcarv, mappendv, repeat_infinite_func, repeat_times_func): Update to new interfaces for managing mutation. * eval.h (lookup_var_l): Declaration updated. * filter.c (trie_add, trie_compress, trie_compress_intrinsic, * build_filter, built_filter_from_list, filter_init): Update to new * interfaces. * gc.c (gc_set): Rewritten to use loc type which provides the exact generation. We do not need the in_malloc_range hack any more, since we have the backpointer to the object. (gc_push): Take loc rather than raw pointer. * gc.h (gc_set, gc_push): Declarations updated. * hash.c (struct hash): The acons* functions use loc instead of val * now. (hash_equal_op, copy_hash, gethash_c, inhash, gethash_n, pushhash,
* * eval.c (me_quasilist): New static function.Kaz Kylheku2014-03-251-4/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | (eval_init): Register me_quasilist as quasilist macro expander. * lib.c (quasilist_s): New global variable. (obj_init): quasilist_s initialized. * lib.h (quasilist_s): Declared. * match.c (do_txreval): Handle quasilist syntax. * parser.l (QWLIT): New exclusive state. Extend lexical grammar to transition to QWLIT state upon the #` or #*` sequence which kicks off a word literal, and in that state, piecewise lexically analyze the QLL, mostly by borrowing rules from quasiliterals. * parser.y (QWORDS, QWSPLICE): New tokens. (n_exprs): Integrate splicing form of QLL syntax. (n_expr): Integrate non-splicing form of QLL syntax. (litchars): Propagate line number info. (quasilit): Fix "string literal" wording in error message. * txr.1: Introduced WLL abbreviation for word list literals, cleaned up the text a little, and documented QLL's.
* * match.c (v_trailer): Fix segfault. The code whichKaz Kylheku2014-03-091-1/+1
| | | | | | helps implement the special interaction between @(accept) and @(trailer) was not handling the situation when there is not current unwind exit point.
* * lib.c (lazy_sub_str): Bugfix: "from" was mistakenly usedKaz Kylheku2014-03-091-1/+1
| | | | | | | in the adjustment of the "to" value. * match.c (search_form): Use predefined constants for -1 and 1 instead of calling num.
* Fixing broken processing of horizontal matching acrossKaz Kylheku2014-03-091-2/+9
| | | | | | | | | | | | | | long lines produced by @(freeform). Once the matching passes about 4000 characters, the "consume_prefix" function kicks in to save memory. Then any code which is not properly written to handle this displaced situation will break. * match.c (h_text, h_var, h_coll, h_parallel, h_fun): Bugfix. The recursive calls to match_line return an absolute position. From this value we must subtract c->base if we are to compare it with c->pos, or update c->pos. If we use the absolute value, we are abruptly jumping ahead in the data.
* * match.c (LOG_MATCH, LOG_MISMATCH): Wouldn't you know it;Kaz Kylheku2014-03-071-2/+2
| | | | | | | | the format strings in these macros contained a workaround for the broken * variable field width syntax, specifying ~*~a where the extra ~ in the middle just feeds a character that the broken state machine expects. These workarounds broke when I fixed the formatting, making -v mode useless.
* * lib.c (assert_s): New global variable.Kaz Kylheku2014-03-061-0/+68
| | | | | | | | | | | | | | | (obj_init): Intern assert symbol, store in assert_s. * lib.h (assert_s): Declared. * match.c (typed_error, v_assert, h_assert): New static functions. (dir_tables_init): Register v_assert and h_assert. Register assert_s as non-data-matching directive. * unwind.c (uw_init): Register assert as a subtype of error. * txr.1: Describe assert.
* * match.c: (v_next): Set the "curfile" in the context to "env" whenKaz Kylheku2014-03-061-1/+2
| | | | | | scanning environment. (open_data_source): Regression: was not setting c->curfile when opening anything.
* * match.c (match_files): Fix it again. The data (nil)Kaz Kylheku2014-03-061-1/+1
| | | | can occur from an interactive/real-time stream.
* Fixing regression caused by the 2014-02-19 change ("Fixed long-runningKaz Kylheku2014-03-051-2/+8
| | | | | | | | | | | | | | | issue ..."). * match.c (open_data_source): if c->data is t, but c->files is nil, set c->data to nil: we cannot possibly open anything later. (match_files): We need to call open_data_source one more time just before processing a line with horizontal material. The previous call(s) to open_data_source might not have opened anything. Before accesing car(c.data) the correct test is consp(c.data), not c.data. In the else clause, we now specificially check for nilp(c.data) which is the correct indicator of no more data. If c.data is any other atom at that point, we have an internal error, for which an assertion is added now.
* Replacing uses of the eq function which are used only as C booleans,Kaz Kylheku2014-02-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | with just using the == operator. Removing cobj_equal_op since it's indistinguishable from eq. Streamlining missingp and null_or_missing_p. * eval.c (transform_op): eq to ==. (c_var_ops): cobj_equal_op to eq. * filter.c (trie_compress, trie_lookup_feed_char, filter_string_tree, html_hex_continue, html_dec_continue): eq to ==. * hash.c (hash_iter_ops): cobj_equal to eq. * lib.c (countq, getplist, getplist_f, search_str_tree, posq): eq to ==. (cobj_equal_op): Function removed. * lib.h (cobj_equal_op): Declaration removed. (missingp): Becomes a simple macro that yields a C boolean instead of t/nil val, because it's only used that way. (null_or_missing_p): Becomes inline function returning int. * match.c (v_output): eq to ==. * rand.c (random_state_ops): cobj_equal_op to eq. * regex.c (char_set_obj_ops, regex_obj_ops): cobj_equal_op to eq. (reg_derivative): Silly if3 expression replaced by null. (regexp): Redundant if2 expression wrapped around eq removed. * stream.c (null_ops, stdio_ops, tail_ops, pipe_ops, string_in_ops, byte_in_ops, string_out_ops, strlist_out_ops, dir_ops, cat_stream_ops): cobj_equal_op to eq. * syslog.c (syslog_strm_ops): cobj_equal_op to eq.
* The C function nullp is being renamed to null, and the rarelyKaz Kylheku2014-02-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | used global variable null which holds a symbol becomes null_s. A new macro called nilp is added that more efficiently checks whether an object is nil, producing a C boolean value rather than t or nil. Most of the uses of nullp in the codebase just become the more streamlined nilp. * debug.c (show_bindings): nullp to nilp * eval.c (lookup_var, lookup_var_l, lookup_fun, lookup_sym_lisp1, do_eval, expand_qquote, expand_quasi, expand_op): nullp to nilp. (op_modplace): nullp to null. (eval_init): Update registration of null and not from C function nullp to null. * filter.c (trie_compress, html_hex_continue): nullp to nil. (filter_string_tree): null to null_s. * hash.c (hash_next): nullp to nilp. * lib.c (null): Variable renamed to null_s. (code2type): null to null_s. (lazy_flatten_scan, chainv, lazy_str, lazy_str_force_upto, obj_print, obj_pprint): nullp to nilp. (obj_init): null to null_s; nullp to null. * lib.h (null): declaration changed to null_s. (nullp): Inline function renamed to null. (nilp): New macro. * match.c (do_match_line): nullp to nilp. * rand.c (make_random_state): Likewise. * regex.c (compile_regex): Likewise.
* Fixing a long-running issue in the TXR pattern language: prematureKaz Kylheku2014-02-191-21/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | opening of files, prior to directives that actually need data. The documentation basically lied that this is the case: namely, the text "A file isn't opened until the query demands material from that file, and then the contents are read on demand, not all at once." This is now a fact. * match.c (non_matching_directive_table): New global variable. (open_data_source): New static function. Contains an almost verbatim migration of the source-opening logic that used to be in match_files. The useless assignment to c->nil is gone, and c->data == t is explicitly tested for. Instead of assuming that only the @(next) directive does not need to have a data source open, the table of non-matching directives is consulted. Opening the data source is now skipped for numerous directives. (match_files): Call open_data_source within the loop. This means that even after processing numerous non-matching directives, we will still correctly set up the data lazy list. (dir_tables_init): Initialize non_matching_directive_table, protect from GC and populate with numerous directives. * txr.1: Improved documentation for @(next :args), and removed a description of the hack that a single @(next) at the top of the query suppressed the opening of the data source.
* * eval.c (subst_vars): Bugfix: results of expressions notKaz Kylheku2014-02-111-3/+4
| | | | | | | | | | | | treated in the same way as variables: lists not stringified, causing expansions with parentheses, and sometimes errors due to unhandled objects. Also, use tostringp instead of format for stringifying objects. stringifying object. Bugfix.k * match.c (subst_vars): Added comment similar to the one in the subst_vars of eval.c. Removed superfluous conversion code where the str variable is already known to be a string.
* Changes to the list collection mechanism to improveKaz Kylheku2014-01-221-17/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the extension of list operations over vectors and strings. * eval.c (do_eval_args, bindings_helper, op_each, subst_vars, supplement_op_syms, mapcarv, mappendv): Switch from list_collect_* macros to functions. * lib.c (copy_list): Switch from list_collect* macros to functions. Use list_collect_nconc for the final terminator. Doing a copy there with list_collect_append was actually wasteful, and now that list_collect_append calls copy_list in places, it triggered runaway recursion. (make_like): Bugfix: list_vector was used instead of vector_list. (to_seq, list_collect, list_collect_nconc, list_collect_append): New functions. (append2, appendv, nappend2, sub_list, replace_list, ldiff, remq, remql, remqual, remove_if, keep_if, proper_plist_to_alist, improper_plist_to_alist, split_str, split_str_set, tok_str, list_str, chain, andf, orf, lis_vector, mapcar, mapcon, mappend, merge, set_diff, env): Switch from list_collect* macros to functions. (replace_str, replace_vec): Allow single item replacement sequence. * lib.h (to_seq): Declared. (list_collect, list_collect_nconc, list_collect_append): Macros removed, replaced by function declarations of the same name. These functions return the new ptail since they cannot assign to it, requiring all uses to be updated to do the assignment of the returned value. (list_collect_decl): Use val rather than obj_t *. * match.c (vars_to_bindings, h_coll, subst_vars, extract_vars, extract_bindings, do_output_line, do_output, v_gather, v_collect): Switch from list_collect* macros to functions. * parser.y (o_elems_transform): Likewise. * regex.c (dv_compile_regex, regsub): Likewise. * txr.c (txr_main): Likewise.
* * arith.c: Revised error messages to refer to Lisp names insteadKaz Kylheku2014-01-151-3/+5
| | | | | | | | | | of C names of functions, or otherwise clarified them. * filter.c: Likewise. * lib.c: Likewise. * match.c: Likewise.
* * match.c (do_txeval): Lift an annoying restriction in the patternKaz Kylheku2014-01-101-2/+4
| | | | | | | | | | language's expression evaluator. Whereas TXR Lisp expressions can be used int the pattern language, preceded by @, it was not possible to evaluate TXR Lisp variables this way. So instead of @var, some clumsy trick had to be used like @(identity var). This is not necessary any more. Code like @(next @*stdin*) will now work. * txr.1: Updated.
* * match.c (v_load): Call yylex_destroy after yyparse, so we don'tKaz Kylheku2014-01-071-0/+1
| | | | | have stray scanner state possibly interfering with a subsquent parse job.
* First cut at signal handling support.Kaz Kylheku2013-12-121-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Makefile (OBJS-y): Include signal.o if have_posix_sigs is "y". * configure (have_posix_sigs): New variable, set by detecting POSIX signal stuff. * dep.mk: Regenerated. * arith.c, debug.c, eval.c, filter.c, hash.c, match.c, parser.y, parser.l, rand.c, regex.c, syslog.c, txr.c, utf8.c: Include new signal.h header, now required by unwind, and the <signal.h> system header. * eval.c (exit_wrap): New function. (eval_init): New functions registered as intrinsics: exit_wrap, set_sig_handler, get_sig_handler, sig_check. * gc.c (release): Unused functions removed. * gc.h (release): Declaration removed. * lib.c (init): Call sig_init. * stream.c (set_putc, se_getc, se_fflush): New static functions. (stdio_put_char_callback, stdio_get_char_callback, stdio_put_byte, stdio_flush, stdio_get_byte): Use new functions to enable signals when blocked on I/O. (tail_strategy): Allow signals across sleep. (pipev_close): Allow signals across waitpid. (se_pclose): New static function. (pipe_close): Use new function to enable signals across pclose. * unwind.c (uw_unwind_to_exit_point): use extended_longjmp instead of longjmp. * unwind.h (struct uw_block, struct uw_catch): jb member changes from jmp_buf to extended_jmp_buf. (uw_block_begin, uw_simple_catch_begin, uw_catch_begin): Use extended_setjmp instead of setjmp. * signal.c: New file. * signal.h: New file.
* Bumping copyrights to 2014 and expressing them as year ranges.Kaz Kylheku2013-12-101-1/+1
| | | | Fixing some errors in copyright comments.
* debug_check calls are the culprit triggering lookaheadKaz Kylheku2013-12-021-4/+2
| | | | | | | | | | | | | in lazy lists. Changing the semantics of the data argument so that it can just be the list. * debug.c (debug): If data is a cons, then replace it with the first item. * match.c (match_files): Pass c.data to debug_check, rather than the value of if2(consp(c.data), car(c.data)) which accesses car(c.data) whether or not we are debugging. (match_fun): Likewise.
* Steps toward fixing an issue: lazy list readahead.Kaz Kylheku2013-12-011-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The problem is that accurate lazy lists are not suitable for real time use, where we want the TXR program to respond immediately to matching some datum. I'm implementing a simple, naive variant of lazy stream lists which simply populates the lazy cons by reading from the stream when the car or cdr fields are accessed. This type of stream can never be nil (empty list) even if the file is empty; in that case it will be (nil) and in general, it will have a spurious nil item at the end instead of ending in a string. (An adjustment was made in match.c to detect this; more will be needed.) I'm adding attributes to streams so streams can now have a "real-time" attribute. When a lazy string list is constructed over a real-time stream, the simple implementation is used. File streams are automatically real-time if (on Unix) they are tied to tty streams. Tail streams are also real-time. More work is needed to achieve the goal of this change, but this is a big step in the right direction. * configure: Detect isatty function. * lib.c (simple_lazy_stream_func): New static function. (lazy_stream_cons): Use simple implementation for real-time streams. * match.c (match_files): Do not call match_line_completely with a data line that is nil (as a result of simple lazy list over a real-time stream). A nil item in a lazy list of strings is treated as eof. * stream.c (real_time_k): New symbol variable. (struct strm_ops): New members: get_prop, set_prop. (struct stdio_handle): New member: is_real_time. (stdio_get_prop, stdio_set_prop): New static function. (stdio_ops, tail_ops, pipe_ops): stdio_get_prop and stdio_set_prop funtions wired in. (make_stdio_stream_common): Attribute streams as real-time if they are tty devices. (make_tail_stream): Tail streams are real-time attributed. (stream_set_prop, real_time_stream_p): New functions. (stream_init): Initialize real_time_k. * stream.h (real_time_k): Declared. (real_time_stream_p, stream_set_prop): Likewise.
* * configure: Added check to detect POSIX sleep function.Kaz Kylheku2013-11-271-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * eval.c (eval_init): Register new open_tail function as intrinsic. * match.c (complex_snarf, complex_stream): Update calls to make_stdio_stream and make_pipe_stream to take fewer arguments. (match_files): Support a stream object as a data source specification in place of a string. * parser.l (parse_reset): Update call to make_stdio_stream to take fewer arguments. * stream.c: Inclusion of <unistd.h> made properly conditional. (struct stdio_handle): pid member defined as pid_t only if we have fork functionality, otherwise defined as int. (tail_get_line, tail_get_char, tail_get_byte): New static functions. (tail_ops): New static structure. (make_stdio_stream_common): New static structure. (make_stdio_stream, make_pipe_stream): These functions lose the input and output parameters, which ended up never used. Reimplemented in terms of new common function. (make_tail_stream): New function. (make_pipevp_stream): Reimplemented in terms of new common function. (open_file, open_command): Simplified by removal of useless local variables and their computation, which used to be extra arguments to make_stdio_stream and make_pipe_stream. (open_tail): New function. (stream_init): Calls to make_stdio_stream updated. * stream.h (make_stdio_stream, make_pipe_stream): Declarations updated. (make_tail_stream, open_tail): Declared. * txr.c (txr_main): Calls to make_stdio_stream updated.
* * match.c (v_load): Bugfix: bindings were propagated in only one of twoKaz Kylheku2013-11-241-1/+2
| | | | | | subcases of a successful load. This bug means that a loaded TXR sub-query was not reliably able to bind variables that are then visible to subsquent directives in the parent.
* Fix for uninitialized struct member curfiles in the match context.Kaz Kylheku2013-11-241-7/+10
| | | | | | | | | This is used for reporting the current match location in debug traces. * match.c (mf_all): Takes new argument curfiles and initializes the match_files_ctx member of the same name. (do_match_line, mf_from_ml, match_filter, match_fun, extract): Pass curfile to ml_all.
* Nasty bug fixed: @(accept) from inside a @(collect) was found not toKaz Kylheku2013-11-211-6/+6
| | | | | | | | | | | | | propagate bindings. The culprit? The bindings_coll variable in the v_collect function being indeterminate by the well-documented and understood action of setjmp. Marking the bindings_coll variable volatile instantly fixed it. I reviewed this code to find any other instance of this oversight. * match.c (v_skip, v_collect): Mark some local variable volatile: precisely those which are used after possibly returning via an unwind, and which might have been modified since setting up the unwind block.
* Remove junk character from error message.Kaz Kylheku2013-10-281-1/+1
|
* Ouch! Turns out the code base has numerous unintendedKaz Kylheku2013-10-241-12/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deviations from C90, like mixed declations and statements. GCC doesn't diagnose these without the --pedantic flag. * configure: GCC's --ansi flag should be spelled -ansi. * lib.c (split_str, obj_print): Reorder declaration before statements. (make_sym): Fix similar problem by eliminating a statement. (funcall1, funcall2, funcall3, funcall4): Use assignment to initialize local array with non-constant elements. This is actually good for performance because we only initialize those parts of the array that we use. * lib.h (struct func): Change functype member to unsigned, since enum-typed bitfields are a GCC extension. * match.c (ml_all, mf_all): Use assignments to initialize local struct with non-constants. (do_txeval, v_collect): Slightly revise unwinding macrology with help of new macros to avoid mixing declarations and statements. (spec_bind): Removed spurious semicolon from macro expansion. (v_gather): Reorder two lines to avoid mixed decls and statements. (match_filter): Move declaration of ret a few lines up, ahead of statements. * unwind.c (uw_pop_until): New function. * unwind.h (uw_pop_until): Declared. (uw_mark_frame, uw_fast_return): New macros.
* Task #11433. Implement continuation of multipleKaz Kylheku2013-10-111-2/+67
| | | | | | | | | | | | | | | | output blocks across the same stream. * match.c (close_s, named_k, continue_k, finish_k): New symbol variables. (v_output): Implement :named, :finish and :continue. (v_close): New static function. (syms_init): New symbols interned. (dir_tables_init): New entry associating v_close function with symbol stored in close_s. * match.h (close_s): Declared. * txr.1: New features documented.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-10 10:39:14 +0000