| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
caused test failure. An empty list not treated as a valid collect
variable list.
|
|
|
|
|
|
| |
(match_line): keyword argument :vars implemented for coll.
* txr.1: Documented :vars.
|
|
|
|
|
| |
(match_files): Implemented :vars in collect.
(match_init): New symbol variable initialized.
|
|
|
|
|
| |
last item on the line, it must match the whole line by returning
success.
|
|
|
|
|
|
|
|
|
| |
(match_line): Implemented :mintimes and :maxtimes, changing
the semantics of :times.
(match_files): Likewise.
(match_init): New keyword variables initialized.
* txr.1: Updated.
|
|
|
|
|
|
|
|
| |
dating back to before October 2009 when txr was put into git.
Basically, unbound variables were not handled right after
the function return, due to the increment step being wrongly
written as ``piter = cdr(aiter)'' in the for loop that processes the
ub_p_a_pairs. Evil cut and paste!
|
|
|
|
|
|
|
|
| |
(match_line): Greedy skip implemented.
(match_files): Likewise.
(match_init): New keyword symbol variable initialized.
* txr.1: Updated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(obj_init): New variable initialized.
* lib.h (eol_s): Declared.
* match.c (match_line): Implemented horizontal skip as and
new eol directive.
(match_lines): Vertical skip defers to horizontal skip if
there is trailing material.
* txr.1: Updated.
* lib.c (eol_s): New symbol variable.
(obj_init): New variable initialized.
* lib.h (eol_s): Declared.
* match.c (match_line): Implemented horizontal skip as and
new eol directive.
(match_lines): Vertical skip defers to horizontal skip if
there is trailing material.
* txr.1: Updated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nested lists. This is in anticipation of future features.
* lib.c (expr_s): New symbol variable.
(obj_init): expr_s initialized.
* lib.h (expr_s): Declared.
* match.c (dest_bind): Now takes linenum. Tests for the meta-syntax
denoted by the system symbols var_s and expr_s, and throws an
error.
(eval_form): Similar error checks added. Also, hack: do not add
file and line number to an exception which begins with a '('
character; just re-throw it. This suppresses duplicate line
number addition when this throw occurs across some nestings.
(match_files): Updated calls to dest_bind.
* parser.l (yybadtoken): Handle new token kind, METAVAR and METAPAR.
(grammar): Refactoring among patterns: TOK broken into
SYM and NUM, NTOK introduced, unused NUM_END removed.
Rule for @( producing METAPAR in nested state.
* parser.y (METAVAR, METAPAR): New tokens.
(meta_expr): New nonterminal.
(expr): meta_expr and META_VAR productions handled.
|
|
|
|
|
|
|
|
|
|
| |
* lib.c (bind2): Function renamed to curry_12_2.
(bind2other): Function renamed to curry_12_1.
(do_bind_2, do_bind2other): Helpers renamed likewise.
(tree_find): Follows rename of bind2.
* match.c (match_files): deffilter code follows bind2 rename
to curry_12_2.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(do_curry_123_2): New static function.
* lib.h (funcall3, curry_123_2): Declared.
* match.c (subst_vars): Bugfix: throw error on unbound variable instead
of ignoring the situation. This bug caused unbound variables in
quasiliterals to be silently ignored.
(eval_form): Function changed to three argument form, so that
it takes a line number for reporting errors. Restructured to catch
the new unbound variable exception from subst_vars, and re-throw
it with a line number. Also, throws exception now instead of returning
nil if itself it detets an unbound variable. Uses of eval_form
no longer have to test the return value for nil, but just assume
it worked.
(match_lines): Currying calls to eval form updated to use
curry_123_2. Test of eval return value eliminated. In function
calls, eval isn't used for reducing symbol arguments to values,
because it now throws in the unbound case, and it's not worth
setting up a catch for this. Instead, assoc is used directly.
|
|
|
|
|
| |
the unbound variable from the argument list can be done
with a destructive operation since that list is a copy.
|
|
|
|
|
|
| |
hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y,
regex.c, regex.h, stream.c, stream.h, txr.1, txr.c, txr.h, unwind.c,
unwind.h, utf8.c, utf8.h: Updated e-mail address.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
are handled in collect/coll. New bindings from the main clause and
last clause must override old bindings. This is done by some
additional set difference operations based on symbol identity.
Otherwise it is possible to end up with multiple bindings for the
same symbol, which is untidy. If the collect clause scrubs a variable
with forget and re-binds it, then combining that environment
with the previous bindings will create a duplicate.
Also, fixed a serious bug with the bindings from the last clause;
the append was wrongly put into the loop that processes the collected
lists.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(set_diff): Optimize common case: list1 and list2
are the same, or list2 is substructure of list1.
Situations in which this won't be the case for variable bindings are
rare.
* lib.h (acons): Declared.
* match.c (match_line): Use acons rather than acons_new, when binding
variables that we know are new (the symbol is unbound).
When computing the set difference over bindings, use cons cell
equality, rather than symbol equality. Symbol equality is wrong
because a binding can be removed, and then a new binding can be
introduced using the same symbol. This must be treated as
a different binding.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
were broken in the face of deletions (local, forget).
For some stupid reason, I had written a destructive routine for
removing elements from an association list, and used it
as the basis for the local and forget directives.
* lib.c (eq_f, car_f): New variables.
(identity_tramp, equal_tramp): Obsolete functions removed.
(apply): Broken function disabled at run time.
(funcall, funcall1, funcall2): Throw meaningful error instead
of aborting.
(alist_remove_test): New static function.
(alist_remove, alist_remove1): Rewritten to be functional
rather than destructive.
(alist_nremove, alist_nremove1): Destructive functions,
using previous implementations of alist and alist_nremove.
(do_sort): Recurses directly rather than via sort. That was
probably why this helper was introduced!
(find, set_diff): New functions.
(obj_init): gc-protect new variables eq_f and car_f, and initialize
them. Initializations for equal_f and identity_f changed to
use equal and identity directly, without the obsolete wrappers.
* lib.h (eq_f, car_f, alist_nremove, alist_nremove1,
find, set_diff): Declared.
* match.c (match_line): Use set_diff to determine what bindings
are new, rather than ldiff and ldiff-like logic which break when
the new bindings do not share structure with the old.
(match_files): Likewise.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bugfix in cases inside coll: was not collecting bindings.
Bugfix for until inside coll: was not seeing bindings
from main clause.
* lib.c (ldiff): New function.
* lib.h (ldiff): Declared.
* match.c (match_line): Implemented last clause. Fixed cases
handling by moving misplaced termination check.
(match_files): Implemented last clause.
* parser.y (until_last): New nonterminal symbol.
(collect_clause): Refactored syntax to support until and last.
(elem): Likewise.
* txr.1: Updated.
|
|
|
|
|
|
| |
(match_files): Remove check against trailer_s not having trailing
material. If it doesn't, it's a vertical directive processed here,
otherwise leave it alone so match_line processed it.
|
|
|
|
|
|
|
|
|
|
|
| |
* lib.h (cons_set): New macro.
* match.c (match_line, match_files): In collect clause handlers,
move variable declarations above goto, and initialize with
cons_set, instead of declaring and initializing with cons_bind.
This eliminates the stupid C++ error that goto skips a variable
initialization (which happens even when it can be trivially
proven that the has no next use at the goto site!)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
need another commit).
* filter.c: Include "gc.h" for prototype of protect.
(struct filter_pair): Use const wchar_t *, so we can assign
literals.
(html_hex_continue): Ditto.
* lib.c (and): Function renamed to andf, since and is a C++
operator.
* lib.h (and): Declaration renamed.
* match.c (match_files): Use of and updated to andf.
|
|
|
|
|
|
|
|
| |
2011-09-28 commit which introduced the double var match.
* match.c (match_line): Handle case where modifier is t.
* parser.y (var_op): Produce modifir as (t) rather than t.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* match.c (choose_s, longest_k, shortest_k): New variables.
(match_line, match_files): Introduced choose directive.
(match_init): Initialize new variables.
* match.h (choose_s): Declared.
* parser.l (yybadtoken): Handle CHOOSE.
(CHOOSE): Clause added for returning this token.
* parser.y: Added #include "match.h".
(CHOOSE): New token symbol.
(choose_clause): New nonterminal symbol.
(clause): choose_clause added.
(all_clause, some_clause, none_clause, maybe_clause,
cases_clause): Abstract syntax tree tweaked.
(choose_clause): New syntax.
(elem): Abstract syntax trees tweaked for many clauses.
New CHOOSE clauses.
(out_clause): New error case for choose_clause.
|
|
|
|
|
|
|
|
|
|
| |
none, maybe and cases directives.
(match_files): Recognize horizontal version of these directives
by the presence of the extra symbol t and do not process.
Also, bugfix in the all directive: not resetting the
all_match flag when short circuiting out.
* parser.y (clause_parts_h, additional_parts_h): New nonterminals.
(elem): New clauses added.
|
|
|
|
|
|
|
|
|
|
|
|
| |
(match_line): Keyword arguments in coll implemented.
(match_init): chars_k variable initialized.
* parser.l (COLL): Lexical syntax changed to allow for
argument material.
* parser.y (elem): Coll syntax rewritten for arguments.
* txr.1: Updated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
symbol variables.
(match_lines): Keyword arguments in collect implemented.
(match_init): New function.
* match.h (match_init): Declared.
* parser.l (COLLECT): Lexical syntax changed for COLLECT to
allow for argument material.
* parser.y (%union): obj renamed to val.
(exprs_opt): New nonterminal.
(collect_clause): Rewritten for arguments.
* txr.c (main): Call to match_init introduced.
|
|
|
|
| |
prepend the next_pat to the specline if it is nil.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
regex variables which also have nested variables.
Previously this code was assuming that the cases were
mutually exclusive, and the parser happened to work that way.
Also, added support for a "double var" match which occurs
when an unbound variable is followed by a regex variable.
This case should be allowed because it makes sense.
It's similar to a variable followed by a regex, except
that the regex is also a variable binding.
* parser.y (o_elems_transform): New function.
(o_elems_opt, o_elems_opt2, quasilit): Transform o_elems with new
function. This is needed because subst_vars doesn't
deal with the nested var syntax for consecutive variables.
(var): New syntax case '{' IDENT exprs '}' elem. This
allows consecutive variables to be nested in all cases.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The test for !data should be done after matching,
before incrementing to the next line. Then it is a true
bottom of the loop test. This commit allows
@(skip)
@first_line
@(skip nil 3)
@(eof)
to correctly match the first line of the input, not
the fourth one from the bottom, since the
second skip has an unbounded range.
|
|
|
|
|
|
|
| |
If a hard skip tries to go beyond EOF, then the query
must fail. However, a skip to exactly EOF is fine.
I.e. data can hit nil at the same time as the right
number of skip iterations is performed.
|
|
|
|
|
|
|
|
|
|
|
|
| |
We should try the match at least once even if there is no data
after a hard skip, so that the query has an opportunity
to do an explicit match for no data, as with @(endp).
This commit makes possible queries like:
@fourth_line_from_bottom
@(skip 1 3)
@(eof)
This query depends on @(skip 1 3) not failing when
it runs out of data, because @(eof) checks for htis.
|
|
|
|
|
|
|
|
|
|
|
| |
(obj_init): New variable initialized.
* lib.h (eof_s): Declared.
* match.c (match_files): New @(eof) directive explicitly
matches end of data.
* txr.1: Updated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
chr function inlined.
* filter.c (trie_value_at, trie_lookup_feed_char): Handle function
case.
(build_filter): New parameter, compress_p.
(html_hex_continue, html_hex_handler): New functions.
(filter_init): Add a function-based node to the from_html trie.
* lib.c (chr): Function removed.
(functionp) New function.
* lib.h (chr): Declaration replaced with inline function.
(functionp): Declared.
* match.c (format_field): Bugfix: failed to apply filter
that came in as an argument.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bugfix in @(throw) when non-symbol is thrown: exception message
referred to the symbol throw rather than the erroneous object.
* filter.c (build_filter_from_list, register_filter): New functions.
* filter.h (register_filter): New function declared.
* lib.c (deffilter_s): New variable defined.
(chain): Function changed from single list argument to variable
argument list to reduce the complexity of use.
(do_and, and): New functions.
(obj_init): deffilter_s initializatio added.
* lib.h (deffilter_s, and): New declarations.
(chain): Declaration updated to new function signature.
(eq): Changed from macro to inline function.
* match.c (do_output_line): Simplified expression involving chain.
(do_output): Likewise.
(match_files): Bugfix in error handling of throw.
Implementation of deffilter.
* txr.1: Documented deffilter.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* filter.c, filter.h: New files.
* Makefile (OBJS): filter.o added.
* gc.c (mark_obj): Mark new alloc field of string objets.
* hash.c (struct hash): New member, userdata.
(hash_mark): Mark new userdata member of hash.
(make_hash): Initialize userdata.
(get_hash_userdata, set_hash_userdata, hashp): New functions.
* hash.h (get_hash_userdata, set_hash_userdata, hashp): New functions
declared.
* lib.c (getplist, string_extend, cobjp): New functions.
(string_own, string, string_utf8): Initialize new alloc field to nil.
(mkstring, mkustring): Initialize new alloc field to actual size.
(length_str): When length is computed and cached, also compute
and cache alloc.
(init): Call filter_init.
* lib.h (string string): New member, alloc.
(num_fast): Macro converted to inline function.
(getplist, string_extend, cobjp): New functions declared.
* match.c (match_line): Follows change of modifier s-exp syntax.
(format_field): New parameter, filter.
New modifier syntax parsed. Filter retrieved, and applied.
(subst_vars): New parameter, filter. Filter is either applied
in this function or passed to format_field, as needed.
(eval_form): Pass nil to new parameter of subst_vars.
(do_output_line): New parameter, filter. Passed down to subst_vars.
(do_output): New parameter, filter. Passed down to do_output_line.
(match_files): Pass nil filter to subst_vars in cat directive.
Output directive refactored to parse keywords, extract the
filter and pass down to do_output.
* parser.y (regex): Generate (sys:regex regex syntax ...)
instead of (regex syntax ...).
(elem, expr): Updated w.r.t. regex syntax change.
(var): Cases '{' IDENT regex '}' and '{' IDENT NUMBER '}'
are removed. new syntax '{' IDENT exprs '}' to handle these
more generally and allow for keywords.
* txr.1: Updated.
|
|
|
|
|
|
| |
lib.h, match.c, match.h, parser.h, parser.l, parser.y, regex.c,
regex.h, stream.c, stream.h, txr.1, txr.c, txr.h, unwind.c, unwind.h,
utf8.c, utf8.h: Updated copyright year.
|
|
|
|
|
|
| |
Leading :nothrow with trailing material is an error now.
* txr.1: Updated. Made note of errors in pipes being asynchronous.
|
|
|
|
| |
features. Support for obsolescent @(next) syntax is gone.
|
|
|
|
|
|
| |
* match.c (match_files): Added a couple of break statements.
* txr.1: Updated.
|
|
|
|
|
|
|
|
| |
a minimum number of lines.
* match.c (match_files): New behavior in skip_s case.
* txr.1: Documented.
|
| |
|
|
|
|
|
| |
lines to match must fail. Extra data lines relative to the spec
are tolerated; extra spec lines unmet by data aren't.
|
|
|
|
|
|
|
|
|
|
| |
If the virtual line is partially matched, the remainder of
the line is folded back into list form. In this case, the
data line number must be incremented. Otherwise the calling
context may conclude that no progress was made, and
skip a line of input. I.e. the unmatched part of the input
is a new line, even if there had originally
been no line break at that point.
|
| |
|
|
|
|
| |
in a code path where it sometimes isn't.
|
|
|
|
|
|
|
| |
discovered on Red Hat EL 4 with gcc 3.4.3.
In the collect loop, set car(success) to nil.
Somehow the generated code hangs on to the last
matching position for a regex, preventing GC.
|
|
|
|
|
|
| |
on 32 bit x86 Fedora. This happens because the lazy list variable
``data'' in the match_files function is optimized to a register,
but a stale value of that variable persists in the backing storage.
|
|
|
|
| |
variable.
|
| |
|
| |
|
|
|
|
| |
in regex module not exposed in header. Etc.
|