The lisp-parse function can now be called multiple times

on the same stream to extract multiple objects; the requirement
that the stream must hold exactly one complete Lisp object
with no following material is now lifted.

* parser.l (YY_INPUT): Modified the macro so that it reads no more
than one character. Though this probably makes the lexer less
efficient, it gives us the important property that the lexer does
not scan ahead into the input stream, hogging data into its buffer
which is then destroyed. This is essential if the lisp-parse function
is to support multiple calls to pull objects one by one out of
a stream.

* parser.y (spec): Use YYACCEPT in the SECRET_ESCAPE_E clause for
pulling a single expression out of the token stream. YYACCEPT
is a trick for not invoking the $accept : spec . $end production
which is implicitly built into the grammar, and which causes
a token of lookahead to occur.  This allows us to read a full
expression without stealing any further token: but only if the
grammar is structured right.
(exprs): This phrase structure now handles the DOTDOT syntax.
There is no such thing as an expr DOTDOT expr expression any more;
it is in the list syntax (and not supported in the dot position).
(expr): Remove DOTDOT syntax.

* txr.1: Updated description of .. syntax, and relaxed the description
of lisp-parse since it now allows multiple calls to extract
multiple objects.

1 files changed, 5 insertions, 2 deletions

diff --git a/parser.y b/parser.y
index 8e7577ee..40c231c1 100644
--- a/parser.y
+++ b/parser.y
@@ -118,7 +118,7 @@ static val parsed_spec;
 spec : clauses                  { parsed_spec = $1; }
      | /* empty */              { parsed_spec = nil; }
      | SECRET_ESCAPE_R regexpr  { parsed_spec = $2; end_of_regex(); }
-     | SECRET_ESCAPE_E expr     { parsed_spec = $2; }
+     | SECRET_ESCAPE_E expr     { parsed_spec = $2; YYACCEPT; }
      | error '\n'               { parsed_spec = nil;
                                   if (errors >= 8)
                                     YYABORT;
@@ -720,9 +720,13 @@ meta_expr : METAPAR exprs ')'   { $$ = rlcp(cons(expr_s, expand($2)), $2); }
           | METAPAR error       { $$ = nil;
                                   yybadtoken(yychar, lit("meta expression")); }
           ;
+
 exprs : expr                    { $$ = rlcp(cons($1, nil), $1); }
       | expr exprs              { $$ = rlcp(cons($1, $2), $1); }
       | expr '.' expr           { $$ = rlcp(cons($1, $3), $1); }
+      | expr DOTDOT exprs       { $$ = rlcp(cons(list(cons_s, $1,
+                                                      car($3), nao),
+                                                 cdr($3)), $1); }
       ;
 
 exprs_opt : exprs               { $$ = $1; }
@@ -743,7 +747,6 @@ expr : SYMTOK                   { $$ = rl(sym_helper($1, t), num(lineno)); }
      | chrlit                   { $$ = rl($1, num(lineno)); }
      | strlit                   { $$ = $1; }
      | quasilit                 { $$ = $1; }
-     | expr DOTDOT expr         { $$ = list(cons_s, $1, $3, nao); }
      ;
 
 regex : '/' regexpr '/'         { $$ = cons(regex_s, $2); end_of_regex();