summaryrefslogtreecommitdiffstats
path: root/parser.c
Commit message (Collapse)AuthorAgeFilesLines
...
* Set the tempfile suffix.Kaz Kylheku2015-09-181-0/+1
| | | | | * parser.c (repl): Configure the .tl suffix in the linenoise object.
* Improved ~/.txr_profile checks, with security.Kaz Kylheku2015-09-181-6/+15
| | | | | | | | | | * parser.c (load_rcfile): Use path-exists-p for the existence check. Since that doesn't throw, it's outside of the catch section. Use path-private-to-me-p to impose a security check on the profile file. If an error exception is caught, show the details. * txr.1: Added notes about security check.
* Change how error locations are reported in REPL.Kaz Kylheku2015-09-171-2/+2
| | | | | | | | | | | | | | Now that we have a multi-line mode, it no longer works that we use the command line number as the parse line number, so that an error from something that was parsed in command 42 comes out as "line:42". This commit changes to the format "expr-42:3": line 3 of expression 42. The expression number is fixed in the parse name, and the line number is derived from the line divisions in the entry itself. * parser.c (repl): Dynamically compute the parse name, such that it contains the command number. Then when parsing, just leave the parse to begin at line 1.
* Whitespace fix.Kaz Kylheku2015-09-171-1/+1
| | | | * parser.c (load_rcfile): Trailing whitespace after declaration.
* Listener configuration variables.Kaz Kylheku2015-09-171-0/+12
| | | | | | | | | | | | | | | | | | * linenoise/linenoise.c (lino_get_multiline): New function. * linenoise/linenoise.h (lino_get_multiline): Declared. * parser.c (listener_hist_len, listener_multi_line_p_s): New symbol global variables. (repl): Set linenoise history length and multi-line mode from the *listener-hist-len* and *listener-multi-line-p* variables on each call. Set the *listener-multi-line* variable from the lino_t object's current state after each linenoise call. (parse_init): Initialize new global variables and register them as special variables. * txr.1: Update sentence which says that history is fixed at 100 lines. Document listener configuration variables.
* Implement ~/.txr_profile mechanism.Kaz Kylheku2015-09-131-0/+34
| | | | | | | | | | | | * parser.c (load_rcfile): New function. (repl): Compute profile file name and try to load it before entering loop. * sysif.c (statp): Linkage becomes external. * sysif.h (statp): Declaration updated. * txr.1: Documented.
* Don't supply trivial completion which matches prefix.Kaz Kylheku2015-09-131-0/+3
| | | | | | | * parser.c (find_matching_syms): Don't add an entry into the history if it is a full match for the completion prefix, because in the UI in then looks as if nothing happened when Tab as pressed.
* Treat comment lines in repl, and plug memory leak.Kaz Kylheku2015-09-111-2/+14
| | | | | | | | | | * parser.c (repl): If we continue the loop due to detecting a blank line, we must free that line, since we are outside of the unwind block (and continue would violate that block if we were). If a line contains nothing but a comment, then enter that line into the history. * txr.1: Documented treatment of commented lines.
* New :read command in repl for direct stdin parse.Kaz Kylheku2015-09-111-1/+23
| | | | | | | | * parser.c (read_eval_ret_last): New function. (repl): Use read_eval_ret_last for :read command. * txr.1: Document :read in new section on direct read from the terminal.
* Better EOF check in read-eval stream function.Kaz Kylheku2015-09-111-1/+10
| | | | | | | | | | | | | | | | | | | | | This makes a difference if we read from a TTY. Without this fix, two Ctrl-D's have to be issued before the code registers EOF and returns. The trick is that we know, after calling the parser, whether it hit EOF, because we have the recent token. If its value is zero, that's the EOF token. We can bail out of the loop instead of calling the parser again which will require another EOF (because prime_parser will not push an EOF token!) * parser.h (parser_eof): Declared. * parser.c (read_eval_stream): After lisp_parse and eval, check for EOF with the parser_eof function, and bail if so. (parser_eof): New function.
* Quit if input form is :quit not its value.Kaz Kylheku2015-09-111-2/+2
| | | | | | | | * parser.c (repl): Subtle difference. Test form's syntax for equality to :quit, not its value. The REPL should not terminate if a form happens to calculate :quit as a value. * txr.1: Updated.
* Don't pass char to <ctype.h> function.Kaz Kylheku2015-09-101-2/+4
| | | | | | | * parser.c (provide_completions): Cast char argument to unsigned char in call to isalnum. The data comes from user input, which could be any byte. The char type may be signed (and usually is) and so may take on negative values.
* Completion doesn't iterate over all packages.Kaz Kylheku2015-09-101-10/+3
| | | | | | | | | | | When the completed symbol is unqualified, let's not complete across all packages, but just the user package. * parser.c (provide_completions): Substitute user_package if package is null, and call find_matching_syms once rather than iterating on all packages. Also, in this case, do not request symbol qualification.
* No completion on symbol in nonexistent package.Kaz Kylheku2015-09-101-0/+2
| | | | | * parser.c (provide_completions): If find_package returns nil, then just bail with no completions.
* C++: need conversions in return value from alloca.Kaz Kylheku2015-09-091-2/+2
| | | | | * parser.c (provide_completions): Use convert macro to cast void * return value of alloca to char *.
* C++: avoid the compl identifier.Kaz Kylheku2015-09-091-5/+5
| | | | | | | * parser.c (find_matching_syms): Rename the compl variable to comple, because compl is a standard C++ macro which provides an alternate name for the ~ operator.
* alloca needs to be declared.Kaz Kylheku2015-09-091-0/+1
| | | | * parser.c: #include ALLOCA_H.
* Refinement in linenoise eof/intr handling.Kaz Kylheku2015-09-081-3/+10
| | | | | | | | * parser.c (repl): Rather than breaking out of the loop when linenoise indicates lino_eof, break out of the loop for any error not equal to lino_intr. If the termination was due to other than lino_eof, print an error message.
* Bugfix: underrun on empty line tab completion.Kaz Kylheku2015-09-081-1/+4
| | | | | | | * parser.c (provide_completions): If strlen(data) is zero, the calculation of ptr places it at data - 1. Instead, let us calculate a zero value for ptr and then bail based on this.
* Load and save repl history.Kaz Kylheku2015-09-071-0/+14
| | | | | | | | | | * parser.c (repl): If a HOME environment variable exists, then load the .txr_history file from the user's home directory and save it when exiting. * sysif.c (getenv_wrap): Static function changed to extern. * sysif.h (getenv_wrap): Declared.
* Print exceptions with proper indenting.Kaz Kylheku2015-09-071-9/+7
| | | | | * parser.c (repl): Use format to print exception, and use ~! to ensure indentation.
* Better message for caught exceptions.Kaz Kylheku2015-09-071-1/+1
| | | | | * parser.c (repl): Use ** prefix on caught exception message. Clearly indicate that the second value is args.
* Store exception in repl variable.Kaz Kylheku2015-09-071-2/+3
| | | | | | | * parser.c (repl): If an exception is caught, don't store nil in the repl result variable and hash. Rather, store a cons cell holding the exception symbol and arguments.
* Reset flex scanner if exception thrown during read.Kaz Kylheku2015-09-061-1/+22
| | | | | | | | | | | | | | We must tear down and rebuild the Flex scanner, otherwise it is left in a bad state causing it to abort. * parser.c (parser_cleanup): After destroying scanner, null out the pointer. (parser_reset): Destroy the Flex scanner, and instantiate a new one in its place. (lisp_parse): Set up an unwind catch which will reset the parser if it was aborted by an exception. * parser.h (parser_reset): Declared.
* Repl will print something for syntax error exceptions.Kaz Kylheku2015-09-061-1/+1
| | | | | | * parser.c (repl): Print a simple fixed message when a syntax error exception is caught, just so the fact is noted.
* Allow evaluation from repl to be interrupted.Kaz Kylheku2015-09-061-0/+7
| | | | | | | | | | | * eval.c (do_eval): Check for pending signals. * parser.c (repl_intr): New static function. (repl): Set up signal handler for SIGINT around REPL. * signal.h (sig_deferred): declared. (sig_check_fast): New inline function/macro. A bit of a rearrangement here.
* Distinguish interrupt indication from linenoise.Kaz Kylheku2015-09-061-2/+6
| | | | | | * parser.c (repl): If linenoise returns a null pointer, but the last error is lino_eof, print "** intr" message and continue; do not quit.
* Distinguish exceptions from results in repl.Kaz Kylheku2015-09-061-0/+1
| | | | | * parser.c (repl): Print "** " sequence in front of exception.
* Erroneous lines must go into history also.Kaz Kylheku2015-09-061-4/+8
| | | | | | * parser.c (repl): If an exception is thrown, we record a nil value for the line backreference variable, and add the line to history.
* repl suppresses syntax error exceptionsKaz Kylheku2015-09-061-1/+3
| | | | | | | | | * parser.c (repl): If an expression is caught of type syntax-error, then just suppress it. It is not informative in this situation because the syntax error diagnostics have appeared on the tty. A syntax error exceptions is thrown out of the reader so that program logic doesn't ignore errors.
* Parenthesis sensitivity for completion.Kaz Kylheku2015-09-061-4/+23
| | | | | | | | | | | | | * eval.c (boundp): Static function becomes extern. * eval.h (boundp): Declared. * parser.c (find_matching_syms): New par parameter lets function determine whether previous character is a an open parenthesis or brace, based on which the set of possible completions is restricted. (provide_completions): Calculate the par parameter and pass to find_matching_syms.
* linenoise completion for txr symbolsKaz Kylheku2015-09-051-0/+114
| | | | | | | | | | | | | | | Implement basic tab completion. * lib.c (package_alist, package_name, package_symbols): New functions. * lib.h (package_alist, package_name, package_symbols): Declared. * parser.c (find_matching_syms, provide_completions): New static functions. (repl): Register provide_completions as completion callback with linenoise.
* Syntax errors refer to REPL line number.Kaz Kylheku2015-09-051-3/+9
| | | | | | | | | | | | | | | | | | * eval.c (eval_init): Registrations of lisp-parse and read must account for new optional argument. * lib.c (func_n5o): New function. * lib.h (func_n5o): Declared. * parser.c (lisp_parse): New argument for passing in line number. This is punched into the parser object. (read_eval_stream): Call to lisp_parse defaults new argument. (repl): Pass repl line number to lisp_parse. * parser.h (lisp_parse): Declaration updated. * txr.c (txr_main): Call to lisp_parse defaults new argument.
* Numbered prompt with backreferencing.Kaz Kylheku2015-09-051-1/+15
| | | | | | | * parser.c (repl): Implement counter which enumerates REPL lines. Implement *<num> variabls which allow the last 100 results to be referenced, as well as *n, *v, and *r variables to have more flexible access to this value history.
* Basic REPL based on linenoise.Kaz Kylheku2015-09-051-0/+82
| | | | | | | | | | | | | | | | * Makefile (OBJS): Only include linenoise.o if have_termios is y. * configure: Adding test for termios. (have_termios): New configure and config.make variable. (gen_config_make): Generate have_termios variable. * parser.c (repl): New function. * parser.h (repl): Declared. * txr.c (help): Summarize new -i option. (txr_main): Implement -i repl.
* Remove useless member from parser structure.Kaz Kylheku2015-08-191-2/+3
| | | | | | | * parser.c (parser_common_init): Use local variable to capture output of yylex_init, rather than yyscan structure member. * parser.h (struct parser): Member yyscan removed.
* Fix parser bug caused by wrong kind of weak hash table.Kaz Kylheku2015-08-191-1/+1
| | | | | | | | | | | | | | This bug causes the parser associated with a stream to suddenly disappear while reading forms from the stream. Parsing continues, but with a new parser which does not carry the lookahead token from the previous parse. So for instance, we miss the opening parenthesis of the next form. * parser.c (parse_init): The stream_parser_hash must be a hash table with weak keys, but not weak values. We want the association to go away only if the stream becomes unreachable, not if the parser becomes unreachable while the stream is still reachable.
* Provide conservative marking interface.Kaz Kylheku2015-08-191-4/+1
| | | | | | | | | | | | | | | | | | | GC now exports a function for marking something that might not be an object, instead of a function for testing. The previous way wasn't integrated with Valgrind properly, and didn't observe the FREE flag. * gc.c (gc_is_heap_obj): Function removed. (mark_obj_maybe): New static function, with body consisting of code moved from mark_mem_region. (mark_mem_region): Moved code replaced by call to mark_obj_maybe. (gc_conservative_mark): New function, wraps mark_obj_maybe. * gc.h (gc_conservative_mark): Declared. (gc_is_heap_obj): Declaration removed. * parser.c (yy_tok_mark): Use gc_conservative_mark instead of gc_is_heap_obj check and gc_mark.
* Improvement in hash bang code.Kaz Kylheku2015-08-121-6/+3
| | | | | * parser.c (read_eval_stream): Simplify hash bang code and avoid creating a string that might not end up being used.
* Use new pushback token priming for single regex parse.Kaz Kylheku2015-08-121-9/+21
| | | | | | | | | | | | | | | | | | | | | | | * parser.h (enum prime_parser): New enum. (prime_parser, prime_scanner, parse): Declarations updated with new argument. * parser.c (prime_parser): New argument of enum prime_parser type Select appropriate secret token for regex and Lisp case. Pass prime selector down to prime_scanner. (regex_parse): Do not prepend secret escape to string. Do not use parse_once function; instead do the parser init and cleanup here and use the parse function. (lisp_parse): Pass new argument to parse, configuring the parser to be primed for Lisp parsing. * parser.l (grammar): Rule producing SECRET_ESCAPE_R removed. (prime_scanner): New argument. Pop the scanner state down to INITIAL. Then unconditionally switch to appopriate state based on priming configuration. * parser.y (parse): New argument for priming selection, passed down to prime parser.
* Crafting a better parser-priming hack.Kaz Kylheku2015-08-121-16/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The method of inserting a character sequence which generates a SECRET_TOKEN_E token is being replaced with a purely token based method. Because we don't manipulate the input stream, the lexer is not involved. We don't have to flush its state and deal with the carry-over of the yy_hold_char. This comes about because recent changes expose a weakness in the old scheme. Now that a top-level expression can have the form expr.expr, it means that the Yacc parser reads one token ahead, to see whether there is a dot or something else. This lookahead token is discarded. We must re-create it when we call yyparse again. This re-creation is done by creating a custom yylex function, which can maintain pushback tokens. We can prime this array of pushback tokens to generate the SECRET_TOKEN_E, as well as to re-inject the lookahead symbol that was thrown away by the previous yyparse. To know which lookahead symbol to re-inject is simple: the scanner just keeps a copy of the most recent token that it returns to the parser. When the parser returns, that token must be the lookahead one. The tokens we keep now in the parser structure are subject to garbage collection, and so we must mark them. Since the YYSTYPE union has no type field, a new API is opened up into the garbage collector to help implement a conservative GC technique. * gc.c (gc_is_heap_obj): New function. * gc.h (gc_is_heap_obj): Declared. * match.c: Include y.tab.h. This is now needed by any module that needs to instantiate a parser_t structure, because members of type YYSTYPE occur in the structure. (parser.h can still be included without y.tab.h, but only an incomplete declaration for the parser strucure is then given, and a few functions are not declared.) * parser.c (yy_tok_mark): New static function. (parser_mark): Mark the recent token and the pushback tokens. (parser_common_init): Initialize the recent token, the pushback tokens, and the pushback stack index. (pushback_token): New static function. (prime_parser): hold_byte argument removed. Body considerably simplified. The catenated stream trick is no longer required. All we do here is set up two pushback tokens and prime the scanner, if necessary, so it is in the right start state for Lisp. * parser.l (YY_DECL): Take over definition of scanning function, renaming to yylex_impl, so we can implement yylex. (grammar): Rule which produces SECRET_ESCAPE_E token removed. (reset_scanner): Function removed. (yylex): New function. * parser.h (struct parser): Now only forward-declared unless y.tab.h has been included. New members, recent_tok, tok_pushback and tok_idx. (yyset_hold_char): Declared. (reset_scanner): Declaration removed. (yylex): Declared (if y.tab.h included). (prime_parser): Declaration updated. (prime_scanner): Declared. * Makefile: express new dependency on existence of y.tab.h of txr.o, match.o and parser.o.
* Diagnose bad consing dot syntax like (a . b . c).Kaz Kylheku2015-08-101-1/+3
| | | | | | | | | | | | | | | | | | | | | * parser.y (r_exprs): Use unique object in the terminating cons to indicate the empty spot where the dotted cdr item will go. Check for misplaced consing dot. (misplaced_consing_dot_check): New static function. Checks for the terminator atom spot being taken already. Thus, the spot may be taken only by the very last reduction, such that the next reduction is r_exprs -> n_exprs where the terminating atom is processed. * parser.c (unique_s): New global variable. (parse_init): Initialize unique_s. * parser.h (unique_s): Declared. * share/txr/stdlib/place.tl (sys:placelet-1): We have a misplaced consing dot here! It was working correctly by "terminating atom propagation" behavior, which allowed (a . b c d) to produce (a c d . b). If a single terminating atom occurred in the middle of a list, it was promoted to the end.
* * parser.c (open_txr_file): Bugfix: the name of the parsedKaz Kylheku2015-07-131-1/+1
| | | | | stream should be the resolved name, not the abstract original, so that error messages correlate to the file.
* Handle setting of parse name through prime_parser.Kaz Kylheku2015-07-101-3/+4
| | | | | | | | | | | | * parser.c (prime_parser): Take name as argument, and install it into parser. (lisp_parser): Pass name to parse, instead of setting it in the parser object. * parser.y (parse): Take name as argument and pass down to prime_parser. * parser.h (prime_parser, parse): Declarations updated.
* Fix wrong name reported for errors in first top-level form.Kaz Kylheku2015-07-101-1/+2
| | | | | * parser.c (read_eval_stream): Get name of original stream, and pass that down to lisp_parse as the name argument.
* Fix off by one error line numbers for .tl files.Kaz Kylheku2015-07-101-3/+7
| | | | | * parser.c (read_eval_stream): Add terminating newline to the first line that was examined for hash bang.
* Remove unused "primer" member from parser_t.Kaz Kylheku2015-07-101-7/+4
| | | | | | | | | | | * parser.c (parser_mark, parser_common_init): Remove reference to primer. (parser): Don't take primer argument, remove reference to member. (ensure_parser): Don't take primer argument, don't pass to parser function. (lisp_parse): Don't pass primer string to ensure_parser. * parser.h (primer_t): Remove primer member. (parser): Declaration updated.
* Bugfix: lexer loses unmatched "hold char" between top-level forms.Kaz Kylheku2015-07-101-6/+11
| | | | | | | | | | | | | | | | | | Test case: file containing 4(prinl 3). Scanner consumes 4 and (. The ( is lost when the scanner is reset for the next call to yyparse, resulting in jut prinl being read and interpreted as a variable. * parser.c (prime_parser): If present, append hold byte to priming string. Takes parser_t * instead of parser, and returns void now. * parser.l (reset_scanner): Now returns int value, the value of the scanner's yy_hold_char variable which is nonzero when the scanner is hanging on to an unmatched byte of input. * parser.h (reset_scanner, prime_parser): Declarations updated. * parser.y (parse): Pass hold byte returned by reset_scanner to prime_parser.
* Parser cleanup: embed scanner in parser.Kaz Kylheku2015-07-091-6/+26
| | | | | | | | | | | | | | | | | | | | | | | * parser.c (parser_destroy): New GC finalizer static function. (parser_ops): Register parser_destroy. (parser_common_init): New function, shared by parse and parse_once. Initializes embedded scanner. (parser_cleanup): New function, shared by parse_once and parser_destroy. (parser): Use parser_common_init. * parser.h (parser_t): New member, yyscan. (reset_scanner, parser_common_init): Declared. * parser.l (reset_scanner): New function. * parser.y (parse_once): Use parser_common_init, and thus perform only a few initializations. Do not define scanner as a local variable. (parse): Call reset_scanner instead of yylex_init since the scanner is being reused, and for the same reason do not call yylex_destroy. GC will do that now.
* Hash-bang support for .tl files.Kaz Kylheku2015-07-021-1/+10
| | | | | | | | | | | | | | | | * parser.c (read_eval_stream): New boolean argument to request hash bang support. * parser.h (read_eval_stream): Declaration updated. * eval.c (sys_load): Pass new thid argument to read_eval_stream, to decline hash bang support. * match.c (v_load): Likewise. * txr.c (txr_main): Request hash bang support from read_eval_stream. Thus files referenced from the txr command line can have a #! line, which is ignored.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-19 23:23:55 +0000