summaryrefslogtreecommitdiffstats
path: root/parser.l
Commit message (Collapse)AuthorAgeFilesLines
* * parser.l (grammar): Do not allow unescaped newline inKaz Kylheku2015-07-231-1/+15
| | | | | | | | | | | word list literals and word list quasiliterals, except in <= 109 compatibility mode. An escaped newline in these literals, together with surrounding whitespace, now produces a single space, except in <= 109 compatibility mode. * txr.1: Documented new rules for WLL's and QLL's, and added compatibility notes.
* Bugfix: lexer loses unmatched "hold char" between top-level forms.Kaz Kylheku2015-07-101-1/+5
| | | | | | | | | | | | | | | | | | Test case: file containing 4(prinl 3). Scanner consumes 4 and (. The ( is lost when the scanner is reset for the next call to yyparse, resulting in jut prinl being read and interpreted as a variable. * parser.c (prime_parser): If present, append hold byte to priming string. Takes parser_t * instead of parser, and returns void now. * parser.l (reset_scanner): Now returns int value, the value of the scanner's yy_hold_char variable which is nonzero when the scanner is hanging on to an unmatched byte of input. * parser.h (reset_scanner, prime_parser): Declarations updated. * parser.y (parse): Pass hold byte returned by reset_scanner to prime_parser.
* Parser cleanup: embed scanner in parser.Kaz Kylheku2015-07-091-0/+8
| | | | | | | | | | | | | | | | | | | | | | | * parser.c (parser_destroy): New GC finalizer static function. (parser_ops): Register parser_destroy. (parser_common_init): New function, shared by parse and parse_once. Initializes embedded scanner. (parser_cleanup): New function, shared by parse_once and parser_destroy. (parser): Use parser_common_init. * parser.h (parser_t): New member, yyscan. (reset_scanner, parser_common_init): Declared. * parser.l (reset_scanner): New function. * parser.y (parse_once): Use parser_common_init, and thus perform only a few initializations. Do not define scanner as a local variable. (parse): Call reset_scanner instead of yylex_init since the scanner is being reused, and for the same reason do not call yylex_destroy. GC will do that now.
* Bugfix: allow @1 in brace variables.Kaz Kylheku2015-06-071-1/+1
| | | | | | | * parser.l (grammar): Scan a METANUM token in the BRACED state also. This allows us to correctly reference op arguments in a quasiliteral, as in `foo @{@1 [1..2] ","} bar`.
* Support trailing semicolon after hex/octal characters.Kaz Kylheku2015-07-021-2/+9
| | | | | | | | | | | | | | | | * parser.l (%option): Remove nounput option since we need yyunput. (grammar): Rule for matching hex and octal escape in SPECIAL state recognizes optional semicolon. In 109 compatibility, this is pushed back into the stream, otherwise consumed. * txr.1: Updated documentation, including compat notes. * genvim.txr (txr_char): Include optional semicolon in match. Corrected some errors where 8 and 9 were being included as matches for octal digits. (txr_error): Default match for \x or \o not followed by digits.
* Third round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-0/+6
| | | | | | * parser.l (char_esc): Recognize \@ escape. (grammar): Add a rule for a \@ escape in quasiliterals, and quasi word list literals.
* Second round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-1/+6
| | | | | | | | | * parser.l: Only shift to QSPECIAL state when @ is followed by a trailing context consisting of certain characters. Not every kind of Lisp object syntax can be introduced with @ in a quasiliteral. Adding a rule to produce an error when @ appears that is not followed by an allowed character.
* First round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-7/+8
| | | | | | | | | | | | | | | | | | | | | * parser.l: Do not try to recognize floating-point literals in QSPECIAL state; that is not possible because @134.3 in a quasiliteral parses as a METANUM followed by ".3". On the other hand, recognize METANUM literals in QSPECIAL state, so that @@123 scans. Recognize @ as a token in QSPECIAL state, so @@abc will scan. When transitioning from QSILIT and QWLIT states to QSPECIAL upon scanning @, return a @ token, which is now parsed in the grammar. * parser.y (quasi_meta_helper): New static function. (q_var): Do not handle SYMTOK any more, only the braced variable syntax. SYMTOK is handled as a n_expr. Braced vars are handled with explicit '@' token, which is now produced by the scanner when it shifts from QSILIT to QSPECIAL. (quasi_item): No longer necessary to recognize various forms here such as quotes and splices. Just recognize a n_expr, preceded by '@'.
* Do not allow unrecognized escapes in regex.Kaz Kylheku2015-04-191-11/+14
| | | | | | | | | | | | * parser.l (REGOP): New regex alias for matching all regex special characters. (grammar): Several rules for regex special characters merged together. New rule introduced to match a special character after a backslash, making it literal. The old rule which makes literal any character after a backslash now throws an error, unless version 105 comaptibility is selected. * txr.1: Documented this behavior change.
* Allow quasiquotes in braces and quasiliterals, and quotes in braces.Kaz Kylheku2015-04-151-6/+1
| | | | | | | | | * parser.l: Consolidate rules for recognizing quote, unquote, and quasiquote. An effect of this is that quasiquotes can now occur in braces and in string quasiliterals. * parser.y (quasi_item): Support quotes and quasiquotes as quasi items: that is to say, i.e. objects denoted by @ in a quasiliteral.
* Diagnose trailing junk in numeric literals.Kaz Kylheku2015-04-141-14/+19
| | | | | | | * parser.l: Combining the handling of hex, octal and binary numeric literals into a single rule. Implementing an additional rule which diagnoses such tokens that have trailing junk. Thus, something like #x1F2AZ is now invalid syntax.
* * parser.c (open_txr_file, regex_parse, lisp_parse): FunctionsKaz Kylheku2014-12-211-77/+0
| | | | | | | moved here from parser.l. * parser.l (open_txr_file, regex_parse, lisp_parse): Functions moved from here to parser.c.
* * Makefile (OBJS): Add parser.o.Kaz Kylheku2014-12-211-1/+1
| | | | | | | | | | * parser.h (parser_s): Declared. (parse_init): Declaration removed. (parser_l_init): Declared. * parser.l (parse_init): Function renamed to parser_l_init. * parser.c: New file.
* Improved error reporting, particularly for macro expansion.Kaz Kylheku2015-02-211-4/+2
| | | | | | | | | | | | | | | | | | | * eval.c (last_form_expanded): New variable. (do_expand): New static function; contains previous expand function. (expand): Becomes a wrapper for do_expand, with re-entry counting. (eval_init): GC-protect last_form_expanded. * eval.h (last_form_expanded): Declared. * parser.l (regex_parse, lisp_parse): Just use a simple word for the name of the regex or string parse location, not the entire expression itself. * unwind.c (uw_throw): Check whether expansion was going on when the unhandled exception was thrown and print additional information.
* * hash.c (hash_begin): Use coerce macro instead of raw C cast.Kaz Kylheku2014-10-251-1/+1
| | | | | | Use cobj_handle so hash argument is validated. * parser.l (YY_INPUT): Use convert macro instead of raw C cast.
* GNU Flex's libfl library provides nothing. Let us not refer to it. ItKaz Kylheku2014-10-241-0/+9
| | | | | | | | | | | | only causes build issues on some systems where it is not provided in the standard location, or is not cross-compiled properly. * Makefile (LEXLIB): Reference to variable removed. * configure (lexlib): Variable removed. (LEXLIB): config.make variable not generated. * parser.l (yywrap): Provide this trivial function as inline.
* * Makefile: Removing trailing spaces.Kaz Kylheku2014-10-241-2/+2
| | | | | | | | | | (GREP_CHECK): New macro. (enforce): Rewritten using GREP_CHECK, with new checks. * arith.c, combi.c, debug.c, eval.c, filter.c, gc.c, hash.c, lib.c, * lib.h, match.c, parser.l, parser.y, rand.c, regex.c, signal.c, * signal.h, stream.c, syslog.c, txr.c, unwind.c, utf8.c: Remove trailing spaces.
* * parser.l (lisp_parse): Bugfix: the error_stream argumentKaz Kylheku2014-10-191-3/+12
| | | | | | | | | must be checked to be a stream before we plant it in place of std_error, otherwise we will get a type exception thrown lower down, which leads to runaway recursion as TXR tries to print the error messages on std_error. * dep.mk: Regenerated.
* Converting cast expressions to macros that are retargettedKaz Kylheku2014-10-171-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to C++ style casts when compiling as C++. * lib.h (strip_qual, convert, coerce): New casting macros. (TAG_MASK, tag, type, wli_noex, auto_str, static_str, litptr, num_fast, chr, lit_noex, nil, nao): Use cast macros. * arith.c (mul, isqrt_fixnum, bit): Use cast macros. * configure (INT_PTR_MAX): Define using cast macro. * debug.c (debug_init): Use cast macro. * eval.c (do_eval, expand_macro, reg_op, reg_mac, eval_init): Use cast macros. * filter.c (filter_init): Use cast macro. * gc.c (more, mark_obj, in_heap, mark, sweep_one, unmark): Use cast macros. * hash.c (hash_double, equal_hash, eql_hash, hash_equal_op, hash_hash_op, hash_print_op, hash_mark, make_hash, make_similar_hash, copy_hash, gethash_c, gethash, gethash_f, gethash_n, remhash, hash_count, get_hash_userdata, set_hash_userdata, hash_iter_destroy, hash_iter_mark, hash_begin, hash_uni, hash_diff, hash_isec): Use cast macros. * lib.c (code2type, chk_malloc, chk_malloc_gc_more, chk_calloc, chk_realloc, chk_strdup, num, c_num, string, mkstring, mkustring, upcase_str, downcase_str, string_extend, sub_str, cat_str, trim_str, c_chr, vector, vec_set_length, copy_vec, sub_vec, cat_vec, cobj_print_op, obj_init): Likewise. * match.c (do_match_line, hv_trampoline, match_files, dir_tables_init): Likewise. * parser.l (grammar): Likewise. * parser.y (parse): Likewise. * rand.c (make_state, make_random_state, random_fixnum, random): Likewise. * regex.c (CHAR_SET_L2_LO, CHAR_SET_L2_HI, CHAR_SET_L1_LO, CHAR_SET_L1_HI, CHAR_SET_L0_LO, CHAR_SET_L0_HI, L0_full, L0_fill_range, L1_full, L1_fill_range, L1_contains, L1_free, L2_full, L2_fill_range, L2_contains, L2_free, L3_fill_range, L3_contains, L3_free, char_set_create, char_set_cobj_destroy, nfa_state_accept, nfa_state_empty, nfa_state_single, nfa_state_wild, nfa_state_set,
* Purge stray occurrences of "void *" from code base.Kaz Kylheku2014-10-171-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * lib.c (cobj_print_op): In the format call, cast the C pointer to val, since the ~p conversion now takes a val rather than void *. (cptr_equal_op, obj_print, obj_pprint): Remove cast to void *, since obj now already has the type that ~p expects. * lib.h (struct any): Use mem_t * instead of void *. * parser.h (yyscan_t): Repeat the definition from inside the Flex-generated lex.yy.c. (yylex_init, yylex_destroy, yyget_extra, yyset_extra): Re-declare using yyscan_t typedef in place of void *. * parser.l (yyget_column, yyerrprepf): Re-declare using yyscan_t. (grammar): Use yyg in place of yyscanner in calls to yyerrprepf. * parser.y (yylex, %lex-param): Use yyscan_t instead of void *. (parse): Use yyscan_t for local variable. * signal.c (stack): Change type from void * to mem_t *. * stream.c (vformat): Conversion specifier p extracts val instead of void *. (run): Use casts that only remove const, not all the way to void *. * txr.1: Documented p conversion specifier of format. * Makefile (OBJS-y): Initialize with := to make sure it is a simple variable, and not a macro. (SRCS): New variable, listing source files. (enforce): New rule for enforcing coding conventions. Currently checks for void * occurrences.
* More type safety, with help from C++ compiler.Kaz Kylheku2014-10-141-22/+18
| | | | | | | | | | | | | | | | | | | | | | | | | * parser.h (scanner_t): New typedef. Cleverly, we use yyguts_t as the basis for this, which pays off in a few places, getting rid of conversions. (parser_t): scanner member redeclared to scanner_t *. (yyerror, yyerr, yyerrof, end_of_regex, end_of_char): Declarations updated. * parser.l (yyerror, yyerrorf, num_esc): scanner argument becomes scanner_t * instead of void *. (yyscanner): Occurrences replaced by yyg: why should we refer to the type-unsafe void *yyscanner, when we know that in the middle of the yylex function we have the properly typed yyguts_t *yyg. (end_of_regex, end_of_char): Argument changes to scanner_t * and name strategically changes to yyg, eliminating the local variable and cast. * parser.y (sym_helper): scanner argument becomes scanner_t * instead of void *. (%parse-param}: void *scnr becomes scanner_t *scnr. (define-transform, yybadtoken): Use scanner_t * instead of void *. (parse): Since yylex_init takes a void **, we use it to initialize a local void *, and then assign it to the structure member.
* C++ upkeep.Kaz Kylheku2014-10-141-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TXR's support for compiling as C++ pays off: C++ compiler finds serious bugs introduced in August 2 ("Big switch to reentrant lexing and parsing"). The yyerror function was being misused; some of the calls reversed the scanner and parser arguments. Since one of the two parameters is void *, this reversal wasn't caught. * parser.l (yyerror): Fix first two arguments being reversed. (num_esc): Change previously correct call to yyerror to follow reversed arguments, so that it stays correct. * parser.y (%parse-param): Change order of these directives so that the scnr parameter is before the parser parameter. This causes the yacc-generated calls to yyerror to have the arguments in the correct order. It also has the effect of changing the signature of yyparse, reversing its parameters. (parse): Update call to yyparse to new argument order. * parser.h (yyparse): Declaration removed. (yyerror): Declaration updated. * regex.c (regex_kind_t): New enum typedef. (struct regex): Use regex_kind_t rather than an enum inside the struct, which has different scoping rules under C++. * txr.c (get_self_path): Fix signed/unsigned warning.
* * eval.c (eval_init): Update registration of lisp-parse and readKaz Kylheku2014-09-021-2/+8
| | | | | | | | | | | | | | | | | | | | to account for new parameter. * lib.c (syntax_error_s): New symbol_variable. (obj_init): New symbol variable initialized. * lib.h (syntax_error_s): Declared. * parser.h (lisp_parse): Declaration updated. * parser.l (lisp_parse): Takes third parameter. * txr.1: Third parameter of read described. * txr.c (txr_main): Pass colon_k to third parameter of lisp_parse to obtain exception throwing behavior. * unwind.c (uw_init): Register syntax-error as subtype of error.
* * parser.l (yyerr): Function removed; it is not used in the lexer,Kaz Kylheku2014-08-071-5/+0
| | | | | | | | | | and converted to a macro in the parser. * parser.y (define_transform): take a parser argument rather than scanner. Set up scnr local variable for yyerr macro. Remove scnr argument from macro calls. (yyerr): New macro. (grammar): Remove scnr argument from yyerr calls.
* Reentrant parser regression.Kaz Kylheku2014-08-071-10/+9
| | | | | | | | | | | * parser.y (yybadtok): New macro. (yybadtoken): Function must take parser argument. (grammar): Replace uses of yybadtoken with yybadtok. * parser.h (yybadtoken): Declaration updated. * parser.l (grammar): Fix incorrect yyprepf calls that are missing the yyscanner parameter.
* * parser.y: Back port from Berkeley Yacc to GNU Bison.Kaz Kylheku2014-08-051-1/+1
| | | | | | | | We need a prototype of yylex that is in scope of the grammar, but YYSTYPE is not defined there. * parser.l: Bison 3 declares yyparse in y.tab.h, so we have to reorder some #includes.
* Big switch to reentrant lexing and parsing.Kaz Kylheku2014-08-021-260/+262
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parser.l (YY_INPUT): Stop relying on removed yyin_stream; refer to stream via yyextra. (yyin_stream, lineno, errors, spec_file_str, prepared_error_message): Global variables removed. (yyget_column, yyset_column): Missing prototypes not generated by flex in bison bridge mode have to be added by us to avoid warning. (yyerror): Takes parser and scanner as parameters. Prepared error message is now in the parser context. Calls to other error handling functions receive scanner context. (yyerr): New function. (yyerrorf, yyerrprepf): Takes scanner argument, chases extra data to get to parser, and refers to parser variables instead of globals. (num_esc): Scanner argument added. (%option reentrant, %option bison-bridge, %option extra-type): New flex options. (grammar): yyscanner added everywhere. (end_of_char): Takes scanner argument. (parse_init): Removed references to yyin_stream and prepared_error_message. (parse_reset): Function renamed to open_txr_file. Returns results via pointers instead of setting global variables. (regex_parse, lisp_parse): Use reentrant parser interface. * parser.y (yyerror): Prototype removed. (yylex): Prototype moved after grammar, with new arguments. (sym_helper, define_transform): Take scanner argument. (make_expr): Takes parser argument. (rlrec): New static function. (rl): Function turned into macro. (mkexp, symhlpr): New macros. (%purse-parser, %parse-param, %lex-param): New Yacc options. (grammar): Actions re-worked for reentrance. Parser and scanner contexts are passed down to helper functions, in some cases via the three new macros. The result of the parse is stored in the syntax_tree member of the parser_t structure instead of a global. The yylex function receives the scanner instance. (get_spec): Function removed. (parse): New function. * parser.h (lineno, errors, yyin_stream, spec_file_str): Declarations removed. (parser_t): New struct. (yyerr): New function declared. (yyparse, yyerror, yyerrorf, end_of_regex, end_of_char, yylex, yylex_destroy): Declarations updated.
* * parser.l: Allow unquotes and splices in QSPECIAL and BRACED states.Kaz Kylheku2014-07-301-1/+6
| | | | | | | | | * parser.y (quasi_item): Support splices as items. * genvim.txr: Syntax highlighting support for unquotes in quasiliterals. * txr.vim: Updated.
* * Makefile, arith.c, arith.h, combi.c, combi.h, configure, debug.c,Kaz Kylheku2014-07-231-16/+16
| | | | | | | | debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, hash.c, hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, syslog.c, syslog.h, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Synchronize license header with LICENSE.
* * parser.l (yylex_destroy): The FLEX_ version are prefixed with YY_Kaz Kylheku2014-06-101-5/+3
| | | | in old flexes too.
* * parser.l: Adding an explicit lexical rule toKaz Kylheku2014-06-051-0/+5
| | | | | | catch backslashes occurring within a regex, not followed by a character. This can happen in dynamically parsed regexes such as "abc\\".
* * parser.l (yylex_destroy): Only rely on the FLEX_ versionKaz Kylheku2014-06-051-0/+2
| | | | | numbers if they are defined. In a newer flex, they are prefixed by YY_.
* Changes for FreeBSD 9.Kaz Kylheku2014-06-041-0/+11
| | | | | | | | | * configure: Use pointer-based test for timegm, because -Werror=implicit-function-declaration does not work in the gcc 4.2 used on FreeBSD 9. * parser.l: Check for Flex 2.5.9 and earlier which don't have yylex_destroy. Thanks to Marcus Breiing.
* * eval.c (me_quasilist): New static function.Kaz Kylheku2014-03-251-31/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | (eval_init): Register me_quasilist as quasilist macro expander. * lib.c (quasilist_s): New global variable. (obj_init): quasilist_s initialized. * lib.h (quasilist_s): Declared. * match.c (do_txreval): Handle quasilist syntax. * parser.l (QWLIT): New exclusive state. Extend lexical grammar to transition to QWLIT state upon the #` or #*` sequence which kicks off a word literal, and in that state, piecewise lexically analyze the QLL, mostly by borrowing rules from quasiliterals. * parser.y (QWORDS, QWSPLICE): New tokens. (n_exprs): Integrate splicing form of QLL syntax. (n_expr): Integrate non-splicing form of QLL syntax. (litchars): Propagate line number info. (quasilit): Fix "string literal" wording in error message. * txr.1: Introduced WLL abbreviation for word list literals, cleaned up the text a little, and documented QLL's.
* Introducing word list literals.Kaz Kylheku2014-03-251-8/+27
| | | | | | | | | | | | | | | | * parser.l (WLIT): New exclusive start state. Extend lexical grammar to transition to WLIT state upon the #" or #*" sequence which kicks off a word literal, and in that state, piecewise lexically analyze the literal, mostly by borrowing rules from other literals. * parser.y (WORDS, WSPLICE): New tokens. (n_exprs): Integrate splicing form of word list literal syntax. (n_expr): Integrate non-splicit for of word list literal syntax. (litchars): Propagate line number info. (wordslit): New grammar rule. * txr.1: Updated.
* Fixing gaping bug in the handling of @-delimited expressionsKaz Kylheku2014-03-201-24/+24
| | | | | | | | | | | | | | | within quasiliterals. This has been a problem for years. Quasiliteral strings existed very early before TXR Lisp was introduced. So it made sense that when @ is seen in a quasiliteral, the lexical analyzer pushed into the SPECIAL state in which directives are recognized, like in the pattern language. I noticed this because there is an @(if) directive now, which prevents `@(if ...)` from being valid. * parser.l (QSPECIAL): New scanner state. This is a state similar to SPECIAL that we enter into when @ is seen in a QSLIT state. In this state we recognize constructs like braced variables, but not certain other features like directives.
* * parser.l: Bugfix. When handling a backslash-newline continuationKaz Kylheku2014-03-151-1/+2
| | | | | | | | | | | | | | | | in the SPECIAL, NESTED and BRACED states, do not pop the state in all three, only in SPECIAL (to terminate the @\ continuation). * txr.1: Eliminate wrong claim that string literals do not split across lines, which is directly contradicted two paragraphs later. Document that quasiliterals also split. * genvim.txr (txr_regex, txl_regex): These definitions change from "syn match" to "syn region" so that the backslash-newline continuation can be properly handled. (txr_string, txr_quasilit): Correctly handle split literals. * txr.vim: Regenerated.
* * parser.l (regex_parse, lisp_parse): Fix neglected handling ofKaz Kylheku2014-03-141-1/+4
| | | | | | | | optional arguments. This problem can cause the symbol : to be planted as the std_error stream, resulting in an error loop that blows the stack. * regex.c (regex_compile): Likewise.
* Implementing @(if)/@(elif)/@(else) in the pattern language.Kaz Kylheku2014-03-131-0/+18
| | | | | | | | | | | | | | | | Input side for now; output later. * parser.y (if_clause, elif_clauses_opt, else_clause_opt): New nonterminals. (IF, ELIF, ELSE): New tokens. (yybadtoken): Handle IF, ELIF, ELSE. * parser.l: Recognize and return new tokens IF, ELIF and ELSE. * txr.1: Documented. * genvim.txr: Updated with if, elsif and else directive keywords. * txr.vim: Regenerated
* * parser.l: Syntactic tweak. Do not allow the "extra"Kaz Kylheku2014-03-091-2/+2
| | | | | | | constituent chars # or ^ to start the symbol name after an @. Prior to this change @^a is a "meta-symbol" whose name is "^a", but ^a is the ^ quasiquote notation followed by symbol a. After this change @^a is meta, applied to quasiquote-a.
* * parser.l: Allowing ^ to be a quote character, and adjusting definitionKaz Kylheku2014-03-031-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of identifiers to rule this out from being the first character of a symbol which has no prefix. Recognize the ^ character as a token in the NESTED state. * lib.c (obj_print, obj_pprint): Render sys:qquote as ^. * parser.y (choose_quote): Function removed. (n_expr): Recognize '^' as quasiquote. Removed all the "smart quote" hacks that try to make quote behave as quote or quasiquote, or try to cancel out unquotes and quotes. * tests/009/json.txr: Fixed to ^ quasiquote. * tests/010/reghash.txr: Likewise. * tests/011/macros-2.txr: Likewise. * tests/011/mandel.txr: Likewise. * tests/011/special-1.txr: Likewise. * txr.1: Updated docs. * genvim.txr: Revamped definitions for txr_ident and txl_ident so that unqualified identifiers cannot start with # or ^, but ones with @ or : in front can start with these characters. * txr.vim: Regenerated.
* * parser.l: Regression: valid floating-point literals taggedKaz Kylheku2014-02-271-9/+9
| | | | | as having trailing junk. The fix is to put the junk-matching rule after the valid rules.
* * parser.l (regex_parse, lisp_parse): Better formattingKaz Kylheku2014-02-261-2/+2
| | | | for the context string that serves in place of a filename.
* * parser.l: Support octal and binary numbers.Kaz Kylheku2014-02-241-0/+44
| | | | | | * txr.1: Documented. * genvim.txr, txr.vim: Updated.
* * parser.l: Disallow syntax like 1.0a, flagging it asKaz Kylheku2014-02-121-0/+14
| | | | | | | | an invalid floating-point token. The problem is that 1a is allowed, for compatibility with other Lisp dialects (it is a symbol) whereas 1.0a was scanning as 1.0 followed by a, which is inconsistent. Some Lisp dialects embedded dots in symbols, and allow 1.0a as a symbol token. We don't.
* * hash.c (hash_mark, hash_grow, make_similar_hash,Kaz Kylheku2014-02-011-1/+1
| | | | | | | | | | | | | | | copy_hash, gethash_l, gethash, gethash_f, gethash_n, hash_count, hash_next, hash_eql, hash_equal): Use num_fast instead of num. (make_hash): An attempt to make a weak-keys hash that has equal-based keys is nonsensical; it is now diagnosed with an exception. Use num_fast instead of num. (hash_process_weak): Call breakpt whenever the weak object(s) due to which entries are being deleted match the value in break_obj. Use num_fast instead of num. * parser.l (parse_init): Bugfix: the forms_to_ln_hash was equal-based, which makes no sense.
* Lexing and parsing improvements, leaving things less hacky than before,Kaz Kylheku2014-01-271-20/+13
| | | | | | | | | | | | | | | | | | | | | | | albeit hacky. * parser.l (BSYM, NSYM): Regex definitions gone. (BT0, BT1, BT2, NT0, NT1, NT2): New regex definitions. (BTREG, BTKEY, NTREG, NTKEY): Rewritten, so that they cannot match a lone @ character as a symbol name. (grammar): Rules for returning METAPAR, METABKT and METAQUO are gone. Instead, we just recognize a @ in the NESTED and BRACED states and return it as a token. * parser.y (METAPAR, METABKT, METAQUO): Token types removed. (meta_expr): Nonterminal symbol removed. ('@'): New token type. (list): Quotes and splices handling removed from this rule. The new token '@' is handled here, on the other hand, because there are places that reference the list rule that need to support @ expressions. (n_expr): Reference to meta_expr removed. Quote, unquote and splice added here. (yybadtoken): Removed references to METAPAR, METABKT and METAQUO.
* Sigh; more lexical-syntactic hacks. This adds handlingKaz Kylheku2014-01-261-6/+17
| | | | | | | | | | for the @' combination, as in @(bind a @'(foo ,bar)) * parser.l: Handle the new METAQUO token. * parser.y (METAQUO): New token. (meta_expr): New "METAQUO expr" case. Added missing METABKT error handling case.
* * configure: Detect platforms which don't reveal declarationsKaz Kylheku2014-01-101-4/+0
| | | | | | | | | | | | | | in C headers which are extensions to standard C, like popen or fileno, in response to standard feature selection macros like -D_POSIX_SOURCE. MinGW and Cygwin are offenders. These platforms hide the declarations when gcc is in -ansi mode, by testing for __STRICT_ANSI__. Turns out, however, that -U__STRICT_ANSI__ on the gcc command line strips this away, causing the declarations to be revealed. * lib.c, parser.l, stream.c, utf8.c: Removed the declarations which compensated for the above problem. Yippee! Fuck you, stupid Cygwin troglodytes, and the MinGW horse you rode in on. http://cygwin.com/ml/cygwin/2011-10/msg00131.html
* * eval.c (eval_init): Renaming lisp-parse to read; lisp-parseKaz Kylheku2014-01-101-2/+4
| | | | | | | | is retained as an obsolescent synonym. All arguments become optional. * parser.l (lisp_parse): Handle nil source. * txr.1: Updated.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-10 13:31:57 +0000