summaryrefslogtreecommitdiffstats
path: root/parser.l
Commit message (Collapse)AuthorAgeFilesLines
* Support trailing semicolon after hex/octal characters.Kaz Kylheku2015-07-021-2/+9
| | | | | | | | | | | | | | | | * parser.l (%option): Remove nounput option since we need yyunput. (grammar): Rule for matching hex and octal escape in SPECIAL state recognizes optional semicolon. In 109 compatibility, this is pushed back into the stream, otherwise consumed. * txr.1: Updated documentation, including compat notes. * genvim.txr (txr_char): Include optional semicolon in match. Corrected some errors where 8 and 9 were being included as matches for octal digits. (txr_error): Default match for \x or \o not followed by digits.
* Third round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-0/+6
| | | | | | * parser.l (char_esc): Recognize \@ escape. (grammar): Add a rule for a \@ escape in quasiliterals, and quasi word list literals.
* Second round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-1/+6
| | | | | | | | | * parser.l: Only shift to QSPECIAL state when @ is followed by a trailing context consisting of certain characters. Not every kind of Lisp object syntax can be introduced with @ in a quasiliteral. Adding a rule to produce an error when @ appears that is not followed by an allowed character.
* First round of quasiliteral-related fixes.Kaz Kylheku2015-06-261-7/+8
| | | | | | | | | | | | | | | | | | | | | * parser.l: Do not try to recognize floating-point literals in QSPECIAL state; that is not possible because @134.3 in a quasiliteral parses as a METANUM followed by ".3". On the other hand, recognize METANUM literals in QSPECIAL state, so that @@123 scans. Recognize @ as a token in QSPECIAL state, so @@abc will scan. When transitioning from QSILIT and QWLIT states to QSPECIAL upon scanning @, return a @ token, which is now parsed in the grammar. * parser.y (quasi_meta_helper): New static function. (q_var): Do not handle SYMTOK any more, only the braced variable syntax. SYMTOK is handled as a n_expr. Braced vars are handled with explicit '@' token, which is now produced by the scanner when it shifts from QSILIT to QSPECIAL. (quasi_item): No longer necessary to recognize various forms here such as quotes and splices. Just recognize a n_expr, preceded by '@'.
* Do not allow unrecognized escapes in regex.Kaz Kylheku2015-04-191-11/+14
| | | | | | | | | | | | * parser.l (REGOP): New regex alias for matching all regex special characters. (grammar): Several rules for regex special characters merged together. New rule introduced to match a special character after a backslash, making it literal. The old rule which makes literal any character after a backslash now throws an error, unless version 105 comaptibility is selected. * txr.1: Documented this behavior change.
* Allow quasiquotes in braces and quasiliterals, and quotes in braces.Kaz Kylheku2015-04-151-6/+1
| | | | | | | | | * parser.l: Consolidate rules for recognizing quote, unquote, and quasiquote. An effect of this is that quasiquotes can now occur in braces and in string quasiliterals. * parser.y (quasi_item): Support quotes and quasiquotes as quasi items: that is to say, i.e. objects denoted by @ in a quasiliteral.
* Diagnose trailing junk in numeric literals.Kaz Kylheku2015-04-141-14/+19
| | | | | | | * parser.l: Combining the handling of hex, octal and binary numeric literals into a single rule. Implementing an additional rule which diagnoses such tokens that have trailing junk. Thus, something like #x1F2AZ is now invalid syntax.
* * parser.c (open_txr_file, regex_parse, lisp_parse): FunctionsKaz Kylheku2014-12-211-77/+0
| | | | | | | moved here from parser.l. * parser.l (open_txr_file, regex_parse, lisp_parse): Functions moved from here to parser.c.
* * Makefile (OBJS): Add parser.o.Kaz Kylheku2014-12-211-1/+1
| | | | | | | | | | * parser.h (parser_s): Declared. (parse_init): Declaration removed. (parser_l_init): Declared. * parser.l (parse_init): Function renamed to parser_l_init. * parser.c: New file.
* Improved error reporting, particularly for macro expansion.Kaz Kylheku2015-02-211-4/+2
| | | | | | | | | | | | | | | | | | | * eval.c (last_form_expanded): New variable. (do_expand): New static function; contains previous expand function. (expand): Becomes a wrapper for do_expand, with re-entry counting. (eval_init): GC-protect last_form_expanded. * eval.h (last_form_expanded): Declared. * parser.l (regex_parse, lisp_parse): Just use a simple word for the name of the regex or string parse location, not the entire expression itself. * unwind.c (uw_throw): Check whether expansion was going on when the unhandled exception was thrown and print additional information.
* * hash.c (hash_begin): Use coerce macro instead of raw C cast.Kaz Kylheku2014-10-251-1/+1
| | | | | | Use cobj_handle so hash argument is validated. * parser.l (YY_INPUT): Use convert macro instead of raw C cast.
* GNU Flex's libfl library provides nothing. Let us not refer to it. ItKaz Kylheku2014-10-241-0/+9
| | | | | | | | | | | | only causes build issues on some systems where it is not provided in the standard location, or is not cross-compiled properly. * Makefile (LEXLIB): Reference to variable removed. * configure (lexlib): Variable removed. (LEXLIB): config.make variable not generated. * parser.l (yywrap): Provide this trivial function as inline.
* * Makefile: Removing trailing spaces.Kaz Kylheku2014-10-241-2/+2
| | | | | | | | | | (GREP_CHECK): New macro. (enforce): Rewritten using GREP_CHECK, with new checks. * arith.c, combi.c, debug.c, eval.c, filter.c, gc.c, hash.c, lib.c, * lib.h, match.c, parser.l, parser.y, rand.c, regex.c, signal.c, * signal.h, stream.c, syslog.c, txr.c, unwind.c, utf8.c: Remove trailing spaces.
* * parser.l (lisp_parse): Bugfix: the error_stream argumentKaz Kylheku2014-10-191-3/+12
| | | | | | | | | must be checked to be a stream before we plant it in place of std_error, otherwise we will get a type exception thrown lower down, which leads to runaway recursion as TXR tries to print the error messages on std_error. * dep.mk: Regenerated.
* Converting cast expressions to macros that are retargettedKaz Kylheku2014-10-171-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to C++ style casts when compiling as C++. * lib.h (strip_qual, convert, coerce): New casting macros. (TAG_MASK, tag, type, wli_noex, auto_str, static_str, litptr, num_fast, chr, lit_noex, nil, nao): Use cast macros. * arith.c (mul, isqrt_fixnum, bit): Use cast macros. * configure (INT_PTR_MAX): Define using cast macro. * debug.c (debug_init): Use cast macro. * eval.c (do_eval, expand_macro, reg_op, reg_mac, eval_init): Use cast macros. * filter.c (filter_init): Use cast macro. * gc.c (more, mark_obj, in_heap, mark, sweep_one, unmark): Use cast macros. * hash.c (hash_double, equal_hash, eql_hash, hash_equal_op, hash_hash_op, hash_print_op, hash_mark, make_hash, make_similar_hash, copy_hash, gethash_c, gethash, gethash_f, gethash_n, remhash, hash_count, get_hash_userdata, set_hash_userdata, hash_iter_destroy, hash_iter_mark, hash_begin, hash_uni, hash_diff, hash_isec): Use cast macros. * lib.c (code2type, chk_malloc, chk_malloc_gc_more, chk_calloc, chk_realloc, chk_strdup, num, c_num, string, mkstring, mkustring, upcase_str, downcase_str, string_extend, sub_str, cat_str, trim_str, c_chr, vector, vec_set_length, copy_vec, sub_vec, cat_vec, cobj_print_op, obj_init): Likewise. * match.c (do_match_line, hv_trampoline, match_files, dir_tables_init): Likewise. * parser.l (grammar): Likewise. * parser.y (parse): Likewise. * rand.c (make_state, make_random_state, random_fixnum, random): Likewise. * regex.c (CHAR_SET_L2_LO, CHAR_SET_L2_HI, CHAR_SET_L1_LO, CHAR_SET_L1_HI, CHAR_SET_L0_LO, CHAR_SET_L0_HI, L0_full, L0_fill_range, L1_full, L1_fill_range, L1_contains, L1_free, L2_full, L2_fill_range, L2_contains, L2_free, L3_fill_range, L3_contains, L3_free, char_set_create, char_set_cobj_destroy, nfa_state_accept, nfa_state_empty, nfa_state_single, nfa_state_wild, nfa_state_set,
* Purge stray occurrences of "void *" from code base.Kaz Kylheku2014-10-171-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * lib.c (cobj_print_op): In the format call, cast the C pointer to val, since the ~p conversion now takes a val rather than void *. (cptr_equal_op, obj_print, obj_pprint): Remove cast to void *, since obj now already has the type that ~p expects. * lib.h (struct any): Use mem_t * instead of void *. * parser.h (yyscan_t): Repeat the definition from inside the Flex-generated lex.yy.c. (yylex_init, yylex_destroy, yyget_extra, yyset_extra): Re-declare using yyscan_t typedef in place of void *. * parser.l (yyget_column, yyerrprepf): Re-declare using yyscan_t. (grammar): Use yyg in place of yyscanner in calls to yyerrprepf. * parser.y (yylex, %lex-param): Use yyscan_t instead of void *. (parse): Use yyscan_t for local variable. * signal.c (stack): Change type from void * to mem_t *. * stream.c (vformat): Conversion specifier p extracts val instead of void *. (run): Use casts that only remove const, not all the way to void *. * txr.1: Documented p conversion specifier of format. * Makefile (OBJS-y): Initialize with := to make sure it is a simple variable, and not a macro. (SRCS): New variable, listing source files. (enforce): New rule for enforcing coding conventions. Currently checks for void * occurrences.
* More type safety, with help from C++ compiler.Kaz Kylheku2014-10-141-22/+18
| | | | | | | | | | | | | | | | | | | | | | | | | * parser.h (scanner_t): New typedef. Cleverly, we use yyguts_t as the basis for this, which pays off in a few places, getting rid of conversions. (parser_t): scanner member redeclared to scanner_t *. (yyerror, yyerr, yyerrof, end_of_regex, end_of_char): Declarations updated. * parser.l (yyerror, yyerrorf, num_esc): scanner argument becomes scanner_t * instead of void *. (yyscanner): Occurrences replaced by yyg: why should we refer to the type-unsafe void *yyscanner, when we know that in the middle of the yylex function we have the properly typed yyguts_t *yyg. (end_of_regex, end_of_char): Argument changes to scanner_t * and name strategically changes to yyg, eliminating the local variable and cast. * parser.y (sym_helper): scanner argument becomes scanner_t * instead of void *. (%parse-param}: void *scnr becomes scanner_t *scnr. (define-transform, yybadtoken): Use scanner_t * instead of void *. (parse): Since yylex_init takes a void **, we use it to initialize a local void *, and then assign it to the structure member.
* C++ upkeep.Kaz Kylheku2014-10-141-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TXR's support for compiling as C++ pays off: C++ compiler finds serious bugs introduced in August 2 ("Big switch to reentrant lexing and parsing"). The yyerror function was being misused; some of the calls reversed the scanner and parser arguments. Since one of the two parameters is void *, this reversal wasn't caught. * parser.l (yyerror): Fix first two arguments being reversed. (num_esc): Change previously correct call to yyerror to follow reversed arguments, so that it stays correct. * parser.y (%parse-param): Change order of these directives so that the scnr parameter is before the parser parameter. This causes the yacc-generated calls to yyerror to have the arguments in the correct order. It also has the effect of changing the signature of yyparse, reversing its parameters. (parse): Update call to yyparse to new argument order. * parser.h (yyparse): Declaration removed. (yyerror): Declaration updated. * regex.c (regex_kind_t): New enum typedef. (struct regex): Use regex_kind_t rather than an enum inside the struct, which has different scoping rules under C++. * txr.c (get_self_path): Fix signed/unsigned warning.
* * eval.c (eval_init): Update registration of lisp-parse and readKaz Kylheku2014-09-021-2/+8
| | | | | | | | | | | | | | | | | | | | to account for new parameter. * lib.c (syntax_error_s): New symbol_variable. (obj_init): New symbol variable initialized. * lib.h (syntax_error_s): Declared. * parser.h (lisp_parse): Declaration updated. * parser.l (lisp_parse): Takes third parameter. * txr.1: Third parameter of read described. * txr.c (txr_main): Pass colon_k to third parameter of lisp_parse to obtain exception throwing behavior. * unwind.c (uw_init): Register syntax-error as subtype of error.
* * parser.l (yyerr): Function removed; it is not used in the lexer,Kaz Kylheku2014-08-071-5/+0
| | | | | | | | | | and converted to a macro in the parser. * parser.y (define_transform): take a parser argument rather than scanner. Set up scnr local variable for yyerr macro. Remove scnr argument from macro calls. (yyerr): New macro. (grammar): Remove scnr argument from yyerr calls.
* Reentrant parser regression.Kaz Kylheku2014-08-071-10/+9
| | | | | | | | | | | * parser.y (yybadtok): New macro. (yybadtoken): Function must take parser argument. (grammar): Replace uses of yybadtoken with yybadtok. * parser.h (yybadtoken): Declaration updated. * parser.l (grammar): Fix incorrect yyprepf calls that are missing the yyscanner parameter.
* * parser.y: Back port from Berkeley Yacc to GNU Bison.Kaz Kylheku2014-08-051-1/+1
| | | | | | | | We need a prototype of yylex that is in scope of the grammar, but YYSTYPE is not defined there. * parser.l: Bison 3 declares yyparse in y.tab.h, so we have to reorder some #includes.
* Big switch to reentrant lexing and parsing.Kaz Kylheku2014-08-021-260/+262
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parser.l (YY_INPUT): Stop relying on removed yyin_stream; refer to stream via yyextra. (yyin_stream, lineno, errors, spec_file_str, prepared_error_message): Global variables removed. (yyget_column, yyset_column): Missing prototypes not generated by flex in bison bridge mode have to be added by us to avoid warning. (yyerror): Takes parser and scanner as parameters. Prepared error message is now in the parser context. Calls to other error handling functions receive scanner context. (yyerr): New function. (yyerrorf, yyerrprepf): Takes scanner argument, chases extra data to get to parser, and refers to parser variables instead of globals. (num_esc): Scanner argument added. (%option reentrant, %option bison-bridge, %option extra-type): New flex options. (grammar): yyscanner added everywhere. (end_of_char): Takes scanner argument. (parse_init): Removed references to yyin_stream and prepared_error_message. (parse_reset): Function renamed to open_txr_file. Returns results via pointers instead of setting global variables. (regex_parse, lisp_parse): Use reentrant parser interface. * parser.y (yyerror): Prototype removed. (yylex): Prototype moved after grammar, with new arguments. (sym_helper, define_transform): Take scanner argument. (make_expr): Takes parser argument. (rlrec): New static function. (rl): Function turned into macro. (mkexp, symhlpr): New macros. (%purse-parser, %parse-param, %lex-param): New Yacc options. (grammar): Actions re-worked for reentrance. Parser and scanner contexts are passed down to helper functions, in some cases via the three new macros. The result of the parse is stored in the syntax_tree member of the parser_t structure instead of a global. The yylex function receives the scanner instance. (get_spec): Function removed. (parse): New function. * parser.h (lineno, errors, yyin_stream, spec_file_str): Declarations removed. (parser_t): New struct. (yyerr): New function declared. (yyparse, yyerror, yyerrorf, end_of_regex, end_of_char, yylex, yylex_destroy): Declarations updated.
* * parser.l: Allow unquotes and splices in QSPECIAL and BRACED states.Kaz Kylheku2014-07-301-1/+6
| | | | | | | | | * parser.y (quasi_item): Support splices as items. * genvim.txr: Syntax highlighting support for unquotes in quasiliterals. * txr.vim: Updated.
* * Makefile, arith.c, arith.h, combi.c, combi.h, configure, debug.c,Kaz Kylheku2014-07-231-16/+16
| | | | | | | | debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, hash.c, hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, syslog.c, syslog.h, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Synchronize license header with LICENSE.
* * parser.l (yylex_destroy): The FLEX_ version are prefixed with YY_Kaz Kylheku2014-06-101-5/+3
| | | | in old flexes too.
* * parser.l: Adding an explicit lexical rule toKaz Kylheku2014-06-051-0/+5
| | | | | | catch backslashes occurring within a regex, not followed by a character. This can happen in dynamically parsed regexes such as "abc\\".
* * parser.l (yylex_destroy): Only rely on the FLEX_ versionKaz Kylheku2014-06-051-0/+2
| | | | | numbers if they are defined. In a newer flex, they are prefixed by YY_.
* Changes for FreeBSD 9.Kaz Kylheku2014-06-041-0/+11
| | | | | | | | | * configure: Use pointer-based test for timegm, because -Werror=implicit-function-declaration does not work in the gcc 4.2 used on FreeBSD 9. * parser.l: Check for Flex 2.5.9 and earlier which don't have yylex_destroy. Thanks to Marcus Breiing.
* * eval.c (me_quasilist): New static function.Kaz Kylheku2014-03-251-31/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | (eval_init): Register me_quasilist as quasilist macro expander. * lib.c (quasilist_s): New global variable. (obj_init): quasilist_s initialized. * lib.h (quasilist_s): Declared. * match.c (do_txreval): Handle quasilist syntax. * parser.l (QWLIT): New exclusive state. Extend lexical grammar to transition to QWLIT state upon the #` or #*` sequence which kicks off a word literal, and in that state, piecewise lexically analyze the QLL, mostly by borrowing rules from quasiliterals. * parser.y (QWORDS, QWSPLICE): New tokens. (n_exprs): Integrate splicing form of QLL syntax. (n_expr): Integrate non-splicing form of QLL syntax. (litchars): Propagate line number info. (quasilit): Fix "string literal" wording in error message. * txr.1: Introduced WLL abbreviation for word list literals, cleaned up the text a little, and documented QLL's.
* Introducing word list literals.Kaz Kylheku2014-03-251-8/+27
| | | | | | | | | | | | | | | | * parser.l (WLIT): New exclusive start state. Extend lexical grammar to transition to WLIT state upon the #" or #*" sequence which kicks off a word literal, and in that state, piecewise lexically analyze the literal, mostly by borrowing rules from other literals. * parser.y (WORDS, WSPLICE): New tokens. (n_exprs): Integrate splicing form of word list literal syntax. (n_expr): Integrate non-splicit for of word list literal syntax. (litchars): Propagate line number info. (wordslit): New grammar rule. * txr.1: Updated.
* Fixing gaping bug in the handling of @-delimited expressionsKaz Kylheku2014-03-201-24/+24
| | | | | | | | | | | | | | | within quasiliterals. This has been a problem for years. Quasiliteral strings existed very early before TXR Lisp was introduced. So it made sense that when @ is seen in a quasiliteral, the lexical analyzer pushed into the SPECIAL state in which directives are recognized, like in the pattern language. I noticed this because there is an @(if) directive now, which prevents `@(if ...)` from being valid. * parser.l (QSPECIAL): New scanner state. This is a state similar to SPECIAL that we enter into when @ is seen in a QSLIT state. In this state we recognize constructs like braced variables, but not certain other features like directives.
* * parser.l: Bugfix. When handling a backslash-newline continuationKaz Kylheku2014-03-151-1/+2
| | | | | | | | | | | | | | | | in the SPECIAL, NESTED and BRACED states, do not pop the state in all three, only in SPECIAL (to terminate the @\ continuation). * txr.1: Eliminate wrong claim that string literals do not split across lines, which is directly contradicted two paragraphs later. Document that quasiliterals also split. * genvim.txr (txr_regex, txl_regex): These definitions change from "syn match" to "syn region" so that the backslash-newline continuation can be properly handled. (txr_string, txr_quasilit): Correctly handle split literals. * txr.vim: Regenerated.
* * parser.l (regex_parse, lisp_parse): Fix neglected handling ofKaz Kylheku2014-03-141-1/+4
| | | | | | | | optional arguments. This problem can cause the symbol : to be planted as the std_error stream, resulting in an error loop that blows the stack. * regex.c (regex_compile): Likewise.
* Implementing @(if)/@(elif)/@(else) in the pattern language.Kaz Kylheku2014-03-131-0/+18
| | | | | | | | | | | | | | | | Input side for now; output later. * parser.y (if_clause, elif_clauses_opt, else_clause_opt): New nonterminals. (IF, ELIF, ELSE): New tokens. (yybadtoken): Handle IF, ELIF, ELSE. * parser.l: Recognize and return new tokens IF, ELIF and ELSE. * txr.1: Documented. * genvim.txr: Updated with if, elsif and else directive keywords. * txr.vim: Regenerated
* * parser.l: Syntactic tweak. Do not allow the "extra"Kaz Kylheku2014-03-091-2/+2
| | | | | | | constituent chars # or ^ to start the symbol name after an @. Prior to this change @^a is a "meta-symbol" whose name is "^a", but ^a is the ^ quasiquote notation followed by symbol a. After this change @^a is meta, applied to quasiquote-a.
* * parser.l: Allowing ^ to be a quote character, and adjusting definitionKaz Kylheku2014-03-031-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of identifiers to rule this out from being the first character of a symbol which has no prefix. Recognize the ^ character as a token in the NESTED state. * lib.c (obj_print, obj_pprint): Render sys:qquote as ^. * parser.y (choose_quote): Function removed. (n_expr): Recognize '^' as quasiquote. Removed all the "smart quote" hacks that try to make quote behave as quote or quasiquote, or try to cancel out unquotes and quotes. * tests/009/json.txr: Fixed to ^ quasiquote. * tests/010/reghash.txr: Likewise. * tests/011/macros-2.txr: Likewise. * tests/011/mandel.txr: Likewise. * tests/011/special-1.txr: Likewise. * txr.1: Updated docs. * genvim.txr: Revamped definitions for txr_ident and txl_ident so that unqualified identifiers cannot start with # or ^, but ones with @ or : in front can start with these characters. * txr.vim: Regenerated.
* * parser.l: Regression: valid floating-point literals taggedKaz Kylheku2014-02-271-9/+9
| | | | | as having trailing junk. The fix is to put the junk-matching rule after the valid rules.
* * parser.l (regex_parse, lisp_parse): Better formattingKaz Kylheku2014-02-261-2/+2
| | | | for the context string that serves in place of a filename.
* * parser.l: Support octal and binary numbers.Kaz Kylheku2014-02-241-0/+44
| | | | | | * txr.1: Documented. * genvim.txr, txr.vim: Updated.
* * parser.l: Disallow syntax like 1.0a, flagging it asKaz Kylheku2014-02-121-0/+14
| | | | | | | | an invalid floating-point token. The problem is that 1a is allowed, for compatibility with other Lisp dialects (it is a symbol) whereas 1.0a was scanning as 1.0 followed by a, which is inconsistent. Some Lisp dialects embedded dots in symbols, and allow 1.0a as a symbol token. We don't.
* * hash.c (hash_mark, hash_grow, make_similar_hash,Kaz Kylheku2014-02-011-1/+1
| | | | | | | | | | | | | | | copy_hash, gethash_l, gethash, gethash_f, gethash_n, hash_count, hash_next, hash_eql, hash_equal): Use num_fast instead of num. (make_hash): An attempt to make a weak-keys hash that has equal-based keys is nonsensical; it is now diagnosed with an exception. Use num_fast instead of num. (hash_process_weak): Call breakpt whenever the weak object(s) due to which entries are being deleted match the value in break_obj. Use num_fast instead of num. * parser.l (parse_init): Bugfix: the forms_to_ln_hash was equal-based, which makes no sense.
* Lexing and parsing improvements, leaving things less hacky than before,Kaz Kylheku2014-01-271-20/+13
| | | | | | | | | | | | | | | | | | | | | | | albeit hacky. * parser.l (BSYM, NSYM): Regex definitions gone. (BT0, BT1, BT2, NT0, NT1, NT2): New regex definitions. (BTREG, BTKEY, NTREG, NTKEY): Rewritten, so that they cannot match a lone @ character as a symbol name. (grammar): Rules for returning METAPAR, METABKT and METAQUO are gone. Instead, we just recognize a @ in the NESTED and BRACED states and return it as a token. * parser.y (METAPAR, METABKT, METAQUO): Token types removed. (meta_expr): Nonterminal symbol removed. ('@'): New token type. (list): Quotes and splices handling removed from this rule. The new token '@' is handled here, on the other hand, because there are places that reference the list rule that need to support @ expressions. (n_expr): Reference to meta_expr removed. Quote, unquote and splice added here. (yybadtoken): Removed references to METAPAR, METABKT and METAQUO.
* Sigh; more lexical-syntactic hacks. This adds handlingKaz Kylheku2014-01-261-6/+17
| | | | | | | | | | for the @' combination, as in @(bind a @'(foo ,bar)) * parser.l: Handle the new METAQUO token. * parser.y (METAQUO): New token. (meta_expr): New "METAQUO expr" case. Added missing METABKT error handling case.
* * configure: Detect platforms which don't reveal declarationsKaz Kylheku2014-01-101-4/+0
| | | | | | | | | | | | | | in C headers which are extensions to standard C, like popen or fileno, in response to standard feature selection macros like -D_POSIX_SOURCE. MinGW and Cygwin are offenders. These platforms hide the declarations when gcc is in -ansi mode, by testing for __STRICT_ANSI__. Turns out, however, that -U__STRICT_ANSI__ on the gcc command line strips this away, causing the declarations to be revealed. * lib.c, parser.l, stream.c, utf8.c: Removed the declarations which compensated for the above problem. Yippee! Fuck you, stupid Cygwin troglodytes, and the MinGW horse you rode in on. http://cygwin.com/ml/cygwin/2011-10/msg00131.html
* * eval.c (eval_init): Renaming lisp-parse to read; lisp-parseKaz Kylheku2014-01-101-2/+4
| | | | | | | | is retained as an obsolescent synonym. All arguments become optional. * parser.l (lisp_parse): Handle nil source. * txr.1: Updated.
* Long overdue MinGW port maintenance.Kaz Kylheku2014-01-091-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Makefile: Use new EXE variable from config.mk. * configure (exe, have_windows_h): New variables. Handle situations with .exe suffix; on MiGW, the rm command doesn't work on executables if the .exe suffix is not given. New tests for localtime_r and gmtime_r. * lib.c: Supply declarations which are missing on MinGW because we use gcc -ansi, because MinGW doesn't follow established conventions like -D_POSIX_SOURCE. Supply definitions for gmtime_r, localtime_r, setenv and unsetenv. * parser.l: Supply declarations which are missing on MinGW. * signal.h (async_sig_enabled): Declare differently based on HAVE_POSIX_SIGS. Misspelled typedef fixed in the code for !HAVE_POSIX_SIGS that has hitherto not been compiled. (sig_mask): Wrap declaration in #ifdef HAVE_POSIX_SIGS because it relies on sigset_t. * stream.c: Supply declarations which are missing on MinGW. Include <windows.h> if we have it. (sleep): Define for Windows. (statf): Handle missing st_blksize and st_blocks members in struct stat. (stream_init): Handle numerous missing S_* macros. * utf8.c: Supply declarations which are missing on MinGW.
* The lisp-parse function can now be called multiple timesKaz Kylheku2014-01-071-9/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | on the same stream to extract multiple objects; the requirement that the stream must hold exactly one complete Lisp object with no following material is now lifted. * parser.l (YY_INPUT): Modified the macro so that it reads no more than one character. Though this probably makes the lexer less efficient, it gives us the important property that the lexer does not scan ahead into the input stream, hogging data into its buffer which is then destroyed. This is essential if the lisp-parse function is to support multiple calls to pull objects one by one out of a stream. * parser.y (spec): Use YYACCEPT in the SECRET_ESCAPE_E clause for pulling a single expression out of the token stream. YYACCEPT is a trick for not invoking the $accept : spec . $end production which is implicitly built into the grammar, and which causes a token of lookahead to occur. This allows us to read a full expression without stealing any further token: but only if the grammar is structured right. (exprs): This phrase structure now handles the DOTDOT syntax. There is no such thing as an expr DOTDOT expr expression any more; it is in the list syntax (and not supported in the dot position). (expr): Remove DOTDOT syntax. * txr.1: Updated description of .. syntax, and relaxed the description of lisp-parse since it now allows multiple calls to extract multiple objects.
* * lib.c (intern): fix the previous diagnostic bug once more with moreKaz Kylheku2013-12-161-11/+7
| | | | | | | | | | feeling. * parser.l (grammar): Recognize package prefixes in symbol tokens. Got rid of special rule for handling lone colon. * parser.y (sym_helper): Catch undefined package as a parsing error rather allowing intern function to throw exception.
* Changing the tokenizer to get rid of IDENT, KEYWORD and METAVARKaz Kylheku2013-12-151-14/+5
| | | | | | | | | | | | | | | | | | | | | token categories, replaced by a single one called SYMTOK. Package prefixes are now recognized and processed in tokens. * lib.c (delete_package): Fix problem in no-such-package error case: it would always report nil as the name. (intern): Fix nonsensical error message: in the no-such-package case it would report that the symbol exists already. * parser.l (grammar): Occurences of KEYWORD, METAVAR, and IDENT scrubbed. All rules reporting any of these now return SYMTOK. The main one of these is greatly simplified. * parser.y (sym_helper): New function. (char_from_name): const qualifier inside param's type declaration. (grammar): IDENT, KEYWORD and METAVAR tokens are gone. New token SYMTOK. Grammar refactored around SYMTOK and using the new sym_helper function. (char_from_name): Updated.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-10 13:47:12 +0000