summaryrefslogtreecommitdiffstats
path: root/parser.h
Commit message (Collapse)AuthorAgeFilesLines
* Copyright year bump 2021.Kaz Kylheku2021-01-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * METALICENSE: 2020 copyrights bumped to 2021. Added note about SHA-256 routines from Colin Percival. * LICENSE, LICENSE-CYG, Makefile, alloca.h, args.c, args.h, arith.c, arith.h, buf.c, buf.h, cadr.c, cadr.h, chksum.c, chksum.h, chksums/crc32.c, chksums/crc32.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, ffi.c, ffi.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, itypes.c, itypes.h, jmp.S, lex.yy.c.shipped, lib.c, lib.h, linenoise/linenoise.c, linenoise/linenoise.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, protsym.c, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/asm.tl, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/compiler.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/copy-file.tl, share/txr/stdlib/debugger.tl, share/txr/stdlib/defset.tl, share/txr/stdlib/doloop.tl, share/txr/stdlib/each-prod.tl, share/txr/stdlib/error.tl, share/txr/stdlib/except.tl, share/txr/stdlib/ffi.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/keyparams.tl, share/txr/stdlib/op.tl, share/txr/stdlib/package.tl, share/txr/stdlib/param.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/pmac.tl, share/txr/stdlib/quips.tl, share/txr/stdlib/save-exe.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/stream-wrap.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/trace.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/vm-param.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, strudel.c, strudel.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, time.c, time.h, tree.c, tree.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, vm.c, vm.h, vmop.h, win/cleansvg.txr, y.tab.c.shipped: Copyright year bumped to 2021.
* New function: txr-parse.Kaz Kylheku2020-04-021-0/+2
| | | | | | | | | | | | txr-parse provides a way for Lisp code to programmatically parse the TXR language and obtain the Lisp represenation. This has hitherto not been available. * eval.c (eval_init): Register txr-parse intrinsic. * parser.c (txr_parse): New function. * parser.h (txr_parse): Declared.
* Copyright year bump 2020.Kaz Kylheku2019-12-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * LICENSE, LICENSE-CYG, METALICENSE, Makefile, alloca.h, args.c, args.h, arith.c, arith.h, buf.c, buf.h, cadr.c, cadr.h, chksum.c, chksum.h, chksums/crc32.c, chksums/crc32.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, ffi.c, ffi.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, itypes.c, itypes.h, jmp.S, lib.c, lib.h, linenoise/linenoise.c, linenoise/linenoise.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, protsym.c, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/asm.tl, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/compiler.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/debugger.tl, share/txr/stdlib/defset.tl, share/txr/stdlib/doloop.tl, share/txr/stdlib/error.tl, share/txr/stdlib/except.tl, share/txr/stdlib/ffi.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/keyparams.tl, share/txr/stdlib/op.tl, share/txr/stdlib/package.tl, share/txr/stdlib/param.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/pmac.tl, share/txr/stdlib/save-exe.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/stream-wrap.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/trace.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/vm-param.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, strudel.c, strudel.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, tree.c, tree.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, vm.c, vm.h, vmop.h, win/cleansvg.txr: Extended copyright notices to 2020.
* parser: rename circ_suppress flag.Kaz Kylheku2019-08-181-2/+2
| | | | | | | | | | | * parser.h (struct parser): eof flag changed to unsigned char. circ_suppress flag renamed to ignore, changed to unsigned char and relocated next to eof to compact together. * parser.c (parser_circ_ref): Follow rename. * parser.y (hash_semi_or_n_expr, hash_semi_or_i_expr, n_exprs, parse): Likewise.
* txr: regression: lack of file name in error messages.Kaz Kylheku2019-07-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was broken on April 21, 2019 in commit 2e36e0feae8d1dd75c8410b365d7dc33b30ce66b (April 21, 2019) which changed parse_once in such a way that the name being passed down was no longer stored in the parser. The ensure_parser function being relied upon doesn't take on the responsibility of propagating the name from the stream to the parser, so the parser ends up with a nil name. Let's just smooth this over by having ensure_parser take a name parameter. If ensure_parser creates a new parser, it puts in the name. If the name is nil, then it is taken from the stream. * eval.c (load): Pass name to ensure_parser. * match.c (v_load): Likewise. * parser.c (parser): Take name parameter, and plant into newly constructed parser. (ensure_parser): Take name parameter. When creating a parser, give it that name, or if name is specified as nil, then give it the name taken from the stream. (parser_set_lineno): Pass nil name to ensure_parser. (lisp_parse_impl, read_file_common): Pass name to ensure_parser. * parser.h (parser, ensure_parser): Declarations updated. * parser.y (parse_once): Pass name to ensure_parser. * txr.c (txr_main): Pass spec_file_str to ensure_parser.
* bugfix: source lineno off by one under hash bang.Kaz Kylheku2019-04-211-0/+1
| | | | | | | | | | | | | | * eval.c (load): When we read and discard a hash bang line, we must set the parser line number to two. * parser.c (parser_set_lineno): New function. * parser.h (parser_set_lineno): Declared. * txr.c (check_hash_bang): New argument, occurs. (txr_main): Track whether hash bang has occurred in a new local variable hb_occurs. Then, before parsing, if hash bang has occurred, set the line number to two.
* parser: always use stream-associated parser for parse_once.Kaz Kylheku2019-04-211-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This refactoring is needed for fixing the off-by-one line number bug when the hash bang line is processed. * eval.c (load): Don't define parser locally; ensure there is one in the stream and use it. * match.c (v_load): Likewise. * parser.c (get_parser_impl): Renamed to parser_get_impl and changed from internal to external linkage. (ensure_parser): Changed to external linkage. (lisp_parser_impl, read_file_common): Follow rename of get_parser_impl. * parser.h (parse_once): Declaration updated. (parser_get_impl, ensure_parser): Declared. * parser.y (parse_once): Take self parameter; drop parser parameter. Ensure a parser to the stream, rather than declaring one locally. Don't clean up the parser when done, just let the stream clean it up. * txr.c (parse_once_noerr): Parser argument is dropped and not passed to parse_once. Program name is passed as self argument to parse_once. (txr_main): When parsing the TXR pattern query, don't define a parser locally; ensure there is one in the stream and use it, like in load and v_load.
* repl: groundwork for recursive listener invocation.Kaz Kylheku2019-04-081-1/+1
| | | | | | | | | | | | | | | | | * parser.c (lino_ctx, repl_level): New static variable. (repl): Increment repl_level on entry and decrement on exit. On the initial entry, allocate the lino_t object, storing it in the new static variable. Free it on outermost exit. The number of > characters in the prompt indicates the nesting level. The function now takes an environment parameter, which is applied to the evaluation of the input form. The intent is to be able to supply symbol macros which expand to function calls implementing debugger commands. (parser_init): Register sys:repl intrinsic. * parser.h (repl): Declaration updated. * txr.c (txr_main): Pass nil environment parameter to repl
* Copyright year bump 2019.Kaz Kylheku2019-01-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * LICENSE, LICENSE-CYG, METALICENSE, Makefile, args.c, args.h, arith.c, arith.h, buf.c, buf.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, ffi.c, ffi.h, filter.c, filter.h, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, itypes.c, itypes.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, protsym.c, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/asm.tl, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/compiler.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/doloop.tl, share/txr/stdlib/error.tl, share/txr/stdlib/except.tl, share/txr/stdlib/ffi.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/keyparams.tl, share/txr/stdlib/op.tl, share/txr/stdlib/package.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/pmac.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/stream-wrap.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/trace.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/vm-param.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, strudel.c, strudel.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, vm.c, vm.h, vmop.h, win/cleansvg.txr: Extended Copyright line to 2018.
* Better identify functions that misuse COBJ-s and hashes.Kaz Kylheku2018-11-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this patch, the cobj_handle, cobj_ops and variants of gethash get an additional argument to identify the caller. Many functions are updated to pass this down. * buf.c (buf_strm): Pass self name to cobj_handle. * eval.c (env_fbind, env_vbind, rt_defvarl, me_case): Pass self name to gethash_c or gethash_e. (load): Pass self name to read_eval_stream and read_compiled_file. (reg_symacro): Pass situation-identifying string to gethash_c. * ffi.c (ffi_type_struct_checked, ffi_closure_struct_checked, ffi_call_desc_checked, uni_struct_checked): Take self name parameter, and pass down to cobj_handle. (ffi_get_type, ffi_get_lisp_type): Take self name and pass down to ffi_type_struct_checked. (union_get_ptr): Take self name and pass to uni_struct_checked. (ffi_union_in, ffi_union_put): Pass self name to union_get_ptr. (ffi_type_compile): Pass self name to ffi_get_lisp_type. (ffi_make_call_desc): Pass self name to ffi_type_struct_checked, ffi_get_type and ffi_call_desc_checked. (ffi_make_closure): Pass self name to ffi_call_desc_checked. (ffi_closure_get_fptr): Take self name, pass to ffi_closure_struct_checked. (ffi_typedef, ffi_size, ffi_alignof, ffi_offsetof, ffi_arraysize, ffi_elemsize, ffi_elemtype, ffi_put_into, ffi_put, ffi_in, ffi_get, ffi_out, make_carray): Pass self name to ffi_closure_struct_checked. (carray_struct_checked): Take self name, pass to cobj_handle. (carray_set_length, carray_dup, carray_own, carray_free, carray_type, length_carray, copy_carray, carray_ptr, buf_carray, vec_carray, list_carray, carray_ref, carray_refset, carray_sub, carray_replace, carray_get_common, carray_put_common, unum_carray, num_carray, put_carray, fill_carray): Pass self name to carray_struct_checked. (carray_blank, carray_buf, carray_cptr): Pass self name ffi_type_struct_checked. (carray_pun): Pass self name to carray_struct_checked and ffi_type_struct_checked. (make_union): Pass self name to ffi_type_struct_checked. (union_members, union_get, union_put, union_in, union_out): Pass self name to uni_struct_checked. (make_zstruct, zero_fill, put_obj, get_obj, fill_obj): Pass self-name to ffi_type_struct_checked. * ffi.h (ffi_closure_get_fptr, union_get_ptr): Declarations updated. * filter.c (trie_add): Pass self-name to gethash_l. * hash.c (make_similar_hash, copy_hash, hash_count, get_hash_userdata, set_hash_userdata, hash_begin, hash_next, hash_uni, hash_diff, hash_isec): Pass self name to cobj_handle. (gethash_c, gethash_e): Take self name parameter and pass down to cobj_handle. (gethash_f): Take self parameter and pass down to gethash_e. (gethash, inhash, gethash_n, sethash, pushhash, remhash, clearhash, hash_update_1): Pass self name to gethash_e or gethash_c. * hash.h (gethash_c, gethash_e, gethash_f): Declarations updated. (gethash_l): Take self name, and pass down to gethash_c. * lib.c (class_check): Take self name parameter and use in type mismatch diagnostic. (use_sym, unuse_sym, symbol_needs_prefix, find_symbol, intern, unintern, intern_fallback, unique, in, sel, obj_print_impl, populate_obj_hash, obj_hash_merge): Pass self name to gethash_f or gethash_l. (symbol_visible, obj_init): Pass situation-identifying string to gethash_e. (cobj_handle, cobj_ops): Take self name parameter and pass down to class_check. * lib.h (class_check, cobj_handle, cobj_ops): Declarations updated. * match.c (v_load): Pass self name to read_compiled_file and read_eval_stream. * parser.c (get_parser_impl): Take self name and pass to cobj_handle. (ensure_parser): Pass situation-identifying string to gethash_c. (parser_circ_def): Pass self-name to gethash_c. (lisp_parser_impl): Pass self name to get_parser_impl and class_check. (lisp_parse, nread, iread): Pass self-name to lisp_parser_impl. (read_file_common): Take self name parameter and pass down to get_parser_impl. (read_eval_stream, read_compiled_file): Take self name and pass down to read_file_common. (load_rcfile): Pass situation-identifying string to read_eval_streem. (get_visible_syms): Pass situation-identifying string to gethash_c. (parser_errors, parser_eof): Pass self name to cobj_handle. * parser.h (read_eval_stream, read_compiled_file): Declarations updated. * parser.y (rlset): Pass self name to gethash_c. * rand.c (make_random_state, random_state_get_vec,l random_fixnum, random_float): Pass self name to cobj_handle. * regex.c (regex_source, regex_print, regex_run): Pass self-name to cobj_handle. (regex_machine_init): Take self name param and pass to cobj_handle. (search_regex, match_regex, match_regex_right, regex_prefix_match, read_until_match): Pass self-name to regex_machine_init. * stream.c (stdio_get_fd): Pass self name to cobj_handle. (generic_get_line): Get COBJ operations via unsafe, diret object access rather than cobj_ops. (set_mode_props): Get object handle via unsafe, direct object access. (stream_fd, sock_family, sock_type, sock_peer, set_sock_peer, get_string_from_stream, get_list_from_stream, stream_set_prop, stream_get_prop, close_stream, get_error, get_error_str, clear_error, get_line, get_char, get_byte, unget_char, unget_byte, put_buf, fill_buf, put_string, put_char, put_byte, flush_stream, seek_stream, truncate_stream, get_indent_mode, test_set_indent_mode, set_indent_mode, get_indent, set_indent, inc_indent, width_check, force_break, get_set_ctx, get_ctx): Pass self name to cobj_ops. (make_delegate_stream): Take self name parameter, pass down to cobj_ops. (record_adapter): Pass self name down to make_delegate_stream. (format): Pass self name to class_check. * struct.c (stype_handle): Pass self name to cobj_handle. (make_struct_type): Pass self name to class_check. * txr.c (read_eval_stream_noerr): Take self name parameter, pass to read_eval_stream. (txr_main): Pass istuation-identifying string to read_compiled_file and read_eval_stream_noerr. * unwind.c (revive_cont): Pass self-name to cobj_handle. * vm.c (vm_desc_struct): Take self name parameter, pass to cobj_handle. (vm_desc_nlevels, vm_desc_nregs, vm_desc_bytecode, vm_desc_datavec, vm_desc_symvec, vm_execute_toplevel, vm_execute_closure, vm_closure_entry): Pass self name to vm_desc_struct. (vm_closure_struct): Take self name parameter, pass to cobj_handle.
* parser: don't generate special lits outside quasiquote.Kaz Kylheku2018-04-041-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The parser generates a sys:hash-lit, sys:struct-lit or sys:vector-lit whenever a hash, struct or vector literal contains unquotes. This allows the quasiquote expander to treat these objects as ordinary list structure when interpolating inside them, and then recognize these symbols and construct the implied real objects. The issue is that these literals are generated even if the unquotes occur outside of a backquote. For instance if a vector literal like #(,a) occurs out of the blue, not in any backquote, this is still a (sys:vector-lit (sys:unquote a)) and not an actual vector. The issue is compounded because this substitution takes place even if there is no actual comma or splice notation. Even the following is a sys:vector-lit: #((sys:unquote x)). In any case, it causes problems for compiled files, because such material can occur in the data vector of a compiled toplevel form. In this patch we modify the parser to keep track of the quasiquote/unquote level. The special literals are generated only when the object occurs inside a quasiquote. * parser.h (struct parser): New member, quasi_level. * parser.c (parser_common_init): Initialize the parser's new quasi_level member. * parser.y (vector, hash, struct): To decide whether to generate the special literal, don't just check whether unquotes occur in the list. Check that we are in a quasiquote, indicated by the quasiquoting level being positive. (i_expr, n_expr): Use a mid-rule actions on the quasiquote, unquote and splice rules to bump the quasiquoting level in one direction before recognizing the object, and then bump in the opposite direction when reducing the rule. (parse): Initialize quasi_level.
* Implement compiled file loading.Kaz Kylheku2018-04-041-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | * eval.c (load): If open_txr_file indicates compiled file by setting txr_lisp_p to character #\o, use read_compiled_file. * match.c (v_load): Likewise. * parser.c (open_txr_file): Recognize the .tlo suffix, and also try to open a .tlo version of an unsuffixed file before trying it as .tl. Indicate a .tlo file by setting txr_lisp_p to the character #\o rather than t. (read_file_common): New static function, made from contents of read_eval_stream. Will either evaluate forms or load compiled code by instantiating virtual machine descriptions and performing their top-level execution. (read_eval_stream): Now a wrapper for read_file_common. (read_compiled_file): New function. * parser.h (read_compiled_file): Declared. * txr.c (help): List new --compiled option. (txr_main): If --compiled is specified, set txr_lisp_p to #\o to load as compiled code. Update error message that -c is not compatible with --lisp or --compiled. If txr_lisp_p is #\o, then use read_compiled_file.
* Copyright year bump 2018.Kaz Kylheku2018-02-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * LICENSE, LICENSE-CYG, METALICENSE, Makefile, args.c, args.h, arith.c, arith.h, buf.c, buf.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, ffi.c, ffi.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, itypes.c, itypes.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, protsym.c, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/doloop.tl, share/txr/stdlib/error.tl, share/txr/stdlib/except.tl, share/txr/stdlib/ffi.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/keyparams.tl, share/txr/stdlib/op.tl, share/txr/stdlib/package.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/pmac.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/stream-wrap.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, strudel.c, strudel.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, win/cleansvg.txr: Extended Copyright line to 2018.
* read, iread: source location recording now conditional.Kaz Kylheku2017-12-291-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recording of source location info incurs a time and space penalty. We don't want to impose this on programs which are just reading large amounts of Lisp data that isn't code. * eval.c (eval_init): Register lisp-parse and read functions to the newly introduced nread function rather than lisp_parse. lisp_parse continues to record source location info unconditionally. * parser.c (rec_source_loc_s): New symbol variable. (parser_common_init): Set the new member of the parser structure, rec_source_loc, according to the current value of the special var *rec-source-loc*. (lisp_parse_impl): New second argument, rlcp_p. If true, it overrides the rec_source_loc member of the parser structure to true. (lisp_parse): Pass true argument to rlcp_p parameter of lisp_parse_impl, so parsing via lisp_parse always records source loc info. (nread): New function. (iread): Pass true argument to rlcp_p parameter of lisp_parse_impl, so *rec-source-loc* controls whether source location info is recorded. (parse_init): Initilize rec_source_loc_s symbol variable, and register the *rec-source-loc* special var. * parser.h (struct parser): New member, rec_source_loc. (rec_source_loc_s, nread): Declared. * parser.y (rlcp_parser): New static function. Like rlcp but does nothing if parser->rec_source_loc is false. (rlc): New macro. (grammar): Replace rlcp uses with rlc, which expands to a call to rlcp_parser. (rlrec): Do nothing if source loc recording is not enabled in the parser. (make_expr, uref_helper): Replace rlcp with rlc. This is possible because these functions have a parser local variable that the macro expansion can refer to. (parse_once): Override rec_source_loc in the parser to 1, so that source loc info is always recorded when parsing is invoked through this function. * txr.1: Documented *rec-source-loc* and added text under read and iread.
* Refactoring hash bang support; hash bang null hack.Kaz Kylheku2017-05-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The hash bang mechanism is handled in one place, and disentangled from all parsing logic. It is also endowed with special powers. * eval.c (load): Pass one less argument to read_eval_stream. * match.c (v_load): Likewise. * parser.c (read_eval_stream): hash_bang_support Boolean argument removed. Hash bang logic removed. (load_rcfile): Pass only two arguments to read_eval_stream. * parser.h (read_eval_stream): Declaration updated. * txr.c (remove_hash_bang_line): Function removed. (check_hash_bang): New static function. (txr_main): Recognize the script file name while still inside the argument processing loop. Open the file, and check for a hash bang line, doing the special processing which can generate more arguments from material after a null byte in the hash bang line. The parse_stream variable is now initialized to nil and doubles as a Boolean indicating whether a stream has been opened. After the loop, we remove the script file from the arguments, if we have an open stream and the spec_file_str matches. read_eval_stream is called only with two arguments. * txr.1: Revised existing documentation and described new features.
* Continuing implementation of buffers.Kaz Kylheku2017-04-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | * Makefile (OBJS): New objects itypes.o and buf.o. * buf.c, buf.h: New files. * itypes.c, itypes.h: New files. * lib.c (obj_print_impl): Handle BUF via buf_print and buf_pprint. (init): Call itypes_init and buf_init. * parser.h (end_of_buflit): Declared. * parser.l (BUFLIT): New exclusive state. (grammar): New rules for recognizing start of buffer literal and its interior. (end_of_buflit): New function. * parser.y (HASH_B_QUOTE): New token. (buflit, buflit_items, buflit_item): New nonterminals and corresponding grammar rules. (i_expr, n_expr): These symbols now generate a buflit; a buffer literal is a kind of expression. (yybadtoken): Handle HASH_B_QUOTE case.
* Bump copyright year to 2017.Kaz Kylheku2017-01-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | * LICENSE, LICENSE-CYG, METALICENSE, Makefile, args.c, args.h, arith.c, arith.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, struct.c, struct.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/except.tl, share/txr/stdlib/getopts.tl, share/txr/stdlib/getput.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/package.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/tagbody.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl: Add 2017 to all copyright headers and strings.
* bugfix: expand macros in a number of directives.Kaz Kylheku2017-01-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | This is the last round of changes on this topic, bringing proper macro expansion to the arguments to @(skip), @(fuzz), @(next), @(call), @(cat), @(load) and @(close). * match.c (match_expand_keyword_args): Only process the keyword arguments if they are followed by an argument. Process @(next) arguments here too: :list and :string take a Lisp expression, but :tlist and :var take an argument which is not a Lisp expression and must be handled properly. Also, expand any non-keyword expression. This handles the <source> argument of @(next). (match_expand_elem): New function. * match.h (match_expand_elem): Declared. * parser.h (expand_meta): Declared. * parser.y (expand_meta): Static function becomes external. (elem): Expand elem other than require or do using match_expand_elem. We don't fold require and do into this because match_expand_elem has a backward compat switch in it that doesn't apply to these.
* parser: fix problems at EOF involving #; syntax.Kaz Kylheku2016-12-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch addresses a problem whereby if a TXR Lisp file ends with an erased object notation such as #;(a b c), there is a syntax error. The strategy is to simplify the grammar so that a single yyparse primed with SECRET_ESCAPE_E or SECRET_ESCAPE_I will read either an object, or just one instance of the #; notation. If #;OBJ is read, then the parse tree is returned as the nao value. The caller knows that #;OBJ must have occurred because there are no errors and the parser isn't at EOF, yet there is no parse tree. Then in lisp_parse we can loop on this situation, and make adjustments elsewhere also. So that iread continues to work, we must separate the parser_eof condition from the lookahead token. Under iread, we were clearing the token in prime_parser_post, but that was having the side effect of making the parser look like it is in EOF. We now preserve the EOF indication in a flag, so we can manipulate the token. * parser.h (struct parser): new member, eof. * parser.c (parser_common_init): Initialize new eof flag in parser structure to zero. (prime_parser_post): Set the eof flag if the parser's most recent token is zero. (lisp_parse_impl): Call the parser repeatedly while there are no errors, and no EOF, yet no object has been produced. This indicates that a #; erasure has been processed. (read_eval_stream): Restructure the logic here for clarity. Do not break the loop if error_val was returned from the parser, but there are no errors, and the parser isn't at EOF. This is behavior is probably redundant with respect to the loop in lisp_parse_impl. (read_eval_ret_last): Bugfixes here. Pass an error indicating value down to lisp_parse, like in read_eval_stream and make the logic similar. (parser_eof): Just return an indication based no the eof flag. * parser.y (hash_semis_n_expr, hash_semis_i_expr, ignored_i_exprs, ignored_n_exprs): Grammar rules removed. (hash_semi_or_n_expr, hash_semi_or_i_expr): New grammar rules. (spec): Retarget SECRET_ESCAPE_E and SECRET_ESCAPE_I cases to new rules. (parse): Clear eof flag to zero.
* New #; syntax for erasing following object.Kaz Kylheku2016-11-071-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parser.c (parser_circ_ref): Don't generate the circular reference if circular suppression is in effect. * parser.h (struct parser): New member, circ_suppress. We use this for suppressing the generation of circular #n# references in erased objects. * parser.l (grammar): Scan #; producing HASH_SEMI token. * parser.y (HASH_SEMI): New token. (hash_semis_n_expr, hash_semis_i_expr, ignored_i_exprs, ignored_n_exprs): New nonterminals, needed for supporting the use of #; in front of top-level forms. (spec): Use hash_semis_n_expr and hash_semis_i_expr instead of n_expr and i_expr. (r_expr): Support object erasure within nested syntax. (yybadtoken): Handle H_SEMI token. (parse): Initialize new circ_suppress member of parser struct to zero. * txr.1: Documented. * genvim.txr (txr_ign_par, txr_ign_bkt, txr_ign_par_interior, txr_ign_bkt_interior): New regions for colorizing erased objects (partial support). (txr_list, txr_bracket, txr_mlist, txr_mbrackets): Include erased objects by including regions txr_ign_par and txr_ign_bkt. * txr.vim, tl.vim: Regenerated.
* Adding notation for cycles and shared structure.Kaz Kylheku2016-10-181-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit implements the parse-side support for handling a notation that exists in ANSI Common Lisp for specifying objects with cycles and shared substructure. * parser.h (struct parser): New members, circ_ref_hash and circ_count. (circref_s, parser_resolve_circ, parser_circ_def, parser_circ_ref): Declared. * parser.c (circref_s): New symbol variable. (parser_mark): Visit the new circ_ref_hash member of the parser structure. (parser_common_init): Initialize new members circ_ref_hash and circ_count of parser structure. (patch_ref, circ_backpatch): New static functions. (parser_resolve_circ, parser_circ_def, parser_circ_ref): New functions. (circref): New static function. (parse_init): Initialize circref_s as sys:circref symbol. Register sys:circref function. * parser.l (grammar): Scan #<num>= and #<num># notation as tokens, extracting their numeric value. * parser.y (HASH_N_EQUALS, HASH_N_HASH): New token types. (i_expr, n_expr): Adding phrases for hash-equalsign and hash-hash syntax. (yybadtoken): Handle new token types in switch. (parse_once): Call parser_resolve_circ after parsing to rewrite any remaining #<num># references in the structure to the objects they denote. (parse): Reset new struct parse members to initial state. Call parser_resolve_circ after parsing to rewrite any remaining #<num># references.
* Detect cycles in rlcp_tree.Kaz Kylheku2016-10-181-0/+8
| | | | | | | | | | | | | This will be required when the parser becomes capable of creating object graphs with cycles. * parser.c (parser_callgraph_circ_check): New function. * parser.h (struct circ_stack): New struct. (parser_callgraph_circ_check): Declared. * parser.y (rlcp_tree_rec): New static function. (rlcp_tree): Reduced to wrapper for rlcp_tree_rec.
* Synchronize license comments with LICENSE.Kaz Kylheku2016-10-011-16/+17
| | | | | | | | | | | | | | | | | | | | * Makefile, args.c, args.h, arith.c, arith.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, ftw.c, ftw.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/awk.tl, share/txr/stdlib/build.tl, share/txr/stdlib/cadr.tl, share/txr/stdlib/conv.tl, share/txr/stdlib/except.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/socket.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/termios.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, socket.c, socket.h, stream.c, stream.h, struct.c, struct.h, sysif.c, sysif.h, syslog.c, syslog.h, termios.c, termios.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Revert to verbatim 2-Clause BSD.
* Copyright year bump.Kaz Kylheku2015-12-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | | * LICENSE, METALICENSE, Makefile, args.c, args.h, arith.c, arith.h, cadr.c, cadr.h, combi.c, combi.h, configure, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, glob.c, glob.h, hash.c, hash.h, jmp.S, lib.c, lib.h, lisplib.c, lisplib.h, match.c, match.h, parser.c, parser.h, parser.l, parser.y, rand.c, rand.h, regex.c, regex.h, share/txr/stdlib/cadr.tl, share/txr/stdlib/except.tl, share/txr/stdlib/hash.tl, share/txr/stdlib/ifa.tl, share/txr/stdlib/path-test.tl, share/txr/stdlib/place.tl, share/txr/stdlib/struct.tl, share/txr/stdlib/txr-case.tl, share/txr/stdlib/type.tl, share/txr/stdlib/with-resources.tl, share/txr/stdlib/with-stream.tl, share/txr/stdlib/yield.tl, signal.c, signal.h, stream.c, stream.h, struct.c, struct.h, sysif.c, sysif.h, syslog.c, syslog.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Add 2016 copyright. * linenoise/LICENSE, linenoise/linenoise.c, linenoise/linenoise.h: Bump one principal author's copyright from 2014 to 2015. The code is based on a snapshot of 2015 upstream work.
* New iread function.Kaz Kylheku2015-11-071-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The read function no longer works like it used to on an interactive terminal because of the support for .. and . syntax on a top-level expression. The iread function is provided which uses a modified syntax that doesn't support these operators on a top-level expression. The parser thus doesn't look one token ahead, and so iread can return immediately. * eval.c (eval_init): Register iread intrinsic function. * parser.c (prime_parser): Only push back the recently seen token when priming for a regular Lisp read. Handle the prime_interactive method by preparing a SECRET_ESCAPE_I token. (lisp_parse_impl): New static function, formed from previous lisp_parse. Takes a boolean argument indicating interactive mode. (prime_parser_post): New function. (lisp_parse): Now a wrapper for lisp_parse_impl which passes a nil to indicate noninteractive read. (iread): New function. * parser.h (enum prime_parser): New member, prime_interactive. (scrub_scanner, iread, prime_parser_post): Declared. * parser.l (prime_scanner): Handle the prime_interactive case the same way as prime_lisp. (scrub_scanner): New function. * parser.y (SECRET_ESCAPE_I): New token type. (i_expr): New nonterminal symbol. Like n_expr, but doesn't support dot or dotdot operators, except in nested subexpressions. (spec): Handle SECRET_ESCAPE_I by way of i_expr. (sym_helper): Before freeing the token lexeme, call scrub_scanner. If the token is registered as the scanner's most recently seen token, the scanner must forget that registration, because it is no longer valid. (parse): Call prime_parser_post. * txr.1: Documented iread.
* Better EOF check in read-eval stream function.Kaz Kylheku2015-09-111-0/+1
| | | | | | | | | | | | | | | | | | | | | This makes a difference if we read from a TTY. Without this fix, two Ctrl-D's have to be issued before the code registers EOF and returns. The trick is that we know, after calling the parser, whether it hit EOF, because we have the recent token. If its value is zero, that's the EOF token. We can bail out of the loop instead of calling the parser again which will require another EOF (because prime_parser will not push an EOF token!) * parser.h (parser_eof): Declared. * parser.c (read_eval_stream): After lisp_parse and eval, check for EOF with the parser_eof function, and bail if so. (parser_eof): New function.
* Reset flex scanner if exception thrown during read.Kaz Kylheku2015-09-061-0/+1
| | | | | | | | | | | | | | We must tear down and rebuild the Flex scanner, otherwise it is left in a bad state causing it to abort. * parser.c (parser_cleanup): After destroying scanner, null out the pointer. (parser_reset): Destroy the Flex scanner, and instantiate a new one in its place. (lisp_parse): Set up an unwind catch which will reset the parser if it was aborted by an exception. * parser.h (parser_reset): Declared.
* Syntax errors refer to REPL line number.Kaz Kylheku2015-09-051-1/+2
| | | | | | | | | | | | | | | | | | * eval.c (eval_init): Registrations of lisp-parse and read must account for new optional argument. * lib.c (func_n5o): New function. * lib.h (func_n5o): Declared. * parser.c (lisp_parse): New argument for passing in line number. This is punched into the parser object. (read_eval_stream): Call to lisp_parse defaults new argument. (repl): Pass repl line number to lisp_parse. * parser.h (lisp_parse): Declaration updated. * txr.c (txr_main): Call to lisp_parse defaults new argument.
* Basic REPL based on linenoise.Kaz Kylheku2015-09-051-0/+3
| | | | | | | | | | | | | | | | * Makefile (OBJS): Only include linenoise.o if have_termios is y. * configure: Adding test for termios. (have_termios): New configure and config.make variable. (gen_config_make): Generate have_termios variable. * parser.c (repl): New function. * parser.h (repl): Declared. * txr.c (help): Summarize new -i option. (txr_main): Implement -i repl.
* New --yydebug option.Kaz Kylheku2015-08-241-0/+2
| | | | | | | | | | | * parser.y (have_yydebug): New global constant. (yydebug_onoff): New function. * parser.h (have_yydebug, yydebug_onof): Declared. (yydebug_onoff): New function. * txr.c (help): List --yydebug option. (txr_main): --yydebug option implemented.
* Remove useless member from parser structure.Kaz Kylheku2015-08-191-1/+0
| | | | | | | * parser.c (parser_common_init): Use local variable to capture output of yylex_init, rather than yyscan structure member. * parser.h (struct parser): Member yyscan removed.
* Use new pushback token priming for single regex parse.Kaz Kylheku2015-08-121-3/+5
| | | | | | | | | | | | | | | | | | | | | | | * parser.h (enum prime_parser): New enum. (prime_parser, prime_scanner, parse): Declarations updated with new argument. * parser.c (prime_parser): New argument of enum prime_parser type Select appropriate secret token for regex and Lisp case. Pass prime selector down to prime_scanner. (regex_parse): Do not prepend secret escape to string. Do not use parse_once function; instead do the parser init and cleanup here and use the parse function. (lisp_parse): Pass new argument to parse, configuring the parser to be primed for Lisp parsing. * parser.l (grammar): Rule producing SECRET_ESCAPE_R removed. (prime_scanner): New argument. Pop the scanner state down to INITIAL. Then unconditionally switch to appopriate state based on priming configuration. * parser.y (parse): New argument for priming selection, passed down to prime parser.
* Crafting a better parser-priming hack.Kaz Kylheku2015-08-121-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The method of inserting a character sequence which generates a SECRET_TOKEN_E token is being replaced with a purely token based method. Because we don't manipulate the input stream, the lexer is not involved. We don't have to flush its state and deal with the carry-over of the yy_hold_char. This comes about because recent changes expose a weakness in the old scheme. Now that a top-level expression can have the form expr.expr, it means that the Yacc parser reads one token ahead, to see whether there is a dot or something else. This lookahead token is discarded. We must re-create it when we call yyparse again. This re-creation is done by creating a custom yylex function, which can maintain pushback tokens. We can prime this array of pushback tokens to generate the SECRET_TOKEN_E, as well as to re-inject the lookahead symbol that was thrown away by the previous yyparse. To know which lookahead symbol to re-inject is simple: the scanner just keeps a copy of the most recent token that it returns to the parser. When the parser returns, that token must be the lookahead one. The tokens we keep now in the parser structure are subject to garbage collection, and so we must mark them. Since the YYSTYPE union has no type field, a new API is opened up into the garbage collector to help implement a conservative GC technique. * gc.c (gc_is_heap_obj): New function. * gc.h (gc_is_heap_obj): Declared. * match.c: Include y.tab.h. This is now needed by any module that needs to instantiate a parser_t structure, because members of type YYSTYPE occur in the structure. (parser.h can still be included without y.tab.h, but only an incomplete declaration for the parser strucure is then given, and a few functions are not declared.) * parser.c (yy_tok_mark): New static function. (parser_mark): Mark the recent token and the pushback tokens. (parser_common_init): Initialize the recent token, the pushback tokens, and the pushback stack index. (pushback_token): New static function. (prime_parser): hold_byte argument removed. Body considerably simplified. The catenated stream trick is no longer required. All we do here is set up two pushback tokens and prime the scanner, if necessary, so it is in the right start state for Lisp. * parser.l (YY_DECL): Take over definition of scanning function, renaming to yylex_impl, so we can implement yylex. (grammar): Rule which produces SECRET_ESCAPE_E token removed. (reset_scanner): Function removed. (yylex): New function. * parser.h (struct parser): Now only forward-declared unless y.tab.h has been included. New members, recent_tok, tok_pushback and tok_idx. (yyset_hold_char): Declared. (reset_scanner): Declaration removed. (yylex): Declared (if y.tab.h included). (prime_parser): Declaration updated. (prime_scanner): Declared. * Makefile: express new dependency on existence of y.tab.h of txr.o, match.o and parser.o.
* Diagnose bad consing dot syntax like (a . b . c).Kaz Kylheku2015-08-101-0/+1
| | | | | | | | | | | | | | | | | | | | | * parser.y (r_exprs): Use unique object in the terminating cons to indicate the empty spot where the dotted cdr item will go. Check for misplaced consing dot. (misplaced_consing_dot_check): New static function. Checks for the terminator atom spot being taken already. Thus, the spot may be taken only by the very last reduction, such that the next reduction is r_exprs -> n_exprs where the terminating atom is processed. * parser.c (unique_s): New global variable. (parse_init): Initialize unique_s. * parser.h (unique_s): Declared. * share/txr/stdlib/place.tl (sys:placelet-1): We have a misplaced consing dot here! It was working correctly by "terminating atom propagation" behavior, which allowed (a . b c d) to produce (a c d . b). If a single terminating atom occurred in the middle of a list, it was promoted to the end.
* * eval.c (force): Default the new second argument of source_loc_str.Kaz Kylheku2015-08-041-1/+1
| | | | | | | | | | | | | | | | | | | (eval_error): Derive location of error from the last_form_evaled, if form doesn't have it. (eval_init): Re-register source-loc-str as binary with an optional arg. * match.c (debuglf, sem_error, file_err, typed_error): Default new argument of source_loc_str. * parser.h (source_loc_str): Declaration updated. * parser.l (source_loc_str): Take second argument which specifies alternative value if the source loc info is not found. * unwind.c (uw_throw): Simplify code thanks to source_loc_str default argument. * txr.1: Document new argument of source-loc-str.
* Handle setting of parse name through prime_parser.Kaz Kylheku2015-07-101-2/+2
| | | | | | | | | | | | * parser.c (prime_parser): Take name as argument, and install it into parser. (lisp_parser): Pass name to parse, instead of setting it in the parser object. * parser.y (parse): Take name as argument and pass down to prime_parser. * parser.h (prime_parser, parse): Declarations updated.
* Remove unused "primer" member from parser_t.Kaz Kylheku2015-07-101-2/+1
| | | | | | | | | | | * parser.c (parser_mark, parser_common_init): Remove reference to primer. (parser): Don't take primer argument, remove reference to member. (ensure_parser): Don't take primer argument, don't pass to parser function. (lisp_parse): Don't pass primer string to ensure_parser. * parser.h (primer_t): Remove primer member. (parser): Declaration updated.
* Bugfix: lexer loses unmatched "hold char" between top-level forms.Kaz Kylheku2015-07-101-2/+2
| | | | | | | | | | | | | | | | | | Test case: file containing 4(prinl 3). Scanner consumes 4 and (. The ( is lost when the scanner is reset for the next call to yyparse, resulting in jut prinl being read and interpreted as a variable. * parser.c (prime_parser): If present, append hold byte to priming string. Takes parser_t * instead of parser, and returns void now. * parser.l (reset_scanner): Now returns int value, the value of the scanner's yy_hold_char variable which is nonzero when the scanner is hanging on to an unmatched byte of input. * parser.h (reset_scanner, prime_parser): Declarations updated. * parser.y (parse): Pass hold byte returned by reset_scanner to prime_parser.
* Parser cleanup: embed scanner in parser.Kaz Kylheku2015-07-091-5/+9
| | | | | | | | | | | | | | | | | | | | | | | * parser.c (parser_destroy): New GC finalizer static function. (parser_ops): Register parser_destroy. (parser_common_init): New function, shared by parse and parse_once. Initializes embedded scanner. (parser_cleanup): New function, shared by parse_once and parser_destroy. (parser): Use parser_common_init. * parser.h (parser_t): New member, yyscan. (reset_scanner, parser_common_init): Declared. * parser.l (reset_scanner): New function. * parser.y (parse_once): Use parser_common_init, and thus perform only a few initializations. Do not define scanner as a local variable. (parse): Call reset_scanner instead of yylex_init since the scanner is being reused, and for the same reason do not call yylex_destroy. GC will do that now.
* Hash-bang support for .tl files.Kaz Kylheku2015-07-021-1/+1
| | | | | | | | | | | | | | | | * parser.c (read_eval_stream): New boolean argument to request hash bang support. * parser.h (read_eval_stream): Declaration updated. * eval.c (sys_load): Pass new thid argument to read_eval_stream, to decline hash bang support. * match.c (v_load): Likewise. * txr.c (txr_main): Request hash bang support from read_eval_stream. Thus files referenced from the txr command line can have a #! line, which is ignored.
* @(load) and @(include) now load Lisp code.Kaz Kylheku2015-06-121-0/+3
| | | | | | | | | | | * match.c (v_load): Check txr_lisp_p flag coming out of open_txr_file and handle the Lisp case usin read_eval_stream. * parser.c (read_eval_stream, get_parser, parser_errors): New functions. * parser.h (read_eval_stream, get_parser, parser_errors): Declared.
* Preparing for lisp loading.Kaz Kylheku2015-06-101-1/+1
| | | | | | | | | | | | | * parser.c (open_txr_file): Rewritten to take new argument which indicates whether to treat an unsuffixed file as TXR or TXR Lisp, and is updated to indicate which is the case by looking at the suffix. * parser.h (open_txr_file): Declaration updated. * match.c (v_load): Follow change in open_txr_file. * txr.c (txr_main): Likewise.
* * parser.c (stream_parser_hash): New static variable.Kaz Kylheku2015-06-071-1/+4
| | | | | | | | | | | | | | | | | | (parser_mark): Mark parser and primer members. (parser, ensure_parser): new argument: primer. (get_parser_impl, ensure_parser): New static functions. (prime_parser): New function. (lisp_parse): Multiple calls to this function on the same stream now logically continue the parse, not resetting the line number to 1. (parse_init): Initialize and gc-protect stream_parser_hash. * parser.h (parser_t): New members, primer and parser. (prime_parser): Declared. (parser): Declaration updated. * parser.y (parse): Now responsible for calling prime_parser.
* * match.c (v_load): Call parse_once rater than parse.Kaz Kylheku2015-06-071-1/+2
| | | | | | | | | | | | | * parser.c (regex_parse): Likewise. * txr.c (txr_main): Likewise. * parser.h (parse): Declaration updated. (parse_once): Declared. * parser.y (parse_once): New function, same as old parse implementation. (parse): Becomes one argument function which works with a previously initialized parser and continues the parse.
* * Makefile (LISP_TO_C_STRING): Strip comments, but not comment lines,Kaz Kylheku2015-05-071-1/+1
| | | | | | | | | | | | | | | | | | so line numbers don't change. * eval.c (eval_init): Fix registrations of lisp-parse and read. * lisplib.c (place_instantiate): Give name to parsed string stream using new lisp_parse argument. * parser.c (lisp_parse): Takes new argument to override name. * parser.h (lisp_parse): Declaration updated. * txr.c (txr_main): Call lisp_parse with four args, defaulting the new one. * txr.1: Documented new argument.
* * Makefile (OBJS): Add parser.o.Kaz Kylheku2014-12-211-1/+4
| | | | | | | | | | * parser.h (parser_s): Declared. (parse_init): Declaration removed. (parser_l_init): Declared. * parser.l (parse_init): Function renamed to parser_l_init. * parser.c: New file.
* Update copyright notices from 2014 to 2015.Kaz Kylheku2015-02-011-1/+1
| | | | | | | | | | | * arith.c, arith.h, combi.c, combi.h, debug.c, debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, hash.c, hash.h, lib.c, lib.h, match.c, match.h, parser.h, rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, stream.h, sysif.c, sysif.h, syslog.c, syslog.h, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: Update. * LICENSE, METALICENSE: Likewise.
* Purge stray occurrences of "void *" from code base.Kaz Kylheku2014-10-171-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * lib.c (cobj_print_op): In the format call, cast the C pointer to val, since the ~p conversion now takes a val rather than void *. (cptr_equal_op, obj_print, obj_pprint): Remove cast to void *, since obj now already has the type that ~p expects. * lib.h (struct any): Use mem_t * instead of void *. * parser.h (yyscan_t): Repeat the definition from inside the Flex-generated lex.yy.c. (yylex_init, yylex_destroy, yyget_extra, yyset_extra): Re-declare using yyscan_t typedef in place of void *. * parser.l (yyget_column, yyerrprepf): Re-declare using yyscan_t. (grammar): Use yyg in place of yyscanner in calls to yyerrprepf. * parser.y (yylex, %lex-param): Use yyscan_t instead of void *. (parse): Use yyscan_t for local variable. * signal.c (stack): Change type from void * to mem_t *. * stream.c (vformat): Conversion specifier p extracts val instead of void *. (run): Use casts that only remove const, not all the way to void *. * txr.1: Documented p conversion specifier of format. * Makefile (OBJS-y): Initialize with := to make sure it is a simple variable, and not a macro. (SRCS): New variable, listing source files. (enforce): New rule for enforcing coding conventions. Currently checks for void * occurrences.
* More type safety, with help from C++ compiler.Kaz Kylheku2014-10-141-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | * parser.h (scanner_t): New typedef. Cleverly, we use yyguts_t as the basis for this, which pays off in a few places, getting rid of conversions. (parser_t): scanner member redeclared to scanner_t *. (yyerror, yyerr, yyerrof, end_of_regex, end_of_char): Declarations updated. * parser.l (yyerror, yyerrorf, num_esc): scanner argument becomes scanner_t * instead of void *. (yyscanner): Occurrences replaced by yyg: why should we refer to the type-unsafe void *yyscanner, when we know that in the middle of the yylex function we have the properly typed yyguts_t *yyg. (end_of_regex, end_of_char): Argument changes to scanner_t * and name strategically changes to yyg, eliminating the local variable and cast. * parser.y (sym_helper): scanner argument becomes scanner_t * instead of void *. (%parse-param}: void *scnr becomes scanner_t *scnr. (define-transform, yybadtoken): Use scanner_t * instead of void *. (parse): Since yylex_init takes a void **, we use it to initialize a local void *, and then assign it to the structure member.
* C++ upkeep.Kaz Kylheku2014-10-141-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | TXR's support for compiling as C++ pays off: C++ compiler finds serious bugs introduced in August 2 ("Big switch to reentrant lexing and parsing"). The yyerror function was being misused; some of the calls reversed the scanner and parser arguments. Since one of the two parameters is void *, this reversal wasn't caught. * parser.l (yyerror): Fix first two arguments being reversed. (num_esc): Change previously correct call to yyerror to follow reversed arguments, so that it stays correct. * parser.y (%parse-param): Change order of these directives so that the scnr parameter is before the parser parameter. This causes the yacc-generated calls to yyerror to have the arguments in the correct order. It also has the effect of changing the signature of yyparse, reversing its parameters. (parse): Update call to yyparse to new argument order. * parser.h (yyparse): Declaration removed. (yyerror): Declaration updated. * regex.c (regex_kind_t): New enum typedef. (struct regex): Use regex_kind_t rather than an enum inside the struct, which has different scoping rules under C++. * txr.c (get_self_path): Fix signed/unsigned warning.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-09 22:41:05 +0000