summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Crafting a better parser-priming hack.Kaz Kylheku2015-08-129-41/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The method of inserting a character sequence which generates a SECRET_TOKEN_E token is being replaced with a purely token based method. Because we don't manipulate the input stream, the lexer is not involved. We don't have to flush its state and deal with the carry-over of the yy_hold_char. This comes about because recent changes expose a weakness in the old scheme. Now that a top-level expression can have the form expr.expr, it means that the Yacc parser reads one token ahead, to see whether there is a dot or something else. This lookahead token is discarded. We must re-create it when we call yyparse again. This re-creation is done by creating a custom yylex function, which can maintain pushback tokens. We can prime this array of pushback tokens to generate the SECRET_TOKEN_E, as well as to re-inject the lookahead symbol that was thrown away by the previous yyparse. To know which lookahead symbol to re-inject is simple: the scanner just keeps a copy of the most recent token that it returns to the parser. When the parser returns, that token must be the lookahead one. The tokens we keep now in the parser structure are subject to garbage collection, and so we must mark them. Since the YYSTYPE union has no type field, a new API is opened up into the garbage collector to help implement a conservative GC technique. * gc.c (gc_is_heap_obj): New function. * gc.h (gc_is_heap_obj): Declared. * match.c: Include y.tab.h. This is now needed by any module that needs to instantiate a parser_t structure, because members of type YYSTYPE occur in the structure. (parser.h can still be included without y.tab.h, but only an incomplete declaration for the parser strucure is then given, and a few functions are not declared.) * parser.c (yy_tok_mark): New static function. (parser_mark): Mark the recent token and the pushback tokens. (parser_common_init): Initialize the recent token, the pushback tokens, and the pushback stack index. (pushback_token): New static function. (prime_parser): hold_byte argument removed. Body considerably simplified. The catenated stream trick is no longer required. All we do here is set up two pushback tokens and prime the scanner, if necessary, so it is in the right start state for Lisp. * parser.l (YY_DECL): Take over definition of scanning function, renaming to yylex_impl, so we can implement yylex. (grammar): Rule which produces SECRET_ESCAPE_E token removed. (reset_scanner): Function removed. (yylex): New function. * parser.h (struct parser): Now only forward-declared unless y.tab.h has been included. New members, recent_tok, tok_pushback and tok_idx. (yyset_hold_char): Declared. (reset_scanner): Declaration removed. (yylex): Declared (if y.tab.h included). (prime_parser): Declaration updated. (prime_scanner): Declared. * Makefile: express new dependency on existence of y.tab.h of txr.o, match.o and parser.o.
* Catenated stream reads must not close last stream.Kaz Kylheku2015-08-121-6/+9
| | | | | | * stream.c (cat_get_line, cat_get_char, cat_get_byte) Only close the head stream when popping it off the list.
* Diagnose ambiguous floats like (a b).4 and x.y.5Kaz Kylheku2015-08-101-0/+30
| | | | | | | | These look like integers involved in qref dot syntax. * parser.l (DOTFLO): New pattern definition. (grammar): New rules for detecting cramped floating literals.
* Dot with no whitespace generates qref syntax.Kaz Kylheku2015-08-105-6/+71
| | | | | | | | | | | | | | | | | | | | | | | a.b.(expr ...).c -> (qref a b (expr ...) c) Consing dot requires whitespace. * eval.c (qref_s): New symbol global variable. (eval_init): Initialize qref_s. * eval.h (qref_s): Declared. * parser.l (REQWS): New pattern definition, required whitespace. (grammar): New rules to scan CONSDOT (space required on both sides) and LAMBDOT (space required after). * parser.y (CONSDOT, LAMBDOT): New token types. (list): (. n_expr) rule replaced with LAMBDOT and CONSDOT. (r_exprs): r_exprs . n_expr consing dot rule replaced with CONSDOT. (n_expr): New n_expr . n_expr rule introduced here for producing qref expressions. (yybadtoken): Handle CONSDOT and LAMBDOT. * txr.1: Documented qref dot.
* Count East Asian Wide and Full Fidth chars as two columns.Kaz Kylheku2015-08-104-7/+74
| | | | | | | | | | | | | | | | | * regex.c (create_wide_cs): New static function. (wide_display_char_p): New function. * regex.h (wide_display_char_p): Declared. * stream.c (put_string, put_char): Use wide_display_char_p to determine whether an extra column need be counted. Also bugfix: iswprint evidently cannot be relied to work over the entire Unicode range, at least not in the C locale. Glibc's version and is reporting valid Japanese characters as unprintable on Ubuntu. As a hack we instead check for control characters and invert the result: control chars are unprintable. * tests/009/json.expected: Updated.
* Handle abc: token syntax.Kaz Kylheku2015-08-101-2/+2
| | | | | | * parser.l (BTREG, NTREG): Allow an empty string symbol name with a nonempty package name. Without this, abc: parses as abc :.
* Diagnose bad consing dot syntax like (a . b . c).Kaz Kylheku2015-08-104-9/+27
| | | | | | | | | | | | | | | | | | | | | * parser.y (r_exprs): Use unique object in the terminating cons to indicate the empty spot where the dotted cdr item will go. Check for misplaced consing dot. (misplaced_consing_dot_check): New static function. Checks for the terminator atom spot being taken already. Thus, the spot may be taken only by the very last reduction, such that the next reduction is r_exprs -> n_exprs where the terminating atom is processed. * parser.c (unique_s): New global variable. (parse_init): Initialize unique_s. * parser.h (unique_s): Declared. * share/txr/stdlib/place.tl (sys:placelet-1): We have a misplaced consing dot here! It was working correctly by "terminating atom propagation" behavior, which allowed (a . b c d) to produce (a c d . b). If a single terminating atom occurred in the middle of a list, it was promoted to the end.
* Remove never-used member of symbol structure.Kaz Kylheku2015-08-093-3/+0
| | | | | | | | * lib.h (struct sym): Remove value member. * lib.c (make_sym): Don't initialize removed member. * gc.c (mark_obj): Do not mark removed member.
* Discontinuing ChangeLog.Kaz Kylheku2015-08-081-0/+8
| | | | | | * ChangeLog: renamed to ChangeLog-2009-2015, and discontinued. Changes will be tracked only in the git commit messages from now on and not duplicated into the ChangeLog.
* Version 111.txr-111Kaz Kylheku2015-08-087-210/+311
|
* C++ upkeep: conversions, clashes, warnings.Kaz Kylheku2015-08-074-5/+16
| | | | | | | | | * glob.c (glob_wrap): Fix signed/unsigned comparison. * stream.c (make_null_stream): Fix convert beign used where coerce is needed. * sysif.c (dup_wrap): Fix use of C++ new keyword.
* C++ static forward issue.Kaz Kylheku2015-08-073-3/+21
| | | | | | | | * lib.h (static_forward, static_def): New macros for dealing with C++ static forward declaration problem. * syslog.c (syslog_strm_ops): Use static forward macros.
* C++ upkeep: resolve multiple definitions of fun_k.Kaz Kylheku2015-08-076-7/+23
| | | | | | | | | | | | | | | * eval.c (fun_k): Global definition removed. (eval_init): Do not initialize fun_k here. * filter.c (fun_k): Definition removed. (filter_init): Do not initialize fun_k. * filter.h (fun_k): Declaration removed. * lib.c (fun_k): Defined in this file now. (obj_init): Initialize fun_k here. * lib.h (fun_k): Declare here.
* * Makefile (SRCS): When top_srcdir is blank, elide the entireKaz Kylheku2015-08-072-1/+8
| | | | | --work-tree argument to git. On Cygwin, git throws a strange error message when --work-tree is given a blank argument.
* * stream.c: Include <wctype.h> header for iswprint.Kaz Kylheku2015-08-072-0/+6
| | | | Needed on Cygwin.
* * sysif.c (w_lstat): If we don't have S_IFLINK, implementKaz Kylheku2015-08-072-0/+9
| | | | w_lstat as an alias to w_stat. This helps build on MinGW.
* Change to different exception for debugger quit.Kaz Kylheku2015-08-072-1/+12
| | | | | | * debug.c (debug_quit_s): New global variable. (debug): Throw debug-quit, not query-error. (debug_init): Initialize debug_quit_s.
* * debug.c (help): Rearrange menu. Show missing quit command.Kaz Kylheku2015-08-062-3/+6
|
* Suppress debug stepping into auto-loaded library code.Kaz Kylheku2015-08-067-8/+86
| | | | | | | | | | | | | | | | | | | | | * debug.c (debug_set_state, debug_restore_state): New functions. * debug.h (debug_state_t): New type. (debug_set_state, debug_restore_state): Declared, and defined as dummy macros in non-debug-support build. * lisplib.c (opt_dbg_autoload): New global variable. (lisplib_try_load): Disable or enable debugging around library loading based on opt_dbg_autoload option. * lisplib.h (opt_dbg_autoload): Declared. * txr.c (help): List --debug-autoload option. (no_dbg_support): New static function to avoid repeated code. (txr_main): Add debugger option. Change duplicate no debug support error messages into calls to no_dbg_support. * txr.1: Document --debug-autoload
* * txr.c (txr_main): Bugfix: debugger long option nonfunctional.Kaz Kylheku2015-08-062-1/+5
|
* Better diagnosis for loose @ forms.Kaz Kylheku2015-08-063-4/+27
| | | | | | | | | | | | | * eval.c (op_meta_error): New static function. (eval_init): Register sys:var and sys:expr as operators that throw error. * parser.y (sym_helper): Take parser_t instead of scanner_t argument so we have access to the name and line number. Obtain scanner internally from parser. Add source location info to (sys:var ...) form. (symhlpr): Retarget macro to pass parser rather than scanner to sm_helper.
* * txr.1: Formatting issue in in-line code block under giterate.Kaz Kylheku2015-08-051-1/+1
|
* * txr.1: pppred formatting problem.Kaz Kylheku2015-08-051-1/+1
|
* New filesystem object testing functions.Kaz Kylheku2015-08-054-0/+339
| | | | | | | | | | * lisplib.c (path_test_set_entries, path_test_instantiate): New static functions. (dlt_register): Registered new functions to dl_table. * txr.1: Documented new functions. * share/txr/stdlib/path-test.tl: New file.
* Adding support for uid and gid manipulation.Kaz Kylheku2015-08-054-0/+187
| | | | | | | | | | | | * configure: Added check for geteuid and related functions. * sysif.c (getuid_wrap, geteuid_wrap, getgid_wrap, getegid_wrap, getgroups_wrap, setuid_wrap, seteuid_wrap, setgid_wrap, setegid_wrap): New static functions. (sysif_init): Register intrinsics getuid, geteuid, getgid, getegid, getgroups, setuid, seteuid, setgid, setegid. * txr.1: Documented new functions.
* New exception type: system-error.Kaz Kylheku2015-08-054-0/+13
| | | | | | | | | * lib.c (system_error_s): New symbol variable. (obj_init): Initialize new variable. * lib.h (system_error_s): Declared. * unwind.c (uw_init): Register system-error exception type.
* * stream.c (stream_init): Register get-indent-mode, set-indent-mode,Kaz Kylheku2015-08-053-1/+163
| | | | | | | | test-set-indent-mode, get-indent, set-indent, inc-indent and width-check intrinsic functions. Register indent-off, indent-data and indent-code variables. * txr.1: Documented stream output indentation API.
* * stream.c (width_check): Just use the publicKaz Kylheku2015-08-052-11/+10
| | | | | put_char function; no need to manipulate column or call put_indent.
* Better diagnostic in funcall family of functions.Kaz Kylheku2015-08-052-5/+22
| | | | | | * lib.c (wrongargs): New static function. (funcall, funcall2, funcall2, funcall3, funcall4): Use wrongargs.
* * eval.c (do_eval): Bugfix: though last_form_evaled isKaz Kylheku2015-08-052-0/+7
| | | | | saved and restored around the execution of a special operator, it is never set the current form.
* * eval.c (bind_args): Use new ~! for proper indentationKaz Kylheku2015-08-043-9/+17
| | | | | | | of multi-line context form. (apply): Use ~! for proper indentation of function code. * unwind.c (uw_throw): Use ~! for proper indentation of code.
* * stream.c (put_string): In indent mode, put_string hasKaz Kylheku2015-08-042-6/+16
| | | | | | | | to process all the characters as if by put_char, (which we now do literally that way). (set_indent_mode): Bugfix: no longer reset the column to zero when turning off indent mode. This is wrong since streams do column counting all the time.
* * stream.c (vformat): Implement ~! format directive for indentation.Kaz Kylheku2015-08-043-10/+80
| | | | | | | Allow negative widths to be specified with a leading minus sign, so that we can indent to the left. * txr.1: Document ~! format directive.
* * stream.c (put_string, put_char): Do not put out the indentationKaz Kylheku2015-08-042-2/+19
| | | | | immediately after outputting a newline. Rather, delay the output of the indentation until some output occurs at column zero.
* * stream.c (inc_indent): If a negative indentation increment goes belowKaz Kylheku2015-08-042-0/+10
| | | | | zero, clamp it at zero. (set_indent): Clamp indentation value to zero.
* * stream.c (vormat): Bugfix: when width specified as *Kaz Kylheku2015-08-042-1/+8
| | | | | | meets a negative argument, the width should be treated as positive and the field left aligned. Instead, the width is treated as zero.
* * stream.c (string_out_put_string): Do not return nil whenKaz Kylheku2015-08-042-2/+9
| | | | buffer calculations overflow, but throw exception.
* Remove useless return values and checks.Kaz Kylheku2015-08-043-78/+63
| | | | | | | | | | | | | | | * stream.c (vformat_align_pre, vformat_align_post): Change to void return. Do not check return value of put_char. (vformat_num, vformat_str): Change to void return. Do not check return value of vformat_align_pre, vformat_aign_post or put_char. (vformat): Do not check return value of vformat_str or vformat_num. nilout exit point no longer needed. (put_string): Do not intercept return value of ops->put_string. Return t. (put_char): Do not intercept return value of ops->put_char. Do not check return value of put_indent. Return t. * txr.1: Document t return of put-char, put-byte and put-string.
* * eval.c (func_get_name): New function.Kaz Kylheku2015-08-044-7/+84
| | | | | | | | | | | | (bind_args): Include the entire context form in argument mismatch errors. (apply): Include the function name, or else source code if it has no name, in argument mismatch erors. (eval_init): Register func-get-name intrinsic. * eval.h (func_get_name): Declared. * txr.1: Documented func-get-name.
* * eval.c (force): Default the new second argument of source_loc_str.Kaz Kylheku2015-08-047-22/+47
| | | | | | | | | | | | | | | | | | | (eval_error): Derive location of error from the last_form_evaled, if form doesn't have it. (eval_init): Re-register source-loc-str as binary with an optional arg. * match.c (debuglf, sem_error, file_err, typed_error): Default new argument of source_loc_str. * parser.h (source_loc_str): Declaration updated. * parser.l (source_loc_str): Take second argument which specifies alternative value if the source loc info is not found. * unwind.c (uw_throw): Simplify code thanks to source_loc_str default argument. * txr.1: Document new argument of source-loc-str.
* * hash.c (hash_revget): New function.Kaz Kylheku2015-08-045-0/+72
| | | | | | | | * hash.h (hash_revget): Declared. * eval.c (eval_init): Registered hash-revget intrinsic. * txr.1: Documented hash-revget.
* * stream.c (indent_mode_put_string): Function removed,Kaz Kylheku2015-08-043-64/+59
| | | | | | | logic hoisted into put_string. (put_string, put_char): Always count column, indent mode or not. * tests/009/json.expected: Updated.
* Define TXR_DBG macro when compiling debug build.Kaz Kylheku2015-08-033-3/+22
| | | | | | | | | * Makefile (dbg/%.o): Pass $(DBG_ONLY_FLAGS) to COMPILE_C_WITH_DEPS recipe macro. * configure (debug_only_flags): New macro. (gen_config_make): Generate DBG_ONLY_FLAGS. Add debug-only-flags to usage help text.
* Custom linker flags support.Christopher Meng2015-08-033-5/+27
| | | | | | | | | | | | * Makefile (PLATFORM_FLAGS): Rename to PLATFORM_CFLAGS. (LDFLAGS): New variable. Includes -lm and PLATFORM_CFLAGS. (LINK_PROG): Use $(LDFLAGS) instead of -lm. * configure (platform_flags): Renamed to platform_cflags. (platform_ldflags): New variable. (gen_config_make): Rename PLATFORM_CFLAGS and generate PLATFORM_LDFLAGS. Usage help text updated.
* Switching some globals to lexical and changing some names.Kaz Kylheku2015-08-0313-158/+240
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * arith.c (arith-init): Changing *flo-...* from special to lexical, and adding un-earmuffed variants. The earmuffed versions are obsolescent. Adding %pi% and %e% global lexicals. Earmuffed versions are also made global lexical, and obsolescent. * eval.c (lookup_global_var, lookup_global_var_l): New functions. (lookup_var): Uses lookup_global_var. (reg_varl): New function. (reg_var): Uses reg_var. (eval_init): Register global lexicals user-package, system-package and keyword-package. Old symbols with earmuffs are obsoleted, and also turned into global lexicals. (top-vb, top-fb): Changed to lexical. * eval.h (lookup_global_var, lookup_global_var_l, reg_varl): Declared. * genvim.txr: Scan ver.tl so that the lib-version variable is included. Extract reg_varl calls. * glob.c (glob_init): glob-err and other variables made lexical. * lib.c (get_user_package, get_system_package, get_keyword_package): Use lookup_global_var_l to avoid searching dynamic scope for lexicals which cannot be dynamically rebound. * share/txr/stdlib/ver.tl (lib-version): New global lexical variable. (*lib-version*): Turned lexical. Obsolescent. * signal.c (sig-init): sig-* variables turned lexical. * sysif.c (sysif-init): s-*, poll-* and w-* variables turned lexical. * syslog.c (syslog-init): log-* variables turned lexical. * txr.c (sysroot-init): stdlib and *txr-version* variables turned lexical. txr-version variable added, and *txr-version* is obsolescent. (txr-main): self-path variable added. *self-path* turns lexical and is obsolescent. * txr.1: Documentation updated. Lexical variables not referred to as special. Special variables referred to as special.
* * txr.c (txr_main): Bugfix: register *args-full*, the variableKaz Kylheku2015-08-032-0/+7
| | | | | described in the documentation! We keep *full-args* for backward compatibility, as an obsolescent feature.
* Introducing global lexicals: defvarl, defparml.Kaz Kylheku2015-08-033-19/+102
| | | | | | | | | | | | | | | | | | | | | * eval.c (defvarl_s, defparm_s, defparml_s, sys_mark_special_s): New symbol variables. (mark_special): Return val rather than void, since it's hoisted into Lisp domain now with func_n1. (op_defvar): Renamed to op_defvarl, and doesn't call mark_special. defvarl is now the special form for defining variables, and special marking is an embellishment added by macros. (me_defparm): Renamed to me_def_variable. Handles defvar, defparm and defparml. (do_expand): Expand defvarl, rather than defvar. (eval_init): Intern defvarl, defparm, defparml and mark-special symbols, and initialize corresponding globals. Register defvarl special operator, retiring defvar. Register defparm to me_def_variable, and register defvar and defparml to the same. * txr.1: Document support for global lexical scopes and the new macros.
* Support lstat and fstat.Kaz Kylheku2015-08-024-14/+106
| | | | | | | | | | | | | | | * stream.h (statf): Declaration removed. * sysif.c (w_stat): Function takes val instead of const wchar_t * as leftmost argument. (w_lstat, w_fstat): New static functions, with same interface as w_stat. (stat_to_list, stat_impl): New static functions. (statp, statl): New static functions. (statf): Function removed, name re-used for new static function. (sysif_init): stat intrinsic registered to statp function, not statf. lstat and fstat intrinsics registered to statl and statf. * txr.1: Documented lstat and fstat.
* Pass pretty flag to cobj print operation.Kaz Kylheku2015-08-017-16/+46
| | | | | | | | | | | | | | | | | | | | | * hash.c (hash_print_op): Take third argument, and call cobj_print_impl rather than cobj_print. * lib.c (cobj_print_op): Take third argument. The object class is * printed with obj_print_impl. (obj_print_impl): Static function becomes extern. Passes its pretty flag argument to cobj print virtual function. * lib.h (cobj_ops): print takes third argument. (cobj_print_op): Declaration updated. (obj_print_impl): Declared. * regex.c (regex_print): Takes third argument, and ignores it. * stream.c (stream_print_op, stdio_stream_print, cat_stream_print): Take third argument, and ignore it. * stream.h (stream_print_op): Declaration updated.
* * eval.c (special_p): Function renamed to special_var_p.Kaz Kylheku2015-08-013-7/+52
| | | | | | | | (expand_opt_params_rec, expand_vars): Follow rename of special_p. (eval_init): Register special-operator-p and special-var-p intrinsics. * txr.1: Document special-operator-p and special-var-p.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-10 14:21:40 +0000