diff options
author | Kaz Kylheku <kaz@kylheku.com> | 2015-08-08 21:28:23 -0700 |
---|---|---|
committer | Kaz Kylheku <kaz@kylheku.com> | 2015-08-08 21:28:23 -0700 |
commit | ac71292135de0f3a18b7d462bcc9379c0fe142d9 (patch) | |
tree | 48058d6ac300b56604b9c2241b669209c86047d1 /ChangeLog-2009-2015 | |
parent | 6bc9935f204d6a79fe4523dd4c1fd287fd240b35 (diff) | |
download | txr-ac71292135de0f3a18b7d462bcc9379c0fe142d9.tar.gz txr-ac71292135de0f3a18b7d462bcc9379c0fe142d9.tar.bz2 txr-ac71292135de0f3a18b7d462bcc9379c0fe142d9.zip |
Discontinuing ChangeLog.
* ChangeLog: renamed to ChangeLog-2009-2015, and discontinued.
Changes will be tracked only in the git commit messages
from now on and not duplicated into the ChangeLog.
Diffstat (limited to 'ChangeLog-2009-2015')
-rw-r--r-- | ChangeLog-2009-2015 | 22389 |
1 files changed, 22389 insertions, 0 deletions
diff --git a/ChangeLog-2009-2015 b/ChangeLog-2009-2015 new file mode 100644 index 00000000..08a518a4 --- /dev/null +++ b/ChangeLog-2009-2015 @@ -0,0 +1,22389 @@ +2015-08-08 Kaz Kylheku <kaz@kylheku.com> + + Discontinuing ChangeLog. + + * ChangeLog: renamed to ChangeLog-2009-2015, and discontinued. + Changes will be tracked only in the git commit messages + from now on and not duplicated into the ChangeLog. + +2015-08-08 Kaz Kylheku <kaz@kylheku.com> + + Version 111. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version and date. + + * share/txr/stdlib/ver.tl: Likewise. + + * tl.vim, txr.vim: Regenerated. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + C++ upkeep: conversions, clashes, warnings. + + * glob.c (glob_wrap): Fix signed/unsigned comparison. + + * stream.c (make_null_stream): Fix convert beign used + where coerce is needed. + + * sysif.c (dup_wrap): Fix use of C++ new keyword. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + C++ static forward issue. + + * lib.h (static_forward, static_def): New macros + for dealing with C++ static forward declaration problem. + + * syslog.c (syslog_strm_ops): Use static + forward macros. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + C++ upkeep: resolve multiple definitions of fun_k. + + * eval.c (fun_k): Global definition removed. + (eval_init): Do not initialize fun_k here. + + * filter.c (fun_k): Definition removed. + (filter_init): Do not initialize fun_k. + + * filter.h (fun_k): Declaration removed. + + * lib.c (fun_k): Defined in this file now. + (obj_init): Initialize fun_k here. + + * lib.h (fun_k): Declare here. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (SRCS): When top_srcdir is blank, elide the entire + --work-tree argument to git. On Cygwin, git throws a strange error + message when --work-tree is given a blank argument. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + * stream.c: Include <wctype.h> header for iswprint. + Needed on Cygwin. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + * sysif.c (w_lstat): If we don't have S_IFLINK, implement + w_lstat as an alias to w_stat. This helps build on MinGW. + +2015-08-07 Kaz Kylheku <kaz@kylheku.com> + + Change to different exception for debugger quit. + + * debug.c (debug_quit_s): New global variable. + (debug): Throw debug-quit, not query-error. + (debug_init): Initialize debug_quit_s. + +2015-08-06 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (help): Rearrange menu. Show missing quit command. + +2015-08-06 Kaz Kylheku <kaz@kylheku.com> + + Suppress debug stepping into auto-loaded library code. + + * debug.c (debug_set_state, debug_restore_state): New functions. + + * debug.h (debug_state_t): New type. + (debug_set_state, debug_restore_state): Declared, and defined + as dummy macros in non-debug-support build. + + * lisplib.c (opt_dbg_autoload): New global variable. + (lisplib_try_load): Disable or enable debugging around + library loading based on opt_dbg_autoload option. + + * lisplib.h (opt_dbg_autoload): Declared. + + * txr.c (help): List --debug-autoload option. + (no_dbg_support): New static function to avoid repeated code. + (txr_main): Add debugger-autoload option. Change duplicate no debug + support error messages into calls to no_dbg_support. + + * txr.1: Document --debug-autoload + +2015-08-06 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Bugfix: debugger long option nonfunctional. + +2015-08-06 Kaz Kylheku <kaz@kylheku.com> + + Better diagnosis for loose @ forms. + + * eval.c (op_meta_error): New static function. + (eval_init): Register sys:var and sys:expr as operators + that throw error. + + * parser.y (sym_helper): Take parser_t instead of scanner_t + argument so we have access to the name and line number. + Obtain scanner internally from parser. Add source location + info to (sys:var ...) form. + (symhlpr): Retarget macro to pass parser rather than scanner to + sm_helper. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + New filesystem object testing functions. + + * lisplib.c (path_test_set_entries, path_test_instantiate): + New static functions. + (dlt_register): Registered new functions to dl_table. + + * txr.1: Documented new functions. + + * share/txr/stdlib/path-test.tl: New file. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + Adding support for uid and gid manipulation. + + * configure: Added check for geteuid and related functions. + + * sysif.c (getuid_wrap, geteuid_wrap, getgid_wrap, getegid_wrap, + getgroups_wrap, setuid_wrap, seteuid_wrap, setgid_wrap, setegid_wrap): + New static functions. + (sysif_init): Register intrinsics getuid, geteuid, getgid, + getegid, getgroups, setuid, seteuid, setgid, setegid. + + * txr.1: Documented new functions. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (system_error_s): New symbol variable. + (obj_init): Initialize new variable. + + * lib.h (system_error_s): Declared. + + * unwind.c (uw_init): Register system-error exception type. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stream_init): Register get-indent-mode, set-indent-mode, + test-set-indent-mode, get-indent, set-indent, inc-indent + and width-check intrinsic functions. Register indent-off, + indent-data and indent-code variables. + + * txr.1: Documented stream output indentation API. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (width_check): Just use the public + put_char function; no need to manipulate column + or call put_indent. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + Better diagnostic in funcall family of functions. + + * lib.c (wrongargs): New static function. + (funcall, funcall2, funcall2, funcall3, funcall4): + Use wrongargs. + +2015-08-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (do_eval): Bugfix: though last_form_evaled is + saved and restored around the execution of a special operator, + it is never set the current form. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_args): Use new ~! for proper indentation + of multi-line context form. + (apply): Use ~! for proper indentation of function code. + + * unwind.c (uw_throw): Use ~! for proper indentation of code. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (put_string): In indent mode, put_string has + to process all the characters as if by put_char, + (which we now do literally that way). + (set_indent_mode): Bugfix: no longer reset the column + to zero when turning off indent mode. This is wrong since + streams do column counting all the time. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Implement ~! format directive for indentation. + Allow negative widths to be specified with a leading minus sign, + so that we can indent to the left. + + * txr.1: Document ~! format directive. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (put_string, put_char): Do not put out the indentation + immediately after outputting a newline. Rather, delay the output of the + indentation until some output occurs at column zero. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (inc_indent): If a negative indentation increment goes below + zero, clamp it at zero. + (set_indent): Clamp indentation value to zero. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vormat): Bugfix: when width specified as * + meets a negative argument, the width should be treated + as positive and the field left aligned. Instead, the + width is treated as zero. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (string_out_put_string): Do not return nil when + buffer calculations overflow, but throw exception. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + Remove useless return values and checks. + + * stream.c (vformat_align_pre, vformat_align_post): Change to void + return. Do not check return value of put_char. + (vformat_num, vformat_str): Change to void return. Do not check return + value of vformat_align_pre, vformat_aign_post or put_char. + (vformat): Do not check return value of vformat_str or vformat_num. + nilout exit point no longer needed. + (put_string): Do not intercept return value of ops->put_string. + Return t. + (put_char): Do not intercept return value of ops->put_char. + Do not check return value of put_indent. Return t. + + * txr.1: Document t return of put-char, put-byte and put-string. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (func_get_name): New function. + (bind_args): Include the entire context form in argument + mismatch errors. + (apply): Include the function name, or else source code + if it has no name, in argument mismatch erors. + (eval_init): Register func-get-name intrinsic. + + * eval.h (func_get_name): Declared. + + * txr.1: Documented func-get-name. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (force): Default the new second argument of source_loc_str. + (eval_error): Derive location of error from + the last_form_evaled, if form doesn't have it. + (eval_init): Re-register source-loc-str as binary with an optional arg. + + * match.c (debuglf, sem_error, file_err, typed_error): Default new + argument of source_loc_str. + + * parser.h (source_loc_str): Declaration updated. + + * parser.l (source_loc_str): Take second argument which specifies + alternative value if the source loc info is not found. + + * unwind.c (uw_throw): Simplify code thanks to source_loc_str + default argument. + + * txr.1: Document new argument of source-loc-str. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_revget): New function. + + * hash.h (hash_revget): Declared. + + * eval.c (eval_init): Registered hash-revget intrinsic. + + * txr.1: Documented hash-revget. + +2015-08-04 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (indent_mode_put_string): Function removed, + logic hoisted into put_string. + (put_string, put_char): Always count column, indent mode or not. + + * tests/009/json.expected: Updated. + +2015-08-03 Kaz Kylheku <kaz@kylheku.com> + + Define TXR_DBG macro when compiling debug build. + + * Makefile (dbg/%.o): Pass $(DBG_ONLY_FLAGS) to COMPILE_C_WITH_DEPS + recipe macro. + + * configure (debug_only_flags): New macro. + (gen_config_make): Generate DBG_ONLY_FLAGS. + Add debug-only-flags to usage help text. + +2015-08-03 Christopher Meng <i@cicku.me> + + Custom linker flags support. + + * Makefile (PLATFORM_FLAGS): Rename to PLATFORM_CFLAGS. + (LDFLAGS): New variable. Includes -lm and PLATFORM_CFLAGS. + (LINK_PROG): Use $(LDFLAGS) instead of -lm. + + * configure (platform_flags): Renamed to platform_cflags. + (platform_ldflags): New variable. + (gen_config_make): Rename PLATFORM_CFLAGS and generate + PLATFORM_LDFLAGS. + Usage help text updated. + +2015-08-03 Kaz Kylheku <kaz@kylheku.com> + + Switching some globals to lexical and changing some names. + + * arith.c (arith-init): Changing *flo-...* from special to + lexical, and adding un-earmuffed variants. The earmuffed + versions are obsolescent. + Adding %pi% and %e% global lexicals. Earmuffed versions are + also made global lexical, and obsolescent. + + * eval.c (lookup_global_var, lookup_global_var_l): New functions. + (lookup_var): Uses lookup_global_var. + (reg_varl): New function. + (reg_var): Uses reg_var. + (eval_init): Register global lexicals user-package, system-package + and keyword-package. Old symbols with earmuffs are obsoleted, + and also turned into global lexicals. + (top-vb, top-fb): Changed to lexical. + + * eval.h (lookup_global_var, lookup_global_var_l, reg_varl): + Declared. + + * genvim.txr: Scan ver.tl so that the lib-version variable + is included. Extract reg_varl calls. + + * glob.c (glob_init): glob-err and other variables made lexical. + + * lib.c (get_user_package, get_system_package, get_keyword_package): + Use lookup_global_var_l to avoid searching dynamic scope for + lexicals which cannot be dynamically rebound. + + * share/txr/stdlib/ver.tl (lib-version): New global lexical variable. + (*lib-version*): Turned lexical. Obsolescent. + + * signal.c (sig-init): sig-* variables turned lexical. + + * sysif.c (sysif-init): s-*, poll-* and w-* variables turned lexical. + + * syslog.c (syslog-init): log-* variables turned lexical. + + * txr.c (sysroot-init): stdlib and *txr-version* variables turned + lexical. txr-version variable added, and *txr-version* is + obsolescent. + (txr-main): self-path variable added. *self-path* turns lexical + and is obsolescent. + + * txr.1: Documentation updated. Lexical variables not referred + to as special. Special variables referred to as special. + + +2015-08-03 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Bugfix: register *args-full*, the variable + described in the documentation! We keep *full-args* for backward + compatibility, as an obsolescent feature. + +2015-08-03 Kaz Kylheku <kaz@kylheku.com> + + Introducing global lexicals: defvarl, defparml. + + * eval.c (defvarl_s, defparm_s, defparml_s, sys_mark_special_s): New + symbol variables. + (mark_special): Return val rather than void, since it's hoisted + into Lisp domain now with func_n1. + (op_defvar): Renamed to op_defvarl, and doesn't call mark_special. + defvarl is now the special form for defining variables, and special + marking is an embellishment added by macros. + (me_defparm): Renamed to me_def_variable. Handles defvar, defparm + and defparml. + (do_expand): Expand defvarl, rather than defvar. + (eval_init): Intern defvarl, defparm, defparml and mark-special + symbols, and initialize corresponding globals. + Register defvarl special operator, retiring defvar. + Register defparm to me_def_variable, and register defvar and + defparml to the same. + + * txr.1: Document support for global lexical scopes and the + new macros. + +2015-08-02 Kaz Kylheku <kaz@kylheku.com> + + Support lstat and fstat. + + * stream.h (statf): Declaration removed. + + * sysif.c (w_stat): Function takes val instead of const wchar_t * + as leftmost argument. + (w_lstat, w_fstat): New static functions, with same interface as w_stat. + (stat_to_list, stat_impl): New static functions. + (statp, statl): New static functions. + (statf): Function removed, name re-used for new static function. + (sysif_init): stat intrinsic registered to statp function, not statf. + lstat and fstat intrinsics registered to statl and statf. + + * txr.1: Documented lstat and fstat. + +2015-08-01 Kaz Kylheku <kaz@kylheku.com> + + Pass pretty flag to cobj print operation. + + * hash.c (hash_print_op): Take third argument, + and call cobj_print_impl rather than cobj_print. + + * lib.c (cobj_print_op): Take third argument. The object class is + * printed with obj_print_impl. + (obj_print_impl): Static function becomes extern. Passes its pretty + flag argument to cobj print virtual function. + + * lib.h (cobj_ops): print takes third argument. + (cobj_print_op): Declaration updated. + (obj_print_impl): Declared. + + * regex.c (regex_print): Takes third argument, and ignores it. + + * stream.c (stream_print_op, stdio_stream_print, cat_stream_print): + Take third argument, and ignore it. + + * stream.h (stream_print_op): Declaration updated. + +2015-08-01 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (special_p): Function renamed to special_var_p. + (expand_opt_params_rec, expand_vars): Follow rename of special_p. + (eval_init): Register special-operator-p and special-var-p + intrinsics. + + * txr.1: Document special-operator-p and special-var-p. + +2015-08-01 Kaz Kylheku <kaz@kylheku.com> + + Big TXR Lisp documentation rearrangement. + + * txr.1: Numerous sections moved around. Miscellaneous formatting + and wording fixes. New sections under TXR Lisp intro section. + +2015-07-31 Kaz Kylheku <kaz@kylheku.com> + + string_output renamed to string_out + + * stream.c (struct string_output): Renamed to string_out. + (string_out_stream_destroy, string_out_byte_callback, + string_out_put_char, strnig_out_put_string, string_out_put_byte, + make_string_output_stream, get_string_from_stream): Follow rename. + +2015-07-31 Kaz Kylheku <kaz@kylheku.com> + + Simplify type check code in stream module. + + * stream.c (get_string_from_stream, get_list_from_stream, + stream_set_prop, stream_get_prop, close_stream, get_line, + get_char, get_byte, unget_char, unget_byte, put_string, + put_char, put_byte, flush_stream, seek_stream): Use cobj_handle and + cobj_ops instead of type_check and type_assert. + (get_indent_mode, test_set_indent_mode, set_indent_mode, + get_indent, set_indent, inc_indent, width_check): Use cobj_handle + and cobj_ops instead of stream->co.handle and stream->co.ops + for safety. + + (vformat, format): Use class_check instead of type_check and + type_assert. + +2015-07-31 Kaz Kylheku <kaz@kylheku.com> + + Multi-line, indented printing of structure. + + * eval.c (op_error): New static function. + (macro_form_p, fboundp): Static to external. + (special_operator_p): New function. + (eval_init): Register macrolet and symacrolet to op_error. + These are recognized and processed by expand, but we want + them in the op table so they are reported by special_operator_p. + + * eval.h (fboundp, macro_form_p, special_operator_p): Declared. + + * hash.c (print_key_val): Break long lines on spaces + between pairs with stream_width_check. + (hash_print_op): Implement split and indented printing. + + * lib.c (obj_print_impl): New static function, resulting + from a merge of obj_print and obj_pprint. Fixes some + wrong-way recursion bugs: obj_pprint recursed into obj_print + in some places. Adds support for multi-line printing of + vectors and lists, with indentation using the new + interfaces in streams. + + * stream.c (strm_base_init): Update initializer. + (put_indent, indent_mode_put_string): New static functions. + (put_string): Use indent_mode_put_string in either of the + two indent modes. + (put_char): Implement indent mode. + (get_indent_mode, test_set_indent_mode, + set_indent_mode, get_indent, set_indent, + inc_indent, width_check): New functions. + + * stream.h (enum indent_mode): New. + (struct strm_base): indent_on member becomes indent_mode. + New members data_width and code_width. + (get_indent_mode, test_set_indent_mode, + set_indent_mode, get_indent, set_indent, + inc_indent, width_check): Declared. + + * tests/009/json.expected: Updated. + + * tests/010/seq.expected: Likewise. + + * tests/011/macros-2.expected: Likewise. + +2015-07-30 Kaz Kylheku <kaz@kylheku.com> + + * filter.c, utf8.c: Fix bad indentation introduced in whitespace + fix on 2013-08-09. + +2015-07-30 Kaz Kylheku <kaz@kylheku.com> + + Correction to COBJ initialization pattern. + + In fact, the previosuly documented process is not correct and still + leaves a corruption problem under generational GC (which has been the + default for some time). + + * HACKING: Document flaw in the initialization pattern previously + thought to be correct, and show fix. + + * hash.c (copy_hash): Fix instance of incorrect pattern. + + * regex.c (regex_compile): Likewise. + +2015-07-30 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Restore accidentally deleted document title. + +2015-07-29 Kaz Kylheku <kaz@kylheku.com> + + Deriving streams from the same base, so + we can give streams some common slots. + + * stream.c (strm_base_init, strm_base_cleanup, strm_base_mark, + stream_destroy_op, stream_mark_op): New functions. + (null_ops): Switch to stream_destroy_op and stream_mark_op. + (make_null_stream): Associate a strm_base instance with the + cobj handle rather than a null pointer. + (struct stdio_handle): Inherit struct strm_base. + (stdio_stream_destroy): Clean up the strm_base part. + (stdio_stream_mark): Mark the strm_base part. + (make_stdio_stream_common): Initialize the strm_base part. + (struct dir_handle): Inherit struct strm_base. + (dir_destroy): Clean up the strm_base part. + (dir_mark): Mark the strm_base part. + (make_dir_stream): Initialize the strm_base part. + (struct string_in): New structure. Replaces ad-hoc cons + cell used for string input streams. + (string_in_stream_mark): Mark new structure. + (string_in_get_line, string_in_get_char, string_in_unget_char, + string_in_get_prop, string_in_get_error, make_string_input_stream): + Convert to new structure. + (string_in_ops): Switch to stream_destroy_op. + (struct byte_input): Inherit struct strm_base. + (byte_in_stream_destroy): No need to check handle for null. + No need to set handle to null after freeing: gc does it. + Clean up the strm_base part. + (byte_in_ops): Switch to stream_mark_op. + (make_string_byte_input_stream): Initialize the strm_base part. + (struct string_output): Inherit struct strm_base. + (string_out_stream_destroy): No need to check handle for null + since the logic elsewhere has changed. + Clean up the strm_base part. + No need to set handle to null. + (string_out_ops): Switch to stream_mark_op. + (make_string_output_stream): Initialize the strm_base part. + (get_string_from_stream): Don't free the handle. + Null out the buffer so->buf whose ownership passes to the string. + (struct strlist_out): New structure. Replaces ad-hoc cons cell + used for string list output stream. + (strlist_mark): Renamed to strlist_out_mark. Mark the strm_base + part. + (strlist_out_put_string, strlist_out_put_char, + make_strlist_output_stream, get_list_from_stream): + Convert to new structure. + (strlist_out_ops): Switch to stream_destroy_op. + Follow rename of strlist_mark. + (struct cat_strm): New structure, replacing ad-hoc list pointer + +2015-07-29 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (finalize): Null out cobj handle after calling the + destructor. This can catch some use-after-free errors. + +2015-07-29 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (common_destroy): Function removed. + (stdio_stream_destroy, dir_destroy): Call close_stream directly. + +2015-07-29 Kaz Kylheku <kaz@kylheku.com> + + * stream.c: Rearranging definitions and declarations so that + code for each stream type is together. Moving catenated + streams above utility functions, so they are together with + other stream definitions. + +2015-07-28 Kaz Kylheku <kaz@kylheku.com> + + Centered fields in format. + + * stream.c (enum align): New enum. + (vformat_align_pre, vformat_align_post): New static functions. + (vformat_num, vformat_str): Argument left renamed to align, and changed + to enum align. vformat_align_pre and vformat_align_post are called to + generate the correct padding for left, center and right alignment. + (vformat): Two-valued local variable left turned into three-valued + align. The ^ prefix on the width field is recognized. + + * txr.1: Documented ^ width prefix. + +2015-07-28 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/txr-case.tl (txr-case-impl): Bugfix, + handle nil case in recursion. + +2015-07-28 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: In introduction of user-defined places, mentioning + placelet as a way of writing place update macros. + Example added under placelet. + +2015-07-25 Kaz Kylheku <kaz@kylheku.com> + + Version 110. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version and date. + + * share/txr/stdlib/ver.tl: Likewise. + + * tl.vim, txr.vim: Regenerated. + +2015-07-25 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Improved highlighting of items in quasiquote. + Lists can be spliced and quoted. Metanums and metavars can + be prefixed by multiple @'s. + +2015-07-25 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Do not refer to installed code via stdlib. + Refer to source tree. + +2015-07-25 Kaz Kylheku <kaz@kylheku.com> + + Quasiliteral tests. + + * tests/012/quasi.expected: New file. + + * tests/012/quasi.tl: New file. + +2015-07-25 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (tst/tets/012/%): Disable --gc-debug for + tests in this directory. + +2015-07-24 Kaz Kylheku <kaz@kylheku.com> + + * tests/012/ifa.tl: New test for it being bound to a place. + +2015-07-24 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: place-form-p must expand place macros. + + * share/txr/stdlib/place.tl (place-form-p): Take + environment parameter. Expand the place form using sys:pl-expand. + + * share/txr/stdlib/ifa.tl (ifa): Pass environment + to place-form-p. + (nthcdr): Pass environment down to place-form-p. + +2015-07-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_quote): Improved diagnostic. + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + Set up blank top_srcdir if building in source directory. + + This means short, relative names are used for C source files when + compiling in the source directory. + + * Makefile (CFLAGS): Exclude -iquote $(top_srcdir) arguments + if top_srcdir is blank. Remove slash after $(top_srcdir), + since the variable includes it now, if its value is not blank. + (SRCS, ABBREV, opt/%-win.o, lex.yy.c, y.tab.c, install): Remove slash + after $(top_srcdir). + (dbg/%.o, opt/%.o): Only include rules for depending + on $(top_srcdir)%.c prerequisite if top_srcdir is not blank. + In the blank case, the rules are redundant. + + * configure: if configuring in the root of the source tree, + establish top_srcdir as blank, otherwise establish is as + an absolute path with a trailing slash. + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + Split off test macros from ifa.tl into common file. + + * Makefile (TESTS_OUT): Don't use find to hunt down tests; + but rather wildcard. This way common.tl is not mistakenly + identified as an independent test file. + + * tests/012/ifa.tl: Removed test code, placed in new file + which is loaded. + + * tests/common.tl: New file. + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + Fixes for building in separate dir. + + * Makefile (CFLAGS): Search for mpi header in $(top_srcdir). + (ABBREV): Remove top_srcdir prefix from source names. + (dbg/%.o, opt/%.o): Extra set of pattern rules for searching + for prerequisites in $(top_srcdir) where primary C sources + are found and in the local dir where some generated C is found. + This way we don't rely on VPATH. + (opt/%.res): Bogus target removed. + (VPATH): Removed in favor of explicitly referencing $(top_srcdir). + Was causing problems for tests. + (lex.yy.c, y.tab.c): Prerequisites referenced in $(top_srcdir). + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (grammar): Do not allow unescaped newline in + word list literals and word list quasiliterals, except + in <= 109 compatibility mode. An escaped newline in + these literals, together with surrounding whitespace, + now produces a single space, except in <= 109 + compatibility mode. + + * txr.1: Documented new rules for WLL's and QLL's, + and added compatibility notes. + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Scan cadr.c, cadr.tl, with-resources.tl, txr-case.tl. + Remove hard-coded txr-case symbols. Recognize operators registered + with in-line intern. + +2015-07-23 Kaz Kylheku <kaz@kylheku.com> + + Adding with-resources macro. + + * share/txr/stdlib/with-resources.tl: New file. + + * lisplib.c (with_resource_set_entries, with_resources_instantiate): + New static functions. + (lisplib_init): Register new functions under dlt_register. + + * txr.1: Document with-resources. + +2015-07-22 Kaz Kylheku <kaz@kylheku.com> + + Adding nthcdr as accessor. + + * eval.c (eval_init): Register nthcdr function. + + * lib.c (nthcdr): New function. + + * lib.h (nthcdr): Declared. + + * share/txr/stdlib/place.tl (nthcdr): New defplace. + + * txr.1: Documented. + +2015-07-22 Kaz Kylheku <kaz@kylheku.com> + + Improve bad argument diagnosis for place macros. + + * eval.c (op_mac_param_bind): New static function. + (eval_init): Register mac-param-bind operator. + + * share/txr/stdlib/place.tl (define-place-macro): Use + mac-param-bind inside a lambda instead of tb macro. + + * txr.1: Document mac-param-bind. + +2015-07-22 Kaz Kylheku <kaz@kylheku.com> + + Implementing second through tenth as places. + + * eval.c (eval_init): Register second through tenth as intrinsic. + + * gencadr.txr: New cadr.c changes encoded. + + * lib.c (second, third, fourth, fifth, sixth): Functions + reimplemented using ref, so they are much more efficient + for vectors and strings. + (seventh, eighth, ninth, tenth): New functions. + + * lib.h (seventh, eighth, ninth, tenth): Declared. + + * share/txr/stdlib/place.tl: place macros defined for + second through tenth. + + * txr.1: Documented. + +2015-07-22 Kaz Kylheku <kaz@kylheku.com> + + * lisplib.c (place_set_entries): Add *place-macro* and + define-place-macro to list of names. + + * share/txr/stdlib/place.tl (*place-macro*): New global hash. + (sys:tigger-load, sys:pl-expand): New functions. + (call-update-expander, call-clobber-expander, call-delete-expander): + Recognize and expand place macros. + (define-place-macro): New macro. + (first, rest): Places redefined using define-place-macro, replacing + the old hack of copying the expanders from one table entry to another. + + * txr.1: Documented place macros. + +2015-07-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print, obj_pprint): Bugfix: incorrect + value returned when printing dwim forms, due to + mutating the obj variable. + +2015-07-21 Kaz Kylheku <kaz@kylheku.com> + + Implementing caar, cadr, cdar and friends. + + * lib.c (init): Call cadr_init. + + * lisplib.c (dl_table, set_dlt_entries, dlt_register): Externalize. + + * lisplib.h (dl_table, set_dlt_entries, dlt_register): Declared. + + * Makefile (OBJS): Add cadr.o. + + * cadr.c: New file. + + * cadr.h: New file. + + * gencadr.txr: New file. + + * share/txr/stdlib/cadr.tl: New file. + + * txr.1: Document cadr accessors. + +2015-07-21 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/place.tl (defplace cdr): Change deletion + semantics so that (del (cdr x)) is symmetric with (del (car x)). + + * txr.1: Update documentation. + +2015-07-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register new split function. + + * lib.c (split_func): New static function. + (partition_split_common): New static function, based on + on contents of partition function. + (partition): Now a wrapper around partition_split_common. + (split): New function. + + * lib.h (split): Documented. + + * txr.1: Documented split. + +2015-07-13 Kaz Kylheku <kaz@kylheku.com> + + * parser.c (open_txr_file): Bugfix: the name of the parsed + stream should be the resolved name, not the abstract original, + so that error messages correlate to the file. + +2015-07-12 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/ifa.tl (ifa): Use placelet to + allow "it" to be mutable when it denotes a place form. + +2015-07-12 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/place.tl (place-form-p): New function. + + * lisplib.c (place_set_entries): Add place-form-p to list of names. + +2015-07-11 Kaz Kylheku <kaz@kylheku.com> + + Expand away sys:lisp1-value based on lexical info. + + * eval.c (sys_lisp1_value_s): New global symbol variable. + (expand_lisp1_value): New static function. + (do_expand): Use expand_lisp1_value. + (eval_init): Initialize sys_lisp1_value_s. + +2015-07-11 Kaz Kylheku <kaz@kylheku.com> + + Let's have placelet and placelet*. + + * share/txr/stdlib/place.tl (placelet*): New macro. + + * lisplib.c (place_set_entries): Add placelet* to list of names. + + * txr.1: Updated to document placelet* and fix mistakes. + +2015-07-11 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/place.tl (defplace dwim): Eliminate + ogetter-sym variable for caching the sequence; use the getter + for the sequence place to access it. Again, this is for placelet, + where the caching is visible. Another way to fix this is to + clobber the cache variable when updating the sequence place. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register bindable as intrinsic. + + * txr.1: Documented. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + New placelet macro. + + * lisplib.c (place_set_entries): Add placelet to list of names. + + * share/txr/stdlib/place.tl (sys:placelet1, placelet): New macros. + (defplace dwim): Do not retrieve the + place's value into a local variable and have the getter + expand to that variable. Rather, have the getter retrieve the + value. A getter that refers to a cached copy breaks + the semantics of placelet, and any place operator which can + evaluate the location after it is stored. + + * txr.1: Documented placelet. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Handle setting of parse name through prime_parser. + + * parser.c (prime_parser): Take name as argument, + and install it into parser. + (lisp_parser): Pass name to parse, instead of setting it + in the parser object. + + * parser.y (parse): Take name as argument and pass + down to prime_parser. + + * parser.h (prime_parser, parse): Declarations updated. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Fix wrong name reported for errors in first top-level form. + + * parser.c (read_eval_stream): Get name of original stream, + and pass that down to lisp_parse as the name argument. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Fix off by one error line numbers for .tl files. + + * parser.c (read_eval_stream): Add terminating newline to + the first line that was examined for hash bang. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Remove unused "primer" member from parser_t. + + * parser.c (parser_mark, parser_common_init): Remove reference to primer. + (parser): Don't take primer argument, remove reference to member. + (ensure_parser): Don't take primer argument, don't pass to parser + function. + (lisp_parse): Don't pass primer string to ensure_parser. + + * parser.h (primer_t): Remove primer member. + (parser): Declaration updated. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: lexer loses unmatched "hold char" between top-level forms. + + Test case: file containing 4(prinl 3). Scanner consumes 4 and (. + The ( is lost when the scanner is reset for the next call to yyparse, + resulting in jut prinl being read and interpreted as a variable. + + * parser.c (prime_parser): If present, append hold byte to priming + string. Takes parser_t * instead of parser, and returns void now. + + * parser.l (reset_scanner): Now returns int value, the value + of the scanner's yy_hold_char variable which is nonzero when + the scanner is hanging on to an unmatched byte of input. + + * parser.h (reset_scanner, prime_parser): Declarations updated. + + * parser.y (parse): Pass hold byte returned by reset_scanner to + prime_parser. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (byte_in_unget_byte): Wrong function name in error message. + +2015-07-10 Kaz Kylheku <kaz@kylheku.com> + + Fix unget-byte and unget-char on catenated streams. + + * stream.c (cat_unget_byte, cat_unget_char): Recursive + call to unget_byte and unget_char with reversed arguments. + +2015-07-09 Kaz Kylheku <kaz@kylheku.com> + + Parser cleanup: embed scanner in parser. + + * parser.c (parser_destroy): New GC finalizer static function. + (parser_ops): Register parser_destroy. + (parser_common_init): New function, shared by parse and parse_once. + Initializes embedded scanner. + (parser_cleanup): New function, shared by parse_once and + parser_destroy. + (parser): Use parser_common_init. + + * parser.h (parser_t): New member, yyscan. + (reset_scanner, parser_common_init): Declared. + + * parser.l (reset_scanner): New function. + + * parser.y (parse_once): Use parser_common_init, and + thus perform only a few initializations. Do not + define scanner as a local variable. + (parse): Call reset_scanner instead of + yylex_init since the scanner is being reused, + and for the same reason do not call yylex_destroy. + GC will do that now. + +2015-07-08 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: define-modify-macro not registered for auto-loading. + + * lisplib.c (place_set_entries): Add define-modify-macro to + list of names. + +2015-07-07 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: allow @1 in brace variables. + + * parser.l (grammar): Scan a METANUM token in the + BRACED state also. This allows us to correctly + reference op arguments in a quasiliteral, as in + `foo @{@1 [1..2] ","} bar`. + +2015-07-07 Kaz Kylheku <kaz@kylheku.com> + + Improved printing of stream objects. + + * stream.c (null_stream_print): Static function removed. + (stream_print_op): New function. Prints stream's name and address. + (null_get_prop): Pull stream name from strm_ops structure + rather than hard-coding to "null-stream". + (null_ops, stdio_ops, tail_ops, pipe_ops, string_in_ops, + byte_in_ops, string_out_ops, strlist_out_ops, dir_ops, + cat_stream_ops): Use stream_print_op in place of cobj_print_op. + Initialize name. + (stdio_stream_print): Use name from ops structure, + and include object address. + (string_in_get_prop): Likewise. + (cat_stream_print): Likewise. + + * stream.h (struct strm_ops): New member, name. + (strm_ops_init): Initialize name. + (stream_print_op): Declared. + + * syslog.c (syslog_strm_ops): Use stream_print_op. Initialize name. + +2015-07-06 Kaz Kylheku <kaz@kylheku.com> + + Bugfix in printed rep of stdio stream. + + * stream.c (stdio_stream_print): Wrong way test on h->pid; + if it is nonzero it must be printed. Was printed wrong: + argument of integer type passed to ~s which expects a val. + +2015-07-06 Kaz Kylheku <kaz@kylheku.com> + + Clarify clear-error and flush return value. + + * stream.c (stdio_clear_error): Return the prior value of h->err. + Do not inspect feof(h->f) or ferror(h->f); just clear the error + status of the stream. + (stdio_close): Set h->err from errno if fclose failed. + (dir_clear_error): Return prior value of h->err. + + * txr.1: Document return value convetion for clear-error, + and flush-stream. + +2015-07-06 Kaz Kylheku <kaz@kylheku.com> + + Tightening behavior for unimplemented stream ops. + + In many cases, if a stream operation is not applicable to a stream, it + is not appropriate to just do nothing and return nil. An error + must be thrown. + + * stream.c (unimpl, unimpl_put_string, unimpl_put_char, + unimpl_put_byte, unimpl_get_line, unimpl_get_char, unimpl_get_byte, + unimpl_unget_char, unimpl_unget_byte, unimpl_seek, null_put_string, + null_put_char, null_put_byte, null_get_line, null_get_char, + null_get_byte, null_close, null_flush, null_seek, null_set_prop, + null_get_error, null_get_error_str, null_clear_error): New + static functions. + (fill_stream_ops): New function. + (null_ops): Use null_put_string, etc., instead of null pointers. + (stdio_clear_error, dir_clear_error, cat_clear_error): Return value now + indicating whether an error was cleared. + (stream_set_prop, stream_get_prop, real_time_stream_p, close_stream, + get_error, get_error_str, clear_error, get_line, get_char, get_byte, + unget_char, unget_byte, put_string, put_char, put_byte, + flush_stream): Do not test whether a function in the ops table + is defined; just call it, and return its value. + (stream_init): Call fill_stream_ops on all stream operation + structures so any null pointers are defaulted to appropriate + functions. + + * stream.h (struct strm_ops): clear_error gets return value. + (fill_stream_ops): Declared. + + * syslog.c (syslog_init): Call fill_stream_ops on syslog_strm_ops. + +2015-07-04 Kaz Kylheku <kaz@kylheku.com> + + Reduce regex duplication in genvim.txr. + + * genvim.txr (bs, hex, at, alpha, alnum, dig, oct, chesc, glyph): + New variables. + *(txr_error, txr_atat, txr_comment, txr_contin, txr_char, + txr_regdir, txr_variable, txr_splicevar, txr_stresc, txr_numesc, + txr_regesc, txr_chr, txr_num, txr_badnum,txr_ident, + txr_braced_ident): Use regex definitions. + +2015-07-03 Kaz Kylheku <kaz@kylheku.com> + + Syntax highlighting fixes: characters, variables, identifers. + + * genvim.txr (txr_variable, txr_splicevar, txr_bracevar): Weren't being + generated into tl.vim. Moved to a common section. + (txr_nested_error): Relocate above other matches. + (txr_ident): Fixed not to match tokens starting with #. + +2015-07-03 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Simplify double plass over BODY into single pass. + +2015-07-03 Kaz Kylheku <kaz@kylheku.com> + + Allow op arguments like @1 to be places. + + * share/txr/stdlib/place.tl: Introducing new defplace + for (sys:var ...) which is treated as if it were a symbol, + in anticipation of the op/do macro placing that syntax + with a symbol. + +2015-07-02 Kaz Kylheku <kaz@kylheku.com> + + Support trailing semicolon after hex/octal characters. + + * parser.l (%option): Remove nounput option since we need + yyunput. + (grammar): Rule for matching hex and octal escape in SPECIAL + state recognizes optional semicolon. In 109 compatibility, + this is pushed back into the stream, otherwise consumed. + + * txr.1: Updated documentation, including compat notes. + + * genvim.txr (txr_char): Include optional semicolon in + match. Corrected some errors where 8 and 9 were being + included as matches for octal digits. + (txr_error): Default match for \x or \o not followed + by digits. + +2015-07-02 Kaz Kylheku <kaz@kylheku.com> + + Hash-bang support for .tl files. + + * parser.c (read_eval_stream): New boolean argument + to request hash bang support. + + * parser.h (read_eval_stream): Declaration updated. + + * eval.c (sys_load): Pass new thid argument to read_eval_stream, + to decline hash bang support. + + * match.c (v_load): Likewise. + + * txr.c (txr_main): Request hash bang support from + read_eval_stream. Thus files referenced from the txr + command line can have a #! line, which is ignored. + +2015-07-02 Kaz Kylheku <kaz@kylheku.com> + + Handle escapes accurately in Vim syntax highlighting. + + * genvim.txr (txr_badesc, txr_stresc, txr_numesc, txr_regesc): New + match categories. + (txr_string, txr_quasilit, txr_regex, txl_regex): No longer use skip= + argument, but rather contain escape categories. + All escapes are colored as Special rather than String, and + unknown or malformed escapes are colored as errors. + +2015-07-01 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Take advantage of \@ escape in quasilit. + + * genvim.txr (txr_escat): New match category containing \@. + Colorized a String. + (txr_quasilit): Contains txr_escat. Thus, \@ looks like any + other escape inside a quasilit. + +2015-07-01 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Bugfix: reproduce the correct opening bracket + before a function link. + +2015-06-27 Kaz Kylheku <kaz@kylheku.com> + + Fix issues in configure found by ShellCheck. + + * configure (have_timegm, tm_gmtoff, tm_tmzone): Unused + variables removed. + (SIZEOF_LONG_LONG_T): Reference to nonexistent variable fixed. + Also, fixed two instances of arithmetic expressions using + unnecessary $ prefixes on variables. + +2015-06-27 Kaz Kylheku <kaz@kylheku.com> + + Fix TXR 97 regression in quasiliterals. + + * eval.c (expand_quasi): When the modifiers argument of a var + item is expanded, the expansion incorrectly places it as the + fourth item rather than third. This is effectively a continuation + of the incomplete regression fix that was applied on 2014-08-29. + The breakage is that `@{(expr) modifier}` ignores modifier + whenever (expr) is an expression that undergoes expansion. + +2015-06-26 Kaz Kylheku <kaz@kylheku.com> + + Third round of quasiliteral-related fixes. + + * parser.l (char_esc): Recognize \@ escape. + (grammar): Add a rule for a \@ escape in quasiliterals, + and quasi word list literals. + + * txr.1: Document \@, and remove the lie that @@ + encodes for a single @ in quasiliterals. + +2015-06-26 Kaz Kylheku <kaz@kylheku.com> + + Second round of quasiliteral-related fixes. + + * parser.l: Only shift to QSPECIAL state when @ is followed + by a trailing context consisting of certain characters. + Not every kind of Lisp object syntax can be introduced + with @ in a quasiliteral. Adding a rule to produce an + error when @ appears that is not followed by an allowed + character. + +2015-06-26 Kaz Kylheku <kaz@kylheku.com> + + First round of quasiliteral-related fixes. + + * parser.l: Do not try to recognize floating-point literals + in QSPECIAL state; that is not possible because @134.3 + in a quasiliteral parses as a METANUM followed by ".3". + On the other hand, recognize METANUM literals in QSPECIAL state, + so that @@123 scans. Recognize @ as a token in QSPECIAL state, + so @@abc will scan. When transitioning from QSILIT and QWLIT + states to QSPECIAL upon scanning @, return a @ token, which + is now parsed in the grammar. + + * parser.y (quasi_meta_helper): New static function. + (q_var): Do not handle SYMTOK any more, only the braced + variable syntax. SYMTOK is handled as a n_expr. + Braced vars are handled with explicit '@' token, which + is now produced by the scanner when it shifts from QSILIT + to QSPECIAL. + (quasi_item): No longer necessary to recognize various + forms here such as quotes and splices. Just recognize a n_expr, + preceded by '@'. + +2015-06-26 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: New filter to add hyperlinks within code in + more places such as examples and summary lists. + +2015-06-24 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: partition example: wrong use of where function. + +2015-06-24 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_from_pairs, hash_list): New functions. + + * hash.h (hash_from_pairs, hash_list): Declared. + + * eval.c (eval_init): Registered hash-from-pairs and hash-list + intrinsic. + + * txr.1: Documented new functions. + +2015-06-24 Kaz Kylheku <kaz@kylheku.com> + + Refactoring n-ary functions to use a single helper. + + * lib.c (nary_op): New function. + (plusv, mulv, logandv, logiorv): Use nary_op. + + * lib.h (nary_op): Declared. + +2015-06-23 Kaz Kylheku <kaz@kylheku.com> + + Hash subset testing. + + * eval.c (eval_init): Register hash-subset and hash-proper-subset + intrinsics. + + * hash.c (hash_subset, hash_proper_subset): New functions. + + * hash.h (hash_subset, hash_proper_subset): Declared. + + * txr.1: New functions documented. + +2015-06-22 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Add ifa.tl library file. + + * tl.vim, txr.vim: Regenerated. + +2015-06-22 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/place.tl: Get rid of big progn around the + whole module. + + * genvim.txr: Handle (def's not preceded by spaces. + +2015-06-21 Kaz Kylheku <kaz@kylheku.com> + + Version 109. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version and date. + + * share/txr/stdlib/ver.tl: Likewise. + + * tl.vim, txr.vim: Regenerated. + +2015-06-21 Kaz Kylheku <kaz@kylheku.com> + + * lisplib.c (ifa_set_entries): Add conda. + + * share/txr/stdlib/ifa.tl (conda): New macro. + + * tests/012/ifa.tl: Adding test for conda. + + * txr.1: Documenting conda. + +2015-06-20 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Rewrite the man2html-generated inner name links with hash + values derived from the title text, so that when sections are inserted + or deleted, the URL's remain stable. The PREAMBLE is gone, and + VERSION is collected when processsing the body. + +2015-06-20 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr (month-name): Filter removed. + Page date is converted to a time value, and later formatted. + +2015-06-20 Kaz Kylheku <kaz@kylheku.com> + + Remove places.h generation hack. + + * Makefile (GEN_HDRS, LISP_TO_C_STRING): Variables removed. + (%.h: %.tl): Rule removed. The place.h header is no longer + generated from place.tl. + + * lisplib.c (place_instantiate): Load place.tl from stdlib directory, + rather than obtaining it from a string literal in generated header + place.h. + + * place.tl: Moved to share/txr/stdlib directory. + + * genvim.txr: Refer to place.tl in stdlib. + +2015-06-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register have function, synonym of true. + + * txr.1: Document have. + +2015-06-19 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented ifa. + + * share/txr/stdlib/ifa.tl: Tightened up the tests for + situations when the macro is ill-formed, following + the improved specification. Also, eval-error is thrown + instead of just error. + +2015-06-19 Kaz Kylheku <kaz@kylheku.com> + + Test ifa macro. + + * Makefile (TEST_OUT): Include .tl files. + (tst/%.out): New rule variant, from .tl prerequisite. + + * tests/012/ifa.expected: New file. + + * tests/012/ifa.tl: New file. + +2015-06-19 Kaz Kylheku <kaz@kylheku.com> + + * parser.c (parser_mark): Do not mark p->syntax_tree if + its value is nao. Introduced on 2015-06-10, "Error handling improvement + in read". + +2015-06-18 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (mkstring): Fix neglect to null terminate. + +2015-06-18 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (cat_str): Detect overflow in the total length + calculation. + +2015-06-18 Kaz Kylheku <kaz@kylheku.com> + + Improvements in equal hashing function. + + * hash.c (equal_hash): For conses and vectors, ensure that + distinct permutations lead to different hash codes. This is done by + accumulating the partial hash with a multiplier, rather than just + adding subhashes. + +2015-06-18 Kaz Kylheku <kaz@kylheku.com> + + Library .txr files become .tl and are autoloaded. + + * lisplib.c (ver_set_entries, ver_instantiate, + txr_case_set_entries, txr_case_instantiate): New + static functions. + (lisplib_init): Register new functions. + + * share/txr/stdlib/txr-case.txr: Reduced to a load for the + corresponding .tl file, retained for backward compatibility. + + * share/txr/stdlib/ver.txr: Likewise. + + * share/txr/stdlib/txr-case.tl: New file, based on previous + contents of corresponding .txr file. + + * share/txr/stdlib/ver.tl: Likewise. + +2015-06-17 Kaz Kylheku <kaz@kylheku.com> + + Adding anaphoric ifa macro. + + * Makefile (install): Install .tl files present in stdlib directory. + (INSTALL): Handle argument 2 being a list. + + * eval.c (load): New function. + + * eval.h (load): Declared. + + * lisplib.c (ifa_set_entries, ifa_instantiate): New functions + to lazily load ifa.tl. + (lisplib_init): Register new functions. + + * txr.c (stdlib_path): New variable. + (sysroot_init): Store the stdlib path in stdlib_path. + + * txr.h (stdlib_path): Declared. + + * share/txr/stdlib/ifa.tl: New file. + +2015-06-17 Kaz Kylheku <kaz@kylheku.com> + + Refactoring of lisplib.c + + * lisplib.c (set_dlt_entries): New static function. + (set_place_dlt_entries): Renamed to place_set_entries. + Uses set_dlt_entries as subroutine. + (place_instantiate): Argument changes from table to function, + which is invoked to null out the hash entries. + (dlt_register): New static function. + (lisplib_init): Use dlt_register to register dynamic loading + of places_tl. + +2015-06-13 Kaz Kylheku <kaz@kylheku.com> + + Version 108. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version and date. + + * share/txr/stdlib/ver.txr: Likewise. + + * tl.vim, txr.vim: Regenerated. + +2015-06-13 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented Lisp loading. + +2015-06-13 Kaz Kylheku <kaz@kylheku.com> + + New --lisp option: treat unsuffixed files as Lisp. + + * txr.c (help): Added help text. + (txr_main): Handle lisp option. + +2015-06-13 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Handle Lisp files. + +2015-06-13 Kaz Kylheku <kaz@kylheku.com> + + Lisp load macro. + + * eval.c (sys_load_s): New symbol variable. + (sys_load, me_load): New static functions. + (eval_init): Initialize sys_load_s, register + sys:load intrinsic function and load macro. + +2015-06-12 Kaz Kylheku <kaz@kylheku.com> + + @(load) and @(include) now load Lisp code. + + * match.c (v_load): Check txr_lisp_p flag coming out of + open_txr_file and handle the Lisp case usin read_eval_stream. + + * parser.c (read_eval_stream, get_parser, parser_errors): + New functions. + + * parser.h (read_eval_stream, get_parser, parser_errors): + Declared. + +2015-06-10 Kaz Kylheku <kaz@kylheku.com> + + Preparing for lisp loading. + + * parser.c (open_txr_file): Rewritten to take new argument + which indicates whether to treat an unsuffixed file as + TXR or TXR Lisp, and is updated to indicate which is the + case by looking at the suffix. + + * parser.h (open_txr_file): Declaration updated. + + * match.c (v_load): Follow change in open_txr_file. + + * txr.c (txr_main): Likewise. + +2015-06-10 Kaz Kylheku <kaz@kylheku.com> + + Error handling improvement in read. + + * parser.y (spec): New grammar production to handle the cases + that SECRET_ESCAPE_E is not followed by anything (the input + ends before any object is scanned, or there is no input + token which starts an object). + + * parser.c (lisp_parse): Deal with EOF indication from parser + (the syntax_tree member of parser_t set to nao). + +2015-06-10 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (yybadtoken): Print unexpected character + literally rather than as a Lisp character literal. + +2015-06-08 Kaz Kylheku <kaz@kylheku.com> + + Code simplification. + + * txr.c (txr_main): Replacing two code blocks with single + call to existing open_txr_file function. + +2015-06-07 Kaz Kylheku <kaz@kylheku.com> + + Critical bugfix for weak hashes. + + * hash.c (hash_mark): Hash tables which have both weak keys + and values were not added to the reachable_weak_hashes list, + and thus not subject to complete hash processing, leading + to corruption. + +2015-06-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.h (mkloc): Rename misleading argument from fun to obj. + +2015-06-07 Kaz Kylheku <kaz@kylheku.com> + + * parser.c (stream_parser_hash): New static variable. + (parser_mark): Mark parser and primer members. + (parser, ensure_parser): new argument: primer. + (get_parser_impl, ensure_parser): New static + functions. + (prime_parser): New function. + (lisp_parse): Multiple calls to this function on the same stream + now logically continue the parse, not resetting the line + number to 1. + (parse_init): Initialize and gc-protect stream_parser_hash. + + * parser.h (parser_t): New members, primer and parser. + (prime_parser): Declared. + (parser): Declaration updated. + + * parser.y (parse): Now responsible for calling prime_parser. + +2015-06-07 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Call parse_once rater than parse. + + * parser.c (regex_parse, lisp_parse): Likewise. + + * txr.c (txr_main): Likewise. + + * parser.h (parse): Declaration updated. + (parse_once): Declared. + + * parser.y (parse_once): New function, same as old parse implementation. + (parse): Becomes one argument function which works with a previously + initialized parser and continues the parse. + +2015-06-07 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (catenated_stream_p, catenated_stream_push): New functions. + (stream_init): Registered new functions as intrinsics. + + * stream.h (catenated_stream_p, catenated_stream_push): Declared. + + * txr.1: Documented catenated-stream-p and catenated-stream-push. + +2015-05-25 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Document new syntactic places framework and place-mutating macros. + +2015-05-25 Kaz Kylheku <kaz@kylheku.com> + + Fix mismanaged dyn_env variable. + + * eval.c (bindings_helper): In the parallel binding case, only allocate + a new dynamic env when it is needed (because a special variable is + being bound), rather than unconditionally. + Consequently, only modify dyn_env when the dynamic environment actually + needs to be extended. Constructs that use bindings_helper are only + wrapped in the dynamic environment save/restore form + sys:with-saved-vars if they bind special variables. + +2015-05-22 Kaz Kylheku <kaz@kylheku.com> + + Ligher weight debug instrumentation. + + This speeds up the TXR Lisp interpreter, because do_eval + sets up a debug frame and uses debug_return. + + * debug.c (debug_block_s): Symbol removed. + (debug_init): Remove initialization of debug_block_s. + + * debug.h (debug_block_s): Declaration removed. + (debug_enter): Do not establish a named block or a catch block; + no time-wasting unwind stack manipulation at all. + The debug_depth variable is managed by the extended setjmp + context now. Provide a return value variable, and a well-defined name to + branch to to exit from the debug block. + (debug_return): Do not use heavy-weight uw_block_return; + simply set the return variable and branch to debug_return_out + label. + + * signal.h (EJ_DBG_MEMB, EJ_DBG_SAVE, EJ_DBG_REST, + EJ_OPT_MEMB, EJ_OPT_SAVE, EJ_OPT_REST): New macros. + (extended_jmp_buf): Define optional global state + variables using EJ_OPT_MEMB. + (extended_setjmp): Save and restore optional + globals using EJ_OPT_SAVE and EJ_OPT_RESTORE. + Now debug_depth is saved and restored if + debugging support is compiled in. + + * match.c (open_data_source): Remove bogus debug_return + invocations which were uncovered here by changes to the macro. + + * eval.c (do_eval, expand_macro): debug_return must now be after + debug_end, because it won't dynamically clean up frames that it doesn't + know about. The set_dyn_env is no longer unreachable in expand_macro; + it is now necessary because debug_return isn't doing the longjmp + that previously restored dyn_env. + +2015-05-22 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (pushnew): New macro. + + * lisplib.c (set_place_dlt_entries): Add pushnew. + +2015-05-22 Kaz Kylheku <kaz@kylheku.com> + + symbol-function, symbol-value and fun become places. + + * eval.c (op_setqf): New function. + (eval_init): Register sys:setqf operator. Also expose + global variable hash tables via sys:top-vb and sys:top-fb. + + * place.tl (sys:get-fb, sys:get-vb): New functions. + (fun, symbol-function, symbol-value): New places defined. + +2015-05-21 Kaz Kylheku <kaz@kylheku.com> + + Introduce defparm operator. + + * eval.c (me_defparm): New function. + (eval_init): Register defparm. + + * txr.1: Documented defparm together with defvar. + defvar documentation is revised. + +2015-05-19 Kaz Kylheku <kaz@kylheku.com> + + Represent place.tl more compactly in image. + + The contents of place.tl are now stored as a UTF-8 ordinary literal + instead of a wide char literal. + + * Makefile (LISP_TO_C_STRING): Generate plain literal. + + * lisplib.c: Convert plain literal to string. + +2015-05-15 Kaz Kylheku <kaz@kylheku.com> + + Variables are now deletable places, if they are global. + + * place.tl (sys:sym-delete-expander): New function. + (get-delete-expander): Retrieve delete expander for + symbols. + +2015-05-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (symbol_value): Retrieve the binding of a symbol + macro also. + (boundp): Return t for symbol macros. + (makunbound, fmakunbound): New functions. + (eval_init): Register makunbound and fmakunbound. + + * txr.1: Document changes to symbol-value and boundp. + +2015-05-14 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (replace_list, replace_str, replace_vec): Handle + the case when from is a vector, for consistency with the sel + function and the dwim operator. + + * txr.1: Document that the third argument of select and replace may be + a vector. + +2015-05-13 Kaz Kylheku <kaz@kylheku.com> + + Recursive lcons force bug (keep-if*, remove-if*). + + * lib.c (rem_lazy_func): Do not access the cdr field of + the lcons that we are in the middle of forcing! + +2015-05-13 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (define-modify-macro): New macro. + + * lisplib.c (set_place_dlt_entries): Add define-modify-macro. + +2015-05-13 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (push): Ensure left-to-right eval order. + +2015-05-12 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (rlet): For the sake of sane evaluation order + in the face of unknown side effects, do not thread symbolic references; + only propagate constants. + +2015-05-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (env_vb_to_fb): Allow null env argument. This + case occurs when (flet () ...) is expanded. + +2015-05-12 Kaz Kylheku <kaz@kylheku.com> + + DWIM places must use Lisp-1 semantics. + + This really only affects code which does something like + (set [f x] y) and f resolves as a function. (The TXR Program + bent over backwards to install a mutable object into a function + binding.) In this situation, we need to update the function binding + f, rather than some variable f. + + * eval.c (op_lisp1_setq, op_lisp1_value): New static functions. + (eval_init): Register sys:lisp1-setq and sys:lisp1-value + special forms. + + * place.tl (sys:*lisp1*): New special variable. + (sys:l1-setq, sys:l1-val): New macros, optimizing + versions of sys:lisp1-setq and sys:lisp1-value special operators. + (sys:sym-place-update-expander, sys:sym-place-clobber-expander): + React to sys:*lisp1* variable by doing symbol access using Lisp-1 + semantics, with help of newly introduced macros and special forms. + (dwim): Bind sys:*lisp1* to true, if main argument is a symbolic place, + so that the update is done using Lisp-1 semantics. Use the + sys:l1-val operator when evaluating arguments. + +2015-05-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (lookup_var_l): Gut this function of its silly + reimplementation of lookup_var. + (op_setq): Use lookup_var instead of lookup_var_l. + Now only outside modules use the lookup_var_l interface. + Error message fix: sys:setq, not setvar. + +2015-05-11 Kaz Kylheku <kaz@kylheku.com> + + Handle vectors and strings in rplaca and rplacd. + + * lib.c (rplaca, rplacd): Use refset and replace + respectively to handle vectors and strings. + Adjust type mismatch error message. + +2015-05-11 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (chr_str_set): Check for literal strings. + +2015-05-11 Kaz Kylheku <kaz@kylheku.com> + + Fix argument count checking regression. + + * eval.c (apply): Revert some changes from 2015-03-13 + which cause the too many arguments case not to be + diagnosed. + +2015-05-10 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (errno): Support (errno) form as place. + +2015-05-09 Kaz Kylheku <kaz@kylheku.com> + + Improve error message related to splice misuse. + + * eval.c (expand_qquote): Corrected awkward error message text + which was saying that a splice "cannot occur as an atom". + +2015-05-09 Kaz Kylheku <kaz@kylheku.com> + + Adding pset operator. + + * place.tl (pset): New macro. + + * lisplib.c (set_place_dlt_entries): Add pset. + +2015-05-09 Kaz Kylheku <kaz@kylheku.com> + + * place.tl (set): Take multiple place/value pairs. Allow zero + arguments. + +2015-05-08 Kaz Kylheku <kaz@kylheku.com> + + Crack down on redefinitions of built-ins. + + * eval.c (builtin, eval_initing): New global variable. + (op_defun, op_defmacro): During initialization, record functions + and macros in builtin hash. + (builtin_reject_test): New static function. + (expand_macrolet): Perform builtin reject test for fbind, lbind, + and macrolet. + (regfun, reg_mac): Add symbol to builtin hash. + (eval_init): GC-protect new hash table variable and initialize it. + Set eval_initing to true over eval initialization. + The flip function is renamed fo flipargs. + (eval_compat_fixup): New function, for dealing with the + operator/function conflict over flip. + + * eval.h (eval_compat_fixup): Declared. + + * lib.c (compat_fixup): Call eval_compat_fixup. + + * tests/011/macros-2.txr: This test was defining a macro called + while which is now illegal. Renamed to whilst. + + * tests/011/macros-2.expected: Regenerated. + + * txr.1: Function flip renamed to flipargs and documented in + Compatibility section. + +2015-05-07 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (LISP_TO_C_STRING): Strip comments, but not comment lines, + so line numbers don't change. + + * eval.c (eval_init): Fix registrations of lisp-parse and read. + + * lisplib.c (place_instantiate): Give name to parsed string stream + using new lisp_parse argument. + + * parser.c (lisp_parse): Takes new argument to override name. + + * parser.h (lisp_parse): Declaration updated. + + * txr.c (txr_main): Call lisp_parse with four args, defaulting + the new one. + + * txr.1: Documented new argument. + +2015-05-06 Kaz Kylheku <kaz@kylheku.com> + + New macro-based framework for assignment places. + + The operators set, inc, dec, pop and others are now macros + which generate code, rather than built-in special forms + that use "C magic". Moreover, new such macros are easy to write, and + several new ones are already available. Moreover, new kinds of + assignable places are easy to create. + + * place.tl: New file. + + * lisplib.c, lisplib.h: New files. + + * Makefile (OBJS): New target, lisplib.o. + (GEN_HDRS): New variable. + (LISP_TO_C_STRING): New recipe macro, with rule. + (clean): Remove generated headers named in $(GEN_HDRS). + + * eval.c (dec_s, push_s, pop_s, flip_s, del_s): Variables removed. + (setq_s): New variable. + (lookup_var, lokup_sym_lisp_1, lookup_var_l, lookup_fun, lookup_mac, + lookup_symac, lookup_symac_lisp1): Trigger the delayed loading of + libraries for undefined global symbols, and re-try the lookup. + (op_modplace, dwim_loc, force_l): Static functions removed. + (op_setq): New static function. + (eval_init): Initialize setq_s; remove initializations of + removed variables; remove registrations for op_modplace; + add registration for sys:setq, sys:rplaca, sys:rplacd, + sys:dwim-set and sys:dwim-del intrinsics. + Call lisplib_init to initialize the dynamic library loading module. + + * lib.c (sys_rplaca, sys_rplacd): New functions, differing + in return value from rplaca and rplacd. + (ref, refset): Handle hash table. + (dwim_set, dwim_del): New functions. + + * lib.h (sys_rplaca, sys_rplacd, dwim_set, dwim_del): Declared. + + * genvim.txr: Include place.tl in scan. + + * tests/010/seq.txr: The del operator test + case no longer throws at run-time but at macro-expansion time, so the + test case is simply removed. + + * tests/010/seq.expected: Updated output. + + * tests/011/macros-2.txr: Reset *gensym-counter* to zero, because + the textual output of the test case includes gensyms, whose numberings + fluctuate with the content of the new Lisp library material. + + * tests/011/macros-2.expected: Updated output. + +2015-05-06 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (make_obj, gc): Move resetting of prev_malloc_bytes + out of make_obj into gc. + +2015-05-05 Kaz Kylheku <kaz@kylheku.com> + + Do some cleanup in progn-like form sequences. + + * eval.c (expand_progn): New function. + (do_expand): Use expand_progn for constructs + which have an implicit progn body. + +2015-05-03 Kaz Kylheku <kaz@kylheku.com> + + Deal with bad quote syntax. + + * eval.c (op_quote): Throw error on bad syntax. + + * lib.c (obj_print, obj_pprint): Do not hide bad quote syntax + using ' notation; print it using ordinary notation. + +2015-05-01 Kaz Kylheku <kaz@kylheku.com> + + Move initialization calls to more suitable place. + + * lib.c (init): Initialize parser, syslog and glob modules + here. + + * txr.c (main): Remove initialization of parser, syslog and glob + modules from here. + +2015-05-01 Kaz Kylheku <kaz@kylheku.com> + + * signal.h (sig_save_disable): Bugfix: macro enabling + rather than disabling async signals. + +2015-05-01 Kaz Kylheku <kaz@kylheku.com> + + Fix corruption triggered by extended gc disabling. + + The issue is that when gc is disabled, the gc function + does nothing. But various code depends on calls to gc() + doing something, like making space available in various + static arrays. + + When gc is disabled for long periods, there are issues, + like array overruns. + + * gc.c (gc): Must no longer be called at all if gc_enabled is false, + and asserts that it is true. Callers must check the gc_enabled + flag and implement appropriate cases. + (make_obj): Only call gc when gc_enabled is true. + If there is no space in the freshobj array after trying gc, + or gc couldn't be tried because it is disabled, then + schedule a full gc. + (gc_set): If the checkobj array is full, only call gc if gc + is enabled, otherwise schedule a full_gc. + (gc_mutated): Do not assume that the mutobj array has room + for another object; only set the object's generation to -1 + and put it into the array if there is room. Similarly to + gc_set, do a gc if there is no room, but if gc is not enabled, then + schedule a full gc. + (gc_wrap): Only call gc if gc_enabled is true, and return t in + that case rathe than nil. + + * txr.1: Document return value of sys:gc function. + +2015-04-30 Kaz Kylheku <kaz@kylheku.com> + + Fix source location for dangling unquotes and splices. + + * parser.y (grammar): Propagate the parser line number to the unquote + or splice form, if it has not received location info from its operand + (because its operand is an atom). In the quasi_item case, we also use + rlcp_tree to make sure the info is propagated through the list being + consed up. + (rlcp_tree): Bugfix: propagate the source location info to every cons + in the list itself, not just into their cars. + +2015-04-29 Kaz Kylheku <kaz@kylheku.com> + + Improved syntax checking: defun, flet, labels, lambda. + + * eval.c (check_lambda_list): New static function. + (op_defun): Argument list checking moved into + check_lambda_list, which is called for defun out of + do_expand. Name checks moved into do_expand. + Thus, defun is checked earlier, at expansion time. + (me_flet_labels): Check the lambda list of each lexical function + with check_lambda_list. + (do_expand): Check the syntax of the lambda form, and + the validity of its argument list. + Check the name of a defun or defmacro for validity. + Check the validity of the defun argument list. + +2015-04-29 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: cannot print (lambda . 42) structure. + + * lib.c (obj_print, obj_pprint): Fix mistake in the test for + properly formed lambda syntax whcih is printed specially. + +2015-04-28 Kaz Kylheku <kaz@kylheku.com> + + Adding poll function. + + * configure: Check for the poll function and set + HAVE_POLL accordingly in config.h. + + * stream.h (fd_k): External declaration added. + + * sysif.c (poll_wrap): New function. + (sysif_init): Registered poll-in, poll-out, poll-err, + poll-pri, poll-rdhup, poll-nval, poll-rdband, poll-wrband + variables and poll function. + + * txr.1: Documented poll. + +2015-04-21 Kaz Kylheku <kaz@kylheku.com> + + Version 107. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise. + + * tl.vim, txr.vim: Regenerated. + +2015-04-25 Kaz Kylheku <kaz@kylheku.com> + + Allow (force ...) to be an assignable place. + + This allows mlet variables to be assignable. + + * eval.c (force_s): New global variable. + (op_modplace): Handle force form. + (me_mlet): Use force_s symbol. + (force_l): New static function. + (eval_init): Initialize force_s variable. Use it in registration + of force function. + + * txr.1: Remove text in mlet definition that variables + are not assignable. Replace with note about an unspecified behavior. + +2015-04-25 Kaz Kylheku <kaz@kylheku.com> + + Introducing mlet macro. + + * eval.c (me_mlet): New static function. + (eval_init): Registered mlet macro. + + * txr.1: Documented mlet. + +2015-04-25 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (symbol_package): If the argument is nil, return + the user package directly, not the value of the *user-package* + variable. + +2015-04-25 Kaz Kylheku <kaz@kylheku.com> + + Remove silly package lookup from keywordp. + + This tiny change yields a 165% (2.65X) speedup in the + tst/tests/011/mandel.txr test case. + + * lib.c (keywordp): Use keyword_package_var instead of the + keyword_package macro which looks up the global environment. + + * parser.y (sym_helper): Likewise. + +2015-04-24 Kaz Kylheku <kaz@kylheku.com> + + Regression: runaway allocation of heaps. + + * gc.c (make_obj): Fix regression introduced on 2014-10-18, + released in TXR 100. Because make_obj discontinues + placing objects into the freshobj_array when the full_gc + flag has been set, it is incorrect to allocate another heap + when the free list is exhausted and the freshobj_array + has room remaining. It has room remaining because it is + disused. + +2015-04-24 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (make_obj, gc): The check for insufficient space in freshobj + array after a gc is moved into the gc function. + +2015-04-24 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (gc): Minor off-by-one: compare gc_counter to FULL_GC_INTERVAL + rather than FULL_GC_INTERVAL - 1, since it is pre-incremented. + +2015-04-22 Kaz Kylheku <kaz@kylheku.com> + + Switch to in-tree mpi directory. + + * Makefile (CFLAGS, MPI_OBJS): Refer to mpi directory without version + number. + (repatch): Target removed. + (distclean): Don't try to remove old unpacked tarball mpi directory. + + * configure (mpi_version, have_quilt, have_patch): Variables removed. + (gen_config_make): Do not generate mpi_version make variable. + Do not test for quilt or patch. No longer extract MPI tarball or + try to apply patches. + + * mpi-1.8.6.tar.gz: File removed. + + * mpi-patches: Directory and all files under it removed. + +2015-04-22 Kaz Kylheku <kaz@kylheku.com> + + delay/force overhaul. + + * eval.c (promise_forced_s, promise_inprogress_s): New symbol + variables. + (me_delay): Change representation of promises so that the + original delay form is stashed there for better reporting + in the force function. Also, propagate the debug info + from the second argument of the form to the entire form; + otherwise it will inherit it from elsewhere. + (force): Rewritten to follow new three-state representation + to detect the recursive case and diagnose it. + (eval_init): Register new symbol variables. + +2015-04-21 Kaz Kylheku <kaz@kylheku.com> + + Version 106 + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise. + +2015-04-19 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: throwing error when trying to print valid regexps. + + * regex.c (print_rec): Only dianose "bad object in regex syntax" + for some atom other than nil, which denotes an empty (sub)expression, + like what results from #// or #/a|/ and such. + +2015-04-19 Kaz Kylheku <kaz@kylheku.com> + + Do not allow unrecognized escapes in regex. + + * parser.l (REGOP): New regex alias for matching all regex + special characters. + (grammar): Several rules for regex special characters merged + together. New rule introduced to match a special character + after a backslash, making it literal. The old rule which makes + literal any character after a backslash now throws an error, + unless version 105 comaptibility is selected. + + * txr.1: Documented this behavior change. + +2015-04-19 Kaz Kylheku <kaz@kylheku.com> + + Improvement in error reporting. + + This fix prevents certain situations when an inappropriate form + is reported, in the default unhandled exception handler, as pertaining + to an error. + + * eval.c (do_eval): Save and restore the last_form_evaled, + in an exception-unsafe way. If we successfully evaluate a form or + subform, we can restore the previous one. + (expand): Save and restore last_form_expanded similarly. We don't + need the unwind block and counter. + +2015-04-18 Kaz Kylheku <kaz@kylheku.com> + + Adding lcons macro. + + * eval.c (me_lcons): New function. + (eval_init): Registered lcons macro. + + * txr.1: Documented lcons. + + * tl.vim, txr.vim: Regenerated. + +2015-04-18 Kaz Kylheku <kaz@kylheku.com> + + Adding getenv, setenv and unsetenv. + + * lib.c (setenv, unsetenv): Changed static functions to external. + Moved them out of the #if !HAVE_TIMEGM block. + + * lib.h (setenv, unsetenv): Declared. + + * sysif.c (getenv_wrap, setenv_wrap, unsetenv_wrap): New functions. + (sysif_init): Registered getenv, setenv and unsetenv. + + * txr.1: Documented getenv, setenv and unsetenv. + + * tl.vim, txr.vim: Regenerated. + +2015-04-18 Kaz Kylheku <kaz@kylheku.com> + + Fix quasistring regression introduced in TXR 81. + + * parser.y (expand_meta): This function must recognize + quasistrings, inside (sys:quasi ...) forms, (sys:var ...) forms + do not denote TXR Lisp variables. These must not be expanded. + Doing so is not only wrong, but the way it was done broke + brace variables by stripping their arguments. + +2015-04-16 Kaz Kylheku <kaz@kylheku.com> + + Adding exit* function which calls _exit. + + * sysif.c (exit_star_wrap): New static function. + (sysif_init): Registered exit* intrinsic. + + * txr.1: Documented exit*. + + * tl.vim, txr.vim: Regenerated. + +2015-04-16 Kaz Kylheku <kaz@kylheku.com> + + Add exec and pipe functions. + + * configure: Check for pipe. + + * sysif.c (exec_wrap, pipe_wrap): New static functions. + (sysif_init): Register exec and pipe intrinsics. + + * txr.1: Documented exec and pipe. + + * tl.vim, txr.vim: Regenerated. + +2015-04-16 Kaz Kylheku <kaz@kylheku.com> + + Adding chmod. + + * configure: Detect chmod. + + * sysif.c (chmod_wrap): New function. + (sysif_init): Register chmod intrinsic. + + * txr.1: Documented. + + * tl.vim, txr.vim: Regenerated. + +2015-04-16 Kaz Kylheku <kaz@kylheku.com> + + Adding dotimes macro. + + * eval.c (me_dotimes): New function. + (eval_init): Registering dotimes macro. + + * txr.1: Documented dotimes. + +2015-04-15 Kaz Kylheku <kaz@kylheku.com> + + Fix escaping issues in open-process on Windows. + + This can now execute programs with spaces in their path names. + Arguments can contain embedded double quotes, and other characters + that are special to cmd.exe, like &, |, (, ), ^ and others. + + * stream.c (string_extend_count, win_escape_cmd): New static functions. + (win_escape_arg): Fix and extend escaping scheme to cover not only + the argument processing scheme implemented by programs which use Microsoft's + C library, but also to smuggle the command line through cmd.exe. + (win_make_cmdline): Use win_escape_cmd to wrap the command. + Escape the quotes which are placed around arguments, so cmd.exe + doesn't interpret them, which will cause it to suppress its processing + of the caret escapes. + +2015-04-15 Kaz Kylheku <kaz@kylheku.com> + + Allow quasiquotes in braces and quasiliterals, and quotes in braces. + + * parser.l: Consolidate rules for recognizing quote, unquote, and + quasiquote. An effect of this is that quasiquotes can now occur in + braces and in string quasiliterals. + + * parser.y (quasi_item): Support quotes and quasiquotes as quasi items: + that is to say, i.e. objects denoted by @ in a quasiliteral. + +2015-04-14 Kaz Kylheku <kaz@kylheku.com> + + Diagnose trailing junk in numeric literals. + + * parser.l: Combining the handling of hex, octal and binary numeric + literals into a single rule. Implementing an additional rule which + diagnoses such tokens that have trailing junk. Thus, something + like #x1F2AZ is now invalid syntax. + +2015-04-11 Kaz Kylheku <kaz@kylheku.com> + + Implementing while* and until* loops. + + * eval.c (while_star_s, until_star_s): New symbol variables. + (me_while_star, me_until_star): New static functions. + (eval_init): Initialize new variables. Register while* and + until* macros. + + * txr.1: Documented while* and until*. + + * txr.vim, tl.vim: Regenerated. + +2015-04-11 Kaz Kylheku <kaz@kylheku.com> + + * sysif.c: Use HAVE_FORK_STUFF to wrap fork, waitpid and dup/dup2. + +2015-04-11 Kaz Kylheku <kaz@kylheku.com> + + Exposing dup and dup2. + + * sysif.c (dup_wrap): New static function. + (sysif_init): Register dup_wrap as dupfd. The name dup + conflicts with an existing function. + + * txr.1: Documented dupfd. + + * txr.vim, tl.vim: Regenerated. + +2015-04-10 Kaz Kylheku <kaz@kylheku.com> + + Functions open-fileno and fileno. + + * stream.c (fd_k): New keyword variable. + (stdio_get_prop): Handle the :fd property by returning + the file descriptor. + (open_fileno): New function. + (stream_init): Initialize fd_k, and register fileno and open-fileno. + + * stream.h (open_fileno): Declared. + + * txr.1: Documented open-fileno and fileno. + + * utf8.c (w_fdopen): New function. + + * utf8.h (w_fdopen): Declared. + +2015-04-10 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (sweep): Fix comment referring to nonexistent identifier. + +2015-04-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): There is an until_s global variable, so + no need to intern the symbol when registering me_until. + +2015-04-09 Kaz Kylheku <kaz@kylheku.com> + + Fork and wait support. + + * sysif.c (work_wrap, wait_wrap, wifexited, wexitstatus, + wifsignaled, wtermsig, wcoredump, wifstopped, wstopsig, + wifcontinued): New static functions. + (sysif_init): Register new intrinsics: functions fork, wait, + w-ifexited, w-exitstatus, w-ifsignaled, w-termsig, + w-coredump, w-ifstopped, w-stopsig, w-ifcontinued; and + variables w-nohang, w-untraced, w-continued. + + * txr.1: Documented. + + * tl.vim, txr.vim: Regenerated. + +2015-04-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (iflet_s, when_s): New global variables. + (me_iflet_whenlet): New static function. + (eval_init): Initialize when_s, iflet_s. Use + when_s for registering me_when. + Register me_iflet_whenlet. + + * txr.1: Documented iflet and whenlet. + + * tl.vim, txr.vim: Regenerated. + +2015-04-05 Kaz Kylheku <kaz@kylheku.com> + + New whilet macro. + + * eval.c (while_s): New symbol variable. + (me_whilet): New static function. + (eval_init): Initialize while_s. Use while_s in registration + of while macro. Register new whilet macro. + + * txr.1: whilet is documented. + + * tl.vim, txr.vim: Regenerated. + +2015-04-01 Kaz Kylheku <kaz@kylheku.com> + + New zap operator. + + * eval.c (zap_s): New global variable. + (op_modplace): Support zap. + (eval_init): Initialize zap_s, and register as operator. + + * txr.1: Documented zap. + +2015-03-31 Kaz Kylheku <kaz@kylheku.com> + + Array overrun fix in apply. + + * eval.c (APPLY_ARGS): New preprocessor symbol, replaces + hard-coded 32 inside apply. + (apply): Use APPLY_ARGS for argument array. + Fix overrun of args array in non-variadic function case when + list has more than APPLY_ARGS elements. + Eliminate superflous "variadic" local variable. + +2015-03-31 Kaz Kylheku <kaz@kylheku.com> + + Deal with spurious retention in function application. + + * gc.h (zap): New inline function. + (z): New macro. + + * lib.c (generic_funcall, funcall1, funcall2, funcall4, do_not, + do_dup, do_iff): Use z macro to burn funargs with no next use. + + * eval.c (apply, apply_frob_args, iapply, call, do_eval, + op_dwim, mapcarv, mappendv, lazy_mapcarv_func, mapdov, do_apf, + do_ipf, callf): Use z macro to burn funargs with no next use. + +2015-03-30 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Bugfix: ~X printing in lower case, like ~x, + when argument is a bignum integer. + +2015-03-29 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/eliminate-locale-dependencies: New patch. + (s_mp_tovalue): Eliminate use of isupper, islower. + + * mpi-patches/series: Add patch. + +2015-03-29 Kaz Kylheku <kaz@kylheku.com> + + New -n option. New "i" mode letter in file opening functions. + + * stream.c (struct stdio_mode): New struct type. + (stdio_mode_init_trivial): New initializer macro. + (parse_mode, format_mode, normalize_mode, set_mode_props): + New static functions. + (make_stdio_stream_common): the isatty check, and automatic marking + of tty device streams as real-time is no longer done, unless + backward compatibility 105 or earlier is requested. + (open_file, open_tail, open_command, open_process): Use new + mode parsing, which supports the "i" flag. + (stream_init): If we have isatty, and standard input is a tty, + then mark the *std-input* stream as real-time. + + * txr.1: Document -n/--noninteractive option. Fix typo where + real-time-stream-p is referred to as stream-real-time-p. + Document "i" flag on open-file and others. + + * txr.c (opt_noninteractive): New global. + (help): Help text for -n/--noninteractive. + (txr_main): Handle new options. + + * txr.h (opt_noninteractive): Declared. + +2015-03-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (prinl, pprinl): Become external functions. + (tprint): New function. + (eval_init): Register tprint as intrinsic. + + * eval.h (prinl, pprinl, tprint): Declared. + + * txr.c (txr_main): New option, -t. + + * txr.1: Documented tprint and -t option. + +2015-03-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): register get-lines as having one optional + argument. + + * lib.c (lazy_stream_cons): Default the argument to std_input. + + * txr.1: Document that the stream is optional in get-lines. + +2015-03-24 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (enforce): Update error error message. + + * configure (have_git): Don't set this variable if + the TXR repository is missing, even if git is installed. + I.e. have_git means have have the tool and the repo. + +2015-03-20 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (int_str): Workaround for wcstol recognizing the 0x prefix when + radix is 16. Also, thrown an error if radix is not in the range + 0 to 36. + + * txr.1: Document int-str's radix range restriction. + +2015-03-17 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (help): Added missing documentation for -e and -p. + Also documented -P. + (txr_main): New option implemented, -P. + + * txr.1: Document -P, and clean up -e and -p documentation. + +2015-03-11 Kaz Kylheku <kaz@kylheku.com> + + Version 105. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise. + + * txr.vim, tl.vim: Regenerated. + +2015-03-11 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (run): In Windows version, fix nasty corruption bug: + we were freeing C strings that were not duplicated, but only borrowed + from string objects. + +2015-03-11 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_process, run): Eliminate utf8name local variable + and associated memory leak. Correctly allocate only nargs + 1 elements + for argv array, not nargs + 2. + +2015-03-10 Kaz Kylheku <kaz@kylheku.com> + + * sysif.c (mkdir_nothrow_exists): New static function. + (ensure_dir): New function. + (sysif_init): ensure_dir registered as intrinsic. + + * txr.1: ensure_dir documented. + +2015-02-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.c (open_txr_file, regex_parse, lisp_parse): Functions + moved here from parser.l. + + * parser.l (open_txr_file, regex_parse, lisp_parse): Functions + moved from here to parser.c. + +2015-02-25 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (OBJS): Add parser.o. + + * parser.h (parser_s): Declared. + (parse_init): Declaration removed. + (parser_l_init): Declared. + + * parser.l (parse_init): Function renamed to parser_l_init. + + * parser.c: New file. + +2015-02-25 Kaz Kylheku <kaz@kylheku.com> + + New ignerr macro. + + * eval.c (me_ignerr): New static function. + (eval_init): Registered ignerr macro. + + * txr.1: Documented. + + * tl.vim, txr.vim: Regenerated. + +2015-02-25 Kaz Kylheku <kaz@kylheku.com> + + Introducing persistent error state on streams. + + * lib.c (cobj_ops): New function. + + * lib.h (cobj_ops): Declared. + + * stream.c (null_ops): Initializer updated. + (struct stdio_handle): New member, err. + (stdio_stream_mark): Mark the err member. + (errno_to_string): New static function. + (stdio_maybe_read_error, stdio_maybe_error): Set persistent error + state. Use errno_to_string_function. + (stdio_put_string, stdio_put_char, stdio_put_byte, stdio_seek, + stdio_get_line): Set errno to zero, so that if the underlying + operations do not set errno on err, we don't misinterpret some pervious + errno value as pertaining to the failed stream operation. + (stdio_get_error, stdio_get_error_str, stdio_clear_error): New static + functions. + (stdio_ops, tail_ops, pipe_ops): Update initializer with new functions. + (string_in_get_error, string_in_get_error_str): New static functions. + (string_in_ops): Update initializer with new functions. + (byte_in_get_error, byte_in_get_error_str): New static functions. + (byte_in_ops): Update initializer with new functions. + (string_out_ops, strlist_out_ops): Update initializer with null + pointers for new functions. + (struct dir_handle): New struct type. + (dir_destroy, dir_mark): New functions. + (dir_get_line): Refactor for struct dir_handle context rather than DIR. + Persist error state. + (dir_close): Refactor for struct dir_handle. + (dir_get_error, dir_get_error_str, dir_clear_error): New static functions. + (dir_ops): Update initializer with new functions. + (make_stdio_stream_common): Initialize new err member. + (make_dir_stream): Refactor for struct dir_handle. + (get_error, get_error_str, clear_error): New functions. + (cat_get_error, cat_get_error_str, cat_clear_error): New static + functions. + (cat_stream_ops): Update initializer with new functions. + (stream_init): Register get-error, get-error-str, clear-error intrinsics. + + * stream.h (struct strm_ops): New function pointer members, get_error, + get_error_str and clear_error. + (strm_ops_init): Macro extended with new arguments for new function pointers. + (get_error, get_error_str, clear_error): Declared. + + * syslog.c (syslog_strm_ops): Update initializer with null + pointers for new functions. + + * txr.1: Documented get-error, get-error-str and clear-error. + +2015-02-21 Kaz Kylheku <kaz@kylheku.com> + + Improved error reporting, particularly for macro expansion. + + * eval.c (last_form_expanded): New variable. + (do_expand): New static function; contains previous expand + function. + (expand): Becomes a wrapper for do_expand, with re-entry + counting. + (eval_init): GC-protect last_form_expanded. + + * eval.h (last_form_expanded): Declared. + + * parser.l (regex_parse, lisp_parse): Just use a simple word for + the name of the regex or string parse location, not the entire + expression itself. + + * unwind.c (uw_throw): Check whether expansion was going on + when the unhandled exception was thrown and print additional + information. + +2015-02-20 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (match_regex_right): Bugfix: zero length matches + should return zero length, rather than nil. This is achieved by trying + the match at one past the last character. + +2015-02-20 Kaz Kylheku <kaz@kylheku.com> + + String-returning wrappers for some regex matching functions. + + * eval.c (eval_init): Register search-regst, match-regst + and match-regst-right intrinsics. + + * regex.c (search_regst, match_regst, match_regst_right): New functions. + + * regex.h (search_regst, match_regst, match_regst_right): Declared. + + * txr.1: Documented new variants. + +2015-02-15 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (print_rec): A compound must use parentheses for + elements which have a higher precedence than catenation. + +2015-02-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (list_collect, list_collect_append): In error + messages about bad appends, do not show the list being + appended because it may be an infinite lazy list. + For instance (append* 3 lazy-list). + +2015-02-12 Kaz Kylheku <kaz@kylheku.com> + + New functions pad and weave. + + * eval.c (pad_func, pad, weave_while, weave_gen, weavev): + New static functions. + (eval_init): Registered pad and weave intrisics. + + * txr.1: Documented pad and weave. + +2015-02-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (lazy_appendv_func, lazy_appendv): Bugfix: append* + was silently ignoring lists after the first atom, instead of + throwing an error. Also, it was not detecting the case that + the last argument is a list which should just be returned, + and instead trying to find its tail in preparation for the + next call to lazy_appendv_func, the consequences being runaway + iteration over an infinite list. + +2015-02-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (symacro_k, fun_k): New keyword variables. + (lexical_lisp1_binding): New static function. + (eval_init): Initialize symacro_k and fun_k; register + new intrinsic function lexical-lisp1-binding. + + * match.h (var_k): Existing external name declared. + + * txr.1: Documented lexical-lisp1-binding. + +2015-02-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Add dependency froml $(EXTRA_OBJS-y) to header + file and config.h, so windows resource file is rebuilt + when the version changes. + +2015-02-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Removing bogus eval which refers to nonexistent + DEP_INSTANTIATE macro. + +2015-02-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (clean): Remove $(EXTRA_OBJS-y) so the Windows .res + file is removed. + +2015-02-08 Kaz Kylheku <kaz@kylheku.com> + + Version 104. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2015-02-07 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (trunc_rem): New function. + + * eval.c (eval_init): Register trunc-rem intrinsic. + + * lib.h (trunc_rem): Declared. + + * txr.1: Documented trunc-rem. + + * tl.vim, txr.vim: Updated. + +2015-02-07 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (CFLAGS): Removed puzzling, unnecessary definitions + of XMALLOC, XCALLOC, XREALLOC and XFREE for $(MPI_OBJS). MPI + does not use such macros and the allocator is already retargetted + to use the TXR one. + + * mpi-patches/use-txr-allocator: In this patch, we don't need + external declaration for chk_malloc or chk_realloc because they + are not used. Only chk_calloc is used. + + * mpi-patches/add-bitops: Refreshed. + + * mpi-patches/add-mp-hash: Likewise. + + * mpi-patches/add-mp-set-intptr: Likewise. + + * mpi-patches/add-mpi-toradix-with-case: Likewise. + + * mpi-patches/bit-search-optimizations: Likewise. + + * mpi-patches/export-mp-eq: Likewise. + + * mpi-patches/faster-square-root: Likewise. + + * mpi-patches/fix-bad-shifts: Likewise. + + * mpi-patches/fix-ctype-warnings: Likewise. + + * mpi-patches/fix-mult-bug: Likewise. + + * mpi-patches/mpi-set-double-intptr: Likewise. + + * mpi-patches/mpi-set-mpi-word: Likewise. + + * mpi-patches/mpi-to-double: Likewise. + +2015-02-07 Kaz Kylheku <kaz@kylheku.com> + + Support txr-win.exe target which has no console window. + + * Makefile (EACH_CONF): Iterate over arguments. + (opt/txr-win.o, dbg/txr-win.o): New special object files. + (opt/%-win.o, dbg/%-win.o): New pattern rule to build special object + files. + ($(PROG)-win, $(PROG)-win-dbg): New program targets. + + * inst.nsi: Install txr-win.exe. + +2015-02-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: fixed flurry of phrasing, formatting fumbles. + +2015-02-07 Kaz Kylheku <kaz@kylheku.com> + + * tests/011/macros-1.txr: Add test for lexical function + shadowing symbol macro. + + * tests/011/macros-1.expected: Updated. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * unwind.c (uw_throw): Detect invalid re-entry and bail. + Cautiously, we use fprintf rather than our streams system, in case some + problem with the streams is involved in this mess. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * unwind.c (uw_throw): Put a txr: prefix on the unhandled + exception in early initialization message. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + Slight internal representation change of string-only exceptions. + + One upshot of all this is that (throw 'foo "msg") now + does exactly the same thing as (throwf 'foo "msg"). + A message-only exception really is a one-string exception argument + list ("message ..."), like the documentation says. + + * unwind.h (struct uw_catch): exception member renamed to args. + (uw_catch): Macro follows structure member rename. + + * eval.c (op_catch): Removed now unnecessary kludge of turning non-list + exception argument list into a one-element argument list. + + * match.c (v_try): Similar hack to the one in op_catch + removed here. + + * unwind.c (uw_unwind_to_exit_point, uw_push_catch): Follows rename of + exception member. + (uw_throw): The exception parameter is renamed to args. The kludge + removed from op_catch re-appears here, because numerous calls + to uw_throw just pass a string as args. It's less of a kludge here + because this is the master entry point to exception processing, + and it straightens out the representation right away. + The exception arguments or message are printed in a clearer way. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + Serious bugfix in the op macro, and derived macro, in + the handling of missing meta-numbers (for + instance (op ... @1 .. @3): 1 and 3 occur, but not 2. + + * eval.c (supplement_op_syms): Bugfix: loop was testing + and updating the wrong variable, causing it to be looking at the first + element of the syms list in every iteration. + Not a regression: this was implemented wrong when the function + was introduced on October 6, 2013. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars): Bugfix. When an ordinary variable + occurs in the quasi syntax (which is the result of substitution + of metanums like @1 by the op macro), if that variable + evaluated to a list form, it was not being converted to + a string. The resulting form was then examined for variable + substitutions as if it were part of the quasi syntax. + This was introduced at the same time as the feature of being + able to use metanums in a quasiliteral (Feb 13, 2012: + last change before Version 57). + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (transform_op): When expanding @rest and metanums + like @1, @2, preserve the (sys:var X args ..) structure + if there are args (but of course with X replaced by the appropriate + gensym). This allows syntax like `@{1 10}` to work in quasiliterals: in + other words, modifiers on interpolated metas, on par with @{a 10}. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * sysif.c (abort_wrap): New static function. + (sysif_init): Registered abort intrinsic. + + * txr.1: Documented abort. Changed Unix Programming heading + to System Programming, since exit, abort and errno are not + Unix-specific, but C-specific. + + * tl.vim, txr.vim: Updated. + +2015-02-06 Kaz Kylheku <kaz@kylheku.com> + + * unwind.c (unhandled_hook_s): New static variable. + (uw_throw): In the unhandled case, check if *unhandled-hook* variable + has a function, and use it instead of the default logic. If it's not a + function, abort with an error message. Clear it so that if the hook + function re-enters this code, it will not be used. Always exit now on + unhandled exceptions; do not abort. + (uw_late_init): New function. + + * unwind.h (uw_late_init): Declared. + + * lib.c (init): Call uw_late_init. + + * txr.1: Documented *unhandled-hook*. + + * genvim.txr: Scan the unwind.c file, since it has a reg_var now. + + * tl.vim, txr.vim: Updated. + +2015-02-05 Kaz Kylheku <kaz@kylheku.com> + + * win/env.nsh: Before the potentially length SendMessage, + print an informative message. + +2015-02-05 Kaz Kylheku <kaz@kylheku.com> + + * win/env.nsh: Improve messages about environment + variable modification. + +2015-02-04 Kaz Kylheku <kaz@kylheku.com> + + Symbol macros shadowed by functions under dwim. + + * eval.c (lookup_symac_lisp1, expand_lisp1, + expand_forms_lisp1): New static functions. + (expand): Handle dwim's arguments via + expand_forms_lisp1, rather than as a generic + compound form. + + * txr.1: Revised text in several places to clarify + this expansion rule. + +2015-02-01 Kaz Kylheku <kaz@kylheku.com> + + Version 103 + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2015-02-01 Kaz Kylheku <kaz@kylheku.com> + + Update copyright notices from 2014 to 2015. + + * arith.c, arith.h, combi.c, combi.h, debug.c, debug.h, eval.c, eval.h, + filter.c, filter.h, gc.c, gc.h, hash.c, hash.h, lib.c, lib.h, + match.c, match.h, parser.h, rand.c, rand.h, regex.c, regex.h, + signal.c, signal.h, stream.c, stream.h, sysif.c, sysif.h, syslog.c, + syslog.h, txr.c, txr.h, unwind.c, unwind.h, utf8.c, utf8.h: + Update. + + * LICENSE, METALICENSE: Likewise. + +2015-02-01 Kaz Kylheku <kaz@kylheku.com> + + Windows installer. + + * inst.nsi: New file. + + * win/env.nsh: New file. + +2015-01-31 Kaz Kylheku <kaz@kylheku.com> + + On Windows, add icon and identifying meta-data to executable. + + * Makefile (EXTRA_OBJS-y): New variable. + (WINDRES): New recipe. + + * configure: Check for presence of windres, + and generate add_win_res variable in config.make. + + * win/txr.xcf: New file: GIMP project for icon. + + * win/txr.ico: New file: icon with three sizes generated + from GIMP project. + + * win/txr.rc: New file. + +2015-01-29 Kaz Kylheku <kaz@kylheku.com> + + Use macro to initialize strm_ops. + + * stream.h (strm_ops_init): New macro. + + * stream.c (null_ops, stdio_ops, tail_ops, pipe_ops, string_in_ops, + byte_in_ops, string_out_ops, strlist_out_ops, dir_ops, + cat_stream_ops): Initialize using macro. + + * syslog.c (syslog_strm_ops): Likewise. + +2015-01-29 Kaz Kylheku <kaz@kylheku.com> + + Use macro to initialize cobj_ops. + + * lib.h (cobj_ops_init): New macro. + + * hash.c (hash_ops, hash_iter_ops): Initialize with cobj_ops_init. + + * rand.c (random_state_ops): Likewise. + + * regex.c (char_set_obj_ops, regex_obj_ops): Likewise. + +2015-01-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (make_ubignum): New static function. + (sign_extend): New function. + + * eval.c (eval_init): Register sign-extend intrinsic. + + * lib.h (sign_extend): Declared. + + * txr.1: Documented sign-extend. + + * tl.vim, txr.vim: Updated. + +2015-01-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: fixed use of nonexistent str-int in @(require) example; + fixed bad formatting under logtrunc and partition. + +2015-01-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (get_opt_param_syms): Fix broken function. This + was choking on keywords like :env in the lambda list, + and failing to handle the nested patterns of macro + lambda lists. Failing case: (defmacro x (: opt :env foo)). + +2015-01-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_macro_params): Bugfix: the colon argument's special + meaning "treat the argument as missing" must apply only to optional + parameters. + +2015-01-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (make_like): Fix regression introduced in 102. + A one element list being converted to a string + treated as an empty list, leading to empty string. + +2015-01-27 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (width): New function. + + * arith.h (width): Declared. + + * eval.c (eval_init): Width registered as intrisinc. + + * txr.1: Documented width. + +2015-01-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (call_f): New global variable. + (do_mapf, mapf): New static functions. + (eval_init): Protect call_f from gc, and initialize it. + Re-register call function using call_f. + Register mapf intrinsic. + + * txr.1: Documented mapf. + +2015-01-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register dupl and swap_12_21 as + the dup and flip intrinsics. + + * lib.c (do_dup): New static function. + (dupl): New function. + + * lib.h (dupl): Declared. + + * txr.1: Documented dup and flip. + +2015-01-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (callf): New static function. + (eval_init): callf registered. + + * txr.1: Documented callf. + +2015-01-22 Kaz Kylheku <kaz@kylheku.com> + + Basic implementation of constantp. Does not + recognize forms like (+ 2 2) as constant. + + * eval.c (constantp_noex, constantp): New functions. + (eval_init): Registered constantp as intrinsic. + + * tl.vim, txr.vim: Updated. + + * txr.1: Documented constantp. + +2015-01-21 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (get_string): New argument, close_after_p. + Close the stream unless close_after_p is specified and is nil, + or when it is missing, an there is no compatibility or it's + newer than version 102. + + * stream.h (get_string): Updated declartion. + + * txr.1: Documented default closing behavior of get-string + and the new optional argument for overriding it. + +2015-01-21 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Minor fix in split-str-set documentation. + +2015-01-21 Kaz Kylheku <kaz@kylheku.com> + + Allow macros to tell what symbols have lexical function + or variable bindings. + + * eval.c (lexical_var_p, lexical_fun_p): New local functions. + (eval_init): Registered as intrinsics. + + * txr.1: Documented lexical-var-p and lexical-fun-p. + + * tl.vim, txr.vim: Updated. + +2015-01-13 Kaz Kylheku <kaz@kylheku.com> + + Version 102 + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * tl.vim, txr.vim: Regenerated. + +2015-01-12 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (zerop): Handle character arguments. + (plusp, minusp): New functions. + + * eval.c (eval_init): Register plusp and minusp. + + * lib.h (plusp, minusp): Declared. + + * txr.1: Documented plusp and minusp, and the + handling of characters by zerop. + +2015-01-12 Kaz Kylheku <kaz@kylheku.com> + + Fix for LLVM wchar_t literals not being four byte + aligned, affecting OS X port. + + * configure: Detect a SIZEOF_WCHAR_T when detecting integer + type that will hold a pointer. In the lit_align test, + if we are on Apple Mac OSX, use a lit_align of 2, + so the logic kicks in for padding literals and handling + misalignment. + + * lib.h (litptr): Add a case for LIT_ALIGN < 4 and + SIZEOF_WCHAR_T == 4. In this case we do the arithmetic + on the pointer using short *, and then convert to + wchar_t. + + * HACKING: New section 2.4.3 about new wchar_t hack. + +2015-01-12 Kaz Kylheku <kaz@kylheku.com> + + * signal.h (sig_save_enable, sig_save_disable): + Use do;while(0) trick instead of local typedef + to cause terminating semicolon to be required. + (sig_restore_enable, sig_restore_disable): + Remove superfluous do { } while (0). + Closing a brace is good enough. + +2015-01-11 Kaz Kylheku <kaz@kylheku.com> + + * glob.c: New file. + (glob_wrap, glob_init): New functions. + (errfunc_thunk): New static function. + + * glob.h: New file. + + * txr.c (main): call glob_init if HAVE_GLOB is defined. + + * configure (have_glob): New variable. + (gen_config_make): Add have_glob to config/config.make. + Detect glob function and set have_glob, and add + HAVE_GLOB to config/config.h. + + * Makefile (OBJS): Include glob.h if have_glob is "y". + + * genvim.txr: Scan glob.c for functions and variables also. + + * txr.1: Documented glob and glob-related variables. + +2015-01-11 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (retest): New phony target for convenience. + +2015-01-11 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (make_like): Bugfix. (make-like nil "abc") + must return the empty string, not nil. Only + lists of non-characters fail to convert to a string. + Since programs may depend on the hitherto wrong, + undocumented behavior, this is made subject to + compatibility. + + * txr.1: Added compatibility note. + +2015-01-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (merge_wrap): Make generic over sequences. + + * txr.1: Update documentation for merge function. + +2015-01-05 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_trailer): Bugfix: not returning new variable + bindings captured in trailer section. Ouch! + +2015-01-01 Kaz Kylheku <kaz@kylheku.com> + + Version 101 + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2015-01-01 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (rebuild, clean, repatch): Fail these targets if + the program is not configured. + (distclean): Providing alternative implementation when the + program is not configured. + +2014-12-31 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (replace_str, replace_vec): Bugfix. The replacement item + sequence may be an infinite list in the case that the from argument + specifies a list of indices. We must avoid trying to calculate its + length before we have detected this case. + +2014-12-31 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fix error. (set [seq index-list] new) corresponds to + replace not to refset. + +2014-12-25 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stream_init): Register make_catenated_stream in + a different form as cat-streams. + + * txr.1: Document cat-streams. + + * tl.vim, txr.vim: Regenerated. + +2014-12-25 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (simple_lazy_stream_func): Bugfix: close the stream + if get_line returns nil. + + * stream.c (cat_get_line, cat_get_char, cat_get_byte): The catenated + stream read operations close an exhausted stream before popping + to the next one. + + * txr.1: Document closing behavior of catenated streams. + +2014-12-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (ap_s, ret_s, aret_s): New symbol variables. + (me_ret): Static function renamed to me_ret_aret. + Handles ret and aret macro expansion. + (eval_init): Initialize ap_s, ret_s and aret_s. + Re-register ret macro using ret_s and bind to me_ret_aret. + Register aret macro. + + * txr.1: Documented aret. + + * tl.vim, txr.vim: Regenerated. + +2014-12-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered intrinsic function unique. + + * lib.c (unique): New function. + (uniq): Becomes wrapper around unique. + + * lib.h (unique): Declared. + + * txr.1: Documented unique, and equivalence between uniq and unique. + + * tl.vim, txr.vim: Regenerated. + +2014-12-23 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (func_n1ov, func_n2ov, func_n3ov): New functions. + + * lib.h (func_n1ov, func_n2ov, func_n3ov): Declared. + +2014-12-23 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (func_n0o): Useless, unused function removed. + + * lib.h (func_n0o): Declaration removed. + +2014-12-18 Kaz Kylheku <kaz@kylheku.com> + + Syntax highlighting issue: keywords in braced variable syntax + not being recognized. + + * genvim.txr (txr_braced_ident): New match kind, specifically + for a keyword. + (txr_bracevar): Bracevar region contains txr_braced_indent. + + * txr.vim, tl.vim: Regenerated. + +2014-12-18 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (copy): Support copying random state objects + vi make_random_state. + + * txr.1: Updated. + +2014-12-17 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (rand32): Bugfix: if the seed object is a random + state, we do not want to make the 8 calls to rand32 to + mix up the state; we need are making a straight copy. + + * txr.1: Document the possibility that the seed object + is a random state. Document the platform-independence of the + integer seed. + +2014-12-17 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (struct rand_state): Change cur member to unsigned. + This generates better code for rand32. + +2014-12-16 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (rstate): New inline function. + (rand32): Use inline function instead of macro. + I compared gcc -O2 output on Intel: no difference. + +2014-12-13 Kaz Kylheku <kaz@kylheku.com> + + Factor out some compiling commands into macros. + + * Makefile (COMPILE_C, COMPILE_C_WITH_DEPS, LINK_PROG): New + macro strings. + (dbg/%.o, opt/*.o, %.o, $(PROG), $(PROG)-dbg, conftest, + conftest2): These targets now use these commands. + +2014-12-11 Kaz Kylheku <kaz@kylheku.com> + + Build bugfix: if a config.h header exists in $(top_srcdir), + then that header is used when building in a separate directory, + instead of the config.h generated in that directory. + To fix this, we move config.h into a config/ subdirectory. + The $(top_srcdir)/config is not in the include search path + when buiding in a separate build directory. + While we are at it, let's move all the configure materials + generated by the configure script into config/. + + * Makefile: include config/config.make at the top. Removing the + rule which asserts the existence of configuration based on + the presence of config.make. + (CFLAGS): Add $(conf_dir) to include search path with -iquote. + (notconfigured): New conditionally-defined target for producing + the error message when the build system is not configured. + (NL, DEP): New variables. + ($(OBJS)): Make dependent on config/config.make and config/config.h + with help of DEP macro. + (opt/lex.yy.o, dbg/lex.yy.o): Express dependency using DEP. + (y.tab.h): Split off as a dependent on y.tab.c rather than + a co-target in the rule. The rule has a body to handle + the situation when y.tab.h is missing for some reason, but + y.tab.c already exists (and so won't be re-made, and so + y.tab.h won't be remade). + + * configure: Require GNU Make 3.81 rather than 3.80. + (conf_dir): New variable. + (config_h, config_make, config_log): New variables. These are used in + place of config.h, config.make and config.log. Add conf_dir to + config.make variable. + +2014-12-11 Kaz Kylheku <kaz@kylheku.com> + + * configure (have_git): New variable and configure test for git. + + * Makefile (SRCS): Only set if git is available. + (enforce): Fail if we don't have git. + +2014-12-10 Kaz Kylheku <kaz@kylheku.com> + + Throwing away old dependency system. + + * Makefile (DEPGEN): New macro variable. + (OPT_OBJS, DBG_OBJS): Define with := assignment. + (OBJS): New variable. + (dbg/%.o, opt/*.o): Use -MMD and -MT options of gcc to generate + dependencies. Also use DEPGEN macro to rewrite each dependency + makefile's rule into a DEP_ variable assignment. + (DEP_INSTANTIATE): New macro variable. + (include dep.mk): Removed, replaced by eval hack that + includes all the .d files and instantiates the rule from + the DEP_ variable in each one. + (opt/lex.yy.o, dbg/lex.yy.o): We need to hard code the dependency + of these on y.tab.h, to force that header to generate. + (DEP_opt/lex.yy.o, DEP_dbg/lex.yy.o): New variables, related + to above. + (depend): Target removed. + + * dep.mk: File removed. + + * depend.txr: File removed. + +2014-12-10 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (ABBREV): Rewrite to take advantage of DEP_ + variables to remove the dependencies from the output. + (ABBREV2): No longer needed, removed. + (dbg/%.o, opt/%.o, %.o): Just use ABBREV instead of ABBREV2; it does + the right thing. Dependency of objects on config.make removed. + +2014-12-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (install-tests): Do not use option -c of + cpio (use old ASCII format). This was unintentional. + On extract, use -m option to preserve timestamps, + like we do in INSTALL. + +2014-12-09 Kaz Kylheku <kaz@kylheku.com> + + Test output goes to tst/ now. + + * Makefile (TESTS_OUT): We don't need to refer to top_srcdir. + Add the tst/ prefix to .out paths. Find must be used with -H + because tests is a symlink for out-of-tree build directories. + (tests/*): All target-specific assignments are moved to tst/tests/* + paths, and top_srcdir is no longer used in the right hand side + values. + (%.out): Becomes (tst/%.out). + (%.ok): need to compute .expected name from .out differently to + account for tst/ prefix. + (tests.clean): Just remove the tst directory. + (install-tests): Remove the tst directory first so test commands + are forced to run (though it is a dry run). + We no longer need to switch to top_srcdir to find the .txr + files, nor do we have to filter out .out files from the tests + tree. We no longer have to switch to top_srcdir in the recursive + make. Also, show the installation of tests using ABBREV3. + + * configure: When configuring in a separate directory, symlink + the tests directory back to the source tree. Since there + are now three things being symlinked, rewrote that as a loop. + The recommendation to build in a separate directory is just a note, not + a warning. + +2014-12-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (($TESTS_OUT)): Do not depend on $(PROG). + This adds superfluous commands to run.sh under make install-tests, + and is also inappropriate if a different txr binary is being tested. + (install-tests): Do not override top_srcdir in the recursive make + call; instead use -C to change to $(top_srcdir). + +2014-12-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (all): Mark as phony target. + +2014-12-09 Kaz Kylheku <kaz@kylheku.com> + + Condensed make output. + + * Makefile (VERBOSE): New variable. + (V, ABBREV, ABBREV1, ABBREV3): New macro variables. + (dbg/%.o, opt/%.o): Use V, ABBREV1 for concensed output. + ($(PROG), $(PROG)-dbg, lex.yy.c, y.tab.c, %.out, + %.ok, INSTALL): Likewise, but use ABBREV. + (tests, GREP_CHECK, config.make, conftest.yacc, + conftest.clean): Use $(V) instead of @. + (tests.clean): Remove @; do not hide remove commands. + (INSTALL): Use $(V) instead of @ and use ABBREV3 to + provide condensed output. + (install): Use $(V) on $(PREINSTALL) so we don't see : + in the output. + (install-tests): Put $(V) on all steps. + +2014-12-07 Kaz Kylheku <kaz@kylheku.com> + + Debug builds optional with --debug-also config option. + + * Makefile (PROG): Variable removed, now set in config.make. + (all): Target now depends on $(BUILD_TARGETS) variable, set + in config.make. + + * configure (debug_also) New variable. + (gen_config_make): Generate PROG and BUILD_TARGETS variables. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (tofloat, toint): Handle characters. + Fix error message in toint wrongly identifying itself as tofloat. + + * txr.1: Document handling of characters. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register in function as intrinsic. + + * lib.c (in): New function. + + * lib.h (in): Declared. + + * txr.1: Documented in. + + * txr.vim, tl.vim: Regenerated. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (set_diff): Bugfix: use member rather than find, + so that a nil set element is handled properly. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (INSTALL): Bugfix: touch -r $(2) does not work + right when $(2) is a wildcard like path/to/*.txr. It touches files in + the source tree to the timestamp of the argument after -r. Putting in + a shell loop to handle this. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (TESTS_TMP): New variable. + (TESTS_OUT): Depends on $(PROG). + (TESTS_OK): Does not dependon $(PROG). + (tests): Does not depend on tests.clean. + (retest): Target removed. + (%.out : %.txr): Generate to temporary file, + then move to .out. + (tests.clean): Remove $(TESTS_TMP). + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (SRCS): Compute from top_srcdir, so we don't get a warning + from git, and "make enforce" works from out of three builds. + +2014-12-05 Kaz Kylheku <kaz@kylheku.com> + + * configure: Bugfix: the "share" directory has to be + symbolically linked. Without this, tests which require txr to have + access to its library do not work. + +2014-12-04 Kaz Kylheku <kaz@kylheku.com> + + TXR now builds optimized and debug at the same time. + Optimized object files are under opt/ and debug + object files are under dbg/. The debug txr executable + is called txr-dbg. + + * Makefile (CFLAGS): $(OPT_FLAGS) is omitted from CFLAGS, + so we can expand it where appropriate. + (ADD_CONF, EACH_CONF): New variables, used as macros. + (DBG_OBJS, OPT_OBJS): New variables. + (dbg/%.o, opt/%.o): New rules. + (all): New target. + ($(PROG)-dbg): New target. + (clean): Remove the object directories with rm -rf. + Remove $(PROG)-dbg. + (depend): Pass $(OPT_OBJS) and $(DBG_OBJS) to depend.txr + Also: various target-specific assignments for object files + had to be split for rel and dbg. + + * depend.txr: Updated to handle dbg/ or rel/ prefix on + object file paths. + + * dep.mk: Regenerated. + +2014-12-04 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (clean): add tests.clean as prerequisite. + (tests): Add tests.clean as prerequiste, remove rm command. + (retest, tests.clean): New rules. + +2014-12-04 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (tests/009/json.out): Use eager assignment + intead of lazy assignment to set TXR_ARGS. + +2014-12-04 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (TXR): New variable. We use this to run txr + rather than ./$(PROG). This way we can override the variable + from the command line if necessary. + (TESTS): Variable removed. + (TESTS_OUT, TESTS_OK): New variables, replace TESTS. + (tests): Prerequisites is now $(TESTS_OK), which are + actual timestamp files. Also depends on $(PROG). + (tests/%/%): Target specific assignments are now for .out + targets rather than .ok targets. + (%.out): New pattern target, made from half of previous %.ok rule. + (%.ok): Smaller rule, only performs diff between .out and .expected, + and touches the .ok stamp. + (%.expected): Bugfix: this rule now just copies .out to .expected, + and its prerequisite is %.out rather than %.txr. + Previously it tried to run txr, but it wouldn't use the correct + options, which depended on the target-specific assignments for + various .ok files. + (install-tests): Bugfix: in the "make -s -n" call use the new TXR + variable to specify the program's path. This fixes the issue that ./ + was prepended to the txr commands, resulting in nonworking run.sh. + (txr-manpage.html): Use TXR to invoke txr rather than PROG. + +2014-12-04 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Adding empty .SUFFIXES: to disable built-in suffixes. + Also set MAKEFLAGS to disable all built-in rules. + The only one rule we used is .c to .o, so we provide it ourselves. + +2014-11-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register lequal and gequal. + + * lib.c (lequal, gequal, lequalv, gequalv): New functions. + + * lib.h (lequal, gequal, lequalv, gequalv): Declared. + + * txr.1: Documented lequal and gequal. + + * txr.vim, tl.vim: Regenerated. + +2014-11-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register less and greater to + the lessv and greaterv functions instead of less and greater. + + * lib.c (lessv, greaterv): New functions. + + * lib.h (lessv, greaterv): Declared. + + * txr.1: Document variadic nature of less and greater. + +2014-11-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register sort-group. + + * lib.c (sort_group): New function. + + * lib.h (sort_group): Declared. + + * txr.1: Documented. + +2014-11-21 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (partition_star_func): Bugfix: doing rplaca(env, seq) + too early, before the loop which adjusts its value. + Restructuring this slightly to avoid duplicated code, by moving the + !first check later. + +2014-11-21 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (partition_by_func): Rename one local variable for clarity. + Remove unessential variable last, and move the next variable + into loop scope. + +2014-11-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (wrap_star, wrap): New functions. + + * eval.c (eval_init): Registered wrap and wrap* intrinsics. + + * lib.h (wrap_star, wrap): Declared. + + * txr.1: wrap and wrap* documented. + +2014-11-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (succ, ssucc, sssucc, pred, ppred, pppred): New functions. + + * eval.c (eval_init): Register new functions as intrinsics. + + * lib.h (succ, ssucc, sssucc, pred, ppred, pppred): Declared. + + * txr.1: Documented. + +2014-11-20 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (where): Argument order reversed, with compat support. + + * lib.h (where): Declaration updated. + + * txr.1: Documented. + +2014-11-20 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sel): Accept a function in place of the index list. + + * txr.1: Documented. + +2014-11-17 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (split_str): If the separator string is empty, + then unless opt_compat is 100 or less, provide a more + consistent behavior, rather than splitting the string + into characters. This latter behavior was never documented. + + * txr.1: Documented. + + * dep.mk: Updated. + +2014-11-15 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (max2, min2): Use the less comparison function + for generic semantics. + + * lib.h (max2, min2): Parameter names changed to avoid suggesting + that the operands are numbers. + + * txr.1: Documentation for min and max updated. + +2014-11-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (opip_s, oand_s, chain_s, chand_s): New global variables. + (macro_form_p): Forward declaration added. + (me_opip): New static function. + (eval_init): Intern new symbols, register opip and oand macros + to me_opip function. Use chain_s and chand_s in registration + for chain and chand. + + * txr.1: Document opip and oand + + * tl.vim, txr.vim: Regenerated. + +2014-11-06 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sub, ref, refset, replace, update, search_list): + Fix cut and paste problem: type_mismatch argument expression + referring to the C function cons rather than the intended + object seq. + +2014-11-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register chand intrinsic. + + * lib.c (do_chand): New static function. + (chandv): New function. + + * lib.h (chandv): Declared. + + * txr.1: Documented chand. + +2014-11-06 Kaz Kylheku <kaz@kylheku.com> + + Allow then-func to be omitted in iff. Cleanup. + + * eval.c (eval_init): Register iff as requiring only one arg. + + * lib.c (do_iff): Do not check thenfun for nil. Just call it. + (iff): Default thenfun to identity. + + * txr.1: Documentation updated. + +2014-11-05 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (iff): Bugfix: forgotten handling of default + argument, resulting in : symbol being called as a function. + + * txr.1: Removing description of situation when the then-function + argument of iff or iffi is specified as nil. This is deprecated + behavior which can be obtained by using the false function. + +2014-10-31 Kaz Kylheku <kaz@kylheku.com> + + New macros tb and tc. + + * eval.c (me_tb, me_tc): New static functions. + (eval_init): Registered tb and tc macros. + + * txr.1: Documented tb and tc. + +2014-10-30 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (chk_grow_vec): New function. + (string_extend): Use chk_grow_vec. + (vec_set_length): Avoid subtractions for comparing numbers. + + * lib.h (chk_grow_vec): Declared. + + * stream.c (snarf_line, string_out_put_string): Use chk_grow_vec. + +2014-10-30 Kaz Kylheku <kaz@kylheku.com> + + Implementing finalization hooks. + + * gc.c (struct fin_reg): New struct type. + (final_list, final_tail, mark_makefresh): New static variables. + (mark_obj): Under generational GC, if make_makefresh is in + effect, set the generation to -1 on all marked objects. + (sweep_one): In an EXTRA_DEBUGGING build, call breakpt + if the object being swept is the one in break_obj. + Under generational GC, place reachable objects that are + in generation -1 the freshobj nursery and assign them to generation 0, + rather than sticking them into the mature generation 1. + (sweep): Under generational gc, reset the freshobj_idx variable + here, so that sweep_one has an empty nursery in which + to place the generation -1 objects. + (prepare_finals, call_finals): New static functions. + (gc): Call prepare_finals before sweep, and call call_finals + just before re-enabling GC and returning. Do not reset freshobj_idx to + zero; this was done in sweep, which may have added entries into it. + (gc_finalize): New function. + (gc_late_init): Register gc_finalize as intrinsic function finalize. + + * txr.1: Documented finalize. + + * HACKING: Documented finalization, described the additional + meaning of the -1 generation, and added a section on debugging with + break_obj and breakpt. + +2014-10-30 Kaz Kylheku <kaz@kylheku.com> + + * gc.h (break_obj): Missing extern added on declaration. + This broke C++ compilation with -DEXTRA_DEBUGGING turned on. + +2014-10-29 Kaz Kylheku <kaz@kylheku.com> + + Vim syntax coloring for standalone TXR Lisp (*.tl) files. + + * genvim.txr (generate): New pattern function, contains + generation logic so we can do it twice. + + * txr.vim: Regenerated. + + * tl.vim: New file. + +2014-10-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (type_check): Function moved to header file inline. + (throw_mismatch): New function. + + * lib.h (throw_mismatch): Declared. + (type_mismatch): Declaration turned into inline function. + +2014-10-25 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_begin): Use coerce macro instead of raw C cast. + Use cobj_handle so hash argument is validated. + + * parser.l (YY_INPUT): Use convert macro instead of raw C cast. + +2014-10-24 Kaz Kylheku <kaz@kylheku.com> + + GNU Flex's libfl library provides nothing. Let us not refer to it. It + only causes build issues on some systems where it is not provided + in the standard location, or is not cross-compiled properly. + + * Makefile (LEXLIB): Reference to variable removed. + + * configure (lexlib): Variable removed. + (LEXLIB): config.make variable not generated. + + * parser.l (yywrap): Provide this trivial function as inline. + +2014-10-24 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Removing trailing spaces. + (GREP_CHECK): New macro. + (enforce): Rewritten using GREP_CHECK, with new checks. + + * arith.c, combi.c, debug.c, eval.c, filter.c, gc.c, hash.c, lib.c, + * lib.h, match.c, parser.l, parser.y, rand.c, regex.c, signal.c, + * signal.h, stream.c, syslog.c, txr.c, unwind.c, utf8.c: Remove + trailing spaces. + +2014-10-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Fix incorrect, registration of + repeat; it must have one optional argument. + +2014-10-22 Kaz Kylheku <kaz@kylheku.com> + + Version 100. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2014-10-21 Kaz Kylheku <kaz@kylheku.com> + + Ensure that a hash reorganization doesn't take place + during a traversal, which could cause items to be visited + twice or skipped. + + * gc.c (full_gc): Changed from static to exter (full_gc): Changed to + internal linkage. + + * gc.h (full_gc): Declared. + + * hash.c (struct hash): New member, usecount. + (struct hash_iter): New member, next. + (reachable_iters): New static variable. + (hash_mark): Reset the usecount of every reachable hash table. + (hash_iter_mark): Add every reachable iterator to reachable_iters + list. + (make_hash, make_similar_hash, copy_hash): Initialize usecount. + (gethash_c): Do not call hash_grow if usecount is nonzero. + (hash_begin): Initialize next field of hash_iter to null. + Increment use count on hash table. + (hash_next): When traversal ends, release use count, and + null out the hash table reference. + (do_weak_tables): New static function. + (do_iters): New static function. + (hash_process_weak): Weak hash processing logic moved + to do_weak_tables. Also calls do_iters. + + * txr.1: Describe hash table traversal, and the assurances + and caveats about inserting and deleting items. + +2014-10-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (interp_fun): If the function doesn't have + specials, then don't bother saving and restoring dynamic + env around the argument binding and evaluation. + +2014-10-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register notf intrinsic function. + + * lib.c (do_not): New static function. + (notf): New function. + + * lib.h (notf): Declared. + + * txr.1: Documented notf. + + * share/txr/stdlib/txr-case.txr (bindable): Eliminated. + (txr-if): Use functional expression, taking advantage of notf. + + * txr.vim: Regenerated. + +2014-10-21 Kaz Kylheku <kaz@kylheku.com> + + * share/txr/stdlib/txr-case.txr: New file. + + * txr.1: Document txr-if, txr-when and txr-case. + + * genvim.txr: Added new macro names. + + * tests/011/txr-case.expected: New file. + + * tests/011/txr-case.txr: New file. + + * txr.vim: Regenerated. + +2014-10-20 Kaz Kylheku <kaz@kylheku.com> + + Source file inclusion implemented: needed for macros. + + * match.c (include_s): New symbol variable. + (v_load): Function extended to handle include semantics. + (include): External wrapper function for doing inclusion + via v_load. + (syms_init): include_s initialized. + + * match.h (include_s): Declared. + (include): Declared. + + * parser.y (check_for_include): New static function. + (clauses_rev): Use check_for_include to replace @(include ..) + directive. + + * txr.1: Documented include. + + * genvim.txr: Added include symbol. + + * txr.vim: Regenerated. + +2014-10-19 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_fun): Bugfix: replace incorrect plain return + with debug_return. This causes a stray debug frame to be left + on the environment stack which turns to garbage, leading to + an invalid longjmp in another debug_return elsewhere + which tries to use that frame. This was diagnosed by valgrind + indicating accesses below the stack frame, and also by glibc + "longjmp causes uninitialized stack frame" abort. + +2014-10-19 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (lisp_parse): Bugfix: the error_stream argument + must be checked to be a stream before we plant it in place + of std_error, otherwise we will get a type exception thrown + lower down, which leads to runaway recursion as TXR tries + to print the error messages on std_error. + + * dep.mk: Regenerated. + +2014-10-19 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (r_exprs): New grammar symbol. r_exprs uses + left-recursive rules to avoid filling the yacc stack, and + returns the items in reverse order. The output of each + r_exprs-generating rule consists of a list which gives + the terminating atom, and then the list contents in reverse order. + (n_exprs): Becomes wrapper for r_exprs which deals with + the terminating atom, and reversed items. + +2014-10-18 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Allow TXR to support large programs, and efficiently so. + (clauses_rev): New grammar symbol. Uses a left-recursive rule that + does not consume an amount of parser stack proportional to the number + of clauses, and sticks to efficient consing, which means that + the list is built up in reverse. + (clauses): Now just a wrapper rule for clauses_rev which nreverses + its output. + (clauses_opt): Retargetted to use clauses_rev instead of clauses, + and reverse its output. + +2014-10-18 Kaz Kylheku <kaz@kylheku.com> + + Deal with situation when GC is disabled and the freshobj array runs out + of room. This could happen when parsing a really large TXR program, + since gc is disabled during parsing. Currently it asserts, which + is not acceptable. + + * gc.c (make_obj): If after gc, the freshobj array has not been + emptied (obviously because gc is disabled), do not assert. + Rather, set the full_gc flag to request a full garbage collection + when gc is re-enabled. Furthermore, only place newly allocated objects + into freshobj if full_gc has not been set. Thus, if we exhaust the + freshobj array while gc is off, the full_gc flag is set, and we + discontinue use of that array. When gc is re-enabled, we will do a full + gc pass. A full gc pass ignores freshobj array, so it doesn't matter + that its use was discontinued. + +2014-10-18 Kaz Kylheku <kaz@kylheku.com> + + * match.c (mf_all): Drop data_lineno parameter. Initialize + the corresponding member based on whether or not data is nil. + (do_match_line, mf_from_ml, match_filter, match_fun, extract): + Do not pass starting line number argument to mf_all. + This fixes a bug when the line number at @(eof) for an empty + file comes out as zero. + (mf_args, v_skip, v_fuzz, v_next, v_gather, v_collect, + open_data_source, match_files): Use zero and one instead of num(0) and + num(1). + +2014-10-17 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_eof): Bugfix: we are at EOF not only when + the remaining data is nil but when it is (nil). This + happens for interactive streams. + +2014-10-17 Kaz Kylheku <kaz@kylheku.com> + + * match.c (dest_bind): Remove the restriction of not allowing + @(expr ...) and @var on the left side of a bind. This + is useful, and necessary for @(line @(lisp expr)) to work: + matching computed line numbers and character positions. + + * txr.1: Document use of Lisp on left hand side of bind, + that there is a restriction on the left hand side of a set, + and that Lisp can be used in a line or chr directive for + computed matches. + +2014-10-17 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: New section Type Safety. + Table of contents regenerated with line numbers. + + * HACKING-toc.txr: New file. + +2014-10-17 Kaz Kylheku <kaz@kylheku.com> + + Converting cast expressions to macros that are retargetted + to C++ style casts when compiling as C++. + + * lib.h (strip_qual, convert, coerce): New casting macros. + (TAG_MASK, tag, type, wli_noex, auto_str, static_str, + litptr, num_fast, chr, lit_noex, nil, nao): Use cast macros. + + * arith.c (mul, isqrt_fixnum, bit): Use cast macros. + + * configure (INT_PTR_MAX): Define using cast macro. + + * debug.c (debug_init): Use cast macro. + + * eval.c (do_eval, expand_macro, reg_op, reg_mac, eval_init): + Use cast macros. + + * filter.c (filter_init): Use cast macro. + + * gc.c (more, mark_obj, in_heap, mark, sweep_one, unmark): Use + cast macros. + + * hash.c (hash_double, equal_hash, eql_hash, hash_equal_op, + hash_hash_op, hash_print_op, hash_mark, make_hash, make_similar_hash, + copy_hash, gethash_c, gethash, gethash_f, gethash_n, remhash, + hash_count, get_hash_userdata, set_hash_userdata, hash_iter_destroy, + hash_iter_mark, hash_begin, hash_uni, hash_diff, hash_isec): Use + cast macros. + + * lib.c (code2type, chk_malloc, chk_malloc_gc_more, chk_calloc, + chk_realloc, chk_strdup, num, c_num, string, mkstring, mkustring, + upcase_str, downcase_str, string_extend, sub_str, cat_str, trim_str, + c_chr, vector, vec_set_length, copy_vec, sub_vec, cat_vec, + cobj_print_op, obj_init): Likewise. + + * match.c (do_match_line, hv_trampoline, match_files, dir_tables_init): + Likewise. + + * parser.l (grammar): Likewise. + + * parser.y (parse): Likewise. + + * rand.c (make_state, make_random_state, random_fixnum, random): + Likewise. + + * regex.c (CHAR_SET_L2_LO, CHAR_SET_L2_HI, CHAR_SET_L1_LO, + CHAR_SET_L1_HI, CHAR_SET_L0_LO, CHAR_SET_L0_HI, L0_full, L0_fill_range, + L1_full, L1_fill_range, L1_contains, L1_free, L2_full, L2_fill_range, + L2_contains, L2_free, L3_fill_range, L3_contains, L3_free, + char_set_create, char_set_cobj_destroy, nfa_state_accept, + nfa_state_empty, nfa_state_single, nfa_state_wild, nfa_state_set, + nfa_free, nfa_run, regex_destroy, regex_mark, dv_compile_regex, + reg_derivative, regex_compile, regex_print, regex_run, + regex_machine_init): Likewise. + + * signal.c (sig_mask): Likewise. + + * stream.c (make_null_stream, stdio_stream_print, stdio_stream_destroy, + stdio_stream_mark, stdio_maybe_read_error, stdio_maybe_error, + stdio_put_char_callback, stdio_get_char_callback, stdio_put_string, + stdio_put_char, stdio_put_byte, stdio_flush, stdio_seek, + stdio_get_prop, snarf_line, stdio_get_line, stdio_get_char, + stdio_get_byte, stdio_unget_char, stdio_unget_byte, stdio_close, + tail_strategy, pipe_close, string_in_stream_mark, string_in_get_line, + string_in_get_char, string_in_unget_char, string_in_get_prop, + byte_in_stream_destroy, byte_in_get_byte, byte_in_unget_byte, + string_out_stream_destroy, string_out_byte_callback, + string_out_byte_flush, string_out_put_string, string_out_put_char, + string_out_put_byte, strlist_mark, strlist_out_put_string, + strlist_out_put_char, make_strlist_output_stream, get_list_from_stream, + dir_get_line, dir_close, make_stdio_stream_common, make_pipevp_stream, + make_string_input_stream, make_string_byte_input_stream, + make_string_output_stream, get_string_from_stream, make_dir_stream, + stream_set_prop, stream_get_prop, close_stream, get_line, get_char, + get_byte, unget_char, unget_byte, vformat, put_string, put_char, + put_byte, flush_stream, seek_stream, open_tail, open_process, run, + cat_stream_print, cat_get_line, cat_get_char, cat_get_byte, + cat_unget_byte, cat_unget_char, cat_get_prop, cat_mark, + make_catenated_stream): Likewise. + + * sysif.c (getcwd_wrap): Likewise. + + * syslog.c (syslog_mark, syslog_put_string, syslog_put_char, + syslog_put_byte, syslog_get_prop, syslog_set_prop, make_syslog_stream): + Likewise. + + * txr.c (get_self_path): Likewise. + + * unwind.c (uw_init): Likewise. + + * utf8.c (utf8_from_uc, utf8_to_uc, utf8_to, utf8_dup_from_uc, + utf8_dup_from, utf8_dup_to, utf8_decode): Likewise. + +2014-10-17 Kaz Kylheku <kaz@kylheku.com> + + Purge stray occurrences of "void *" from code base. + + * lib.c (cobj_print_op): In the format call, cast + the C pointer to val, since the ~p conversion now + takes a val rather than void *. + (cptr_equal_op, obj_print, obj_pprint): Remove cast to + void *, since obj now already has the type that ~p expects. + + * lib.h (struct any): Use mem_t * instead of void *. + + * parser.h (yyscan_t): Repeat the definition from inside + the Flex-generated lex.yy.c. + (yylex_init, yylex_destroy, yyget_extra, yyset_extra): Re-declare + using yyscan_t typedef in place of void *. + + * parser.l (yyget_column, yyerrprepf): Re-declare using yyscan_t. + (grammar): Use yyg in place of yyscanner in calls to yyerrprepf. + + * parser.y (yylex, %lex-param): Use yyscan_t instead of void *. + (parse): Use yyscan_t for local variable. + + * signal.c (stack): Change type from void * to mem_t *. + + * stream.c (vformat): Conversion specifier p extracts val + instead of void *. + (run): Use casts that only remove const, not all the way to void *. + + * txr.1: Documented p conversion specifier of format. + + * Makefile (OBJS-y): Initialize with := to make sure it is a + simple variable, and not a macro. + (SRCS): New variable, listing source files. + (enforce): New rule for enforcing coding conventions. + Currently checks for void * occurrences. + +2014-10-16 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (gcd): Fix semantics. If either operand is + zero, return the other operand. + (lcm): New function. + + * eval.c (eval_init): Retarget registration of gcd to + variable argument gcdv function. Register lcm. + + * lib.c (gcdv, lcmv): New functions. + + * lib.h (gcdv, lcm, lcmv): Declared. + + * txr.1: Re-document gcd with coverage of lcm. + + * txr.vim: Regenerated. + +2014-10-16 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (gcd, lognot): Bugfix: bignum results + in fixnum range not normalized. + +2014-10-16 Kaz Kylheku <kaz@kylheku.com> + + * match.c (dest_bind): More detailed log message for variable + mismatch. + +2014-10-16 Kaz Kylheku <kaz@kylheku.com> + + New @(line) and @(chr) directives. + + * match.c (line_s): New variable. + (h_chr, v_line): New static functions. + (syms_init): line_s initialized. + (dir_tables_init): Register v_line and h_chr. + + * match.h (line_s): Declared. + + * txr.1: Document @(line) and @(chr) directives. + + * txr.vim: Regenerated. + +2014-10-15 Kaz Kylheku <kaz@kylheku.com> + + * match.c (subst_vars): Fix buggy rendering of TXR Lisp expressions + that evaluate to lists. For instance `@(list)` renders to the + string "nil", and `@(list 1 2)` renders as "(1 2)". The desired + behavior is "" and "1 2", respectively. + (do_output_line): In output directives, there is a similar problem. + A @(list) in the middle of an output block turns to nil, and + a @(list 1 2) renders in parentheses as (1 2). Furthermore, + there is the additional problem that no filtering is applied + to the interpolated value. These behaviors are subject to the + compatibility option, since they change the externally visible + behavior of TXR programs. + + * txr.1: Document that empty lists in @(output) variable substitutions + turn into nothing. Document value of 100 for -C option, describing + the above issue. + +2014-10-14 Kaz Kylheku <kaz@kylheku.com> + + More type safety, with help from C++ compiler. + + * parser.h (scanner_t): New typedef. Cleverly, we use + yyguts_t as the basis for this, which pays off in + a few places, getting rid of conversions. + (parser_t): scanner member redeclared to scanner_t *. + (yyerror, yyerr, yyerrof, end_of_regex, end_of_char): + Declarations updated. + + * parser.l (yyerror, yyerrorf, num_esc): scanner argument becomes + scanner_t * instead of void *. + (yyscanner): Occurrences replaced by yyg: why should we + refer to the type-unsafe void *yyscanner, when we know that + in the middle of the yylex function we have the properly + typed yyguts_t *yyg. + (end_of_regex, end_of_char): Argument changes to scanner_t * and name + strategically changes to yyg, eliminating the local variable and cast. + + * parser.y (sym_helper): scanner argument becomes + scanner_t * instead of void *. + (%parse-param}: void *scnr becomes scanner_t *scnr. + (define-transform, yybadtoken): Use scanner_t * instead of void *. + (parse): Since yylex_init takes a void **, we use it to initialize + a local void *, and then assign it to the structure member. + +2014-10-14 Kaz Kylheku <kaz@kylheku.com> + + C++ upkeep. + + TXR's support for compiling as C++ pays off: C++ compiler finds + serious bugs introduced in August 2 ("Big switch to reentrant lexing + and parsing"). The yyerror function was being misused; some of + the calls reversed the scanner and parser arguments. Since one + of the two parameters is void *, this reversal wasn't caught. + + * parser.l (yyerror): Fix first two arguments being + reversed. + (num_esc): Change previously correct call to yyerror to follow reversed + arguments, so that it stays correct. + + * parser.y (%parse-param): Change order of these directives + so that the scnr parameter is before the parser parameter. + This causes the yacc-generated calls to yyerror to have + the arguments in the correct order. It also has the + effect of changing the signature of yyparse, reversing its parameters. + (parse): Update call to yyparse to new argument order. + + * parser.h (yyparse): Declaration removed. + (yyerror): Declaration updated. + + * regex.c (regex_kind_t): New enum typedef. + (struct regex): Use regex_kind_t rather than an enum + inside the struct, which has different scoping rules + under C++. + + * txr.c (get_self_path): Fix signed/unsigned warning. + +2014-10-14 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Round of fixes. + +2014-10-13 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register greater function as intrinsic. + + * lib.c (gt_f, lt_f): Global variables removed. + (greater_f): New variable. + (greater): New function. + (find_max, pos_max): Use greater_f as default for testfun, + rather than gt_f. + (find_min, pos_min): Use less_f as default for testfun, + rather than lt_f. + (obj_init): Remove references to gt_f and lt_f. + GC-protect and initialize greater_f. + + * lib.h (greater_f): Declared. + (gt_f, lt_f): Declarations removed. + + * txr.1: Documented greater, and use of less and greater + in pos-min, pos-max, find-min and find-max. + + * txr.vim: Regenerated. + +2014-10-13 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr (sortuniq): Remove unnecessary function, + since (sortuniq list) can be done as (sort (uniq list)). + +2014-10-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (merge_wrap): New static function + (eval_init): Register less as intrinsic. Retarget merge intrinsic to + merge_wrap for proper argument defaulting which is missing from merge, + and which we don't want to introduce there since internal calls + to merge don't erquire it. Change registration of sort so it has + only one required argument, not two. + + * lib.c (less_f): New global variable. + (less_tab): New static array. + (less_tab_init): New static function. + (less): New function. + (sort): Default lessfun argument to the less function. + (obj_init): GC-protect the less_f variable. + Initialize it with a function object made from the less function. + (init): Call less_tab_init. + + * lib.h (enum type): New enumeration member MAX_TYPE, an alias + for the largest type. + (less_f, less): Declared. + + * txr.1: Documented new less function, and that the lessfun + argument in sort and merge is optional, defaulting to less. + + * txr.vim: Regenerated. + +2014-10-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register chr_isblank and chr_isunisp as + intrinsics. + + * lib.c (chr_isblank, chr_isunisp): New functions. + + * lib.h (chr_isblank, chr_isunisp): Declared. + + * regex.h (spaces): Declaration for existing variable added. + + * txr.1: Documented chr-isblank and chr-isunisp. + + * genvim.txr: Add missing sysif.c. + + * txr.vim: Regenerated. + +2014-10-09 Kaz Kylheku <kaz@kylheku.com> + + Fix gc safety issue in abs_path_p function. + + * stream.c (ap_regex): New static variable. + (abs_path_p): Remove local reg variable; replace + uses of reg with ap_regex. + (stream_init): gc-protect ap_regex. + +2014-10-09 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stream_init): No need to gc-protect + std_input, std_output, std_debug, std_error and std_null. These are + not ordinary variables but macros which expand to locations in the + variable binding environment that are already protected. This call was + actually doing nothing, because the symbol involved in the expression + std_input is not yet initialized and so &std_input yields a null + pointer which causes protect to bail. + +2014-10-08 Kaz Kylheku <kaz@kylheku.com> + + Moving system interface functions to separate module out of + the eval and stream modules. + + * Makefile (OBJS): Add sysif.o. + + * dep.mk: Regenerated. + + * eval.c (errno_wrap, daemon_wrap, exit_wrap, usleep_wrap, getpid_wrap, + getppid_wrap, env_hash): Functions moved to sysif.c and changed + to static functions. + (eval_init): Registrations of functions moved to sysif.c. + + * lib.c (init): Call sysif_init. + + * stream.c (w_stat, statf, mkdir_wrap, chdir_wrap, getcwd_wrap, + makedev_wrap, minor_wrap, major_wrap, mknod_wrap): Functions + moved to sysif.c and become static functions. + (stream_init): Registration of stat moved to sysif.c. + (open_files, open_files_star): Bugfix: no longer erroneously included + in #ifdef HAVE_UNISTD_H block. + + * stream.h (mkdir_wrap, chdir_wrap, getcwd_wrap, makedev_wrap, + minor_wrap, major_wrap, mknod_wrap, symlink_wrap, link_wrap, + readlink_wrap): Declarations removed. + + * sysif.c: New file. + + * sysif.h: New file. + +2014-10-08 Kaz Kylheku <kaz@kylheku.com> + + Add test case for recent breakage. + + * tests/006/freeform-3.expected: New file. + + * tests/006/freeform-3.txr: New file. + + * tests/006/passwd: New file. + + * Makefile (TXR_ARGS): Set up for new test case. + +2014-10-08 Kaz Kylheku <kaz@kylheku.com> + + Fix 2011-12-03 regression. The freeform directive code + relies on lazy_str_get_trailing_list and the behavior + of that function changed because (split-str "" any-separator) + changed from returning nil to returning the empty string. + The resulting behavior change of lazy_str_get_trailing_list was later + described in the documentation, thereby codifying it. + + This patch changes the lazy_str_get_trailing_list behavior, + which is poor, and causes infinite looping. We do not want + an extra empty string prepended, because it looks like + a spurious line. + + * lib.c (lazy_str_get_trailing_list): if split_str produces + a the list (""), then just return the unmaterialized list from + the lazy string without prepending that one-element list to it. + + * txr.1: Updated documentation for lazy-str-get-trailing-list. + +2014-10-05 Kaz Kylheku <kaz@kylheku.com> + + Version 99. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * Makefile: Improve binary packaging rules. + + * regex.c: #include <stdarg.h> added. + +2014-10-04 Kaz Kylheku <kaz@kylheku.com> + + Printing of regular expression objects implemented. + + * regex.c (regex_print): New static function. + (regex_obj_ops): Registered regex_print. (print_class_char, + paren_print_rec, print_rec): New static functions. + + * dep.mk: Regenerated. + +2014-10-04 Kaz Kylheku <kaz@kylheku.com> + + Keep regex source code in regex objects, in anticipation + of pretty-printing. Fix object construction bugs. + + * regex.c (struct regex): New member, source. + (regex_mark): Ensure source is visited by garbage collector. + (regex_compile): Store regex_sexp in source. + Fix violations of section 3.2 of HACKING document. + +2014-10-03 Kaz Kylheku <kaz@kylheku.com> + + Eliminating the extra list wrapping applied to regular + expression objects in the syntax tree. The parser + just puts out a #<regex ...> instead of (#<regex ...> regex-syntax). + + * eval.c (do_eval): We no longer need the hack of + treating (#<regex> ...) as a special form which + evaluates to #<regex>. + (expand): We no longer have to skip over regex syntax, + so the case is removed. + + * match.c (h_var, do_txeval, do_match_line): regexp cases are no longer + subcases of consp but stand on their own. In do_match_line, we + introduce a COBJ case into the type switch for regexes. + + * parser.y: regexes are now compiled in the regex and lisp_regex + grammar rules instead of the dependent rules, and are not wrapped in + extra syntax. + +2014-10-03 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_var): Fix regression introduced in 2014-08-11 + commit. The incompleteness of that change broke the case of an unbound + variable followed by a bound variable. The value of the + second variable was still being wrapped in the old complicated + representation before being pushed to the front of the spec. + + * txr.1: Replace bogus text which says that variables are not + bound to regexes, and so regex matches from variable substitutions + do not arise. This works fine after this change. + +2014-10-03 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: rules that rely on values from config.make + now have config.make as a prerequisite. + +2014-10-02 Kaz Kylheku <kaz@kylheku.com> + + Using unified COBJ representation for both regex kinds, + rather than the list-based notation for derivative-based + regexes, and an encapsulated COBJ for NFA-based regexes. + + * lib.c (compiled_regex_s): Variable removed. + (obj_init): Initialization of compiled_regex_s removed. + + * lib.h (compiled_regex_s): Declaration removed. + + * regex.c (struct regex, regex_t): New type. + (regex_destroy): Object is now a regex_t, not nfa_t. + (regex_mark): New function. + (regex_obj_ops): Register regex_mark operation. + (reg_nullable, reg_derivative): Remove cases that handles + compiled_regex_s. + (regex_compile): Output of dv_compile_regex becomes + a cobj nwo. Output of nfa_compile_regex must be + embedded in regex_t structure. + (regexp): Drop the check for compiles_regex_s. + (regex_nfa): Function removed. + (regex_run, regex_machine_init): Use cobj_handle to retrieve regex_t * + pointer and dispatch appropriate code based on regex->kind. + +2014-09-30 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Add PayPal donation button. + + * txr.1: Avoid escapes at the end of a line; man2html doesn't + handle this properly. + +2014-09-30 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: TXR is now bolded in the title. + + * txr.1: More formatting and wording fixes. + +2014-09-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (do_and): Fix andf again: (andf) not producing + a function that returns t, as documented. + +2014-09-26 Kaz Kylheku <kaz@kylheku.com> + + Version 98. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * dep.mk, txr.vim: Regenerated. + +2014-09-26 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Substantially revised with rich troff markup, + and restructured in some places. + + * genman.txr: No longer generate the whole document in + monospaced font. Add hyperlinks. + +2014-09-25 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (do_and, do_or): Fix broken andf and orf, + lacking the semantics of returning the last value, + or the first true value, respectively. + +2014-09-22 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (txr-manpage.pdf): New target. + +2014-09-10 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (compat_fixup): void return changed to int. + Returns minimum supported emulation. + + * lib.h (compat_fixup): Declaration fixed. + + * txr.c (compat): Replace hard-coded min version + by return value of compat_fixup. + +2014-09-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (help): List new --compat option. + (requires_arg, do_fixnum_opt, compat, array_dim, gc_delta): New static + functions. + (txr_main): Use do_fixnum_opt for handling options with integer + argument. Add --compat alias for -C. + + * txr.1: Documented --compat. Added missing description of --debugger. + +2014-09-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Cleaning up option handling code. + Better checking for misused long options. + +2014-09-09 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register intrinsic partition* function. + + * lib.c (partition_star_func): New static function. + (partition_star): New function. + + * lib.h (partition_star): Declared. + + * txr.1: Documented partition*. + +2014-09-09 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register intrinsic partition function. + + * lib.c (partition_func): New static function. + (partition): New function. + + * lib.h (partition): Declared. + + * txr.1: Documented partition. + +2014-09-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (help): Revised help text. + +2014-09-08 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (MALLOC_DELTA_THRESH): Macro remaned to DFL_MALLOC_DELTA_THRESH. + (opt_gc_delta): New global variable. + (make_obj): Use opt_gc_delta rather than MALLOC_DELTA_THRESH. + (gc_set_delta, gc_wrap): New static functions. + (gc_late_init): New function. + + * gc.h (gc_late_init): Declared. + + * genvim.txr: scan gc.c also. + + * lib.c (init): call gc_late_init. + + * txr.1: Document new --gc-delta option and the functions + gc and gc-set-delta. + + * txr.c (help): Help text for --gc-delta. + (txr_main): Parse --gc-delta option. + + * txr.h (opt_gc_delta): Declared. + +2014-09-06 Kaz Kylheku <kaz@kylheku.com> + + Make the garbage collector aware of malloced bytes, to + prevent skyrocketing memory use when the program manipulates + heap objects that have large amounts of malloc memory + attached, like very large bignums. + + * gc.c (MALLOC_DELTA_THRESH): New preprocessor symbol. + (prev_malloc_bytes): New static variable. + (make_obj): when the value of malloc_bytes jumps by MALLOC_DELTA_THRESH + or more since the last gc, trigger gc. + +2014-09-03 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (symbol_setname): New static function. + (obj_init): Change exception symbol names with underscores + to use friendly dashes instead. + (compat_fixup): New function. + + * lib.h (compat_fixup): Declared. + + * txr.1: Change occurrences of exception symbols from + underscores to dashes. + + * txr.c (txr_main): Call compat_fixup when -C option + is processed. + +2014-09-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Document -C option. + + * txr.c (opt_compat): New global variable. + (help): Describe -C option. + (txr_main): Process -C, and set opt_compat. + Ensure -C does not clump. + + * txr.h (opt_compat): Declared. + +2014-09-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Update registration of lisp-parse and read + to account for new parameter. + + * lib.c (syntax_error_s): New symbol_variable. + (obj_init): New symbol variable initialized. + + * lib.h (syntax_error_s): Declared. + + * parser.h (lisp_parse): Declaration updated. + + * parser.l (lisp_parse): Takes third parameter. + + * txr.1: Third parameter of read described. + + * txr.c (txr_main): Pass colon_k to third parameter of lisp_parse + to obtain exception throwing behavior. + + * unwind.c (uw_init): Register syntax-error as subtype of error. + +2014-09-02 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (arith_init): Register some variables: *flo-dig*, + *flo-min*, *flo-max*, *flo-epsilon*, *pi* and *e*. + + * genvim.txr: Include arith.c in scan for symbols. + + * lib.c (init): arith_init() must now be called after eval_init(). + + * txr.1: Documented new variables.. + + * txr.vim: Updated. + +2014-08-29 Kaz Kylheku <kaz@kylheku.com> + + Version 97. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * dep.mk: Regenerated. + +2014-08-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_quasi): Regression: fix broken handling + of @{var mod} syntax triggering a bogus assertion. Mods are the third + element in the abstract syntax now, not fourth. There is no next link. + Removing bogus comment referring to a nonexistent function. + +2014-08-29 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Fix regression introduced in 94: broken @(load). + +2014-08-25 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (top): Renamed to gc_prot_top, turned extern. + (prot1, rel1, mark): Follow rename. + + * gc.h (gc_prot_top): Declared. + + * signal.h (extended_jmp_buf): New member, gc_pt. + (extended_setjmp): Save and restore gc_prot_top. + +2014-08-25 Kaz Kylheku <kaz@kylheku.com> + + GC correctness fixes: make sure we pin down objects for which we borrow + low level C pointers, while we execute code that can cons memory. + + * lib.c (list_str): Protect the str argument. + (int_str): Likewise. + + * regex.c (search_regex): protect the haystack string, + while using the h pointer to its data, since regex_run + can use the derivative-based engine which conses. + + * stream.c (vformat_str): Protect str argument, since + put_char might conceivably cons. + (vformat): Protect fmtstr. + +2014-08-14 Kaz Kylheku <kaz@kylheku.com> + + Version 96. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * txr.vim: Regenerated. + +2014-08-13 Kaz Kylheku <kaz@kylheku.com> + + Uprooting stupidities in handling of output variables. + + * parser.y (o_elems_transform): Remove useless function which + was only unwrapping the strange parse of output vars. + (o_elems_opt, rep_elem, quasilit, wordsqlit): Eliminate + o_elems_transform call. + (o_var, q_var): Eliminate the phrase structure rules which + match an extra o_elem or quasi_item, and which incorporate + them into the var syntax tree element. Place the modifiers + into the third position, not fourth. + + * eval.c (subst_vars): Eliminate handling of "pat" + element. Actually that was not even there thanks to + o_elems_transform being applied: dead code. Pull modifiers + from the third element of the var form now, not fourth. + + * match.c (subst_vars): Similar changes as in the match.c + subst_vars function. Here the pat variable is even more obviously + useless; if it is not nil, it is just punted back to the spec. + +2014-08-13 Kaz Kylheku <kaz@kylheku.com> + + Fix regression in previous change: we must match a compound text + element whole, and not break it up. + + * match.c (search_match): Take a spec argument. + (h_var): Turn a text element into a one-element spec and process + with search_match. + + * txr.1: Updated text about matching of variables + followed by a directive or function, and about consecutive + variables via directive. + +2014-08-12 Kaz Kylheku <kaz@kylheku.com> + + When a variable is delimited by some form other than + the contents of a variable, fixed string or regex, we now + use the entire tail of the specline to find the match. + So for instance @var@(trailer)foo works as intuition + might expect. + + * match.c (search_form): Static function removed. + (search_match): New static function based on search_form. + Does not handle regexes, and does not update c->bindings. + (h_var): Renamed local variable pat to next. Added a few + missing rlcp's. Combined the cases when pat is a cons to + one block so consp isn't repeatedly tested. Function now + handles a var followed by (sys:text ...) elements specially; + the first element of the text block is pulled out and + matched. Implemented "var delimiting spec" general case + which matches the entire tail of the spec at successive + character positions until a match is found, and the skipped + text goes into the variable. + +2014-08-11 Kaz Kylheku <kaz@kylheku.com> + + First cut at restructuring how variable matching works in the pattern + language. The goal is to remove the strict behavior of using only one + element of context after a variable. + variable form at parse time: we unravel that first. + + * parser.y (grammar): Simplifying the phrase structure + rule for the var element. All the variants that have + a trailing elem are removed. The abstract syntax + changes; the modifier moves to the third position in the + list. + + * match.c (h_var): Matching change: the element which follows + a variable is now pulled from the specline rather than the variable + syntax, which is how it should have been done in the first place. The + modifiers are pulled from a different spot in the variable syntax. + +2014-08-09 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (filter_init): Expose the trie-lookup-begin, + trie-value-at and trie-lookup-feed-char functions as intrinsics. + + * txr.1: Document exposed functions. + +2014-08-08 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: ret operator does not generate functions + that take any number of arguments. + + * eval.c (me_ret): Generate (op identity (progn @rest <expr>)) + rather than of (op identity <expr>). + + * txr.1: Update misleading equivalence. + +2014-08-07 Kaz Kylheku <kaz@kylheku.com> + + Version 95. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2014-08-07 Kaz Kylheku <kaz@kylheku.com> + + No getppid on MinGW. + + * configure: Add test for getppid. + + * eval.c (getppid_wrap, eval_init): use HAVE_GETPPID + +2014-08-07 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (yyerr): Function removed; it is not used in the lexer, + and converted to a macro in the parser. + + * parser.y (define_transform): take a parser argument rather + than scanner. Set up scnr local variable for yyerr macro. + Remove scnr argument from macro calls. + (yyerr): New macro. + (grammar): Remove scnr argument from yyerr calls. + +2014-08-07 Kaz Kylheku <kaz@kylheku.com> + + Reentrant parser regression + + * parser.y (yybadtok): New macro. + (yybadtoken): Function must take parser argument. + (grammar): Replace uses of yybadtoken with yybadtok. + + * parser.h (yybadtoken): Declaration updated. + + * parser.l (grammar): Fix incorrect yyprepf calls + that are missing the yyscanner parameter. + +2014-08-05 Kaz Kylheku <kaz@kylheku.com> + + Version 94. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * txr.vim, dep.mk: Updated. + +2014-08-05 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (make_random_state): Make the seeding behavior + portable when the seed is a fixnum or bignum. The goal is that the same + values of seed should produce the same random sequences on any + platform. + +2014-08-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Fix incorrect registration of + random-fixnum, which doesn't allow it to be called + with no arguments. + +2014-08-05 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Back port from Berkeley Yacc to GNU Bison. + We need a prototype of yylex that is in scope of the grammar, + but YYSTYPE is not defined there. + + * parser.l: Bison 3 declares yyparse in y.tab.h, so we have to + reorder some #includes. + +2014-08-02 Kaz Kylheku <kaz@kylheku.com> + + Big switch to reentrant lexing and parsing. + + * parser.l (YY_INPUT): Stop relying on removed yyin_stream; + refer to stream via yyextra. + (yyin_stream, lineno, errors, spec_file_str, + prepared_error_message): Global variables removed. + (yyget_column, yyset_column): Missing prototypes not generated by flex + in bison bridge mode have to be added by us to avoid + warning. + (yyerror): Takes parser and scanner as parameters. Prepared error + message is now in the parser context. Calls to other error handling + functions receive scanner context. + (yyerr): New function. + (yyerrorf, yyerrprepf): Takes scanner argument, chases extra data to + get to parser, and refers to parser variables instead of globals. + (num_esc): Scanner argument added. + (%option reentrant, %option bison-bridge, %option extra-type): New + flex options. + (grammar): yyscanner added everywhere. + (end_of_char): Takes scanner argument. + (parse_init): Removed references to yyin_stream and + prepared_error_message. + (parse_reset): Function renamed to open_txr_file. Returns + results via pointers instead of setting global variables. + (regex_parse, lisp_parse): Use reentrant parser interface. + + * parser.y (yyerror): Prototype removed. + (yylex): Prototype moved after grammar, with new arguments. + (sym_helper, define_transform): Take scanner argument. + (make_expr): Takes parser argument. + (rlrec): New static function. + (rl): Function turned into macro. + (mkexp, symhlpr): New macros. + (%purse-parser, %parse-param, %lex-param): New Yacc options. + (grammar): Actions re-worked for reentrance. Parser and scanner + contexts are passed down to helper functions, in some cases + via the three new macros. The result of the parse is stored + in the syntax_tree member of the parser_t structure instead + of a global. The yylex function receives the scanner instance. + (get_spec): Function removed. + (parse): New function. + + * parser.h (lineno, errors, yyin_stream, spec_file_str): + Declarations removed. + (parser_t): New struct. + (yyerr): New function declared. + (yyparse, yyerror, yyerrorf, end_of_regex, end_of_char, + yylex, yylex_destroy): Declarations updated. + (yylex_init, yyget_extra, yyset_extra): Declared. + (parse_reset, rl): Declaration removed. + (open_txr_file): Declaration added. + (parse): New function. + + * match.c (v_load): Use new reentrant parser interface. + + * txr.c (txr_main): Stop using parser-related global variables; + call parser using new reentrant interface. + +2014-08-02 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (interrupt_count): New global variable. + (sig_handler): Increment and decrement interrupt count. + If the interrupt count is already positive, treat + that as if async signals were not enabled: do not + invoke handlers. + +2014-08-01 Kaz Kylheku <kaz@kylheku.com> + + * configure (have_sys_time): New variable. + Added check for setitimer/getitimer which also + checks for <sys/time.h>. + + * signal.c (sig_init): Register itimer-real, itimer-virtual, + itimer-prof variables and getitimer and setitimer functions. + (tv_to_usec): New static function. + (getitimer_wrap, setitimer_wrap): New functions. + + * signal.h (getitimer_wrap, setitimer_wrap): Declared. + + * txr.1: Documented itimers. + +2014-08-01 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (set_sig_handler): Don't use SA_ONSTACK + except for signals for which we use alt stack. + + * signal.h (sig_save_enable): Do sig_check before + enabling async signals. This caused the !async_sig_enabled assertion + to trip inside make_obj. + +2014-08-01 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (sig_hup, sig_int, sig_quit, sig_ill, sig_trap, sig_abrt, + sig_bus, sig_fpe, sig_kill, sig_usr1, sig_segv, sig_usr2, sig_pipe, + sig_alrm, sig_term, sig_chld, sig_cont, sig_stop, sig_tstp, sig_ttin, + sig_ttou, sig_urg, sig_xcpu, sig_xfsz, sigtalrm, sig_prof, sig_poll, + sig_sys, sig_winch, sig_iot, sig_stkflt, sig_io, sig_lost, sig_pwr): + Remove unused global variables variables. + + * signal.h: Remove declarations thereof. + +2014-07-30 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (getpid_wrap, getppid_wrap): New static functions. + (eval_init): Registered getpid and getppid intrinsics. + + * signal.c (kill_wrap): New static function. + (sig-init): Registered kill intrinsic function. + + * txr.1: Documented getpid, gettpid and kill. + +2014-07-30 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Allow unquotes and splices in QSPECIAL and BRACED states. + + * parser.y (quasi_item): Support splices as items. + + * genvim.txr: Syntax highlighting support for unquotes in + quasiliterals. + + * txr.vim: Updated. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register uniq function. + + * lib.c (uniq): New function. + + * lib.h (uniq): Declared. + + * txr.1: Documented uniq. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (repeatv): Renamed to repeat. Turned into function + with one optional argument, reflecting existing behavior. + (eval_init): Registration of repeat updated. + + * txr.1: Fixed incorrect documentation which falsely suggests + that repeat takes multiple lists, and doesn't mention + the optional count. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus, minus, mul): Fix missing breaks + to switch statements, leading to strange error + messages when wrong types are passed in. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register nconc as intrinsic. + + * lib.c (nconcv): New function. + + * lib.h (nconcv): Declared. + + * txr.1: Documented nconc. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (reduce_left, reduce_right): Nullify incoming sequence + argument so empty vectors and strings can be processed. + +2014-07-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (giterate_func, giterate): Adjusting semantics of giterate to + include seed item in the sequence. The while test is applied to + each item before it is passed through the function. Also, + eliminated useless rplacd operation. + + * txr.1: Updated documentation of giterate. + +2014-07-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register partition-by intrinsic. + + * lib.c (partition_by_func): New static function. + (partition_by): New function. + + * lib.h (partition_by): Declared. + + * txr.1: Documented partition-by. + +2014-07-28 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (rising_product): Fix wrong m == n case, + which breaks the n_perm_k function for k == 1. + + * txr.1: Fix typo: n-choose-k written in place of n-perm-k. + +2014-07-27 Kaz Kylheku <kaz@kylheku.com> + + Fix 2014-02-05 safety regression: unchecked use of non-function objects + in some contexts that require functions. + + * lib.c (funcall, funcall1, funcall2, funcall3, funcall4): + check type(fun) before dereferencing to see whether there + are optional args. + +2014-07-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (giterate_func, giterate): New static functions. + (eval_init): Registered giterate as intrinsic. + + * txr.1: Documented giterate. + +2014-07-26 Kaz Kylheku <kaz@kylheku.com> + + * unwind.h (uw_block_begin): Get rid of the unused typedef trick, + replace with do { } while (0), since we don't have any instances of a + uw_block_begin followed by declarations. + (uw_catch_begin): Move SYMVAR and EXCVAR inside the do/while block, + so that the expansion doesn't begin with a declaration. + +2014-07-23 Kaz Kylheku <kaz@kylheku.com> + + * Makefile, arith.c, arith.h, combi.c, combi.h, configure, debug.c, + debug.h, eval.c, eval.h, filter.c, filter.h, gc.c, gc.h, hash.c, + hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y, + rand.c, rand.h, regex.c, regex.h, signal.c, signal.h, stream.c, + stream.h, syslog.c, syslog.h, txr.c, txr.h, unwind.c, unwind.h, + utf8.c, utf8.h: Synchronize license header with LICENSE. + +2014-07-22 Kaz Kylheku <kaz@kylheku.com> + + * match.c (subst_vars): Bugfix: I neglected to apply the + filter which is in effect to the result of interpolating a TXR Lisp + expression, oops! + +2014-07-22 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (METANUM): Forgotten %right declaration for this + token has been resulting in a shift-reduce conflict. + +2014-07-22 Kaz Kylheku <kaz@kylheku.com> + + * configure: Add a check, in the case that we cannot make an + executable, whether this is due to being required to use C99. + For instance, the Solaris environment requires compilation + using the C99 dialect if _XOPEN_SOURCE is set to 600 or higher. + + * debug.c: When compiling as C99, we have to obey the special + C99 conventions for instantiating inline functions. + + * hash.c: Likewise. + + * lib.c: Likewise. + + * parser.y: Likewise. + + * unwind.c: Likewise. + +2014-07-21 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + + * arith.c (log2): Change to static. + +2014-07-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (me_case): Fix quoting issue in expansion. + +2014-07-20 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (set_sig_handler): Set up and tear down alternate + stack for SIGBUS also, not only for SIGSEGV. On Linux, + when stack growth collides with another memory mapping, + it triggers a SIGBUS not SIGSEGV, which occurs when the + process limit on stack size is exceeded. In either + situation, we need an alternative stack. + +2014-07-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (caseq_s, caseql_s, casequal_s, memq_s, memql_s, memqual_s, + eq_s, eql_s, equal_s): New symbol variables. + (me_case): New static function. + (eval_init): Initialize new variables. Register caseq, caseql and + casequal macros. + Re-register memq, memql, memqual, eq, eql and equal using + new symbol variables. + + * txr.1: Document case, caseql and casequal. + +2014-07-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register juxt as intrinsic. + + * lib.c (do_juxt): New static function. + (juxtv): New function. + + * lib.h (juxtv): Declared. + + * txr.1: Documented juxt. + +2014-07-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (divi): Support one-argument form. + Use "/" name in error reporting, not "divi". + + * eval.c (eval_init): Change registration of / so only + one argument is required out of two. + + * txr.1: Document one-argument division. + +2014-07-20 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Fixed highlighting issues in numbers followed by newline. + + * txr.vim: Regenerated. + +2014-07-19 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (search_str): Support negative starting index. + Hoist uselessly repeated c_str operation out of loop. + + * txr.1: Document negative starting index for search-str. + +2014-07-19 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_construct): Nullify the pairs argument so that + it works correctly with an empty vector. + + * txr.1: Write missing documentation for hash-construct. + +2014-07-19 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (mapdov): New function. + (eval_init): Register mapdo intrinsic. + + * lib.c (mapdo): New function. + + * lib.h (mapdo): Declared. + + * txr.1: Documented mapdo. Documented zero-argument case + for mapcar, mapcar*, mappend and mappend*. + +2014-07-19 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (put_strings, put_lines): New functions. + (stream_init): Registered new functions as intrinsics. + + * stream.h (put_strings, put_lines): Declared. + + * txr.1: Documented. + +2014-07-19 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register get-lines as a synonym for lazy-stream-cons. + + * stream.c (get_string): New function. + + * stream.h (get_string): Declared. + (stream_init): get_string registered as get-string intrinsic. + + * txr.1: Documented get-string, and get-lines as a synonym for + lazy-stream-cons. Documented the behavioral difference of + lazy-stream-cons lazy lists for real-time and non-real-time streams. + +2014-07-18 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (conftest, conftest2): Link math + library so we can test for math functions. + + * arith.c (log2_init): New static function. + (log2, logtwo): New functions. + (l2): New static variable. + (arith_init): Call log2_init. + + * configure (lang_flags): Switching _XOPEN_SOURCE + from 500 to 600 to reveal log2. + Adding test for log2. + + * eval.c (eval_init): Register log2 intrinsic. + + * lib.h (logtwo): Declared. + + * txr.1: Documented log2. + +2014-07-15 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_do, v_require): Set up and tear down environment frame, + like other situations that evaluate TXR Lisp from the pattern language. + Otherwise obscure things will go wrong. + (h_do): Same as above, and additionally, add the forgotten + call to install the bindings into the match context. + +2014-07-15 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_eol): Fix broken horizontal @(eol). + It should be returning next_spec_k, rather + than bindings, which indicate a complete match. + +2014-07-15 Kaz Kylheku <kaz@kylheku.com> + + * tests/001/query-1.txr: Remove bogus public domain header. + + * tests/001/query-2.txr: Likewise. + + * tests/001/query-3.txr: Likewise. + + * tests/001/query-4.txr: Likewise. + + * tests/002/query-1.txr: Likewise. + +2014-07-15 Kaz Kylheku <kaz@kylheku.com> + + Fix old, known syntax highlighting problems between symbols, + integers and floating literals. + + * genvin.txr (txr_num): The general solution is to over-match + a number by one character: a non-token constituent, and then + subtract that from the highlight region using me=e-1. + This solves the prefix ambiguities between numbers and symbols. + (txr_badnum): New match: matches floating literals with trailing + junk, which are highlighted as errors. + (txr_directive, txr_list, txr_bracket, txr_mlist, txr_mbracket): + All of these contain a txr_badnum. + + * txr.vim: Regenerated. + +2014-07-15 Kaz Kylheku <kaz@kylheku.com> + + Fix broken regex highlighting. + + * genvim.txr (txl_regex): Missing syntactic region added. + (txr_regex): Start pattern is just a slash; no optional # in front. + + * txr.vim: Regenerated. + +2014-07-14 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Allow Lisp code executed via -e and -p options to + see the bindings set up by prior -D options. + +2014-07-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register interpose and lconsp as intrinsics. + + * lib.c (lconsp, interpose): New functions. + (lazy_interpose_func, lazy_interpose): New static functions. + + * lib.h (lconsp, interpose): Declared. + + * txr.1: Documented lconsp and interpose. + +2014-07-10 Kaz Kylheku <kaz@kylheku.com> + + Version 92. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * share/txr/stdlib/ver.txr: Likewise + +2014-07-10 Kaz Kylheku <kaz@kylheku.com> + + * tests/011/macros-2.txr: Added test for labels shadowing macro, + and let shadowing symacro. + + * tests/011/macros-2.expected: Regenerated + + * txr.vim: Regenerated. + +2014-07-10 Kaz Kylheku <kaz@kylheku.com> + + Implementing local function binding constructs. + + * eval.c (fbind_s, lbind_s, flet_s, labels_s): New symbol globals. + (env_fb_to_fb): New static function. + (lookup_mac): Implement the same shadow check for function macros + that is done for symbol macros, because we now have local functions + that can shadow local macros. + (fbindings_helper, op_fbind, make_fun_shadowing_env, + expand_fbind_vars, me_flet_labels): New static functions. + (expand): Add cases for fbind and lbind. + (eval_init): Intern the four new symbols. + Register sys:fbind and sys:lbind operators. + Register flet and labels macros. + + * txr.1: Documented flet and labels. + +2014-07-08 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bindings_helper): New parameter to indicate that + the caller needs the return value, so the function + can avoid consing up a list that will be thrown away. + (op_let): Pass nil argument to new parameter of bindings_helper. + (op_each, op_for): Pass t to indicate that the return value + is needed. + +2014-07-08 Kaz Kylheku <kaz@kylheku.com> + + * LICENSE: Adding copyright for MPI Library. + + * METALICENSE: Adding notes about MPI. + +2014-07-07 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: install the LICENSE and METALICENSE files into the data + directory. + + * txr.c (help): Help text updated to document --license option. + (license): New function. + (txr_main): Implement --license option. + + * unwind.h (uw_catch): Add cast to suppress warning about unused symbol. + + * txr.1: Document --license option. + +2014-07-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (replace_list, sel): fix use of unused variable. + +2014-07-07 Kaz Kylheku <kaz@kylheku.com> + + * LICENSE: Small wording change. Add pointer to METALICENSE. + + * METALICENSE: New file. + +2014-07-07 Kaz Kylheku <kaz@kylheku.com> + + * LICENSE: Switching to simplified two-clause BSD license, + but with modifications. Adding back the no-liability + part of the disclaimer, but with abbreviated text. + Also, rewriting the initial paragraph to make it clear + that the two conditions cover redistribution, and that + use of the software is tantamount to agreeing with the + disclaimer. + +2014-07-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Spell check. + +2014-07-06 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Corrections from Roman Mishin + +2014-07-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apf_s, ipf_s): New symbol variables. + (me_ap, me_ip): Generate a much simpler macro-expansion: + a call to apf or ipf, rather than lambda syntax. + (me_ado, me_ido): New static functions. + (eval_init): Initialize apf_s and ipf_s variables. + Use them in the registration calls for apf and ipf. + Register ado and ido macros. + + * txr.1: Streamlined documentation for ap and ip, + and added ado and ido. + +2014-07-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apf, ipf): Bugfix: do_apf and do_ipf + should be registered as variadic functions. + +2014-07-02 Kaz Kylheku <kaz@kylheku.com> + + Version 91. + + * RELNOTES: Updated. + + * configure, txr.1: Bumped version. + + * dep.mk, txr.vim: Regenerated. + +2014-07-02 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: use $(EXE) in install: recipe. + + * configure: set up EXE variable in config.make, not exe. + +2014-07-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented usleep. + +2014-07-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc): Support indexing using a list of positions, + such as obtained by the where function. + + * lib.c (replace_list, replace_str, replace_vec): Allow the + from argument to be a list of index positions, possibly empty. + + * txr.1: Condensed syntactic descriptions under dwim operator. + Range Indexing section no longer says that the value nil + can be used as either endpoint of a range. This will not + work any longer since a "from" value of nil looks like an + empty list of indexes. Documented new behavior under + replace, and shortened documentation for replace-list, + replace-str and replace-vec. + +2014-06-30 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sel): Bugfix: not implemented correctly for sequences + which are lists. + +2014-06-28 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (bit): New function. + + * eval.c (eval_init): Register bit as intrinsic. + + * lib.h (bit): Declared. + + * mpi-patches/add-bitops (mp_bit): New function. + + * txr.1: Documented bit + +2014-06-28 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/add-bitops (mp_and, mp_or, mp_xor, mp_shift): Plug memory + leaks caused by wrongly initializing the temporary destination operand + for mp2_comp with a size, which mp2_comp then clobbers by doing the same + thing. Also plug memory leaks that happen in in out-of-memory return + case. + +2014-06-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (where, sel): Extend into hashes. + + * txr.1: document hash support for select and where. + +2014-06-27 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: apply_intrinsic and iapply must not destructively + manipulate argument lists. + + * eval.c (apply_frob_args): Rewrite to non-destructive + one-pass version. + (iapply): Likewise. + + * lib.c (term): New function. + + * lib.h (term): Declared. + +2014-06-26 Kaz Kylheku <kaz@kylheku.com> + + Fixes to bignum bit operations affecting pretty much all bit operations + when bit field operands are negative, affecting logand, logor, logxor, + lognot, logtrunc, logtest and ash. In addition, logtest was found to + return the logical inverse of its correct value. + + * arith.c (logtest): Fix broken boolean polarity + of return value. + + * mpi-patches/add-bitops (mp_2comp): Fix incorrect + treatment of negative values. + (mp_and): Fix incorrectly ordered statements, which + cause failure when operands are negative. + +2014-06-26 kaz kylheku <kaz@kylheku.com> + + * eval.c (iapply_s): new global variable. + (iapply, me_ip): new static functions. + (do_apf): Bugfix: use apply_intrinsic, not apply. + (do_ipf, ipf): New static functions. + (eval_init): initialize iapply_s. register me_ip macro + expander, and the iapply and ipf functions. + + * txr.1: Documented iapply, ipf and ip. + +2014-06-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (last): Bugfix: reversed null test. + +2014-06-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): register range_regex and tok_where + as intrinsics. + + * lib.c (tok_where): New function. + + * lib.h (tok_where): Declared. + + * regex.c (range_regex): New function. + + * regex.h (range_regex): Declared. + + * txr.1: Documented tok-where and range-regex. + +2014-06-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (search): Bugfix in type mismatch error message: + trying to print cons function pointer as value. + +2014-06-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (mapcarv): Use mapcar_listout, so list_of_lists can be + a non-list sequence. + (eval_init): Register transpose and zip as intrinsics. + + * lib.c (curry_12_1_v): New static function. + (transpose, mapcar_listout): New functions. + (mapcar): Redefined in terms of mapcar_listout. + + * lib.h (transpose, mapcar_listout): Declared. + + * txr.1: Documented transpose and zip. + +2014-06-20 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: macros not being expanded in expansions embedded in + quasilierals: i.e. the forms X and Y in `@{X}` + and `@{X Y}`, where X and Y can be Lisp symbol macros or compound + forms that is a macro call. + + * eval.c (expand_quasi): Handle the var forms in a quasi. + + * parser.y (n_exprs_opt, q_var): New grammar nonterminals. + q_var is a clone of o_var, but with different construction behavior. + It fixes the bug that o_var applies expand_meta to embedded + Lisp forms, which is not appropriate for TXR Lisp quasiliterals. + (quasi_item): Derive q_var rather than o_var. + +2014-06-20 Kaz Kylheku <kaz@kylheku.com> + + Optimization: add missing tail updates to some list + collecting loops. + + * lib.c (tuples_func, where, sel): Catch return value of + list_collect and update tail variable. + + * match.c (do_txeval): Likewise. + + * parser.y (expand_meta): Likewise for list_collect_nconc. + +2014-06-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (do_apf, apf): New functions. + (eval_init): Register apf as intrinsic. + + * txr.1: Document apf. + +2014-06-20 Kaz Kylheku <kaz@kylheku.com> + + Improve error reporting. + + * eval.c (eval_error): Use last_form_evaled if form is null. + (apply): Substitute apply symbol if ctx_form is null. + (apply_intrinsic): Do not pass fake (apply) context form to + apply, just pass nil. + +2014-06-19 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (identity_s): New global variable. + (me_ret, tf, nilf, do_retf, retf): New static functions. + (eval_init): Initialize identity_s, and use it for registration + of identity. Register ret macro, and the retf, tf and nilf + functions. + + * txr.1: Documentation for ret, retf, tf and nilf. + +2014-06-19 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: dwim operator contradicts the documentation + and intended design. + + * eval.c (do_eval): When calling do_eval_args to evaluate + the arguments of a compound form that is a function call, + do not pass down the lookup function, but substitute &lookup_var. + Passing down the lookup function means tha all evaluation + enclosed in (dwim ...) or [...] follows the Lisp-1 style. + +2014-06-19 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (me_ap): New static function. + (eval_init): Use new list_f instead of func_n0v(identity). + Register multi as intrinsic. Register me_ap as ap macro. + + * lib.c (list_f): New global variable. + (multi): New function. + (multi_sort): Use list_f in place of func_n0v(identity). + (obj_init): gc-protect and initialize list_f. + + * lib.h (list_f, multi): Declared. + + * txr.1: Documented multi and ap. + +2014-06-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register member and member_if as intrinsics. + + * lib.c (member, member_if): New functions. + + * lib.h (member, member_if): Declared. + + * txr.1: Documented. + + * txr.vim: Regenerated. + +2014-06-17 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (generic_funcall): Bugfixes: support symbols. + Removed dubious statement which clamps nargs to the number + of fixed parameters, breaking variadic calls. + Test case: (mapcar 'list '(1 2)) -> ((1) (2)) + Note: generic_funcall is only used when non-function objects + are used as functions; variadic funcalls were not broken. + +2014-06-17 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (generic_funcall): Bugfixes: support symbols. + Removed dubious statement which clamps nargs to the number + of fixed parameters, breaking variadic calls. + Test case: (mapcar 'list '(1 2)) -> ((1) (2)) + +2014-06-17 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fix broken examples for some and none functions. + +2014-06-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (not_null): New static function. + (eval_init): Use null_f in existing registration of null + and not. Add registration for not_null as "true", + and for null_f as "false". + + * txr.1: Documented true and false. + +2014-06-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): register tuples as intrinsic. + + * lib.c (tuples_func): New static function. + (tuples): New function. + + * lib.h (tuples): Declared. + + * txr.1: Documented. + +2014-06-16 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print): Render character DC00 as "pnul". + Clean up code which chooses rendering for characters. + Print C0 and C1 control characters, as well as D800-DFFF, + FFFE and FFFF and characters above FFFF using hex; + others are printed using the #\<char> notation. + + * parser.y (char_from_name): map "pnul" to DC00. + + * txr.1: Documented pnul, clarified character + printing rules, and added a cautionary note about + possible ambiguity in printing. + +2014-06-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register pos_max, pos_min, find_max, + find_min and seqp as intrinsics. + + * lib.c (gt_f, lt_f): New variables. + (to_seq): renamed to toseq. + (seqp): New function. + (minmax): New static function. + (replace_str, replace_vec): Follow to_seq renaming. + (find_max, find_min, pos_max, pos_min): New functions. + (obj_init): gc-protect and initialize gt_f and lt_f. + + + * lib.h (gt_f, lt_f): Declared. + (to_seq): Declaration updated to toseq. + (seqp, find_max, find_min, pos_max, pos_min): Declared. + + * txr.1: Updated. + +2014-06-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): where and sel registered as intrinsics + where and select. + + * lib.c (generic_funcall): Support a sequence as an argument + to a sequence. + (where, sel): New functions. + + * lib.h (where, sel): Declared. + + * txr.1: Documented. + +2014-06-15 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: range and range* broken when "to" argument is + omitted. This was broken in version 89, by the + 2014-04-08 commit. + + * eval.c (rangev_func, range_star_v_func): Only perform + numeric tests between from and to if to is not nil. + +2014-06-12 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Install share/txr/stdlib/*.txr material. + + * match.c (do_txeval): If a variable is not in the bindings, fall + back on treating it as a TXR Lisp dynamic variable. This allows + us to refer to the stdlib variable from a quasistring in a + @(load ...) directive. + + * txr.c (sysroot_init): Register new variable, *txr-version*. + + * share/txr/stdlib/ver.txr: New file. + +2014-06-12 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): use the abs_path_p function instead of + checking for leading slash. + + * stream.c (abs_path_p): New function. + (stream_init): Register abs_path_p as abs-path-p. + + * stream.h (abs_path_p): Declared. + + * txr.1: Documented abs-path-p. + + * dep.mk: Updated. + +2014-06-11 Kaz Kylheku <kaz@kylheku.com> + + Version 90 + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * txr.spec: Bumped version. + + * txr.vim, dep.mk: Regenerated. + + * RELNOTES: Updated + +2014-06-11 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (txr.o): Pass TXR_VER preprocessor symbol on gcc command + line. + (PREINSTALL): New variable holding a step for the install recipe. + (pax tar zip): New targets. + + * configure (txr_ver): Version now added to config.make, passed + through to txr.c. + (gen_config_make): bindir, datadir and mandir are established + using gmake's regular macro assignment (=) rather than + expanding assignment (:=). This allows us to override the + prefix variable after configure time. + + * lib.h (wli_noex): New macro. + (wli): Retarget to wli_noex, so that argument is subject + to macro replacement. + + * txr.c (version): Use TXR_VER defined on compiler command line, + rather than hard-coded string literal. + +2014-06-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Change registration of string_cmp + to cmp_str. Add registrations for str_eq, str_lt, + str_gt, str_le, and str_lt. + + * lib.c (string_cmp): Name changes to cmp_str, and the + function fixed so that it actually works. + The name change doesn't affect anyone because the function + was too broken to use due to the incorrect type dispatch. + (string_lt): Name changes to str_lt. + (str_eq, str_gt, str_le, str_ge): New functions. + + * lib.h (string_cmp, string_lt): Declarations renamed. + (str_eq, str_gt, str_le, str_ge): New declarations. + + * txr.1: Document string-cmp to cmp-str rename, that + string-lt is deprecated, and the new str<, str>, + str>=, str<= and str= functions. + +2014-06-10 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (PROG): Removing ./ prefix from variable name; adding it + to invocations of $(PROG) in some rules. + (txr.o): Pass several strings as macros on the command line: + TXR_REL_PATH, EXE_SUFF and PROG_NAME. + + * configure (bindir, datadir, mandir): These variables become + just relative paths from the prefix. + + * txr.c (sysroot): Use the TXR_REL_PATH, EXE_SUFF and PROG_NAME + preprocessor symbols defined on the command line to avoid + hard-coding strings like "bin/txr" and "bin/txr.exe" which + actually should reflect the value of the bindir variable. + +2014-06-10 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (get_self_path): Fix spelling of GetModuleFileName. + Add missing parentheses in expression. + (sysroot_init): On Windows, filter progpath to change + backslashes to forward slashes. + +2014-06-10 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (yylex_destroy): The FLEX_ version are prefixed with YY_ + in old flexes too. + +2014-06-09 Kaz Kylheku <kaz@kylheku.com> + + New variable stdlib, with a sysroot mechanism to compute + the path based on the "sysroot" where it is actually installed. + + * txr.c (progname_8u, progpath): New static variables. + (get_self_path, sysroot_helper, sysroot, sysroot_init): New + static functions. Sysroot creates a stdlib variable. + (main): Initialize progname_u8 value, and call sysroot_init. + +2014-06-09 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (match_str): Extended to suffix testing, with a negative + start argument. + + * txr.1: Documented. + +2014-06-09 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: fix broken tests; numerous test cases output + bindings, and need the -B option. One test case does not + need the -l option which now implies -B. + +2014-06-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): New option --args. + Also, put in missing check for -f being erroneously clumped. + + * txr.1: Documented. + +2014-06-09 Kaz Kylheku <kaz@kylheku.com> + + The dumping of bindings and printing of false must now + be explicitly requested by the -B option. + + * match.c (opt_nobindings): Variable removed. + (opt_print_bindings): New variable. + (extract): Print bindings or "false" if opt_print_bindings is true. + + * stream.c (output_produced): Variable removed. + (stdio_put_string, stdio_put_char, stdio_put_byte): Remove + update of output_produced. + + * stream.h (output_produced): Declaration removed. + + * txr.1: Documentation updated. + + * txr.c (txr_main): Option 'b' does nothing. 'B', 'l', 'a', + and '--lisp-bindings' set opt_print_bindings to 1. + + * txr.h (opt_nobindings): Declaration removed. + (opt_print_bindings): Declared. + + * unwind.c (uw_throw): When exiting due to a query error or + file error, print false when opt_print_bindings is true. + +2014-06-06 Kaz Kylheku <kaz@kylheku.com> + + Fixing issue with list-like iteration over generic sequences, + namely that empty strings and vectors are not nil. + The nullify function is introduced. It is also exposed to + users, as is the existing make_like function. + + * eval.c (mapcarv, mappendv, lazy_mapcar, lazy_mapcarv): + Use nullify to handle non-list inputs correctly. + (eval_init): Registering make_like and nullify as intrinsics. + + * lib.c (copy_list, to_seq, list_collect_nconc, list_collect_append, + reverse, lazy_appendv_func, lazy_appendv, ldiff, memq, memql, + memqual, remq, remql, remqual, remove_if, keep_if, rem_lazy_rec, + remq_lazy, remql_lazy, remqual_lazy, remove_if_lazy, keep_if_lazy, + countqual, countql, countq, count_if, some_satisfy, all_satisfy, + none_satisfy, do_chain, chainv, do_and, andv, do_or, orv, + cat_vec, assoc, assql, mapcar, mapcon, mappend, sort, multi_sort, + find, find_if, posqual, posql, posq, pos, pos_if, set_diff, + search): Use nullify for correctness. Some functions fixed + so return sequence matches type of input sequence. + (nullify): New function. + + * lib.h (nullify): Declared. + + * txr.1: Documented nullify and ake-like. + +2014-06-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register new search function as intrinsic. + + * lib.c (search_list): New static function. + (search): New function. + + * lib.h (search): New function declared. + + * txr.1: Documented. + + * txr.vim: Regenerated. + +2014-06-05 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Adding an explicit lexical rule to + catch backslashes occurring within a regex, not followed by a + character. This can happen in dynamically parsed regexes + such as "abc\\". + +2014-06-05 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (yylex_destroy): Only rely on the FLEX_ version + numbers if they are defined. In a newer flex, they are prefixed + by YY_. + +2014-06-05 Kaz Kylheku <kaz@kylheku.com> + + * configure (lang_flags): Removing -D_BSD_SOURCE + from lang_flags. Adding a test for determining + which flag reveals BSD functions. This is due to the + braindamaged way feature selection macros work on + FreeBSD. There is no way to say "give me only the + functions from a certain version of the Unix spec, + plus traditional BSD functions", so we have to resort to + using the internal symbol __BSD_VISIBLE. Also, changing + the detection test for daemon function to the + pointer-based approach. + +2014-06-04 Kaz Kylheku <kaz@kylheku.com> + + * configure (lang_flags): drop -D_POSIX_C_SOURCE=199309L. + This is subsumed by -D_XOPEN_SOURCE=500. + +2014-06-04 Kaz Kylheku <kaz@kylheku.com> + + * configure: Typo in diagnostic output: wspawnlp. + +2014-06-04 Kaz Kylheku <kaz@kylheku.com> + + Changes for FreeBSD 9. + + * configure: Use pointer-based test for timegm, because + -Werror=implicit-function-declaration does not work in the + gcc 4.2 used on FreeBSD 9. + + * parser.l: Check for Flex 2.5.9 and earlier which don't + have yylex_destroy. Thanks to Marcus Breiing. + +2014-06-04 Kaz Kylheku <kaz@kylheku.com> + + * configure: Changes for building on Solaris 10 using the + /usr/bin/xpg4/sh shell. Solaris' /bin/sh does not like test + -e. Adding break to the new shell existence testing loop so we + use the first shell in the list. Solaris wants _XOPEN_SOURCE=500 + to reveal certain functions. Fix breakage in the make utility + detection, and also use the $make variable everywhere instead + of make. + +2014-06-04 Kaz Kylheku <kaz@kylheku.com> + + * configure: Defend against legacy, non-POSIX /bin/sh implementations. + We boostrap using #!/bin/sh, but try to detect a better shell + and re-execute the script with that shell. Furthermore, we now + set up the SHELL variable in config.make, so that make uses that + shell for executing build recipes. + +2014-06-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_lisp1): Function removed. + (op_dwim, dwim_locs): Removing silly logic of separately calling + eval_lisp1 on the functor, and eval_lisp1_args on its arguments, since + lisp1 evaluation treats all position the same way. + +2014-06-03 Kaz Kylheku <kaz@kylheku.com> + + Documenting the return value of rplaca and rplacd, and changing + it to be the cell, rather than the new value. + + * eval.c (transform_op): Fix use of rplacd that uses return value. + (force): Likewise. + + * lib.c (rplaca, rplacd): Return cons. + (rem_lazy_rec, obj_init): Fix use of rplacd that uses return value. + + * txr.1: Documented. + +2014-05-20 Kaz Kylheku <kaz@kylheku.com> + + The call operator should be a function! + + * eval.c (call): New static function. + (eval_args, op_call): Static functions removed. + (eval_init): call_s registered as operator rather than function. + + * txr.1: Updated. + +2014-05-10 Kaz Kylheku <kaz@kylheku.com> + + Version 89 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * txr.spec: Bumped version. + + * RELNOTES: Updated + +2014-05-10 Kaz Kylheku <kaz@kylheku.com> + + String type related bugfixes: neglecting to handle all three kinds in + some places. In particular, the test case + + echo : | ./txr -c '@a:@a' - + + breaks because of neglected LIT in do_match_line. + + * arith.c (tofloat, toint): Handle LIT type in switch. + + * lib.c (ref, refset, replace, update): Handle LSTR type. + + * match.c (do_match_line, do_output_line): Handle LSTR and LIT + objects in switch. + +2014-04-14 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (get_line, get_char, get_byte): Fix outdated, incorrect + optional argument defaulting logic. + +2014-04-13 Kaz Kylheku <kaz@kylheku.com> + + * eval.c, gc.c, rand.c, regex.c, signal.c: Remove inclusion of unneeded + headers. + +2014-04-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (make_var_shadowing_env): Remove unnecessary test for + colon symbol. + +2014-04-08 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rangev_func, range_star_v_func): Use numeric + equivalence comparison for end test, rather than eql. + +2014-04-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (eql): Bugfix: not handling floating-point types! + Two objects which are equal floating-point values must be considered + eql even if they are distinct objects (not eq). + +2014-04-04 Kaz Kylheku <kaz@kylheku.com> + + Version 88 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. Generational GC is now + enabled by default. + + * txr.spec: Bumped version. + + * RELNOTES: Updated + + * txr.vim: Regenerated. + +2014-03-29 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (CHECKOBJ_VEC_SIZE, FRESHOBJ_VEC_SIZE): Increase + to 8 * HEAP_SIZE from 2 * HEAP_SIZE to reduce frequency + of garbage collections. + +2014-03-29 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Updating generational GC notes in light of changes. + +2014-03-29 Kaz Kylheku <kaz@kylheku.com> + + Change to how locations are passed around, for the sake of generational + GC. The issue being solved here is the accuracy of the gc_set function. + The existing impelmentation is too conservative. It has no generation + information about the memory location being stored, and so it assumes + the worst: that it is a location in the middle of a gen 1 object. + This is sub-optimal, creating unacceptable pressure against the + checkobj array and, worse, as a consequence causing unreachable gen 0 + objects to be tenured into gen 1. + + To solve this problem, we replace "val *" pointers with a structure + of type "loc" which keeps track of the object too, which lets us + discover the generation. + + I tried another approach: using just a pointer with a bitfield + indicating the generation. This turned out to have a serious issue: + such a bitfield goes stale when the object is moved to a different + generation. The object holding the memory location is in gen 1, but the + annotated pointer still indicates gen 0. The gc_set function then + makes the wrong decision, and premature reclamation takes place. + + * combi.c (perm_init_common, comb_gen_fun_common, + rcomb_gen_fun_common, rcomb_list_gen_fun): Update to new interfaces + for managing mutation. + + * debug.c (debug): Update to new interfaces for managing mutation. + Avoid loc variable name. + + * eval.c (env_fbind, env_fbind): Update to new interfaces + for managing mutation. + (lookup_var_l, dwim_loc): Return loc type and update to new interfaces. + (apply_frob_args, op_modplace, op_dohash, transform_op, mapcarv, + mappendv, repeat_infinite_func, repeat_times_func): Update to new + interfaces for managing mutation. + + * eval.h (lookup_var_l): Declaration updated. + + * filter.c (trie_add, trie_compress, trie_compress_intrinsic, + * build_filter, built_filter_from_list, filter_init): Update to new + * interfaces. + + * gc.c (gc_set): Rewritten to use loc type which provides the exact + generation. We do not need the in_malloc_range hack any more, since + we have the backpointer to the object. + (gc_push): Take loc rather than raw pointer. + + * gc.h (gc_set, gc_push): Declarations updated. + + * hash.c (struct hash): The acons* functions use loc instead + of val * now. + (hash_equal_op, copy_hash, gethash_c, inhash, gethash_n, pushhash, + remhash, set_hash_userdata, hash_next, group_by, hash_keys, + hash_values, hash_pairs, hash_uni, hash_isec, hash_update, + hash_update_1): Updated to new interfaces for managing mutation. + (hash_grow): Needs hash table argument to call the new form of set. + (hash_process_weak): We need to use valptr to recover the val * + from the loc that we receive from some functions now. + + * hash.h (gethash_c, gethash_f): Declarations updated. + (gethash_l): Inline function's prototype changes. + + * lib.c (rplaca, rplacd, car_l, cdr_l, listref_l, tail, lastcons, last, + ltail, list_collect, list_collect_nconc, list_collect_append, nreverse, + lazy_appendv_func, lazy_appendv): Update to new interfaces. + (malloc_low_bound, malloc_high_bound): Static variables removed. + (adjust_bounds): Static function removed. + (chk_malloc, chk_malloc_gc_more, chk_calloc, chk_realloc): + Calls to adjust_bounds removed. + (in_malloc_range): Function removed. + (get_plist_f, string_extend, length_str, replace_str, make_sym, + delete_package, intern, rehome_sym, get_user_package, + get_system_package, get_keyword_package, func_get_env, vec_set_length, + vecref, vecref_l, vec_push, simple_lazy_stream_func, lazy_stream_func, + lazy_str, lazy_str_force, lazy_str_force_upto, acons_new, acons_new_c, + aconsql_new, aconsql_new_c, alist_nremove, alist_nremove1, merge, + sort_list, refset, obj_init): Update to new interfaces. + + * lib.h (loc): New typedef. + (gc_set): Interface change. + (mkloc_fun): New inline function. + (mkloc, mkcloc, nulloc, nullocp, deref, valptr): New macros + (set): Interface change: takes loc as first argument, + rather than lvalue expression. + (mpush): Interface change: second argument is loc, rather + than lvalue. + (keyword_package, user_package, system_package, gensym_counter): Use + deref rather than dereference operator. + (in_malloc_range): Declaration removed. + (car_l, cdr_l, listref_l, tail, lastcons, ltail, getplist_f, + get_user_package, get_system_pckage, get_keyword_package, + vecref_l, acons_new_c, aconsql_new_c): Declarations updated. + (list_collect_decl): Updated to follow new interfaces. + + * match.c (dest_set, h_coll, v_gather, v_collect, v_cat, v_output, + v_filter): Updated to follow new interfaces. + + * parser.y (expand_meta, rlset): Updated to follow new interfaces. + + * rand.h (random_state): Use deref. + + * stream.c (string_in_get_line, string_in_get_char, + string_in_unget_char, strlist_out_put_string, strlist_out_put_char): + Updated to follow new interfaces. + + * stream.h (std_input, std_output, std_debug, std_error, + std_null): Use deref macro. + (lookup_var_l): Declaration updated. + + * syslog.c (syslog_put_string, syslog_put_char, syslog_put_byte, + syslog_set_prop): Updated to follow new interfaces. + + * unwind.c (uw_register_subtype): Updated to follow + new interfaces. + +2014-03-29 Kaz Kylheku <kaz@kylheku.com> + + Generational GC tweaks. + + * gc.c (make_obj): If we have room in the freshobj array, + but are out of objects, then call more. Without this, + we don't take proper advantage of this nursing area. + (gc): Set the full_gc flag after doing gc, in preparation + for next time. If we know full GC is coming, we can + adjust some behaviors. Increase the threshold for calling the more() + function to be the same as in the non-generational case: + less than 3/4 of the size of a heap scavenged. + (gc_set): Now does nothing if it is known that + a full gc is coming. Also, in the checkobj array overflow + case when we invoke gc, there is no point in adding obj + to the array, since it must have been promoted to the mature + generation. + (gc_mutated): Don't bother storing the object in the + array if a full GC is coming. + +2014-03-29 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (string_time): If possible, change the timezone in the + struct tm from "GMT" to "UTC", so that the time_string_utc + function will use UTC for the %Z format. + +2014-03-27 Kaz Kylheku <kaz@kylheku.com> + + Fix a bug arising from putting generation 1 objects into the + checkobj array (via the mut macro that expands to gc_mutated). + The garbage collector assumes that checkobj has only generation 0 + objects, which all exist in the freshobj array, which is subject + to a sweep. So gen 1 objects in checkobj are never cleaned up + properly: they do not have their REACHABLE flag reset, or + their generation restored to 1. To fix this, a new array for these + objects is introduced separate from checkobj. + + * gc.c (MUTOBJ_VEC_SIZE): New preprocessor symbol. + (mutobj, mutobj_idx): New static array and integer. + (mark_obj): Check for REACHABLE flag before checking the full_gc + flag and generation, since those cost additional memory accesses. + (mark): Mark the objects in the new mutobj array. + (sweep): Sweep the objects in the mutobj array. + (gc): Reset mutobx_idx to zero after gc. + (gc_set): Rearrange logic. In the case that the checkobj array + is full and a gc is done to make room, there is no point in + adding to the array: the gc pass moves all babies to generation 1, + so the object that was passed into the function is no longer a baby. + (gc_mutated): Rewrite in terms of mutobj rather than checkobj, + fixing the bug. + + * HACKING: Improved documentation of GC. Describe mut macro + and mutobj array. + +2014-03-27 Kaz Kylheku <kaz@kylheku.com> + + More generational GC fixes. One GC fix. + + * combi.c (perm_init_common, comb_gen_fun_common, + rcomb_gen_fun_common): Use set macro instead of plain assignment. + + * hash.c (hash_grow, copy_hash, hash_update_1): Use set macro + instead of plain assignment. + + * lib.c (nreverse, lazy_appendv_func, lazy_appendv, + vec_push, refset): Use set macro instead of plain assignment. + (make_package): Assign all fields of the newly created PKG + object before calling a function which can trigger GC. + + * parser.y (rlset): Use set macro. + +2014-03-27 Kaz Kylheku <kaz@kylheku.com> + + Fix generational GC regression caused by fixes + in in 2014-03-12, when prof was introduced. + The attempt to fix a bug made things worse. + + * lib.c (adjust_bounds): New static function. + (chk_malloc, chk_calloc, chk_realloc): Replace existing logic with + call to adjust_bounds. + (chk_malloc_gc_more): Bugfix: add missing call to adjust_bounds. + +2014-03-26 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Update for WLL's and QLL's. + + * txr.vim: Regenerate. + +2014-03-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (me_quasilist): New static function. + (eval_init): Register me_quasilist as quasilist macro expander. + + * lib.c (quasilist_s): New global variable. + (obj_init): quasilist_s initialized. + + * lib.h (quasilist_s): Declared. + + * match.c (do_txreval): Handle quasilist syntax. + + * parser.l (QWLIT): New exclusive state. + Extend lexical grammar to transition to QWLIT state upon + the #` or #*` sequence which kicks off a word literal, + and in that state, piecewise lexically analyze the QLL, + mostly by borrowing rules from quasiliterals. + + * parser.y (QWORDS, QWSPLICE): New tokens. + (n_exprs): Integrate splicing form of QLL syntax. + (n_expr): Integrate non-splicing form of QLL syntax. + (litchars): Propagate line number info. + (quasilit): Fix "string literal" wording in error message. + + * txr.1: Introduced WLL abbreviation for word list literals, + cleaned up the text a little, and documented QLL's. + +2014-03-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_quasi): Bugfix: incorrect logic, failing + to macro-expand the embedded forms in a quasiliteral except + when they are the very first item. + +2014-03-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (yybadtoken): Add missing cases for new token + types WORDS and WSPLICE. + +2014-03-25 Kaz Kylheku <kaz@kylheku.com> + + Introducing word list literals. + + * parser.l (WLIT): New exclusive start state. + Extend lexical grammar to transition to WLIT state upon + the #" or #*" sequence which kicks off a word literal, + and in that state, piecewise lexically analyze the literal, + mostly by borrowing rules from other literals. + + * parser.y (WORDS, WSPLICE): New tokens. + (n_exprs): Integrate splicing form of word list literal syntax. + (n_expr): Integrate non-splicit for of word list literal syntax. + (litchars): Propagate line number info. + (wordslit): New grammar rule. + + * txr.1: Updated. + +2014-03-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register last function as intrinsic. + + * lib.c (last): New function. + + * lib.h (last): Declared. + + * txr.1: Documented last. + +2014-03-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register empty as intrinsic. + + * lib.c (copy): Bugfix: handle lazy strings. Also, handle hash + tables via copy_hash. + (length): Bugifx: handle lazy strings. Also, handle hash tables + via hash_count. + (empty): New function. + + * lib.h (empty): Declared. + + * txr.1: Documented. + +2014-03-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (not_s): New symbol var. + (me_unless, me_while, m_until): New static functions. + (eval_init): Register macros unless, while and until. + + * txr.1: Document unless, while and until. + +2014-03-22 Kaz Kylheku <kaz@kylheku.com> + + Version 87 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated + + * txr.vim: Regenerated. + +2014-03-22 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (w_fopen, w_popen, w_freopen, w_remove, w_rename): Revert + 2014-03-14 changes. + +2014-03-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (me_when): New static function. + (eval_init): Register "when" macro. + + * txr.1: Documented when macro. + +2014-03-19 Kaz Kylheku <kaz@kylheku.com> + + Second round of bugfixing on the theme of the previous commit. + Here were are changing the output clause to recognize all special + clause tokens that are not used in output and turning them into + regular Lisp. So @(if a b c) in an output clause works once again, + recognized as IF exprs_opt ')' syntax, and turned into Lisp. Other + things work that didn't work before like @(and), @(or) and so forth. + + * parser.y (make_expr): New static function. + (not_a_clause): New nonterminal. + (out_clause): Remove error-catching productions for match-side + clauses. + (o_elems): Now consists of a mixture of o_elems and not_a_clause's. + +2014-03-19 Kaz Kylheku <kaz@kylheku.com> + + Fixing gaping bug in the handling of @-delimited expressions + within quasiliterals. This has been a problem for years. + Quasiliteral strings existed very early before TXR Lisp was introduced. + So it made sense that when @ is seen in a quasiliteral, the + lexical analyzer pushed into the SPECIAL state in which directives + are recognized, like in the pattern language. I noticed + this because there is an @(if) directive now, which prevents + `@(if ...)` from being valid. + + * parser.l (QSPECIAL): New scanner state. This is a state + similar to SPECIAL that we enter into when @ is seen in a QSLIT state. + In this state we recognize constructs like braced variables, but not + certain other features like directives. + +2014-03-19 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (int_flo): Fix non-handling of negative values. + +2014-03-19 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (tofloat, toint): New functions. + + * arith.h (tofloat, toint): Declared. + + * eval.c (eval_init): tofloat and toint registered + as intrinsics. + + * txr.1: Documented. + +2014-03-16 Kaz Kylheku <kaz@kylheku.com> + + Version 86 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated + + * dep.mk: Regenerated. + +2014-03-16 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (win_make_cmdline): I think I cracked the code of + passing escaped arguments to Microsoft's popen and system + functions. The first argument cannot be quoted, but the + subsequent ones can. It is the quoting of the first argument + that causes the whole thing to be quoted. So we just + avoid quoting the first argument. (If it has spaces, too bad; + don't do that). + (open_process): Fix neglected optional argument handling + in Windows version of this function. + +2014-03-16 Kaz Kylheku <kaz@kylheku.com> + + * configure: Detect _wspawn* functions. + + * stream.c (run): Implement using _wspawnvp on MinGW. + +2014-03-15 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Bugfix. When handling a backslash-newline continuation + in the SPECIAL, NESTED and BRACED states, do not pop the state in all + three, only in SPECIAL (to terminate the @\ continuation). + + * txr.1: Eliminate wrong claim that string literals do not split across lines, + which is directly contradicted two paragraphs later. Document that + quasiliterals also split. + + * genvim.txr (txr_regex, txl_regex): These definitions change from "syn match" + to "syn region" so that the backslash-newline continuation can be properly + handled. + (txr_string, txr_quasilit): Correctly handle split literals. + + * txr.vim: Regenerated. + +2014-03-15 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (text): Allow the EMPTY token (the @(empty) syntax) + to be a text. + + * txr.1: Documented @(empty). + +2014-03-14 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (regex_parse, lisp_parse): Fix neglected handling of + optional arguments. This problem can cause the symbol : to be + planted as the std_error stream, resulting in an error loop + that blows the stack. + + * regex.c (regex_compile): Likewise. + +2014-03-14 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (w_fopen, w_popen, w_freopen, w_remove, w_rename): We + have similar functions in the MSVCRT library used by MinGW, + so if _WIN32 is defined, our functions now just wrap those. + +2014-03-14 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (run): Fix MinGW build failures. The run function doesn't + work right with arguments, though: the win command line is broken. + +2014-03-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Make seed argument optional in make-random-state. + + * rand.c (make_random_state): Do argument defaulting on seed. + Also, mix getpid() into the seed. + (random_fixnum): Bugfix: do proper defaulting on optional + agument, rather than relying on nil. + (random): Fix 2014-02-05 regression. This was totally broken, + ignoring the random state passed in and using the global + random state. This function must only use the state passed in; + there is no defaulting to the global random state. + + * txr.1: Documenting that seed is optional in make-random-state. + Describing what guarantees can be expected with regard to + calls made close together temporally. + +2014-03-13 Kaz Kylheku <kaz@kylheku.com> + + Implementing @(if)/@(elif)/@(else) in the pattern language. + Input side for now; output later. + + * parser.y (if_clause, elif_clauses_opt, else_clause_opt): New nonterminals. + (IF, ELIF, ELSE): New tokens. + (yybadtoken): Handle IF, ELIF, ELSE. + + * parser.l: Recognize and return new tokens IF, ELIF and ELSE. + + * txr.1: Documented. + + * genvim.txr: Updated with if, elsif and else directive keywords. + + * txr.vim: Regenerated + +2014-03-13 Kaz Kylheku <kaz@kylheku.com> + + On platforms with sigaltstack, TXR programs can now catch the + segmentation fault that occurs when running out of stack space, + and escape by throwing an exception. + + Also, bugfix: save and restore the gc enable/disable state. Without + this, if we are ever running cod in a gc disabled state and it jumps + out, gc stays disabled. + + * configure: added check for sigaltstack. + + * gc.h (gc_enabled): Declaration added for existing variable. + + * signal.c (is_cpu_exception): New static function. + (sig_handler): For cpu_exception type signals that pertain + to the execution of some instruction, turn on async_sig_enabled, + so that the signal is not deferred. Otherwise we will just + return without calling the user-defined handler, restart + the instruction and get into a loop. Also, disable gc around + the handler just in case. One issue is that we might be on + an alternate stack, which gc won't like. + (setup_alt_stack, teardown_alt_stack): New static functions. + (set_sig_handler): If we have sigaltstack, and are asked + to set up a SEGV handler, then set it up on the alternate + stack. + + * signal.h (extended_jmp_buf): Adding new member, gc. + (extended_setjmp, extended_longjmp): use gc member + to save and restore the gc_enable state across + setjmp and longjmp. + +2014-03-13 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_process): If execvp fails, use errno as the + exit status. + (sh, run): New static functions. + (stream_init): sh and run registered as intrinsics. + +2014-03-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (iffi): Bugfix: was still using incorrect, outdated optional + argument defaulting logic. + +2014-03-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (plus_s, prof_s): New symbol global variables. + (op_prof, me_pprof): New static functions. + (eval_init): Intern prof symbol, store in prof_s. + Captured interned + symbol in plus_s. Register prof operator and pprof + macro. + + * gc.c (gc_bytes): New global variable. + (more): Use nse function chk_malloc_gc_more instead of chk_malloc. + (make_obj): Increment gc_bytes. + + * lib.c (malloc_bytes): New global variable. + (chk_malloc, chk_realloc): Increment malloc_bytes. + (chk_calloc): Bugfix: incorrect size in recursion into oom_realloc. + Incorrect calculation of malloc_high_bound. Increment malloc_bytes. + (chk_malloc_gc_more): New function. + + * lib.h (alloc_bytes_t): New typedef. + (malloc_bytes, gc_bytes): Declared. + (chk_malloc_gc_more): Declared. + + * stream.c (format_s): New symbol global. + (stream_init): format_s inited. + format_s used to register formatv function. + + * stream.h (format_s): Declared. + + * txr.1: Documented prof and pprof. + + * genvim.txr: Recognize reg_fun calls with intern + followed by a preceding assignment or other syntax. + + * txr.vim: Updated. + +2014-03-11 Kaz Kylheku <kaz@kylheku.com> + + * configure: new test for fcntl. + + * stream.c (open_process): Fixed off-by one erroneous value of nargs, + causing memory leak of one string. Fixed memory leak on fork failure. + Fixed a deadlock that can occur in the pipe close function when + multiple pipes are in existence. This is fixed by setting the + FD_CLOEXEC flag on the pipe file descriptor. Without this, one child + process can hold another's pipe open, causing that other one not to + terminate when we're trying to shut it down, resulting in that child + blocked on a write, while we block on waitpid. + +2014-03-11 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_process): In the event of fdopen failure, + kill the child process less abruptly by hitting it with SIGINT and + SIGTERM, rather than with SIGKILL. Also, collect the child + process with waitpid. + +2014-03-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registration of url_encode and url_decode + moved to filter.c. + + * filter.c (trie_compress_intrinsic, html_encode, html_decode): New + static functions. + (filter_init): Register make_trie, trie_add, trie_compress_intrinsic, + filter_string_tree, filter_equal, html_encode and html_decode + as intrinsics. Move registration of url_encode and url_decode here. + + * genvim.txr: Look for registrations in filter.c too. + + * txr.1: Documented. + + * txr.vim: Updated. + +2014-03-11 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_process): Close the original pipe file descriptor + in the child process after dup2-ing it to standard input or standard + output, so the child doesn't have to references to the pipe. + +2014-03-10 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (pipe_close): Restructured the function a bit. + Do not throw exceptions for normal process terminations, only for + abnormal ones (terminated by a signal). Return the termination + status instead of just t. + + * txr.1: Fixed the neglected documentation of the optional boolean + argument of close-stream. Described the behaviors with regard + to pipes in more detail. + +2014-03-09 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_trailer): Fix segfault. The code which + helps implement the special interaction between @(accept) and + @(trailer) was not handling the situation when there is not + current unwind exit point. + +2014-03-09 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Syntactic tweak. Do not allow the "extra" + constituent chars # or ^ to start the symbol name after an @. + Prior to this change @^a is a "meta-symbol" whose name is "^a", + but ^a is the ^ quasiquote notation followed by symbol a. + After this change @^a is meta, applied to quasiquote-a. + +2014-03-09 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (lazy_sub_str): Bugfix: "from" was mistakenly used + in the adjustment of the "to" value. + + * match.c (search_form): Use predefined constants for -1 and 1 + instead of calling num. + +2014-03-09 Kaz Kylheku <kaz@kylheku.com> + + Fixing broken processing of horizontal matching across + long lines produced by @(freeform). Once the matching + passes about 4000 characters, the "consume_prefix" + function kicks in to save memory. Then any code which is + not properly written to handle this displaced situation + will break. + + * match.c (h_text, h_var, h_coll, h_parallel, h_fun): Bugfix. + The recursive calls to match_line return an absolute position. + From this value we must subtract c->base if we are to + compare it with c->pos, or update c->pos. If we use the + absolute value, we are abruptly jumping ahead in the data. + +2014-03-08 Kaz Kylheku <kaz@kylheku.com> + + Issue: match_regex and search_regex were continuing to feed characters + to the regex machine even when there is no transition available. This + was due to the broken return value protocol of regex_machine_feed. For + instance for the regex / +/ (one or more spaces), after matching some + spaces, it would report REGM_INCOMPLETE for additional non-space + characters, never reporting REGM_FAIL. + + * regex.c (regm_result_t): Block comment added, documenting protocol. + (regex_machine_feed): Return REGM_FAIL if there are no transitions for + the given character, even a partial match has been recorded. This is + a signal to stop feeding more characters. At that point, the function + can be called with a null character to distinguish the three cases: + fail, partial or full match. + (search_regex): Now when the search loop gets a REGM_FAIL, it can + no longer assume that nothing was matched and the search must restart + at the next position. Upon the REGM_FAIL signal, it is necesary to + seal the search by feeding in the 0 character. Only if that returns + REGM_FAIL is it a no match situation. Otherwise it is actually a match! + +2014-03-08 Kaz Kylheku <kaz@kylheku.com> + + Version 85 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated + + * txr.vim: Updated. + +2014-03-08 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Added missing documentation for @(forget)/@(local). + +2014-03-08 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (rand_init): Oops! Spectacular silliness here + broke the symbol module: two intern calls in consecutive + lines, one for the name without "earmuffs", one with, + leading to the situation that the C code looks for a + dynamic variable that does not exist. + +2014-03-07 Kaz Kylheku <kaz@kylheku.com> + + * match.c (LOG_MATCH, LOG_MISMATCH): Wouldn't you know it; + the format strings in these macros contained a workaround for + the broken * variable field width syntax, specifying + ~*~a where the extra ~ in the middle just feeds a character + that the broken state machine expects. These workarounds broke when I + fixed the formatting, making -v mode useless. + +2014-03-07 Kaz Kylheku <kaz@kylheku.com> + + Version 84 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated + + * txr.vim: Updated. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (upop): New function. + + * lib.h (upop): Declared. + + * txr.c (txr_main): Two bugfixes. One is that the argument - + was being pushed back twice resulting in *args* being ("-" "-"). + This is because the option processing loop checked for "-" and pushed + it back into args, and then some logic after the loop pushed arg back + into args again. But, these pushes were wrong because they push + back a different cons cell; we would like to be able to + do (ldiff *full-args* *args*). This is solved by upop, which provides + one element of undo. After upop, we can restore the prior list + from the undo save location. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (assert_s): New global variable. + (obj_init): Intern assert symbol, store in assert_s. + + * lib.h (assert_s): Declared. + + * match.c (typed_error, v_assert, h_assert): New static functions. + (dir_tables_init): Register v_assert and h_assert. + Register assert_s as non-data-matching directive. + + * unwind.c (uw_init): Register assert as a subtype + of error. + + * txr.1: Describe assert. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + * match.c: (v_next): Set the "curfile" in the context to "env" when + scanning environment. + (open_data_source): Regression: was not setting c->curfile when opening + anything. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apply_intrinsic, lazy_mapcar): Changed linkage to external. + + * eval.h (apply_intrinsic, lazy_mapcar): Declarations added. + + * stream.c (open_files, open_file_star): New functions. + (stream_init): Registered new functions as intrinsics. + + * txr.1: Documented open-files and open-files*. Added to make-catenated-stream documentation. + + * genvim.txr: Replace bunch of code with open-files. + + * txr.vim: Regenerated. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): Fix it again. The data (nil) + can occur from an interactive/real-time stream. + +2014-03-06 Kaz Kylheku <kaz@kylheku.com> + + g++ regressions. + + * eval.c (env_k): Duplicate global variable definition removed. + + * lib.c (vector, vec_set_length): Fixed signed/unsigned comparison + warnings. + + * stream.h (stdin_s, stdout_s, stddebug_s, stderr_s, stdnull_s): + Declarations were definitions due to missing extern. + +2014-03-05 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Don't push back arg into arg_list if arg is nil. + +2014-03-05 Kaz Kylheku <kaz@kylheku.com> + + Fixing regression caused by the 2014-02-19 change ("Fixed long-running + issue ..."). + + * match.c (open_data_source): if c->data is t, but c->files + is nil, set c->data to nil: we cannot possibly open anything later. + (match_files): We need to call open_data_source one more time just + before processing a line with horizontal material. The previous + call(s) to open_data_source might not have opened anything. Before + accesing car(c.data) the correct test is consp(c.data), not c.data. In + the else clause, we now specificially check for nilp(c.data) which is + the correct indicator of no more data. If c.data is any other atom at + that point, we have an internal error, for which an assertion is added + now. + +2014-03-05 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Fix broken parsing of parameteric width and + precision (i.e. given by *). The simplest way to do this is to add a + state vf_star which is similar to vf_digits, and reuses much of the + logic via a goto. Fixing forgotten --fmt (push back char) when moving + from vf_precision to vf_spec state due to unhandled character. + Also, allowing a leading zero before a precision given as * + so that computed precisions can be combined with zero padding. + + * txr.1: Documented leading zero better so it is clear it + combines with *. Restructured documentation. + +2014-03-05 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (put_line): Default string argument to empty string. + (stream_init): Change how put_line is registered to take advantage of + new optional parameter. + + * txr.1: Updated. + +2014-03-04 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (logten): New function. + + * eval.c (eval_init): logten registered as intrinsic. + + * lib.h (logten): Declared. + + * txr.1: Documented. + +2014-03-03 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Allowing ^ to be a quote character, and adjusting definition + of identifiers to rule this out from being the first character of a + symbol which has no prefix. Recognize the ^ character as a token in the + NESTED state. + + * lib.c (obj_print, obj_pprint): Render sys:qquote as ^. + + * parser.y (choose_quote): Function removed. + (n_expr): Recognize '^' as quasiquote. Removed all the "smart quote" + hacks that try to make quote behave as quote or quasiquote, or try to + cancel out unquotes and quotes. + + * tests/009/json.txr: Fixed to ^ quasiquote. + + * tests/010/reghash.txr: Likewise. + + * tests/011/macros-2.txr: Likewise. + + * tests/011/mandel.txr: Likewise. + + * tests/011/special-1.txr: Likewise. + + * txr.1: Updated docs. + + * genvim.txr: Revamped definitions for txr_ident and txl_ident so that + unqualified identifiers cannot start with # or ^, but ones with @ or : + in front can start with these characters. + + * txr.vim: Regenerated. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (lex.yy.c): Remove lex.yy.c before trying to regenerate it. + (y.tab.c, y.tab.h): Remove y.tab.c before running yacc. Write-protect + y.tab.c, not $@, which could expand to the y.tab.h target if that is + what triggered the rule. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Incorporating more corrections from Roman Mishin. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (make_env_intrinsic): New static function. + (eval_init): Register new intrinsics make-env, env-fbind and env-vbind. + + * txr.1: Documented. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + Version 83 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated + + * dep.mk: Updated. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand): Bugfix. When processing a defvar, should mark + the variable as special at expansion time. Without this, binding + forms in other parts of the tree won't be expanded properly + to bring about the special semantics. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (unquotes_occur): Fix use of unquote_s rather than sys_unquote_s + which breaks backquote, caught by tests/010/reghash.txr. + I thought fixed this already! But I must have made the change to y.tab.c + rather than parser.y. + + * Makefile (lex.yy.c, y.tab.c): Make these files readonly to prevent + unintended edits. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Skip lines that contain system_package. + + * txr.vim: Regenerated. + +2014-03-02 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: catenated stream objects not hooked into garbage collector, + causing premature collection of the catenated stream list. + + * stream.c (cat_mark): new function. + (cat_stream_ops): use cat_mark for gc mark function. + +2014-03-01 Kaz Kylheku <kaz@kylheku.com> + + New quasiquote idea: let's have two quasiquote macros sharing one + expander. One macro based on sys:qquote, sys:unquote and sys:splice, + and the other based on qquote, unquote and splice in the user package. + The read syntax puts out the sys: one. + + * eval.c (expand_qquote): Takes three additional arguments: the + qquote, unquote and splice symbols to recognize. + The invalid splice diagnostic is adjusted based on which backquote + we are expanding. + (me_qquote): Look at the symbol in the first position of the form + and then expand either the internal quasiquote macro or the public one, + passing the right symbols into expand_qquote. + (eval_init): Register error-throwing stub functions + for the sys_qquote_s, sys_unquote_s and sys_splice_s symbols. + Register a macro for sys_qquote_s. + + * lib.c (sys_qquote_s, sys_unquote_s, sys_splice_s): New symbol + variables. + (obj_init): Initialize new variables. Change qquote_s, + unquote_s and splice_s to user package. + (obj_print, obj_pprint): Convert only sys_qquote_s, + sys_unquote_s and sys_splice_s to the read syntax. + The quote_s, unquote_s and splice_s symbols are not + treated specially. + + * lib.h (sys_qquote_s, sys_unquote_s, sys_splice_s): Declared. + + * parser.y (n_expr): Use sys_qquote_s, sys_unquote_s and + sys_splice_s rather than qquote_s, unquote_s and splice_s. + (unquotes_occur): Likewise. + + * txr.1: Documented. + +2014-03-01 Kaz Kylheku <kaz@kylheku.com> + + * configure: changed numerous "if ! conftest" tests to "if conftest", + swapping around the consequent and alternative clauses. + +2014-03-01 Kaz Kylheku <kaz@kylheku.com> + + * configure (conftest, conftest_o): New functions + Use functions throughout tests instead of repeated code for removing + target, invoking make with redirection and testing for existence of + target. + +2014-03-01 Kaz Kylheku <kaz@kylheku.com> + + * tests/011/special-1.txr: Add some coverage for evaluation + of a re-bound special under the Lisp-1 evaluation of the [ ] + notation. This test case would have failed three commits + back. + +2014-03-01 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_qquote): Another bugfix: not recognizing a trailing + atom that comes out of recursive call, wrapped in (quote ...), + resulting in '(,a . b) expanding to (append 'a . quote b) + rather than (append (list 'a) (quote b)); i.e. (append (list 'a) 'b). + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (lookup_sym_lisp1): Bugfix: wasn't following the dynamic + environment at all, and still had vestiges of support for the the old + cptr based global variables. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: Use target-specific assignment + to suppress TXR_DBG_OPTS for tests under tests/011. + The --gen-gc makes mandel.txr run too slow. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (self_evaluating_p, maybe_quote): New functions. + (expand): Use maybe-quote form macro-time, to not quote + result unnecessarily. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * tests/011/mandel.expected: New file. + + * tests/011/mandel.txr: New file. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + Turn *gensym-counter* into proper special variable. + + * eval.c (eval_init): Save *gensym-counter* symbol in gensym_counter_s + symbol variable right after interning, and use zero as the inital value + rather than the gensym_counter variable which is removed now. + + * lib.c (gensym_counter_s): New symbol variable. + (gensym_counter): Variable removed. + (gensym): Slight refactoring to avoid a double variable lookup. + Also, for generational GC correctness, use the set macro to update it, + since the variable could live inside heap object and the counter + could overflow to bignums which are heap objects. + (obj_init): Remove initialization of gensym_counter. + + * lib.h (gensym_counter_s): Declared. + (gensym_counter): Declaration removed, replaced by macro. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + Change in the design of how special variables work, to fix the broken + re-binding. C code now has to go through the dynamic environment lookup + to access things like *random-state*, or *stdout*. As part of this, + I'm moving some intrinsic variable and function initializations out of + eval.c and into their respective modules. Macros are are used to make + global variables look like ordinary C variables. This is very similar + to the errno trick in POSIX threads implementations. + + * eval.c (looup_var, lookup_var_l): Restructured to eliminate silly + goto, the cobjp handling is gone. + (reg_fun, reg_var): Internal function becomes external. + reg_var registers a simple cons cell binding now, without any + C pointer tricks to real C global variables. + (c_var_mark): Static function removed. + (c_var_ops): Static struct removed. + (eval_init): Numerous initializations for streams, syslog, rand, + signals and others moved to their respective modules. + The new symbol variables user_package_s, keyword_package_s + and system_package_s are interned here, and the variables are + created in a special way. + + * eval.h (reg_var, reg_fun): Declared. + + * gc.c (prot1): Added assert that the loc pointer isn't null. + This happened, and blew up during garbage collection. + + * lib.c (system_package, keyword_package, user_package): Variables + removed these become macros. + (system_package_var, keyword_package_var, user_package_var): New + global variables. + (system_package_s, keyword_package_s, user_package_s): New + symbol globals. + (get_user_package, get_system_package, get_keyword_package): New + functions. + (obj_init): Protect new variables. Initialization order of modules + tweaked: the modules sig_init, stream_init, and rand_init are moved + after eval_init because they register variables. + + * lib.h (keyword_package, system_pckage, user_package): Variables + turned into macros. + (system_package_var, keyword_package_var, user_package_var): Declared. + (system_package_s, keyword_package_s, user_package_s): Declared. + (get_user_package, get_system_package, get_keyword_package): Declared. + + * rand.c (struct random_state): Renamed to struct rand_state to + avoid clash with new random_state macro. + (random_state): Global variable removed. + (random_state_s): New symbol global. + (make_state, rand32, make_random_state, random_fixnum, random): + Follow rename of struct random_state. + (rand_init): Reference to random_state variable gone. Using + reg_var to create the *random-state* variable that is referenced + from C using the random_stat macro. + + * rand.h (random_state): Variable removed, replaced by macro + that performs dynamic lookup. + + * signal.c (sig_init): References to all the sig_* global variables + removed. The signal-related reg_var and reg_fun calls from eval.c moved + here. + + * stream.c (std_input, std_output, std_debug, std_error, std_null): + Variables removed. + (s_ifmt, s_ifsock, s_iflnk, s_ifreg, s_ifblk, s_ifdir, + s_ifchr, s_ififo, s_isuid, s_isgid, s_isvtx, s_irwxu, + s_irusr, s_iwusr, s_ixusr, s_irwxg, s_irgrp, s_iwgrp, + s_ixgrp, s_irwxo, s_iroth, s_iwoth, s_ixoth): Variables removed. + (stdin_s, stdout_s, stddebug_s, stderr_s, stdnull_s): New symbol + globals. + (stream_init): References to removed variables gone. Moved + stream-related initializations here from eval.c. The global + streams are set up differently. + + * stream.h (std_input, std_output, std_debug, std_error, std_null): + Variable declarations replaced by macros. + (lookup_var_l): Declared. + (s_ifmt, s_ifsock, s_iflnk, s_ifreg, s_ifblk, s_ifdir, + s_ifchr, s_ififo, s_isuid, s_isgid, s_isvtx, s_irwxu, + s_irusr, s_iwusr, s_ixusr, s_irwxg, s_irgrp, s_iwgrp, + s_ixgrp, s_irwxo, s_iroth, s_iwoth, s_ixoth): Declarations removed. + + * syslog.c (log_pid_v, log_cons_v, log_ndelay_v, log_odelay_v, + log_nowait_v, log_perror_v, log_user_v, log_daemon_v, log_auth_v, + log_authpriv_v, log_emerg_v, log_alert_v, log_crit_v, log_err_v, + log_warning_v, log_notice_v, log_info_v, + log_debug_v, std_log): Variables removed. + (syslog_init): References to removed variables removed. + Moved syslog-related initializations here out of eval_init. + + * syslog.h:x (std_log): Declration removed. + + * txr.c (self_path, prog_args_full, prog_args): Variables gone. + (txr_main): References to removed varaibles are gone. + Moved registration of special variables out of eval_init + here. + + * txr.h (self_path, prog_args_full, prog_args): Declarations gone. + + * tests/011/special-1.txr: Test case modified to properly test + special variables. Previously it produced the expected output + even though *stdout* wasn't rebound properly. + + * tests/011/special-1.expected: Updated. + + * genvim.txr: Updated to follow variable and function registration + moves. It has to scan more files than just eval.c. Produces identical + contents, so no change to txr.vim. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_defvar): Remove the same-named symbol macro when a + variable is defined. + (op_defsymacro): Remove the same-named variable when a symbol macro is + defined. + (op_defun): Throw excpetion if an attempt is made to define a + special operator as a function. Remove the same-named macro when a + function is defined. + (op_defmacro): Throw excpetion if an attempt is made to define a + special operator as a macro. REmove the same-named function + when a macro is defined. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_qquote): Fix broken '(,x . ,y) case, which + is generating (append (list x) . y) instead of (append (list x) y). + Also, added a nil case which is now necessary to prevent + '(,x) from generating (append (list x) nil), though this is + effectively an optimization, which is pointless, since + the expander isn't optimizing overall. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + * tests/010/output-clauses.expected: New file. + + * tests/010/output-clauses.txr: New file. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + About time for some new regression tests. + + * tests/011/macros-1.expected: New file. + + * tests/011/macros-1.txr: New file. + + * tests/011/macros-2.expected: New file. + + * tests/011/macros-2.txr: New file. + + * tests/011/special-1.expected: New file. + + * tests/011/special-1.txr: New file. + +2014-02-28 Kaz Kylheku <kaz@kylheku.com> + + Bugfixing semantics of symbol macro hiding by variable bindings. + + * eval.c (expand_vars): Expand vars needs to know whether it's + handling a sequential (let*-like) or parallel (let-like) binding. + The init form of a variable sees the previous macro environment, + but that variable must, for the remaining variables, shadow any + previous symbol macro. + (expand): In expand we must inform expand_vars whether we have + a sequential or parallel binding construct. Moreover, there was + a huge bug here: the new_menv (with the shadows) was passed to + expand_vars, so that all the variables were hiding same-named + symbol macros from all their initforms. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (delay_s): Global variable removed. + (eval_init): Initialiation of delay_s removed. + Symbol now interned in reg_mac call for registering me_delay . + + * genvim.txr: Updated to recognize a reg_mac call + with intern. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Improved documentation for the dwim operator. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + Version 82 + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date, improved the documentation + for the copy function and copy-list, and fixed the + an instance where the delay macro was called a function. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim: Updated. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Typos, grammar and formatting fixes mailed in + by Roman Mishin. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Regression: valid floating-point literals tagged + as having trailing junk. The fix is to put the junk-matching rule after + the valid rules. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (copy): New function. + + * lib.h (copy): Declared. + + * eval.c (eval_init): Registered copy function as intrinsic. + + * txr.1: Added missing documentation for length. Documented copy. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Updated with regard to how operators are registered in + in eval_init. Also, scans reg_mac registrations now. + + * txr.vim: Refreshed. The previously missed "delay" operator is now + listed, thanks to reg_mac. + +2014-02-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (maybe_progn): New function. + (expand_macrolet, expand_symacrolet): Use maybe_progn + to eliminate unnecessary progn wrapping. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (gun_s): New global variable. + (me_gun): New static function. + (eval_init): New gun symbol interened, me_gun registered + as intrinsic macro. + + * txr.1: Documented gun. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + Converting expander special case code transformations into + formal macros that are in the top_mb table, make their symbols + fboundp and can be expanded with macroexpand. + + * eval.c (mefun_t): New typedef name. + (expand_macro): If the expander is a cobj, then pull out the C function + and call it, otherwise realize the interpreted macro as before. + (me_gen, me_delay): New static functions, replace expand_gen + and expand_delay. + (expand_qquote): Renamed to me_quote. + (expand_gen, expand_delay): Renamed to me_gen and me_delay, + with an interface adjustment and moved. + (expand_op): Renamed to me_op. + (expand): Removed qquote, gen, delay, op, and do handling, since + these operators are now macros. + Removed the unnecessary expansion of with-saved-vars. + (reg_op, reg_fun): Assert that the symbol is not nil, to catch + initialization order issues. One just showed up: op_do was + interned in match.c, which is initialized later. + (reg_mac): New static function + (eval_init): Intern do_s, because match.c hasn't done it yet + at this point. Register me_gen, me_delay, me_op (twice) and me_qquote + as intrinsic macros. + + * txr.1: Documented those operators as macros. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (reg_op): New static function. + (eval_init): Register operators with reg_op instead of direct sethash + calls. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (make_like): If the first element of list isn't + a character, then don't try to make it into a string; + keep it as a list. This allows [mapcar list "abc" "def"] + to work intuitively rather than blow up. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (lit_s): New symbol. + (code2type): Return lit_s for LIT type. + (typeof): Return lit_s for TAG_LIT. + (replace_str): Error message wording. + (obj_init): lit_s interned. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (do_eval): Set last_form_evaled just before + dispatching function, so it is not clobbered by argument evaluations. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_op): Use rlcp_tree when constructing the + dwim_body, since the append2 copies list structure. + + * parser.y (n_exprs): propagate source loc info from both + constituents, giving precedence to the left one, rather than just taking + it from the left one and ignoring the second constituent. This fixes + cases of missing location info. The left constituent n_expr is often + a symbol, and those do not have location info. We want a case like + like ((x) y) to take from (x), and (x (y)) to take it from (y), + and so on. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (list): For @n_expr, get the source location info + from the @ token. + (n_expr): Bugfix: do not attribute a symbol with location info. + (quasilit): Obtain location info from quasi_items, and only + if that is unavailable, then from lineno. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + Version 81 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim: Updated. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (regex_parse, lisp_parse): Better formatting + for the context string that serves in place of a filename. + +2014-02-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_place): Unnecessary, buggy function removed. + It was handling silly special cases with the net effect of being + nearly equivalent to just expand, while failing to expand places + which are compound macro calls. It did diagnose nonexistent place + types at expansion time, which isn't worth much. + (expand): All the expansions for the mutating operator sset, inc, dec, + push, pop and flip were also pointless and have been removed. they did + the precise equivalent of doing expand(rest(form)), and consing the + symbol to the front, which is the fallback case for function call + forms. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * configure: Make it an error to have implicit function declarations by + means of -Werror=implicit-function-declarations. Fix broken mkdir and + mknod tests. Regenerate config.make if we add -U__STRICT_ANSI__ + to the compiler options, so that later tests benefit from this. + Regenerate config.make right away if adding -D_DARWIN_C_SOURCE. + This allows the makedev test to pass. + + * stream.c: Include <sys/types.h> for makedev. + (mknod_wrap): Don't rely on makedev wrappers to be there; test. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (spec): Bugfix: SECRET_ESCAPE_E should use n_expr + not expr. expr is subject to expand_meta. + (n_expr): Do not associate source info with character + literals. + (expand_meta): Bugfix: when walking forms in a collecting loop, + propagate the source info to them. Bugfix: attach source + info to var and expr expansions. + (rl): Rewritten in terms of rlset. + (rlset): Only set source info for an object that doesn't already + have it. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_init): Revert qquote, unquote and splice + to the system namespace, for hygiene. + + * txr.1: Updated. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): HAVE_MAKEDEV not HAVE_MKDEV. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (getcwd_wrap): Implement correct test of getcwd's + return value. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * configure: Use -Werror= on the missing prototypes and + strict prototypes warnings; these should be errors. + +2014-02-25 Kaz Kylheku <kaz@kylheku.com> + + * configure: Added feature tests for makedev, link/symlink/readlink, + mkdir and mknod. + + * eval.c (eval_init): Wrap #ifdefs around the registrations of + the wrappers for these functions. + + * stream.c (mkdir_wrap): Wrap in #ifdef HAVE_MKDIR, and + provide a Windows version if HAVE_MKDIR is missing, but + HAVE_WINDOWS_H is true. + (makedev_wrap, major_wrap, minor_wrap): Wrap with #if HAVE_MAKEDEV. + (mknod): Use #if HAVE_MKNOD. + (symlink_wrap, link_wrap, readlink_wrap): Wrap with #if HAVE_SYMLINK. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (debug): Fix 2013-12-02 regression, which leaves the + debugger not printing the current line of data. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Intern symlink_wrap, link_wrap, readlink_wrap. + + * stream.c (symlink_wrap, link_wrap, readlink_wrap): New functions. + + * stream.h (symlink_wrap, link_wrap, readlink_wrap): Declared. + + * txr.1: Documented. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register forgotten s_ifsock variable. + Register mkdir_wrap, chdir_wrap, getcwd_wrap, makedev_wrap, minor_wrap, + major_wrap and mknod_wrap. + + * stream.c (mkdir_wrap, chdir_wrap, getcwd_wrap, makedev_wrap, + minor_wrap, major_wrap, mknod_wrap): New functions. + (stream_init): Initialize forgotten variable s-ifsock. + + * stream.h (s_ifsock): Declared. + (mkdir_wrap, chdir_wrap, getcwd_wrap, makedev_wrap, + minor_wrap, major_wrap, mknod_wrap): Declared. + + * txr.1: Forgotten documentation for remove-path and rename-path + added. Forgotten mention of s-ifsock. Documented new Unix functions. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Support octal and binary numbers. + + * txr.1: Documented. + + * genvim.txr, txr.vim: Updated. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (modifiers): Bugfix: list element not subject to expansion + of Lisp forms denoted by @. + (expand_meta): Bugfix: failure to expand vars, which can be + symbol macros now. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print, obj_pprint): Render quasi-quote hash and + vector literals using their original notation. + + * parser.y (unquotes_occur): Takes new argument, level. + Only finds quotes which are at the given quasiquoting level. + Finally, this is the right semantics. In the first version of this + function, we were not eager enough: we neglected to find unquotes + that were wrapped in nested quasiquotes. Then we were too eager: + finding any unquotes, even ones belonging to the inner backquotes. + (vector, hash, choose_quote): Pass zero to unquotes_occur function. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_qquote): Bugfix. Was not handling an unquote + in the dotted position: (qquote x1 .. xn . (unquote form)), + which looks like the structure (qquote x1 .. xn unquote form). + Fixed by a hack: after recursively expanding the (unquote form) + part (rest of the form), we check whether the unexpanded + version has unquote at the front. If so, we know the expansion is just + form, and use it to emit the suitable expansion. + +2014-02-24 Kaz Kylheku <kaz@kylheku.com> + + Symbol macros. + + * eval.c (top_smb, defsymacro_s, symacrolet_s): New global variables. + (lookup_symac, get_opt_param_syms, get_param_syms, op_defsymacro, + expand_symacrolet, make_var_shadowing_env): New static functions. + (expand_tree_cases, expand_catch_clause): Install shadowing environment + so lexical bindings hide any symbol macrolets. + (expand_place): Fix neglect to expand an atomic form, which breaks + symbol macros used as places. + (expand): Expand symbol macros, expand symacrolet binding forms. + Make sure symbol macros are shadowed in the lexical binding + constructs. Take advantage of return value of rlcp_tree in a + few places. + (macro_form_p): Support for symbol macros; bugfix: not handling + default argument. + (macroexpand_1): Streamlined, and support added for symbol macros. + (eval_init): Protect top_smb from gc. Create new hash, stored in + top_smb. Initialize defsymacro_s and symacrolet_s. + Register op_defsymacro. + + * parser.y (rlcp_tree): Return the to form instead of useless t and nil. + + * txr.1: Documented. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Document quasiquote operator syntax. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (env_fbind, env_vbind): Use acons_new_c, and provide + a much more useful return value: the binding cell itself, + rather than the symbol. + (bind_args): Bugfix: env_vbind was still being called in one case, + leading to a neglect to bind a special variable properly. + (bindings_helper): Changed interface. Bugfix: in sequential binding, + actually bind the successive environments so closures will behave + 100% correctly. Returns a list of the actual bindings, in which + special variables are not distinguished in any way. + (op_let, op_for): Conform to new bindings_helper interface. Use the + lexical environment that it returns instead of making a new one. + (op_each): Use the environment and list of bindings out of + bindings_helper. The bindings are used for stepping the lists, + and that includes stepping any special vars. The lexical environment + is used for evaluating the body. Thus, we need no special processing + for special vars here any more. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bindings_helper): This must now bind dynamic values + rather than just assign to them. Got rid of the superfluous variable + saving array. Fixed the problem in recognizing the special_s symbol (it + is bindable). + (op_with_saved_vars): This simplifies, since it no longer needs + to save individual variables in an array, only to set up and + tear down a new dynamic environment frame. + (expand_vars): No longer returns two values with a cons. + Takes a form argument for error reporting and a pointer to a boolean + just to report whether there are special vars without listing them. + (expand_save_specials): The with-saved-specials form doesn't need + a var list any more, so the expander is updated not to stick them in. + (expand): Update calls to expand_vars to new interface. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (get_string_from_stream): Bugfix: do not abort if + stream is not a string stream, but throw a proper error exception. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + Get special variable overriding working in function and macro + parameter lists. + + There is a bugfix here too (see eval_init below). + + * eval.c (special_s): New global variable. This symbol is used + as a marker in parameter lists denoting expander-generating + syntax that gives information about specials. + (lookup_var, looup_var_l): Bugfix: walk the dynamic chain properly. + Fallback from the lexical chain to the dynamic. + (env_vbind_special): New static function. + (bind_args, bind_macro_params): Detect special list in params. Use + env_vbind_special to bind variables either in the dynamic environment + or the lexical one. + (expand_opt_params, expand_params): Renamed to expand_opt_params_rec + and expand_params_rec, respectively. Now take extra argument for + accumulating list of special variables found in the param list. + (expand_params): New static function. + (set_dyn_env): New static function. + (interp_fun, expand_macro): Set up and tear down new dynamic + environment around parameter list evaluation and body evaluation. This + will take any new dynamic bindings. + (bindings_helper, op_each): The special_s symbol is now used instead of + colon_k for marking specials. + (op_defun): Recognize the specials added to the parameter list + so as not to t report that as not a bindable symbol. + (op_catch): Set up an tear down new dynamic environment around + the evaluation of the catch clause param binding and body. + (expand_vars): use colon_s symbol instead of colon_k for marking + special var. + (eval_init): Bugfix: gc-protect recently added dyn_env variable. + Intern special symbol into special_s variable. + +2014-02-23 Kaz Kylheku <kaz@kylheku.com> + + Introducing some changes for improved handling of special variables. + Saving and restoring of individual variables is very silly + and it's making it difficult to implement function parameters + that are rebindings of special variables. + + Let's have a global pointer to a dynamic environment called dyn_env. + Let's make it part of the extended_jmp_buf structure, so that it's + implicitly saved and restored across exception handling. + + Special variable lookups go through the dyn_env chain, and fall back on + the global bindings. + + To override some specials, we just push a new dynamic environment and + stick them there. (As a bonus, the bindings can even be repeated in the + lexical environment (i.e. the same objects), so they can be found + faster. We have to make sure we remove that environment when we + leave the scope in the normal way. If we unwind out, it is done + automatically by extended_longjmp mechanism. + + * eval.c (dyn_env): New global variable. + (lookup_var, lookup_var_l): If env is nil, look in the dyn_env + first, and only if that fails, look in the global bindings top_vb. + + * signal.h (extended_jmp_buf): New member, de, for saving/restoring + dyn_env. This structure is now used whether or not we have signals. + (extended_setjmp, extended_longjmp): Updated to save and restore + dyn_env, and to do it regardless of whether there is POSIX signal + support. + (dyn_env): Declared here. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (symbol_function): Retrieve the global macro binding if the + function lookup fails, and if that fails, retrieve the special + operator binding. + (fboundp): Report t if there is a global macro binding. + + * txr.1: Use "global" rather than "toplevel". Added note that + the fun operator doesn't see macro bindings. Documented that + symbol-function and fboundp see global macro bindings, + and that symbol-function can retrieve a special operator binding. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (prinl, pprinl): New functions. + (eval_init): Registered as intrinsics. + + * txr.1: Documented. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_init): changing the quote, qquote, unquote + and splice symbols to be in the regular user package, + rather than the system package. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Allow the (. expr) syntax to denote expr. + + * eval.h: Declare existing lambda_s extern variable. + + * lib.c (obj_print, obj_pprint): print (lambda sym ...) + as (lambda (. sym) ...) and (lambda sym) as (lambda (. sym)). + + * txr.1: document it. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + Implemented macrolet. + + * eval.c (macrolet_s): New variable. + (lookup_mac, expand_macrolet): New static functions. + (expand): Recognize and expand macrolet form. + use lookup_mac to resolve macros rather than gethash, + since we have lexical macros now. + (eval_init): Intern macrolet symbol. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_macro_params, expand_macro, macroexpand_1, macroexpand): + Rename mac_env variable to menv for consistency. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + Preparation for lexical macros: we need to pass a macro + environment down through the expander call hierarchy. + + * eval.c (expand_opt_params, expand_params, expand_tree_cases, + expand_tree_case, expand_forms, val expand_cond_pairs, val + expand_place, expand_qquote, expand_vars, expand_quasi, expand_op, + expand_catch_clause, expand_catch, expand): All expanders get new + parameter, menv. expand_forms and expand handle a nil value of menv. + (eval_intrinsic): Pass nil macro environment to expand. + (eval_init): Update intrinsic registration for expand. + + * eval.h (expand, expand_forms): Declarations updated. + + * parser.y (expand_meta): Gets macro env parameter. + (elem, o_elem, exprs, expr): Pass nil to expand_forms and expand_meta. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + Replacing uses of the eq function which are used only as C booleans, + with just using the == operator. + + Removing cobj_equal_op since it's indistinguishable from eq. + + Streamlining missingp and null_or_missing_p. + + * eval.c (transform_op): eq to ==. + (c_var_ops): cobj_equal_op to eq. + + * filter.c (trie_compress, trie_lookup_feed_char, filter_string_tree, + html_hex_continue, html_dec_continue): eq to ==. + + * hash.c (hash_iter_ops): cobj_equal to eq. + + * lib.c (countq, getplist, getplist_f, search_str_tree, + posq): eq to ==. + (cobj_equal_op): Function removed. + + * lib.h (cobj_equal_op): Declaration removed. + (missingp): Becomes a simple macro that yields a C boolean instead + of t/nil val, because it's only used that way. + (null_or_missing_p): Becomes inline function returning int. + + * match.c (v_output): eq to ==. + + * rand.c (random_state_ops): cobj_equal_op to eq. + + * regex.c (char_set_obj_ops, regex_obj_ops): cobj_equal_op to eq. + (reg_derivative): Silly if3 expression replaced by null. + (regexp): Redundant if2 expression wrapped around eq removed. + + * stream.c (null_ops, stdio_ops, tail_ops, pipe_ops, string_in_ops, + byte_in_ops, string_out_ops, strlist_out_ops, dir_ops, + cat_stream_ops): cobj_equal_op to eq. + + * syslog.c (syslog_strm_ops): cobj_equal_op to eq. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + The C function nullp is being renamed to null, and the rarely + used global variable null which holds a symbol becomes null_s. + + A new macro called nilp is added that more efficiently checks whether + an object is nil, producing a C boolean value rather than t or nil. + + Most of the uses of nullp in the codebase just become the more + streamlined nilp. + + * debug.c (show_bindings): nullp to nilp + + * eval.c (lookup_var, lookup_var_l, lookup_fun, lookup_sym_lisp1, + do_eval, expand_qquote, expand_quasi, expand_op): nullp to nilp. + (op_modplace): nullp to null. + (eval_init): Update registration of null and not from C function + nullp to null. + + * filter.c (trie_compress, html_hex_continue): nullp to nil. + (filter_string_tree): null to null_s. + + * hash.c (hash_next): nullp to nilp. + + * lib.c (null): Variable renamed to null_s. + (code2type): null to null_s. + (lazy_flatten_scan, chainv, lazy_str, lazy_str_force_upto, + obj_print, obj_pprint): + nullp to nilp. + (obj_init): null to null_s; nullp to null. + + * lib.h (null): declaration changed to null_s. + (nullp): Inline function renamed to null. + (nilp): New macro. + + * match.c (do_match_line): nullp to nilp. + + * rand.c (make_random_state): Likewise. + + * regex.c (compile_regex): Likewise. + +2014-02-22 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Bugfix, I think. :) We do not need to call the expander + for a Lisp expression in a quasistring, and doing so leads to a + potentially incorrect double expansion because the whole thing + will be expanded again, either by the TXR Lisp quasi operator's + expander (if this is a quasiliteral in Lisp) or in the TXR + pattern language, by expand_meta, which hunts down down @ expressions + and expands them. + +2014-02-21 Kaz Kylheku <kaz@kylheku.com> + + Implementing special variables with local overriding. + All predefined globals become special. + + * eval.c (special, with_saved_vars_s): New global variables. + (mark_special, special_p): New functions. + (bindings_helper): Takes new argument, include_specials. + Now processes the special colon syntax for denoting special variables, + setting up their values, taking care to observe whether the + binding is parallel or sequential. + (op_let, op_for): Pass new argument to bindings_helper. + (op_each): Pass new argument to bindings_helper with a value of it, + and deal with the colon annotations that emerge in the bindings. + (op_defvar, regvar): Mark symbol as special. + (op_with_saved_vars): New static function. + (expand_vars): Takes new argument, returns a cons. Detects special + variables among the vars and produces the colon syntax. + (expand_catch_clause): Bugfix: this was using expand_vars on + a parameter list. Now properly uses expand_params. + (expand_save_specials): New static function. + (expand): For the operators that are binding constructs, handle + the new form of expand_vars which returns information about + special variables. If specials occur, then generate the + with-saved-vars form around the expansion which will save and restore + their values. The expansion of vars done by expand_vars, together + with the run-time actions of bindings_helper, do the rest. + Speaking of which, the new with-saved-vars operator form is now + expanded here too. + (eval_init): Protect new variables special and with_saved_vars_s. + Initialize special with new hash table. Store new interned + symbol in with_saved_vars_s. Register op_with_save_vars in op_table. + + * txr.1: Documented specials. + +2014-02-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars): Change throwing of query_error_s + to eval_error call. This is a leftover from when this was cloned + from its counterpart in match.c. However, I can't think of a way + this call can ever be reached. + +2014-02-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (macro_form_p, macroexpand_1, macroexpand): New static + functions. + (eval_init): Registered new functions. + + * txr.1: Documented. + +2014-02-20 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (unquotes_occur): Bugfix: we should not terminate + the recursion early if we see a quote. This would be true if + the only quotes were those generated by the parser based on + calls to choose_quote. However, it breaks for something + like an explicitly coded '(sys:quote ,form), which becomes + (sys:quote (sys:quote ,form)), leaving a dangling unquote. + +2014-02-19 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/faster-square-root (mp_sqrt): Bugfix: was computing square + roots that were incorrect in the last digit/bit, because it was not + generating the guess mask all the way down to bit zero. Also, added + an early test to bail the loop when an the guess at the root happens + to be right. + + * mpi-patches/add-bitops: Refreshed. + + * mpi-patches/fix-ctype-warnings: Likewise. + + * mpi-patches/mpi-to-double: Likewise. + +2014-02-19 Kaz Kylheku <kaz@kylheku.com> + + Fixing a long-running issue in the TXR pattern language: premature + opening of files, prior to directives that actually need data. + The documentation basically lied that this is the case: namely, + the text "A file isn't opened until the query demands material from + that file, and then the contents are read on demand, not all at + once." This is now a fact. + + * match.c (non_matching_directive_table): New global variable. + (open_data_source): New static function. Contains an almost + verbatim migration of the source-opening logic that used + to be in match_files. The useless assignment to c->nil is gone, + and c->data == t is explicitly tested for. Instead of assuming + that only the @(next) directive does not need to have a data source + open, the table of non-matching directives is consulted. Opening + the data source is now skipped for numerous directives. + (match_files): Call open_data_source within the loop. This + means that even after processing numerous non-matching directives, + we will still correctly set up the data lazy list. + (dir_tables_init): Initialize non_matching_directive_table, + protect from GC and populate with numerous directives. + + * txr.1: Improved documentation for @(next :args), and removed + a description of the hack that a single @(next) at the top of the + query suppressed the opening of the data source. + +2014-02-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (env_hash): new function. + (eval_init): Register env and env_hash functions. Register prog_args + and prog_args_full as *args* and *full-args*. + + * lib.c (timegm_hack): Invalidate env_list, after mucking with + the environment via setenv and unsetenv. + + * txr.c (prog_args_full, prog_args): New global variables. + (txr_main): Command-line processing converted to use TXR's + library. Populates prog_args_full and prog_args. + + * txr.h (prog_args_full, prog_args): Declared. + + * txr.1: Documented *args*, *full-args*, env and env-hash. + +2014-02-18 Kaz Kylheku <kaz@kylheku.com> + + The mode argument in some stream-opening functions becomes optional. + + * eval.c (eval_init): Change registration for open_file, open_tail + and open_command. + + * stream.c (open_file, open_tail, open_command): mode_str argument + defaulted. + + * txr.1: Updated. + +2014-02-18 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_tail): Fix 2013-12-02 regression: + seek_to_end_p argument being ignored, with the + behavior being no initial seek to the end. + + * txr.1: Clarified the behavior of the seek-to-end-p + option. + +2014-02-18 Kaz Kylheku <kaz@kylheku.com> + + * combi.c (rperm_gen_full): Update ptail with return value + of list_collect. This is not necessary for correctness, but it spares + list_collect from having to search for the tail on each call. + +2014-02-17 Kaz Kylheku <kaz@kylheku.com> + + Version 80 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim, dep.mk: Updated. + +2014-02-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_args, op_defvar, op_defun, op_defmacro, op_modplace): + Fix sybol misspelling. + +2014-02-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register gensym function as the gensym + intrinsic, rather than gensymv. + Register gensym_counter as *gensym-counter*. + + * lib.c (gensym): Handle missing prefix argument by defaulting + the prefix to "g". + (gensymv): Function removed. + + * lib.h (gensymv): Declaration removed. + + * txr.1: Fixed omission: missing documentation for gensym. + Documented *gensym-counter*. + +2014-02-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand): Bugfix: do not expand any part of the macro + form via expand_forms. This is completely wrong since only the + macro knows what material is evaluated and what isn't. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_tree_case): Removed superfluous handling of + nil parameter list; this is correctly handled by bind_macro_params. + Added support for the colon symbol as an indication that + the case is declined. + + * txr.1: Updated. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + New destructuring operators. + + * eval.c (tree_case_s, tree_bind_s): New symbol variables. + (bind_macro_params): Bugfix: inappropriate exception thrown when atom + matched against parameter list. Bugfix: nil being returned when + atom matches empty parameter list. Added support for a new convention: + if loose_p is the colon keyword, then exceptions are not thrown + for destructuring mismatches; nil is returned instad. + (op_tree_case, expand_tree_cases, expand_tree_case, op_tree_bind): + New static functions. + (expand): Handle tree_case_s and tree_bind_s. + (eval_init): Intern tree-case and tree-bind symbols. + Register the corresponding operator functions op_tree_case and + op_tree_bind under these symbols in op_table. + + * txr.1: Documented tree-case and tree-bind operators. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_macro_params): Bugfix: enforce mismatch on + superfluous material. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + In the spirit of the previous hack, here is another hack to + alleviate a long-standing pain: when an exception happens in + TXR's library somewhere, the program dies without leaving + a clue about what code was being evaluated when that happened. + What we can do is have the evaluator publish the most recent + compound form it has processed by stashing it in a variable. + Then when an unhandled exception occurs, we can peek at that + and try to pull out source location info. + + * eval.c (last_form_evaled): New variable. + (do_eval): When evaluating a compound form, stash it in + last_form_evaled. + (eval_init): Protect last_form_evaled from gc. + + * eval.h (last_form_evaled): Declared. + (eval_error_s): Existing variable declared. + + * unwind.c: Has to include "eval.h" for the above variable + and "parser.h" for the source_loc function. + (uw_throw): When an exception is unhandled, if + last_form_evaled has source info, add it to the diagnostic. + But not if the exception is eval-error; because errors from + the evaluators already have the info. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + Nice idea: how about a function which walks the tree structure and + back-fills some missing source code location info. We apply this to + macro expansions. If some error occurs in expanded code, this way it + is referenced to the line where the macro *call* occurs. + Not only is this better than nothing, it may be better than tracing + it to the macro definition. Ideally, we would have both places: + ("the error is in the code expanded from this macro, at this site"). + + * eval.c (expand): Use rlcp_tree to back-fill source info in + macro expansion by taking it from the unexpanded form. + + * parser.h (rlcp_tree): Declared. + + * parser.y (rlcp_tree): New function. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + Bugfixes: not propagating source loc info in quasiliterals. + + * eval.c (expand_quasi): Add some rlcp's here. + + * parser.y (o_var, quasi_items, o_elems_transform): Likewise. + +2014-02-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand): Replace blatant tail calls with + a backwards goto. + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_macro_params): Bugfix: the :whole parameter must + capture the entire form, not just the arguments. + (expand): Apply rlcp to the result of macro expansion, if it has + not set up source code location. + (eval_init): Register rlcp as intrinsic. + + * txr.1: Start of macro documentation. Documented rlcp. + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + First cut at implementation of macros. + + * eval.c (top_mb, defmacro_s, macro_time_s, whole_k, env_k): New global + variables. + (expand_params): Recurse to handle macro parameter lists too. + (bind_macro_params, op_defmacro, expand_macro): New static functions. + (expand): Evaluate defmacro forms and macro-time forms at expansion + time. Recognize and expand macros (albeit not yet with proper lexical + scoping: local bindings are not able to shadow a macro). + (eval_init): Protect top_mb from GC and initialize it. + Intern new symbols defmacro, macro-time, and :whole. + Register defmacro operator in op_table. + + * match.h (env_k): Added declaration for existing external variable. + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: parameters of defun were not being treated by expander. + They need to be since they can contain evaluable initforms for default + arguments. Ditto with lambda; lambda parameters were being treated + using expand_vars, which is not appropriate. + + * eval.c (expand_opt_params, expand_params): New static functions. + (expand_vars): Bugfix: added some missing rlcp's to propagate + line number info. + (expand): Move lambda to separate case, and expand its params + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_args, op_modplace): Use ~s consistently + in error messages. + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bindings_helper): Fixed misspelling of "symbol" in exception + message. + +2014-02-15 Kaz Kylheku <kaz@kylheku.com> + + A trivial change in the UTF-8 decoder allows TXR to handle null bytes + in text. + + * utf8.h (UTF8_ADMIT_NUL): New preprocessor symbol. + (utf8_decoder): New member, flags. + + * utf8.c (utf8_decoder_init): Initialize flags to 0. + (utf8_decode): If a null byte is encountered in the input, + then convert it to 0xDC00, rather than keeping it as zero, + unless flags contains UTF8_ADMIT_NUL. + + * txr.1: Document handling of null bytes. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_update): Avoid double cdr. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (inhash): Simplify with gethash_c. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + Replacing acons_new_l and aconsq_new_l interfaces with ones + that return the new or old cons cell rather than a pointer + to its cdr field. + + * eval.c (transform_op): use of acons_new_l replaced with + acons_new_c. + + * hash.c (struct hash): acons_new_l_fun member replaced + with acons_new_c_fun. + (make_hash, make_similar_hash): initialize acons_new_l_fun + member using either acons_new_c or aconsql_new_c. + (gethash_l): function becomes an inline in hash.h. + (gethash_c): new function, based on gethash_l. + (inhash, gethash_n): updated w.r.t struct hash change. + + * hash.h (gethash_c): declared. + (gethash_l): becomes an inline wrapper for gethash_c. + + * lib.c (acons_new_l, aconsql_new_l): functions removed. + (acons_new_c, aconsql_new_c): new functions. + (obj_init): use gethash_c and rplacd instead of gethash_l + and set. + + * lib.h (acons_new_l, aconsql_new_l): declarations removed. + (acons_new_c, aconsql_new_c): declared. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (inhash): Simplify code with gethash_f. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered hash_update_1 as intrinsic. + + * hash.c (hash_update_1): New function. + + * hash.h (hash_update_1): Declared. + + * txr.1: Documented hash-update-1. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (inhash): Rename local variable from new to new_p. + This is consistent with usage elsewhere in the module, + and more importantly, keeps our code C++. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (class_check): Bugfix: the function was assuming that + the argument is a pointer type object, leading to a crash + if given a fixnum integer or string literal. + +2014-02-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register inhash as intrinsic. + + * hash.c (inhash): New function. + + * hash.h (inhash): Declared. + + * txr.1: Documented inhash. Also, added surprisingly missing + documentation for gethash! + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + Different approach: optional arguments on hash-isec and hash-uni allow + for more flexible joining of data from the hash tables. + + * eval.c (eval_init): Remove hash_guni and hash_gisec. Change + registration for hash_uni and hash_isec to three arguments with + one optional. + + * hash.c (hash_uni): Third parameter introduced, join_func. + The default behavior changes: in the two argument case, + clashing keys prefer the value from hash1 rather than hash2. + For this reason, we now iterate over hash2 first, then hash1. + (hash_guni): Removed. + (hash_isec): Third parameter introduced, join_func. + (hash_gisec): Removed. + + * hash.h (hash_uni, hash_isec): Declarations updated. + (hash_guni, hash_gisec): Delarations removed. + + * txr.1: Documentation updated. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register hash_guni and hash_gisec as intrinsics. + + * hash.c (hash_guni, hash_gisec): New functions. + (hash_isec): Bugfix: since gethash was naively used, keys in hash2 + associated with the value nil were erroneously omitted from the + intersection. + + * hash.h (hash_guni, hash_gisec): Declared. + + * txr.1: Documented new functions. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Disallow syntax like 1.0a, flagging it as + an invalid floating-point token. The problem is that 1a is allowed, + for compatibility with other Lisp dialects (it is a symbol) whereas + 1.0a was scanning as 1.0 followed by a, which is inconsistent. + Some Lisp dialects embedded dots in symbols, and allow 1.0a + as a symbol token. We don't. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (struct hash): New member, equal_fun. + (hash_equal_op): Short circuited logic: whenever we pull + identical cells from either hash, we don't have to go + through the pending lookaside list. + (make_hash, make_similar_hash): Initialize new structure member. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_equal_op, hash_hash_op): New static functions. + (hash_ops): New functions registered in table of operations. + + * txr.1: Documentation for equal function updated to explain + how two hashes are equal. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + Undoing bogus optimization, which can only work when objects + are treated as immutable. + + * hash.c (last_equal_key, last_equal_hash): Variables removed. + (equal_hash, hash_process_weak): All references to removed + variables scrubbed. + +2014-02-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (some_satisfy, all_satisfy, none_satisfy): Fix coding + mistake from two commits ago leading to a regression. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register new functions posqual, posql, + posq, pos, and pos_if as intrinsics. + + * lib.c (posqual, posql, posq, pos, pos_if): New functions. + + * lib.h (posqual, posql, posq, pos, pos_if): Declared. + + * txr.1: Documented + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Turn a require argument into an optional + one for the functions some, all and none. + + * lib.c (some_satisfy, all_satisfy, none_satisfy): Add defaulting + behavior for pred parameter. + + * txr.1: Document that the predicate function is optional + in calls to some, all and none. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + Version 79 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (if_fun): Bugfix: forgotten handling of optional + alternative argument. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars): Bugfix: results of expressions not + treated in the same way as variables: lists not stringified, + causing expansions with parentheses, and sometimes errors + due to unhandled objects. Also, use tostringp instead of + format for stringifying objects. + stringifying object. Bugfix.k + + * match.c (subst_vars): Added comment similar to the one in + the subst_vars of eval.c. Removed superfluous conversion code where the + str variable is already known to be a string. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apply): Fix regression in dwim operator: not handling + nil object properly. Since dwim now trivially delegates to apply, + apply must recognize nil. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * combi.c (comb_hash_while_fun, comb_hash_gen_fun, comb_hash): New + static functions. + (comb): Support hash tables. + + * hash.c (print_key_val): When values are nil, print in a more condensed + way by omitting the second element. This notation is accepted as + input already by the parser. + (hash_insert_pair): New function. + + * txr.1: Description of comb updated to indicate that it works over + hashes. + +2014-02-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Remove inclusion of unneeded headers: <limits.h> + and <dirent.h>. + +2014-02-10 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (OBJS): New object file, combi.o. + + * dep.mk: Updated. + + * eval.c: Removed combination and permutation functions; they + now reside in combi.c. + (generate): Changed from static to external linkage. + + * eval.h (generate): Declared. + + * combi.c: New file. + + * combi.h: New file. + + * txr.vim: Regenerated. + +2014-02-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rcomb_gen_fun_common): Streamlined implementation. + Got rid of stack that is consed up on each call. + +2014-02-10 Kaz Kylheku <kaz@kylheku.com> + + Relaxed behavior: don't throw errors for impossible permutations, + but return an empty list. + + * eval.c (perm_init_common): Do not throw error; return a nil + state if permutation length exceeds sequence length. + (perm_vec, perm_list, perm_str): Check for null return from + perm_init_common and return empty list. + (k_conses): Do not throw error; return empty list. + (comb_list_gen_fun): Check for nil value out of k_conses. + (comb): For vectors and strings, check length against k and + return nil if necessary. For lists, comb_list_gen_fun handles it. + + * txr.1: Section order rearranged, and updated. + +2014-02-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rcomb_while_fun, rcomb_gen_fun_common, + rcomb_list_gen_fun, rcomb_list, rcomb_vec_gen_fun, rcomb_vec, + rcomb_str_gen_fun, rcomb_str, rcomb): New static functions. + (eval_init): Register rcomb as intrinsic. + + * txr.1: Documented rcomb. + +2014-02-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rperm, perm_str): Just in case, return a mutable empty + string, rather than null_string, which is a literal. + (k_conses, comb_while_fun, comb_gen_fun_common, + comb_list_gen_fun, comb_list, comb_vec_gen_fun, comb_vec, + comb_str_gen_fun, comb_str, comb): New static functions. + (eval_init): Registered comb as instrinsic. + + * txr.1: Documented comb. + +2014-02-09 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rperm_list, rperm_vec, rperm_str): n variable renamed + to k, for consistency with rperm. + (rperm): Likewise, and the behavior in the k == zero case is + changed to return a single empty permutation. + (perm_while_fun, perm_index, perm_gen_fun_common, perm_init_common, + perm_vec_gen_fill, perm_vec_gen_fun, perm_vec, + perm_list_gen_fill, perm_list_gen_fun, perm_list, + perm_str_gen_fill, perm_str_gen_fun, perm_str, perm): New + static functions. + (eval_init): perm registered as intrinsic. + + * lib.c (vecref_l): Bugfix: allow negative indices, just like vecref. + + * lib.h (three, four): New macros. + + * txr.1: Updated documentation for rperm. Documented perm. + +2014-02-08 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (vector, vec_set_length, cat_vec): When the vector size + overflows the size_t type that is passed to malloc, throw an exception. + +2014-02-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (nperm_while_fun, nperm_gen_fun, nperm_list, + nperm_vec_gen_fun, nperm_vec, nperm_str_gen_fun, nperm_str, nperm): New + static functions. + (eval_init): nperm registered as intrinsic. + + * txr.1: Documented nperm function. + +2014-02-06 Kaz Kylheku <kaz@kylheku.com> + + Version 78 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim: Updated. + +2014-02-06 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (int_str): Regression: handle base argument properly. + +2014-02-06 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_grow, make_hash, make_similar_hash, copy_hash): + Pass second argument to vector. + + * lib.c (vector): Takes additional argument specifying the value + for the slots of the vector. + (vector_list, sub_vec): Pass second argument to vector. + + * lib.h (vector): Declaration updated. + + * eval.c (eval_init): Register vector as two-argument function + with one required arg. + + * txr.1: Updated. + +2014-02-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_dwim): Gutted down to just a few lines. + Basically the dwim operator is just a Lisp-1 version of the call + operator now. It doesn't have to do anything funny with non-function + objects, since they are callable. + + * lib.c (chr_str, chr_str_set, vecref, vecref_l): Replace + inappropriate internal assertions with error exceptions. + + * unwind.h (numeric_assert, range_bug_unless): Unused macros + removed. + +2014-02-05 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (generic_funcall): If a cons cell is passed as + one argument to a sequence being used as a function, split it into two + arguments. This is consistent with the DWIM operator behavior. + + * txr.1: Document callable objects. + +2014-02-05 Kaz Kylheku <kaz@kylheku.com> + + Allow sequences and hashes to be called as functions. + This is already supported in the DWIM operator. + + * eval.c (apply): If object isn't a function, gather the + arguments into an array and delegate to generic_funcall. + + * lib.c (generic_funcall): Changed from static to external linkage. + Supports sequences and hashes as functions. Error messages + fixed not to refer to "funcall". + (funcall, funcall1, funcall2, funcall3, funcall4): Do not throw + exception if fun is not of FUN type; instead, delegate + to generic_funcall. Error messages fixed not to refer to "funcall". + + * lib.h (generic_function): Declared. + +2014-02-05 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (lognot): Conform to new scheme for defaulting optional args. + + * eval.c (apply): Unconditionally use colon_k for missing optional + args, for intrinsic functions. + (eval_intrinsic, rangev, rangev_star, errno_wrap): Conform to new + scheme for defaulting optional args. + (reg_fun_mark): Function removed. + (eval_init): Switch reduce_left and reduce_right back to reg_fun + registration. + + * hash.c (gethash_n): Conform to new scheme for defaulting optional + arguments. + + * lib.c (sub_list, replace_list, remove_if, keep_if, remove_if_lazy, + keep_if_lazy, tree_find, count_if, some_satisfy, all_satisfy, + none_satisfy, search_str, match_str, match_str_tree, sub_str, + replace_str, cat_str, tok_str, intern, rehome_sym, sub_vec, + replace_vec, lazy_str, sort, multi_sort, find, find_if, set_diff, + obj_print, obj_pprint): Conform to new scheme for defaulting optional + arguments. + (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, func_n1, + func_n2, func_n3, func_n4, func_n5, func_n6, func_n7, func_f0v, + func_f1v, func_f2v, func_f3v, func_f4v, func_n0v, func_n1v, + func_n2v, func_n3v, func_n4v, func_n5v, func_n6v, func_n7v): + Remove references to removed mark_missing_args member of struct func. + (func_set_mark_missing): Function removed. + (generic_funcall): Unconditionally use colon_k for missing optional + args, for intrinsic functions. + + * lib.h (struct func): mark_missing_args member removed. + (func_set_mark_missing): Declaration removed. + (default_arg, default_bool_arg): New inline functions. + + * rand.c (random): Left argument is not optional. + (rnd): Conform to new scheme for defaulting optional arguments. + + * regex.c (search_regex, match_regex): Conform to new scheme for + defaulting optional arguments. + + * stream.c (unget_char, unget_byte, put_string, put_char, put_byte, + put_line): Conform to new scheme for defaulting optional arguments. + + * syslog.c (openlog_wrap): Conform to new scheme for defaulting + optional arguments. + + * txr.1: Remove the specification that nil is a sentinel value in + default arguments, where necessary. Use consistent syntax for + specifying variable parts in argument lists. A few errors and omissions + addressed. + +2014-02-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_args): Support optional parameters in the + form (sym initform present-p-sym). Also, support the convention + that a value of : explicitly passed for an optional argument produces + the same behavior as if that argument value were missing. + + * txr.1: Document new conventions. + +2014-02-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apply): Pass missing optional arguments as colon_k. + to functions for which this is requested. + (reg_fun_mark): New static function. + (eval_init): Register reduce_left and reduce_right as requiring + marking for missing optionals. + + * lib.c (func_set_mark_missing): New function. + (generic_funcall): Pass missing optional arguments as colon_k + to functions for which this is requested. + (reduce_left, reduce_right): Handle missing values of init and key. + (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, func_n1, + func_n2, func_n3, func_n4, func_n5, func_n6, func_n7, func_f0v, + func_f1v, func_f2v, func_f3v, func_f4v, func_n0v, func_n1v, + func_n2v, func_n3v, func_n4v, func_n5v, func_n6v, func_n7v): + Initialize new mark_missing_args member of struct func. + + * lib.h (struct func): New bitfield member, mark_missing_args. + (func_set_mark_missing): Declared. + (missingp, null_or_missing_p): New inline functions. + + * txr.1: Updated descriptions of reduce-left and reduce-right. + +2014-02-03 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (nullp): Function removed. + + * lib.h (nullp): New inline function. + (eq): Removed useless parentheses: it's not a macro. + +2014-02-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_modplace): Fixing a few argument count mismatches + in format calls. + +2014-02-03 Kaz Kylheku <kaz@kylheku.com> + + Tighten up environment visibility semantics of default argument + init forms, while streamlining the implementation. + + * eval.c (bind_args): Get rid of opt_init_parm array hack; + turns out we already have a useful function for extending an + environment object, namely env_vbind: hitherto unused! We now make a + blank environment upfront and extend it destructively with env_vbind. + Except that after evaluating an init form, we create a new environment: + this is in case a lexical closure was made by the init form, in which + event we don't want to be destructively manipulating the environment + any more. + + * txr.1: Clarified. + +2014-02-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Fix incorrect registration of + list_star_intrinsic, diagnosed by compiler. + +2014-02-02 Kaz Kylheku <kaz@kylheku.com> + + Adding list* since we get it "for free" thanks to the + new helper function that supports apply. + + * eval.c (list_star_intrinsic): New static function. + (eval_init): Register list_star_intrinsic as list*. + + * txr.1: Document list*. + +2014-02-02 Kaz Kylheku <kaz@kylheku.com> + + append can now take additional leading arguments before the list. + + * eval.c (apply_frob_args): New static function. + (apply_intrinsic): Process arguments with apply_frob_args. + (eval_init): apply_intrinsic registered differently, as a + variadic function with one mandatory arg. + + * lib.c (lastcons): New function. + + * lib.h (lastcons): Declared. + + * txr.1: Updated append documentation. + +2014-02-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apply): Support string and vector arglist. + (do_eval_args): Support string or vector in dot position. + + * lib.c (tolist): New function. + + * lib.h (tolist): Declared. + + * txr.1: Document how apply and dot position in compound forms + supports strings as well as vectors. + +2014-02-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (do_eval_args): If the dotted position of the argument + list evaluates to a vector, then convert the vector to a list. + + * txr.1: Document compound forms, and how they allow the dotted + position and how it may be a vector. + +2014-02-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register vector_list function a second time + under the name vec, as a variadic function. + + * txr.1: Document vec. + +2014-02-01 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_mark, hash_grow, make_similar_hash, + copy_hash, gethash_l, gethash, gethash_f, gethash_n, + hash_count, hash_next, hash_eql, hash_equal): Use + num_fast instead of num. + (make_hash): An attempt to make a weak-keys hash that has + equal-based keys is nonsensical; it is now diagnosed with + an exception. Use num_fast instead of num. + (hash_process_weak): Call breakpt whenever the weak object(s) due to + which entries are being deleted match the value in break_obj. + Use num_fast instead of num. + + * parser.l (parse_init): Bugfix: the forms_to_ln_hash + was equal-based, which makes no sense. + +2014-02-01 Kaz Kylheku <kaz@kylheku.com> + + Export break_obj. + + * gc.c (break_obj): Change to external linkage. + (mark): Use t variable instead of re-evaluating obj->t.type. + + * gc.h (break_obj): Declared. + +2014-02-01 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (d): Disable garbage collection around the dump. + It's importa for this function not to have the side effect + of triggering garbage collection, when it is used for + debugging issues that show up under --gc-debug. + +2014-01-31 Kaz Kylheku <kaz@kylheku.com> + + * configure: remove stray debugging printf from the + GNU Make check. This was introduced in May 2012. + +2014-01-30 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Support floating-point constants. + Do not color embedded decimal integers in symbols as integers. + + * txr.vim: Regenerated. + +2014-01-30 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_op): When the variant is do, and the symbol + is an operator, then suppress the addition of the rest + parameter in the dotted cons position at the end of the form, + since this mechanism for applying additional arguments is only + supported by function calls, not by special forms. + + * lib.h (and2, and3): New macros. + +2014-01-30 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_op): Fix broken do operator. In the case of the do + operator, we must feed the entire form to the expander, not the + individual forms. That is to say (do operator arg) must expand the + form (operator arg), whereas (op fun arg) just expands the forms fun + and arg individually. + +2014-01-29 Kaz Kylheku <kaz@kylheku.com> + + Version 77 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim: Updated. + +2014-01-29 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (match_regex_right): Fix not returning value. + +2014-01-29 Kaz Kylheku <kaz@kylheku.com> + + Default argument initializer scoping rule change, allowing + things like (defun foo (s : (l (length s))) ...). + Default arguments can be initialized by expressions that + refer to the arguments. + + * eval.c (bind_args): By means of a local array, defer the evaluation + of optional argument init forms until the lexical environment, + including all the parameters, is captured. Then valuates the forms + in the array, and set the variable values. + + * txr.1: Clarify the new scoping rules. + +2014-01-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered existing make_catenated_stream as + intrinsic. + + * stream.c (cat_get_line, cat_get_char, cat_get_byte): Do not + remove the last stream from the stream list, so that there is + some stream object there which can take the unget-char or + unget-byte operations. + (cat_unget_byte, cat_unget_char): New static functions. + (cat_stream_ops): unget_char and unget_byte function pointers + filled in. + + * txr.1: Documented catenated-streams. + +2014-01-28 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (struct stdio_handle): New member, is_rotated. + Moved mode member down. + (make_stdio_stream_common): Initialize is_rotated. + (tail_strategy): Substantially rewritten in order to address + a possible race condition, when a file is rotated. + The issue is that even when the file disappears, or when the file has + been replaced, we must continue reading from the old stream until the + end, and only then can we switch to the newly rotated file. + +2014-01-28 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (remove_path, rename_path): New functions. + + * stream.h (remove_path, rename_path): Declared. + + * utf8.c (w_remove, w_rename): New functions. + + * utf8.h (w_remove, w_rename): Declared. + + * eval.c (eval_init): Registered remove_path and rename_path + as intrinsics. + +2014-01-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (meta_meta_p, meta_meta_strip): New static functions. + (transform_op): Recognize compounded metas, and strip one level off. + (eval_init): Intern sys:expand function so we have access to the + form expander from TXR Lisp. + + * lib.c (obj_print, obj_pprint): Fix: wasn't rendering metanumbers. + + * parser.y (list): Support @ in front of anything. If it's an atom, + treat it similarly to a metasymbol or metanumber. + + * txr.1: Documented meta-meta arguments in nested op. + + * genvim.txr, txr.vim: Support coloring for compounded meta syntax. + +2014-01-27 Kaz Kylheku <kaz@kylheku.com> + + Lexing and parsing improvements, leaving things less hacky than before, + albeit hacky. + + * parser.l (BSYM, NSYM): Regex definitions gone. + (BT0, BT1, BT2, NT0, NT1, NT2): New regex definitions. + (BTREG, BTKEY, NTREG, NTKEY): Rewritten, so that they cannot + match a lone @ character as a symbol name. + (grammar): Rules for returning METAPAR, METABKT and METAQUO + are gone. Instead, we just recognize a @ in the NESTED + and BRACED states and return it as a token. + + * parser.y (METAPAR, METABKT, METAQUO): Token types removed. + (meta_expr): Nonterminal symbol removed. + ('@'): New token type. + (list): Quotes and splices handling removed from this rule. + The new token '@' is handled here, on the other hand, because + there are places that reference the list rule that need to support + @ expressions. + (n_expr): Reference to meta_expr removed. Quote, unquote and splice + added here. + (yybadtoken): Removed references to METAPAR, METABKT and METAQUO. + + * txr.vim, genvim.txr: There can now be whitespace in TXR Lisp + like @ (a b c) and @ [1 2 3]. + +2014-01-27 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (yybadtoken): Handle METAQUO in switch. + +2014-01-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (reduce_left, reduce_right): changing the behavior so that + the initial value is optional. this creates the possibility that + the effective list of operands is empty, in which case the function + must support a call with no arguments, just like in the common lisp + reduce. + + * txr.1: rewrote reduce-left and reduce-right documentation. + +2014-01-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print, obj_pprint): Bugfix: there may be + additional expressions forms in (sys:var <sym>) after + the sym. These were not being printed. Such syntax occurs + often in the TXR pattern language. + +2014-01-26 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (match_regex_right): New function. + + * regex.h (match_regex_right): Declared. + + * eval.c (eval_init): Register match_regex_right as instrinsic. + +2014-01-26 Kaz Kylheku <kaz@kylheku.com> + + Implementing more correct treatment of meta forms + like @[...] and @(...) occurring in the TXR pattern language. + + The previous behavior is that the parser always expands the interior + of these forms when they occur. This is wrong. + + These forms only denote TXR Lisp (and so require expansion) when they + occur in a directive, inside a sub-expression that is not already known + to be Lisp. + + For instance in @(do @(op foo)), the inner @(op foo) should not be + subject to expansion. The reason that the argument forms of @(do) + are TXR Lisp. The @(op foo) form denotes (sys:expr foo), and that + operator currently has no meaning; and so we should not expand it. + The previous, buggy behavior would turn the @(op ..) into + a @(lambda ...). + + Another example is @(bind a @(list @(op foo))) where @(list ...) + denotes TXR Lisp and so the interior of the form should be expanded. + However, the @(op foo) should not be expanded into @(lambda ...) + + Expanding @(...) forms is not currently harmful, but it interferes + with code that wants to use the @(...) syntax for its own use, + + The solution involves adding shims in the parser so that the expansion + is only applied when expressions are reduced to the top level + within a directive, and then to walk the expressions, looking for + the @ syntax and expanding only the outermost occurrence thereof. + + * parser.y (expand_meta): New static function. + (n_exprs n_expr): New nonterminal symbols. + (elem): The arguments of the list elem (representing a generic + directive) now need to be put through expand_meta when it is not + @(do ...) or @(require ...). + (list): Use n_exprs instead of exprs. + (meta_expr): Do not call expand, and use n_expr(s) instead of expr(s). + (exprs, expr): These rules no become just a shim which expands + the outer-most metas. The actual parsing is represented by n_expr + and n_exprs ("n" stands for nested), which behave just like the + old expr and exprs. + +2014-01-26 Kaz Kylheku <kaz@kylheku.com> + + Sigh; more lexical-syntactic hacks. This adds handling + for the @' combination, as in @(bind a @'(foo ,bar)) + + * parser.l: Handle the new METAQUO token. + + * parser.y (METAQUO): New token. + (meta_expr): New "METAQUO expr" case. Added missing METABKT + error handling case. + +2014-01-24 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_update): New function. + + * hash.h (hash_update): Declared. + + * lib.c (update): New function. + + * lib.h (update): Declared. + + * eval.c (eval_init): Register hash_update and update as intrinsics. + + * txr.1: Documented. + +2014-01-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand): Bugfix in expansion of dohash: neglected + to insert result form into the expanded output. + +2014-01-23 Kaz Kylheku <kaz@kylheku.com> + + Version 76 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim, dep.mk: Updated. + + * genvim.txr: Add missing op and do. + +2014-01-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_op): Takes sym argument to distinguish op and do. + The dwim operator is deleted when the symbol is do. + (expand): Expand do_s form with expand_op, not only op_s. Pass down + symbol. + + * txr.1: Document do operator. + +2014-01-23 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (do_eval): Fix incorrect, recently introduced code + rearrangement that gives precedence to operator lookup over function + lookup. + +2014-01-23 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: @(require ...) not expanding forms. + + * eval.c (expand_forms): Static function becomes external. + (expand_form): Remove case which handles do_s. + + * eval.h (expand_forms): Declared. + + * parser.y (elem): Expand both do_s and require_s forms by + using expand_forms. + +2014-01-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (conses, lazy_conses, func_set_env): New functions. + (lazy_conses_func): New static function. + + * lib.h (conses, lazy_conses, func_set_env): Declared. + + * eval.c (eval_init): conses, lazy_conses and func_set_env registered + as intrinsics. + + * txr.1: Documented. + +2014-01-22 Kaz Kylheku <kaz@kylheku.com> + + Changes to the list collection mechanism to improve + the extension of list operations over vectors and strings. + + * eval.c (do_eval_args, bindings_helper, op_each, + subst_vars, supplement_op_syms, mapcarv, mappendv): Switch from + list_collect_* macros to functions. + + * lib.c (copy_list): Switch from list_collect* macros to functions. + Use list_collect_nconc for the final terminator. Doing a copy + there with list_collect_append was actually wasteful, and now + that list_collect_append calls copy_list in places, it triggered + runaway recursion. + (make_like): Bugfix: list_vector was used instead of vector_list. + (to_seq, list_collect, list_collect_nconc, list_collect_append): New + functions. + (append2, appendv, nappend2, sub_list, replace_list, ldiff, remq, + remql, remqual, remove_if, keep_if, proper_plist_to_alist, + improper_plist_to_alist, split_str, split_str_set, tok_str, + list_str, chain, andf, orf, lis_vector, mapcar, mapcon, mappend, + merge, set_diff, env): Switch from list_collect* macros to functions. + (replace_str, replace_vec): Allow single item replacement sequence. + + * lib.h (to_seq): Declared. + (list_collect, list_collect_nconc, list_collect_append): Macros + removed, replaced by function declarations of the same name. + These functions return the new ptail since they cannot assign + to it, requiring all uses to be updated to do the assignment + of the returned value. + (list_collect_decl): Use val rather than obj_t *. + + * match.c (vars_to_bindings, h_coll, subst_vars, extract_vars, + extract_bindings, do_output_line, do_output, v_gather, v_collect): + Switch from list_collect* macros to functions. + + * parser.y (o_elems_transform): Likewise. + + * regex.c (dv_compile_regex, regsub): Likewise. + + * txr.c (txr_main): Likewise. + +2014-01-22 Kaz Kylheku <kaz@kylheku.com> + + Support function versions of if, and, or so that partial + evaluation like (op or @1 42) or (op if (eq @1 foo) bar xyzzy) + is possible. + + * eval.c (do_eval): Change precedence between operator and + function lookup to favor operators. This is important since + there are several operators now which are also functions. + (if_fun, or_fun, and_fun): New static functions. + (eval_init): New functions registered as intrinsics. + + * txr.1: Documented that if, and, or exist as both functions + and operators. + +2014-01-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_instrinsic): Changed to external linkage. + + * eval.h (eval_intrinsic): Declared. + + * txr.c (spec_file): Global variable removed. + (txr_main): Support for -e and -p options. Minor code cleanup. + + * txr.1: Documented new options. + +2014-01-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (mapcarv, mappendv): Convert output to type of leftmost + input sequence. + + * lib.c (make_like): New function. + (reverse, remq, remql, remqual, remove_if, keep_if, mapcar, mapcon, + mappend, set_diff): Convert output list to type of leftmost input + sequence. + + * lib.h (make_like): Declared. + + * txr.1: Documented. + +2014-01-21 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (car, cdr, ldiff): Extend to handle vectors and strings. + Thereby, numerous previously list-only operations in TXR Lisp + now magically handle strings and vectors. + + * txr.1: Documented. + +2014-01-20 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (broken_down_time_list): New static function. + (time_fields_local, time_fields_utc): New functions. + + * lib.h (time_fields_local, time_fields_utc): Declared. + + * eval.c (eval_init): Intern time_fields_local and time_fields_utc + as intrinsic functions. + + * txr.1: Documented. + +2014-01-16 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (group_by): New function. + + * hash.h (group_by): Declared. + + * eval.c (eval_init): group_by registered as group-by intrinsic. + + * txr.1: Documented. + +2014-01-16 Kaz Kylheku <kaz@kylheku.com> + + Version 75 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim: Updated. + +2014-01-15 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print, obj_pprint): Convert the internal representation + of @sym and @(expr ...) syntax back to these notations instead + of leaving them as (sys:var sym) and (sys:expr expr ...). + +2014-01-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (append_each_s, append_each_star_s): New symbol variables. + (op_each, expand): Support append-each and append-each*. + (eval_init): Initialize new symbol variables, and register + new operators. + + * txr.1: Document append-each and append-each*. + +2014-01-14 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Wrote missing documentation for throw, throwf and error. + +2014-01-14 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stdio_seek): After a successful seek, discard the character + that may have been put back with unget_char. + +2014-01-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bind_args): Implemented default values for optional + arguments. + (op_defun): Permit the syntax. + + * txr.1: Documented. + +2014-01-13 Kaz Kylheku <kaz@kylheku.com> + + Support for pushing back bytes and characters into streams. + + * stream.c (null_ops, stdio_ops, tail_ops, pipe_ops, + string_in_ops, byte_in_ops, string_out_ops, strlist_out_ops, + dir_ops, cat_stream_ops): Structure definition updated + with new initializers for two new virtuals. + (stdio_handle): New member, unget_c. + (snarf_line, stdio_get_char): Handle pushed-back character in + h->unget_c. + (stdio_unget_char, stdio_unget_byte, string_in_unget_char, + byte_in_unget_byte): New static functions. + (make_stdio_stream_common): Initialize unget_c member. + (unget_char, unget_byte): New functions. + + * stream.h (struct strm_ops): New virtuals: unget_char and unget_byte. + (unget_char, unget_byte): New functions declared. + + * syslog.c (syslog_strm_ops): Two new initializers. + + * eval.c (eval_init): Registered unget_char and unget_byte as + intrinsics. + + * txr.1: Documented. + +2014-01-13 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (put_byte): Bugfix: was checking whether the put_char + virtual function is not null, rather than put_byte. + +2014-01-13 Kaz Kylheku <kaz@kylheku.com> + + Bugfix in regex char ranges affecting ranges whose upper end + corresponds to the high bit of a bitmap cell: for instance + the character \x7f when the cell size is 32 bits. + + * regex.c (BITCELL_ALL1): Unused macro removed. + (BITCELL_BIT): New macro to replace occurrences of a repeated + expression. + (CHAR_SET_INDEX, CHAR_SET_BIT): Updated to use BITCELL_BIT. + (L0_fill_range): Bugfix: the mask1 calculation was producing all-zero + in the condition bt1 == BITCELL_BIT; it should produce an all-ones + mask. + +2014-01-13 Kaz Kylheku <kaz@kylheku.com> + + Version 74 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES, txr.vim, dep.mk: Updated. + +2014-01-13 Kaz Kylheku <kaz@kylheku.com> + + * debug.h (debug_check): Fix broken build when debugger is disabled: + the debug-disabled stub function needs six arguments. + +2014-01-12 Kaz Kylheku <kaz@kylheku.com> + + Some bignum-related optimizations + + * arith.c (plus, minus, mul, trunc): Optimize cases + where bignum is opposite 1 or 0. + +2014-01-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (rising_product): New static function. + (n_choose_k, n_perm_k): New functions. + + * arith.h (n_choose_k, n_perm_k): Declared. + + * eval.c (eval_init): New functions interned. + + * txr.1: Documented. + +2014-01-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (to_float): Print function name as ~a rather than ~s, + so it doesn't have quotes around it. + (cum_norm_dist): New function. + + * arith.h (cum_norm_dist): Declared. + + * eval.c: Include arith.h. + (eval_init): Register cum_norm_dist as intrinsic. + + * txr.1: Documented cum-norm-dist. + +2014-01-10 Kaz Kylheku <kaz@kylheku.com> + + * configure: Detect platforms which don't reveal declarations + in C headers which are extensions to standard C, like popen or fileno, + in response to standard feature selection macros like -D_POSIX_SOURCE. + MinGW and Cygwin are offenders. These platforms hide the declarations + when gcc is in -ansi mode, by testing for __STRICT_ANSI__. Turns out, + however, that -U__STRICT_ANSI__ on the gcc command line strips this + away, causing the declarations to be revealed. + + * lib.c, parser.l, stream.c, utf8.c: Removed the declarations which + compensated for the above problem. Yippee! Fuck you, stupid Cygwin + troglodytes, and the MinGW horse you rode in on. + http://cygwin.com/ml/cygwin/2011-10/msg00131.html + +2014-01-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Renaming lisp-parse to read; lisp-parse + is retained as an obsolescent synonym. All arguments become optional. + + * parser.l (lisp_parse): Handle nil source. + + * txr.1: Updated. + +2014-01-10 Kaz Kylheku <kaz@kylheku.com> + + * configure (lang_flags): Let us switch from using _XOPEN_SOURCE=2 + to _POSIX_C_SOURCE=199309L and _BSD_SOURCE. + In the .exe suffix test, try harder to redirect ls's output. + Test for fork stuff now needs <sys/types.h> evidently; + <unistd.h> does not necessarily pid_t, even though it defines + functions that return and accept pid_t! + The timegm test no longer needs _SVID_SOURCE since _BSD_SOURCE + covers it, and so we end up not using _SVID_SOURCE any more. + New tests for usleep and nanosleep. + The test for setenv is not conditional on timegm failing. + + * eval.c: Include <time.h> and conditionally <windows.h>. + (usleep_wrap): New function. + (eval_init): New usleep_wrap function registered as usleep intrinsic. + + * lib.c: Only define setenv and unsetenv if we don't have timegm, + because only in that situation they are used. + +014-01-10 Kaz Kylheku <kaz@kylheku.com> + + * match.c (do_txeval): Lift an annoying restriction in the pattern + language's expression evaluator. Whereas TXR Lisp expressions can be + used int the pattern language, preceded by @, it was not possible + to evaluate TXR Lisp variables this way. So instead of @var, + some clumsy trick had to be used like @(identity var). This is not + necessary any more. Code like @(next @*stdin*) will now work. + + * txr.1: Updated. + +2014-01-10 Kaz Kylheku <kaz@kylheku.com> + + * configure: Generate HAVE_VALGRIND as #define-d to 1, rathern + than just fdefined. + + * gc.c: Use #if HAVE_VALGRIND instaed of #ifdef HAVE_VALGRIND, + consistently with other HAVE_* config variables. + + * lib.c: Likewise. + + * lib.h: Likewise. + + * txr.c: Likewise. + + * txr.h: Likewise. + +2014-01-09 Kaz Kylheku <kaz@kylheku.com> + + Fix broken Json parsing test case on MinGW. + + * stream.c (vformat): Fix incorrect code for normalizing + exponent fields in the output of sprintf's floating-point + conversion. For ~e, the logic would let through a leading zero. + For ~a/~s conversion, it was comparing to 0 instead of '0', + not squashing leading zeros at all. + +2014-01-09 Kaz Kylheku <kaz@kylheku.com> + + * stream.c: Eliminate useless #else section containing #error + +2013-01-09 Kaz Kylheku <kaz@kylheku.com> + + Long overdue MinGW port maintenance. + + * Makefile: Use new EXE variable from config.mk. + + * configure (exe, have_windows_h): New variables. + Handle situations with .exe suffix; on MiGW, the rm command + doesn't work on executables if the .exe suffix is not given. + New tests for localtime_r and gmtime_r. + + * lib.c: Supply declarations which are missing on MinGW because + we use gcc -ansi, because MinGW doesn't follow established conventions + like -D_POSIX_SOURCE. Supply definitions for gmtime_r, localtime_r, + setenv and unsetenv. + + * parser.l: Supply declarations which are missing on MinGW. + + * signal.h (async_sig_enabled): Declare differently based on + HAVE_POSIX_SIGS. + Misspelled typedef fixed in the code for !HAVE_POSIX_SIGS + that has hitherto not been compiled. + (sig_mask): Wrap declaration in #ifdef HAVE_POSIX_SIGS because + it relies on sigset_t. + + * stream.c: Supply declarations which are missing on MinGW. + Include <windows.h> if we have it. + (sleep): Define for Windows. + (statf): Handle missing st_blksize and st_blocks members in struct + stat. + (stream_init): Handle numerous missing S_* macros. + + * utf8.c: Supply declarations which are missing on MinGW. + +2013-01-08 Kaz Kylheku <kaz@kylheku.com> + + Version 73 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2014-01-07 Kaz Kylheku <kaz@kylheku.com> + + The lisp-parse function can now be called multiple times + on the same stream to extract multiple objects; the requirement + that the stream must hold exactly one complete Lisp object + with no following material is now lifted. + + * parser.l (YY_INPUT): Modified the macro so that it reads no more + than one character. Though this probably makes the lexer less + efficient, it gives us the important property that the lexer does + not scan ahead into the input stream, hogging data into its buffer + which is then destroyed. This is essential if the lisp-parse function + is to support multiple calls to pull objects one by one out of + a stream. + + * parser.y (spec): Use YYACCEPT in the SECRET_ESCAPE_E clause for + pulling a single expression out of the token stream. YYACCEPT + is a trick for not invoking the $accept : spec . $end production + which is implicitly built into the grammar, and which causes + a token of lookahead to occur. This allows us to read a full + expression without stealing any further token: but only if the + grammar is structured right. + (exprs): This phrase structure now handles the DOTDOT syntax. + There is no such thing as an expr DOTDOT expr expression any more; + it is in the list syntax (and not supported in the dot position). + (expr): Remove DOTDOT syntax. + + * txr.1: Updated description of .. syntax, and relaxed the description + of lisp-parse since it now allows multiple calls to extract + multiple objects. + +2014-01-07 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Call yylex_destroy after yyparse, so we don't + have stray scanner state possibly interfering with a subsquent + parse job. + +2014-01-06 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Add forgotten until and last keywords. + + * txr.vim: Regenerated. + +2013-12-25 Kaz Kylheku <kaz@kylheku.com> + + Merry C++mas! + + * configure: Avoid using "new" as identifier in signals test case. + + * signal.c (set_sig_handler): Fix warning about { 0 } initializer. + (sig_mask): Avoid using "new" as identifier. + + +2013-12-20 Kaz Kylheku <kaz@kylheku.com> + + Fixing a bug and performance issue. + + Problem: we are using sigsetjmp but with a jmp_buf structure; + it requires sigjmp_buf! + + Performance issue: sigsetjmp is a dog which makes system calls. + Solution: let's roll our own cached version of sigprocmask which + only calls the real sigprocmask when the mask changes. Then + our extended_setjmp will just use regular setjmp, plus our own + custom signal saving and restoring based on the cached version. + + * signal.c (sig_blocked_cache): New variable. + (set_sig_handler): Use our sig_mask instead of sigprocmask. + (mem_set_bits, mem_clr_bits): New static functions. + (sig_mask): New function. + + * signal.h (extended_jmp_buf): New member, blocked. + (extended_setjmp): save blocked signals by peeking into + sig_blocked_cache, and restore using sig_mask. + (sig_blocked_cache, sig_mask): Declared. + +2013-12-17 Kaz Kylheku <kaz@kylheku.com> + + Version 72 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-12-17 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stdio_seek): Bugfix. Was returning current position + whenever offset is zero, regardless of whence argument. + +2013-12-17 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (yybadtoken): Handle HASH_H. + +2013-12-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_defvar): Fix the semantics to be similar to Common Lisp: + no effect if the variable already exists. + + * txr.1: Documented defvar and lisp-parse. + +2013-12-16 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (intern): fix the previous diagnostic bug once more with more + feeling. + + * parser.l (grammar): Recognize package prefixes in symbol tokens. + Got rid of special rule for handling lone colon. + + * parser.y (sym_helper): Catch undefined package as a parsing + error rather allowing intern function to throw exception. + +2013-12-15 Kaz Kylheku <kaz@kylheku.com> + + Changing the tokenizer to get rid of IDENT, KEYWORD and METAVAR + token categories, replaced by a single one called SYMTOK. + Package prefixes are now recognized and processed in tokens. + + * lib.c (delete_package): Fix problem in no-such-package + error case: it would always report nil as the name. + (intern): Fix nonsensical error message: in the no-such-package case it + would report that the symbol exists already. + + * parser.l (grammar): Occurences of KEYWORD, METAVAR, and IDENT + scrubbed. All rules reporting any of these now return + SYMTOK. The main one of these is greatly simplified. + + * parser.y (sym_helper): New function. + (char_from_name): const qualifier inside param's type declaration. + (grammar): IDENT, KEYWORD and METAVAR tokens are gone. + New token SYMTOK. Grammar refactored around SYMTOK and using + the new sym_helper function. + (char_from_name): Updated. + +2013-12-14 Kaz Kylheku <kaz@kylheku.com> + + Support for parsing Lisp expression out of strings and streams. + New catenated streams make the Yacc hack possible. + + * eval.c (eval_init): Register lisp_parse as intrinsic. + + * parser.h (lisp_parse): Declared. + + * parser.l: New lexical hack to produce SECRET_ESCAPE_E token. + (regex_parse): Move declaration before statements. + (lisp_parse): New function. + + * parser.y (SECRET_ESCAPE_E): New token type. + (spec): New production rule for single expression. + + * stream.c (cat_stream_print, cat_get_line, cat_get_char, + cat_get_byte, cat_get_prop): New static functions. + (cat_stream_ops): New static function. + (make_catenated_stream): New function. + + * stream.h (make_catenated_stream): Declared. + +2013-12-13 Kaz Kylheku <kaz@kylheku.com> + + New stream property: name. Some streams can report + their name: some kind of string. + + * stream.c (name_k): New variable. + (null_get_prop): New static function. + (null_ops): Wire null_get_prop into ops structure. + (stdio_get_prop): Report h->descr as name. + (string_in_get_prop): New function. + (string_in_ops): Wire string_in_get_prop into ops structure. + (stream_init): Initialize name_k. + + * stream.h (name_k): Declared. + + * syslog.c (syslog_get_prop): Report "syslog" as + stream name. + +2013-12-13 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (sig_handler): Pass two arguments to signal + handler rather than 1. The new argument is the value t, + telling the handler that it's asynchronously invoked in + a real signal context. If the signal handler returns + true, then mark it deferred so that it can re-trigger + in a synchronous context. + (sig_check): Pass nil as a the second argument to signal handlers, + indicating that they are synchronously called, not + in a signal handler context. + + * txr.1: Added documentation. + +2013-12-13 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (set_sig_handler): Disable signals around this + function because it manipulates the mask of deferred + signals which can also be manipulated by an async signal. + +2013-12-13 Kaz Kylheku <kaz@kylheku.com> + + * signal.c (set_sig_handler): Clear the deferred flag + for a signal that is being set to ignored or default. + + * txr.1: Documented signals. + +2013-12-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered vars for signal numbers. + + * signal.c (sig_hup, sig_int, sig_quit, sig_ill, sig_trap, sig_abrt, + sig_bus, val sig_fpe, sig_kill, sig_usr1, sig_segv, sig_usr2, sig_pipe, + sig_alrm, val sig_term, sig_chld, sig_cont, sig_stop, sig_tstp, + sig_ttin, val sig_ttou, sig_urg, sig_xcpu, sig_xfsz, sigtalrm, + sig_prof, val sig_poll, sig_sys, sig_winch, sig_iot, sig_stkflt, + sig_io, sig_lost, sig_pwr): New variables. + (sig_init): New variables initialized. + + * signal.h: New variables declared. + +2013-12-12 Kaz Kylheku <kaz@kylheku.com> + + First cut at signal handling support. + + * Makefile (OBJS-y): Include signal.o if have_posix_sigs is "y". + + * configure (have_posix_sigs): New variable, set by detecting POSIX + signal stuff. + + * dep.mk: Regenerated. + + * arith.c, debug.c, eval.c, filter.c, hash.c, match.c, parser.y, + parser.l, rand.c, regex.c, syslog.c, txr.c, utf8.c: Include new + signal.h header, now required by unwind, and the <signal.h> system + header. + + * eval.c (exit_wrap): New function. + (eval_init): New functions registered as intrinsics: exit_wrap, + set_sig_handler, get_sig_handler, sig_check. + + * gc.c (release): Unused functions removed. + + * gc.h (release): Declaration removed. + + * lib.c (init): Call sig_init. + + * stream.c (set_putc, se_getc, se_fflush): New static functions. + (stdio_put_char_callback, stdio_get_char_callback, stdio_put_byte, + stdio_flush, stdio_get_byte): Use new functions to enable + signals when blocked on I/O. + (tail_strategy): Allow signals across sleep. + (pipev_close): Allow signals across waitpid. + (se_pclose): New static function. + (pipe_close): Use new function to enable signals across pclose. + + * unwind.c (uw_unwind_to_exit_point): use extended_longjmp instead of + longjmp. + + * unwind.h (struct uw_block, struct uw_catch): jb member changes from + jmp_buf to extended_jmp_buf. + (uw_block_begin, uw_simple_catch_begin, uw_catch_begin): Use + extended_setjmp instead of setjmp. + + * signal.c: New file. + + * signal.h: New file. + +2013-12-12 Kaz Kylheku <kaz@kylheku.com> + + * configure (config_flags): Variable removed. This was more trouble than it + was worth, and only solved problems caused by not checking that the test + program was made all the way to an executable. + +2013-12-12 Kaz Kylheku <kaz@kylheku.com> + + * configure: Fix warnings in test cases for HAVE_FORK_STUFF + and HAVE_DAEMON. + + * eval.c: daemon support must be wrapped in #if HAVE_DAEMON + +2013-12-11 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (txr_main): Change stderr to line buffered mode. + +2013-12-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): log_authpriv_v registered as log-authpriv. + + * syslog.c (log_authpriv_v): New variable. + (syslog_init): New variable initialized. + + * syslog.h (log_authpriv_v): Declared. + + * txr.1: Documented log-authpriv, and put in notes about testing + for variables that may not be present, including log-perror. + +2013-12-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (symbol_value, boundp, fboundp): New functions. + (eval_init): New functions registred as intrinsics. + + * txr.1: Documented. + +2013-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c, hash.c, lib.c, rand.c, stream.c, syslog.c: Removing + #include <assert.h> since none of these modules uses the standard C + assert macro. + +2013-12-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): closelog_wrap interned. + + * syslog.c (closelog_wrap): New function. + + * syslog.h (closelog_wrap): Declared. + + * txr.1: Documented. + +2013-12-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register streamp, real_time_stream_p, + stream_get_prop and stream_set_prop as intrinsics. + + * stream.c (stream_get_prop): New function. + + * stream.h (stream_get_prop): Declared. + + * syslog.c (syslog_get_prop): Bugfix: check for prio_k not real_time_k. + + * txr.1: Updated. + +2013-12-10 Kaz Kylheku <kaz@kylheku.com> + + syslog support; bitwise logior and logand functions become variadic. + + * Makefile: Use -iquote to restrict our #include search paths from + being processed for #include <...>. Add syslog.o to OBJS-y if + have_syslog is y. + + * configure (have_syslog): New variable, set by detecting syslog API. + + * eval.c (eval_init): logand and logior registrations changed to + go to variadic versions. New syslog variables and functions registered. + + * lib.c (logandv, logiorv): New functions. + + * lib.h (logandv, logiorv): Declared. + + * txr.c (main): Call syslog_init. + + * syslog.c: New file. + + * syslog.h: New file. + + * txr.1: Updated. + + * txr.vim: Regenerated. + +2013-12-07 Kaz Kylheku <kaz@kylheku.com> + + Version 71 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-12-06 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (end_of_regex): Fix start condition stack underflow + during regex_parse. + +2013-12-06 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (regex_parse): Match the simplified grammar now + expected in the parser, with no slashes around the regex. + Change prefix for diagnostic message. + +2013-12-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Update registration of regex-compile + to reflect that it has two arguments now. + + * parser.y (grammar): Update calls to regex_compile to + pass two arguments. Since we don't expect regex_compile to + parse, we specify the error stream as nil. + (spec): The "secret syntax" for a regex is simplified + not to include the slashes. This provides better diagnostics for + unterminated syntax and requires less string processing to generate. + Also, the form returned doesn't have the regex symbol + consed onto it, which parse_regex just throws away. + + * regex.c (regex_compile): Now takes a stream argument. + + * regex.h (regex_compile): Declaration updated. + + * txr.1: Updated + +2013-12-06 Kaz Kylheku <kaz@kylheku.com> + + Fixing some old-style coding that became obsolete + around November 2009. + + * lib.c (lazy_str): Use the efficient lit("...") that + doesn't allocate memory instead of string(L"..."). + (lazy_str_get_trailing_list): Likewise. + + * stream.c (open_process): Likewise. + + * txr.c (remove_hash_bang_line): Likewise. + +2013-12-05 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (regex_compile): Handle string input. + + * regex.h (regex_compile): Don't call argument + regex_sexp, since it can be a string. + + * txr.1: Updated. + +2013-12-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered regex_parse as new + intrinsic function and std_null as new variable. + + * parser.h (yylex_destroy): Existing function declared. + + * parser.l (regex_parse): New function. + New lexical syntax added which returns SECRET_ESCAPE_R. + + * parser.y (SECRET_ESCAPE_R): New token. + (spec): Added syntactic variant which lets us + smuggle a regex into the parser easily. + + * stream.c:x (std_null): New global variable. + (null_stream_print): New static function. + (null_ops): New static structure. + (make_null_stream): New function. + (stream_init): Protect and initialize std_null. + + * stream.h (std_null, make_null_stream): Declared. + + * txr.1: New features documented: regex-parse, *stdnull*. + + * txr.c (txr_main): Call yylex_destroy after parsing the program now + that I know about this function; this can free up some memory. + +2013-12-02 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stdio_set_prop): Fix reversed boolean. + +2013-12-02 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (tail_strategy): Execute the strategy code also + in the case that the stream's FILE * handle is null. This handles + opening the file for the first time. + (make_stdio_stream_common): Do not use the FILE * handle if it is null. + (open_tail): Do not open the file immediately and error out. + This is undesirable because log files might not exist at the time + open_tail is called on them. Instead, produce a stream which contains + a null file handle, and use tail_strategy to poll for the file + to come into existence. + + * utf8.c (w_freopen): Just in case freopen doesn't like a null + pointer for the existing stream, use fopen instead if that is the case. + +2013-12-02 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Missing catch and finally keywords added. + + * txr.vim: Regenerated. + +2013-12-02 Kaz Kylheku <kaz@kylheku.com> + + debug_check calls are the culprit triggering lookahead + in lazy lists. Changing the semantics of the data argument + so that it can just be the list. + + * debug.c (debug): If data is a cons, then replace it with + the first item. + + * match.c (match_files): Pass c.data to debug_check, rather + than the value of if2(consp(c.data), car(c.data)) which accesses + car(c.data) whether or not we are debugging. + (match_fun): Likewise. + +2013-12-01 Kaz Kylheku <kaz@kylheku.com> + + Steps toward fixing an issue: lazy list readahead. + The problem is that accurate lazy lists are not suitable for + real time use, where we want the TXR program to respond immediately + to matching some datum. + + I'm implementing a simple, naive variant of lazy stream lists + which simply populates the lazy cons by reading from the stream when + the car or cdr fields are accessed. This type of stream can never + be nil (empty list) even if the file is empty; in that case + it will be (nil) and in general, it will have a spurious nil + item at the end instead of ending in a string. + (An adjustment was made in match.c to detect this; more + will be needed.) + + I'm adding attributes to streams so streams can now have a + "real-time" attribute. When a lazy string list is constructed over + a real-time stream, the simple implementation is used. + File streams are automatically real-time if (on Unix) they are tied + to tty streams. Tail streams are also real-time. + + More work is needed to achieve the goal of this change, + but this is a big step in the right direction. + + * configure: Detect isatty function. + + * lib.c (simple_lazy_stream_func): New static function. + (lazy_stream_cons): Use simple implementation for real-time streams. + + * match.c (match_files): Do not call match_line_completely + with a data line that is nil (as a result of simple lazy list + over a real-time stream). A nil item in a lazy list of strings + is treated as eof. + + * stream.c (real_time_k): New symbol variable. + (struct strm_ops): New members: get_prop, set_prop. + (struct stdio_handle): New member: is_real_time. + (stdio_get_prop, stdio_set_prop): New static function. + (stdio_ops, tail_ops, pipe_ops): stdio_get_prop and + stdio_set_prop funtions wired in. + (make_stdio_stream_common): Attribute streams as real-time + if they are tty devices. + (make_tail_stream): Tail streams are real-time attributed. + (stream_set_prop, real_time_stream_p): New functions. + (stream_init): Initialize real_time_k. + + * stream.h (real_time_k): Declared. + (real_time_stream_p, stream_set_prop): Likewise. + +2013-11-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions countqual, countql, countq + and count_if registered as intrinsics. + + * lib.c (countqual, countql, countq, count_if): New functions. + + * lib.h (countqual, countql, countq, count_if): Declared. + + * txr.1: New functions documented. + +2013-11-29 Kaz Kylheku <kaz@kylheku.com> + + * configure (config_flags): New variable, allowing us to + have stricter diagnosis for configure tests. + (have_timegm, need_svid_source, need_bsd_source): New + variables. sys/stat.h test only declares static data and + compiles an object file. Adding tests for timegm, tzset, + setenv and unsetenv. + + * eval.c (eval_init): Register new intrinsic, make_time_utc. + + * lib.c (make_time_impl): New static function. + (make_time): Reimplemented as call to make_time_impl. + (timegm_hack): New conditionally-defined static function. + (make_time_utc): New function. + + * lib.h (make_time_utc): Declared. + + * txr.1: make-time-utc documented. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (make_time): We must subtract from the 1-12 + month for the tm_mon member of struct tm; we + were adding 1 instead. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (tail_calc): New function + (tail_strategy): Handle the situation when the file disappears. We + cannot throw an error, but must poll the filesystem + for the file to reappear. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * configure: Detect the daemon function. + + * eval.c: Include <unistd.h> conditionally. + (errno_wrap, daemon_wrap): New functions. + (eval_init): Registered errno_wrap and daemon_wrap as intrinsics + under the names daemon and errno. + + * txr.1: Documented errno and daemon in new UNIX PROGRAMMING + section. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented open-tail and make-time. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + Extending intrinsic functions to go up to 7 arguments. + Adding wrapper for mktime. + + * eval.c (apply): Handle function codes N5 through N7. + (eval_init): Register make_time as intrinsic. + + * lib.c (auto_k): New keyword symbol variable. + (equal, generic_funcall): Handle N5-N7. + (func_n5, func_n6, func_n7, func_n5v, func_n6v, func_n7v): New + functions. + (obj_init): Initialize auto_k. + (make_time): New function. + + * lib.h (functype_t): New enum members: N5, N6, N7. + (struct func): New members: n5, n6, n7, n5v, n6v, n7v. + (auto_k, func_n5, func_n6, func_n7, func_n5v, func_n6v, func_n7v, + make_time): Declared. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (make_stdio_stream_common): Initialize new + mode member of struct stdio_ops to nil. + +2013-11-28 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (struct stdio_handle): New member, mode. + (stdio_stream_mark): Mark the new member during gc. + (stdio_seek): When we seek, we should reset the utf8 machine. + (tail_strategy): New function. + (tail_get_line, tail_get_char, tail_get_byte): Use + tail_strategy for polling the file at EOF. + (open_tail): Store the mode in the file handle. + + * utf8.c (w_freopen): New function. + + * utf8.h (w_freopen): Declared. + +2013-11-27 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (conftest.clean): Use @ to suppress output. + + * configure (have_unistd): New variable. Set true by every successful + test that compiles something that contains #include <unistd.h>. + HAVE_UNISTD_H is conditionally generated in config.h based on this variable. + Minor cleanup. + + * parser.l: Inclusion of <unistd.h> wrapped in #if/#endif. + + * stream.c: Conditional inclusion of <unistd.h> based on new HAVE_UNISTD_H symbol. + +2013-11-27 Kaz Kylheku <kaz@kylheku.com> + + * configure: Added check to detect POSIX sleep function. + + * eval.c (eval_init): Register new open_tail function as intrinsic. + + * match.c (complex_snarf, complex_stream): Update calls to + make_stdio_stream and make_pipe_stream to take fewer arguments. + (match_files): Support a stream object as a data source specification + in place of a string. + + * parser.l (parse_reset): Update call to make_stdio_stream to take + fewer arguments. + + * stream.c: Inclusion of <unistd.h> made properly conditional. + (struct stdio_handle): pid member defined as pid_t only if we have fork + functionality, otherwise defined as int. + (tail_get_line, tail_get_char, tail_get_byte): New static functions. + (tail_ops): New static structure. + (make_stdio_stream_common): New static structure. + (make_stdio_stream, make_pipe_stream): These functions lose the input + and output parameters, which ended up never used. Reimplemented + in terms of new common function. + (make_tail_stream): New function. + (make_pipevp_stream): Reimplemented in terms of new common function. + (open_file, open_command): Simplified by removal of useless local + variables and their computation, which used to be extra arguments to + make_stdio_stream and make_pipe_stream. + (open_tail): New function. + (stream_init): Calls to make_stdio_stream updated. + + * stream.h (make_stdio_stream, make_pipe_stream): Declarations updated. + (make_tail_stream, open_tail): Declared. + + * txr.c (txr_main): Calls to make_stdio_stream updated. + +2013-11-26 Kaz Kylheku <kaz@kylheku.com> + + Stream-seeking functionality. + + Bugfix in stdio_flush: check FILE * handle for null. + + Minor cleanups. + + * eval.c (eval_init): Register seek_stream as intrinsic. + + * stream.c (from_start_k, from_current_k, from_end_k): New symbol + variables. + (strm_whence): New enum. + (strm_ops): New member, seek. + (stdio_maybe_write_error): Renamed to stdio_maybe_error; takes + new string argument to describe action. + (stdio_put_string, stdio_put_char, stdio_put_byte): Updated call to + stdio_maybe_error. + (stdio_flush): Updated call to stdio_maybe_error. Check + handle for null. + (stdio_seek): New static function. + (stdio_ops): Added stdio_seek. + (pipe_ops, string_in_ops, byte_in_ops, string_out_ops, + strlist_out_ops, dir_ops): Added explicit zero entries and comments for + unimplemented functions. + (seek_stream): New function. + (stream_init): New keyword symbols interned. + + * stream.h (from_start_k, from_current_k, from_end_k): New + variables declared. + (dev_k, ino_k, mode_k, nlink_k, uid_k, gid_k, rdev_k, size_k, + blksize_k, blocks_k, atime_k, mtime_k, ctime_k, s_ifmt, s_iflnk, + s_ifreg, s_ifblk, s_ifdir, s_ifchr, s_ififo, s_isuid, s_isgid, s_isvtx, + s_irwxu, s_irusr, s_iwusr, s_ixusr, s_irwxg, s_irgrp, + s_iwgrp): Existing extern variables declared. + (seek_stream): New function declared. + + * txr.1: Documented seek-stream. + +2013-11-24 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Bugfix: bindings were propagated in only one of two + subcases of a successful load. This bug means that a loaded TXR + sub-query was not reliably able to bind variables that are then visible + to subsquent directives in the parent. + +2013-11-24 Kaz Kylheku <kaz@kylheku.com> + + Fix for uninitialized struct member curfiles in the match context. + This is used for reporting the current match location in debug traces. + + * match.c (mf_all): Takes new argument curfiles + and initializes the match_files_ctx member of the same name. + (do_match_line, mf_from_ml, match_filter, match_fun, extract): + Pass curfile to ml_all. + +2013-11-22 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Change how the hard-coded symbols (end, and, or) are + added, and also add the missing rep to these. + + * txr.vim: Regenerated. + +2013-11-21 Kaz Kylheku <kaz@kylheku.com> + + Version 70 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-11-21 Kaz Kylheku <kaz@kylheku.com> + + Nasty bug fixed: @(accept) from inside a @(collect) was found not to + propagate bindings. The culprit? The bindings_coll variable in the + v_collect function being indeterminate by the well-documented and + understood action of setjmp. Marking the bindings_coll variable + volatile instantly fixed it. I reviewed this code to find any other + instance of this oversight. + + * match.c (v_skip, v_collect): Mark some local variable volatile: + precisely those which are used after possibly returning via an + unwind, and which might have been modified since setting up + the unwind block. + +2013-11-05 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Handle symbols whose C names end with _star_s, + and also ones with underscores. + + * txr.vim: Updated. + +2013-10-28 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (clean): depends on conftest.clean + (conftest.clean): New target. + + * configure: Use "make conftest.clean" to clean up conftest stuff. Note + that there was a bug here: brace expansion was used, but the shell is + /bin/sh, so conftest.err wasn't being removed. + +2013-10-27 Kaz Kylheku <kaz@kylheku.com> + + Fix 2013-10-06 regression in op syntax. This happens when a @var + syntax appears in an op form, crashing the expander with infinite + recursion at program load time, before the code is run. + + * eval.c (transform_op): The fi == var check to handle a + trailing op variable in the dot position of a form + needs to ensure that it really is @<num> type var, and + not the @<sym> type for instance. + +2013-10-27 Kaz Kylheku <kaz@kylheku.com> + + Fix jaw-dropping bug introduced on 2009-11-15, first affecting + the 022 release. + + Efficient string literal objects (tag bits 11) were introduced on that + day. + + The null_string object was changed to a literal, and that object is + efficiently returned by some string manipulation functions in cases + when it is obvious that the result is an empty string. + + However, the LIT type code was not added to a switch statement in + the variable matching code, which continued to recognize only the + type STR as a string, to this day. + + A test case for this is + + echo ":x" | ./txr -c "@x:@x" - + + which results in an "unsupported object in spec" error. + + * match.c (h_var): Use stringp to test for string rather than + type(..) == STR. Add LIT and LSTR to the switch statement + so that literals an lazy strings are handled if they occur. + (do_output_line): Add LIT and LSTR to switch. + +2013-10-27 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Include syntactic directives which aren't extracted from + sources because actions aren't registered for them. + + * txr.vim: Regenerated. + +2013-10-24 Kaz Kylheku <kaz@kylheku.com> + + Ouch! Turns out the code base has numerous unintended + deviations from C90, like mixed declations and + statements. GCC doesn't diagnose these without the + --pedantic flag. + + * configure: GCC's --ansi flag should be spelled -ansi. + + * lib.c (split_str, obj_print): Reorder declaration before statements. + (make_sym): Fix similar problem by eliminating a statement. + (funcall1, funcall2, funcall3, funcall4): Use assignment to initialize + local array with non-constant elements. This is actually good for + performance because we only initialize those parts of the array that + we use. + + * lib.h (struct func): Change functype member to unsigned, + since enum-typed bitfields are a GCC extension. + + * match.c (ml_all, mf_all): Use assignments to initialize local + struct with non-constants. + (do_txeval, v_collect): Slightly revise unwinding macrology with help + of new macros to avoid mixing declarations and statements. + (spec_bind): Removed spurious semicolon from macro expansion. + (v_gather): Reorder two lines to avoid mixed decls and + statements. + (match_filter): Move declaration of ret a few lines up, ahead of + statements. + + * unwind.c (uw_pop_until): New function. + + * unwind.h (uw_pop_until): Declared. + (uw_mark_frame, uw_fast_return): New macros. + +2013-10-23 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: Split long lines of keywords. + + * txr.vim: Updated. + +2013-10-23 Kaz Kylheku <kaz@kylheku.com> + + * genvim.txr: New file. + + * txr.vim: Updated using genvim.txr. + +2013-10-23 Kaz Kylheku <kaz@kylheku.com> + + * configure: Change to header block comment generated in config.h + +2013-10-23 Kaz Kylheku <kaz@kylheku.com> + + Version 69 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-10-21 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fixed documentation bug. expt with three or more + arguments is a right-associative reduction, not left. + +2013-10-11 Kaz Kylheku <kaz@kylheku.com> + + Task #11433. Implement continuation of multiple + output blocks across the same stream. + + * match.c (close_s, named_k, continue_k, finish_k): New symbol + variables. + (v_output): Implement :named, :finish and :continue. + (v_close): New static function. + (syms_init): New symbols interned. + (dir_tables_init): New entry associating v_close + function with symbol stored in close_s. + + * match.h (close_s): Declared. + + * txr.1: New features documented. + +2013-10-11 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fix reversed arguments in documentation for time-string-local + * and time-string-utc. + +2013-10-07 Kaz Kylheku <kaz@kylheku.com> + + Version 68 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-10-06 Kaz Kylheku <kaz@kylheku.com> + + Improving behavior of op and fixing a bug. + + Explicitly specifying @rest using (op ... . @rest) + did not work at all. + + The then-func agument of iff and iffi may now be nil. + + * eval.c (format_op_arg): New static function. + (transform_op): Handle dotted lists ending in @rest + or @<num>. + (supplement_op_syms): New static function. + (expand_op): Add missing numeric arguments, + so that all 1 through n are in the list. + Trailing rest is now added under different + conditions. + + * lib.c (do_iff): Give thenfun the same + behavior on nil that elsefun enjoys. + + * txr.1: Updated. + +2013-10-06 Kaz Kylheku <kaz@kylheku.com> + + New feature: :vars argument in repeat and rep directives in an output + block, for specifying variables to include in iteration whose + presence repeat is not able to deduce. + + * match.c (extract_bindings): New argument, vars, specifies + additional variables to consider. + (do_output_line, do_output): Process :vars argument of repeat + and rep directive. + + * txr.1: Updated. + +2013-08-09 Kaz Kylheku <kaz@kylheku.com> + + * filter.c, utf8.c: Tabs changed to spaces. For some reason, filter.c + used 4 space tabs and utf8.c used 8 space tabs, inconsistently. + +2013-07-13 Kaz Kylheku <kaz@kylheku.com> + + Version 67 + + txr.1: Documented line continuation for string literals, + supported since Version 38 but not clarified properly. + + RELNOTES: Updated. + +2013-07-13 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Diagnose unrecognized escape sequences in string + literals and quasiliterals and after @\. + Support "\ " (backslash space) escape in string literals + and quasiliterals. + +2013-07-12 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fix slipup, where call is referred to as funcall. + +2013-07-12 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (version): Bumped. + + * txr.1: Bumped version, set date and documented string-cmp. + + * configure (txr_ver): Bumped. + + * eval.c (eval_init): Forgotten string-cmp registered as intrinsic. + + * txr.vim: Highlighting for string-cmp. + + * RELNOTES: Updated. + +2013-07-12 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Bringing up to date. + +2013-06-11 Kaz Kylheku <kaz@kylheku.com> + + * match.c (require_s): New variable. + (v_require): New static function. + (syms_init): Initialize require_s. + (dir_tables_init): Add new entries into v_directive_table + and h_directive_table for new require directive. + + * match.h (require_s): Declared. + + * txr.1: Added do and require directives to the directive summary + section. Documented new require directive. + Fixed int-str documentation to clarify that the radix is + optional. + +2013-06-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): tok-str acquires new parameter + + * lib.c (tok_str): New parameter, keep_sep. + + * lib.h (tok_str): Declaration updated. + + * txr.1: Documentation for tok-str updated. + +2013-06-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): lazy-str's third argument is optional. + Added lazy-stringp. Changing names of length-str-{gt,ge,lt,le} + to be consistent with the >, >=, < and <= functions. + + * lib.c (lazy_stream_func): Greatly simplified implementation. + The lazy list now continues by means of recursing via + an optimized version of lazy_stream_cons called lazy_stream_cont. + The environment structure is simplified to just hold the next item, + rather than a pointless list. The pointless setting of lcons->lc.func + to nil is also removed; this is always done by the caller. + (lazy_stream_cont): New static function, similar to lazy_stream_cons, + but optimized by not consing up a new function and new environment + cell. + (lazy_stream_cons): The environment for the update function + is simplified to just a single cons. + + * txr.1: Documented lazy string functions and lazy-stream-cons. + +2012-03-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): lazy string related functions become intrinsics. + + * lib.c (string_cmp): New function. + + * lib.h (string_cmp): Declared. + +2013-05-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register tok_str as intrinsic. + + * lib.c (tok_str): New function. + + * lib.h (tok_str): Declared. + + * txr.1: Documented. + +2013-05-20 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Corrected wrong function name in documentation + for split-str-set. + +2013-05-20 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Misleading documentation updated for split-str, which + takes a regex, not just a fixed string separator. Clarified + the corner case behaviors too. + +2013-05-16 Kaz Kylheku <kaz@kylheku.com> + + Version 66 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2013-05-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsics, time-string-local and + time-string-utc. + + * lib.c (string_time): New static function. + (time_string_local, time_string_utc): New functions. + + * lib.h (time_string_local, time_string_utc): Declared. + + * txr.1: Documented. + + * RELNOTES: Updated. + +2013-05-15 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_fun): Support debug stop on the function + prior to the call. The first data line number is 1, not zero, + if there is data. + + * txr.1: Added usage example for match-fun. + +2013-05-14 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented time and url encoding/decoding functions. + +2013-05-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): State argument in random-fixnum should be + optional. + + * txr.1: Documented random functions as well as range and range* + +2013-05-14 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: Updated in preparation for release. + + * eval.c (eval_init): Expose delete-package, rehome-sym + and packagep. + + * lib.c (make_package, intern): use ~s formatting for package name in + error message. + (packagep, delete_package, rehome_sym): New functions. + + * lib.h (packagep, delete_package, rehome_sym): Declared. + + * txr.1: Documented process functions and packages. + +2013-01-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New instrinsic function iffi registered. + + * lib.c (iff): Reversed argument names corrected. No functional + change. + (iffi): New function. + + * lib.h (iffi): Declared. + + * txr.1: Documented iffi. + +2013-01-10 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (help): Help text updated. + (debug): Fix conflicting command assignment. + Show form is now 'o'. + +2012-10-31 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (int_str): Fix gaping bug introduced by previous commit, + which could have been caught by running the regression test suite. + The revised function was returning small integers as bignums. + +2012-10-30 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (string_lt): Bugfix: wcscmp returns some value less than zero, + not specifically -1. + +2012-10-30 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/config-types: Include the header as "config.h" + rather than "../config.h" because the dependency generation script + checks for this as a special case and treats it relative to the + build directory. + + * dep.mk: Regenerated. + +2012-10-01 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (bignum_from_long): New function. + + * arith.h (bignum_from_long): Declared. + + * lib.c (int_str): Streamlined. Only use mp_read_radix in the case when + wcstol fails, because now we have bignum_from_long to handle all values + of long. Ensure that the bignum is normalized, in case it falls in the + fixnum range (does not happen on our usual platforms). + +2012-09-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c: Allow the test form of a for loop to be omitted, + defaulting to a true test, allowing an infinite loop to + be expressed as (for () () () ...). + + * txr.1: Documented. + +2012-09-24 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: Correct incorrect 2011 years on all 2012 dates. + How did that happen? + +2012-09-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (apply): Fix in error message: hard coded "apply" instead + of using ctx_form. + +2012-09-24 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: internal funcall functions not handling functions + with optional arguments. + + * lib.c (generic_funcall): New static function, based on apply + from eval.c. + (funcall, funcall1, funcall2, funcall3, funcall4): If the function + being called has optional arguments, then go through generic_funcall. + +2012-09-24 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (logand): Fix incorrect return value. + +2012-09-18 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (logtest): New function. + + * eval.c (eval_init): Registered logtest. + Registered s-ifmt, s-iflnk, s-ifreg, s-ifblk, s-ifdir, + s-ifchr, s-ififo, s-isuid, s-isgid, s-isvtx, s-irwxu, + s-irusr, s-iwusr, s-ixusr, s-irwxg, s-irgrp, s-iwgrp, + s-ixgrp, s-irwxo, s-iroth, s-iwoth, s-ixoth variables. + + * lib.h (logtest): Declared. + + * stream.c (s_ifmt, s_ifsock, s_iflnk, s_ifreg, s_ifblk, + s_ifdir, s_ifchr, s_ififo, s_isuid, s_isgid, s_isvtx, s_irwxu, s_irusr, + s_iwusr, s_ixusr, s_irwxg, s_irgrp, s_iwgrp, s_ixgrp, s_irwxo, s_iroth, + s_iwoth, s_ixoth): New global variables. + + * stream.h (s_ifmt, s_ifsock, s_iflnk, s_ifreg, s_ifblk, + s_ifdir, s_ifchr, s_ififo, s_isuid, s_isgid, s_isvtx, s_irwxu, s_irusr, + s_iwusr, s_ixusr, s_irwxg, s_irgrp, s_iwgrp, s_ixgrp, s_irwxo, s_iroth, + s_iwoth, s_ixoth): Declared. + + * txr.1: Documented logtest and s-* variables for stat, + as well as open-file and open-directory. + +2012-09-17 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Slight change in ~s directive. For a floating + point integer, .0 is now always shown if the precision is not given, + or if it is given and nonzero. Previous behavior of ~s was add .0 only + when a precision is not specified. The new behavior is more + sensible and consistent with documentation. + + * txr.1: Typo fixed in related documentation. + +2012-09-17 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (comp_trunc, logcomp): logcomp renamed to lognot, + consistent with Common Lisp. + + * eval.c (eval_init): logcomp to lognot. + + * lib.h (logcomp): Declaration updated. + +2012-09-17 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (highest_significant_bit): New static function. + (comp_clamp): Bugfix: avoid shifting left into sign bit. Function + renamed to comp_trunc. + (logtrunc, ash): New functions. + + * eval.c (eval_init): Registered logtrunc and ash intrinsics. + + * lib.h (logtrunc, ash): Declared. + + * mpi-patches/add-bitops (s_highest_bit_mp): Forward declaration for + added. + (mp_clamp_comp): Bugfix in handling remainder bits. Function + renamed to mp_trunc_comp. + (mp_trunc, mp_shift): New functions. + +2012-09-17 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (highest_significant_bit): New static function. + (comp_clamp): Bugfix: avoid shifting left into sign bit. + (logclamp, ash): New functions. + + * eval.c (eval_init): Registered logclamp and ash intrinsics. + + * lib.h (logclamp, ash): Declared. + + * mpi-patches/add-bitops (s_highest_bit_mp): Forward declaration for + added. + (mp_comp_clamp): Bugfix in handling remainder bits. + (mp_clamp, mp_shift): New functions. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/add-bitops: Bugfixes: mp_2comp is extended to properly + allow arbitrarily wide complements (which causes the code to access + beyond the a argument's digits array). A similar fix is applied in + the new mp_clamp_comp function. Incorrect initializations of mp_int + fixed in all the logic functions: mp_init was used instead of + mp_init_size. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (logand, logior, logxor): Bugfix: result needs to be + normalized, otherwise we end up with fixnum-range bignums. + (comp_clamp): New function. + (logcomp): Changed to two argument form. If second argument + is present (not nil) then call comp_clamp. + + * eval.c (eval_init): Change registration of logcomp to allow + optional argument. + + * lib.h (logcomp): Declaration updated. + + * mpi-patches/add-bitops: New mp_clamp_comp function implemented. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + Adding complementing function. + + * arith.c (logcomp): New function. + + * eval.c (eval_init): logcomp registered as intrinsic. + + * lib.h (logcomp) declared. + + * mpi-patches/add-bitops: Fixed bugs in mp_xor. Implemented mp_comp. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + Starting work on adding bit operations. The semantics is that + negative integers behave as an "infinite bit two's complement". + + * arith.c (logand, logor, logxor): New functions. + + * eval.c (eval_init): New intrinsic functions logand, logior, logxor. + + * lib.h (logand, logor, logxor): Declared. + + * mpi-patches/series: New patch, add-bitops. + + * mpi-patches/add-bitops: New file. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Fix bug in ~x format directive for printing + integers in hex. When we use the printf's %x conversion specifiers for + fixnums, we get incorrect results when the values are negative, because + the argument is actually treated as an unsigned integer. + The end result is that (format t "~x" -1) produces FFFFFFFF + rather than -1. Also, merged together mindless code duplication. + +2012-09-16 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Implemented hexadecimal integer constants. + These will be very useful since bit operations are about + to be implemented. + + * txr.1: Documented. + +2012-09-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsics: make-similar-hash, copy-hash, + hash-uni, hash-diff, hash-isec. + + * hash.c (make_similar_hash, copy_hash, hash_uni, hash_diff, + hash_isec): New functions. + + * hash.h (make_similar_hash, copy_hash, hash_uni, hash_diff, + hash_isec): Declared. + + * txr.1: Updated. + + * txr.vim: Highlighting for these new functions. + +2012-09-11 Kaz Kylheku <kaz@kylheku.com> + + * configure: Added test for <sys/stat.h> + + * eval.c: New intrinsic functions "stat" and "prop". + + * stream.c: Include <sys/stat.h> if we have it. + (w_stat, statf): New functions. + (val dev_k, ino_k, mode_k, nlink_k, uid_k, + val gid_k, rdev_k, size_k, blksize_k, blocks_k; + val atime_k, mtime_k, ctime_k): New sybol variables. + (stream_init): Intern new keywords symbols. + + * stream.h (statf): Declared. + + * txr.1: prop documented. Stub for stat created. + +2012-09-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): new instrinsic function /= registered. + + * lib.c (numneqv): New function. + + * lib.h (numneqv): Declared. + + * txr.1: New function documented. + +2012-09-10 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_collect): Bug in processing of @(last) directive. + When a match for the @(last) material occured at the end of data, + c->data was being mistakenly set to nil rather than t before breaking + out of the loop, wreaking havoc. This is not a regression; this + was there since the inception of @(last) between versions 38 and 39. + +2012-09-06 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented string library. + +2012-09-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Follow function renames. + + * hash.c (make_hash): Likewise. + + * lib.c (assq): Renamed to assql for consistency. + (aconsq_new): Renamed to aconsql_new. + (aconsq_new_l): Renamed to aconsql_new_l. + (alist_remove_test): Use equal rather than eq. Association lists + use equal equality by default. + (alist_nremove): Use memqual rather than memq. + (alist_nremove1): Use equal rather than eq. + (merge): Bugfix: unnecessary consing caused by using append + instead of nconc on list pieces that are already destructively + chopped up. This has implications for the efficiency of sort + over lists! + (multi_sort_less): Implement key functions. + (multi_sort): Interface change: arguments rearranged, and new + argument to specify key functions. + + * lib.h (assoc, assq, assql, aconsq_new, aconsq_new_l): Declarations + renamed. + (multi_sort): Declaration updated. + + * txr.1: Documented alist library, list sorting and completed + documenting lazy library. + + * txr.vim: multi-sort highlighted. + +2012-09-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Lots of new documentation. Major rearrangement of document, + with new headings. + +2012-09-01 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (symbol_function): Bugfix: return the function rather than + the whole binding. + +2012-09-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Minor corrections, and documented most stream functions, + except directory-related ones. + +2012-09-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented print, pprint, tostring and tostringp. + +2012-08-31 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Bugfix: under the ~a and ~s directives, + apply field formatting to the object not only if a nonzero width has + been specified, but also if precision has been specified. + + * txr.1: documented stream global variables and format. + +2012-08-31 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand): Bugfix: failure to handle regular + expression syntax, resulting in (set ...) syntax being regarded as + assignment (due to another recent change). + +2012-08-30 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented chain, andf, orf and iff. + +2012-08-30 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_intrinsic): Bugfix: take the expanded form instead + of throwing it away. + (expand): Recurse into expand_place for (set ...) forms. + + * txr.1: Documented eval. + +2012-08-30 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented all functions related to hashing. + +2012-08-29 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (multi_sort_less): Fixing semantics of return value. Individual + sorted lists are returned, rather than a list of zipped tuples. + +2012-08-29 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (multi_sort_less): Change the semantics so that when the + list of the functions is empty, the left item is considered less + than the right, thereby preserving the order. + +2012-08-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (mapcarv): Changed to external linkage. + + * eval.h (mapcarv): Declaration added. + (eval_init): New intrinsic multi-sort registered. + + * lib.c (multi_sort_less): New static function. + (multi_sort): New function. + + * lib.h (multi_sort): Declared. + + * txr.1: stub section added. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Registered open-command and open-process + intrinsics. open-pipe is now deprecated but stays for backward + compatibility as a synonym for open-command. + + * stream.c (open_pipe): Renamed to open_command. + (open_pipevp): Renamed to open_process. + + * stream.h (open_pipe, open_pipevp): Declarations updated. + + * txr.1: Documentation headings updated. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (open_pipev): Bugfix: program name is included + as first element of argv. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + Implement open_pipev in terms of popen for Windows which + does not have for or exec. We could use CreateProcess and CreatePipe, + et cetera, but it won't buy us anything because the whole point + of this function is to improve the argument passing, and CreateProcess + takes a single command line string, not too different from popen. + + * stream.c (pipev_close, make_pipev_stream): Surrounded with + HAVE_FORK_STUFF ifdef. + (pipe_close): Choice of close strategy conditional on HAVE_FORK_STUFF. + (open_pipev): Conditionally defined in two ways now. + (win_escape_arg, win_make_cmdline): New static functions. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + * configure: New test added for fork, pipe, exec and waitpid. + Produces HAVE_FORK_STUFF in config.h. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (w_fopen, w_popen): Removing unnecessary casts of + return values of ut8_dup_to. + + * match.c (complex_open): Likewise. + + * stream.c (w_opendir): Likewise. + +2012-05-18 Kaz Kylheku <kaz@kylheku.com> + + Implementing new pipe function to get around the limitation + that popen accepts a complete command. We need something which + accepts a program name, and a list of arguments, so that + we don't have to assemble together a correctly quoted string. + popen needs an alternative interface resembling execvp. + + * eval.c (eval_init): New intrinsic registered, open-pipe-args. + + * stream.c (struct stdio_handle): New member, pid. + (stdio_stream_print): Print the pid, if it is nonzero. + (pipevp_close): New close function. + (pipe_close): If h->pid is nonzero, it's a new-style pipe, which + must be closed with pipev_close. + (make_stdio_stream, make_pipe_stream): Initialize new stdio_handle + member to zero. + (make_pipevp_stream): New static function. + (open_pipevp): New function. + + * stream.h (open_pipevp): Declared. + + * txr.1: open-pipe-args added to stub section heading. + +2012-05-17 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_collect): Implemented semantics for repeat symbol. + (dir_tables_init): Register dispatch for repeat to v_collect + function. + + * parser.y (collect_repeat): New nonterminal symbol. + (clause): Removed repeat_clause error case because that now clashes + with the syntax in collect_clause. + (collect_clause): Repeat syntax implemented, with help of + collect_repeat. + (out_clause): Error case for collect_clause removed due to + syntactic clash. + + * txr.1: Added mention of @(collect :vars nil) and documented + @(repeat) as the shorthand. + +2012-05-17 Kaz Kylheku <kaz@kylheku.com> + + * configure: Do not capture the stderr of GNU Make in the + configure step; send it to /dev/null. + + * txr.spec: Adding specfile for building RPMs. + +2012-04-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (range_v_func, range_v_star_func): Restore the order of + arguments to plus, so that the from object is on the left. This + change was introduced in a commit on 2012-02-14, causing + character ranges to break due to bug that addition did not + commute for char + fixnum. Although the previous commit fixed + the regression, it's still good to have the object on the + left in case there are some future data types in case there + is ever some form of addition which does not commute. + +2012-04-24 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus): Fixed broken (+ fixnum char) case. + +2012-04-20 Kaz Kylheku <kaz@kylheku.com> + + Fixes for configuring and building in a separate directory. + + * configure: MPI is now extracted and patched in the build directory, + not in the source directory. + + * Makefile (CFLAGS): Refer to mpi headers relative to current + directory, not $(top_srcdir). + (repatch): Run MPI repatching steps in correct directory. + (clean): Remove extracted MPI from build directory. + +2012-04-20 Kaz Kylheku <kaz@kylheku.com> + + Version 65 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-04-20 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (regex_space_chars): Variable removed. + (char_set_addr_str): New function. + (char_set_compile): Use char_set_addr_str to + add spaces to set. + (init_special_char_sets): Use char_set_addr_str to + add spaces to set. Bugfix: word_cs, cword_cs wrongly initialized. + (regex_init): Removed reference to regex_space_chars. + +2012-04-20 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (regtoken): New nonterminal symbol. + (regterm): REGTOKEN production factored out to regtoken. + (regclass): Reverted prior commmit's changes. + (regclassterm): Reverted prior commit, removing REGTOKEN + production for character classes, and introduced a regtoken + production. So now the keyword symbols are part of the + character class abstract syntax. + (regtoken): New production rule. + + * regex.c (regex_space_chars): Converted to internal linkage. + (char_set_compile): Handle token keywords in character class + abstract syntax. + + * regex.h (regex_space_chars): External declaration removed. + +2012-04-19 Kaz Kylheku <kaz@kylheku.com> + + First cut at implementing \s, \d, \w, \S, \D and \W regex tokens. + + * lib.c (init): Call regex_init. + + * parser.l: return new REGTOKEN kind. + + * parser.y (REGTOKEN): New token type. + (REGTERM): Translate REGTERM to keyword. + (regclass): Restructured to handle inherited nodes as lists. + (regclassterm): Produce $$ as list. Add handling for REGTOKEN + occurring inside character class by expanding it. This might not + be the best approach. + (yybadtoken): Handle REGTOKEN in switch. + + * regex.c (struct any_char_set, struct small_char_set, + struct displaced_char_set, struct large_char_set, + struct xlarge_char_set): New bitfield member, stat. + (char_set_create): New parameter for indicating static char set. + (char_set_destroy): Do not free a static char set. + (char_set_compile): Pass zero to new parameter of char_set_create. + (spaces): New static array. + (space_cs, digit_cs, word_cs, cspace_cs, cdigit_cs, cword_cs): New + static pointers to char_set_t. + (init_special_char_sets, nfa_compile_given_set): New static function. + (nfa_compile_regex, dv_compile_regex): Handle new character set token + keywords. + (space_k, digit_k, word_char_k, cspace_k, cdigit_k, cword_char_k, + regex_space_chars): New variables. + (regex_init): New function. + + * regex.h (space_k, digit_k, word_char_k, cspace_k, cdigit_k, + cword_char_k, regex_space_chars, regex_init): Declared. + + * txr.1: Documented. + +2012-04-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsic functions remq*, remql*, + remqual*, remove-if*, keep-if*. + + * lib.c (rem_lazy_func, rem_lazy_rec): New static functions. + (remq_lazy, remql_lazy, remqual_lazy, remove_if_lazy, + keep_if_lazy): New functions. + + * lib.h (remq_lazy, remql_lazy, remqual_lazy, remove_if_lazy, + keep_if_lazy): Declared. + + * txr.1: New functions documented. + +2012-04-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): find-if intrinsic registered. + + * lib.c (find): First and second arguments reversed. + The item should be first. + (find_if): New function. + + * lib.h (find): Declaration updated. + (find_if): Declaration added. + + * txr.1: Stub section. + +2012-04-14 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Added notes on generational garbage collection. + +2012-04-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions remove-if and keep-if. + + * lib.c (remove_if, keep_if): New functions. + + * lib.h (remove_if, keep_if): Declared. + + * txr.1: Documented. + +2012-04-13 Kaz Kylheku <kaz@kylheku.com> + + * configure: Restructuring configure script to be able to detect + nonexistent options. Variable default values are established first, + which simplifies the syntax. The scanning loop detects attempts + to create configuration variables that do not exist. + The var_given convention is folded into the parsing. (If var + exists, and var_given exists, and the user specifies var, + then var_given is set to y to indicate that the value came + from the user.) + +2012-04-13 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: multiple uses of @(next :args) seeing junk "args" string + in argument list. Fix is not to rely on the hack of using the first + element of the list of files to hold the name of the current file. + + * match.c (match_files_ctx): New member, curfile. + (mf_all): Initialize curfile. + (mf_args): Set curfile to "args". + (mf_file_data): Initialize curfile. + (v_skip, v_fuzz, v_gather, v_collect): Use c->curfile rater than + first(c->files) in debug calls. + (freeform_prepare, match_files): Pass c->curfile to ml_all constructor + of match_line_ctx rather than rather than first(c->files). + +2012-04-12 Kaz Kylheku <kaz@kylheku.com> + + Improve the regex Lisp syntax by allowing strings to specify + character compounds. I.e. the syntax "foo" is equivalent to the + cumbersome canonical form (compound #\f #\o #\o). + + * regex.c (nfa_compile_regex, dv_compile_regex): Use chrp function + instead of typeof. Handle stringp case by forming a compound out of the + characters and recursing. Check for some bad objects in the regex + that would never come out of our regex parser but could occur + in a "hand crafted" syntax tree. + +2012-04-11 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Fix misleading comment example. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + Adding patch to MPI to disable its Makefile so people don't try + to work around TXR build problems by running make inside + the MPI directory, which is wrong. + + Also fixing an incorrect use of ctype.h functions in + MPI and one case of a superfluous warning. + + * mpi-patches/series: new patches added. + + * mpi-patches/disable-make: New file. + + * mpi-patches/fix-ctype-warnings: New file. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + More NetBSD porting. + + * Makefile (EXTRA_FLAGS): Handy new variable for additional + ad-hoc CFLAGS. Needed by a new test in configure. + + * configure: Check for annoying warnings about char being used + as an array subscript when calling the macros from <ctype.h>. + If this occurs, suppress it by #undef-ing the macros. + (lang_flags): Switching fromm _XOPEN_SOURCE to _XOPEN_SOURCE=2 + because this is needed on NetBSD to obtain declarations of + popen and pclose. + + * stream.c (pipe_close): If we don't have the WIFCONTINUED macro, + then define it to expand to zero. It's absent on NetBSD and they + already seem to have a hack for this because there is a warning about + the function not being defined, but then the program links anyway. + Let's do it properly and not rely on their hack. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (INT_PTR_MAX_MP): New static variable. + (in_int_ptr_range): New function. + (arith_init): Initialize INT_PTR_MAX_MP. + + * arith.h (in_int_ptr_range): Declared. + + * lib.c (c_num): Allow bignums to be converted to a cnum, if + they are in range, rather than allowing only fixnums. + + * rand.c (make_random_state): Now that we have such a function, + initialize random seed using time value from time_sec_usec rather than + from time and clock. clock is bad for random seeding because it + measures virtual time since the start of the process. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + * configure: Switch from _POSIX_C_SOURCE=2 to _XOPEN_SOURCE. + We will be relying on the gettimeofday function which only came + into POSIX in 2001, but was in Unix long before then. + + * HACKING: Replace notes about -D_POSIX_C_SOURCE which are not true + any more. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (conftest.ccver): New phony target. + + * configure: The test for non-working 128 bit integers in the gcc + used by NetBSD 5.1 is not good enough to catch the bug. + Adding a test which will assume that gcc 4.3.x and lesser + does not have working 128 bit types. + +2012-04-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Expose regex-compile and regexp as intrinsics. + + * lib.c (obj_init): Change spelling of nongreedy operator and put + it into the user package so that it is available for use with + regex-compile. + + * regex.c (match_regex, search_regex): Bugfix: optional start + position argument argument not defaulting to zero. + + * txr.1: Documented regex-compile and regexp. + + * txr.vim: Highlighting regex-compile and regexp. + +2012-04-09 Kaz Kylheku <kaz@kylheku.com> + + Port to NetBSD (5.1). + + * arith.c (bignum_dbl_ipt): Added missing #if HAVE_DOUBLE_INTPTR_T + around function + + * configure: NetBSD's shell is too pathetic to expand "$@" properly + when there are no positional arguments, so I applied the ${@+"$@"} + trick. + (make): New variable. GNU make might be known only as gmake, + so we now detect the command for our own use within the configure + script and also for giving the user advice on what command to use for + building. Put in a fix so that the attempt to run $make --version + does not bail the script under "set -e" if that command terminates + unsuccessfully. + The check for clashing names has been moved earlier, so that + the renaming #define's appear early in config.h (so config.h itself + can rely on the renames). The identifier longlong_t has been added + to the potential clash list. NetBSD exposes this identifier in spite + of -D_POSIX_SOURCE=2, and it clashes with our longlong_t. + The tests for integers wider than long long has been modified + to actually compile a multiplication of two long-long-s into + the wider precision. On the NetBSD system, with gcc 4.1.3, I found + that the __int128 type is there, but doesn't actually work; + compilation of arith.c bails with an internal compiler error. + We now get this internal error at configure time and avoid using + the type. + +2012-04-09 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/add-mp-hash (mp_hash): Fix incorrect + code on platforms where mp_digit is smaller than long. + This was left shifting a mp_digit by MP_DIGIT_BIT. + It was found by compiling on NetBSD with a gcc 4.1.3 + which has a broken__int128_t, forcing us to use 32 bit + mp_digit with a 64 bit mp_word. + +2012-04-08 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (bignum): Previously static function now exposed as external. + + * arith.h (bignum): Declared. + + * configure: Added check for tm_gmtoff and tm_tmzone fields + being present in struct tm. + + * eval.c (eval_init): New intrinsic functions: time, time-usec. + + * lib.c (num): If the cnum is outside of the fixnum range, then + construct a bignum. + (time_sec, time_sec_usec): New functions. + + * lib.h (mut): Slight change to macro to eliminate compiler warning. + (time_sec, time_sec_usec): Declared. + + * txr.1: Stub section for time and time-usec. + + * txr.vim: Highlighting for time and time-usec. + +2012-04-08 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Fixed accidental breakage. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.c: Revert earlier change: config.h must be included before the + <windows.h> section because that header is conditionally included + based on one of the config constants. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + Fixes to get configure ccname=g++ working on OSX Lion. + + * Makefile (conftest.syms): Use -n flag in nm so that the output + is not alphabetically sorted, but numerically. We need this + to get the symbols ordered by increasing offset. + + * configure (read_syms): New function. Factors out logic used + in two places for reading the output of nm. On OSX Lion, it looks + like we do not get symbol sizes but offsets only, when compiling + with g++. The symbols are in an S section. When compiling with + gcc, we get a common C section with symbol sizes. + So the hack is to use the deltas between offsets to get the sizes. + The objects had to be re-ordered in decreasing rank so alignment + doesn't create padding that will get counted as the size. + Interleaved dummy objects of type char should also work. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + * configure: Print done and terminate line after checking for + clashing symbols. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + * configure: Added new check for some clashing external names, + which we can redefine out of the way in config.h. + + * lib.c: config.h was being included before <windows.h>. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented int-flo, flo-int, search-regex, + match-regex and regsub. + +2012-04-07 Kaz Kylheku <kaz@kylheku.com> + + Rounding out hash table functionality with lazy lists that + can walk table content in different ways. + + * eval.c (op_dohash): Follow interface change of hash_next. + (eval_init): hash-keys, hash-values, hash-pairs and hash-alist + intrinsics introduced. + + * filter.c (trie_compress): Follow interface change of hash_next. + + * hash.c (hash_next): Silly interface which takes a pointer to + the iterator has changed to just take the iterator. The function + unambiguously returns nil when the iteration ends, so there + is no need to set the iterator variable to nil. + (maphash): Follows interface change of hash_next. + (hash_keys_lazy, hash_values_lazy, hash_pairs_lazy, hash_alist_lazy): + New static functions. + (hash_keys, hash_values, hash_pairs, hash_alist): New functions. + + * hash.h (hash_next): Declaration updated. + (hash_keys, hash_values, hash_pairs, hash_alist): Declared. + + * lib.c (make_half_lazy_cons): New way of constructing lazy cons, + with the car field specified. It simplifies situations when the + previous cons computes the car of the next one. Why hadn't I thought of + this before? + + * lib.h (make_half_lazy_cons): Declared. + + * txr.1: Doc stubs for new hash functions. + + * txr.vim: Highlighting for new hash functions. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + Version 64 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: @[...] syntax not marked as "contained" because it + can freely occur, and is useful in @(output). + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (FRESHQ_SIZE): Preprocessor symbol renamed to FRESHOBJ_VEC_SIZE. + (freshobj, make_obj): Object and function definitions follow rename. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (mark_obj, sweep_one, gc_is_reachable): Check for gen > 0 rather + than gen == 0. This allows gen == -1 objects to be considered the + same as gen == 0, and traversed. + (gc_set, gc_mutated): When a gen 0 object is added to the checkobj + array, set its generation to -1. This prevents duplicates in + the checkobj array. Also, it fixes a bug: an vector marked as + mutated was not being traversed due to being in generation 1. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + Code cleanup and tweaking. + + * gc.c (BACKPTR_VEC_SIZE): Preprocessor symbol renamed to + CHECKOBJ_VEC_SIZE. + (FULL_GC_INTERVAL): Increased to 40 since the modified + algorithm now leaves less work for the full gc to do. + (backptr, backptr_idx): Static variables renamed to + checkobj and checkobj_idx. + (mark): Follows rename of backptr and backptr_idx. + (gc): Commented out handy printf added. + (gc_set): Use in_malloc_range check to avoid adding to + the check array pointers which are being stored in non-heap locations, + since non-heap locations are already GC roots. + (gc_mutated): Follows variable renaming. + (gc_push): Just do the push using gc_set. + + * lib.c (malloc_low_bound, malloc_high_bound): New variables. + (chk_malloc, chk_calloc, chk_realloc): Updated malloc_low_bound + and malloc_high_bound. + (in_malloc_range): New function. + + * lib.h (in_malloc_range): Declared. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + The mut macro should only be used for vectors or vector-like objects + which hold direct references to other objects and must be used + each time a mutation takes place. + + * eval.c (op_dohash): invocations of mut macro removed. + Comment rewritten. + + * lib.c (sort_list): Use set macro for mutating assignment. + Do not invoke mut on sorted list; it won't work anyway, because + it doesn't mean what the code wants it to mean: that the list will be + fully traversed during gc marking. + +2012-04-05 Kaz Kylheku <kaz@kylheku.com> + + Bunch of fixes. + + * gc.c (gc_mutated): Return the value. + + * gc.h (gc_mutated): Declaration updated. + + * hash.c (remhash): Fix unsafe assignment to use set macro. + + * lib.c (sort): Fix wrong use of mut macro on the list + before it is sorted rather than after. + + * lib.h (mut): Trivial version of macro updated to return argument. + + * unwind.c (uw_init): The toplevel environment's match_context + should be gc_protected. Though this is probably not used, + which is why it has not been a problem. + +2012-04-04 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_grow, gethash_l, gethash, gethash_f): Replace + useless use of *vecref_l() with vecref(). + +2012-04-04 Kaz Kylheku <kaz@kylheku.com> + + * configure (gen_gc): Default to off. + Help section added for gen_gc variable. + + * gc.c (gc): Some missing CONFIG_GEN_GC added. + +2012-04-04 Kaz Kylheku <kaz@kylheku.com> + + Code cleanup. + + * gc.c (backptr_oflow): Static variable removed. + (freshq_head, freshq_tail, partial_gc_count): Static variables removed. + (freshq): Array renamed to freshobj. + (full): Variable renamed to full_gc. + (freshobj_idx): New varaible. + + (make_obj): Add newly born objects to freshobj array rather than + freshq. If freshobj array is full on entry to this function, + trigger gc to empty it. make_obj no longer updates the free_tail; + the gc routine takes care of restoring this invariant. + (mark_obj): Follows rename of full_gc. Some code was not wrapped + in #if CONFIG_GEN_GC. + (mark, sweep_one): Follow rename of full_gc. + (sweep): On entry, restore free_tail invariant in the empty + free_list case. Code which processes freshq from tail to head + replaced by simple array walk of freshobj. Code wrapped properly + in #if CONFIG_GEN_GC. + (gc): Logic for triggering full gc simplified. + Check added for situations when a partial gc is called when + the free list empties, and it doesn't liberate enough memory. + This prevents the situation of partial gc being called over and over + again by make_obj, squeezing less and less memory each time until + finally it returns 0 objects, and more() is called. + (gc_is_reachable): Follows rename of full_gc, and #if CONFIG_GEN_GC + added. + (gc_set, gc_mutated): Simplified. Check if the backptr array + is full and trigger gc if so to flush it, then just add to the array. + +2012-04-03 Kaz Kylheku <kaz@kylheku.com> + + Performance tweaking and fixes. + + * gc.c (BACKPTR_VEC_SIZE): Increase greatly, so that we don't + trigger gc due to overflow of the backptr array. This is not likely + to yield a lot of free objects except in a full GC. + (FULL_GC_INTERVAL): From 10 to 20. + (gc): Take a not of whether or not gc was entered with free_list + being exhausted or not. Call more() only if the free_list was + empty, and a full sweep was done. + Reset partial_gc_count only when a full gc is triggered. + +2012-04-03 Kaz Kylheku <kaz@kylheku.com> + + Fix failing test case tests/006/freeform-1.txr. + + * lib.c (lazy_str_force, lazy_str_force_upto): Use set macro + when assigning lim. This won't cause a problem unless lim is + in the bignum range, however. + (acons_new, aconsq_new): When overwriting the cdr value of + the existing entry, use set. This is the smoking gun; + these functions are used for manipulating bindings. + (sort): After sorting a list, we must mark it as having + been mutated. If a list contains only mature conses or only + fresh conses, there is no problem. But if it contains a mixture, + then sorting could reverse their relationship, causing mature + conses to backpoint to the fresh ones. + (obj_init): Use set when installing the t symbol into the user package. + +2012-04-03 Kaz Kylheku <kaz@kylheku.com> + + Generational GC showing signs of working. One test case in + test suite fails. + + * gc.c (FRESHQ_SIZE): New preprocessor symbol. + (backptr_oflow, freshq, freshq_head, freshq_tail): New static + variables. + (make_obj): Place newly allocated generation 0 object into + freshq. If freshq becomes full, transfer oldest item into + generation 1. + (mark_obj): If doing only a partial gc, then do not mark + objects which are not generation 0. + (mark_mem_region): Valgrind support: cannot mark t.type field undefined + because it is a bitfield. Just mark the first SIZEOF_PTR bytes + of the object defined. + (mark): Under partial gc, mark the table of back pointers. + (sweep_one): New static function from the prior guts of sweep. + Reachable objects now get promoted to generation 1. + (sweep): Under partial gc, sweep just the freshq which identifies + the generation 0 objects, rather than the entire linked list of all the + heaps. + (gc): Trigger full gc also if the backptr list has overflowed + due to gc having been disabled. + Under generational gc, reset the static variables afterward: + clear the list of backpointers, and the freshq. + (gc_is_reachable): Under partial gc, report any mature object + as reachable. + (gc_set, gc_mutated): Handle backptr array overflow situation + when gc is disabled. + (gc_push): Bugfix: it is the newly pushed cons cell that has to be + marked as a root, not the value being pushed. + + * hash.c (sethash): Use set macro for storing value. + + * lib.h (set, mut, mpush): Fix wrong-way #if test for these macros. + The trivial versions were being defined uner CONFIG_GEN_GC and vice + versa! + +2012-04-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_modplace): push replaced with mpush (mutating push). + + * gc.c (gc_push): New function. + + * gc.h (gc_push): Declared. + + * hash.c (pushhash): Use mpush. + + * lib.c (push): Reverted to unsafe operation. TODO comment replaced + with warning. + (lazy_flatten_scan): push occurence commented as safe. + (lazy_stream_func): Unsafe push replaced with mpush. + + * lib.h (mpush): New macro. + +2012-04-02 Kaz Kylheku <kaz@kylheku.com> + + * configure: Support a gen-gc configuration variable which + causes CONFIG_GEN_GC to be defined as 1 in config.h. + + * eval.c (op_defvar, dwim_loc, op_modplace, transform_op): Handle + mutating assignments via set macro. + (op_dohash): Inform gc about mutated variables. TODO here. + + * filter.c (trie_add, trie_compress): Handle mutating assignments + via set macro. + + * gc.c (BACKPTR_VEC_SIZE, FULL_GC_INTERVAL): New preprocessor symbols. + (backptr, backptr_idx, partial_gc_count, full): New static variables. + (make_obj): Initialize generation to zero. + (gc): Added logic for deciding between full and partial gc. + (gc_set, gc_mutated): New functions. + + * gc.h (gc_set, gc_mutated): Declared. + + * hash.c (hash_mark): Changed useless use of vecref_l to vecref. + (gethash_f): Use set when assigning through *found since it + is a possible mutation. + + * lib.c (car_l, cdr_l, vecref_l): Got rid of loc macro uses. Using the + value properly is going to be the caller's responsibility. + (push): push may be a mutation, so use set. + (intern): Uset set to mutate a hash entry. + (acons_new_l, aconsq_new_l): Use set when replacing *list. + + * lib.h (PTR_BIT): New preprocessor symbol. + (obj_common): New macro for defining common object fields. + type_t is split into two bitfields, half a pointer wide, + allowing for generation to be represented. + (struct any, struct cons, struct string, struct sym, struct package, + struct func, struct vec, struct lazy_cons, struct cobj, struct env, + struct bignum, struct flonum): Use obj_common macro to defined + common fields. + (loc): Macro removed. + (set, mut): Macros conditionally defined for real functionality. + (list_collect, list_collect_nconc, list_collect_append): Replace + mutating operations with set. + + * match.c (dest_set, v_cat, v_output, v_filter): Replace + mutating operations with set. + + * stream.c (string_in_get_line, string_in_get_char, + strlist_out_put_string, strlist_out_put_char): Replace mutating + operations with set. + + * unwind.c (uw_register_subtype): Replace mutating operation with set. + +2012-04-02 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (vec_set_length): Use set instead of assignment. + (vecref_l): Use loc to lift address of cell. + (replace_vec): Use macro mut to indicate the object is being + mutated. + + * lib.h (mut): New macro. + +2012-04-01 Kaz Kylheku <kaz@kylheku.com> + + Start of ground-work for ephemeral GC. We must add some abstraction + to places where we potentially assign a reference to a younger object + inside a field located in an older object (chronological + backreference) and also where we take the address of an object + field, making it possible that the user of the address will do so. + + This patch does not take care of vectors. + + No, this is not an April Fool's joke. + + * eval.c (env_fbind, env_vbind, env_replace_vbind, lookup_var, + lookup_sym_lisp1): Use set macro instead of assignment. + + * hash.c (hash_grow, set_hash_userdata, hash_next): + Use set macro instead of assignment. + + * lib.c (rplaca, rplacd, string_extend, length_str, replace_str, + rehome_sym, lazy_stream_func, lazy_str, lazy_str_force, + lazy_str_force_upto, obj_init): Use set macro instead of assignment. + + (car_l, cdr_l): Use loc instead of address-of operator. + + * lib.h (set, loc): New macros. + +2012-03-31 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (last_equal_key, last_equal_hash): New static variables. + (equal_hash): Caching optimization implemented. + (eql_hash): Optimization extended to those objects that have + equal semantics. + (hash_process_weak): Clear the cached hash during gc. + +2012-03-31 Kaz Kylheku <kaz@kylheku.com> + + If one of the blocks which are subordinate to a @(trailer) + happen to request a successful termination by invoking @(accept) + the position must not advance into the trailer material. + + * match.c (v_trailer): Added an unwind protect which + detects that an accept is taking place and adjusts the return value to + restrict the input position at the point given to trailer. + (accept_fail): Use uw_block_return_proto instead of uw_block_return + and pass the symbol as the protocol identifier. + + * unwind.c (uw_current_exit_point): New function. + (uw_block_return): Function renamed to uw_block_return_proto; + takes new parameter which is stored in the block structure. + + * unwind.h (struct uw_block): New member, protocol. + (uw_block_return): Becomes an inline wrapper for uw_block_return_proto. + (uw_block_return_proto, uw_current_exit_point): Declared. + + * txr.1: Interaction between @(trailer) and @(accept) documented. + +2012-03-30 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_var): Disallow the variable named by the symbol t + by throwing an exception. Allow nil, but wherever nil occurs, + do not produce a binding. + + * txr.1: State the restrictions against using t in the section + on Variables and also describe the nil ignore feature. + +2012-03-30 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Correction: backtracking does NOT take place into a block + which completed. + +2012-03-30 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documenting the debugger with an example session. + +2012-03-30 Kaz Kylheku <kaz@kylheku.com> + + Version 63 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-03-30 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (num_str): Much more accurate test for deciding whether + to treat the number as floating or integer. We can't just look + for the presence of E, e or . because these coudl be part of + trailing junk for instance "123XYZE." should convert + to the integer 123, where "XYZE." is trailing junk. + + * txr.1: Documented int-str, flo-str and num-str. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (numeq): Fix misplaced parenthesis. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (min2, max2): Semantics tweak. If the numbers are equal, + favor the left one. + + * txr.1: Documented min and max. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (numeq): New function. + (exptmod): Bugfix: was no normalizing the bignum, ouch. + Also was reporting "non-integral operands" for other + errors. + + * eval.c (eval_init): Registered = intrinsic function. + + * lib.c (numeqv): New function. + + * lib.h (numeq, numeqv): Declared. + + * txr.1: Documented expt, sqrt, isqrt, exptmod, fixnump, bignump, + integerp, floatp, numberp, zerop, evenp, oddp, >, <, >=, <= and =. + + * txr.vim: Highlight = + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (gcd): Allow zeros. Don't issue "non-integral" + exception if MPI fails. + (floorf, ceili): Map integer argument to itself. + (tang, asine, acosi): New functions. + + * eval.c (eval_init): New intrinsics: tan, asin, acos. + + * lib.h (tang, asine, acosi): Declared. + + * txr.1: Documented gcd, abs, floor, ceil, sin, cos, tan + asin, acos, atan, log, and exp. + + * txr.vim: Highlighting for tang, asine, acosi. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (dmod): New static function. + (mod): Use dmod instead of fmod directly, to calculate + the correct semantics for combinations of + negative operands in the floating point domain also. + + * txr.1: Documented /, trunc and mod. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented +, - and *. + +2012-03-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: num-str added to doc stub. + + * txr.vim: num-str added. + +2012-03-28 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print, obj_pprint): Do not use the #<lazy-string ..> + notation for lazy strings that have been forced. + +2012-03-27 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Compensate for differences in printf + implementations with regard to printing floating point exponents. + by deleting any plus sign and leading zeros after the 'e'. + + * tests/009/json.expected: Regenerated. + +2012-03-26 Kaz Kylheku <kaz@kylheku.com> + + Filtering on lists and nested lists is hereby made to work. + For instance given @(bind a ("a" "b" "c")) it is now possible + to do @(filter :upcase a) whereby a promptly takes on the value + ("A" "B" "C"). + + * filter.c (string_filter): Function renamed to string_tree_filter. + (compound_filter): Follows rename. + (filter_string): Function renamed to filter string tree. + Can filter tree of strings, or possibly other objects, + if the filter function allows. + (filter_equal): No special case test for objects that are strings. + Just put them through the filter. + + * filter.h (filter_string): Declaration updated. + + * match.c (format_field, subst_vars, v_filter): Follow rename. + +2012-03-26 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_output): Bugfix: we should flush the stream + after each @(output) block. Otherwise if output blocks + that go to standard output are interleaved with output blocks + which pipe to some command which then goes to standard out, + the output won't be in the proper order. + +2012-03-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsic num-str registered. + + * filter.c (tonumber_k, tointeger_k, tofloat_k, hextoint_k): + New keyword variables. + (filter_init): New variables initialized; new filters registered. + + * filter.h (tonumber_k, tointeger_k, tofloat_k, hextoint_k): + Declared. + + * lib.c (num_str): New function. + + * lib.h (num_str): Declared. + + * txr.1: New filters documented. + +2012-03-26 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (to_float): Fix unterminated argument list in throwf. + + * lib.c (funcall): Likewise. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (rebind_s): New symbol variable. + + * lib.h (rebind_s): Declared. + + * match.c (v_rebind): New static function. + (dir_tables_init): Registered rebind_s to v_rebind, + and also to hv_trampoline in the horizontal directive table. + + * txr.1: Documented it. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + Bug #35989 + + * match.c (syms_init): text_s must be in the system + package because it's not a user-visible operator. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + Performance improvement in the GC: keep at least one heap's worth + of free space, so programs close to exhausting a heap do not + waste cycles frequently calling the collector. + + * gc.c (more): Do not assert that free_list is null; this will + not be the case any more. + (make_obj): Comment added regarding why we the free_tail variable + is fixed up. + (sweep): Now returns a count of the objects that are free. + (gc): If sweep reports that less than 75% of the objects are free + then let us add another heap. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register match-str and match-str-tree + intrinsics. + + * lib.c (match_str, match_str_tree): Default position to zero. + + * txr.1: Doc stubs created. + + * txr.vim: Highlighting for match-str and match-str-tree. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: code like @(skip)@{var /partial/} where + the regular expression does not match all the way to + the end of the line was getting by the check for + a complete match. + + * match.c (do_match_line): Loses the second parameter + named completely. The check whether the line was matched + completely is done higher up, in match_line_completely. + This is needed because do_match_line has some early + successful return cases which bypass the check. + (match_line): Remove second paramter in call to do_match_line. + (match_line_completely): Do the check here that the line + was matched completely. Nothing can get by this. + (v_freeform): Do notpass second nil argument to do_match_line. + +2012-03-24 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (search_str): If start_num is nil, default it to zero. + This is needed for this to work right as an optional argument. + +2012-03-23 Kaz Kylheku <kaz@kylheku.com> + + Version 62 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2012-03-23 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: Updated. + + * txr.1: Describe floating-point constants. + +2012-03-23 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (TXR_ARGS): Pass new file to tests/009/json.txr test. + + * tests/009/json.expected: Updated. + + * tests/009/json.txr: Updated source. Translates to a more native + representation with vectors and hash tables. Numbers go to + floating point instead of remaining as strings. + + * tests/009/pass1.json: New file: a test case from json.org. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (expo): New function. + + * eval.c (eval_init): expo registered as intrinsic exp. + + * lib.h (expo): Declared. + + * txr.1: Added to stub heading. + + * txr.vim: Highlighting for exp. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (transform_op): use integerp instead of numberp. + Not all numbers are integers now, and that situation + requires an integer. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Bugfix: was not allowing e-notation floats + with no decimal point like 1E1. + + * stream.c: (vformat): Keep track of whether or not precision was + given in precision_p local variable. + When printing #<bad-float> pass a precision of 0 + to vformat_str, not precision, since precision does not apply. + In ~f and ~e, if the precision was not given, default + it to 3. + Restructured float printing in ~a and ~s. It now just uses sprintf's %g + with a precision. If user does not specify precision, it defaults + to DBL_DIG to print the number with reasonable accuracy. + A .0 is added if it sprintf produces an integer, and the conversion + is ~s rather than ~a. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + Fix sqrt confusion. There must be a separate isqrt + for the integer square root. + + * arith.c (sqroot_fixnum): Renamed back to isqrt_fixnum. + (sqroot): Rewritten to handle only floating-point square root. + (isqrt): New function, based on previous sqroot, + handles only integers. + + * eval.c (eval_init): New intrinsic, isqrt. + + * lib.h (isqrt): New declaration. + + * txr.1: Doc stubs. + + * txr.vim: Highlighting for isqrt. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (floorf, ceili, sine, cosi, atang, loga): New functions. + + * eval.c (eval_init): New intrinsic functions registered: + floor, ceil, sin, cons, atan, log. + + * lib.h (floorf, ceili, sine, cosi, atang, loga): Declared. + + * txr.1: Doc stub section for new functions. + + * txr.vim: Highighting added. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (int_flo): If sprintf produces something + that doesn't begin with a digit, it's most likely NaN or Inf. + We can turn that into an exception. + + * stream.c (vformat): If sprintf produces a non-number, + turn it into the printed representation #<bad-float>. + +2012-03-22 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (to_float): New static function. + (divi): Uses to_float. + (zerop, gt, lt, ge, le, expt): Floating support. + (isqrt_fixnum): Static function renamed to sqroot_fixnum. + (isqrt): Renamed to sqroot. Floating support. + (evenp, oddp, exptmod, gcd): Work with integers, not floats. + + * eval.c (eval_init): intrinsic registration of sqrt follows rename of + isqrt to sqroot. + + * lib.h (isqrt): Declaration replaced. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (divi): New function. + + * eval.c (eval_init): divi registered as / intrinsic. + + * lib.h (divi): Declared. + + * txr.1: divi added to stub heading. + + * txr.vim: / operator highlighted. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (mod): Floating support. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (trunc): Floating support. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus, minus, mul): Removing unnecessary type checks, + which are already implied by the switch case. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Doc stubs for new functions floatp, integerp, + float-str, int-flo and flo-int. + + * txr.vim: Highlighting for new functions. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * Makefile: link in -lm, which is needed now on some systems. + + * arith.c (plus, minus): Eliminated some unnecessary (double) casts. + (abso, mul): Floating support. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (neg): Floating-point support. + + * parser.l: FLO and FLODOT cases had to be reordered because + the lex trailing context counts as part of the match length, + causing 3.0 to be matched as three characters with 0 as + the trailing context. The cases are split up to eliminate + a flex warning. + + * stream.c (vformat): Support bignum in floating point + conversion. Bugfixes: floating point conversion was + accessing obj->fl.n instead of using n. + Changed some if/else ladders to switches. + +2012-03-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus): Minor code simplification. + (minus): Floating point support. + + * mpi-patches/mpi-to-double (mp_to_double): Re-apply lost + bugfix: index incremented instead of decremented. + Didn't refresh patch last time, then did a make distclean. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + Regression fix: 1..3 scans incorrectly into 1. .3 tokens. + + * parser.l (SGN, EXP, DIG): New regex definitions. + (FLO): Do not recognize numbers of the form 123. + Decimal point must be followed either by exponent, or digits + (which may then be followed by an exponent). + (FLODOT): New token type, recognizes 123. + (grammar): Recognize FLODOT as a floating point number, + only if it not trailed by another dot, and + recognize FLO unconditionally. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus): Completed implementation of bignum-float + and float-bignum cases. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Use larger num_buf buffer so we don't + overrun. IEEE double floats can go to e+-308. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (flo_int): New function. + + * eval.c (eval_init): flo-int registered as intrinsic. + + * lib.h (flo_int): Declared. + + * mpi-patches/series: Added mpi-to-double to patch stack. + (mp_to_double): New MPI function. + + * mpi-patches/mpi-to-double: New file. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus): Optimization: use num_fast when + result is in the fixnum range. + Implemented FLNUM cases, except for adding a FLNUM + to BGNUM. + (minus, mul): Use num_fast when the cnum value is in the fixnum range. + (int_flo): New function. + + * eval.c (eval_init): Register int-flo intrinsic. + + * lib.c (c_flo): New function. + + * lib.h (TYPE_SHIFT, TYPE_PAIR): New macros, carried over + from the lazy strings branch. + (c_flo, int_flo): Declared. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (FLO): Adjusted syntax of floating point numbers + to allow leading or trailing decimal. + +2012-03-19 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): num_buf increased to 256 because we + are now printing floating point numbers into it, letting + the C library handle precision which can generate many digits. + We cap the precision at at 128. New format specifiers ~e + and ~f implemented, which loosely correspond to those of printf. + The ~s and ~a directives handle floats similarly to ~g in + printf, except that they ensure that a decimal point is printed + for the non-exponential notation. + +2012-03-19 Kaz Kylheku <kaz@kylheku.com> + + * configure (uintptr): New variable. Indicates whether unsigned + version of intptr_t is available and should be generated in config.h + as uintptr_t. + + * eval.c (eval_init): New intrinsic functions floatp, + integerp, flo-str. + + * gc.c (finalize): Handle FLNUM case. Rearranged + cases so that all trivially returning cases are + together. + (mark): Handle FLNUM case. + + * hash.c (hash_double): New function. + (equal_hash): Handle FLNUM via hash_double. + (eql_hash): Likewise. + + * lib.c: <math.h> is included. + (float_s): New symbol variable. + (code2type, equal): Handle FLNUM case in switch. + (integerp): New function; does the same thing + as integerp before. + (numberp): Returns t for floats. + (flo, floatp, flo_str): New functions. + (obj_init): Initialize new float_s variable. + (obj_print, obj_pprint): Handle FLNUM case in switch. + Printing does not work yet; needs work in stream.c. + + * lib.h (enum type): New enumeration FLNUM. + (struct flonum): New struct type. + (union obj): New member, fl. + (float_s, flo, floatp, integerp, flo_str): Declared. + + * parser.l (FLO): New token pattern definition. + Scans to a NUMBER token. + Corrected uses of yylval.num to yylval.val. + + * parser.y (%union): Removed num member from yystype. + +2012-03-20 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (debug): Breakpointing now takes into account + the module file name, not only the line number. Breakpoints work on + source locations rather than line numbers. Boy, this was easy. + Keep the breakpoint list free of duplicates. Issue a message + if a nonexistent breakpoint is asked to be deleted. + +2012-03-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): url_decode has two parameters now, + so we make the second one optional. + + * filter.c (topercent_k, frompercent_k): New keyword + variables. + (url_encode, url_decode): Take a second parameter, space_plus. + This determines whether or not to apply the rule that + a space encodes as a + character. + (filter_init): Initialize new keyword variables, and register :topercent + and :frompercent filters. Fix the previous registrations of :tourl and + :fromurl using currying. + + * filter.h (urlencode, urldecode): Declarations updated. + (topercent_k, frompercent_k): Declared. + + * txr.1: Documented. + +2012-03-17 Kaz Kylheku <kaz@kylheku.com> + + Changing type function to not blow up on nil, which makes a lot of code + simpler. A pseudo type code is introduced called NIL with value 0. + + * lib.h (enum type): New enumeration value, NIL. + (type): Function accepts object nil and maps it to code NIL. + + * eval.c (dwim_loc, op_dwim): test for nil obj and goto hack is gone, + just handle NIL in the switch. + + * gc.c (make_obj, mark): Handle new NIL type code in switch. + + * hash.c (equal_hash): Handle NIL in the switch instead of nil test. + + * lib.c (code2type): Map new NIL type code to null. + (typeof, typecheck): Code simplified. + (class_check, car): Move nil test into switch. + (eql, equal, consp, bignump, stringp, lazy_stringp, + symbolp, functionp, vectorp, cobjp): Simplified. + (length, sub, ref, refset, replace, obj_print, obj_pprint): Handle NIL + in switch instead of nil test. goto hack removed from refset. + + * match.c (do_match_line, do_output_line): switch condition simplified. + + * regex.c (regexp): Simplified. + (regex_nfa): Assert condition simplified. + +2012-03-17 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (digit_value): static function moved. + (html_hex_continue): Use digit_value instead of hex digits string + literal. + +2012-03-16 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (do_chain): More useful behavior. The first + function, if any, is treated as variadic. Subsequent + functions are monadic. + (chain, chainv): Turn do_chain into variadic. + +2012-03-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsic functions + chain, andf, orf, iff. + + * lib.c (chainv): New function. + (do_and, do_or): Generalized to handle functions of + any arguments via apply. + (andf, orf): Turn do_and and do_or into variadic function instead of a + monadic function. + (do_iff): New static function. + (andv, orv, iff): New functions. + + * lib.h (chainv, andv, orv, iff): New functions declared. + + * txr.1: Doc stubs created. + + * txr.vim: Updated. + +2012-03-16 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (quicksort): Bugfix: incorrect loop from 0 rather than from + leading to unbounded recursion. + +2012-03-15 Kaz Kylheku <kaz@kylheku.com> + + Version 61 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * eval.c (op_modplace): Fix warning about uninitialized variable. + No bug. + + * filter.c: gcc compilation regresion: missing <stdio.h> breaks inclusion + of "stream.h" header. Strangely, didn't show up when configured for + compiling with g++ on Ubuntu. + + * match.c (match_filter): Fixed ununsed variable warning. + + * txr.vim: Bunch of missing keywords added. + + * dep.mk: Regenerated. + +2012-03-15 Kaz Kylheku <kaz@kylheku.com> + + * tests/010/block.expected: New file. + + * tests/010/block.txr: New file. + + * tests/010/reghash.expected: New file. + + * tests/010/reghash.txr: New file. + +2012-03-14 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: Updated for upcoming 61. + +2012-03-14 kaz kylheku <kaz@kylheku.com> + + * txr.1: Documented hash and vector quasiliterals. + +2012-03-14 Kaz Kylheku <kaz@kylheku.com> + + Support quasiquoting over vectors also, and a bugfix for hash + quasiquoting. We cannot use the same symbol for the + literal form from the parser, and for the expanded form, + because this creates a confusion when there are multiple + nestings of quasiquote expansion. + + * eval.c (vector_lit_s, vector_list_s, hash_lit_s): New symbol + variables. + (hash_construct_s): Relocated here from hash.c. + (expand_qquote): Part of bugfix: look for hash_lit_s + instead of has_construct_s. Translate to a hash_construct_s + form which is no longer recognizes as a hash literal. + Implementing recognition of a quasiquote vector literal, + handled similarly. + (eval_init): Initialize vector_lit_s, vector_list_s, + hash_list_s and hash_lit_s. + Use vector_list_s when registering vector_list function. + + * eval.h (vector_lit_s, vector_list_s, hash_lit_s, + hash_constuct_s): Declared. + + * hash.c (hash_construct_s): Variable removed + and relocated into eval.c. + (hash_init): Initialization of hash_construct_s removed. + + * hash.h (hash_construct_s): Declaration removed. + + * parser.y: (vector): Action updated to generate + a (vec-lit ...) form if the object contains unquotes, + otherwise generate a vector object. + (hash): Generate hash-lit form, not a + hash-construct form. + +2012-03-14 Kaz Kylheku <kaz@kylheku.com> + + Allow quasi-quoting over hash table literals, + to express dynamic hash table construction + + * eval.c (expand_qquote): Recognize hash-construct + forms: expand the hash arguments and pairs separately, + then rewrite to a new hash-construct form. + (eval-init): hash-construct intrinsic function added. + + * hash.c (hash_construct_s): New symbol variable. + (hash_construct): New function. + (hash_init): Initialize hash_construct_s. + + * hash.h (hash_construct_s, hash_construct): Declared. + + * parser.y (hash): Rule rewritten to emit either a literal + hash table object, or a hash-construct form, based on + whether quasiquote unquotes occur within the syntax. + (hash_from_notation): Function removed. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + Change: @(block) requires @(end) from now on. + Blocks no longer extend to the end of the surrounding + scope. + + * match.c (v_block): Rewrite for new syntax. + + * parser.l (BLOCK): New token type handled. + + * parser.y (BLOCK): New token. + (block_clause): New nonterminal grammar symbol. + (clause): Collateral fix: replaced a bunch of + list(X, nao) forms with cons(X, nil). + Introduced block_clause as a constituent of clause. + + * txr.1: Revamped documentation of block, and + wrote about using blocks for reducing nested + skips and reducing backtracking in general. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (ID_END): Bugfix: ID_END was defined incorrectly + for the current way in which an identifier token is recognized. + As a result @(collect-ing) was being interpreted as @(collect -ing). + It should be the complement of NSCHR. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + * regex.c (regsub): the replacement argument + can now be a function of one argument which maps + the original piece of text matched by the regex + to a replacement text. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (stdio_put_string, stdio_put_char, stdio_put_byte): Do not + consider data sent to std_error to be output for the purposes of + the output_produced flag. Otherwise the program behavior changes + in -v mode; it will not print bindings. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + Implementing URL filtering. + + * eval.c (eval_init): New intrinsic functions: url-encode, url-decode. + + * filter.c (tourl_k, fromurl_k): New keyword variables. + (is_url_reserved, digit_value): New static functions. + (url_encode, url_decode): New functions. + (filter_init): Intialize new keyword variables and register + new :tourl and :fromurl filters. + + * filter.h (tourl_k, fromurl_k, url_encode, url_decode): Declared. + + * txr.1: Updated. + + * txr.vim: Likewise. + +2012-03-13 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (string_out_byte_flush): Bugfix. Do not loop inside this + function. This must not flush out more than one character out of this + small buffer, except when we are flushing out the last data. + The correct operation is predicated on the assumption that + a complete character can be pulled out. That's why we move the + buffer to the front after consuming it, and do not automatically + flush until there are four bytes. + (string_out_put_string): We loop the call to string_out_byte_flush + here because when a request comes in to write a Unicode character, + we flush all the bytes, even if the tail of those bytes forms + an incomplete sequence that turns into U+DCxx codes. + (get_string_from_stream): Use the same loop termination test + as in string_out_put_string, for consistency. In that function + it is needed to prevent infinite looping in the case when + the string_out_put_string is being called from string_out_byte_flush + and is thus re-entering it. + + * tests/010/strstream.expected: New file. + + * tests/010/strstream.txr: New file. + +2012-03-12 Kaz Kylheku <kaz@kylheku.com> + + Implementing put_byte for string output stream. + This does the Right Thing with a mixture of bytes and characters. + Incomplete byte sequences. + + * stream.c (struct strm_ops): Changing byte argument of put_byte + to int, since the put_byte API function can just pass down that + value after validating it. + (stdio_handle): Use available typedef. + (stdio_put_byte): Follow interface change in strm_ops. + Do not validate the range of a byte; the put_byte higher + level function does that now. + (struct string_output): New members: ud, byte_buf, head, tail. + (string_out_byte_callback, string_out_byte_flush): New static + functions. + (string_out_put_string): Flush any UTF-8 bytes in the byte buffer + before putting the string. + (string_out_put_byte): New static function, implementation for + put_byte on string output streams. + (string_out_ops): string_out_put_byte wired in. + (make_string_output_stream): Initialize new members of + of struct string_output. + (get_string_from_stream): Flush any UTF-8 bytes in the byte buffer + before retrieving the string. + (put_byte): Validate that the byte is in range. Pass byte + as C int down to the put_byte virtual. + +2012-03-12 Kaz Kylheku <kaz@kylheku.com> + + Plugging memory leak. + + * stream.c (byte_in_stream_destroy): New function. + (byte_in_ops): Use new function instead of noop stub. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Update. Adding remq, remql, remqual, regsub, *stddebug* + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + Bug #35718. Workaround good enough to get some code working. + + * eval.c (cons_find): New function. + (expand_op): Use cons_find rather than tree_find to look for + rest_gensym. + + * regex.c (regsub): Rearranged arguments so that the string + is last. This is better for partial evaluaton via the op + operator. + + * regex.h (regsub): Updated declaration. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New intrinsic function, regsub. + + * regex.c (regsub): New function. + + * regex.h (regsub): Declared. + + * txr.1: Doc stub added. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (split_str): Separator can be a regex now. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + Version 60 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + + * txr.vim: Handle : symbol properly. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus): Showstopper bug: plus(bignum, fixnum) + broken for fixnum <= 0. Bad pointer passed to MPI. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/add-mp-hash (mp_hash): Fixed use of uninitialized + variable on platforms where the MP digit is smaller than a long integer. + (Not anything TXR is known to run on). Changed algorithm to take the + first and last digit and add them together, rather than just taking the + last digit. The last digit will be zeros for numbers that contain 2 as a + factor with a large enough multiplicity. + + * mpi-patches/add-mpi-toradix-with-case: Refreshed. + + * mpi-patches/bit-search-optimizations: Likewise. + + * mpi-patches/faster-square-root: Likewise. + + * mpi-patches/fix-bad-shifts: Likewise. + + * mpi-patches/fix-mult-bug: Likewise. + +2012-03-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_defun): Bugfix: documentation says that defun supports + the same parameter list as lambda, and that is the intent. But + this was broken, since op_defun was expecting the parameter list + to be a proper list containing only bindable symbols, ruling out + the use of the consing dot for rest parameter as well as the colon + keyword symbol for optional parmeters. + +2012-03-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Expose remq, remql and remqual. + + * txr.1: Documented. + +2012-03-03 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sub_str): Removed second check for lazy string that + can't ever come out true. + + * match.c (search_form, h_var, h_coll, h_parallel, h_fun): Handle + position t emanating from match_line, indicating match to end of line. + (h_skip): When skipping to the end of line (empty spec), just + return t as the position rather than the end of the line. This avoids + calculating the length of the line, which forces a lazy string. + (do_match_line): Near the beginning of the loop, if the position is t, + then substitute the length of the line. + (freeform_prepare): Return the freeform line limit value. + (v_freeform): Check for t coming out of match line and do the + conversion back to the trailing list in that case, but only if + the freeform was limited by number of lines. + +2012-03-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (search_form): search_form works with relative positions now, + but match_line returns absolute, and so needs to be offset by -c->base. + (h_trailer, h_fun): Bugfix: return the absolute position, rather than + relative c->pos. The return value of these functions becomes the return + value of do_match_line, so the semantics has to agree. + +2012-03-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (sem_error, file_err): Bugfix: pull text formatted source + location using source_loc_str rather than raw object using source_loc. + +2012-03-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Fixed broken functionality. Failing to evaluate + directives following the load and consume input properly. + Check for errors during the parse and throw an exception. + +2012-03-01 Kaz Kylheku <kaz@kylheku.com> + + Fixing two instances of unintentional O(n*n) behavior and poor memory use + that occur in horizontal matching of basic text patterns. + + * lib.c (match_str, match_str_tree): New functions. + + * lib.h (match_str, match_str_tree): Declared. + + * match.c (do_match_line): Use match_str_tree and match_str when matching + strings and string lists, respectively, rather than stupidly calling + search functions and then asserting that the match was found at the + starting position. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (do_match_line): Function takes new argument, "completely". + The check for completely matching a line is now done within + do_match_line. + (match_line): Pass nil to do_match_line, specifying that a prefix + match is okay. + (match_line_completely): New interface to do_match_line, which + requests a match to the end of the line. + (v_freeform): Pass nil to do_match_line: freeform needs + incomplete match semantics. + (match_files): Use match_line_completely instead of match_line. + By doing it this way, we do not need to compute the length of + the original line and compare it to the absolute position. + This saves time and memory since computing the length of a lazy + string forces it. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (consume_prefix): This memory optimization should only + be done for lazy strings, otherwise it just causes unnecessary + memory use by duplicating the line, and inefficiency via + thanks to allocator churn. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (numberp): Fix bad type check: null pointer dereference when + object is nil. + + * match.c (do_match_line): Bugfix for incorrect treatment of long + lines. Must return the absolute position from the start of the original + line (plus(c->pos, c->base)), rather than just c->pos, which only + measures from the start of a line that may have been chopped by + consume_prefix. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: rlcp function was incorrect for new way of storing + line number info. + + * parser.h (rlset): Declared. + (rlcp): Use rlset. + + * parser.y (rlset): New function. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (help, debug): New g command for tweaking log level. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * debug.c: Missing d command implemented. + Condense the output to 8 times the screen width, for more context. + Condense the output in vertical mode (when the entire input line + is shown) not only character mode. + + * lib.c (remq, remql, remqual): New functions. + + * lib.h (remq, remql, remqual): Declared. + +2012-02-29 Kaz Kylheku <kaz@kylheku.com> + + Version 59 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-02-28 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sub_list, replace_list, lazy_sub_str, sub_str, replace_str, + sub_vec, replace_vec): Regression: replace incorrect zerop(to) + test with to == zero, because to is not necessarily a number. + +2012-02-27 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Recognize backslash-newline break in regexes. + +2012-02-27 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Substantially revised for better lexical accuracy + and highlighting of errors. + +2012-02-27 Kaz Kylheku <kaz@kylheku.com> + + * match.c (extract): Check for negative value of opt_nobindings, + which forces the printing. + + * txr.c (help): Under -b, mention that printing the word false is + suppressed also. Added documentation for -B. + (main): Implemented -B option. + + * txr.1: Documented -B option and added clarifying text under -b + option explaining the conditions under which bindings are printed. + +2012-02-26 Kaz Kylheku <kaz@kylheku.com> + + Fix the issue properly, once and for all, of how to determine + that output has taken place and suppress the printing of bindings. + + * debug.c (debug): std_output replaced with std_debug. + + * eval.c (eval_init): Registered new *stddebug* variable. + + * stream.c (std_debug): New variable. + (stdio_put_string): Check that stream is other than + std_debug, to determine that output has taken place. + + * stream.h (std_debug): Declared. + + * txr.1: Added *stddebug* to documentation stub heading. + +2012-02-26 Kaz Kylheku <kaz@kylheku.com> + + Fixing long-time (pre-GIT) bug. The object (nil) was stupidly used to + represent empty optional output clauses, distinguishing them from + missing clauses. This creates an ambiguity, so that an @(output) + block which puts out a single empty line is treated as empty. + Present but empty clauses are now represented by t. + + * match.c (do_output_line): Check for t and bail. + (do_output): Check for t instead of (nil) and bail. + + * parser.y (o_elems_opt2): Nonterminal deleted. + (out_clauses_opt): Empty case generates nil. + (req_parts_opt): o_elems_opt2 replaced by o_elems_opt. + (repeat_rep_helper): Function now keeps track of which + clauses were specified. For those that were specified, but + empty, it substitutes t. + + * tests/008/empty-clauses.expected: New file. + + * tests/008/empty-clauses.txr: New file. + +2012-02-26 Kaz Kylheku <kaz@kylheku.com> + + Bug #35625 + + * parser.l (BSCHR, BSYM, BTOK): New lexical definitions. + (BRACED): New state. + (grammar): Refactored so that braced variables are now handled + in the BRACED state, allowing for lexical differences between + braced variables and Lisp. This allows us to have + the /regex/ syntax in braces, but /regex/ is just a symbol + in the Lisp. The new #/ token is recognized and returned + as HASH_SLASH. All rules reformatted to a more easily + maintainble convention. + + * parser.y (HASH_SLASH): New token. + (modifiers, lisp_regex): New nonterminals. + (var): Grammar changed to use modifiers nonterminal instead of exprs. + (var_op): Rule moved closer to var. + (expr): Produces lisp_regex rather than regex. + (yybadtoken): Handle HASH_SLASH in the switch statement. + Bugfix: HASH_BACKSLASH was not handled. + + * txr.1: Documented #/regex/ syntax. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Updated copyright year. + + * arith.h: Likewise. + + * debug.c: Added copyright header. + + * debug.h: Updated copyright year. + + * eval.c: Likewise. + + * eval.h: Likewise. + + * filter.c: Likewise. + + * filter.h: Likewise. + + * gc.c: Likewise. + + * gc.h: Likewise. + + * hash.c: Likewise. + + * hash.h: Likewise. + + * lib.c: Likewise. + + * lib.h: Likewise. + + * match.c: Likewise. + + * match.h: Likewise. + + * parser.h: Likewise. + + * regex.c: Likewise. + + * regex.h: Likewise. + + * stream.c: Likewise. + + * stream.h: Likewise. + + * txr.c: Likewise, and e-mail address. + + * txr.h: Updated copyright year. + + * unwind.c: Likewise. + + * unwind.h: Likewise. + + * utf8.c: Likewise. + + * utf8.h: Likewise. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * tests/008/filtenv.expected: New file. + + * tests/008/filtenv.txr: New file. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + Bug #34652 + + * match.c (do_txeval): Establish a dynamic env frame around + evaluation of quasiliteral and around embedded TXR Lisp expression + (which may contain quasiliterals) and stick the bindings there + via set_match_context. This way if filte functions are invoked through + a quasiliteral, they can see bindings. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New put-byte function interned. + + * stream.c (struct strm_ops): New member, put_byte. + (stdio_put_byte): New function. + (stdio_ops, pipe_ops, string_in_ops, byte_in_ops, + string_out_ops, strlist_out_ops, dir_ops): Updated. + (put_byte): New function. + + * stream.h (put_byte): Declared. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (num_esc): Skip octal indicator 'o' if present. + This is needed for character constants. + (CHRLIT): Fix broken hex constants, being treated as octal. + + * txr.1: Document octal character constants. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + Version 58 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + + * eval.c (eval_init): Updated. + + * rand.c (rand): Function renamed to rnd due to a clash with + standard C library (that does not show up when compiling as C++). + + * rand.h (rand): Declaration renamed to rnd. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.l (source_loc_str): Missing function re-written. + (Accidentally originally written in lex.yy.c file!) + (parse_reset): If file can't be opened, try adding .txr suffix + to it. + + * txr.1: Suffix-handling behavior documented. + +2012-02-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (clause): "Doh" moment. We don't need the special + transformation of the load syntax because the parent location + is already associated with the syntax. + + * match.c (v_load): Pull out source location info from + the form itself. + +2012-02-24 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (debug): Use new way of getting line number. + + * eval.c (eval_error): Use source_loc_str to get source location. + + * match.c (debuglf, sem_err, file_err): Likewise. + + * parser.h (source_loc_str): Declared. + + * parser.l (parse_init): form_to_ln_hash must be equal based now. + + * parser.y (rl): Store new form of read-time source location info. + + * txr.1: Documented load. + +2012-02-24 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_load): Sanity checking on target path. + Check if it is absolute and do not substitute parent + file's directory. + +2012-02-24 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_var): Eliminated uses of rl function, in favor of + rlcp. Only the parser should use rl to establish location info. + +2012-02-24 Kaz Kylheku <kaz@kylheku.com> + + First cut at @(load) directive. Incomplete: debug location info + needs to record file name, not only line number; absolute paths + not handled, etc. + + * match.c (load_s): New symbol variable. + (v_load): New static function. + (syms_init): load_s initialized. + (dir_tables_init): Load directive registered. + + * match.h (load_s): Declared. + + * parser.h (parse_reset): New function declared. + + * parser.l (spec_file_str): Global variable moved from txr.c. + (parse_reset): New function. + + * parser.y (clause): Special handling for @(load ...) directive. + parent file path inserted into the syntax at parse time, + so when the load directive executes, it can load the file from + the same directory as the parent file. + + * txr.c (spec_file_str): Global variable moved to parser.l. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * tests/010/seq.txr: New file. + + * tests/010/seq.expected: New file. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sub_list, replace_list, lazy_sub_str, sub_str, + replace_str, sub_vec, replace_vec): Implementing floating zero + behavior. This is necessary so that the important case index .. (+ + index 1) for selecting one element as a subrange works even if index is + negative. + + * txr.1: Floating zero behavior for ranges documented. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc): del operator's return value is the + deleted range of values. Missing del cases added for single + index cases. + + * lib.c (quicksort): Fix pivot selection one more time. + (length): Fix bad argument in "not a sequence" error. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (quicksort): Incorrect pivot selection. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (del_s): New symbol variable. + (eval_error): nostatic added to suppress a g++ warning. + (dwim_loc): Takes new value as the original unevaluated form so + it can control evaluation. Support for del operator added. + (op_modplace): Bugfixes: new value form was being unconditionally + evaluated and out of order w.r.t. other forms. Now there is + left to right order. The form is evaluated later, since for most + operators it is on the right, with the exception of push. + dwim places can now optionally control the evaluation of newform. + del operator supported in gethash case as a special case. + error case added for del: if it's not handled in gethash or + dwim_loc, it is an error. + (eval_init): del_s initialized, and del operator added to table. + + * txr.vim: syntax highlighting for del. + + * txr.1: Documented del. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (init): Hash bugfix: added missing call to hash_init. + Previously this function did nothing, but now it interns + some important symbols. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (remhash): Rewrote buggy function. + It was decrementing the hash count without checking that + something was deleted from the chain. The deletion was done + incorrectly, without regard for the kind of comparison fucntion + used by the hash table. + +2012-02-22 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Intrinsic bindings for sub, ref, refset + and replace. + + * lib.c (do_sort): Static function renamed to sort_list. + (swap, quicksort, sort_vec): New static functions. + (sort): Made generic over lists, vectors and strings. + (refset): New function. + + * lib.h (sort): Declaration updated (parameter name change). + (refset): Declared. + + * txr.1: Mention refset. + + * txr.vim: Updated with refset. + +2012-02-21 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Third argument of gethash becomes optional. + +2012-02-21 Kaz Kylheku <kaz@kylheku.com> + + Introducing optional arguments. + + * debug.c (help, show_bindings): put_string arguments reversed. + + * eval.c (bind_args): Support colon notation in interpreted + function lambda lists for optional arguments. Improved error checking. + (apply): Allow optional arguments to be left out. + (dwim_loc): Reversed arguments to replace_str, replace_vec, + replace_list. + (eval_init): Numerous intrinsics now have arguments that are optional. + New function rand introduced which reverses arguments relative to + random. New intrinsic function hash introduced for alternative + construction of hashes. + + * gc.c (sweep): Reversed arguments to put_char. + + * hash.c (weak_keys_k, weak_vals_k, equal_based_k): New keyword + symbol variables. + (hashv): New function. + (hash_init): Intern new symbols. + + * hash.h (weak_keys_k, weak_vals_k, equal_based_k, hashv): Declared. + + * lib.c (colon_k): New keyword symbol variable. + (replace_list, replace_str, replace_vec): Arguments rearranged. + (tree_find): testfun becomes optional argument. + (int_str): base becomes optional argument. + (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, + func_n1, func_n2, func_n3, func_n4, func_f0v, func_f1v, + func_f2v, func_f3v, func_f4v, func_n0v, func_n1v, + func_n2v, func_n3v, func_n4v, func_interp): Initialize optargs to zero. + (func_n0o, func_n1o, func_n2o, func_n3o, func_n4o): New functions. + (cobj_print_op): Reversed arguments to put_string. + (find): testfun and keyfun become optional arguments. + (replace): Parameters rearranged and arguments rearranged in calls to + replace_list, replace_str and replace_vec. + (obj_init): colon_k initialized. + (obj_print, obj_pprint): Arguments reversed, and stream defaults + to std_output. Arguments reversed in calls to put_char and put_string. + (dump): Arguments reversed in call to put_char. + + * lib.h (struct func): sizes of minparam, fixparam bitfields + adjusted. New bitfield optargs. New unnamed bitfield added so + the previous ones add up to 16 bits. + (colon_k): Declared. + (func_n0o, func_n1o, func_n2o, func_n3o, func_n4o): Declared. + (replace_list, replace_str, replace_vec, replace): Declarations updated. + + * match.c (debuglf, dump_shell_string, dump_byte_string, dump_var, + do_output_line, extract): Reversed arguments to put_char and + put_string. + + * parser.l (yyerrorf): Reversed arguments to put_char. + (grammar): Recognize : symbol in NESTED state. This is a keyword + symbol with the empty string as its name. + + * rand.c (rand): New function. + + * rand.h (rand): Declared. + + * stream.c (strlist_out_put_string): Reversed arguments to put_string. + (strlist_out_put_char): Reversed arguments to put_char. + (get_line, get_char, get_byte): Stream defaults to std_input. + (vformat_num, vformat_str, vformat): Reversed arguments to put_char. + (put_string, put_char, put_line): Arguments reversed; stream defaults + to std_output. + + * stream.h (put_string, put_line, put_char): Declarations updated. + + * txr.c (oom_realloc_handler): Arguments to put_line reversed. + + * unwind.c (uw_throw): Likewise. + + * txr.1, txr.vim: Updated. + +2012-02-19 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Corrected misleading text in description of output + variables. Lists do not have to be wrapped in @(repeat)/@(rep). + Indexing notation works on strings and vectors. + +2012-02-19 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (sub, ref, replace): New functions. + + * lib.h (sub, ref, replace): Declared. + + * match.c (format_field): Generic indexing using new functions. + + * txr.1: Documentation stub. + + * txr.vim: Highlighting for new functions. + +2012-02-18 Kaz Kylheku <kaz@kylheku.com> + + * match.c (extract_vars): If a brace var is actually an + expression, then recurse into the expression to find + variables. This allows us to do things like @{`@x`} inside + a @(repeat) or @(rep). + +2012-02-18 Kaz Kylheku <kaz@kylheku.com> + + Allow braced output variables to actually be arbitrary substitutions. + + * eval.c (subst_vars): Treat the variable as an arbitrary + expression rather than just a symbol. + + * match.c (subst_vars): Likewise. + + * parser.y (o_var): Further simplification. The first item in + an output var is an expr and not an IDENT. + + * txr.1: Updated. + + * txr.vim: Likewise. + +2012-02-18 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (o_var): Remove productions from grammar by using exprs_opt. + +2012-02-17 Kaz Kylheku <kaz@kylheku.com> + + Reimplementation of how TXR decides whether to dump bindings + or not. This is now done right inside the standard output stream. + + * match.c (output_produced): Variable removed. + (complex_open): Assignment to output_produced removed. + + * stream.c (output_produced): New global variable. + (stdio_put_string, stdio_put_char): Set output_produced + to t if the target of the output is stdout. + + * stream.h (output_produced): Declared. + + * txr.h (output_produced): Declaration removed. + +2012-02-17 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc): 2012-02-13 fix didn't do it. + Here is a simpler way. We can let nil be treated as a list with a + simple goto. This solves all cases. + +2012-02-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Register match-fun. + + * match.c (v_do): Store match context. + (match_fun): New function. + + * match.h (match_fun): Declared. + + * stream.c (streamp): New function. + + * stream.h (streamp): Declared. + + * txr.1: Stub section for match-fun. + + * txr.vim: Highlight match-fun. + +2012-02-16 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_funcall): Function renamed to match_filter. + + * match.h (match_funcall): Declaration updated. + + * filter.c (get_filter): Updated. + +2012-02-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_catch): Treat the nil exception object as an empty + list, and not the atom nil. + + * match.c (v_try): Do not transform an atomic exception x into + the list ((t . x)). Just to (x). This (t . x) representation is + a vestige of from the time when the exception consisted of the + old-style return values from txeval. + +2012-02-16 Kaz Kylheku <kaz@kylheku.com> + + TXR Lisp gets exception handling. + + * eval.c (op_catch, expand_catch_clause, + expand_catch): New static functions. + (expand): Handle catch operator. + (eval_init): Added catch operator to op_table, + and registered new functions throw, throwf and error. + + * txr.1: Documented catch operator and created + stub section for throw, throwf and error. + + * txr.vim: Updated. + + * unwind.c (uw_throwfv, uw_errorfv): New functions. + + * unwind.h (uw_throwfv, uw_errorfv): Declared. + +2012-02-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rangev_func, rangev, range_star_v_func, + range_star_v): Regression: handle the case where the range is + open-ended (to is nil). + +2012-02-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rangev): If a descending range is specified, + but the step is omitted, the step should be negative one + rather than one. + (range_star_v_func, range_star_v): New static functions. + (eval_init): New function, range*, registered. + + * txr.1: Stub section for range is also for range*. + + * txr.vim: Recognize range* function. + +2012-02-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rangev_func): Improved termination test. The sequence + not only if it achieves the endpoint, but if it crosses it. + +2012-02-13 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc): Allow assignment to the dwim place when it + evaluates to nil, by building a form targetting the place, + and recursing into op_modplace. Also, optimization: don't use eval + on a form built from the operator; just go to op_modplace directly. + +2012-02-13 Kaz Kylheku <kaz@kylheku.com> + + Version 57 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-02-13 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars): Handle lone symbols (not wrapped + in sys::var form) as variables. This allows interpretation of metanums + in quasiliterals, because these are replaced by variables in the + op expander. + + * parser.l: Recognize METANUM in QSILIT state. + + * parser.y (quasi_item): METANUM parsed in quasiliteral. + + * txr.vim: Metanums highlighted properly. + +2012-02-13 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc, op_dwim): Bugfix: hash indexing requiring + at least two arguments, rather than at least one. + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Added note about register save areas, which can contribute + to spurious retention. + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (breakpoints, last_command): linkage changed to static. + (cols): New static variable. + (debug): Print context intelligently, fitting into the width of + the screen. + (debug_init): Try to get terminal width, from the COLUMNS variable. + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (lazy_flatten): Bugfix: function was assuming that + the input is a list, and not handling the case + atom -> (atom) like its non-lazy counterpart. + This broke @(next :list expr). + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (obj_print): Print control characters in string and + character literals as hex escapes, followed by semicolon if + necessary. Don't use iswprint function since it is locale-specific + and concludes that non-ASCII characters are unprintable. + Changed print syntax for lazy strings. + (obj_pprint): Changed print syntax for lazy strings. + + * parser.l: Bugfix in hex/octal character constant. + num_esc(yytext) was called rather than num_esc(yytext+1). + + * parser.y (chrlit): Bugfix: missing case for hex and octal + constants which are given by a LITCHAR token. + + * stream.c (vformat): Bugfix: strings were being printed as if using ~a + even under ~s. + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + Task #11486 + + * match.c (h_coll): Call consume_prefix in the loop. + +2012-02-12 Kaz Kylheku <kaz@kylheku.com> + + Task #11486: continuation. Take into account base position + in debug reporting. + + * debug.c (debug): New argument, base. + + * debug.h (debug_check): New argument, base. + (debug): Declaration updated. + + * eval.c (do_eval): Pass new argument of debug_check as nil. + + * match.c (LOG_MISMATCH, LOG_MATCH): Take into account base + when displaying character position. + (do_match_line): Pass base position to debug_check. + (match_files): Pass nil as base to debug_check. + +2012-02-11 Kaz Kylheku <kaz@kylheku.com> + + Task #11486: continuation. Now able to @(skip) through + a large file horizontally under @(freeform) in constant memory. + Position reporting is incorrect. + + * match.c (match_line_ctx): New member, base. + (ml_all): Initialize new member to zero. + (consume_prefix): New static function. + (do_match_line): New static function, contains implementation of + match_line. Calls consume_prefix. + (match_line): Becomes a wrapper for do_match_line. + (search_form, h_skip): Call consume_prefix. + (freeform_prepare): New function. External to avoid inlining. + (v_freeform): Uses do_match_line, to down context by reference, + and also refactored to avoid spurious retention of c->data. + + * unwind.c (uw_push-block, uw_push_env, uw_push_debug, + uw_push_catch): Zero out the structure, otherwise the unused + fields can cause spurious retention. + +2012-02-09 Kaz Kylheku <kaz@kylheku.com> + + Task #11486: preparatory refactoring. + + * match.c (h_match_func): Changed to one-argument form. + (LOG_MISMATCH, LOG_MATCH): References to structure members + changed from member selection . to ->. + (h_text, h_var, h_skip, h_coll, h_parallel, h_trailer, + h_fun, hv_trampoline, h_define, h_do): Changed to one-argument form + that takes a pointer to the context. + (match_line): Local pointer named c used so that the + LOG_MISMATCH and LOG_MATCH macros can be used. + +2012-02-09 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (lazy_sub_str): New static function. + (sub_str): Bugfix: handle t values of from from and to, and negative + indices, just like sub_vec. Special handling for lazy strings + introduced. If to is the value t, then the a lazy string can be + produced. + +2012-02-05 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Updated. + + * txr.vim: Fixed mistakes in a few regex character classes, + causing upper case letters not to be recognized as part + of a variable name, and character hex constant. + +2012-02-05 Kaz Kylheku <kaz@kylheku.com> + + Version 56 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-02-05 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Allow semicolon terminator on hex and octal + escapes in a regex. Removed o escape character from octal + constants in strings and quasiliterals: no such thing is documented. + Octal constants can be semicolon-terminated like hex ones. + + * txr.1: Documented semicolons after octal constants in + string literals and after hex and octal constants in regexes. + +2012-02-05 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (utf8_from_uc, utf8_decode): Some cascaded if tests converted + to a switch on the upper nybble value. This also fixes an unfortunate + bug. The test for the two byte case was written as + ch >= 0xc2 && ch <= 0xE0. That should have been ch < 0xE0. + Versions of TXR up to 55 have been incorrectly decoding some UTF-8. + +2012-02-04 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (utf8_from_uc): Bugfix: incorrect condition in character + range check (less than minimum *and* U+DCxx, rather than *or*). + Also, we must check for out of range characters. UTF-8 sequences + beginning with F4 can code beyond 0x10FFFF. + (utf8_decode): Check for characters beyond 0x10FFFF. + +2012-02-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (rest_s, op_s): New variables. + (do_eval_args): Allow calls specified by improper lists + like (x y . z) where the z expression must evaluate to a list + that turns into addition arguments to be applied. + (transform_op, expand_op): New static functions. + (expand): Call expand_op. + (eval_init): Initialize rest_s and op_s. Use rest_s + to register rest function. + + * lib.c (gensym): New function based on gensymv. + (gensymv): Now calls gensym. + + * lib.h (gensym): Declared. + + * parser.l: Parse @ followed by digits as a new kind of token, + METANUM. + + * parser.y (METANUM): New token. + (meta_expr, exprs): Missing rlcp's added. + (expr): METANUM variant introduced. + (yybadtoken): Handle METANUM. + + * txr.1: Documented one-symbol argument list of lambda. + Documented op. Closed some unbalanced parentheses. + + * txr.vim: Highlight op. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (utf8_from_uc, utf8_decode): Use upper case for hex constants. + If bytes decode to U+DCxx, treat this sequence as invalid. This + way we can't be fooled by an attacker into accepting some U+DCxx which + on output we will then convert to byte xx. + (utf8_to_uc): Use upper case for hex constants. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (utf8_to_uc, utf8_encode): Do not encode surrogate code + points (U+DC00 to U+DCFF) as multi-byte UTF8 sequences. We use + that range for invalid bytes on input, so on output the best thing + to do is to reproduce the original bytes. E.g the code U+DCA0 + will produce the byte A0. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: UTF-8 handling clarified. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * utf8.c (utf8_from_uc, utf8_decode): Impose a minium value on the + decoded character based on which UTF-8 case it is from. This rejects + overlong forms. + + * utf8.h (struct utf8_decoder): New member, wch_min. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Document that TXR accepts UTF-8 overlong forms. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Move error match before other cases and simplify. + Comment are colorized properly again. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + This test case would have caught the prior regression. + + * Makefile (TXR_ARGS): Defined for new test case. + + * tests/010/align-columns.dat: New file. + + * tests/010/align-columns.expected: New file. + + * tests/010/align-columns.txr: New file. + +2012-02-02 Kaz Kylheku <kaz@kylheku.com> + + * match.c (format_field): Regression. Was not handling modifiers + which are compound expression (e.g. computed field widths). + This was broken by the first commit of 2012-01-26. + +2012-02-01 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (lookup_sym_lisp1): New function. + (do_eval, do_eval_args): New static functions. + (eval, eval_args): Become wrappers for do_eval and do_eval_args, + respectively. + (eval_lisp1, eval_args_lisp1): New static functions. + (dwim_loc, op_dwim): Use eval_lisp1 and eval_args_lisp1 instead + of eval and eval_args. + + * parser.y (meta_expr): Bugfix: expand the whole dwim expression, + rather than its arguments, which are not an expression. + + * txr.1: Updated with notes that dwim really does Lisp-1 style + evaluation. + +2012-01-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: More color categories. More accurate matching of @ elements. + +2012-01-27 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Support hex and octal escapes in string and quasiliterals, + as the documentation says. Also support an optional trailing ; + delimiter in hex escapes. + + * txr.1: Documented. + +2012-01-27 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Properly show @[...] inside quasiliteral. + +2012-01-27 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc, op_dwim): Relax the requirement on hash + indexing arguments. The default value can be omitted, + defaulting to nil. + +2012-01-27 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_begin): Missing type check added, otherwise + passing in an object that is not a hash results in corruption + or crashing behavior. + + * lib.c (class_check): Improved to a one-step check with a clear + message. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + Version 55 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_modplace): Fix warning caught by gcc, but not g++. + + * lib.c (replace_list): Fix code that is valid C++ but not C. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (replace_list): Always convert the input items to a list, + even in the trivial case that an empty list is being replaced. + Allow a string to be the replacement (split into a list of + characters). + (replace_str): Bugfix in assignment from vector; wrong index + used over source vector. + (split_str): If the splitting set is empty, just split the + string into characters instead of getting into an infinite loop. + (replace_vec): Allow replacement source to be a string. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus, minus): Better wording in error messages. + + * eval.c (dwim_loc): Assignments to string indices and ranges + supported. New arguments for this purpose. + (op_modplace): Use new dwim_loc interface for returned value. + (op_dwim): Support assignment to string ranges. + (eval_init): replace_str registered. + + * lib.c (string_extend): If the argument is a number, let it + specify the amount by which to extend the string. + (replace_str): New function. + + * lib.h (replace_str): Declared. + + * txr.1: Updated. + + * txr.vim: Updated. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (listref, listref_l): Negative indices must have semantics + consistent with vecref and ranges. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (cat_str): Throw error if one of the list elements + is not a character or string instead of silently returning nil. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: More discussion of ranges. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * match.c (format_field): Removed useless use of cat_str (no longer + needed because str is already a string). The purpose was to reduce + character to strings. + (subst_vars): Some of the new logic in format_field must be replicated + in the case when format_field is not called because the variable + has no modifiers. Lists must be converted to a space-separated string. + Bugfix here: occurence of pat and modifiers is not mutually exclusive. + +2012-01-26 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc, dwim_op): Eliminated redundant re-evaluation + of range arguments. They are already evaluated since the cons + expression is evaluates as part of the dwim arglist. + Replaced some open code with function calls to the new listref + and listref_l functions. + (tostring, tostringp): made extern and moved to lib.c. + + * lib.c (listref, listref_l): New functions. + (tostring, tostringp): moved here from eval.c. + + * lib.h (listref, listref_l, tostring, tostringp): Declared. + + * match.c (format_field): Handle index and range references. + + * txr.1: Documented new output variable syntax. + +2012-01-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_loc): Handles full responsibility for assigning + to list and array ranges. + (op_modplace): Pass extra arguments to dwim_loc so it can + do the job for ranges. If dwim_loc returns 0, it means + that it did everything. + (op_dwim): Support list and array ranges. + + * txr.1: Documented. + +2012-01-25 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (zerop): Misspelling in error message. + + * lib.c (sub_list, replace_list, sub_vec, replace_vec): + Allow the value t to specify one element past the end, so that t t + refers to zero-length sequence just past the end of the array or list. + Also, fixed out of bounds memmoves in replace_vec. + +2012-01-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions registered. + + * lib.c (sub_list, replace_list, vectorp): New functions. + (sub_vec): Allow negative indices from end of array. + (replace_vec): New function. + + * lib.h (sub_list, replace_list, vectorp, replace_vec): Declared. + + * parser.l (DOTDOT): Scan .. as new token. + + * parser.y (DOTDOT): New token. + (expr): New syntax with DOTDOT. + (yybadtoken): Handle DOTDOT. + + * txr.vim: Added new functions. Also missing append* and dwim. + + * txr.1: Updated. + +2012-01-25 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim (txr_chr): Fix for highlighting named characters + like #\newline. + +2012-01-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dwim_s): New symbol variable. + (dwim_loc, op_dwim): New static functions. + (op_modplace): Support assignment to dwim forms + with the help of dwim_loc. + (expand_place): Handle dwim places. + (eval_init): Initialize dwim_s. Register dwim operator + in op_table. + + * eval.h (dwim_s): Declared. + + * lib.c (chr_str, chr_str_set): Allow negative indices to index + backwards from end of string. + (vecref, vecref_l): Allow negative indices to index from + rear of array. + (obj_print, obj_pprint): Render (dwim ...) forms as [...]. + + * parser.l: Peoduce new METABKT token type for @[, + and '[', ']' tokens. + + * parser.y (METABKT): New token. %type declaration for '['. + (list): Support square-bracket style of list, translated + into dwim form. + (meta_expr): Support @[...] variant. + (yybadtoken): Handle METABKT in switch. + + * txr.1: Documented [...] syntax and dwim operator. + + * txr.vim: Updated. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + Version 54 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (help): Added missing help for w command. + (debug): In backtrace, show the renaming pairs for unbound + variables (up_p_a_pairs) if they are present. + + * debug.h (debug_begin): Renamed to debug_frame. + + * eval.c (eval): Wrap debug_begin/debug_end around function dispatch, + so TXR Lisp functions are included in backtraces. + + * match.c (h_fun): Follow rename of debug_begin to debug_frame. + Pass in evaluated args, not the original ones. + (v_fun): Likewise. + + * unwind.c (uw_push_debug): bindings argument renamed to env. + Bugfix: args argument was being assigned to ub_p_a_pairs. + + * unwind.h (struct uw_debug): Member bindings renamed to env. + (uw_push_debug): Declaration updated. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (last_command): Do not initialize with lit(); + this is not a constant expression in C. + (debug): Handle the situation here. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (help): Filled in. + (debug): Some commands changed due to duplicates. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_fun): Removing all debugging instrumentation. + (match_files): Moving debug_check out of directive case so + it covers all forms handled by loop. All this makes the n + command in the debugger work better: not skip over + function calls or horizontal material. + +2012-01-21 Kaz Kylheku <kaz@kylheku.com> + + Improved debugging. Debug nesting depth counter maintained + and used for next/step/finish stepping. + + * Makefile (OBJS): debug.o moved to OBJS-y or OBJS-. + (OBJS-y, OBJS-): New variables. + $(PROG): Depends on OBJS-y also. + clean: clean $(OBJS-y). + depend: include $(OBJS-y) in dependency generation. + + * configure: Underscores and dashes are interchangeable in + configure variables. + (yaccname_given, yacc_given): Default value is y, not yes. + (debug_support): New config variable. + (CONFIG_DEBUG_SUPPORT): New config.h symbol. + + * debug.c (debug_depth): New global variable. + (debug_block_s): New symbol variable. + (next_depth): New static variable. + (debug): Renamed some commands. Introduced separate next, step + and finish. + (debug_init): debug_block_s initialized. + + * debug.h (debug_depth, debug_block_s): Declared. + (debug_enter, debug_leave, debug_return): New macros. + (debug_check, debug_init): Conditionally defined based on + if this is a debug build. + + * dep.mk: Regenerated. + + * eval.c (eval): Instrumented with debug_enter, debug_leave, + debug_return. + + * match.c (match_line, v_fun, match_files): Likewise. + + * txr.c (txr_main): Bail if -d or --debug used in build + that lacks debug support. + +2012-01-19 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (last_command): Initialize to empty string rather + than nil, otherwise hitting enter tries to repeat the nil command. + (show_bindings): New function. Prints all levels of bindings. + (debug): Flip the corresponding print flags after printing the current + form or data, so they are not printed for every prompt. + On EOF from standard input, substitute the q command. + If enter is hit and there is no last command, just re-print the prompt. + The v command uses show_bindings to dump the environment. + + * eval.c (eval): When calling debug_check, pass the env objects, rather + than the bindings it contains. + +2012-01-19 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (car_l, cdr_l): Bugfix: do not call the lazy cons + force function if it is already nil, and set it to nil afterward. + +2012-01-12 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Make lazy_appendv function + available as append*. + + * txr.1: Documented. + +2012-01-11 Kaz Kylheku <kaz@kylheku.com> + + Before releasing 53, there is this. + + * eval.c (c_var_mark): Bugfix: we cannot use cptr_get from + within the garbage collector because of its type check. + Bugfix: synchronize the shadow binding with the variable's + current contents so we don't hang on to a stale object. + +2012-01-11 Kaz Kylheku <kaz@kylheku.com> + + Version 53 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2012-01-11 Kaz Kylheku <kaz@kylheku.com> + + TXR Lisp regression in C global variables. + + * eval.c (struct c_var): New struct type. + (lookup_var, lookup_var_l): cptr type bindings now point to a struct + c_var, which has to be handled properly here. + (c_var_mark): New static function. + (c_var_ops): New static struct. + (reg_var): Register variables using struct c_var to provide + a pointer to the location and a cached cons that can be + returned as a binding. + +2012-01-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (each_s, each_star_s, collect_each_s, collect_each_star_s): + New symbol variables. + (op_each): New static function. + (expand): Handle the four new operators. + (eval_init): Intern new symbols, register new operators. + + * txr.1: Documented each, each*, collect-each and collect-each*. + + * txr.vim: Updated. + +2012-01-11 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): list_str registered. + + * lib.c (list_str): New function. + + * lib.h (list_str): Declared. + + * txr.1: Doc stub section created. + + * txr.vim: Updated. + +2012-01-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (generate): Bugfix: do not call gen_fun before + testing while_pred. + +2012-01-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (tostring, tostringp): New static functions. + (eval_init): New functions registered. + + * txr.1: Stub sections created. + + * txr.vim: Updated. + +2012-01-10 Kaz Kylheku <kaz@kylheku.com> + + Spat of new features having to do with lazy processing. + + * eval.c (prog1_s, gen_s, generate_s, delay_s, promise_s): New symbol + variables. + (eval_prog1, op_prog1, expand_gen, expand_delay): New static functions. + (expand): Handle gen and delay. + (lazy_mapcar_func, lazy_mapcar, lazy_mapcarv_func, lazy_mapcarv, + lazy_mappendv): New static functions. + (rangev_func, rangev, generate_func, generate, repeat_infinite_func, + repeat_times_func, repeatv, force): New static functions. + (eval_init): New operators and functions interned. + lazy-flatten renamed to flatten*. + + * lib.c (null_f): New global variable. + (ltail, lazy_appendv): New functions. + (lazy_appendv_func): New static function. + (obj_init): null_f protected and initialized. + + * lib.h (null_f, ltail, lazy_appendv): Declared. + + * txr.1: Documented. + + * txr.vim: Updated. + +2012-01-09 Kaz Kylheku <kaz@kylheku.com> + + Non-broken way to achieve intent of previous commit. + + * eval.c (subst_vars): Do not evaluate modifiers + as an argument list locally. Pass form-evaluating + function to format_field. + + * match.c (format_field): Modified to accept new argument, + a one-argument function for reducing a form to a value. + Error checking for invalid modifiers made stricter. + (subst_vars): Do not evaluate modifiers as an argument + list. Pass form-evaluating function to format_field. + + * match.h (format_field): Declaration updated. + +2012-01-09 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars): Evaluate the + modifiers, so expressions can be used. + + * match.c (subst_vars): Likewise, + but using txeval. + +2012-01-07 Kaz Kylheku <kaz@kylheku.com> + + Version 52 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. Wrong December dates fixed. + +2012-01-06 Kaz Kylheku <kaz@kylheku.com> + + * match.c (fuzz_s): New symbol variable. + (v_fuzz): New static function. + (syms_init): fuzz_s initialized. + (dir_tables_init): v_fuzz entered into v_directive_table. + + * txr.1: Documented @(fuzz). + +2012-01-06 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_gather): Implemented until/last clause. + + * parser.y (gather_parts, additional_gather_parts): New nonterminals. + (gather_clause): Syntax refactored for until/last clause. + + * txr.1: Updated. + +2012-01-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Fix regression introduced in + 2011-12-29 commit. We can't use mod_s, because the module + which sets up that variable is not yet initialized. + +2012-01-01 Kaz Kylheku <kaz@kylheku.com> + + Make C globals in TXR Lisp properly assignable, so that for instance + assigning *stdout*, it really overwrites the underlying C variable. + + * eval.c (lookup_var): Handle new kind of toplevel binding. + If the hash value is a cptr, it points to a val storage location. + (lookup_val_l): New function. + (op_modplace): Get location of variable using lookup_val_l + rather than assuming there is a cons-based binding. + (reg_var): Argument changed to val * pointer. + Register the variable as a cptr referencing the location. + (eval_init): reg_var calls pass address of each global. + + * eval.h (lookup_var_l): Declared. + +2012-01-01 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New gensym function registered. + + * lib.c (gensym_counter): New variable. + (gensymv): New function. + (obj_init): Initialize gensym_counter. + + * lib.h (gensym_counter, gensymv): Declared. + +2011-12-30 Kaz Kylheku <kaz@kylheku.com> + + * match.c (counter_k): New keyword symbol variable. + (do_output_line): Process new :counter argument of rep. + (do_output): Ditto, for repeat. + (syms_init): Intern new keyword symbol. + + * match.h (counter_k): Declared. + + * parser.l (REPEAT, REP): Lexical syntax changed to + allow arguments. + + * parser.y (repeat_rep_helper): Takes extra argument, representing + the repeat/rep args. This is inserted into the second position + of the output list. + (repeat_clause, rep_elem): Extract repeat/rep arguments and + pass to repeat_rep_helper. + (yybadtoken): Do not put quotes around the word "number". + + * txr.1: Updated. + +2011-12-29 Kaz Kylheku <kaz@kylheku.com> + + New functionality: mod and modlast directives in repeat and rep. + + * eval.c (eval_init): Use new symbol variable mod_s instead + of calling intern. + + * match.c (mod_s, modlast_s): Symbol variables defined. + (do_output_line): mod and modlast directives implemented under rep. + (do_output): likewise under repeat. + (syms_init): Initialize new symbol variables. + + * match.h (mod_s, modlast_s): Declared. + + * parser.l (MOD, MODLAST): Parse new token types. + + * parser.y (MOD, MODLAST): New tokens. + (repeat_parts_opt, rep_parts_opt): New syntax. + (repeat_rep_helper): Handle mod and modlast syntax. + + * txr.1: Updated. + + * txr.vim: Updated. + +2011-12-29 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (repeat_rep_helper): Bugfix. Circular lists + were being created here when clauses of the same kind appear multiple + times. The problem is that append2 no longer copies the second list, + which the code was relying on it to do. + +2011-12-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Useless sentence under reduce-left and reduce-right removed. + Missing Description headings added. + +2011-12-28 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Updated for recent man page changes. + +2011-12-28 Kaz Kylheku <kaz@kylheku.com> + + Version 51 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-28 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Capitalize TXR where it makes sense. + Introductory text rewritten. + +2011-12-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (LOG_MATCH): Use < in format directive instead of -. + + * rand.c (random): Add back missing declaration. + +2011-12-28 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (quasi_item): Switch from var to o_var. This fixes + cases like `@a@(foo)@b` where foo was being translated + to (foo) rather than (sys:expr foo). + +2011-12-27 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/shrink-mpi-int (mpi_int): Fixed terrible bug in this + patch, resulting in an insufficient bit field width for representing + the allocation size of the MPI integer on 32 bit platforms. + +2011-12-27 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (make_state): Use ANSI C syntax for prototyped function of no + arguments. This snuck through due to working with a C++ compiler. + (random): Fixed unused variable warning that happens on 32-bit-pointer + platforms. + +2011-12-25 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Formatting fixes. + +2011-12-25 Kaz Kylheku <kaz@kylheku.com> + + * dep.mk: Overdue update. + +2011-12-25 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_next): Change flatten to lazy_flatten in the + correct place. In the previous commit I did it in the code that handles + the obsolescent :var syntax. + +2011-12-25 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New function interned. + + * lib.c:x (lazy_flatten_scan, lazy_flatten_func): New static functions. + (lazy_flatten): New function. + + * lib.h (lazy_flatten): Declared. + + * match.c (v_next): Use lazy_flatten instead of flatten for + processing a :list source. This means that @(next :list ...) + can be used to process infinite lazy lists. + + * txr.1: Documented lazy-flatten. + +2011-12-23 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (rand32): Moved. + (make_random_state): After initializing, retrieve eight + random numbers to clear pathological initial behavior + leading to duplicate values. + +2011-12-23 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (highest_bit): Changing to external linkage. + + * arith.h (highest_bit): Declared. + + * rand.c (random): Rewrote using different algorithm which + ensures even distribution, and avoids doing a bignum mod + operation. + +2011-12-23 Kaz Kylheku <kaz@kylheku.com> + + Version 50 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-23 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (memql): New function. + (some_satisfy): Return the first non-nil result, rather than t. + (all_satisfy): Return the value of the last item, if all items + are processed. + + * lib.h (memql): Declared. + + * txr.1: Documented memq, memql, memqual, tree-find, some, + all, none, eq, eql and equal. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented copy-list, reverse, nreverse, ldiff and flatten. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented reduce-left and reduce-right. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + Bug #35010 + + * match.c (extract_bindings): Make sure there are no duplicate + variables among the extracted bindings. This is needed because + of the other changes. + (do_output_line, do_output): In handling the rep/repeat directives, + append the original bindings to the extracted set bindings for the + variables which just occur in the clause, so that Lisp code can see all + of the variables. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): If width is specified for ~s or ~a, and + the object is not a string or number, then print it to a string + and treat it as a string, adjusting it within the field. + Also, do not simply abort on an unknown format directive + but throw a proper exception. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Left-adjusted field is now specified + using < rather than '-'. The +, space and leading 0 are + specified on the precision, not the width. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (random): Fix for 64 bit fixnums: stick two random numbers + together. Otherwise for fixnum moduli, we get only a 32 bit number no + matter what the modulus is. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (vformat): Combine ~a and ~s cases, so numbers and + strings are printed the same way under ~s and ~a. The only difference + is printing other kinds of objects. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + Bug #35026 + + * stream.c (format_num): New argument: sign character. + Rewrote to handle precision, width, zero padding and + leading sign similarly to printf. + (vformat): New syntax accepted: a space or + before + the width specifies that a positive sign is to be + explicitly written as a space or + character. + Pass one more argument to vormat_num calls. + Bugfix: go back to vf_init state after processing ~~. + +2011-12-22 Kaz Kylheku <kaz@kylheku.com> + + Bug #35136 and cleanup. + + * arith.c (plus, minus, mul, gt, ge, lt, le, exptmod, gcd): Remove + trailing abort; we already marked uw_throwf as noreturn. This hack + should not be needed in functions where the last statement is a throw. + (trunc, expt): Repeated error case handled in one place. Temp variable + used to avoid two calls to mp_clear. Call to abort removed. + (mod): Repeated error handed in one place. Plugged memory leak + by moving throw past mp_clear calls. Call to abort removed. + (isqrt): Repeated error case handed in one place. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Fixed to char literal syntax. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/bit-search-optimizations (s_highest_bit): + It will take days to completely wipe the egg off my face. + I forgot to fix this code for unsigned integers before + pasting it into MPI. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (normalize): Linkage changed to extern. + + * arith.h (normalize): Declared. + + * rand.c (random): Bugfix: normalize the bignum before returning it. + + * txr.1: Doc stubs for PRNG functionality. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * rand.c: Added comment about source of algorithm. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * rand.c (random): Bugfix: not building up sufficiently large bignums. + Work properly when mp_digit is smaller than 32 bits. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (OBJS): new object file, rand.o. + + * eval.c: Includes rand.h header. + (eval_init): New variable and functions from rand module registered. + + * lib.c: Includes rand.h header. + (init): Call rand_init. + + * rand.c: New file. + + * rand.h: New file. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + Bug #35139 + + Better fix. + + * parser.y (YYEOF): If YYEOF is not defined, define it as zero. + (yybadtoken): Undo previous changes: do not test for zero. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + Bug #35139 + + * parser.y (yybadtoken): The current token (yychar) is 0 on byacc + rather than YYEOF or YYEMPTY, so we have to handle that. + +2011-12-21 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (distclean): use rm -rf on mpi directory. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + Test case for bug #35137 + + * tests/007/except-2.expected: New file. + + * tests/007/except-2.txr: New file. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New function registered. + + * lib.c (cat_vec): New function. + + * lib.h (cat_vec): Declared. + + * txr.1: Documentation stub. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + Bug #35137 + + * unwind.c (uw_unwind_to_exit_point): When jumping to a catch frame, + do not mark it invisible. + + * unwind.h (uw_catch): Flip the matches to nil so that this catch + frame can no longer be identified as an unwind point by uw_throw, + and thus will not be re-entered for the purposes of handling + an exception. It remains visible for the purposes of running the + clean up code. + (uw_unwind): Prior to executing cleanup forms, flip the visibility + to 0. This means that the frame will no longer be re-entered + for any reason. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + Streamlining exception handling macros a little bit. + + * eval.c (op_unwind_protect): Use uw_simple_catch_begin, + and remove the uw_catch (exsym, exvals) clause. Put + explicit braces around the unwind code even though it + is only one statement. + + * match.c (do_txeval): Got rid of empty uw_unwind clause. + This is not needed any longer. + (v_try): Got rid of explicit uw_do_unwind calls. + + * unwind.h (uw_simple_catch_begin): New macro. + (uw_do_unwind): Macro removed. + (uw_catch): Added goto uw_unwind_label at the front. + This way if the previous clause falls through, + control goes to the unwind logic. + (uw_unwind): Got rid of initial break. Previous + clause should fall through to unwind logic, + whether it is the main clause, or one of the catches. + (uw_catch_end): Default case aborts, because + we don't expect this. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + Critical regression. Hash lookup was crashing on some platforms + due to negative hashing values being reduced modulo table size + to a negative array index. + + * hash.c (equal_hash, eql_hash): Ensure + that value returned is in the range [0,NUM_MAX]. + (hash_obj): Unused function removed. + (cobj_hash_op): Use hashing similar to eql hash for + other kinds of references. + (hash_eql, hash_equal): Removed bogus % NUM_MAX reduction. + + * hash.h (hash_obj): Declaration removed. + +2011-12-20 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions registered as intrinsics. + + * lib.c (copy_vec, sub_vec): New functions. + + * lib.h (copy_vec, sub_vec): Declared. + + * txr.1: Stub sections created. + +2011-12-19 Kaz Kylheku <kaz@kylheku.com> + + Version 049 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-19 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (subst_vars, op_quasi_list, expand_quasi): New static + functions. + (expand): New case for quasiliterals. + (eval_init): Register quasi literal as special operator. + + * match.c (format_field): Linkage changed to external. + + * match.h (format_field): Declared. Declarations rearranged. + +2011-12-18 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bindings_helper): Fix format arguments. + (eval_init): Registered new functions: symbol-function, + func-get-form, func-get-env, functionp, interp-fun-p. + + * lib.c (nappend2, getplist_f, improper_plist_to_alist): + tail variable renamed to avoid clash in macro. + (func_get_form, func_get_env, interp_fun_p): New functions. + + * lib.h (func_get_form, func_get_env, interp_fun_p): Declared. + (list_collect): Fix macro not to throw error, but handle the case. + + * match.c (vars_to_bindings, extract_bindings): tail variable + renamed to avoid clash in macro. + + * txr.1: Documentation stubs. + +2011-12-16 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (equal_hash): Eliminating displacement from character + hashes. Simplifying some code. + (eql_hash): Handle fixnums, characters and literals specially, + rather than hashing all value types the same way. The shift + applicable for object pointers causes adjacent integers to clash. + +2011-12-16 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_vars): Bugfix: use expand_forms rather than + expand on a list of forms. + +2011-12-16 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: iskeyword updated. + +2011-12-15 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (appendv): bugfix: wrong way test. + (vector_list): Wrong zero used, resulting in vector(nil) being called. + +2011-12-15 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): not added as synonym for null. + + * lib.c (copy_list): Use list_collect_append rather than + list_collect_terminate. + (append2, appendv): Simplified using new list_collect_append. + (nappend2): Simplified using new list_collect_nconc. + + * lib.h (list_collect): Added check for accidental usage + of list_collect after list_append, since PTAIL has different + semantics. + (list_collect_nconc, list_collect_append): Semantics fixed so that + append collecting works more like the Common Lisp append function, + allowing trailing atoms or a lone atom. The meaning of PTAIL is + changed, however. Now PTAIL actually tracks the head of the most + recently appended segment. Each append operation has to first + traverse the previously added piece to get to the end. + + (list_collect_terminate): Macro removed. + + * match.c (v_gather): Removed useless use of list_collect_terminate. + + * parser.y: Some headers added that are needed by list_collect. + + * txr.1: Documented append, list, atom, null, not, consp, make-lazy-cons, + lcons-fun, listp, proper-listp, length-list, mapcar, mappend, and apply. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + @# comments are becoming obsolescent. @# comments + can now be used. Within nested forms, + Lisp-compatible ; comments are suported. + + * parser.l: Support @# and ; comments. + + * txr.1: Documentation updated. + + * txr.vim: Updated. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (car, cdr): Set the lazy cons function to nil + after calling it. + (rplacd): Do not set the lazy cons function to nil + in. + + * txr.1: Documented a bunch of functions. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): Removed registration for vec_get_fil. + Renamed vec_set_fill to vec-set-length. + + * hash.c (equal_hash): vec_fill to vec_length name change. + (hash_grow, make_hash): No need to call vec_set_length. + + * lib.c (equal, vecref, vec_push, length_vec, list_vector, + obj_print, obj_pprint): vec_fill to vec_length name change. + (vector): Argument now represents actual length, not just allocated + size. + (vec_get_fill): Function removed; did exactly the same thing + as length_vec. + (vec_set_fill): Function renamed to vec_set_length. + (vector_list): Allocate a 0 length vector initially. + + * lib.h (enum vecindex): member changes name from vec_fill + to vec_length. + (vector): Parameter name changed. + (vec_set_fill): Redeclared. + (vec_get_fill): Declaration removed. + + * txr.1: Doc stubs updated. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (car, cdr): Semantics fix for lazy conses. + Ignore the return value of the lazy cons function: do not + return nil if the function returns nil. + This useless behavior was a source of inconvenience in lazy + cons programming, requiring the lazy function to return + non-nil in addition to installing the car and cdr fields. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (abso): broken for fixnums. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Highlight hash prefix and quote. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_dohash): Esbatlish anonymous block. + + * txr.1: Finished documenting special operators. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Fix empty NAME section. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (minus): Allow difference between characters. + +2011-12-14 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus, minus, gt, lt, ge, le): Handle character operands. + + * eval.c (eval_init): New functions interned. + + * lib.c (num_chr, chr_num): New functions. + + * lib.h (num_chr, chr_num): Declared. + + * txr.1: Documentation stubs. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + Version 048 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (exptmod, gcd): New functions. + + * eval.c (eval_init): New functions registered as intrisics. + + * lib.h (exptmod, gcd): Declared. + + * txr.1: Documentation stubs added. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (evenp, oddp): New functions. + + * eval.c (eval_init): New functions registered as intrinsics. + + * lib.h (evenp, oddp): Declared. + + * txr.1: Documentation stub updated. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (highest_bit): Linkage changed to static. + (abso, isqrt): New functions. + (isqrt_fixnum): New static function. + + * eval.c (eval_init): Registered abs, sqrt and numberp instrinsics. + + * lib.c (numberp): New function. + + * lib.h (numberp, abso, isqrt): Declared. + + * mpi-patches/series: New patch added. + + * mpi-patches/faster-square-root: New patch added. + + * txr.1: Documentation stubs for new functions. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (expt): Fix broken bignum x fixnum combination. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (repatch): New phony target. + (distclean): Remove mpi directory. + +2011-12-13 Kaz Kylheku <kaz@kylheku.com> + + Patch to shrink mpi-int to three words on 32 bit platforms, + so that obj_t stays four pointers wide. + + * mpi-patches/series: New patch added. + + * mpi-patches/shrink-mpi-int: New file. + +2011-12-12 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/bit-search-optimizations (s_highest_bit): Added static + storage class specifier. + + * mpi-patches/fix-mult-bug (s_mp_sqr): More braindamage found in MPI. + This function performs additions and multiplication mp_digit, + expecting a mp_word precision result without casting. This function + is needed for exponentiation. + +2011-12-12 Kaz Kylheku <kaz@kylheku.com> + + Git rid of some some loops in MPI where it is searching for + the highest bit, replacing them with an adapation of the + bit searching function used in arith.c. + + * mpi-patches/series: Patch added. + + * mpi-patches/bit-search-optimizations: New file. + +2011-12-12 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (expt): New function. + + * eval.c (eval_init): Registering new intrinsic functions, + reduce-left, reduce-right and expt. + + * lib.c (minusv): Return one instead of num(1). + (exptv, reduce_right): New functions. + + * lib.h (expt, exptv, reduce_right): Declared. + + * txr.1: Blank sections for new functions. + +2011-12-12 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/fix-mult-bug: One more flaw discovered in + s_mp_mul_d and added to patch. This one caused malloc corruption and + crashes, because the incorrect arithmetic causes the function + to think that the multiplication will not be needing another + digit, but then there is a carry out which does spill into + a new digit. + + * mpi-patches/series: Arg! Somehow the patch fix-bad-shift + went missing from the series file, even though the patch + itself is in the GIT repository. + +2011-12-06 Kaz Kylheku <kaz@kylheku.com> + + Version 047 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (zerop, gt, lt, ge, le): Functions from lib.c reimplemented + with bignum support. + + * eval.c (eval_init): Added bignump and zerop as intrinsic function. + Renamed numberp to fixnump. + + * lib.c (zerop, gt, lt, ge, le): Functions removed. + (numeq): Unused function removed. + + * lib.h (numeq): Declaration removed. + + * txr.1: Sections for zerop and bignump created. Changed reference + to numberp to fixnump. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (plus, mul): Plugged mpi_int memory leaks. + (trunc): Plugged memory leaks. Straightened out semantics + with negative modulus. (Residue comes out negative). + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (trunc): Error messages prefixed with trunc:. + (mod): New function, reimplementation of removed mod from lib.c. + + * lib.c (mod): Function removed. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + Bignum division implemented. More portability bugs found in MPI: + code like 1 << n, where n exceeds the width of the type int. + + * arith.c (trunc): New function, reimplementation of removed + trunc from lib.c. + + * lib.c (trunc): Removed. + + * mpi-patches/fix-bad-shifts: New file. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c (ABS): New macro. + (plus, minus): Bugfix: must not pass signed values to mp_add_d and + mp_sub_d functions. + (mul): Must not pass signed value to mp_mul_d. Also, fixed type check + on wrong argument in the (TAG_PTR, TAG_NUM) case. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + Removing this crutch; it's not that useful. + + * arith.txr: File removed. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Regenerated. + + * arith.txr (normalize): Bugfix: was not turning +/- NUM_MAX bignums + into fixnums. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Regenerated. + + * arith.txr (highest_bit): Missing #else added, fixing + SIZEOF_PTR == 4 case. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Regenerated. + + * arith.txr (highest_bit): Oops, half the logic for + the 64 bit case was missing due to to a cut and paste mistake. + +2011-12-11 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: Regenerated. + + * arith.txr (highest_bit): New function. + (mul): Use highest_bit instead of shift based algorithm. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim (txr_atat): New match. The @@ sequence is recognized + properly and highlighted. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + Bignum support in mult function. + + * arith.c: Regenerated. + + * arith.txr (CNUM_BIT): New constant. + (bignum, bignum_dbl_ipt): New static functions. + (@{add-fname}): Use bignum function. + (mul): New functions, rewrite of mul from lib.c. + + * lib.c (mul): Function removed. + + * mpi-patches/add-mp-set-intptr (mp_set_intptr): Revised patch. + Local variable v should be int_ptr_t not unsigned long. + Also, the mp_set interface doesn't set the sign; it's an unsigned + interface. We must do that ourselves. + + * mpi-patches/fix-mult-bug: The main multiplication function is + also broken in the same way, requiring the cast. + + * mpi-patches/mpi-set-double-intptr: Fixed use of wrong type for + local variable v. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/mpi-set-mpi-word: Bugfix and refresh. + + * mpi-patches/mpi-set-double-intptr: New file. + + * mpi-patches/series (mpi-set-double-intptr): Patch added. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * configure: add to config.h the type double_intptr_t, which + is twice the size of intptr_t. It may not be available, so + there is a HAVE_ macro to detect it. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions added as intrinsics. + + * hash.c (hash_eql, hash_equal): New external functions. + + * hash.h (hash_eql, hash_equal): Declared. + + * txr.1: Sections added. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * mpi-patches/add-mp-hash: Rewrote mp_hash to only hash enough + low-order bit material from the bignum to fill an unsigned long. + We don't need to walk the entire bignum. If the low order + digit of the bignum is at least as large as an unsigned long, + we just take that as the hash, otherwise we take enough of the + digits to fill an unsigned long. For negative numbers, we just + invert the bits of the hash. + + * mpi-patches/add-mpi-toradix-with-case: Refreshed. + + * mpi-patches/fix-mult-bug: Refreshed. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (mulv): Recognize cases to eliminate a wasteful mul call + with an initial element of 1. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (plusv): Recognize cases to eliminate a wasteful plus call + with an initial element of zero. + +2011-12-10 Kaz Kylheku <kaz@kylheku.com> + + * arith.c: File is now generated using TXR. + (NOOP): New macro. + (plus): Use NOOP macro. + (minus, neg): Function moved here from lib.c and rewritten + for bignum support. + + * lib.c (minus, neg): Functions removed. + + * arith.txr: New file. + +2011-12-09 Kaz Kylheku <kaz@kylheku.com> + + * configure: Fix patching without quilt. + +2011-12-09 Kaz Kylheku <kaz@kylheku.com> + + Build and pass test suite on Cygwin. + + * configure (longlong, ulonglong, superlong, usuperlong): Initialize + these variables so that if the detection tests fail, the script + does not access unbound variables. + Avoid adding junk like .bss.* into config.h. + + * mpi-patches/config-types: Fixed wrong use of nonexistent + SIZEOF_LONG_T. + +2011-12-09 Kaz Kylheku <kaz@kylheku.com> + + (Applies to previous commit.) + + * mpi-patches/config-types: Added missing definitions + of MP_DIGIT_SIZE in two cases. + +2011-12-09 Kaz Kylheku <kaz@kylheku.com> + + Bignum support, here we go! + + Bignums, based on Michael Fromberger's MPI library, are integrated + into the input syntax, stream output, equality testing, the garbage + collector, and hashing. + + The plus operation handles transitions between fixnums and bignums. + Other operations are still fixnum only. + + * Makefile (CFLAGS): Add mpi directory to include file search. + (OBJS): Include new arith.o module and all of MPI_OBJS. + (MPI_OBJS, MPI_OBJS_BASE): New variables. + + * configure (mpi_version, have_quilt, have_patch): New variables. + Script detects whether patch and quilt are available. Unpacks + mpi library, applies patches. Detects 128 bit integer type. + Records more information in config.h about the sizes of types. + + * dep.mk: Updated. + + * depend.txr: Make work with paths that have directory components. + + * eval.c (eval_init): Rename of nump to fixnump. + + * gc.c (finalize, mark_obj): Handle BGNUM case. + + * hash.c: (hash_c_str): Changed to return unsigned long + instead of long. + (equal_hash): Handle BGNUM case. + (eql_hash): Handle bignums with equal-hash, but other + objects as eq. + + * lib.c (num_s): Variable renamed to fixnum_s. + (bignum_s): New symbol variable. + (code2type): Follow rename of num_s. Handle BGNUM case. + (typeof): Follow rename of num_s. + (eql): Handle bignums using equal, and other types using eq. + (equal): Handle BGNUM case. + (chk_calloc): New function. + (c_num): Wording change in error message: is not a fixnum. + (nump): Renamed to fixnump. + (bignump): New function. + (plus): Function removed, reimplemented in arith.c. + (int_str): Handle integers which are too large for wcstol + using bignum conversion. Base 0 is no longer passed to + wcstol but converted to 10 because the special semantics + for 0 would be inconsistent for bignums. + (obj_init): Follow rename of num_s. Initialize bignum_s. + (obj_print, obj_pprint): Handle BGNUM. + (init): Call arith_init. + + * lib.h: Includes "mpi.h", as an exception to the project rule against + headers including headers. + (enum type): New enumeration member, BGNUM. + (struct bignum): New struct type. + (union obj): New member bn. + (mp): New inline function. + (num_s): Redeclared as fixnum_s. + (bignum_s, chk_calloc, bignump): Declared. + (nump): Redeclared as fixnump. + + * match.c (h_var, h_line, h_skip, h_coll, h_fun, format_field, v_skip, + v_freeform, v_collect, v_match_files): Follow nump to fixnump rename. + + * parser.l (NUM): New token type. Split up the parsing of identifiers + and numbers once again. But since every number is also lexically also + lexically an identifier, we put the action first. The action for + making numbers handles bignums. It produces object numbers, not + C numbers (change in yystype union). + + * parser.y (%union): num changes type from cnum to val. + + * stream.c (vformat): Handle bignums in numeric conversions. + + * arith.c: New file. + + * arith.h: New file. + + * mpi-1.8.6.tar.gz: New file. + + * mpi-patches/add-mp-hash: New file. + + * mpi-patches/add-mp-set-intptr: New file. + + * mpi-patches/add-mpi-toradix-with-case: New file. + + * mpi-patches/config-types: New file. + + * mpi-patches/export-mp-eq: New file. + + * mpi-patches/fix-mult-bug: New file. + + * mpi-patches/fix-warnings: New file. + + * mpi-patches/series: New file. + + * mpi-patches/use-txr-allocator: New file. + +2011-12-08 Kaz Kylheku <kaz@kylheku.com> + + C++ maintenance. + + * eval.c (and_s, or_s): Redundant variables removed. + + * match.h (do_s): extern storage class specifier added. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_defun): Transform a function body by inserting + a named block around it, thereby imitating a Common Lisp feature. + (op_for): Establish an anonymous block around the loop body, + test form and increment forms. + + * txr.1: Documented named block in defun. Documented for and for *. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Updated with all operators and functions. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: flip operator documented. Bad syntax for pop fixed. + Blank section for list-vector function added. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_modplace): If the operator is push, then reverse + the arguments. We want (push item list) for compatibility with CL. + (expand): Bugfix: some of the cases were constructing new + forms using unexpanded pieces from the original form. + Added separate case for push, which handles the reversed arguments. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (debug): Fix regression: repeat last command by hitting + Enter stopped working. This was broken by recent bugfixes in the + string splitting functions, which introduced a semantics change. + + * eval.c (flip_s, vecref_s): New symbol variables. + (op_modplace): New places (vecref ...) and (flip ...). Bugfix: dec + operator was incrementing. + (expand_place): Handle vecref and flip. Bugfix: pop has no + third argument and so is now handled by the same case as flip. + Bugfix: if a modify form has no third argument, then do not + resynthesize it with a nil third argument. + (eval_init): Initialize new symbol variables. + Register new flip operator. Register new list_vectory function + as intrinsic. + + * lib.c (rplacd): When modifying the cdr field of a lazy cons, + then lapse the lazy function to nil! This is needed by user-defined + lazy conses, and it makes sense to do it this way rather than + put in some explicit interface. + (list_vector): New function. + + * lib.h (list_vector): Declared. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (lookup_var, lookup_fun): Reversing assoc arguments. + (eval_init): New intrinsics. + + * hash.c (struct_hash): assoc_fun parameters reversed. + (gethash, gethash_f, gethash_n): Likewise. + + * lib.c (assoc, assq): Reversing parameters. + (find_package, acons_new, acons_new_l, aconsq_new): Reversing + arguments to assoc adn assq. + + * lib.h (assoc, assq): Declarations updated. + + * match.c (dest_set, dest_bind, h_var, h_coll, h_parallel, h_fun, + subst_vars, do_txeval, v_next, v_parallel, v_gather, v_collect, + v_flatten, v_cat, v_output, v_filter, f_fun, match_funcall): + Reversing arguments to assoc. + + * unwind.c (uw_get_func, uw_exception_subtype_p, uw_register_subtype): + Reversing arguments to assoc. + + * txr.1: Blank sections created for new functions. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Blank sections created for character functions. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New functions registered as intrinsics. + + * lib.c (chr_toupper, chr_tolower): New functions. + + * lib.h (chr_toupper, chr_tolower): New functions declared. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: In the CHRLIT state, return a nonblank character as an + IDENT token. This allows for character literals like #\$. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): New character functions registered. + + * lib.c (c_num): Generalized to convert characters to numbers also. + This allows functions like gt and lt to work with characters. + (chr_isalnum, chr_isalpha, chr_isascii, chr_iscntrl, chr_isdigit, + chr_isgraph, chr_islower, chr_isprint, chr_ispunct, chr_isspace, + chr_isupper, chr_isxdigit): New functions added. + + * lib.h: (chr_isalnum, chr_isalpha, chr_isascii, chr_iscntrl, + * chr_isdigit, chr_isgraph, chr_islower, chr_isprint, chr_ispunct, + * chr_isspace, chr_isupper, chr_isxdigit): New functions declared. + (c_true): New macro. + +2011-12-07 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (progn_s): New symbol variable. + (op_progn): New static function. + (eval_init): Initialize new variable, register progn operator. + + * txr.1: progn documented. + +2011-12-06 Kaz Kylheku <kaz@kylheku.com> + + Version 046 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-06 Kaz Kylheku <kaz@kylheku.com> + + * stream.c (find_char): New function. + (string_in_get_line): Following up TODO. Fixed broken + function. Now get_line on a string stream properly returns characters + up to and not including the next newlne character, and also consumes + the newline character. Other cases are handled properly, also: + the stream being at EOF already, or at the last line not being + newline-terminated. + +2011-12-06 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_unwind_protect): Fixed uninitialized variable + warning. + (eval_init): New functions registered: typeof and vector functions, + as well as length_list. + + * lib.c (length): Function renamed to length_list, because it is + list specific. + (length_vec, size_vec, vector_list): New functions. + (length): New function, generic over lists, vectors and strings. + + * lib.h (length_list, length_vec, size_vec, vector_list): Declared. + + * match.c (h_var, h_fun, robust_length, v_deffilter, v_fun): Use + length_list instead of length. + + * parser.l: Introduced # token. + + * parser.y (vector): New nonterminal. + (expr): vector is a kind of expr. + (chrlist): Bugfix: single-character syntax was not working; + for instance #\x to denote the charcter x. + (lit_char_helper): Use length_list instead of length. + + * stream.c (string_in_get_line): Bugfix: this was using + the wrong length function: length was being applied to a string. + The genericity of length makes that correct now, but changing + to length_str anyway. + + * txr.1: Blank sections created for functions. Vector syntax + documented. + +2011-12-06 Kaz Kylheku <kaz@kylheku.com> + + * configure: Forgot to treat octal number in the processing + of conftest.syms. Removed useless eval. + +2011-12-05 Kaz Kylheku <kaz@kylheku.com> + + Version 045 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-05 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_cond): Fixed behavior for singleton clauses. + (eval_init): Use existing function objects car_f, cdr_f, + eq_f, eql_f and equal_f. Added identity to function table. + + * lib.h (eql_f): Missing declaration added. + + * txr.1: Documented cond, and, if, or, defun, inc, dec, + set, push and pop. + +2011-12-04 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (force_regular_quotes): Function removed. + (list): Prior commit reversed. + + * txr.1: Prior commit reversed. + + * RELNOTES: No semantics clarification in quasiquote; bugfixes only. + +2011-12-04 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_qquote_error, op_unquote_error): New static functions. + (expand_qquote): Bugfix: missing case added to handle directly quoted + quasiquote. + (eval_init): Error-catching pseudo-operators registered in + op_table. + + * parser.y (force_regular_quotes): New function. + (list): Quotes within unquotes and splices are regular. + + * txr.1: Clarified new rules. Removed description of ,'form and ,*'form + special syntax. + +2011-12-03 Kaz Kylheku <kaz@kylheku.com> + + Expose lazy lists in TXR Lisp. + + * eval.c (eval_init): New intrinsic functions. + + * lib.c (rplaca, rplacd, lcons_fun): New functions. + (make_lazycons): Renamed to make_lazy_cons, relocated + and turned into external function. + (lazy_stream_func, lazy_stream_cons): Follow rename + of make_lazycons. + + * lib.h (rplaca, rplacd, make_lazy_cons, lcons_fun): + Declared. + + * txr.1: Stub sections created. + +2011-12-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (uw_protect_s, return_s, return_from_s): New symbol + variables. + (op_unwind_protect, op_block, op_return, op_return_from): + New static functions. + (expand): Removed case for call, if, and, and or. These operators + evaluate all their arguments, so the code walker can treat them + as a function calls. + Added case for block and return-from. + (eval_init): New symbols interned. New operator functions + registered in op_table. + + * txr.1: Blank sections added. + +2011-12-03 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (split_str, split_str_set): Bugfix: access beyond the end of + the input string. + +2011-12-03 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): String and character functions + exposed as intrinsics. + + * txr.1: Blank sections created. + +2011-12-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Added stub sections for new functions. + +2011-12-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c: Symbol related intrinsic functions and variables made + available: + + * lib.h (sym_name): Dangling declaration removed. + +2011-12-02 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (list): unquote and splice actions look inside the + argument form. If an unquote or splice are applied to a quoted + form, its quote becomes a regular quote. + This behavior is necessary to make ,',form work in nested + quotes, otherwise the ' is a quasiquote which captures + the comma in ,form, reducing ,',form to ,form. + + * txr.1: Documented this special behavior. + +2011-12-02 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_qquote): Bugfix: removed bogus recognition + and processing of regular quote form. This broke nested + backquote processing, and quasiquote forms containing + non-quasi-quotes like like '(a 'b ,c). + +2011-12-02 Kaz Kylheku <kaz@kylheku.com> + + Version 044 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Started Lisp documentation. Updated description of + symbol syntax. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (int_str): Return nil rather than 0 if no digits are extracted + at all. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_skip, h_coll, v_skip, v_collect): Evaluate the arguments. + (do_txeval): Optimization: short-circuit out if the expression is nil, + without establishing the exception handler. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_skip): Bugfix: Nov 12 commit caused regression: + skip min/max arguments not working! + + * RELNOTES: Updated. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + Dropping the silly cons return value from txeval. + Two interfaces are provided to the function. One throws + on unbound variable, the other which evaluates them to the + symbol noval_s (used in exception handling). + + * match.c (do_txeval): New static function. + (txeval): Functionality moved to do_txeval. + (txeval_allow_ub): New static function. + (vars_to_bindings, h_fun, v_freeform, v_next, v_merge, v_bind, v_set, + v_cat, v_output, v_deffilter, v_fun): No need to + use cdr to get the value from txeval. + (v_throw): Use txeval_ub_allowed, since unbound variables + are allowed in throw. + (v_try): Detect unbound arguments by checking for noval_s rather than + nil. No need to use cdr. + +2011-12-01 Kaz Kylheku <kaz@kylheku.com> + + * match.c (eval_form): Function renamed to txeval so its is + not confused with the Lisp evaluation functions. + (vars_to_bindings, h_fun, v_freeform, v_next, v_merge, v_bind, v_set, + v_cat, v_output, v_throw, v_deffilter, v_fun): Updated. + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + * lib.h (or2): Restore macro version of or2, because we need + the sequencing! Making it an inline function broke the tests. + But we can't have multiple evaluation either, so it's going to use + a temporary lexical variable. + (uses_or2): Macro which declares the lexical variable needed by or2. + + * debug.c (debug): add uses_or2. + + * eval.c (eval_intrinsic, op_modplace): Likewise. + + * lib.c (lazy_str, lazy_str_force_upto, lazy_str_get_trailing_list): + Likewise. + + * match.c (h_parallel, v_freeform, v_parallel, v_output): Likewise. + + * parser.y (unquotes_occur): Likewise. + + * stream.c (format): Likewise. + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + Removing useless hash table. + + * parser.h (ln_to_forms_hash): Declaration removed. + + * parser.l (ln_to_forms_hash): Variable removed. + (parse_init): Initialization and protection of ln_to_forms_hash + removed. + + * parser.y (rl): Update of ln_to_forms_hash removed. + + * txr.1: + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + * configure (extra_debugging): New variable. EXTRA_DEBUGGING + conditionally generated in config.h. + + * gc.c (break_obj): New static variable. + (mark_obj): Debugging feature: if the object is the one stored in + break_obj and not yet reached, then call breakpt. + (deheap): New debugging function for viewing regions of the heaps. + + * lib.c (breakpt): New function. + + * lib.h (breakpt): Declared. + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (hash_process_weak): Fix regression caused by a mistake + in the the 2010-01-26 commit, prior to release 033. When processing a + table with weak values, this function was mistakenly testing the keys + rather than values for for reachability. I noticed this when a test + case that should run in constant memory showed unwarranted accumulation + of memory. + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (op_modplace): Bugfix: conflation of new value and + increment value. Separate new value and increment value, and check + number of arguments. + + * lib.h (or2): Turned into inline function due to multiple + argument evaluation. + +2011-11-30 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: New operators added. + +2011-11-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (bindings_helper): Fix uninitialized variable. + +2011-11-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (dohash_s): New symbol variable. + (op_dohash): New static function + (expand): New case for dohash_s. + Bugfix for do_s: expand was used rather than expand_forms. + (eval_init): dohash_s initialized and entered into op_table. + +2011-11-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): hashp and maphash functions registered. + + * hash.c (maphash): New function. + + * hash.h (maphash): Declared. + +2011-11-29 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (expand_vars): Bugfix: was not handling vars + of the form var, only (var initform). + +2011-11-29 Kaz Kylheku <kaz@kylheku.com> + + Support assignment to (car ...) and (cdr ...). + + * eval.c (car_s, cdr_s): New symbol variables. + (op_modplace): Cases for car and cdr added. + (expand_place): Likewise. Calls abort should the + cases fall through rather than returning 42. + (expand): Bugfix: for and for* case not propagating + source location info. Bugfix: expansion for do added. + (eval_init): car_s and cdr_s initialized and used + in place of previous intern calls. + + * parser.y (elem): Removed wrong logic for expanding the + do form. It was expanding only the first argument. + +2011-11-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (let_star_s, for_s, for_star_s): New symbols. + (env_replace_vbind, bindings_helper): New static functions. + (op_let): Refactored to allow for let* form. Code for setting + up bindings moved into bindings helper, shared by for loop. + (op_for, expand_vars): New static functions. + (expand): Bugfix: let case was neglecting to walk the var + initialization forms. This is done via expand_vars now. + let_star_s added to this case to handle let* and let at + the same time. New case added for for and for*. + (eval_init): let_star_s, for_s, and for_star_s initialized, + and entered into op_table. + +2011-11-28 Kaz Kylheku <kaz@kylheku.com> + + * eval.c (eval_init): More functions. + + * txr.vim: More highlighting. + +2011-11-28 Kaz Kylheku <kaz@kylheku.com> + + Adding streams functions to Lisp evaluator. + + * eval.c (op_let): Bugfix: was not evaluating var init forms. + (reg_var): New static function. + (eval_init): Registered numerous stream functions and the + three standard streams. + + * lib.c (obj_print, obj_pprint): Modified to return a value. + (init): eval_init called after stream_init, because eval + needs the three standrad streams prepared. + + * lib.h (obj_print, obj_pprint): Declarations updated. + + * stream.c (format): Support t as a shorthand for standard output. + (formatv, open_directory, open_file, open_pipe): New functions. + (w_opendir): New static function. + + * stream.h (formatv, open_directory, open_file, open_pipe): + Declared. + + * txr.vim: set iskeyword such that keyword can contain special + characters. Set b:current_syntax to "lisp". + (txl_keyword): New keyword category populated with TXR Lisp keywords + defined as separate category. + (txr_list): Contains txl_keyword. + (txr_meta): Contains txl_keyword and txr_list. + +2011-11-28 Kaz Kylheku <kaz@kylheku.com> + + mapcar, mappend and apply functions. + fun operator. + + * eval.c (apply_s): New symbol variable. + (apply): Handle functions specified as symbols. Use symbol from context + form in error reporting. + (apply_intrinsic): New function. + (interp_fun): Bugfix: removed evaluation of arguments, since + arguments are already evaluated. + (op_call): Simplified by not having to handle symbols, + since apply does. + (op_fun): New function. + (expand): Handle special form fun. + (mapcarv, mappendv): New functions. + (eval_init): Initialize apply_s. Register op_fun function + in op_table. Register mapcar, mappend and apply functions. + +2011-11-28 Kaz Kylheku <kaz@kylheku.com> + + Added evaluation support for quote and quasiquote with unquotes. + New functions list, append and eval. Code walking framework for + expanding quasiquotes. quotes right now. + + * eval.c (let_s, lambda_s, call_s, cond_s, if_s, and_s, or_s + defvar_s, defun_s, list_s, append_s): New symbol variables. + (eval_intrinsic, op_quote, expand_forms, expand_cond_pairs, + expand_place, expand_qquote): New static functions. + (expand): New external function. + (eval_init): Initialize new symbol variables. Use newly defined symbol + variables to register functions. Also, new functions: quote, append, + list and eval. + + * eval.h (expand): Declared. + + * lib.c (appendv): New function. + (obj_init): quote and splice operator symbols moved into system + package. + (obj_print, obj_pprint): Support for printing quotes and splices. + + * lib.h (appendv): Declared. + + * match.c (do_s): New symbol variable. + (syms_init): New variable initialized. + (dir_tales_init): New variable used instead of intern. + + * match.h (do_s): Declared. + + * parser.y (elem): @(do) form recognized and its argument passed + through the new expander. + (o_elem, quasi_item): Pass list through expander. + (list): Use choose_quote to decide whether to put + regular quote or quasiquote on quoted list. + (meta_expr): Fixed abstract syntax so the expression is a single + argument of the sys:expr, rather than multiple arguments. + (unquotes_occur, choose_quote): New static function. + +2011-11-26 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (expr): Set source location info on elements. + (strlit): Set location info. + +2011-11-26 Kaz Kylheku <kaz@kylheku.com> + + * match.c (subst_vars): Handle expr_s, so that + Lisp expressions can be interpolated into quasiliterals. + (extract_vars): Avoid recursing into expressions marked + with expr_s. + (do_output_line): Handle expr_s so that Lisp expressions + can be interpolated into output. + + * parser.y (o_elem, quasi_items): Handle list expressions, + annotated with expr_s. + +2011-11-26 Kaz Kylheku <kaz@kylheku.com> + + Task #11436 + + Lisp interpreter added. + + * gc.c (finalize, mark_obj): Handle ENV objects. + + * hash.c (struct hash): acons_new_l_fun function + pointer order of arguments change. + (equal_hash): Handle ENV. + (make_hash, gethash_l): Use cobj_handle for + type safety. Follow change in acons_new_l. + (gethash, gethash_f, remhash, hash_count, + hash_get_userdata, hash_set_userdata, hash_next): Use cobj_handle. + (gethash_n): New function. + + * hash.h (gethash_n): Declared. + + * lib.c (env_s): New symbol variable. + (code2type, equal): Handle ENV. (plusv, minusv, mul, mulv, trunc, mod, + gtv, ltv, gev, lev, maxv, minv, int_str): New functions. + (rehome_sym): New static function. + (func_f0, func_f1, func_f2, func_f3, func_f4, func_n0, func_n1, + func_n2, func_n3, func_n4): Initialize new fields of struct func. + (func_f0v, func_f1v, func_f2v, func_f3v, func_f4v, + func_n0v, func_n1v, func_n2v, func_n3v, func_n4v, + func_interp): New functions. + (apply): Function removed: sanely re-implemented in new eval.c file. + (funcall, funcall1, funcall2, funcall3, funcall4): Handle + variadic and interpreted functions. + (acons, acons_new, acons_new_l, aconsq_new, aconsq_new_l): Reordered + arguments for compatibility with Common Lisp acons. + (obj_init): Special hack to prepare hash_s symbol, which is + needed for type checking inside the hash table funtions invoked + by make_package, at a time when the symbol is not yet interned. + Initialize new env_s variable. + (obj_print, obj_pprint): Handle ENV. Fix confusing rendering of + of function type. + (init): Call new function eval_init. + + * lib.h (enum type): New enumeration member ENV. + (struct func): functype member changed to bitfield. + New bitfied members minparam and variadic. + New members in f union: f0v, f1v, f2v, f3v, + f4v, n0v, n1v, n2v, n3v, n4v. + (struct env): New type. + (union obj): New member e of type struct env. + (env_s): Variable declared. + (plusv, minusv, mul, mulv, trunc, mod, gtv, ltv, gev, lev, maxv, minv, + int_str): New functions declared. + (func_f0v, func_f1v, func_f2v, func_f3v, func_f4v, + func_n0v, func_n1v, func_n2v, func_n3v, func_n4v, + func_interp): Likewise. + (apply): Declaration removed, and re-introduced in eval.h. + (acons, acons_new, acons_new_l, aconsq_new, aconsq_new_l): Declarations + updated to new argument order. + + * match.c (bindable): static function moved to eval.c, where + it becomes external. + (h_var, h_coll, h_parallel, h_fun, v_parallel, v_gather, + v_collect, v_merge, v_fun): Follows argument order change in + acons functions. + (subst_vars): Print atoms other than strings. + (eval_form): Support @(...) syntax for evaluating Lisp forms. + (v_do, h_do): New functions. + (dir_tables_init): Insert v_do and h_do into tables. + + * parser.l: Token syntax for numbers and symbols merged. + Symbols in a nested context can consist of various additional + characters. Useless code removed from action for '('/METAPAR. + + * stream.c (format): Bugfix in type checking, in the + case that the stream argument is nil and defaults to a string stream. + + * txr.vim: Updated for new token syntax. Fixed uses of + unescaped + operator. + + * unwind.c (uw_set_func) + * unwind.h (numeric_assert, range_bug_unless): Missing + whitespace in message added. + + * Makefile (OBJS): eval.o added. + + * dep.mk: Updated. + + * eval.c: New file. + + * eval.h: New file. + +2011-11-24 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (getplist_f): New function. + + * lib.h (getplist_f): Declared. + + * match.c (v_collect, h_coll): Use getplist_f to distinguish + the case that :vars is explicitly specified as (). In this + case, no bindings escape from the collect. + + * tests/008/soundex.txr: This test case broke due to using + :vars () and yet counting on the variable to exist. + + * RELNOTES: Updated. + +2011-11-24 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_funcall): Set source location info for + generated function call. + +2011-11-24 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (texts, elem): Fixed incorrect use of rl rather + than rlcp. Added forgotten rlcp on result of optimize_text. + + * RELNOTES: Updated. + +2011-11-20 Kaz Kylheku <kaz@kylheku.com> + + Version 043 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + + * RELNOTES: Updated. + +2011-11-23 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (remove_hash_bang_line): Recognize multiple + syntax possibilities. A hash bang could be buried in a (text ...) + compound, or it could just be a string (thanks to the text form + optimization). + +2011-11-23 Kaz Kylheku <kaz@kylheku.com> + + Optimization: if all the elements of (text ...) are + strings, then replace the (text ...) by the catenation + of those strings. + + * parser.y (optimize_text): New function. + (elem): Use optimize_text. + +2011-11-23 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (plus, minus): Fixed wrong assertion which would incorrectly + fire for inputs that do not overflow. + + * match.c (search_form): Fixed incorrect loop test which could + lead to nonterminating behavior. + + * RELNOTES: Updated. + +2011-11-23 Kaz Kylheku <kaz@kylheku.com> + + Semantics change. If a variable is followed by a mixture + of text and regular expressions, that whole mixture is + considered to follow the variable and used for matching. + + The earlier semantics change whereby a single unescaped + space denotes the regular expression / +/ broke the + simple case @a word. It caused the @a to be followed + not by the text " word" but by just the regular expression + element. + + With this change @a word means that a is followed by + the regex / +/ and "word". + + * match.c (text_s): New symbol variable. + (h_text): New function. + (syms_init): Initialize new symbol variable. + (dir_tables_init): Hook h_text into horizontal directives table. + + * match.h (text_s): Declared. + + * parser.y (text, texts): New nonterminals. + (elem): TEXT, SPACE and regex are now handled under texts + grammar production. All texts are run together and produce + an item which looks like (text items ...). + + * txr.1, RELNOTES: Updated. + + * txr.c (remove_hash_bang_line): Updated to find #! buried + in (text ...) syntax. + +2011-11-22 Kaz Kylheku <kaz@kylheku.com> + + * configure: Fix environ test case for C++. + +2011-11-22 Kaz Kylheku <kaz@kylheku.com> + + * match.c (search_form): Bugfix: we must search to one character + position after the end of the line, otherwise we can never match + @(eol). + (h_eol): Bugfix: do not return t, but the line length. + + * txr.1: Warn users about @var@(bind ...) pitfall. + + * RELNOTES: Updated. + +2011-11-20 Kaz Kylheku <kaz@kylheku.com> + + Version 042 + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2011-11-20 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (char_from_name): const on wchar_t *. + +2011-11-20 Kaz Kylheku <kaz@kylheku.com> + + Bug #34630 + + * parser.y (repeat_clause, rep_elem): Allow empty body. + (yybadtoken): Handle unexpected newline with different message. + + * RELNOTES: Updated. + +2011-11-20 Kaz Kylheku <kaz@kylheku.com> + + Relaxing :vars in collect/coll a little bit. + + * match.c (h_coll, v_collect): Only throw an error about + missing required variables if the collect iteration collected some new + variables. This allows strict collects with :vars to have + some cases which explicitly match and skip unwanted material, + without binding variables. Also, print all missing variables in the + diagnostic. + + * txr.1: Mention this special exception. + + * RELNOTES: Updated. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (tests/008/soundex.ok): New test case. + (TXR_ARGS): Specified for new test case. + + * tests/008/soundex.expected: New file. + + * tests/008/soundex.txr: New file. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + * RELNOTES: New file. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + Bug #34866 + + * match.c (h_skip): Bugfix. Return the length of the line if + the skip is to the end of line, not the value t. + + * txr.1: Clarify that @var@(skip)text is useless. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_deffilter): Even better. Just evaluate + the arguments individually. Now @(deffilter a b ..) + is possible where these evaluate to suitable + lists of strings. + + * txr.1: Documented. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + deffilter grows in power: it can take quasistrings. + + * lib.c (cdr_f): New global variable. + (funcall1, funcall2, funcall3, funcall4): Fix unterminated + arguments in uw_throwf call by using uw_throw instead. + (do_or): New static function. + (orf): New function. + (obj_init): gc_protect and initialize cdr_f. + + * lib.h (cdr_f, orf): Declared. + + * match.c (v_deffilter): Treat the table as forms to be evaluated which + must reduce to strings, rather than literal strings. + + * txr.1: Documented. + +2011-11-19 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (yybadtoken): Use ~a to print bad character rather than #\ + notation. + +2011-11-18 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Regression IDENT, '{' and '}' + must be on the same precedence level and right + associative. Without this consective braced + variables don't work, etc. + +2011-11-18 Kaz Kylheku <kaz@kylheku.com> + + * gc.c (mark_mem_region): Use the Valgrind API only to mark + the type field as accessible, not the whole object that + we are checking. Marking the whole object accessible hides + uninitialized field bugs! + + * lib.c: And found a bug already: lazy_str was not completely + initializing all of the object fields (ls.prefix, ls.list) + before invoking memory allocating operations, making it + possible for the garbage collector to encounter uninitialized + object areas. + +2011-11-18 Kaz Kylheku <kaz@kylheku.com> + + Added a JSON parsing test case. This flushed out a bug which crashed + the garbage collector (uninitialized fields in function objects). + + * Makefile: Defined TXR_ARGS and TXR_OPTS for new test case. + + * hash.c (hash_begin): Construction of cobj modified to obey + the correct procedure described in HACKING. + + * lib.c (func_n3, func_n4): These functions neglected to initialize + the env member of the function structure. + + * tests/009/json.expected: New file. + + * tests/009/json.txr: New file. + + * tests/009/webapp.json: New file. + +2011-11-17 Kaz Kylheku <kaz@kylheku.com> + + Task #11598. + + * match.c (resolve_k): New keyword symbol variable. + (h_parallel, v_parallel): Implement :resolve keyword in @(some) + directive. + (syms_init): New symbol variable initialized. + + * parser.l: Allow (some) to have argument material. + + * parser.y (some_clause, elem): SOME syntax adjusted. + + * txr.1: Documented new :resolve keyword in @(some). + +2011-11-17 Kaz Kylheku <kaz@kylheku.com> + + Adding quote and unquote read syntax to list forms, resembling + Lisp. The difference is that splice is spelled ,* because @ + already means something, and that there is only one quote operator. + None of this does anything; it is only syntax. + + * lib.c (quote_s, qquote_s, unquote_s, splice_s): New variables. + (obj_init): New variables initialized. + + * lib.h (quote_s, qquote_s, unquote_s, splice_s): Declared. + + * parser.l: Added recognition rules. + + * parser.y (SPLICE): New symbolic token. + (list): Added new syntax for quote and splicing. + +2011-11-17 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_fun, v_fun): Bugfix! copy_list should be used for copying + the bindings, not copy_alist. Otherwise functions cannot destructively + update a binding, which is useless. We want a function not to + manipulate the binding list, but to be able to manipulate the + contents of bindings. + (match_files_ctx): Declaration moved ahead of match_line. + (v_fun): Forward declaration added. + (match_line): Allow vertical functions to be called from + a horizontal context, in a limited way. + + * txr.1: Mention the possibility of a call from a horizontal + context falling back on a vertical function. + +2011-11-17 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Bugfix: precedence of { } must be low, close to that of + IDENT, otherwise @{var}@(foo) doesn't parse. + +2011-11-16 Kaz Kylheku <kaz@kylheku.com> + + Allow directives after variable to be a kind of negative match. + + * match.c (search_form): bugfix: return correct match extent. + + * parser.y: Adjusting associativity and precedence of directives, IDENT, + and grouping tokens once again. This is so that a var followed by + a directive will turn into one elem, rather than the var being + reduced to an elem first. + + * txr.1: Revised documentation to mroe clearly define the concept + of a negative match, broken into subsections. Some sections + belonging to syntax were moved to an appropriate location. + Subsections added to description of form syntax. + Explanation of directive-driven syntax. + +2011-11-16 Kaz Kylheku <kaz@kylheku.com> + + Variable matches can span over function calls. + Function calls following variables have searching semantics. + + * match.c (ml_specline_pos, search_form): New static functions. + (h_var): Handle functions and regexes in a common way. + + * parser.y: Adjusted precedence of IDENT and ( so that + @var@(func) are parsed into a single var element. + + * txr.1: Documented. + +2011-11-15 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Update for new character constant syntax. + +2011-11-15 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_var): when manipulating specline, propagate the + source locatio info. + (v_skip): Don't use specline for trace messages, because + it may be nil. Use the skip spec. + + * parser.h (rl): Declared. + (rlcp): New inline function. + + * parser.y (rl): Static declaration removed. Function becomes + extern. + (clause): Propagate location info from clause to clause list + backbone. + (collect_clause, COLL): Bugfix: car/cdr mixup in location info. + (elem): Use rlcp function to abbreviate code. + (o_elems_opt, o_elems_opt2, o_elem): Set location info. + +2011-11-15 Kaz Kylheku <kaz@kylheku.com> + + Changing read syntax for character literals, because we are going to + need the single quote in the Lisp way for suppressing evaluation, + eventually. + + I'm going with a Scheme-compatible syntax for character literals. + It has a richer repertoire of standard character names than Common + Lisp, and has a x convention for coding characters in hex. + + * lib.c (obj_print): Print characters in a Scheme-like way. + + * parser.h (end_of_char): New function declared. + + * parser.l (grammar): Implement rules for #\ syntax, with + involving new HASH_BACKSLASH token. + (end_of_regex): Enhancement: added check that end_of_regex is + called in correct state, like the one in end_of_char. + (end_of_char): New function. + + * parser.y (repeat_rep_helper, o_elems_transform, define_transform, + lit_char_helper): Functions changed to static. + (rl): Function moved down, past the grammar section. + (HASH_BACKSLASH): New terminal symbol. + (chrlit): Grammar redesigned. + (char_from_name): New function. + + * txr.1: Character syntax documented. + +2011-11-14 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: horizontal directives were being treated as vertical, + and the trailing material silently ignored. + For instance @(bind a 1)@(bind b 2). This was going to v_bind, + v_bind does not check for the trailing material and doe snot + call decline_s. The result was that b was not bound. + Correct behavior is to process these binds in match_line. + + * match.c (match_line): Check if a directive IS found in the vertical + table, and if so report a different error message. The fallback + case is that there is no such function or directive. + (v_next): Do not check for obsolete syntax any more. This case + will not occur any more due to the following changes. + (match_files): Do not defer opening the file if the data starts + with an incorrectly written next directive. + Do not look up and process a vertical directive or function + call if it is followed by more material in the same line. + Thus vertical directives can longer receive trailing material. + This fixes the bug of horizontal directives being treated as + vertical + +2011-11-13 Kaz Kylheku <kaz@kylheku.com> + + * debug.c (debug): Eliminated duplicate code. + Implemented better way of printing character context. + +2011-11-13 Kaz Kylheku <kaz@kylheku.com> + + Adding a debugger. This is an experimental prototype. + + * Makefile (OBJS): New object file debug.o. + + * dep.mk: Updated. + + * match.c (h_fun): Use debug_begin and debug_end macros + to set up a debug frame for backtracing. + (match_line, match_files): Call debug_check to give debugger a chance + to instrument call. + (v_fun): Use debug_begin and debug_end macros to set up a debug frame + for backtracing. Call debug_check to give debugger a chance to + instrument call. + + * stream.c (struct strm_ops): New function pointer, flush. + (stdio_maybe_write_error): Wrong word in error message corrected. + (stdio_flush): New static function. + (stdio_ops, pipe_ops): New function entered into tables. + (flush_stream): New function. + + * stream.h (flush_stream): Declared. + + * txr.c (help): New options documented. + (main): call to debug_init added. New debug options parsed and + opt_debugger set accordingly. + + * unwind.c (uw_push_debug, uw_current_frame): New function. + + * unwind.h (uw_frtype): New enumeration member UW_DBG. + (struct uw_debug): New frame variant. + (union uw_frame): New member, db. + (uw_push_debug, uw_current_frame): Declared, + + * debug.c: New file. + + * debug.h: New file. + +2011-11-13 Kaz Kylheku <kaz@kylheku.com> + + Fix regression in earlier commit: + + "Eliminate line numbers from the abstract syntax + tree representation of the TXR query." + + * match.c (match_funcall): Remove spurious object being + added to the front of a form where a line number used + to be. + +2011-11-13 Kaz Kylheku <kaz@kylheku.com> + + * match.c: Removed <assert.h> + + * txr.c: Likewise. + +2011-11-13 Kaz Kylheku <kaz@kylheku.com> + + Bug #34813 + + * match.c (v_freeform): Fail if the data is null, + to avoid a false positive match as an empty line, followed by the type + error of incrementing a nonexistent data line number. + +2011-11-12 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Correctly record line number info for regex. + +2011-11-12 Kaz Kylheku <kaz@kylheku.com> + + Improved line number reporting in errors and debug traces. + + * match.c (debugf): Function removed. + (dest_bind, v_output, v_eof): Use debuglf instead of + debugf, and sem_error instead of uw_throwf. + (match_files): Likewise, and file_err is called + with form. + +2011-11-12 Kaz Kylheku <kaz@kylheku.com> + + Eliminate line numbers from the abstract syntax + tree representation of the TXR query. + + * match.c (debuglf, sem_error, file_err, eval_form): Line number argument replaced + with the form to which the situation pertains. Location information is + pulled from the hash table entry associated with the form. + (dest_set, dest_bind, eval_form, vars_to_bindings): Context argument + renamed since it isn't a line number. + (struct match_line_ctx): spec_lineno member removed. + (ml_all, ml_bindings_specline): lineno parameter removed. + (LOG_MISMATCH, LOG_MATCH, h_var, h_skip, h_coll, h_parallel, + match_line): Pass elem to debuglf instead of line number. + as context. + (h_trailer, h_eol): define elem for LOG_MISMATCH and LOG_MATCH macros. + (h_fun): Pass elem variable to debuglf instead of line number. + Body stored as a simple cons cell once again (no line number). + (do_output_line): Line number parameter removed. Pass specline to + sem_error instead of line number. + (do_output): Adjusted for one less parameter in do_output_line. + (mf_from_ml): Pass one less parameter to ml_all. Conversion of + specline to spec is just a wrapping into a nested list, + with no line number. + (spec_bind): Linenumber variable parameter removed from macro. + Definition simplified. + (v_skip): Pass specline to debuglf instead of spec_linenum, + which is no longer computed. + (v_trailer): Use new definition of specline. Pass first_spec + to sem_error instead of spec_linenum. + Computation of ff_specline no longer has to skip line number. + (v_freeform, v_block, v_accept_fail, v_next, v_parallel, v_gather, + v_collect, v_merge, v_bind, hv_trampoline, v_cat, v_output, + v_try, v_defex, v_throw, v_deffilter, v_filter, match_funcall): Use new + definition of specline. Pass first_spec to sem_error instead of + spec_linenum. (v_forget_local): Specline computed differently since + there is no linenumber to skip. + (h_define): Back to implified representation of function with + no extra cell for line number. + (v_define, v_fun): Pass first_spec to sem_error instead of + spec_linenum. Back to implified representation of function with no + extra cell for line number. + (match_files): first_spec_item computed differently. + Pass first_spec to sem_error instead of spec_linenum. + + * parser.h (source_loc): Declared. + + * parser.l (source_loc): New function. + + * parser.y:x (grammar): Removed line numbers from abstract sytnax + tree. A few more places needed the annotation of forms with location + info, and a couple of cases of the need to propagate the info was + identified. Use extra cons cell as output of until_last to propagate + the line number from the symbol to the use. + + * txr.c (remove_hash_bang_line): No longer has to look past + line number. + +2011-11-12 Kaz Kylheku <kaz@kylheku.com> + + Infrastructure for storing line number information + outside of the code, in hash tables. + + * filter.c (make_trie, trie_add): Update to three-argument + make_hash. + + * hash.c (struct hash): New members, hash_fun, assoc_fun + acons_new_l_fun. + (ll_hash): Renamed to equal_hash. + (eql_hash): New static function. + (cobj_hash_op): Follows ll_hash rename. + (hash_grow): Use new function indirection to call hashing function. + (make_hash): New argument to specify type of hashing. Initialize new + members of struct hash. + (gethash_l, gethash, remhash): Use function indirection for hashing and + chain search and update. + (pushhash): New function. + + * hash.h (make_hash): Declaration updated with new parameter. + (pushhash): Declared. + + * lib.c (eql_f): New global variable. + (eql, assq, aconsq_new, aconsq_new_l): New functions. + (make_package): Updated to new three-argument make_hash. + (obj_init): gc-protect and initialize new variable eql_f. + + * lib.h (eql, assq, aconsq_new, aconsq_new_l): Declared. + + * match.c (dir_tables_init): Updated to there-argument make_hash. + + * parser.h (form_to_ln_hash, ln_to_forms_hash): Global variables + declared. + + * parser.l (form_to_ln_hash, ln_to_forms_hash): New global variables. + (grammar): Set yylval.lineno for tokens that are classified to + that type in parser.y. + (parse_init): Initialize and gc-protect new global variables. + + * parser.y (rl): New static helper function. + (%union): New member, lineno. + (ALL, SOME, NONE, MAYBE, CASES, CHOOSE, GATHER, + AND, OR, END, COLLECT, UNTIL, COLL, OUTPUT, REPEAT, + REP, SINGLE, FIRST, LAST, EMPTY, DEFINE, + TRY, CATCH, FINALLY, ERRTOK, '('): Reclassified as lineno type. + In the grammar, these keywords can thus provide a stable line number + from the lexer. + (grammar): Numerous rules updated to add constructs to the + line number hash tables via the rl helper. + + * dep.mk: Updated. + + * Makefile (depend): Use the installed, stable txr in the + system path to update dependencies rather than locally built ./txr, to + prevent the problem that txr is broken because out out-of-date + dependencies, and thus cannot regenerate dependencies. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + Bug #34799: errors in horizontal functions + reported to caller line number. + + * match.c (ml_bindings_specline): Extended with extra argument + (h_coll): Pass nil for new argument of ml_bindings_specline. + (h_fun): Extract line number from stored function. + Pass line number to ml_bindings_specline. + (h_define, v_define): Store function as a cons cell containing + the line number and body. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Document -l/--lisp-bindings. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + * match.c (opt_nobindings, opt_arraydims): Global + variables moved from parser.l. + (opt_lisp_bindings): New variable. + (dump_bindings): Dump Lisp syntax bindings + on standard output if opt_lisp_bindings is set. + (v_cat): Do not complain about trailing material; + this is not compatible with horizontal cat. + + * parser.l (opt_nobindings, opt_arraydims): Moved + to match.c. + + * txr.c (txr_main): New options, --lisp-bindings + and the equivalent -l. + + * txr.h: opt_lisp_bindings declared. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + Task #11583 + + * match.c (dir_tables_init): Mapping flatten_s, forget_s, + local_s, merge_s, set_s, cat_s and filter_s to hv_trampoline + function, thereby making all these directives work in + horizontal contexts in one fell swoop. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + Task #11583 + + More generic approach. + + * match.c (h_bind): Function removed. + (hv_trampoline): New function. + (dir_tables_init): hv_trampoline installed in + h_directive_table instead of h_bind. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Fixed wrong error message. + +2011-11-10 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_fun): Bugfix: if there is material after + the function call, decline it; it is a horizontal context. + + * txr.1: Discussion and examples of calls that are + in a horizontal context. + +2011-11-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented horizontal function definitions and calls + +2011-11-09 Kaz Kylheku <kaz@kylheku.com> + + Task #11583 + + @(bind) in horizontal mode. + + * match.c (mf_from_ml, h_bind): New functions. + (dir_tables_init): h_bind entered into table. + +2011-11-09 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_fun, v_fun): Slightly more informative tracing from failed + function calls. + +2011-11-09 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Missing coll keyword added. + +2011-11-08 Kaz Kylheku <kaz@kylheku.com> + + Task #11431. First cut at horizontal match functions. + + * match.c (h_fun): New function. + (match_line): Rearranged not to do hash lookup if the directive is a + regex or list. If hash lookup fails, try it as a horizontal function. + (h_define): New function. Handles horizontal function syntax embedded + in line. + (v_define): Handle the horizontal function syntax occuring + on a line by itself. The function info is now stored as a cons cell + whose car is the vertical function and cdr the horizontal one. + (v_fun): Adjust to new function storage convention. + (dir_tables_init): h_define entered in table. + * parser.y: Added syntax for horizontal define. + +2011-11-06 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Make sure whitespace is recognized after @. + +2011-11-06 Kaz Kylheku <kaz@kylheku.com> + + Task #11581 & bugfix. + + * match.c (noval_s): New symbol variable. + (vars_to_bindings): Use a default value of noval_s to indicate a + required variable, rather than nil, which would not allow + an optional variable with a default value of nil. + (h_coll, v_collect): Check default value against noval_s, rather than nil. + (v_gather): Support :vars keyword. + (syms_init): Initialize new symbol variable. + + * txr.1: Documented gather's :vars parameter. + +2011-11-06 Kaz Kylheku <kaz@kylheku.com> + + Task #11581 + + * match.c (gather_s): New keyword variable. + (v_gather): New function. + (syms_init): gather_s initialized. + (dir_tables_init): v_gather entered into table. + + * match.h (gather_s): Declared. + + * parser.l: GATHER token scanning added. + + * parser.y: GATHER token added. gather_clause nonterminal added. + + * txr.1: New directive documented. + + * txr.vim: gather keyword introduced. + +2011-11-05 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (env): Fixed inappropriate cut-and-pasted error messages. + Check for failure of GetEnvironmentStringsW, and call + FreeEnvironmentStringsW is called. + +2011-11-05 Kaz Kylheku <kaz@kylheku.com> + + * match.c (dir_tables_init): Bugfix: horizontal @(some) + directive not included in dispatch table. + +2011-11-05 Kaz Kylheku <kaz@kylheku.com> + + * configure: Bugfixes. Before the compiler tests, we must + remove the conftest executable, to make sure that the next test will + try to re-make it. The configure runs fast enough that the new + conftest.c does not always have a timestamp which is newer than + previous conftest executable. + +2011-11-05 Kaz Kylheku <kaz@kylheku.com> + + Task #11442. Make work on MingW. + + * configure: Test for environ and GetEnvironmentStrings. + + * lib.c: Conditionally include <windows.h>. + (env): Implemented for POSIX and Windows with #ifdefs. + +2011-11-05 Kaz Kylheku <kaz@kylheku.com> + + Task #11442. Access to environment variables. + + * lib.c (env_list): New static variable. + (env): New function. + (match): Declaration of nonexistent function removed. + (obj_init): New variable gc-protected. + + * lib.h (env): Declared. + + * match.c (env_k): New symbol variable. + (v_next): Implemented :env. + + * txr.1: @(next :env) described. + +2011-11-04 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (ll_hash): Added a break in the case that handles + pointer hashing of identity-equal objects. Without this, if the + pointer size is not 4 or 8, we fall through to the next case. + +2011-11-04 Kaz Kylheku <kaz@kylheku.com> + + * txr.c (help): Change year from 2009 to 2011. + +2011-11-03 Kaz Kylheku <kaz@kylheku.com> + + * tests/008/students.txr: Use disciplined collect with :vars. + +2011-11-03 Kaz Kylheku <kaz@kylheku.com> + + * tests/008/students.txr: Regexes removed. + +2011-11-02 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Added missing keywords. + +2011-11-01 Kaz Kylheku <kaz@kylheku.com> + + * genman.txr: Use filter for mapping month digits to names. + Added comment about where to find the right man2html. + +2011-11-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.vim: Added installation instructions. + +2011-11-01 Kaz Kylheku <kaz@kylheku.com> + + Syntax highlighting for Vim. + + * txr.vim: New file. + +2011-10-30 Kaz Kylheku <kaz@kylheku.com> + + Version 041 + + Bugfixes: + + Runaway recursion in @(block) directive, introduced in 040. + + Fixed bug in matching list variable against text, at the same + time clarifying semantics to longest-match. + + Fixed potential excessive memory use caused by refactoring in 040. + + Features: + + New :append keyword in @(output) to append instead of overwriting. + + Variable contents can be treated as input sources using :string + and :list keywords in @(next). Variables can be treated as output + destinations using :into keyword in @(output). + + New @(set) directive for destructive assignment to a variable. + + New filters: :upcase and :downcase. + + @(bind) can now compare left and right objects through filters. + + Filters can now be chained into compound filters. + + Pattern matching functions can be used as filters. + + Shorthand notation in @(deffilter) when multiple strings map to the + same replacement string. + + @(cat) directive changes syntax. + + Error handling improvements in parser: no more reams and reams of + errors. + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2011-10-30 Kaz Kylheku <kaz@kylheku.com> + + We don't include headers in headers in this project. + + * parser.h: Do not include <stdio.h> + + * regex.c: Include <limits.h> + + * regex.h: Do not include <limits.h> + +2011-10-30 Kaz Kylheku <kaz@kylheku.com> + + Bug #34691 + + Changing the parameter passing convention for vertical directives. + They take one parameter which is a pointer, rather than a copy + of the structure. They do not have to perform a structure + assignment when returning next_spec_k. + + * match.c (v_match_func): Typedef updated to new function + signature. + (v_skip, v_trailer, v_freeform, v_block, v_accept, v_accept, v_next, + v_parallel, v_collect, v_flatten, v_forget, v_forget, v_merge, v_bind, + v_set, v_cat, v_output, v_define, v_try, v_defex, v_throw, v_deffilter, + v_filter, v_eof, v_fun): Refactored. + (match_files): Updated dispatch logic to new style calls. + (match_funcall): Updated to new way of calling v_fun. + +2011-10-29 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Grammar fixes. Expanded on lazy strings a little bit. + Added something about mem_t *, and a few extra words here and there, + including a blurb about a Valgrind debugging caveat. + +2011-10-27 Kaz Kylheku <kaz@kylheku.com> + + Bug #34657 + + * txr.1: Added explanations about the differences between + empty streams and empty lines, and to watch out when passing + empty strings to @(next :string ...). + +2011-10-26 Kaz Kylheku <kaz@kylheku.com> + + Bugfix: prepared_error_message variable needs to be gc-protected. + + * parser.h (parse_init): Declared. + + * parser.l (parse_init): New function. + + * txr.c (main): Call parse_init. + (txr_main): No need to gc-protect yyin_stream since parse_init does it. + +2011-10-26 Kaz Kylheku <kaz@kylheku.com> + + Parse error handling improvements. + + * parser.l (prepared_error_message): New static variable. + (yyerror): Emit and clear prepared error message. + (yyerrprepf): New static function. + (yybadtoken): Function moved into parser.y. + (grammar): For irrecoverable lexical errors, stash error message + with yyerrprepf and return the special error token ERRTOK to generate a + syntax error. I could find no other interface to the parser to make it + cleanly exit. + + * parser.y (ERRTOK): New terminal symbol, does not appear anywhere + in the grammar. + (spec): Bail after 8 errors, recover to nearest newline, and + use yyerrok to clear error situation. + (YYEOF): Provided by Bison, conditionally defined for other yacc-s. + (yybadtoken): Function moved from parser.l. Checks for the next + token being YYEMPTY or YYEOF, and also handles ERRTOK. + + * stream.c (vformat_to_string): New function. + (format): If stream is nil, format to string and return it. + + * stream.h (vformat_to_string): Declared. + +2011-10-26 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_cat): Bugfix: unterminated variable argument list. + + * tests/001/query-3.txr: Updated to new cat syntax. + +2011-10-26 Kaz Kylheku <kaz@kylheku.com> + + Fixed lame @(cat) directive, without obsolescence phase. + + * match.c (v_cat): Rewritten. + + * txr.1: Documented. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + * configure: put in set -u to trap unbound variables, + and fixed resulting errors that were found. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + * match.c (filter_s): New symbol variable. + (v_filter): New function. + (syms_init): New symbol variable initialized. + (dir_tables_init): New function entered into table. + + * txr.1: Documented new filter directive. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + dep.mk: Regenerated. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + Shorthand for filters which map multiple texts to a common + replacement text. + + * filter.c (build_filter_from_list): Allow tuples to denote + multiple keys mapping to the same value. + + * lib.c (do_curry_123_2, do_curry_123_1): New static functions. + (curry_123_2, curry_123_1): New functions. + + * lib.h (curry_123_2, curry_123_1): New functions declared. + + * match.c (v_deffilter): Allow tuples of strings rather than + just pairs. + + * txr.1: Updated. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + * parser.y: Remove mention of nonexistent terminal \\ + from %right associativity clause. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (fun_k): New keyword variable. + (function_filter): Use :fun keyword symbol instead of fun. + (filter_init): New keyword variable initialized. + + * filter.h (upcase_k, downcase_k, fun_k): Declared. + + * txr.1: Updated. + +2011-10-25 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_bind): Use sem_error to throw errors with line number + info. + +2011-10-24 Kaz Kylheku <kaz@kylheku.com> + + Bugs #34641, #34629. + + * lib.c (search_str_tree): If multiple strings from the needle tree + matching within within the haystack string, then take the leftmost + match. If there are multiple matches at the same leftmost position, + take the longest one. + +2011-10-24 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (function_filter): New function. + (get_filter): Handle (fun ...) syntax. + + * match.c (v_bind): Establish dynamic environment frame around + dest_bind, and stash the bindings there so filters can have access + to the bindings. + (v_output): Likewise, around do_output calls. + (v_fun): New function. + (match_files): Function handling broken out into v_fun. + (match_funcall): New function. + + * match.h (match_funcall): Declared. + + * unwind.c (uw_push_env): Initialize match_context. + (uw_get_match_context, uw_set_match_context): New functions. + + * unwind.h (struct uw_dynamic_env): New member, match_context. + (uw_get_match_context, uw_set_match_context): Declared. + + * txr.1: Documented function filters. + +2011-10-24 Kaz Kylheku <kaz@kylheku.com> + + Turning attention to some plumbing. + + * unwind.c (uw_env_stack): New static variable. + (uw_unwind_to_exit_point): Maintain correct uw_env_stack + during unwinding. + (uw_find_env): Just retrieve the env stack pointer; no search. + (uw_push_env): Store a pointer to the previous + environmental frame and just initialize the bindings to nil. + No need to cons up a copy of the bindings from the previous frame. + (uw_get_func): Perform a search through the environment stack. + + * unwind.h (struct uw_dynamic_env): New member, up_env. + +2011-10-23 Kaz Kylheku <kaz@kylheku.com> + + * tests/007/except-1.txr: Use next :list instead + of piping from echo command. As a result, this test case should run on + MingW. + +2011-10-23 Kaz Kylheku <kaz@kylheku.com> + + * match.c (list_k, string_k): New keyword symbol variables. + (v_next): Implement :list and :string keywords. + (syms_init): New keyword variables initialized. + NOTE: the :var keyword is deprecated. + + * txr.1: Documented :list and :string. + +2011-10-23 Kaz Kylheku <kaz@kylheku.com> + + * match.c (h_skip): Bugfix: bad agument list in debugf call. + +2011-10-22 Kaz Kylheku <kaz@kylheku.com> + + Task #11474 + + * filter.c (filter_equal): Takes two filters instead of one. + (lfilt_k, rfilt_k): New keyword variables. + (filter_init): New keyword variables initialized. + + * filter.h (filter_equal): Declaration updated. + (lfilt_k, rfilt_k): Declared. + + * lib.c (funcall4): New function. + (do_curry_1234_34): New static function. + (curry_1234_34): New function. + (do_swap_12_21): New static function. + (swap_12_21): New function. + + * lib.h (funcall4, curry_1234_34, swap_12_21): Declared. + + * match.c (dest_bind): Swap use the function argument swapping + combinator when calling tree find such that the value + being searched is on the left and pattern material is on the right. + (v_bind): Implemented :lfilt and :rfilt. + + * txr.1: Documented :lfilt and :rfilt. + +2011-10-22 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (get_filter_trie): Function renamed to get_filter. A filter + is not necessarily a trie. + (string_filter, compound_filter): New functions. + (get_filter): Recognize a compound filters and return a function + which implements it. + + * filter.h (get_filter_trie): Declaration renamed. + + * match.c (format_field, v_bind, v_output): Follow get_filter_trie + rename. Error message text updated. + + * txr.1: Describe compound filters. + +2011-10-22 Kaz Kylheku <kaz@kylheku.com> + + Task #11474 + + * filter.c (filter_equal): New function. + (upcase_k, downcase_k): New keyword variables. + (filter_init): New keyword variables initialized, + and new upcase and downcase filters registered. + + * filter.h (filter_equal): Declared. + + * lib.c (tree_find): Takes new argument, the equality test function. + (upcase_str, downcase_str): New functions. + (do_curry_123_23): New static function. + (curry_123_23): New function. + + * lib.h (tree_find): Declaration updated. + (upcase_str, downcase_str, curry_123_23): Declared. + + * match.c (dest_bind): Updated to take equality function. + Uses it and passes it down to tree_find. + (v_bind): Filter feature implemented. + (h_var, v_try): Add equal_f to dest_bind argument list. + + * txr.1: Updated to describe new filters and bind arguments. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_collect, v_coll): Establish empty list bindings + for all :vars in the event that the collect turns up nothing. + + * txr.1: Document behavior. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_collect): Regression bugfix. Make it work like the comment + says: until/last clause has visibility to uncollated bindings from + collect. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + Implementing @(set) directive for assigning to variables + destructively. + + * match.c (dest_set, v_set): New static functions. + (dir_tables_init): Add v_set to vertical directives hash table. + + * txr.1: Documented. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + * match.c (v_output): When appending output to a variable, + flatten the previous contents so we can append to a single + string, or to deeply nested list, etc. + + * txr.1: Documented these new extensions to next and output. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + New features. Strling list output streams in stream + library, allow output to be captured as a list of strings + representing lines (in contrast to string streams which capture + a single string). + + The output directive can output to a variable, + and next can scan over a variable. + + * lib.c (span_str, compl_span_str, break_str): New functions. + + * lib.h (span_str, compl_span_str, break_str): New functions declared. + + * match.c (into_k, var_k): New keyword variables. + (mf_file_data): New static function. + (v_next): Refactored argument handling. Added support for :var + keyword. + (v_output): Added support for :into keyword. + + * stream.c (strlist_mark, strlist_out_put_string, + strlist_out_put_char): New static functions. + (strlist_out_ops): New static struct. + (make_strlist_output_stream, get_list_from_stream): New functions. + + * stream.h (make_strlist_output_stream, get_list_from_stream): New + functions declared. + +2011-10-21 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (proper_plist_to_alist, improper_plist_to_alist): New + functions. + + * lib.h (proper_plist_to_alist, improper_plist_to_alist): New + functions declared. + + * match.c (append_k): New keyword symbol variable. + (complex_open): New append argument. + (v_output): Streamlined parsing of keywords. + Support :append keyword. + + * txr.1: Output directive's keyword documentation revised. + +2011-10-20 Kaz Kylheku <kaz@kylheku.com> + + Bug #34609 + + * match.c (v_block): Regression induced by rabid refactoring. Block + must apply remaining directives to data, excluding itself, otherwise + runaway recursion takes the place of correct behavior. + +2011-10-19 Kaz Kylheku <kaz@kylheku.com> + + Version 040 + + Single unescaped space behaves like @/ +/ regex. + + Ported to native Windows via MinGW. + + Bugfixes for Cygwin and more robust handling of + errors arising from Windows not having proper + Unicode support (16 bit wide characters only). + + Nasty GC bug fixed for all platforms, exposed by gcc 4.5.2, x86_64. + + [Internal] The huge functions match_line and match_files have been + broken up into functions dispatched by hash table lookup on directive + symbols. + + [Internal] Hashing of some objects improved. + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2011-10-19 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + * match.c (repeat_spec_k): New symbol variable + (h_match_func): New typedef. + (elem_bind): New macro. + (h_var, h_skip, h_coll, h_parallel, h_trailer, h_eol): New functions. + (match_line): Remaining directives moved to functions. + (syms_init): New symbol variable initialized. + (dir_tables_init): New functions entered into hash table. + +2011-10-19 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + Refactoring match_files to make it easier to break up into + subfunctions, similarly to what was done with match_files. + + * match.c (match_line_ctx): New struct type. + (ml_all, ml_specline, ml_bindings_specline): New functions. + (LOG_MISMATCH, LOG_MATCH): Macros moved outside of function, + updated to refer to structure members rather than local + variables. + (match_line): Takes only one argument now. + All recursive calls updated. + (v_freeform): Call to match_line updated. + (match_files): Likewise. + +2011-10-19 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + * match.c (v_accept_fail, v_next, v_parallel, v_collect, + v_flatten, v_forget_local, v_merge, v_bind, v_cat, v_output, + v_try, v_define, v_defex, v_throw, v_deffilter, v_eof): + New functions. + (match_files): Remaining directives moved to functions. + (dir_tables_init): New functions entered into hash table. + +2011-10-19 Kaz Kylheku <kaz@kylheku.com> + + * hash.c (ll_hash): Hashing of pointers should take into + account alignment, otherwise only values divisible by the + alignment occur. This patch takes into considerations that + val values are pointers to object descriptors in a heap which + are four words wide, and so most likely aligned to 16 byte + boundaries (32 bit systems) or 32 byte boundaries (64 bit). + We need to shift. + +2011-10-18 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + * match.c (v_block): New function. + (match_files): Block directive moved to function. + (dir_tables_init): v_block entered into table. + +2011-10-18 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + * match.c (spec_bind): New macro. + (v_freeform): New function. + (match_files): Freeform logic moved to function. + (dir_tables_init): v_freeform entered into table. + +2011-10-18 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + * match.c (same_data_k): Symbol variable renamed to next_spec_k. + (v_skip): Restructured not to return next_spec_k when there + are no more specs, but rather thread directly to what match_file + will do anyway, namely return the bindings and data position. + (v_trailer): New functions. + (match_files): Trailer logic moved to function. + (syms_init): Follows renaming of variable. + (dir_tables_init): GC bugfix: did not protect global hash + tables again, like in filter.c previously. + +2011-10-17 Kaz Kylheku <kaz@kylheku.com> + + Task #11425 + + Vertical skip directive moved into function dispatched + via hash table. Test suite passes. + + * lib.c (cptr_s): New symbol variable. + (cptr_equal_op): New static function. + (cptr_equal_op, cptr, cptr_get): New functions. + (cptr_ops): New static structure. + (obj_init): New variable initialized. + + * lib.h (cptr_s, cptr, cptr_get): Declared. + + * match.c (decline_k, same_data_k): New symbol variables. + (v_match_func): New typedef. + (v_skip): New function. + (match_files): Check symbol in v_directive_table and dispatch + the associated function if an entry exists. + Skip directive handling moved to v_skip function. + (syms_init): Initialize new symbol variables. + (dir_tables_init): Enter v_skip into v_directive_table under + skip_s symbol. + +2011-10-16 Kaz Kylheku <kaz@kylheku.com> + + Quick and dirty port to MinGW. + + * configure: Test for presence of <sys/wait.h> added. + Conditionally generates HAVE_SYS_WAIT variable in config.h. + + * stream.c: Include <sys/wait.h> conditionally. + (pipe_close): Do not test ermination status with WIFEXITED, etc. + if there is no <sys/wait.h> header. + +2011-10-16 Kaz Kylheku <kaz@kylheku.com> + + * configure: reduced post-configure advice to just point + to the INSTALL guide. + + * INSTALL: New file. + +2011-10-16 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (trie_filter_string): Fix warning about uninitialized + variable (not a bug, but compiler cannot prove that). + +2011-10-15 Kaz Kylheku <kaz@kylheku.com> + + Task #11425. Refactoring match_files to make it easier to + break up into subfunctions. Arguments are packaged into a structure, + so that subfunctions won't have to all have big argument lists. + + * match.c (h_directive_table, v_directive_table): New variables. + (match_files_ctx): New structure. + (mf_all, mf_args, mf_data, mf_spec, mf_spec_bindings): New functions. + (match_files): Takes only one argument now, the context + structure. data_lineno variable is a dynamic number. + Recursive calls to match_files are handled by creating contexts + as appropriate with the helper functions. + The old local variable data is now part of the context. + (syms_init, dir_tables_init): New functions. + (match_init): Just calls syms_init and dir_tables_init. + +2011-10-15 Kaz Kylheku <kaz@kylheku.com> + + Fixed broken GC on x86_64 (Ubuntu 11, gcc 4.5.2). + The issues is that due to the aggressive function inlining + in the gc module, the mark_mem_region function is not real + subroutine. The address of its local variable &gc_stack_top + ended up excluding the machine context saved by setjmp in + the parent function. I.e. the buffer was not between the + computed stack top and bottom. Thus registers were not being + scanned for references to values. I added a little abstraction + to the machine context in the process of fixing this. + + * gc.c (struct mach_context, mach_context_t): New type. + (save_context): New macro. + (mark): Takes two new arguments, pointer to the stack top and + machine context. It scans the machine context explicitly rather + than relying it to be on the stack, between the top and bottom. + This context is in fact only object within the garbage collector part + of the activation chain that we need to scan. + (gc): Use new abstraction to save machine context. Local variable + is used to derive the stack top here. The stack top is the top + of the stack above the activation frames in the garbage collector + itself. The gc has nothing on its stack that should be scanned, + except for the machine context, which is now handled explicitly. + +2011-10-15 Kaz Kylheku <kaz@kylheku.com> + + * configure: POSIX Portability. Use = instead of == + in test expressions. This was revealed by ubuntu's dash. + +2011-10-13 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (elem): Amending previous change. A single space + should only denote multiple spaces, not mixtures of spaces and + tabs. WE have to be careful with tabs because they can be + semantically different from spaces (e.g. file with tab delimited + fields which can be blank, empty or have leading or trailing spaces.) + + * txr.1: Updated. + +2011-10-13 Kaz Kylheku <kaz@kylheku.com> + + * Makefile (%.ok: %.txr): Use unified diff for showing + differences between expected and actual test output. + + * parser.l (yybadtoken): Handle new terminal symbol, SPACE. + New rule for producing SPACE token out of an extent of + tabs and spaces. + + * parser.y (SPACE): New terminal symbol. + (o_var): New nonterminal. I noticed that the var rule was + being used for output elements, and the var rule refers to + elem rather than o_elem. A new o_var rule is a simplified + duplicate of var. + (elem): Handle SPACE token. Transform to regex if it is + a single space, otherwise to literal text. + (o_elem): Handle SPACE token in output. + + * tests/001/query-2.txr: This query depends on matching + single spaces and so needs to use escapes. + + * tests/001/query-4.txr, test/001/query-4.expected: New test + case, based on query-2.txr. It produces the same output, + but is simpler thanks to the new semantics of space. + + * txr.1: Documented. + +2011-10-12 Kaz Kylheku <kaz@kylheku.com> + + Bug #34538 + + * lib.h (wli): This macro now does the pointer displacement by 1. + (auto_str, static_str): #if/#else/#endif gone. These functions just + add the type tag. The + 1 logic was incorrect; it should have + been + sizeof(wchar_t). But even that was not right because other + code expects a wchli_t * to point to the first character, + such as the string_out_put_char function. + +2011-10-10 Kaz Kylheku <kaz@kylheku.com> + + Improved support for broken unicode. + Regex support for extra-large character sets not compiled in + if wchar_t is not wide enough for it. + The utf-8 properly throws exceptions when encountering characters + that it cannot represent, instead of silently ignoring the + situation and continuing with incorrectly computed data. + + * regex.c (FULL_UNICODE): New macro. + (CHAR_SET_L3, CHAR_SET_L2_LO, CHAR_SET_L2_HI): Only defined + if full unicde is available. + (CHSET_XLARGE, cset_L3_t, struct xlarge_char_set, + L2_full, L3_fill_range, L3_contains): Ditto. + (unon char_set): Member x1 present only under FULL_UNICODE. + (char_set_destroy, char_set_add, char_set_add_range, + char_set_contains): CHSET_XLARGE cases only available on + FULL_UNICODE. + (char_set_compile): Default cst variable to CHSET_LARGE. + + * utf8.c (FULL_UNICODE): New macro. + (conversion_error): New function. + (utf8_from_uc): Throw error if not FULL_UNICODE and character is + outside the BMP. + (utf8_decode): Likewise. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Documented portability hacks for narrow wchar_t. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + Version 039 + + Ported to Cygwin. + + Horizontal modes for @(trailer), @(skip). + + New :greedy keyword in skip which can be given instead of max + distance to give it longest match semantics. + + @(collect) and @(coll) support a new clause, @(last). + + The :times keyword in @(collect) and @(coll) introduced in the + previous release has a different meaning. The keywords :mintimes + and :maxtimes are added, and :maxtimes behaves like :times + did previously. + + There is a :vars keyword in @(collect) and @(coll) to have + some control over what bindings are collected, as well + as error checking for missing bindings and defaulting behavior. + + New @(eol) directive for explicitly matching the end of + the input or end of a line. + + New lexical syntax: @(...) and @abc is allowed within expressions. + This produces a special structural syntax with no assigned meaning yet. + + Awful bug fixed in function calling: if a function was called with + multiple unbound variables, and bindings were produced for them, + only one variable was being propagated to the calling environment. + + Bugfixes to binding environment handling in the face of + @(local)/@(forget) directive. + + Fix for the issue of unbound variables being silently ignored + in some contexts, like quasiliterals. An exception is now thrown. + + Bugfix for an issue with consecutive variables in output. + + Bugfix for an issue with horizontal @(cases) not collecting + bindings. + + Bugfix for an issue with @(until) inside @(coll) not seeing + bindings from main clause. + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + One more swing at this with the axe. + + * lib.h (wini, wref): New macros. + + * stream.c (string_out_put_char): Rewritten with macros to eliminate + preprocessor #if test. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + * lib.h (wli, lit_noex): We need null characters on both ends + so that this hack is correct for null strings. When recovering + the wchar_t pointer from a null literal object, we wil increment + unconditionally, since it always points to a null character. + We end up skipping past null terminator #1, but safely landing on #2. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + Following up to previous commit's TODO. + + * filter.c (struct filter_par): wchar_t becomes wchli_t. + + * lib.h (wchli_t): New type: an incomplete structure type, + so that a pointer to this type is incompatible with anything else. + (wli): Macro produces const wchli_t * pointer instead of + const wchar_t *. + (auto_str, static_str): Accept a const wchli_t * instead + of const wchar_t *, making it impossible to misuse these + functions by passing in a literal. + + * stream.c (string_out_put_char): These type changes showed + this hack to have a bug. Confronted with the need to cast + from const wchar_t * to const wchli_t *, it's obvious that + the conversion has to be done properly with the + 1 in the + one platform case, but not the other. + + * txr.c (version): Type changed to const wchli_t. + + * txr.h (version): Declaration updated. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + Ported to Cygwin. + + TODO: there should be some type safety with the new wli macro + so that if it is forgotten, there will be a diagnostic. + + * configure (lit_align): New configuration variable + and configuration test. Generates LIT_ALIGN in config.h. + Fixed the integer-holds-pointer test for the different output + from the nm program on Cygwin. The arrays become common symbols + marked C which do not show an offset attribute, only size: + one less column. + + * filter.c (to_html_table, from_html_table): wrap wide string + literals with the wli macro. This must be done from now on for + all literals and initializes of arrays that are going to be + directly converted to type tagged val-s. + + * lib.h (wli): New macro. + (auto_str, static_str, litptr, lit_noex): Handle wide literals on + platforms where they are aligned to only two bytes, such that we don't + have two bits in the pointer. We can still add our 11 bit type tag, but + then when recovering the pointer to the data, we have may have + to fix up the pointer. + + * parser.l: Another portability issue here. Flex generates a scanner + which has #include <unistd.h> in the middle, after the source file's + own #includes which can introduce macros. On Cygwin, there is some + hygiene problem whereby our "noreturn" macro causes the <unistd.h> + header to generate bad syntax and fail to compile. Stupid Cygwin + and even stupider flex! The workaround is to include <unistd.h> + at the top in the flex source. + + * stream.c (string_out_put_char): This is one more place where + the string literal handling hack spreads. + + * txr.c (version): Wrap string in wli. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + * dep.mk: Regenerated. Too easy to neglect this file. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + * match.c (vars_to_bindings): Regression fix: recent commit + caused test failure. An empty list not treated as a valid collect + variable list. + +2011-10-09 Kaz Kylheku <kaz@kylheku.com> + + * configure: Fixed indentation. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Removed references to obsolete @(next) variant. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * match.c (vars_to_bindings): New function. + (match_line): keyword argument :vars implemented for coll. + + * txr.1: Documented :vars. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * match.c (vars_k): New symbol variable. + (match_files): Implemented :vars in collect. + (match_init): New symbol variable initialized. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Augment example of @/.*/ being used to skip to the + end of the line with @(skip) which is now better style, + since it avoids reaching for regexes. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line): Skip directive bugfix. If skip is the + last item on the line, it must match the whole line by returning + success. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * match.c (mintimes_k, maxtimes_k): New keyword variables. + (match_line): Implemented :mintimes and :maxtimes, changing + the semantics of :times. + (match_files): Likewise. + (match_init): New keyword variables initialized. + + * txr.1: Updated. + +2011-10-08 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Formatting. + +2011-10-07 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): Fixed spectacular bug in function calling, + dating back to before October 2009 when txr was put into git. + Basically, unbound variables were not handled right after + the function return, due to the increment step being wrongly + written as ``piter = cdr(aiter)'' in the for loop that processes the + ub_p_a_pairs. Evil cut and paste! + +2011-10-07 Kaz Kylheku <kaz@kylheku.com> + + * match.c (greedy_k): New keyword symbol variable. + (match_line): Greedy skip implemented. + (match_files): Likewise. + (match_init): New keyword symbol variable initialized. + + * txr.1: Updated. + +2011-10-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (eol_s): New symbol variable. + (obj_init): New variable initialized. + + * lib.h (eol_s): Declared. + + * match.c (match_line): Implemented horizontal skip as and + new eol directive. + (match_lines): Vertical skip defers to horizontal skip if + there is trailing material. + + * txr.1: Updated. + +2011-10-07 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (flatten_helper): Function removed. + (flatten): Recurse directly, using func_n1. + +2011-10-07 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: fxed wrong word. + +2011-10-06 Kaz Kylheku <kaz@kylheku.com> + + Extending syntax to allow for @VAR and @(...) forms inside + nested lists. This is in anticipation of future features. + + * lib.c (expr_s): New symbol variable. + (obj_init): expr_s initialized. + + * lib.h (expr_s): Declared. + + * match.c (dest_bind): Now takes linenum. Tests for the meta-syntax + denoted by the system symbols var_s and expr_s, and throws an + error. + (eval_form): Similar error checks added. Also, hack: do not add + file and line number to an exception which begins with a '(' + character; just re-throw it. This suppresses duplicate line + number addition when this throw occurs across some nestings. + (match_files): Updated calls to dest_bind. + + * parser.l (yybadtoken): Handle new token kind, METAVAR and METAPAR. + (grammar): Refactoring among patterns: TOK broken into + SYM and NUM, NTOK introduced, unused NUM_END removed. + Rule for @( producing METAPAR in nested state. + + * parser.y (METAVAR, METAPAR): New tokens. + (meta_expr): New nonterminal. + (expr): meta_expr and META_VAR productions handled. + +2011-10-06 Kaz Kylheku <kaz@kylheku.com> + + Renaming the currying combinators according to new scheme. + + * lib.c (bind2): Function renamed to curry_12_2. + (bind2other): Function renamed to curry_12_1. + (do_bind_2, do_bind2other): Helpers renamed likewise. + (tree_find): Follows rename of bind2. + + * match.c (match_files): deffilter code follows bind2 rename + to curry_12_2. + +2011-10-06 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (funcall3, curry_123_2): New functions. + (do_curry_123_2): New static function. + + * lib.h (funcall3, curry_123_2): Declared. + + * match.c (subst_vars): Bugfix: throw error on unbound variable instead + of ignoring the situation. This bug caused unbound variables in + quasiliterals to be silently ignored. + (eval_form): Function changed to three argument form, so that + it takes a line number for reporting errors. Restructured to catch + the new unbound variable exception from subst_vars, and re-throw + it with a line number. Also, throws exception now instead of returning + nil if itself it detets an unbound variable. Uses of eval_form + no longer have to test the return value for nil, but just assume + it worked. + (match_lines): Currying calls to eval form updated to use + curry_123_2. Test of eval return value eliminated. In function + calls, eval isn't used for reducing symbol arguments to values, + because it now throws in the unbound case, and it's not worth + setting up a catch for this. Instead, assoc is used directly. + +2011-10-05 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): In function calls, the deletion of + the unbound variable from the argument list can be done + with a destructive operation since that list is a copy. + +2011-10-04 Kaz Kylheku <kaz@kylheku.com> + + * LICENSE, Makefile, configure, filter.c, filter.h, gc.c, gc.h, hash.c, + hash.h, lib.c, lib.h, match.c, match.h, parser.h, parser.l, parser.y, + regex.c, regex.h, stream.c, stream.h, txr.1, txr.c, txr.h, unwind.c, + unwind.h, utf8.c, utf8.h: Updated e-mail address. + +2011-10-04 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line, match_files): Another correction to how bindings + are handled in collect/coll. New bindings from the main clause and + last clause must override old bindings. This is done by some + additional set difference operations based on symbol identity. + Otherwise it is possible to end up with multiple bindings for the + same symbol, which is untidy. If the collect clause scrubs a variable + with forget and re-binds it, then combining that environment + with the previous bindings will create a duplicate. + Also, fixed a serious bug with the bindings from the last clause; + the append was wrongly put into the loop that processes the collected + lists. + +2011-10-04 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (acons): New function. + (set_diff): Optimize common case: list1 and list2 + are the same, or list2 is substructure of list1. + Situations in which this won't be the case for variable bindings are + rare. + + * lib.h (acons): Declared. + + * match.c (match_line): Use acons rather than acons_new, when binding + variables that we know are new (the symbol is unbound). + When computing the set difference over bindings, use cons cell + equality, rather than symbol equality. Symbol equality is wrong + because a binding can be removed, and then a new binding can be + introduced using the same symbol. This must be treated as + a different binding. + +2011-10-04 Kaz Kylheku <kaz@kylheku.com> + + Bugfixes to the semantics of binding environments, which + were broken in the face of deletions (local, forget). + For some stupid reason, I had written a destructive routine for + removing elements from an association list, and used it + as the basis for the local and forget directives. + + * lib.c (eq_f, car_f): New variables. + (identity_tramp, equal_tramp): Obsolete functions removed. + (apply): Broken function disabled at run time. + (funcall, funcall1, funcall2): Throw meaningful error instead + of aborting. + (alist_remove_test): New static function. + (alist_remove, alist_remove1): Rewritten to be functional + rather than destructive. + (alist_nremove, alist_nremove1): Destructive functions, + using previous implementations of alist and alist_nremove. + (do_sort): Recurses directly rather than via sort. That was + probably why this helper was introduced! + (find, set_diff): New functions. + (obj_init): gc-protect new variables eq_f and car_f, and initialize + them. Initializations for equal_f and identity_f changed to + use equal and identity directly, without the obsolete wrappers. + + * lib.h (eq_f, car_f, alist_nremove, alist_nremove1, + find, set_diff): Declared. + + * match.c (match_line): Use set_diff to determine what bindings + are new, rather than ldiff and ldiff-like logic which break when + the new bindings do not share structure with the old. + (match_files): Likewise. + +2011-10-03 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Starte dodcumenting the forgotten merge directive. + +2011-10-03 Kaz Kylheku <kaz@kylheku.com> + + Implemented new last clause for collect and coll. + Bugfix in cases inside coll: was not collecting bindings. + Bugfix for until inside coll: was not seeing bindings + from main clause. + + * lib.c (ldiff): New function. + + * lib.h (ldiff): Declared. + + * match.c (match_line): Implemented last clause. Fixed cases + handling by moving misplaced termination check. + (match_files): Implemented last clause. + + * parser.y (until_last): New nonterminal symbol. + (collect_clause): Refactored syntax to support until and last. + (elem): Likewise. + + * txr.1: Updated. + +2011-10-02 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (rep_elem): Bugfix: forgotten o_elems_transform on + syntax tree of o_elems constituent, leading to problems with + consecutive variables in a @(rep). + +2011-10-02 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line): Handle trailer_s directive. + (match_files): Remove check against trailer_s not having trailing + material. If it doesn't, it's a vertical directive processed here, + otherwise leave it alone so match_line processed it. + +2011-10-02 Kaz Kylheku <kaz@kylheku.com> + + Compiles as C++ again. + + * lib.h (cons_set): New macro. + + * match.c (match_line, match_files): In collect clause handlers, + move variable declarations above goto, and initialize with + cons_set, instead of declaring and initializing with cons_bind. + This eliminates the stupid C++ error that goto skips a variable + initialization (which happens even when it can be trivially + proven that the has no next use at the goto site!) + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + Version 038 + + New eof directive. + + Fixes in skip directive to work very well with eof. + + Consecutive variable matching semantics improved; concept of double + variable match introduced for unbound variable followed by + regex variable. + + Directives collect and coll have keyword arguments for more control + over their behavior. + + Parallel directives (all, some, none, ...) are available in + horizontal mode. + + New choose directive for selecting one of numerous alternatives + + GC bugfix in new filtering code. + + The code has an issue compling with GNU C++ instead of C, + which is something that is supported by this project. + Not a release-blocking issue. Not easy to fix without + restructuring some code. + + * txr.c (version): Bumped. + + * txr.1: Bumped version and set date. + + * configure (txr_ver): Bumped. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + Maintaining C++ compiling (except for two issues that will + need another commit). + + * filter.c: Include "gc.h" for prototype of protect. + (struct filter_pair): Use const wchar_t *, so we can assign + literals. + (html_hex_continue): Ditto. + + * lib.c (and): Function renamed to andf, since and is a C++ + operator. + + * lib.h (and): Declaration renamed. + + * match.c (match_files): Use of and updated to andf. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + HACKING: Clarified that --vg-debug is also needed to turn on on + the Valgrind support at run-time, in addition to building it in. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + New test case, covering some filtering from HTML/XML. + + * Makefile: Defined TXR_ARGS for new test case. + + * tests/008/students.expected: New file. + * tests/008/students.txr: New file. + * tests/008/students.xml: New file. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + * filter.c (filters, filter_init): Serious gc bug fixed: neglected to + inform the garbage collector about the filters global variable. + Ouch! + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + New test case under tests/008. + + * Makefile: Made previous TXR_ARGS for 008 specific + to tokenizing test case, and introduced separate TXR_ARGS + for this test case. + + * tests/008/configfile: New file. + + * tests/008/configfile.expected: New file. + + * tests/008/configfile.txr: New file. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + Tokenizing test case, exercising for @(coll :gap 0) + and horizontal @(choose :shortest ...). + + * Makefile: Defined TXR_ARGS for tests/008 directory. + + * tests/008/data: New file. + * tests/008/tokenize.expected: New file. + * tests/008/tokenize.txr: New file. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + New test case, covering exception handling across nested + function invocations. + + * Makefile (TEST): Test targets marked as .PHONY, because they are. + + * tests/007/except-1.expected: New file. + * tests/007/except-1.txr: New file. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + * parser.y (all_clause, some_clause, none_clause, maybe_clause, + cases_clause, choose_clause, elem): Regression bug fix: bad list calls + in parser, lacking nao terminator. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + Regression bug fix: longest match variables broken by + 2011-09-28 commit which introduced the double var match. + + * match.c (match_line): Handle case where modifier is t. + + * parser.y (var_op): Produce modifir as (t) rather than t. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Documented choose and horizontal mode for paralle + constructs. + +2011-10-01 Kaz Kylheku <kaz@kylheku.com> + + New directive: choose. + + * match.c (choose_s, longest_k, shortest_k): New variables. + (match_line, match_files): Introduced choose directive. + (match_init): Initialize new variables. + + * match.h (choose_s): Declared. + + * parser.l (yybadtoken): Handle CHOOSE. + (CHOOSE): Clause added for returning this token. + + * parser.y: Added #include "match.h". + (CHOOSE): New token symbol. + (choose_clause): New nonterminal symbol. + (clause): choose_clause added. + (all_clause, some_clause, none_clause, maybe_clause, + cases_clause): Abstract syntax tree tweaked. + (choose_clause): New syntax. + (elem): Abstract syntax trees tweaked for many clauses. + New CHOOSE clauses. + (out_clause): New error case for choose_clause. + +2011-09-30 Kaz Kylheku <kaz@kylheku.com> + + * HACKING: Updated with debugging hints. + +2011-09-29 Kaz Kylheku <kaz@kylheku.com> + + * txr.1: Clarified consecutive variables and documented double + variable match. + +2011-09-29 Kaz Kylheku <kaz@kylheku.com> + + * parser.l: Implemented backslash continuations in SPECIAL + state, regexes and string literals. + + * txr.1: Documented. + +2011-09-29 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line): Implemented horizontal all, some, + none, maybe and cases directives. + (match_files): Recognize horizontal version of these directives + by the presence of the extra symbol t and do not process. + Also, bugfix in the all directive: not resetting the + all_match flag when short circuiting out. + * parser.y (clause_parts_h, additional_parts_h): New nonterminals. + (elem): New clauses added. + +2011-09-29 Kaz Kylheku <kaz@kylheku.com> + + * match.c (chars_k): New variable. + (match_line): Keyword arguments in coll implemented. + (match_init): chars_k variable initialized. + + * parser.l (COLL): Lexical syntax changed to allow for + argument material. + + * parser.y (elem): Coll syntax rewritten for arguments. + + * txr.1: Updated. + +2011-09-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (mingap_k, maxgap_k, gap_k, times_k, lines_k): New + symbol variables. + (match_lines): Keyword arguments in collect implemented. + (match_init): New function. + + * match.h (match_init): Declared. + + * parser.l (COLLECT): Lexical syntax changed for COLLECT to + allow for argument material. + + * parser.y (%union): obj renamed to val. + (exprs_opt): New nonterminal. + (collect_clause): Rewritten for arguments. + + * txr.c (main): Call to match_init introduced. + +2011-09-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line): Bugfix in double var. Do not + prepend the next_pat to the specline if it is nil. + +2011-09-28 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_line): Logic restructured to allow for + regex variables which also have nested variables. + Previously this code was assuming that the cases were + mutually exclusive, and the parser happened to work that way. + Also, added support for a "double var" match which occurs + when an unbound variable is followed by a regex variable. + This case should be allowed because it makes sense. + It's similar to a variable followed by a regex, except + that the regex is also a variable binding. + + * parser.y (o_elems_transform): New function. + (o_elems_opt, o_elems_opt2, quasilit): Transform o_elems with new + function. This is needed because subst_vars doesn't + deal with the nested var syntax for consecutive variables. + (var): New syntax case '{' IDENT exprs '}' elem. This + allows consecutive variables to be nested in all cases. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * parser.y ('{', '}'): Nope, still not right. + These must have exactly the same precedence as + IDENT for this to work right, of course. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * parser.y ('{', '}'): Bugfix: precedence of these + terminals was causing @foo@foo to be parsed differently + from @foo@{foo}. We need consecutive variables to be + specially folded in the syntax under a single var_s + node. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): One more fix to this, argh. + The test for !data should be done after matching, + before incrementing to the next line. Then it is a true + bottom of the loop test. This commit allows + @(skip) + @first_line + @(skip nil 3) + @(eof) + to correctly match the first line of the input, not + the fourth one from the bottom, since the + second skip has an unbounded range. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): Another bugfix to skip. + If a hard skip tries to go beyond EOF, then the query + must fail. However, a skip to exactly EOF is fine. + I.e. data can hit nil at the same time as the right + number of skip iterations is performed. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): Bugfix in skip directive. + We should try the match at least once even if there is no data + after a hard skip, so that the query has an opportunity + to do an explicit match for no data, as with @(endp). + This commit makes possible queries like: + @fourth_line_from_bottom + @(skip 1 3) + @(eof) + This query depends on @(skip 1 3) not failing when + it runs out of data, because @(eof) checks for htis. + +2011-09-27 Kaz Kylheku <kaz@kylheku.com> + + * lib.c (eof_s): New symbol variable. + (obj_init): New variable initialized. + + * lib.h (eof_s): Declared. + + * match.c (match_files): New @(eof) directive explicitly + matches end of data. + + * txr.1: Updated. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + Version 037 + + Short-circuiting behavior for @(all) and @(none). + + Obsolete forms of @(next) and @(output) syntax are gone. + + New filtering feature for substitutions in output. Filtering to and + from HTML built in, plus user-defined filtering with deffilter. + + Bugfixes: wrong error message in throw; lack of support for escaping + backslashes in literals and regexes. + + * txr.c (version): Bumped to 037. + + * txr.1: Set version to 037 and bumped date. + + * configure: Bumped txr_ver to 037. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + Support &#NNNN; decimal escapes also. + + * filter.c (html_hex_continue): Bail with nil if no digits + are collected. The &#x; syntax is not translated to anything. + (html_dec_continue): New function. + (html_hex_handler): Function renamed to html_numeric_handler. + (filter_init): Change function-based trie node over to + html_numeric_handler. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + Support &#xNNNN; hex escapes in html. Bugfix in field formatting. + chr function inlined. + + * filter.c (trie_value_at, trie_lookup_feed_char): Handle function + case. + (build_filter): New parameter, compress_p. + (html_hex_continue, html_hex_handler): New functions. + (filter_init): Add a function-based node to the from_html trie. + + * lib.c (chr): Function removed. + (functionp) New function. + + * lib.h (chr): Declaration replaced with inline function. + (functionp): Declared. + + * match.c (format_field): Bugfix: failed to apply filter + that came in as an argument. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + Bugfixes: Consistent escaping in various literals. Double + backslash codes for single backslash. Output clause can be empty. + + * parser.l (char_esc): Backslash handled. + Use internal_error rather than abort. + (REGCHAR, LITCHAR): Backslash added to lexical syntax. + + * parser.y (output_clause): Allow empty output clause. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + New feature: @(deffilter) + + Bugfix in @(throw) when non-symbol is thrown: exception message + referred to the symbol throw rather than the erroneous object. + + * filter.c (build_filter_from_list, register_filter): New functions. + + * filter.h (register_filter): New function declared. + + * lib.c (deffilter_s): New variable defined. + (chain): Function changed from single list argument to variable + argument list to reduce the complexity of use. + (do_and, and): New functions. + (obj_init): deffilter_s initializatio added. + + * lib.h (deffilter_s, and): New declarations. + (chain): Declaration updated to new function signature. + (eq): Changed from macro to inline function. + + * match.c (do_output_line): Simplified expression involving chain. + (do_output): Likewise. + (match_files): Bugfix in error handling of throw. + Implementation of deffilter. + + * txr.1: Documented deffilter. + +2011-09-26 Kaz Kylheku <kaz@kylheku.com> + + Trie compression. Hash table iteration. + Bugfix in typeof. + + * filter.c (trie_compress): New function. + (trie_value_at, trie_lookup_feed_char, filter_string): Handle cons cell + nodes in trie. + (build_filter): Call trie_compress. + + * gc.c (cobj_destroy_op): Function renamed to cobj_destroy_stub_op + since it doesn't do anything. + (cobj_destroy_free_op): New function. + * hash.c (struct hash_iter): New type. + (hash_destroy): Function removed. + (hash_ops): Reference to hash_destroy replaced with + cobj_destroy_free_op. + (hash_count, hash_iter_mark, hash_begin, hash_next): New functions. + (hash_iter_ops): New static structure. + * hash.h (hash_count, hash_begin, hash_next): New functions declared. + * lib.c (hash_iter_s): New symbol variable. + (typeof): Bugfix: TAG_LIT type tag not handled. + (vecref): New function. + (obj_init): Initialize hash_iter_s. + * lib.h (cobj_destroy_op): Declaration renamed. + (cobj_destroy_free_op, vecref): New functions declared. + (hash_iter_s): New variable declared. + * stream.c (string_in_ops, byte_in_ops): cobj_destroy_op + renamed to cobj_destroy_stub_op. + +2011-09-25 Kaz Kylheku <kaz@kylheku.com> + + Filtering from HTML implemented. + + * filter.c (from_html_k): New variable. + (to_html_table): New static array. + (filter_init): Intern new symbol. Instantiate new filter + and store in filters hash. + +2011-09-25 Kaz Kylheku <kaz@kylheku.com> + + Filtering feature for variable substitution in output. + + * filter.c, filter.h: New files. + + * Makefile (OBJS): filter.o added. + + * gc.c (mark_obj): Mark new alloc field of string objets. + + * hash.c (struct hash): New member, userdata. + (hash_mark): Mark new userdata member of hash. + (make_hash): Initialize userdata. + (get_hash_userdata, set_hash_userdata, hashp): New functions. + + * hash.h (get_hash_userdata, set_hash_userdata, hashp): New functions + declared. + + * lib.c (getplist, string_extend, cobjp): New functions. + (string_own, string, string_utf8): Initialize new alloc field to nil. + (mkstring, mkustring): Initialize new alloc field to actual size. + (length_str): When length is computed and cached, also compute + and cache alloc. + (init): Call filter_init. + + * lib.h (string string): New member, alloc. + (num_fast): Macro converted to inline function. + (getplist, string_extend, cobjp): New functions declared. + + * match.c (match_line): Follows change of modifier s-exp syntax. + (format_field): New parameter, filter. + New modifier syntax parsed. Filter retrieved, and applied. + (subst_vars): New parameter, filter. Filter is either applied + in this function or passed to format_field, as needed. + (eval_form): Pass nil to new parameter of subst_vars. + (do_output_line): New parameter, filter. Passed down to subst_vars. + (do_output): New parameter, filter. Passed down to do_output_line. + (match_files): Pass nil filter to subst_vars in cat directive. + Output directive refactored to parse keywords, extract the + filter and pass down to do_output. + + * parser.y (regex): Generate (sys:regex regex syntax ...) + instead of (regex syntax ...). + (elem, expr): Updated w.r.t. regex syntax change. + (var): Cases '{' IDENT regex '}' and '{' IDENT NUMBER '}' + are removed. new syntax '{' IDENT exprs '}' to handle these + more generally and allow for keywords. + + * txr.1: Updated. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + Numeric constants become real constants. + + Vector code cleanup. + + * lib.h (zero, one, two, negone, maxint, minint): Extern declarations + removed, macros introduced for these identifiers. + + * lib.c (zero, one, two, negone, maxint, minint): File scope + definitions removed. + (vector): Use vec_alloc and vec_fill enums instead of constants. + (obj_init): Remove references to removed definitions. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + * LICENSE, Makefile, configure, gc.c, gc.h, hash.c, hash.h, lib.c, + lib.h, match.c, match.h, parser.h, parser.l, parser.y, regex.c, + regex.h, stream.c, stream.h, txr.1, txr.c, txr.h, unwind.c, unwind.h, + utf8.c, utf8.h: Updated copyright year. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + * match.c, parser.y: Support for old output syntax removed. + Leading :nothrow with trailing material is an error now. + + * txr.1: Updated. Made note of errors in pipes being asynchronous. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + * tests/002/query-1.txr: Old next syntax rewritten to new. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + * match.c (match_files): Some cleanup in preparation of new + features. Support for obsolescent @(next) syntax is gone. + +2011-09-23 Kaz Kylheku <kaz@kylheku.com> + + Semantics tweak: short circuiting behavior for @(all) and @(none). + + * match.c (match_files): Added a couple of break statements. + + * txr.1: Updated. + +2011-09-22 Kaz Kylheku <kaz@kylheku.com> + + Version 036 + + Extension to @(skip). + + * txr.c (version): Bumped to 036. + + * txr.1: Set version to 036. + + * configure: Bumped txr_ver to 036. + +2011-09-22 Kaz Kylheku <kaz@kylheku.com> + + Useful second argument in skip directive for skipping + a minimum number of lines. + + * match.c (match_files): New behavior in skip_s case. + + * txr.1: Documented. + +2010-10-05 Kaz Kylheku <kkylheku@gmail.com> + + Version 035 + + Fixes garbage-collection-related problem affecting @(freeform) + that was revealed by "make tests" on x86-64 system, gcc 4.3.2. + + Fixes show-stopper stupidity, again: a query that matches + the end of data terminates successfully rather than failing. This + is because version 032 did not properly address the breakage + introduced in the 2010-01-21 change to match.c involving the dataline + variable. The "fix" only prevented the program from aborting in that + situation. + + * txr.c (version): Bumped to 035. + + * txr.1: Set version to 035. + + * configure: Bumped txr_ver to 035. + +2010-10-04 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): Bugfix. A (sub)query that runs out of data + lines to match must fail. Extra data lines relative to the spec + are tolerated; extra spec lines unmet by data aren't. + +2010-10-03 Kaz Kylheku <kkylheku@gmail.com> + + * lib.h (lazy_string): Fix incorrect comment. + + * lib.c (split_str, split_str_set): It is necessary to protect input + parameters against GC, because we cache their internal pointers, + after which we no longer refer to the objects themselves. Moreover, + we perform object allocation, and then keep using the internal + pointers. + +2010-09-30 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fix formatting problem. + +2010-03-01 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fix inaccuracies: files are not read into memory all at + once, and a query doesn't execute if it had errors. + +2010-02-28 Kaz Kylheku <kkylheku@gmail.com> + + Version 034 + + Patched up broken @(freeform) directive. + + * txr.c (version): Bumped to 034. + + * txr.1: Set version to 034. + + * configure: Bumped txr_ver to 034. + +2010-02-28 Kaz Kylheku <kkylheku@gmail.com> + + New testcases for freeform. + + * tests/006/data: New UTF-8 file. + + * tests/006/freeform-1.txr: Likewise. + + * tests/006/freeform-1.expected: Likewise. + + * tests/006/freeform-2.txr: Likewise. + + * tests/006/freeform-2.expected: Likewise. + + * Makefile (TXR_ARGS): New target-specific assignment to set + data for test case set 006. + +2010-02-27 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (length_str_gt, length_str_ge, length_str_lt, length_str_le): + Added missing support for literal string type. + +2010-02-27 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (search_str): Bugfix for empty haystack case: checks for end + of string must use postincrement on the index, otherwise the access + goes past the null terminator. + +2010-02-27 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_lines): Bugfix in freeform directive. + If the virtual line is partially matched, the remainder of + the line is folded back into list form. In this case, the + data line number must be incremented. Otherwise the calling + context may conclude that no progress was made, and + skip a line of input. I.e. the unmatched part of the input + is a new line, even if there had originally + been no line break at that point. + +2010-02-27 Kaz Kylheku <kkylheku@gmail.com> + + * lib.h (split_str_sep): Declared. + + * lib.c (split_str_sep): New function. + (split_str): Semantics changed; the second argument + is not a set of separator characters (like in split_str_sep) + but rather a separator string. Fixed bug: if the input + string is empty, the output list is empty. This caused + infinite looping behavior in @(freeform). + +2010-02-24 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (init_str): Bugfix: copy only len characters, not len + 1, so + that we don't clobber the null terminator in the target string, or try + read past the end of the source data. This affects the @(freeform) + directive. + +2010-01-26 Kaz Kylheku <kkylheku@gmail.com> + + Version 033 + + Addressed exponential memory behaviors + in derivative-based regex matching. + + * txr.c (version): Bumped to 033. + + * txr.1: Set version to 033. + + * configure: Bumped txr_ver to 033. + +2010-01-26 Kaz Kylheku <kkylheku@gmail.com> + + hash.c (hash_process_weak): There is no point in fixing up + the type codes of spuriously reached nodes; reached objects + will not be removed by weak processing and so it's better + to just detect those situations and short-circuit. + +2010-01-26 Kaz Kylheku <kkylheku@gmail.com> + + Optimization in derivative-based regex engine. + Exponential memory consumption behavior was observed when + matching the input aaaaaa.... + against the regex a?a?a?a?....aaaa.... + The fix is to eliminate common subexpressions + from the derivative for the or operator. + + * lib.c (memqual, mapcon): New functions + + * lib.h (memqual, mapcon): Declared. + + * regex.c (flatten_or, unflatten_or, + unique_first, reduce_or): New functions. + (reg_derivative): Apply reduce_or + to the constructed disjunction. + +2010-01-25 Kaz Kylheku <kkylheku@gmail.com> + + Fixing weak hash tables. + + * gc.c, gc.h (REACHABLE, FREE): Moved to header. + + * hash.c (hash_mark): Fix broken list push code. + (has_process_weak): Defend against conservative + garbage collector. We cannot trust that the + conses which make up the chain backbone + and hash entry pairs are unmarked, because the + hash vectors might be reached through + spurious references. + +2010-01-25 Kaz Kylheku <kkylheku@gmail.com> + + Version 032 + + Fix showstopper stupidity. + + * match.c (match_files): Fix incorrect change involving dataline + variable made on 2010-01-21; failure to check for end of data. + + * txr.c (version): Bumped to 032. + + * txr.1: Set version to 032. + + * configure: Bumped txr_ver to 032. + +2010-01-25 Kaz Kylheku <kkylheku@gmail.com> + + Version 031 + + Addresses some spurious object retention problems related to the GC's + conservative scan of the stack. + + * txr.c (version): Bumped to 031. + + * txr.1: Set version to 031. + + * configure: Bumped txr_ver to 031. + +2010-01-25 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): Workaround for GC issue + discovered on Red Hat EL 4 with gcc 3.4.3. + In the collect loop, set car(success) to nil. + Somehow the generated code hangs on to the last + matching position for a regex, preventing GC. + +2010-01-24 Kaz Kylheku <kkylheku@gmail.com> + + * stream.c (vformat_num): Fix bad width calculation. + +2010-01-21 Kaz Kylheku <kkylheku@gmail.com> + + Fix for unbounded memory growth problem reproduced with GCC 4.4.1 + on 32 bit x86 Fedora. This happens because the lazy list variable + ``data'' in the match_files function is optimized to a register, + but a stale value of that variable persists in the backing storage. + + * gc.h (gc_hint): New macro. + (gc_hint_func): Declared. + + * gc.c (gc_hint_func): New function. + + * match.c (match_files): Use gc_hint on the data lazy list. + +2010-01-21 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): Reduce scope, and bogus use of, dataline + variable. + +2010-01-19 Kaz Kylheku <kkylheku@gmail.com> + + Version 030 + + Fixed grammar conflicts. + + R1~R2 syntax supported in regexes. + + * txr.c (version): Bumped to 030. + + * txr.1: Set version to 030. + + * configure: Bumped txr_ver to 030. + +2010-01-19 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (regex): Getting rid of empty '/' '/' production + again. + (regexpr): Re-introducing empty production; this time using + %prec LOW trick to give this interpretation the lowest + possible precedence. Thus expressions like /&/ work again. + (regbranch): New production to allow R1~R2 to be valid. + + * txr.1: Documented. + +2010-01-19 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (grammar): The ^ character is no longer considered + a special regex token, just a regular character. + + * parser.y (LOW): New phony terminal symbol, used as place holder + for lowest precedence. + (grammar): Fixed numerous conflicts in regex section by refactoring. + The regex nonterminal no longer has an empty derivation. + A regex character class no longer has an empty derivation; this is + handled by special rules. Ambiguity around ^ is resolved; this is + parsed as a regular character and specially recognized. + Ambiguity between catenation of terms and postfix operators + resolved in favor of shift by giving catenation low + precedence using %prec LOW. + +2010-01-18 Kaz Kylheku <kkylheku@gmail.com> + + Version 029 + + Performance optimizations of derivative-based regexes. + + New syntax: [] and [^]. + + Saner semantics for % operator. + + * txr.c (version): Bumped to 029. + + * txr.1: Set version to 029. + + * configure: Bumped txr_ver to 029. + +2010-01-18 Kaz Kylheku <kkylheku@gmail.com> + + * regex.c (reg_derivative_list, reg_derivative): Recognition + of cases to reduce consing. In reg_derivative_list, we avoid + consing the full or expression if either branch is t, and + also save a cons when the first element has a null derivative. + In reg_derivative the oneplus and zeroplus cases are split, + since zeroplus can re-use the input expression, when it's + just a one-character match, deriving nil. + +2010-01-18 Kaz Kylheku <kkylheku@gmail.com> + + Adjust semantics of non-greedy operator R%S, to avoid the broken + case whereby R%S matches nothing at all when S is not empty + but equivalent to empty, or more generally when S is nullable. + A much nicer definition is ``the intersection of R* and + the set of all strings that do not contain a non-empty substring + that matches S, followed by S''. + + * regex.c (dv_compile_regex): Adjust syntactic sugar for the % + operator, taking advantage of the reg_nullable function to keep the + simpler syntactic sugar for cases where S is not nullable. + + * txr.1: Document accordingly. + +2010-01-17 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (regterm, regclass): Relocate handling + of empty [] into regterm, via empty derivation. + +2010-01-16 Kaz Kylheku <kkylheku@gmail.com> + + Regex syntactic tweaks: support the [] syntax + to match no character and [^] as its complement, + being synonymous with the wildcard dot. + + * parser.y (regterm): Added new productions. + + * txr.1: Documented. + +2010-01-16 Kaz Kylheku <kkylheku@gmail.com> + + Version 028. + + Code cleanup. + + New additional regex implementation using regex derivatives, + providing new operators: regex complement, intersection, + non-greedy match. + + Regex syntax bugfixes. + + * txr.c (version): Bumped to 028. + + * txr.1: Bumped version to 028. + + * configure: Bumped txr_ver to 028. + + * match.c (dest_bind): Remove spurious syntax. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Get rid of parens from regex operator descriptions. + Correct wrong text: all operators can take an empty regex. + Clarify escaping rules within a character class. + Describe Kleene and non-greedy behavior more accurately. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + * genman.txr, txr.1: Encode version differently; extract + from text during HTML conversion. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + Automate the maintenance of the HTML-ized man page. + + * Makefile (txr-manpage.html): New target, generated from txr.1 + man page. + + * genman.txr: New txr query to transform the output of man2html. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + Implemented non-greedy operator. + + * lib.c (nongreedy_s): New symbol globals. + (obj_init): New symbol interned. + + * lib.h (nongreedy_s): Declared. + + * parser.l (grammar): Support % as a regex operator. + + * parser.y (grammar): Define '%' nonterminal, + on th esame precedence level as '*'. + (regterm): Add the % expression as a term. + (regchar): Recognize % as ordinary character in a character + class. Also, bugfix: recognize & and ~ similarly. + + * regex.c (dv_compile_regex): Implement % as a syntactic sugar + via an algebraic transformation to a more complex expression. + (regex_requires_dv): A regex containing the % operator requires + derivatives. + + * txr.1: Documented %; moved exotic regex notes to end of document. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + * regex.c (reg_derivative_list): Bugfix: wrong algebra, + taking a double derivative of the first item. + +2010-01-15 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fix accidental edit garbage. + +2010-01-14 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fix accidental .b, which should have been .B. + Revised description of regex operators. Added section + on intersection and complement, which may not be familiar + to regex users. + +2010-01-14 Kaz Kylheku <kkylheku@gmail.com> + + * regex.c (reg_derivative): Bugfix: remove invalid + algebraic reductions in the derivative for the or operator. + +2010-01-13 Kaz Kylheku <kkylheku@gmail.com> + + Bugfix: allow unescaped / to be used in regex character classes. + + To do this, we no longer make the lexer look for the terminating + slash which ends the regex syntax. This is driven by the parser, + which calls a special function in the lexer to indicate that + the regex parsing is done. + + * parser.h (end_of_regex): New function declared. + + * parser.l (REGCLASS): Unused start condition removed. + (grammar): A slash character in the REGEX start condition is now simply + returned as an operator token; no popping of the state stack takes + place. The scanner stays in REGEX mode. + (end_of_regex): New function. + + * parser.y (regex): Call end_of_regex when a regex is successfully + scanned through to terminating slash, or if a syntax error occurs. + (regchar): Can derive a / terminal now, thus including it in a + regex character class. + +2010-01-13 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (precedence): bugfix: character classes like this [^*] + being treated as a non-complemented set of two characters. + +2010-01-13 Kaz Kylheku <kkylheku@gmail.com> + + Dynamically determine which regex implementation to use: + NFA or derivatives. The default behavior is NFA, with + derivatives used if the regular expression contains + uses of complement or intersection. The --dv-regex + option forces derivatives always. + + * regex.c (opt_derivative_regex): Default value is 0 now. + (regex_requires_dv): New function. + (regex_compile): If regex_requires_dv function reports + true, or if the opt_derivative_regex flag is true, + treat the regex with the derivative-based implementation. + + * txr.c (txr_main): Implemented --dv-regex option + to set the opt_derivative_regex flag. + +2010-01-13 Kaz Kylheku <kkylheku@gmail.com> + + * lib.h (c_num): Remove redundant declaration. + +2010-01-13 Kaz Kylheku <kkylheku@gmail.com> + + Impelement derivative-based regular expressions. + + * lib.c (chset_s, compiled_regex_s): New symbol globals. + (obj_init): New symbols interned. + + * lib.h (chest_s, compiled_regex_s): Declared. + + * match.c (match_line, match_files): Use regexp predicate + function instead of typeof for detecting regex. + + * parser.y (regexpr, regbranch, regterm): Minor syntactic refactoring. + + * regex.h (union nfa_state, nfa_state_t, struct nfa, enum nfam_result, + nfa_machine_t, nfa_compile_regex, nfa_free, nfa_run, + nfa_machine_reset, nfa_machine_init, nfa_machine_cleanup, + nfa_machine_feed, nfa_machine_match_span, regex_nfa): Declarations + for internal material removed from header, some moved into regex.c. + + * regex.c: Includes txr.h now to get declaration of new option global. + (union nfa_state, nfa_state_t, struct nfa, + nfa_compile_regex, nfa_free, nfa_run, regex_nfa): Declarations + moved from regex.h. + (enum nfam_result, nfa_machine_reset, nfa_machine_init, + nfa_machine_cleanup, nfa_machine_feed, nfa_machine_match_span): + Renamed from nfam_* and nfa_machine_* to regm_* and regex_machine_*. + Functions made static. Regex machine is now polymorphic: the + machine is instantiated based on whether the regex is NFA or + derivative type, and the behavior of the functions is type dependent. + (nfa_machine_t): Renamed to regex_machine_t, now typedef name for union + regex_machine. + (struct dv_machine, union regex_machine): New types. + (struct nfa_machine): New member is_nfa. A few members rearranged, + so that union common members are at the start of the structure. + (opt_derivative_regex): New global added. + (char_set_compile, char_set_cobj_destroy): New function. + (char_set_cobj_ops): New static structure. + (nfa_compile_set): Refactored to use char_set_compile; made static. + (nfa_compile_list): New function. + (nfa_compile_regex): Refactored to follow new syntax from parser.y; + made static. + (nfa_free, nfa_run, regex_nfa): Made static. + (dv_compile_regex, reg_nullable_list, reg_nullable, + reg_derivative_list, reg_derivative, dv_run): New functions. + (regex_compile): Can compile either kind of regex now. + (search_regex, match_regex): Decoupled from dependency on NFA + implementation. + + * txr.h (opt_derivative_regex): Declared. + + * dep.mk: Regenerated. + +2010-01-06 Kaz Kylheku <kkylheku@gmail.com> + + Remove incorrect implementation of extended + regex operations (complement, intersection). + The syntax extensions documentation are retained. + + * regex.c (struct any_char_set, struct small_char_set, struct + displaced_char_set): refs field removed. + (nfa_kind_t): Removed enum members nfa_super_accept, + nfa_reject, nfa_compl_empty, nfa_compl_wild, + nfa_compl_single, nfa_compl_set. + (nfa_super_accept_state, nfa_is_accept_state): Removed. + (char_set_create, char_set_destroy): Reverted. + (char_set_clone): Removed. + (nfa_state_empty_convert, nfa_state_merge): Reverted. + (nfa_compl_state, nfa_compl): Removed. + (nfa_compile_regex, nfa_all_states, nfa_closure, nfa_move): Reverted. + +2010-01-06 Kaz Kylheku <kkylheku@gmail.com> + + Some fine tuning in regex grammar. + + * parser.y (regex): Empty regex handled by + allowing regex to generate empty, rather than + a special case production for '/' '/'. Thus + empty subexpressions are possible. + (regbranch, regterm): Complement is handled + in regbranch, so that it has lower precedence + than aggregation. + +2010-01-05 Kaz Kylheku <kkylheku@gmail.com> + + Implemented the regular expression ~ and & operators. + This turns out to be easy to do in NFA land. + The complement of an NFA has exactly the same number + and configuration of states and transitions, except + that the states have an inverted meaning; and furthermore, + failed character transitions are routed to an extra + state (which in this impelmentation is permanently + allocated and shared by all regexes). The regex & + is implemented trivially using DeMorgan's. + + Also, bugfix: regular expressions like A|B|C are allowed + now by the syntax, rather than constituting syntax error. + Previously, this would have been entered as (A|B)|C. + + * lib.c (comp_s, and_s): New symbol globals. + (obj_init): New symbols interned. + + * lib.h (comp_s, and_s): Declared. + + * parser.l (grammar): Provide new ~ and & tokens in REGEX state. + + * parser.y (regexpr): Constituents of '|' are regexprs, + rather than regbranches (see bugfix note above). + The '&' operator is added. + (regterm): The '~' operator is added. + + * regex.c (struct any_char_set, struct small_char_set, struct + displaced_char_set): refs field added. + (nfa_kind_t): New enum members nfa_super_accept, + nfa_reject, nfa_compl_empty, nfa_compl_wild, + nfa_compl_single, nfa_compl_set. + (nfa_super_accept_state): New static structure. + (nfa_is_accept_state): New inline function. + (char_set_create): Initialize reference count to 1. + (char_set_destroy): Decrement refcount, free if zero. + (char_set_clone): New function. + (nfa_state_empty_convert, nfa_state_merge): Handle nfa_reject state, + the complement of nfa_accept. + (nfa_compl_state, nfa_compl): New functions. + (nfa_compile_regex): Handle new operators. + (nfa_all_states, nfa_closure): Handle new state types. + (nfa_move): Handle new types according to special rules: + the new complemented states that have character transitions have a next + move to the super-accept state if they do not match the input + character. + + * txr.1: Documented new regex operators. + +2009-12-17 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (make_package, find_package): Eliminate declaration + in the middle of statement block. + + * lib.h (TAG_MASK): Becomes type cnum rather than long. + (nao): Based off 1 rather than -1 to avoid left shift of + negative number. + +2009-12-09 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (YYINPUT): Fix signed/unsigned comparison. + +2009-12-09 Kaz Kylheku <kkylheku@gmail.com> + + * hash.c (sethash): New function. + + * hash.h (sethash): Declared. + + * lib.c (cobj_handle): New function. + + * lib.h (cobj_handle): Declared. + +2009-12-08 Kaz Kylheku <kkylheku@gmail.com> + + All COBJ operations have default implementations now; + no null pointer check over struct cobj_ops operations. + + New typechecking function for COBJ objects. + + * gc.c (finalize): Assume function pointer destroy + is not null. + (cobj_destroy_op): New function. + (mark_obj): Assume function pointer mark is not null. + (cobj_mark_op): New function. + + * hash.c (ll_hash): Assume function pointer hash + is not null. + (cobj_hash_op): New function. + (hash_equal): Function removed. + (hash_ops): Replaced hash_equal with cobj_equal_op. + + * lib.c (class_check, cobj_equal_op): New functions. + + * lib.h (cobj_equal_op, cobj_destroy_op, cobj_mark_op, + cobj_hash_op): Declarations added. + (system_package, user_package, class_check): Declaration added. + + * regex.c (regex_equal): Function removed. + (regex_obj_ops): regex_equal replaced with cobj_equal_op. + + * stream.c (common_equal): Function removed. + (stdio_ops, pipe_ops, string_in_ops, byte_in_ops, + string_out_ops, dir_ops): common_equal replaced + with cobj_equal_op, and all previously null + function pointers populated with default functions. + +2009-12-05 Kaz Kylheku <kkylheku@gmail.com> + + More void * to mem_t * conversion. + + * stream.c (stdio_put_char_callback, stdio_get_char_callback, + stdio_put_string, stdio_put_char, stdio_snarf_line, stdio_get_char): + Convert void * to mem_t *. + + * utf8.c (utf8_encode, utf8_decode): Convert void * to mem_t *. + + * utf8.h (utf8_encode, utf8_decode): Update declarations. + +2009-12-04 Kaz Kylheku <kkylheku@gmail.com> + + Eliminate the void * disease. Generic pointers are of mem_t * + from now on, which is compatible with unsigned char *. + No implicit conversion to or from this type, in C or C++. + + * hash.c (make_hash): Convert void * to mem_t *. + + * lib.c (oom_realloc, chk_malloc, chk_realloc, vec_set_fill, + cobj, init): Convert to using mem_t *. + + * lib.h (mem_t): New typedef. + (struct cobj): Convert void * to mem_t *. + (oom_realloc, chk_malloc, chk_realloc, init): Declarations updated. + + * regex.c (regex_compile): Convert void * to mem_t *. + + * stream.c (snarf_line, string_out_put_string, make_stdio_stream, + make_pipe_stream, make_string_input_stream, + make_string_byte_input_stream, make_string_output_stream, + get_string_from_stream): Convert void * to mem_t *. + + * txr.c (oom_realloc_handler): Convert void * to mem_t *. + +2009-12-03 Kaz Kylheku <kkylheku@gmail.com> + + * gc.c (heap_min_bound, heap_max_bound): New static globals. + (more): Update heap_min_bound and heap_max_bound. + (in_heap): Do early rejection tests on the pointer. If it's + not aligned, or it's completely outside of the bounding + box of the heap area, short circuit to false. + +2009-12-03 Kaz Kylheku <kkylheku@gmail.com> + + Version 027. + + Code cleanup. + + gc-related bugfix. + + Improved file copying semantics of make install, + and adherence for DESTDIR convention. + + * txr.c (version): Bumped to 027. + + * txr.1: Bumped version to 027. + + * configure: Bumped txr_ver to 027. + +2009-12-03 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (CFLAGS): Better test for g++, when removing + warning options not appropriate for g++. Sometimes + g++ may be called something that dosn't end in g++, + like g++4. + +2009-12-03 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (YY_NO_UNPUT): Removed superfluous #define. This is not + needed because suppressing generation of unput is requested via + the %option. In scanners generated by the legacy version of + flex, 2.5.4, still widely in use. this redundancy leads to + a multiple #define YY_NO_UNPUT and a compiler warning. + +2009-12-03 Kaz Kylheku <kkylheku@gmail.com> + + Fix for failing test suite on MIPS machine, due to + gc failing to mark a local variable in txr_main. + + * txr.c (txr_main): Changed from internal linkage to external. + This prevents gcc -O2 from inlining txr_main into main. + We need separate stack frames for main and txr_main, + in order to be sure that when walking to the bottom of + stack pointer, we visit all locals in main. This is the + whole reason why there is a separate txr_main. + +2009-12-02 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (tests): Don't depend on the executable. Otherwise, + during make install-tests, if it doesn't exist in the + install directory, a gcc compile command gets deposited + into the run.sh generated script. + (install-tests): Fixes to make this work when using + a separate build directory. Split the cpio -p job + into a cpio -i piping into cpio -o. + +2009-12-02 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (install-tests): New target. Provides a way to make the + test cases part of the installation, and a generated script to run the + commands on the installation host. + +2009-12-02 Kaz Kylheku <kkylheku@gmail.com> + + Fix annoyances with dependency generation, such as picking up local + files that are not in the project. + + * Makefile (depend): Rule passes object file names as arguments + to depend.txr script. + + * depend.txr: Changed to take names of object files from command line, + rather than scanning the directory for all .c files. Switched + to new style next directives, using quasiliterals. + + * dep.mk: Regenerated. + +2009-11-28 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (CFLAGS): If the compiler matches the pattern %g++, + then remove some C-front-end-specific warnings from CFLAGS, + which the g++ front end will complain about. + +2009-11-28 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (CFLAGS): add -Dlint to CFLAGS when compiling y.tab.o. + This suppresses some warnings from a byacc-generated parser, + and gets rid of a useless static sccsid array. May help with + Bison-generated parser also. + +2009-11-28 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l: Use flex options to suppress generation of the + unused functons yyunput and yyinput, thus getting rid of + some compiler diagnostics. + +2009-11-28 Kaz Kylheku <kkylheku@gmail.com> + + Code cleanup. All private functions static. Private stuff + in regex module not exposed in header. Etc. + + * configure (diag_flags): Add -Wmissing-prototypes and + -Wstrict-prototypes. + + * gc.c (more): Turn into prototyped definition with (void). + + * gc.h (unmark): Declared. + + * hash.c (hash_equal, hash_destroy, hash_mark, hash_grow): Private + functions defined static. + + * lib.c (flatten_helper, do_bind2, do_bind2other): Likewise. + + * lib.h (make_package, merge, d): Declared. + + * match.c (dump_shell_string, dump_byte_string, dump_var, + dump_bindings, depth, weird_merge, bindable, dest_bind, match_line, + format_field, subst_vars, eval_form, complex_open, complex_snarf, + complex_stream, robust_length, bind_car, bind_cdr, extract_vars, + extract_bindings, do_output_line, do_output, match_files): Private + functions defined static. + (map_leaf_lists, complex_close): Unused functions removed. + + * parser.h (yyerror): Declared. + + * regex.c (bitcell_t, BITCELL_ALL1, CHAR_SET_SIZE, + chset_type_t, cset_L0_t, cset_L1_t, cset_L2_t, cset_L3_t, + struct any_char_set, struct small_char_set, struct displaced_char_set, + struct large_char_set, struct xlarge_char_set, union char_set, + nfa_kind_t, struct nfa_state_accept, struct nfa_state_empty, + struct nfa_state_single, struct nfa_state_set, struct nfa_state, + struct nfa_machine): Definitions moved here from regex.h file. + (L0_fill_range, L0_contains, L1_full, L1_fill_range, L1_contains, + L1_free, L2_full, L2_fill_range, L2_contains, L2_free, L3_fill_range, + L3_contains, L3_free, char_set_create, char_set_destroy, + char_set_compl, char_set_add, char_set_add_range, char_set_contains, + nfa_state_accept, nfa_state_empty, nfa_state_single, nfa_state_wild, + nfa_state_free, nfa_state_shallow_free, nfa_state_set, + nfa_state_empty_convert, nfa_state_merge, nfa_make, nfa_combine, + nfa_compile_set, nfa_all-states, nfa_closure, nfa_move): Private + functions defined static. + + * regex.h (bitcell_t, BITCELL_ALL1, CHAR_SET_SIZE, + chset_type_t, cset_L0_t, cset_L1_t, cset_L2_t, cset_L3_t, + struct any_char_set, struct small_char_set, struct displaced_char_set, + struct large_char_set, struct xlarge_char_set, union char_set, + nfa_kind_t, struct nfa_state_accept, struct nfa_state_empty, + struct nfa_state_single, struct nfa_state_set, struct nfa_state, + struct nfa_machine): Definitions removed. + (char_set_created, char_set_destroy, char_set_compl, char_set_add, + char_set_add_range, char_set_contains nfa_state_accept, + nfa_state_empty, nfa_state_single, nfa_state_wild, nfa_state_set, + nfa_state_free, nfa_state_shallow_free, nfa_state_merge): Extern + declarations removed. + + * stream.c (stdio_stream_print, stdio_stream_destroy, + stdio_stream_mark, stdio_get_char, stdio_get_byte, + string_in_stream_mark, vformat_str): Private functions defined static. + + * txr.c (oom_realloc_handler, help, hint, + remove_hash_bang_line): Likewise. + + * unwind.c (uw_unwind_to_exit_point): Likewise. + +2009-11-28 Kaz Kylheku <kkylheku@gmail.com> + + * configure: Workaround in banner code for coreutils printf %.*s bug. + +2009-11-27 Kaz Kylheku <kkylheku@gmail.com> + + Switching to DESTDIR convention for install. + Make install step does some things more correctly now, + without relying on the install program. + + * configure: Help text doesn't refer to ``Makefile variables'' + but ``make variables'', or ``variables in config.make''. + The install_prefix variable becomes DESTDIR now in + config.make. + + * Make (INSTALL): New rule body macro. + (install): Uses of mkdir -p and cp switched to a call + to the INSTALL macro. + +2009-11-26 Kaz Kylheku <kkylheku@gmail.com> + + Version 026. + + Fixed wchar_t build problem in parser.y. + + Improved configure script to auto-detect yacc program. + Txr works with either Berkeley yacc (byacc) or Bison. + + Fixed some two uninitialized memory bugs. + + Valgrind API is now used to integrate GC memory manager + with valgrind. + + The symbols nothrow and args in the next directive + are now keyword symbols, written :nothrow and + :args. (Breaks backward compatibility; sorry!) + + * txr.c (version): Bumped to 026. + + * txr.1: Bumped version to 026. + + * configure: Bumped txr_ver to 026. + +2009-11-26 Kaz Kylheku <kkylheku@gmail.com> + + Not all systems have a yacc alias for the yacc program. + txr is known to work with two yacc implementations: GNU Bison + and Berkeley yacc. Let's add some auto-detection for yacc. + + * Makefile: use "include" rather than "-include" for + including config.make, so that make fails if the file does + not exist. + (conftest.yacc): New target. Just outputs the value of the + variable expansion of $(YACC). + + * configure (yaccname): New variable. + (gen_config_make): New function. + Steps added to test for existence of various yaccs. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + * gc.c (mark_mem_region): Bugfix: do not mess with the valgrind + accessibility of the heap object if valgrind debugging is not enabled. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (grammar): Fixes for bison 2.4.1. Remove superfluous action + in chrlit. Include <stdlib.h> for abort. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + Refinements to Valgrind support. + + * gc.c (mark_mem_region): If a pointer from the stack is valid + for the heap, it may point to a free object, which is marked + in accessible. We must grant the garbage collector access + to the object. If the object is free, close off access. + This is not 100% correct, because if the object is accessible + but undefined, then we end up flipping it to defined. + (sweep): Before sweeping each heap, mark the entire block as defined. + This is necessary because sweep accesses blocks, which may be free, + and thus inaccessible. Then, during the sweep, any block + which is already free must be marked inaccessible again. + This means that the remaining blocks that are reachable become defined. + Here that is okay, because gc has marked all those blocks. If any + of them had uninitialized members, that would have been caught + by valgrind during the marking phase, if not sooner. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + More Valgrind support. New option --vg-debug which turns on + Valgrind protection of free blocks. This works independently + of --gc-debug. + + * gc.c (opt_vg_debug): New conditionally defined global variable. + (more): Mark entire heap of free blocks inaccessible, if + vg debugging is enabled. + (make_obj): If vg debugging enabled, mark returned block as accessible, + but undefined, and take care to grant self temporary access while + manipulating the free list. + (finalize): Removed old debugging logic of not freeing strings + and vectors during gc debug. If the null pointers are ever a problem + during debugging, they can be checked inside obj_print, and + turned into #<garbage ...> notation. + (sweep): Switch to FIFO free block recycling if vg debugging is + enabled, just like when gc debugging is enabled. + Mark freed blocks as inaccessible, careful to grant self + temporary access while manipulating the free list. + + * txr.c (txr_main): Parse the --vg-debug option. + + * txr.h (opt_vg_debug): Conditionally declared. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + Fix a build breakage that may happen on some platforms. + The parser.y file includes "utf8.h", which uses the the type wint_t. + It also includes "lib.h" which uses "wchar_t". But it fails + to include any headers which define these types. + The generated y.tab.c picks up wchar_t by the Bison-inserted + inclusion of <stdlib.h>, so that's how we got that. But wint_t does not + come from any of the headers---if they are standard-conforming. + + * parser.y: Add inclusion of <stddef.h> and <wchar.h>. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + More valgrind integration. Vector objects keep displaced pointers + to vector data; they point to element 0 which is actually the third + element of the vector. If an object is only referenced by interior + pointers, Valgrind reports it as possibly leaked. This change + conditionally adds a pointer to the true start of the vector, + if Valgrind support is enabled. + + * lib.h (struct vec): vec_true_start, new member. + + * lib.c (vector, vec_set_fill): Maintain vec_true_start. + +2009-11-25 Kaz Kylheku <kkylheku@gmail.com> + + First stab at Valgrind integration. First goal: eliminate false + positives when gc is accessing uninitialized parts of the stack. + + * configure (valgrind): New variable. Defaults to false (do not + build valgrind support). New check for whether the valgrind API + is actually avilable if --valgrind is selected. + (HAVE_VALGRIND): Conditionally added to config.h. + + * gc.c: Conditionally include valgrind memcheck.h header. + (mark_mem_region): After pulling out a value from the stack, + mark that copy as defined memory using VALGRIND_MAKE_MEM_DEFINED. + (mark): Removed check for a registered root variable pointer + being null; this cannot happen, unless someone registers a + null pointer, or the stack is trashed. The comment about a + possible null was misleading. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Fix uninitialized memory locations. + + * hash.c (make_hash): Uninitialized h->count member. + + * lib.c (mkustring): Preallocated string buffer to have its + null terminator byte initialized, because the caller + does not do so (e.g. see lit_har_helper in parser.y). + The calling module is responsible for initializing all API-accessible + parts of the string, but the null belongs to the string implementation. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Switching to keyword symbols for :args and :nothrow. + + * lib.c (args_s, nothrow_s): Renamed to args_k and nothrow_k. + (flattn_s): Renamed to flatten_s. + (obj_init): args_k and nothrow_k interned in keyword package. + + * lib.h (args_s, nothrow_s, flattn_s): Declarations updated. + + * match.c (match_files): Follow name changes. + + * tests/004/query1.txr: Changed nothrow to :nothrow. + + * txr.1: Documentation updated. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + /Now/ this can be released as 025. + + utf8.c (utf8_from_uc): Fix bug introduced several commits ago (porting + to C++). Caught by regression test suite. Found using git bisect. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Version 025 + + External changes: + + Flattening an empty list produces an empty list, not (()), + which is a list containing an empty list. + + Tightened up semantics of bind, merge and other forms. + Fixed false positives in binding. + + More bugfixes in the parser leading to garbage error messages. + (Still no regression test cases for error cases, oops). + + Fixed crash in regexp function. + + Symbol packages added. Keyword symbols (symbols in keyword package) + introduced. + + Clarified semantics that t, nil and keywords evaluate to themselves. + + Fixed bugs in the system for building in a separate directory. + + Configuration script now tests the compiler for sanity, and runs + compiler-based tests to detects which integer type to use for casting + an obj_t * value to a number, and what specifiers to use for + inline functions. + + Internal changes: + + Macros replaced with inline functions. + + Uses of obj_t * replaced with val typedef everywhere. + + Exceptions occuring during early initialization no longer + lead to an infinite recursion due to streams not working. + + The long type is no longer used, but a configured typedef. + + Configure script now spits out a "config.h" header that is + widely included. + + Symbol globals renamed to _s naming scheme. + + Code made portable to C++. A new configure flag --ccname make it easier + to switch compilers. + + * txr.c (version): Bumped to 025. + + * txr.1: Bumped version to 025. + + * configure: Bumped txr_ver to 025. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Auto-detect what specifiers to use for inline functions. + Allow compiler command to be set independently of full path + for easier compiler switching. + + * Makefile (conftest.o): Target removed. What this rule does + is already an implicit rule; and nowhere else in the Makefile + are there rules for .c -> .o. + (conftest2): New target, for two-translation-unit config test program. + (INLINE_FLAGS): Removed. + + * configure (ccname, inline): New variables. + (inline_flags): Variable removed. INLINE_FLAGS not generated + any more in config.make. Added test for what inline specifiers to use, + which is turned into #define INLINE ... in the config.h header. + + * lib.h: (tag, is_ptr, is_num, is_chr, is_lit, type, auto_str, + static_str, litptr): Changed from inline to INLINE. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Changes to make the code portable to C++ compilers, which + can be taken advantage of for better diagnostics. + + * gc.c (more, mark_obj, sweep, unmark): Obey stricter C++ rules + with regard to enumerations. + (make_obj): Avoid using C++ keyword "try". + + * lib.c: Removed duplicate definitions of objects, found by C++. + (chk_malloc, chk_realloc): Casts needed when converting from void *. + (list): Discovered and fixed lack of va_end. + (trim_str, acons_new_l): Avoid use of C++ keyword "new". + (make_sym): Follow rename of struct member. + + * lib.h (struct sym): Renamed val member to value. + (null): Added missing declaration. + + * match.c (enum fpip_close, struct fpip): Moved and named enum out of + struct. + + * regex.c (L0_full): Cast added in signed/unsigned comparison. + (L1_fill_range, L2_fill_range, L3_fill_range, char_set_create): + Don't mark static blank structures const; then they need initializers + in C++. + (char_set_compl, char-set_destroy, char_set_contains, nfa_compile_set): + Avoid using the C++ keyword "compl". + + * regex.h (struct any_char_set, struct small_char_set, struct + displaced_char_set, struct large_char_set, struct xlarge_char_set): + Renamed compl member to comp. + + * utf8.c (utf8_from_uc, ut8_decode): Obey stricter C++ rules + with regard to enumerations. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Fixed broken yyerrorf. It was still taking char *, and passing + that as an object to vformat, resulting in #<garbage: ...> + output. + + * parser.h (yybadtoken): Declaration updated. + + * parser.l (yybadtoken): Redefined to take val argument. + The tok stays as int; this is closely coupled to yacc, + so why bother with num(). + + * parser.y (grammar): Fix occurences of yybadtoken to pass + proper literal objects using the lit macro, or nil in + the one case when there is no context. + +2009-11-24 Kaz Kylheku <kkylheku@gmail.com> + + Renaming global variables that denote symbols, such that they + have a _s suffix. + + * lib.c (cons_t, str_t, chr_t, num_t, sym_t, pkg_t, fun_t, vec_t, + stream_t, hash_t, lcons_t, lstr_t, cobj_t var, regex, set, cset, wild, + oneplus zeroplus, optional, compound, or, quasi skip, trailer, block, + next, freeform, fail, accept all, some, none, maybe, cases, collect, + until, coll define, output, single, frst, lst, empty, repeat, rep + flattn, forget, local, mrge, bind, cat, args try, catch, finally, + nothrow, throw, defex error, type_error, internal_err, numeric_err, + range_err query_error, file_error, process_error): Symbol globals + renamed to cons_s, str_s, chr_s, num_s, sym_s, pkg_s, fun_s, vec_s, t, + cons_s, str_s, chr_s, num_s, sym_s, pkg_s, fun_s, vec_s, stream_s, + hash_s, lcons_s, lstr_s, cobj_s, var_s, regex_s, set_s, cset_s, wild_s, + oneplus_s, zeroplus_s, optional_s, compound_s, or_s, quasi_s, skip_s, + trailer_s, block_s, next_s, freeform_s, fail_s, accept_s, all_s, + some_s, none_s, maybe_s, cases_s, collect_s, until_s, coll_s, define_s, + output_s, single_s, first_s, last_s, empty_s, repeat_s, rep_s, + flattn_s, forget_s, local_s, merge_s, bind_s, cat_s, args_s, try_s, + catch_s, finally_s, nothrow_s, throw_s, defex_s, error_s, type_error_s, + internal_error_s, numeric_error_s, range_error_s, query_error_s, + file_error_s, process_error_s, + (code2type, typeof, make_package, intern, obj_init): Symbols + references follow rename. + + * lib.h (cons_t, str_t, chr_t, num_t, sym_t, pkg_t, fun_t, vec_t, + stream_t, hash_t, lcons_t, lstr_t, cobj_t var, regex, set, cset, wild, + oneplus zeroplus, optional, compound, or, quasi skip, trailer, block, + next, freeform, fail, accept all, some, none, maybe, cases, collect, + until, coll define, output, single, frst, lst, empty, repeat, rep + flattn, forget, local, mrge, bind, cat, args try, catch, finally, + nothrow, throw, defex error, type_error, internal_err, numeric_err, + range_err query_error, file_error, process_error): Declarations + updated. + + * hash.c (make_hash): Symbol references follow rename. + + * match.c (sem_error, file_err, dump_var, match_line, subst_vars, + eval_form, complex_stream, extract_vars, do_output_line, do_output, + match_files): Likewise. + + * parser.y (grammar, repeat_rep_helper, define_transform): Likewise. + + * regex.c (nfa_compile_set, nf_compile_regex, regex_compile, + regexp, regex_nfa): Likewise. + + * stream.c (stdio_maybe_read_error, stdio_maybe_write_error, + stdio_close, pipe_close, make_stdio_stream, make_pipe_stream, + make_string_input_stream, make_string_byte_input_stream, + make_string_output_stream, get_string_from_stream, make_dir_stream, + close_stream, get_line, get_char, get_byte, vformat, format, + put_string, put_char): Likewise. + + * txr.c (txr_main): Likewise. + + * unwind.c (uw_throw, uw_errorf, type_mismatch, uw_register_subtype, + uw_init): Likewise. + + * unwind.h (internal_error, numeric_assert, range_bug_unless); + Likewise. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * configure (platform_flags, remove_flags): New config variables. + + * Makefile (CFLAGS): Take into account new flags. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + Follow up on 64 bit compilation warnings. + + * lib.c (chr, chrp): Do not convert directly between wchar_t and + the pointer type; go through cnum intermediate value. + + * stream.c (vformat): Fix bad cast from pointer to int; this was + missed in the conversion to cnum because it should have been a + cast to long originally. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (conftest.o): revert change that took CFLAGS from + this target. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * configure: Don't rely on higher precision arithmetic from the build + machine's shell. POSIX requires shell arithmetic to be only signed + long. We can't compute the INT_PTR_MAX constant in the shell, but + rather generate a constant C expression to compute it. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + Reporting of compile errors during configuration for easier + configure debugging. + + * Makefile (conftest): Pass all of the CFLAGS when building + conftest. This way bad compiler options are caught + right in the basic compiler sanity test. + + * configure: Compiler jobs are redirected to temporary error + file conftest.err which is dumped if there is a failure. + Parting text is improved: the user should not blindly + trust the success of the configuration but check its sanity. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * configure: Bugfix in parsing configuration variables + which contain the = character. + + * Makefile (conftest.o): Pass full CFLAGS to configuration test + builds. If some flags don't work with the compiler, this should + be caught. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (CFLAGS): Added -I. so current directory is first + in the include search path. This is needed for finding + generated header files, when building in a separate directory. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (chk_malloc, chk_realloc): Fix diagnosable conversion, + caught by gcc 4.1.1. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + * configure (cross): Print out value of $cross in --help. + + * depend.txr: Add "config.h" to list of headers that are not + prefixed with $(top_srcdir). + + * dep.mk: Regenerated. + +2009-11-23 Kaz Kylheku <kkylheku@gmail.com> + + Improving portability. It is no longer assumed that pointers + can be converted to a type long and vice versa. The configure + script tries to detect the appropriate type to use. Also, + some run-time checking is performed in the streams module + to detect which conversions specifier strings to use for + printing numbers. + + * Makefile (conftest, conftest.o, conftest.syms): New targets. + Used by the configure script. + + * configure (intptr, nm): New configuration variables. + Generating config.make is no longer the last step; compiler tests are + performed after config.make is set up, so that rules in the Makefile + can be used for doing the compiling. (This is the cleanest way to do + it, since the paths to the tools may contain Make variable expansion + syntax). New steps are added to try to detect whether the compiler has + a wider integer type than the c89 long, and which of the available + types (including, potentially, the extra wide type) is suitable for + holding a pointer. Results are generated into a header config.h. + + * dep.mk: Regenerated. + + * lib.h (NUM_MAX, NUM_MIN): Now derived from INT_PTR_MAX and + INT_PTR_MIN macros, which come from config.h. + (cnum): New typedef name. + (cobj ops, tag, auto_str, static_str, litptr, lit_noex): + Changed long to cnum. + (num, c_num): Declaration updated. + + * lib.c (equal, length, num, c_num, plus, minus, neg, search_str, + cat_str, vector, vec_set_fill, obj_print, obj_pprint): Changed + long to cnum. + + * gc.c (mark_obj): Changed long to cnum. + + * hash.c (stuct hash, ll_hash, hash_mark, hash_grow, + hash_process_weak): Changed long to cnum. + + * match.c (complex_open, do_output_line, do_output, match_files): + Changed long to cnum. + + * parser.h (lineno): Declaration updated. + + * parser.l (lineno): Redefined as cnum. + (grammar): Changed long to cnum. + + * parser.y (%union/yystype): num member changed to cnum. + of config.h added. + + * regex.c (nfa_run, nfa_machine_match_span, search_regex): + Changed long to cnum. + + * regex.h (struct nfa_machine): Members last_accept_pos and count + changed to cnum. + (nfa_run, nfa_machine_match_span): Declarations updated. + + * stream.c (struct fmt): New type. + (fmt_tab): New static array. + (num_fmt): New static pointer. + (detect_format_string): New function. + (vformat): Changed long to cnum. Formatting of numbers uses + num_fmt. + (stream_init): Call detect_format_string. + + * txr.c, unwind.c, utf8.c: include config.h. + + * unwind.h (internal_error): Local declaration of num updated. + +2009-11-21 Kaz Kylheku <kkylheku@gmail.com> + + Introducing symbol packages. Internal symbols are now in + a system package instead of being hacked with the $ prefix. + Keyword symbols are provided. In the matcher, evaluation + is tightened up. Keywords, nil and t are not bindeable, and + errors are thrown if attempts are made to bind them. + Destructuring in dest_bind is strict in the number of items. + String streams are exploited to print bindings to objects + that are not strings or characters. Numerous bugfixes. + + * lib.h (enum type, type_t): new member: PKG. + (struct stym): New member: package. + (struct package): New type. + (union obj, obj_t): New member pk. + (interned_syms): Declaration removed. + (keyword_package, pkg_t): Declared. + (intern, acons_new_l): Declarations updated. + (find_package, symbol_package, keywordp): Declared. + + * lib.c (interned_syms): Definition removed. + (packages, pkg_t, system_package, keyword_package, user_package): New + global variables. + (code2type, equal, obj_pprint): Handle PKG case. + (symbol_package, make_package, find_package, keywordp): New functions. + (make_sym): Initialize package field of symbol. + (intern): Takes package argument. Rewritten using packages, + which use hash tables to store symbols. + (acons_new_l): Takes extra pointer argument to return an extra + value. + (obj_init): Updated to handle packages. The orders of some + initializations have to change. The way nil is added as a symbol is + quite different, and a special hack for the symbol t is used. + Most symbols go into the user_package, but symbols that were + previously namespaced with $ go to the system package. + (obj_print): SYM cases now considers the packge of a symbol. + Symbols in the user package are printed as before. + Symbols with no package are printed using #: notation; + keywords with : notation; and all others with their package prefix. + PKG case is handled. + + * gc.c (finalize): Handle PKG case. + (mark_obj): For SYM, mark the new package member. Handle PKG case. + + * hash.h (gethash_l): Declaration updated. + + * hash.c (ll_hash): Handle PKG case. + (gethash_l): Extra argument added to distinguish new addition + from existing find. + + * match.c (dump_var): Dumps any object now by printing to + a string with a string stream. + (bindable): New function. + (dest_bind): Tightened up to distinguish bindable symbols + from non-bindable. Symbols that stand for themselves, including nil, + can only match themselves. Destructuring matches have to + match in the number of elements: dot notation can be used + to match superfluous elements. + (eval_form): Tightened up to recognize bindable symbols. + (match_files): Various directives honor non-bindable symbols (cat, + merge, flatten). + + * parser.l (yybadtoken): Handle KEYWORD case. + (grammar): TOK can start with : . Returned as KEYWORD terminal, + with a lexeme that no longer has the : character. + + * parser.y (KEYWORD): New nonterminal. + (grammar): Calls to intern given extra parameter. + In the expr rule, KEYWORD turned into symbol in keyword package. + + * regex.c (regexp): Bugfix: dereferencing non pointer. + + * stream.c (vformat): Bugfixes in state machine: handling + of prefix digits; printing of numbers in ~s. + + * txr.c (txr_main): Intern calls updated. + + * txr.1: Updated with information about nil, t and keywords. + More details about destructuring matching in bind. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * unwind.c (uw_throw): If streams are not initialized, + we have an unhandled exception too early in initialization. + Use C stream to print an error message and abort. + Using the nil stream variable will just cause a recursion bomb. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (intern): Symbol interning to hash tables. + (obj_init): interned_syms must be created as a hash + table. Rearranged the order of some initializations so + the vector code called by hash works. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (dest_bind): Fix breakage from two commits ago; + was falling through to unsuccessful return in the + consp case. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (grammar): Fix error actions that do not assign + a value to $$. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (dest_bind): Extended to handle more general forms + by using eval_form rather than direct symbol binding lookups. + False positive return fixed. + (match_line): Fixed merge to use eval_from + rather than direct symbol binding. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (flatten): Semantics change. The flatten function + should not map nil -> (nil), but nil -> nil. + +2009-11-20 Kaz Kylheku <kkylheku@gmail.com> + + Changing ``obj_t *'' occurences to a ``val'' typedef. (Ideally, + we wouldn't have to declare object variables at all, so why + use an obtuse syntax to do so?) + + * lib.h (val): New typedef name. Used throughout. + + * gc.c, gc.h, hash.c, hash.h, lib.c, match.c, match.h, parser.h, + parser.l, parser.y, regex.c, regex.h stream.c,, stream.h, txr.c, + unwind.c, unwind.h: Replace obj_t * with val almost everywhere. + Low-level gc functinos that work with arrays of obj_t use + obj_t *. Seeing pointer arithmetic on a val doesn't make sense. + In macros we use obj_t *, to reduce the chances of clashing + with some local variable called val. + +2009-11-19 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fixed mangled formatting of exception handling example. + +2009-11-19 Kaz Kylheku <kkylheku@gmail.com> + + Get rid of macros in favor of safer inline functions. + + The recent auto_str("byte str") error could have been caught + at compile time. + + * Makefile (CFLAGS): Include expansion of INLINE_FLAGS. + + * configure (inline_flags): New variable. + (INLINE_FLAGS): New variable generated in config.make. + + * lib.h (tag, is_ptr, is_num, is_chr, is_lit, type, + auto_str, static_str, litptr): Function-like macros converted to + functions. + +2009-11-19 Kaz Kylheku <kkylheku@gmail.com> + + Version 024 + + Fixed show-stopper breakage in parse error diagnostic function. + + Fixed bug introduced back in 015: collects that don't yield + any variable bindings being wrongly treated as failed. + + * txr.c (version): Bumped to 024. + + * txr.1: Bumped version to 024. + +2009-11-19 Kaz Kylheku <kkylheku@gmail.com> + + Use unsigned char * as allocator return value. + + * lib.c (chk_malloc, chk_realloc): Return unsigned char *. + + * lib.c (chk_malloc, chk_realloc): Declarations updated. + + * utf8 (utf8_dup_to_uc): Remove cast to unsigned char *. + +2009-11-18 Kaz Kylheku <kkylheku@gmail.com> + + Following-up on diagnostics obtained by running code through C++ + compiler. Idea: allocator functions return char * instead of void *, + like malloc did in classic pre-ANSI C. That way we are forced to + use a cast except when the target pointer is char * already. + + * lib.c (progname): Duplicate definition of global removed. + (equal): Some default: cases to switch statements added. + (chk_malloc): Returns char *. + (chk_realloc): Returns char *, but takes void * on the way in. + That way we get C++-like behavior. + (chk_strdup): Oops, this returned void * instead of wchar_t *. + c++ catches boo boo. + (stringp): Added default: case to switch. + (vec_set_fill): Cast return value of chk_realloc. + + * lib.h (chk_malloc, chk_realloc, chk_strdup): Declarations updated. + + * parser.h (lineno): extern qualifier added to prevent duplicate + definitions of. + + * regex.c (nfa_free, nfa_run, nfa_machine_init, regex_compile): Cast + return value of chk_malloc. + + * stream.c (snarf_line, get_string_from_stream): Cast return value of + chk_realloc. + +2009-11-18 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_line, match_files): Fix broken behavior of collect + that doesn't match anything. It is not a failed match, as the + documentation makes perfectly clear. Collect/coll were introduced + in txr-006 and had the proper non-failing semantics. + However, in txr-015, during code restructuring, a bug crept in. + When changing to a different debugiging function, for some reason + I added the nil returns. + +2009-11-18 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (yyerror): Total breakage: can't take auto_str of char * + string. + (yyerrorf): Total breakage: arguments of wrong types. Detected + by vformat as garbage. + +2009-11-18 Kaz Kylheku <kkylheku@gmail.com> + + txr.1: Clarified handling of UTF-8, now that it's precise and portable. + +2009-11-18 Kaz Kylheku <kkylheku@gmail.com> + + Version 023 + + Minor bugfix. + + Code cleanup. Portability. + + Completely removed dependency on C99 wide character stream functions, + and character encoding support from glibc. All UTF-8 encoding + and decoding is done by the program itself. + + Removed the use of all GNU extensions and C99 syntax. + + txr now requires a C90 compiler, and POSIX 1003.1 and 1003.2. + + * txr.c (version): Bumped to 023. + + * txr.1: Bumped version to 023. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + More removal of C99 wide character I/O, and tightening up + of standard conformance. + + * configure (lang_flags): Specify -D_POSIX_C_SOURCE=2 to obtain + POSIX 1003.1 and POSIX 1003.2 functions from the headers, + without GNU extensions. Specify -std=c89 to get C89 conformance + from gcc. + + * match.c (dump_byte_string): New function. + (dump_shell_string): Retargetted to object streams. + (dump_var, dump_bindings): Retargetted to object streams. + Changed back to using a byte string for the array index prefixes, + to avoid using the wide-character swprintf. + + * parser.l (grammar): Eliminate wcsdup uses in favor of chk_strdup. + Not only is wcsdup a GNU extension, it doesn't have the OOM check. + + * stream.c: Added <sys/wait.h> header to define WIFEXITED and others. + + * txr.c: Added include of <stdarg.h>. Removed <locale.h>, + (main): Removed setlocale call. Not needed any more, since wide + stream and string I/O is no longer used from the C library. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + Removing use of C99 wide character I/O. + + * stream.c (BROKEN_POPEN_GETWC): Macro removed. Work around no + longer needed since the program does not call getwc. + (struct stdio_handle): #ifdef text removed. + New member added: utf8 decoder. + (stdio_maybe_read_error, stdio_maybe_write_error): Treat + null handle as an exception rather than nil return. No need + to check ferror in stdio_maybe_write_error, since there is no + need to distinguish an end-of-file situation from error. + (stdio_put_char_callback, stdio_get_char_callback): New functions. + (stdio_put_string, stdio_put_char): Retargetted to utf8 encoder. + Null handle treated as separate kind of error. + (snarf_line, stdio_get_line, stdio_get_char): Retargetted to utf8 + decoder. + (pipe_close): #ifdef text removed. + (make_stdio_stream): utf8 decode initialized. + (make_pipe_stream): utf8 decoder initialized. #ifdef text removed. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + Warning fixes. + + * hash.c (hash_ops): Add missing initializer. + + * match.c (complex_open): Add missing intializer to ret variable. + + * regex.c (regex_obj_ops): Add missing initializer. + + * stream.c (stdio_ops, pipe_ops, string_in_ops, byte_in_ops, + string_out_ops, dir_ops): Likewise. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (chrp): Fix broken is_chr(num) call. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + * regex.c (nfa_all_states, nfa_closure): visited parameter + should be unsigned. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + Fixes for compliance to C89. + + * lib.c (init): Do not define variable after statements. + + * match.c (match_files): Likewise. + + * regex.h (struct any_char_set, struct small_char_set, struct + displaced_char_set, struct large_char_set, + struct xlarge_char_set): do not use enum bitfields, which is a GCC + extension. + + * unwind.h (enum uw_frtype, uw_frtype_t): Combine into one + declartion, eliminating forward enum reference which is a GCC + extension. + (uw_block_begin): Add dummy typedef to macro so that it requires + a following semicolon. Without this, if the macro use is followed + by a semicolon, that semicolon looks like a null statement. A + subsequent declaration thus follows a statement and is not conforming + to C89. Also added an opening do. + (uw_block_end): Add while(0) to match do in uw_block_begin. + (uw_env_begin, uw_env_end): Add do/while(0) to macro pair, so + uw_env_end reuqires a semicolon. + (uw_catch_begin, uw_catch_end): Likewise. + +2009-11-17 Kaz Kylheku <kkylheku@gmail.com> + + Version 022 + + Fix for bug 28033: crash in string output stream. + (Used by exception handling). + + New kernel object type introduced which allows C string literals + to be used as first-class objects. + + Use of printf-like C formatting eliminated from the code base. + + The dependency on C99 wide character I/O is now minimized. + + * txr.c (version): Bumped to 022. + + * txr.1: Bumped version to 022. + +2009-11-16 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (rebuild): New target. Tired of doing make clean; make. + +2009-11-16 Kaz Kylheku <kkylheku@gmail.com> + + Big round of changes to switch the code base to use the stream + abstraction instead of directly using C standard I/O, + to eliminate most uses of C formatted I/O, + and fix numerous bugs, such variadic argument lists which + lack a terminating ``nao'' sentinel. + + Bug 28033 is addressed by this patch, since streams no longer provide + printf-compatible formatting. The native formatter is extended with + some additional capabilities to take over. + + The work on literal objects is expanded and they are now used + throughout the code base. + + Fixed bad realloc in string output stream: reallocating by number + of wide chars rather than bytes. + + * gc.c (sweep): Debugging code switched from fprintf to format. + + * lib.c (typ_check, type_check2, car, cdr, car_l, cdr_l, list, + num, chrp, apply, cobj_print_op, dump): Retargetted, with help of new + literals, to new funtions that take string objects, rather than raw C + strings. + (obj_print, obj_pprint): Revamped with support for LIT type. + Retargetted to not use C style I/O functions in streams. + + * lib.h (lit): Macro retargetted to another macro so that it expands + its argument. + (lit_noex): New macro, like lit, but does not macro-expand argument. + (auto_str): New macro. + (static_str): New macro. + + * match.c (debugf, debuglf, sem_error, file_err): Converted from C + string to string object. + (dest_bind, match_line, LOG_MISMATCH, LOG_MATCH, match_files): + Retargetted to new interfaces that take string objects rather than raw + C strings. + (complex_stream): New function. + (do_output_line, do_output, extract): Retargetted from C streams to + object streams. + + * parser.h (yyerrorf): Declaration updated. + + * parser.l (yyerror): Call new yyerrorf interface, using auto_str + macro to dress up C string as a temporary object. + (yyerrorf): Changed from C strings to object strings. + (yybadtoken, grammar): Retargetted to new yyerrorf. + + * stream.c (strm_ops): put_string and put_char function pointers + changed to take object strings rather than C strings. vcformat and + vformat virtuals removed. C formatting is not supported, and vformat is + handled above the stream switch level in one place for all streams. + (common_vformat, stdio_vcformat, string_out_vcformat, cformat, + put_cstring, put_cchar): Functions removed. + (stdio_stream_print, stdio_stream_destroy, stdio_maybe_write_error, + stdio_put_string, stdio_put_char, stdio_close, pipe_close, + string_out_put_char, make_pipe_stream, make_string_input_stream, + make_string_output_stream, make_dir_stream, close_stream, + get_line, put_line, get_char, put_char, put_string): Retargetted to new + string object interfaces. + (stdio_ops, pipe_ops): stdio_vcformat and common_vcformat initializers + (string_out_ops): string_out_vcformat and common_vcformat initializers + removed. + (string_in_ops, byte_in_ops, dir_ops): Two null initializers removed. + (string_out_put_string): Converted to object string interface. + Unnecessary chk_realloc call suppressed. + (get_string_from_stream): Fixed bad call to realloc with incorrect + size. + (vformat_num, vformat_str): New functions, helper to vformat. + (vformat): Rewritten. Is now the formatting engine. + (format, put_string, put_char): Interface converted from C string to + object string. + + * stream.h (vformat, format): Declarations updated. + (vcformat, cformat, put_cstring, put_cchar): Declarations removed. + + * txr.c (oom_realloc_handler, help, txr_main): Retargetted to object + stream and strings. + + * unwind.c (uw_throw, type_mismatch, uw_register_subtype): Retargetted + to new string object interfaces. + (uw_throwf, uw_errorf): Interface changed from C string to object + string. + (uw_throwcf, uw_errorcf): Functions removed. + + * unwind.h (uw_throwf, uw_errorf, type_mismatch): Declarations updated. + (uw_throwcf, uw_errorcf): Declarations removed. + (internal_error): Macro interface changed and retargetted to + object strings. Also, num hygiene problem worked around with local + extern declaration. + (numeric_assert, range_bug_unless): Retargetted to object strings. + + * utf8.c (utf8_to, utf8_dup_from_uc, utf8_dup_from, + utf8_dup_to_uc): Casts of chk_malloc return value added. + +2009-11-15 Kaz Kylheku <kkylheku@gmail.com> + + Use the 11 tag bit pattern to denote a new type: LIT. This is a + pointer to a C static string, intended for literals. We can now + treat literal strings as light-weight objects. + + * lib.h (TAG_MASK): Ensure the constant expr has long type. + (TAG_LIT): New macro. + (enum type, type_t): New enum member, LIT. + + * gc.c (finalize, mark_obj): Handle LIT type. + + * hash.c (ll_hash): Likewise. + + * lib.c (code2type, equal, stringp, length_str, c_str, + obj_print): Likewise. + (obj_init): Intern symbols using literal strings. + (type): Parentheses added to macro expansion. + (is_lit, lit, litptr): New macros. + +2009-11-15 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (chr): Take wchar_t argument, not int. Dropped range check. + (c_chr): Return wchar_t not int. + + * lib.h (chr, c_chr): Declarations updated. + +2009-11-15 Kaz Kylheku <kkylheku@gmail.com> + + Version 021. + + Text is represented using wide characters now. Queries and data + are parsed as UTF-8, so extended characters can be directly used. + Numeric character escapes can go up to \x10FFF. (More limited on + platforms where wchar_t is 16 bit). Regular expressions support + extended characters, directly or through escapes. Regex character set + matches can use full Unicode range. New test case 005 exercises + some of these features over Japanese text. + + Failed exit status of pipes, and file close errors are exceptions now. + + Bug fixed in regex character classes. + + Fixed off-by-one error in lazy string implementation, which broke + some uses of the @(freeform) directive. + + Fixed all instances of gc bug 28086: objects being prematurely + reclaimed. This showed up when compiling for profiling (gcc -pg). + + The --cc argument of the configure script works properly now. + + Numbers and characters are unboxed types now, encoded directly in + the (obj_t *) value. Lowest two bits of (obj_t *) are a tag + distinguishing characters, integers and pointers. The program + performs better from not having to cons memory when operating + on numbers and characters. + + Discovered bug in glibc: getwc function segfaults when applied to + stream returned by popen. Worked around this bug. Bug is filed + as 10958 in glibc bugzilla. + + Internals: + + Hash tables implemented. Hash tables support weak keys and values. + + * configure, hash.c, lib.c, stream.c, utf8.c: Removed trailing + from some lines. + + * txr.c (version): Bumped to 021. Removed trailing whitespace. + + * txr.1: Bumped version to 021. + +2009-11-14 Kaz Kylheku <kkylheku@gmail.com> + + Provide both char * and unsigned char * interfaces in UTF-8 module. + Fix unsigned and plan char * mixing. + + * utf8.c (utf8_from_uc, utf8_to_uc, utf8_dup_from_uc, + utf8_dup_to_uc): New functions. + (utf8_from): Fix type of backtrack pointer to unsigned char *. + + * utf8.h (utf8_from_uc, utf8_to_uc, utf8_dup_from_uc, + utf8_dup_to_uc): Declared. + + * lib.c (string_utf8): Changed to take char * argument. + + * lib.h (string_utf8): Declaration updated. + +2009-11-14 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (depend): Marked phony and $(PROG) prerequisite dropped. + (clean, distclean, tests, install): Phony targets marked phony. + + * dep.mk: Regenerated. + +2009-11-14 Kaz Kylheku <kkylheku@gmail.com> + + * configure (cc): Compute variable properly. + +2009-11-14 Kaz Kylheku <kkylheku@gmail.com> + + Fixes for bug 28086. When constructing a cobj, whose associated + C structure contains obj_t * references, we should initialize + that C structure after allocating the cobj. If we initialize + the structure first, it may end up having the /only/ references + to the objects. In that case, the objects are invisible to the + garbage collector. The subsquent allocation of the cobj itself + then may invoke gc which will turn these objects into dust. + The result is a cobj which contains a handle structure that + contains references to free objects. The fix is to allocate + the handle structure, then the cobj which is associated with + that handle, and then initialize the handle, at which point it + is okay if the handle has the only references to some objects. + Care must be taken not to let a cobj escape with a partially + initialized handle structure, and not to trigger gc between + allocating the cobj, and initializing the fields. + + * hash.c (make_hash): Fix cobj construction order. + + * stream.c (make_stdio_stream): Fix cobj construction order. + (make_pipe_stream): Fix cobj construction order. Also + noticed and fixed a bug: h->descr field not being initialized + in the currently enabled BROKEN_POPEN_GETWC variant of the code. + +2009-11-13 Kaz Kylheku <kkylheku@gmail.com> + + New testcase which does some UTF-8 scanning, Unicode regexes, + and @(freeform). + + * tests/005/data: New UTF-8 file. + + * tests/005/query-1.txr: Likewise. + + * tests/005/query-1.expected: Likewise. + + * Makefile (TXR_ARGS): New target-specific assignment to set + data for test case set 005. + +2009-11-13 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (symbolp): Bugfix: function crashed on NUM argument. + (lazy_str): Fix for gc correctness: object from make_obj must be + completely intialized before any gc-triggering operation is invoked, + otherwise the garbage collector will be traversing an object + whose fields contain old garbage. + (lazy_str_force_upto): Off-by-one error. To force the object + up to index position N, means forcing up to length N+1. + This bug can make it look like a lazy string is much shorter + than it really is. + +2009-11-13 Kaz Kylheku <kkylheku@gmail.com> + + Allow -c scripts to not have a trailing newline. + Test suite exercises -c now. + + txr.c (txr_main): If the script specified with -c is not terminated + by a newline, just add a newline. On the shell command line, it's a + nuisance to have to add the extra line before closing the quote. + It's also awkward in scripting, because the shell (or at + least Bash 3.0) does not produce a final terminating newline in command + substitution syntax like -c "$(cat file)". The last newline in + the file is trimmed, and has to be explicitly added in the script + itself, which is wrong in the case when the file is empty. + + Makefile (TXR_SCRIPT_ON_CMDLINE): New target-specific variable, + arbitarily set for test 002. + (%.ok: %.txr): Rule updated to honor TXR_SCRIPT_ON_CMDLINE + variable, passing the script body to txr using -c rather than + as a file argument. + + txr.1: Document -c change. + +2009-11-13 Kaz Kylheku <kkylheku@gmail.com> + + Previous commit broke UTF-8 lexing, by changing the get_char + semantics on the input stream to wide character input. + Also, reading a query the command line (-c) must + read bytes from a UTF-8 encoding of the string. + We introduce a new get_byte function which can extract bytes + from streams which provide it. + + * parser.l (YYINPUT): Call get_byte instead of get_char. + + * stream.c (struct strm_ops): New function pointer, get_byte. + (stdio_get_byte): New function. + (stdio_ops, pipe_ops): Add new function. + (string_in_ops, string_out_ops, dir_ops): Null pointer added. + (struct byte_input): New struct type. + (byte_in_get_byte): New function. + (byte_in_ops): New structure. + (make_string_byte_input_stream, get_byte): New functions. + + * stream.h (make_string_byte_input_stream, get_byte): New functions. + + * txr.c (txr_main): Make a byte input stream from the command + line spec, rather than a string input stream. + +2009-11-12 Kaz Kylheku <kkylheku@gmail.com> + + Continuing wchar_t conversion. Making sure all stdio calls + use wide character functions so that there is no illicit + mixing. (But the goal is to replace this usage with txr streams). + + * lib.c (list, cobj_print_op, obj_print, obj_pprint): Use + wide string literals and I/O functions. + + * match.c (debuglcf): Converted to wchar_t. + (dump_bindings, match_line, match_lines, extract): Use wide string + literal and I/O function. + + * parser.h (yyerrorf): Declaration updated. + + * parser.l (yyerror): Use wide-string yyerrorf. Users of + yyerror continue to pass multibyte strings. + (yyerrorf): Converted to wchar_t. + (yybadtoken, grammar): Use wide string literals to call yyerrorf. + + * stream.c (struct strm_ops): vcformat changed to wchar_t. + (stdio_vcformat, string_out_vcformat, vcformat, cformat): Likewise. + + * stream.h (vformat, vcformat, cformat): Declarations updated. + + * txr.c (oom_realloc_handler, help, hint, txr_main): Use wide string + literals and I/O functions. + + * unwind.c (uw_throwcf, uw_errorcf): Converted to wchar_t. + + * unwind.h (uw_throwcf, uw_errorcf): Declarations updated. + (internal_error, numeric_assert, range_bug_unless): Macros + fixed to use wide string literals. + +2009-11-12 Kaz Kylheku <kkylheku@gmail.com> + + * utf8.c (utf8_from): Fix total breakage. + Was writing out incomplete wide characters on internal + state transtions while traversing a single multi-byte character. + Also, improved handling of bad bytes close to EOF: if EOF + occurs in a multi-byte character, it will backtrack, + and skip one bad byte, etc. + (utf8_encode, utf8_decoder_init, utf8_decode): New functions. + + * utf8.h (enum utf8_state): New enum. + (struct utf8_decoder, utf8_decoder_t): New struct. + (utf8_encode, utf8_decoder_init, utf8_decode): Declared. + +2009-11-12 Kaz Kylheku <kkylheku@gmail.com> + + Documenting extended characters in man page. + Cleaned up some more issues related to extended characters. + + * parser.l (grammar): Added error sctions for invalid UTF-8 bytes. + + * stream.c (BROKEN_POPEN_GETWC): New macro. Enables workaround + for a glibc bug, whereby getwc blows up when applied to a FILE * + stream returned from a popen call. + (struct strm_ops): put_char function takes wchar_t. + (common_format): Use wchar_t rather than int. + (stdio_put_string): fputws returns -1, not EOF. + (stdio_put_char, put_cchar): Character argument changed to wchar_t. + Output done with putwc used instead of putc. + (snarf_line, stdio_get_char): Use getwc to read from the stream. + (pipe_close, make_pipe_stream): Implement workaround form glibc bug. + + * stream.h (put_cchar): Declaration updated. + + * txr.1: Added notes about international characters. + +2009-11-12 Kaz Kylheku <kkylheku@gmail.com> + + Regular expression module updated to do unicode character sets. + Most of the changes are in the area of representing sets. + + Also, a bug was found in the compilation of regex character sets: + ranges straddling two adjacent blocks of 32 characters were + not being added to the character set. However, ranges falling + within a single 32 block, or spanning three or more such blocks, + worked properly. This bug is not tickled by common ranges + such as A-Z, or 0-9, which land within a 32 block. + + * regex.h (BITCELL_LIT): Macro removed. + (CHAR_SET_SIZE): Macro does not depend on UCHAR_MAX any more, + but hard-codes a set size of 256. UCHAR_MAX means nothing to us any + more since we are using wchar_t. The number 256 is simply an + arbitrarily chosen size for representing the small character + sets (or the leaves of the radix tree for representing large sets). + (chset_type_t): New enum typedef. + (cset_L0_t, cset_L1_t, cset_L2_t, cset_L3_t): New array typedefs. + (struct char_set): Replaced by union char_set. + (struct any_char_set, struct small_char_set, struct displaced_char_set, + struct large_char_set, struct xlarge_char_set): New struct types. + (char_set_clear): Declaration removed. + (char_set_create, char_set_destroy): Declared. + (char_set_add, char_set_add_range, char_set_contains, + nfa_state_single, nfa_state_set, nfa_machine_feed): Declarations + updated for wchar_t. + (struct nfa_state_single): member ch changed to wchar_t. + + * regex.c (char_set_clear): Function removed. + (CHAR_SET_L0, CHAR_SET_L1, CHAR_SET_L2, CHAR_SET_L3, CHAR_SET_L2_L0, + CHAR_SET_L2_HI, CHAR_SET_L1_L0, CHAR_SET_L1_HI, CHAR_SET_L0_L0, + CHAR_SET_L0_HI): New macros. + (L0_full, L0_fill_range, L0_contains, L1_full, L1_fill_range, + L1_contains, L1_free, L2_full, L2_fill_range, L2_contains, + L2_free, L3_fill_range, L3_contains, char_set_create, + char_set_destroy): New functions. + (char_set_compl): Works using a flag rather than by actually + computing a complemented set. Also, is no longer a toggle (and + was never used that way). + (char_set_add, char_set_add_range, char_set_contains): Polymorphic over + the different set types. + (nfa_state_single, nfa_move, nfa_run, nfa_machine_feed): Converted + to wchar_t. + (nfa_state_free): Use char_set_destroy to free set. + (nfa_state_set): Does not construct the set internally but + takes it as a parameter. + (nfa_compile_set): Rewritten to perform two passes over the + s-expression representing the list of characters and ranges + making up the set. The first pass determines what representation + will be used for the set. The second pass stuffs the characters and + ranges into the set. + +2009-11-11 Kaz Kylheku <kkylheku@gmail.com> + + * txr.c (main): call setlocale to set the LC_CTYPE to en_US.UTF-8, + so that the C library streams do the encoding. Once the program + is weaned from C library wide character stream I/O, this can go away. + +2009-11-11 Kaz Kylheku <kkylheku@gmail.com> + + Big conversion to wide characters and UTF-8 support. + This is incomplete. There are too many dependencies on + wide character support from the C stream I/O library. + The regex code does not handle wide characters properly. + Character type is still int in some places, rather than wchar_t. + Test suite passes though. + + * hash.c (hash_str): Converted to wchar_t. + + * lib.c (progname, type_check, type_check2, type_check3, + car, cdr, car_l, cdr_l, equal, chk_strdup, string_own, + string, mkstring, mkustring, init_str, length_str, + c_str, search_str, sub_str, cat_str, split_str, trim_str, + chrp, apply, lazy_str, lazy_str_get_trailing_list, + cobj, obj_init, obj_print, obj_pprint, init): Converted to wchar_t. + (vector): Cast of chk_malloc return value added. + (string_utf8): New function. + + * lib.h (struct string): Member str changed to wchar_t *. + (progname, chk_strdup, string_own, string, init_str, + c_str, init): Declarations updated. + (string_utf8): Declared. + + * match.c (debugf, debuglf, sem_error, file_err, dump_shell_string, + dump_var, dump_bindings, dest_bind, match_line, do_output_line, + do_output, match_files): Converted to wchar_t. + + * parser.h (spec_file): Declaration updated. + + * parser.l (yy_errorf, char_esc, num_esc): Converted to wchar_t. + (ASC, ASCN, U, U2, U3, U4, UANY, UNANN, UONLY): New named + regexes, used for lexing utf-8. + (grammar): Converted to wchar_t and utf-8 handling. + + * parser.y (%union/yystype): lexeme member changed to wchar_t *, + chr member changed to wchar_t. + + * regex.c (nfa_run): Input string is wchar_t *. + (search_regex): String from haystack is wchar_t *. + + * regex.h (nfa_run): Declaration updated. + + * stream.c (struct strm_ops, common_vformat, stdio_stream_print, + stdio_maybe_read_error, stdio_maybe_write_error, stdio_put_string, + stdio_put_char, snarf_line, stdio_get_line, stdio_close, pipe_close, + struct string_output, string_out_put_string, string_out_put_char, + string_out_vcformat, dir_get_line, make_string_output_stream, + get_string_from-stream, make_dir_stream, get_line, get_char, + vformat, vcformat, format, cformat, put_string, put_cstring, + put_char, put_cchar, stream_init): Converted to wchar_t. + + * stream.h (vformat, format, put_cstring): Declarations updated. + + * txr.c (version, progname, spec_file, oom_realloc_handler, + help, hint, remove_hash_bang_line, main, txr_main): Converted + to wchar_t. + + * txr.h (version, progname): Declarations updated. + + * unwind.c (uw_throw, uw_throwf, uw_errorf, type_mismatch, + uw_register_subtype): Converted to wchar_t. + + * unwind.h (uw_throwf, uw_errorf, type_mismatch): Declarations updated. + + * utf8.c, utf8.h: New files. + +2009-11-10 Kaz Kylheku <kkylheku@gmail.com> + + hash.c (hash_grow): Rewritten to avoid resizing the vector + in place, and thus having to pulling all conses into a big list. + TODO: avoid recomputing the hash function over the keys. + We could enhance cons cells with two more fields without using + additional storage. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + Changing representation of objects to allow for unboxed characters. + Now numbers and characters fit into a cell. We lose one more bit + from the range of numbers. + + * lib.h (TAG_SHIFT, TAG_MASK, TAG_NUM, TAG_PTR, NUM_MASK, NUM_MIN, + is_ptr, is_num): Macros updated. + (is_chr, tag): New macros. + (struct chr): Removed. + (union obj): Updated. + + * lib.c (typeof, equal, chr, chrp, c_chr, obj_print): Updated. + + * hash.c (ll_hash): Characters aren't pointers any longer; + use abstract accessor. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + Add hash removal. + + * hash.c (remhash): New function. + + * hash.h (remhash): Declared. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + Add hash table growth. + + hash.c (hash_grow): New function. + (l_gethash): Renamed to gethash_l. Increment count; if load + factor gets to two, call hash_grow to double the size. + + hash.h (l_gethash): Declaration changed to gethash_l. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + Changing representation of objects to allow the NUM type to be + unboxed. If the lowest bit of the obj_t * pointer is 1, then + the remaining bits are a number. A lot of assumptions are made: + - the long type can be converted to and from a pointer + - two's complement. + - behavior of << and >> operators when the sign bit is involved. + + * lib.h (TAG_SHIFT, TAG_MASK, TAG_NUM, TAG_PTR, NUM_MASK, NUM_MIN, + is_ptr, is_num, type): New macros. + (struct num): Removed. + (nao): Redefined, so that it doesn't have the numeric tag. + + * lib.c (typeof, type_check2, type_check3, car, car_l, cdr, cdr_l, + equal, consp, atom, listp, num, c_num, nump, plus, minus, + stringp, lazy_stringp, obj_print, obj_pprint): Fixed these + functions to use the new number representation, and not to deference + the obj_t * poitner if it is actually a number. + (obj_init): Adjusted values of maxint and minint. + + * gc.c (mark_obj, gc_is_reachable): Avoid dereferencing numbers. + + * hash.c (ll_hash): Likewise. + + * match.c (match_line, do_output_line): Likewise. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + First cut at hash tables. One known problem is allocation during gc, + due to use of boxed numbers for vector access. + + * gc.c (gc): Disable gc when doing garbage collection, in case + something tries to allocate memory during gc, triggering a recursive + gc, which would be very bad. Also, call the new function, + hash_process_weak, in between the mark and sweep phases. + (gc_is_reachable): New function. + + * gc.h (gc_is_reachable): Declared. + + * lib.c (hash_t): New symbol global. + (acons_new_l): New function. + (obj_init): New symbol interned. + + * lib.h (hash_t, acons_new_l): Declared. + + * hash.c, hash.h: New files. + + * Makefile: New target, hash.o. + + * dep.mk: Regenerated. + +2009-11-06 Kaz Kylheku <kkylheku@gmail.com> + + Throw exception on stream error during close, or I/O operations. This + is needed for pipes that terminate abnormally or return failed + termination. Pipe and stdio streams have an extra description field + so they are printed in a readable way. + + * lib.c (process_error): New global defined. + (obj_init): New symbol interned. + (lazy_stream_func): Pass t to close_stream, so exception + is thrown if the close fails. + (lazy_stream_cons): Ditto. + + * lib.h (process_error): Declared. + + * match.c (complex_snarf): Pass new desr argument to make_stdio_stream + and make_pipe_stream. + + * stream.c (strm_ops): New argument on close function pointer. + (common_destroy): Close without throwing exception. For objects + being finalized, we don't care if the close works or not; the + program has shown that it doesn't care about the stream by + letting it become unreachable, so we don't bother the program + by throwing an exception. + (stdio_handle): New struct. + (stdio_stream_print, stdio_stream_destroy, stdio_stream_mark, + stdio_maybe_read_error, stdio_maybe_write_error): New functions. + (stdio_put_string, stdio_put_char, stdio_get_line, + stdio_get_char, stdio_vcformat, stdio_close): Updated to new + handle format, and throw errors now. + (stdio_ops, pipe_ops): Redirected to new functions stdio_stream_print, + stdio_stream_destroy and stdio_stream_mark. + (pipe_close): Updated to new handle format. Parses status from + pclose and throws exceptions appropriate to the situation. + (dir_close): Takes extra argument. + (make_stdio_stream, make_pipe_stream): New argument added. + (make_string_output_stream): Some casts added. + (close_stream): Pass new argument down to virtual function. + (stream_init): Pass new argument to make_stdio_stream + when creating streams for stdin, stdout and stderr. + + * stream.h (make_stdio_stream, make_pipe_stream, close_stream): + Declarations updated. + + * txr.c (txr_main): Pass new argument to make_stdio_stream. + + * unwind.c (uw_init): Register process_error. + +2009-11-01 Kaz Kylheku <kkylheku@gmail.com> + + Version 020 + + Improved documentation. + + Building via configure script. + + Support for cross compiling support. + + Support for building in separate build directory. + + Internal bugfixes. + + Portability bugs fixed; works on x86-64 GNU/Linux. + +2009-11-01 Kaz Kylheku <kkylheku@gmail.com> + + Bug ID 27898: Directory order dependencies in test case. + Converted some directories to text files. + + * tests/002/proc/*/task: Directories removed. + + * tests/002/proc/*/tasks: Files created. + + * tests/002/query-1.txr: Query updated. + + * tests/002/query-1.expected: Expected output updated. + +2009-11-01 Kaz Kylheku <kkylheku@gmail.com> + + Bug ID 27895: Calls to protect have an argument list terminated + by the integer constant 0 rather than a proper null pointer constant. + + lib.c (obj_init): Properly terminate argument list of protect call. + + stream.c (stream_init): Likewise. + + unwind.c (unwind_init): Likewise. + + txr.c (txr_main): two-argument protect calls rewritten using prot1. + +2009-11-01 Kaz Kylheku <kkylheku@gmail.com> + + Bug ID 27899: Garbage collection problem: method of locating stack + bottom is unreliable due to the unpredictable allocation order of local + variables. The addresses of stack_bottom_0 and stack_bottom_1 variables + do not necessarily bracket the others which means that some local + variables in main can be out of the reach of the garbage collector: + our stack bottom is wrongly in the middle of the frame. + + * lib.c (init): Removed one of the stack bottom parameters, so there + is only one. This is passed straight down to gc_init. + Also noticed that the oom_realloc variable was not being set + from the oom parameter. + + * lib.h (init): Declaration updated. + + * txr.c (txr_main): New static function. + (main): Calls init, and then txr_main. The idea is that txr_main + should get fresh stack frame. So the stack_bottom variable in main + should be outside of that stack frame. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (equal): Fix broken LSTR and FUN cases. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + Got "make tests" working in separate build directory, + with .out files going to local tests/ tree. + + * Makefile (depend): Refer to depend.txr and dep.mk + using $top_srcdir; no need for symlinks. + Changed a few more ./txr references to use $(PROG). + (TESTS): Path munging to generate targets with local paths. + (%.ok): Fixed diff logic to compare between .expected file + in $(top_srcdir) and local .out file. + + * configure: Don't generate symlinks for tests and dep.mk. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + Got "make install" working. + + * Makefile (install): New target. + + * configure (mandir, bindir): New variables. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + Got build to work in separate build directory. + + * Makefile (CFLAGS): Added -I flag to point header inclusion to the + source directory. + (PROG): New variable to hold program name. + (VPATH): Variable set, as a quick and dirty way to get GNU make + to find the prerequisites back in the source directory. + * configure: Added steps to symlink the tests directory and dep.mk. + * depend.txr: Modified to generate the dependencies with + correct references to the top_srcdir, with the exception of + locally generated headers. + * dep.mk: Regenerated. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + Build configuration via configure script, with cross compiling support. + (Tested by cross-compiling txr on an x86 GNU/Linux system + to run on a MIPS-based GNU/Linux system). + + * configure: New script. + * Makefile: (OPT_FLAGS, LANG_FLAGS, DIAG_FLAGS, DBG_FLAGS, + LEX_DBG_FLAGS, TXR_DBG_OPTS, LEXLIB): Variables removed; + these are now generated in config.make by configure. + (config.make): New target to print friendlier diagnostic if + the build is not configured. + (distclean): New target to do clean, plus remove config.make. + +2009-10-22 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (YY_INPUT): Kill tabs with spaces (how did they sneak in?). + Fix possible use of uninitialized ch. + +2009-10-21 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Fixed misleading wording (separation versus termination). + Added Introduction headings to some major sections. + Improved exception handling intro. + +2009-10-21 Kaz Kylheku <kkylheku@gmail.com> + + Version 019 + + Regexps can be bound to variables. + + New freeform directive. + + * txr.c (version): Bump. + * txr.1: Bump version and date. + + * lib.c, match.c, regex.c, regex.h, stream.c: Trailing whitespace + removed from lines. + +2009-10-21 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Documented freeform. + +2009-10-21 Kaz Kylheku <kkylheku@gmail.com> + + Change the freeform line catenation semantics to termination + rather than separation. + + * lib.h (lazy_str): Declaration updated. + + * lib.c (lazy_str): Tack terminator onto initial prefix + string. Parameter renamed. Also, terminator string cached + in the object. + (lazy_str_force, lazy_str_force_upto): Terminate, rather + than separate. + + * match.c (match_files): sep variable renamed to term. + +2009-10-21 Kaz Kylheku <kkylheku@gmail.com> + + * gc.c (mark_obj): Bugfix: recurse over recently added + member, opts, in the lazy_string structure. + +2009-10-20 Kaz Kylheku <kkylheku@gmail.com> + + Got regex working over lazy strings from freeform. + + Bugfixes. + + * lib.c (length_str): Fixed recursion to wrong length function. + (lazy_str_force): March down list properly. Update lazy + string's limit value. + + * match.c (match_line): Convert to lazy-string-aware style; i.e. + avoidance of triggering a force of the whole string. + (match_files): Bugfix in argument processing of freeform directive. + + * regex.h (nfam_result_t): New typedef. + (nfa_machine_reset): New function declaration. + (nfa_machine_feed): Updated declaration. + + * regex.c (nfa_machine_init): Refactor to use nfa_machine_reset. + (nfa_machine_feed): Return nfam_result_t rather than just int. + (search_regex, match_regex): Refactor to work with lazy strings well. + +2009-10-20 Kaz Kylheku <kkylheku@gmail.com> + + Implement custom separator and limit in freeform. + + * lib.h (lazy_string): New struct member, opts. + (lazy_str): Declaration updated. + + * lib.c (lazy_str): New constructor parameters to set the + seprator string and numeric line limit. + (lazy_str_force, lazy_str_upto): Honor the line limit, + and use the separator string if provided. + + * match.c (match_files): Process the arguments for freeform + directive. + +2009-10-20 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (sub_str): Avoid invoking c_str which forces the + lazy string. + +2009-10-20 Kaz Kylheku <kkylheku@gmail.com> + + Start of implementation for freestyle matching. + + Lazy strings implemented, incompletely. + + Changed string function to implicitly strdup; non-strdup + version changed to string_own. Fixed wrong uses of strdup + rather than chk_strdup. + + Functions added to regex module to provide regex matching + as a state machine to which characters are fed. + + * lib.h (type_t): New enum member LSTR, for lazy strings. + (lstr_t, freestyle, type_check3, string_own): Declared. + (string): Parameter changed to const char *. + (lazy_stringp, split_str, lazy_str, lazy_str_force_upto, + lazy_str_force, lazy_str_get_trailing_list, length_str_gt, + length_str_ge, length_str_lt, length_str_le): Declared. + + * lib.c (lstr_t, freestyle): New symbol globals. + (code2type, obj_print, obj_pprint, equal): Extended to handle LSTR. + (type_check3): New function. + (string_own): New function; does what string used to do. + (string): Duplicates the string with strdup, so callers don't have to. + (mkstring, copy_str, trim_str): Use string_own. + (stringp): A lazy string is a kind of string. + (lazy_stringp): New function. + (length_str, c_str, search_str, sub_str, chr_str, + chr_str_set): Handle lazy strings. + (split_str): New function. + (lazy_str, lazy_str_force_upto, lazy_str_force, + lazy_str_get_trailing_list, length_str_gt, length_str_ge, + length_str_lt, length_str_le): New functions. + (obj_init): New symbols interned. Eliminated strdup calls. + + * gc.c (finalize, mark_obj): Changed to handle LSTR type. + Eliminated default case from switch so we get a gcc + diagnostic if a case is not handled. + + * match.c (match_files): Eliminated strdup calls. + Added freeform directive. + + * parser.y (grammar): Changed string calls to string_own. + + * stream.c (stdio_get_line, get_string_from_stream): + Changed string calls to string_own. + (dir_get_line): Eliminated chk_strdup call. + + * txr.c (remove_hash_bang_line, main): Eliminated strdup calls. + + * regex.h (nfam_result): New union. + (nfa_machine, nfa_machine_t): New struct and typedef. + (nfa_machine_init, nfa_machine_cleanup, nfa_machine_feed, + nfa_machine_match_span): New functions declared. + + * regex.c (nfa_machine_init, nfa_machine_cleanup, nfa_machine_feed, + nfa_machine_match_span): New functions defined. + +2009-10-18 Kaz Kylheku <kkylheku@gmail.com> + + Trivial change allows regexps to be bound to variables, + and used for matching. This Just Works because of + the way match_line treats variables. + + * match.c (eval_form): Check for a regexp form and return + it as a value representing itself. + * regex.c (regexp): New function. + * regex.h (regexp): Declared. + +2009-10-17 Kaz Kylheku <kkylheku@gmail.com> + + * deps.mk: Updated. + +2009-10-17 Kaz Kylheku <kkylheku@gmail.com> + + Version 018 + + Bugfixes: mistakes in debugging calls; infinite looping + bug in collect; skip directive not advancing match + by proper number of lines bug. + + * match.c (debuglcf): Cosmetic fix. + (match_files): After recognizing nothrow in the file spec, + replace it by a string. A few places expect first(files) to be + a string. The skip directive must return whatever + return value it obtained from the nested match_files call, + and not substitute the current line number, so that the + caller can proceed past the correct number of lines that + were matched. Fixed obj_t * being passed to %s printf specifier + in debug printf. Collect directive must make progress even + if the nested spec makes no progress (returns successfully, + but with the original line number). + * txr.c (version): Bump. + * txr.1: Bump version and date. + * txr/tests/004/query-1.txr: New test case. + * tests/004/query-1.expected: Expected result for new test case. + +2009-10-17 Kaz Kylheku <kkylheku@gmail.com> + + Version 017 + + Bugfix in exception subtype definition (defex). + + Tail recursion in marking function of garbage collector. + + -f option for specifying query file, allowing more + options to follow, useful in hash-bang scripting and + other situations. + + * txr.c: (version): Bump to 016 + * txr.1: Bump version to 016. + +2009-10-17 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Documented defex. + * unwind.c (uw_register_subtype): Bugfix: if the subtype + exists already, we must not delete it and create a new entry, but + destructively point its entry to its assigned supertype. + An exceptions is thrown rather than abort for attempts + to make t a subtype of something other than itself. + An attempt to make something other than nil a subtype of nil + is diagnosed. Attempts to redefine the relationship + between two types if they are already connected by one; + this covers circularity and other cases, while still allowing + a relaxed order of definition. + +2009-10-17 Kaz Kylheku <kkylheku@gmail.com> + + * gc.c (mark_obj_tail): New macro. + (mark_obj): Optimized with manual tail recursion. + The funtion will no longer generate long call stacks + for long lists. Descending to the car field of + a cons is still recursive, but ``car-heavy'' + trees are rare. + +2009-10-16 Kaz Kylheku <kkylheku@gmail.com> + + Resurrect -f option, with different meaning. + We need "-f query-file" so that hash-bang scripts + can be written which can pass options to txr. + + * txr.c (help, main): Inplement and document -f. + Also bugfix: do not throw file open errors as exceptions + of type error, because these cause an abort, potentially leading to a + core dump. They are now thrown as file_error. + * txr.1: Documented -f. + +2009-10-16 Kaz Kylheku <kkylheku@gmail.com> + + Implemented @(next arg) for treating the command line + as an input source. + + * txr.1: Updated, and fixed a few unrelated mistakes. + * lib.c (dir): Removed unused symbol globa. + (args): New symbol global. + * lib.h (dir): Declaration removed. + (args): Declared. + match.c (match_files): Implemented @(next arg). + Had to hack laziness to the file opening logic in match_files. + If the function is entered with a spec whose first + directive is @(next), then it defers opening the first + file in the list of files (since it will be immediately + abandoned in favor of another input source). + This prevents an error in the situation when the + arguments do not name files, and there is a @(next args) + directive to process them as an input source. + +2009-10-16 Kaz Kylheku <kkylheku@gmail.com> + + Version 016 + + Catch clauses with parameters. + + Directive for throwing exceptions: throw. + + Directive for defining exception types: defex. + + -f option renamed to -c. + + * txr.c: (version): Bump to 016 + * txr.1: Bump version to 016. + +2009-10-16 Kaz Kylheku <kkylheku@gmail.com> + + * txr.c (help, main): Changed -f argument to -c. This + is consistent with the -c argument of the shell; + -f looks like awk's -f option, which specifies a file, + not a literal script body. + + * txr.1: Updated. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: Grammar, spelling. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + * parser.y (clauses_opt): Long overdue nonterminal added. + (define_clause) simplified with clauses_opt. + (try_clause): Error handling improved. + (catch_clauses_opt): Catch and finally clauses can be empty. + Error cases added. + * txr.1: Updated. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): Use alist_remove1 for a one + element removal. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + * unwind.c (uw_throw): Add program prefix before + unhandled exception text. Print it in the + standard notation if it's not a string literal. + * match.c (sem_error, file_err): Don't stick program + prefix into exception text. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + * unwind.c (uw_exception_subtype_p, uw_init): + Slight change in representation for exception subtypes, + saving one node in the list. + +2009-10-15 Kaz Kylheku <kkylheku@gmail.com> + + New throw and defex directives, catches with arguments. + + * lib.c (defex, throw): New symbol globals. + (obj_init): Symbols interned. + * lib.h (defex, throw): Declared. + * match.c (match_files): Implemented throw and defex. + Argument handling in catches. + * unwind.c (uw_register_subtype): Returns right + argument, so we can cleverly use it with reduce_left. + * unwind.h (uw_register_subtype): Declaration updated. + * txr.1: Updated. + +2009-10-14 Kaz Kylheku <kkylheku@gmail.com> + + Version 015 + + Code restructuring. + + Corruption bugfix in gc-debugging code. + + The nil symbol more properly implemented. + + Semantics change: collect treated as a failed match if it + does not collect anything. + + Bugfix in function argument reconciliation: must only + be done for unbound parameters. + + New @(local) directive (synonym of forget) for expressing + local variables in functions. + + Quasi-literals: backquote-delimited literals that contain interpolated + variables. Useful in next, output, bind and function calls. + + Hygiene: some implementation-inserted syntax tree elements + are now in their own namespace so they can't clash with user-defined + constructs. + + Rewritten streams implementation. + + Exception handling: try/catch/finally. + + Exceptions used internally and externally. + + File errors are mapped to exceptions now. + + Hash bang (#!) scripting supported. + + New -f paramater, allowing entire query to be specified + as argument rather than from a file or stdin. + + * txr.c: (version): Bump to 014. + * txr.1: Bump version to 014. More documentation about + exceptions. + +2009-10-14 Kaz Kylheku <kkylheku@gmail.com> + + Support for hash bang execution, and embedding query + in a command line argument. + + * txr.c (remove_hash_bang_line): New function. + (main): Added -f option. Initialize and gc-protect yyin_stream, and + use it in all places where yyin was previously set up. + Diagnose when -a, -D and -f are wrongly clumped with other options. + Remove the first line of the query if it starts with #!. + * parser.h (yyin): Declaration removed. + (yyin_stream): Declared. + * parser.l (YY_INPUT): Macro defined. + (yyin_stream): New global. + * stream.c (string_in_get_line, string_in_get_char): Bugfix: + wrong length function used. + (string_in_ops): Bugfix: wrong get_char function wired in. + (get_char): New function. + * stream.h (get_char): Declared. + * txr.1: -f option documented. + +2009-10-14 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (obj_print, obj_pprint): Print #<garbage ...> + syntax if an object has a bad type code; do not just return + without printing anything. + +2009-10-14 Kaz Kylheku <kkylheku@gmail.com> + + Code cleanup and documentation. + + * txr.1: Start documenting quasiliterals, exception handling and + nothrow in next and output. + * parser.y (catch_clauses_opt): Add missing empty production, so that + a try block doesn't have to have a finally clause. + * lib.h (or2, or3, or4): New macros. + * match.c (match_files): Allow output and next forms which just + have one argument that is nothrow, as documented. + * stream.c common_vformat, string_out_vcformat, string_out_vcformat, + make_string_output_stream, make_dir_stream, close_stream, get_line, + vformat, vcformat, format, cformat, put_string, put_cstring, + put_char): Switch to new style type assertions. + +2009-10-13 Kaz Kylheku <kkylheku@gmail.com> + + New syntax for next and output directives, taking advantage + of quasi-literals. Non-throwing behavior can be specified in + both using nothrow. The old syntax is supported, and has + the old semantics (non-throwing). Hence, the test cases + pass again without modification. + + File open errors thrown as file_error type. + + * lib.c (nothrow, file_error): New symbol globals. + (obj_init): New symbols interned. + * lib.h (nothrow, file_error): Declared. + * match.c (file_err): New function. + (eval_form): Bugfix: if input is nil, or an atom other than a symbol, + return the value hoisted into a cons. A nil return strictly means, + unbound variable. + (match_files): Support new syntax for next and and output. + Throw open errors as file_err. + * parser.l (grammar): Change how OUTPUT is returned to the + style similar to DEFINE, so interior forms can be parsed. + * parser.y (grammar): Fix up output_clause with new syntax. + * unwind.c (uw_throw): Do not abort on unhandled file_error, + but terminate with a failed status. + (uw_init): Register file_error as a subtype of error exception. + +2009-10-13 Kaz Kylheku <kkylheku@gmail.com> + + First cut at working try/catch/finally implementation. + + * lib.c (try, catch, finally): New symbol globals. + (obj_init): New symbols interned. + * lib.h (try, catch, finally: Declared. + * parser.y (TRY, CATCH, FINALLY): New tokens. + (try_clause, catch_clauses_opt): New nonterminal grammar symbols. + * parser.l (yybadtoken): TRY, CATCH and FINALLY handled. + (grammar): New cases for try, catch and finally. + * unwind.h (struct uw_catch): New member called visible. + (uw_continue): New parameter added. + (uw_exception_subtype_p): Declared. + (uw_catch_begin): Macro rewritten to use switch logic + around setjmp. + (uw_do_unwind, uw_catch, uw_unwind): New macros. + (uw_catch_end): Rewritten to close switch, and automatically + continue the unwinding if the block is entered as an unwind. + * unwind.c (uw_unwind_to_exit_point): Exception catching + frames made invisible via new flag prior to control passing to them. + longjmp code 2 introduced for distinguishing a catch from + an unwind. Visibility flag is checked and invisible frames + are skipped. + (uw_push_catch): cont member of the unwind frame initialized to zero. + (exception_subtype_p): Renamed to uw_exception_subtype_p, changed + to extern. Fixed wrong order of arguments to assoc. + (uw_throw): Honor visibility flag: do not consider invisible + catch frames. + (uw_register_subtype): sup/sub mixup bugfix. + (uw_continue): Takes extra argument: the continuation frame + that (re)establishes the exit point for the unwinding. + This allows nested unwinding action to take place in a finally, + and then to continue to the original exit point. + * match.c (match_files): Handling for try directive added. + +2009-10-13 Kaz Kylheku <kkylheku@gmail.com> + + * parser.l (yybadtoken): Bugfix: added missing LITCHAR case. + * unwind.h (internal_error): Fixed broken macro. + * match.c (match_line, match_files): sem_error bugfix: used %a instead + of ~a. + (match_files): Wrap block handler in compound statement, otherwise the + macroexpansion declares a variable in the middle of a statement, which + is a gcc extension to C90 (or a C99 feature, + but we aren't using C99). + +2009-10-08 Kaz Kylheku <kkylheku@gmail.com> + + Exception handling for query errors. + Verbose logging decoupled from yyerror functions. + Superior object-oriented formatting used for cleaner code. + + * lib.c (query_error): New symbol global. + (obj_init): New symbol interned. + * lib.h (query_error): Declared. + * match.c (output_produced): Variable changed to external linkage. + (debugf, debuglf, debuglcf, sem_error): New static functions. + (dest_bind, match_line, match_files): Regtargetted away from + the yyerrorf and yyerrorlf functions to use debugf, + debuglf, debuglcf for logging and sem_error for throwing + query errors as exceptions. + * parser.h (spec_file_str): New global declared. + * parser.l (yyerror): Calls yyerrorf instead of yyerrorlf; + lets yyerrorf increment error count. + (yyerrorf): Loses level argument. + (yyerrorlf): Function removed. + (yybadtoken): Retargetted from yyerrorlf to yyerrorf. + (grammar): yyerrorf call fixed up. + * txr.c (spec_file_str): New global defined. + (main): Protects new global against gc, and initializes it. + * unwind.c (uw_throw): If an unhandled exception is of + type query_error, it results in an exit rather than abort. + The false string is conditionally printed. + (uw_init): Register query_error as subtype of error. + +2009-10-08 Kaz Kylheku <kkylheku@gmail.com> + + Exception handling framework implemented. + + * lib.c (cobj_t, error, type_error, internal_err, numeric_err, + range_err): New symbol globals. + (prog_string): New string global. + (code2type): New static function. + (typeof): Rewritten using code2type. + (type_check, type_check2): New static functions. + (car, cdr, list, plus, minus, length_str, chr_p, chr_str, + chr_str_set, apply, funcall, funcall1, funcall2, + vec_get_fill, vecref_l, lazy_stream_cons): Checks and + assertions rewritten using new functions and macros. + (obj_init): prog_string protected from gc. + New symbols interned. + (init): uw_init() call moved after obj_init() because + it needs stable symbols. + * lib.h (cobj_t, error, type_error, internal_err, numeric_err, + range_err, prog_string, type_check, type_check2): Declared. + * match.c (dump_var, complex_snarf, complex_close): abort + calls rewritten to use exception handling. + * regex.c (nfa_all_states, nfa_closure, nfa_move): Likewise. + * stream.c (string_out_vcformat): Bugfix: fill index not updated. + (make_string_output_stream): Bugfix: initial buffer not null terminated. + (get_string_from_stream): New function. + * stream.h (get_string_from_stream): Declared. + * txr.c (main): Some error prints turned to throws. + * unwind.c (unwind_to_exit_point): Supports UW_CATCH frames, + whose finalization logic has to be invoked during unwinding, + and as target exit points. + (uw_init): Installs exception symbols into + subtyping hirearchy. + (uw_push_catch, exception_subtype_p, uw_throw, uw_throwf, + uw_errorf, uw_throwcf, uw_errorcf, type_mismatch, + uw_register_subtype, uw_continue): New functions. + (exception_subtypes): New static global. + * unwind.h (noreturn): New macro, conditionally defined on __GNUC__. + (enum uw_frtype): New member, UW_CATCH. + (struct uw_catch): New struct type. + (union uw_frame): New member, ca. + (uw_push_catch, exception_subtype_p, uw_throw, uw_throwf, + uw_errorf, uw_throwcf, uw_errorcf, type_mismatch, + uw_register_subtype, uw_continue): New functions declared. + (uw_catch_begin, uw_catch_end, internal_error, type_assert, + bug_unless, numeric_assert, range_bug_unless): New macros. + +2009-10-07 Kaz Kylheku <kkylheku@gmail.com> + + Rewritten streams implementation. + + * stream.h, stream.c: New files. + * Makefile (OBJS): New object file stream.o. + * dep.mk: Dependencies updated. + * gc.c (finalize): STREAM case removed. Call destroy only if not null. + (mark_obj): STREAM case removed. + * lib.c (push, pop): New functions. + (equal): STREAM case removed. + (sub_str): Allow from parameter to be nil, defaulting to zero. + (stdio_line_read, stdio_line_write, stdio_close, stdio_line_stream, + pipe_close, pipe_line_stream, dirent_read, dirent_close, + dirent_stream, stream_get, stream_pushback, stream_put, + stream_close): Functions removed. + (stream_ops dirent_stream_ops, stdio_line_stream_ops, + struct stream_ops, pipe_line_stream_op): Static structs removed. + (lazy_stream_func, lazy_stream_cons): Retargetted to new streams. + (cobj_print_op): Likewise. + (init): Disables and restores GC, instead of doing it in obj_init. + (obj_print): Retargetted to new streams. + (obj_pprint): New function. + (obj_init): Does not manipulate gc_state any more, moved to init. + Call to stream_init added. + (d, snarf): Retargetted to new streams. + (snarf_line): Removed, now appears in stream.c, retargetted + to new streams. + * lib.h (enum type): STREAM removed. + (struct stream, struct stream_ops): Removed. + (struct cobj_ops): Retargetted to new streams. + (union obj): sm member removed. + (push, pop, obj_pprint): Declared. + (stdio_line_stream, pipe_line_stream, dirent_stream, stream_get, + stream_pushback, stream_put, stream_close, snarf_line): Removed. + (cobj_print_op, dump, snarf): Modified. + * match.c (dump_bindings, complex_snarf): Retargetted to new streams. + * txr.c (main): format used to dump bindings and specs in verbose mode. + +2009-10-07 Kaz Kylheku <kkylheku@gmail.com> + + Implemented quasi-literals: string literals which may + contain variables to be interpolated. + + Also, took care of a hygiene problem with respect to some + parser-generated forms, which must be invisible to the user. + + * Makefile (LEX_DB_FLAGS): New variable; helpful + in generating a lexical analyzer with debug tracing. + * parser.l (nesting, closechar): Static variables removed. + (char_esc): Add \` escape for quasi-literals. + (stack): New %option, to generate a scanner which has + a start condition stack. + (QSILIT): New start condition. + (grammar): Refactored to use start condition stacks. + Quasi-literal lexical analysis added. + * parser.y (lit_char_helper): New function, for factoring out + some common logic between string literals and quasi literals. + (quasilit, quasi_item, quasi_items): New grammar symbols and + production rules. + (strlit): Rule shortened with new helper function. + Bugfix: error case assigns nil to $$. + (chrlist): Bugfix: error case assigns nil to $$. + (LITCHAR): Added to %prec table to fix shift-reduce problem. + (expr): Production now can generate a quasilit. + * lib.c (quasi): New symbol global. + (obj_init): Intern quasi as "$quasi", so the user can + make a function called quasi. Also, var and regex are now interned + with the names "$var" and "$regex" for the same reason. + * lib.h (quasi): Declared. + * match.c (eval_form): Rewritten with recursive processing + to handle deeply embedded variables, as well as quasi-strings. + (subst_vars): Handles quasi-strings. + (match_files): Function calls now use eval_form for function + argument evaluation, except of course in the special case that if an + argument is a symbol, it may be unbound. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): No error message for merging to + a symbol which is already bound; the existing behavior + is to destructively update the binding, which is useful, + and so the error is pointless. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + Introduce local as synonym to forget. It does exactly the + same thing; a previous binding is forgotten. This spelling + is nicer for functions. + * lib.h (local): Declared. + * lib.c (local): Defined. + (obj_init): New symbol interned. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + Bugfix: function parameter reconciliation (after function call + completes) must only consider the unbound parameters. + Otherwise false mismatches result if the function destructively + manipulated some bindings of bound parameters. + E.g. @(define foo (a)) is called as @(foo "bar") and internally + it rebinds bound parameter a to "baz". This situation is + not a mismatch. The rebinding is thrown away. + + * match.c (match_files): When processing a function call, + keep an alist which associates arguments and unbound parameters. + Then, after the function call, process the alist, rather + than the full parameter list. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + Semantics change: collect fails if it does not collect + anything. Non-failing behavior can be obtained by + wrapping with @(maybe) (but no such workaround for coll yet). + + * match.c (match_line): Return nil if coll collected nothing. + (match_files): Return nil if collect collected nothing. + + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + Bugfix: nil must be on the list of interned symbols. + + * lib.c (sym_name): Function removed. This was like + symbol_name but did not accept nil. + (intern): Use symbol_name instead of sym_name, allowing + nil to be on the list of interned symbols. + (obj_init): Add nil to interned_syms list. + (nil_string): Changed from "NIL" to "nil". + * match.c (dest_bind): Treat nil as a value, not a symbol. + (match_files): Treat nil as a value when it's + a function argument. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + * gc.c (more): Bugfix: free_tail was incorectly calculated, + thereby destroying the validity of the FIFO recycling algorithm + used when GC debugging is enabled. This showed up as mysterious + assertions and crashes. + (mark_obj): Do not abort if a free object is marked. + (mark_mem_region): Renamed bottom and top variables to low + and high. The naming was confusing inverted relative + to that in the caller. + (sweep): Abort if somehow a block is free and marked reachable. + +2009-10-06 Kaz Kylheku <kkylheku@gmail.com> + + * match.c (match_files): Fixed nonexitent symbol warning for merge + directive (complained about wrong symbol). + +2009-10-05 Kaz Kylheku <kkylheku@gmail.com> + + Refactoring matching code. + + * lib.h (cobj_ops): New function pointer, mark. + * gc.c (mark_obj): For a COBJ type, call the mark function + if the pointer is non-null. + (gc_mark): New public function, wrapper that calls + the private mark_obj. Implementations of mark for COBJ + objects will need to call this. + * gc.h (mark_obj): Declared. + * regex.c (regex_obj_ops): Explicitly initialize mark function pointer + to null. + +2009-10-05 Kaz Kylheku <kkylheku@gmail.com> + + Code restructuring. + + * Makefile (match.o): New object file. + (depend): New rule for generating dep.mk, using txr. + (lib.o, lex.yy.o, regex.o, y.tab.o unwind.o, txr.o, match.o, gc.o): + Dependency rules removed. + * dep.mk: New make include file; captures dependencies. Generated + by new depend rule in Makefile, using txr. + * depend.txr: Txr query to generate dependencies. + * extract.y: File renamed to parser.y + (output_produced): Variable removed, + moved into new file match.c. + (dump_shell_string, dump_shell_string, dump_var, dump_bindings, depth, + weird_merge, map_leaf_lists, dest_bind, eval_form, match_line, + format_field, subs_vars, complex_open, complex_open_failed, + complex_close, complex_snarf, robust_length, bind_car, bind_cdr, + extract_vars, extract_bindings, do_output_line, do_output, + match_files, extract): Functions removed, added to match.c. + (struct fpip): Definition removed, added to match.c + (<stdlib.h>, <string.h>, <ctype.h>, <errno.h>, <setjmp.h>, + "gc.h", "unwind.h"): Unneeded headers removed. + * match.c: New file. + * extract.l: Renamed to parser.l. + * extract.h: Renamed to parser.h. + (opt_loglevel, opt_nobindings, opt_arraydims, version, progname): + Declarations moved to txr.h. + (extract): Dclaration moved to match.h. + * txr.h, match.h: New headers. + * gc.h (opt_gc_debug): Moved to txr.h. + +2009-10-03 Kaz Kylheku <kkylheku@gmail.com> + + Version 014 + + New cases directive. + + New define directive: user-defined dynamically scoped functions. + + String literals in bind and function calls. + + EOF in the middle of a line handled properly. + + * extract.l (version): Bump to 014. + * txr.1: Bump version to 014. + + +2009-10-02 Kaz Kylheku <kkylheku@gmail.com> + + New cases directive. + + * extract.l (yybadtoken): Add case for CASES. + (grammar): Tokenize cases directive. + * extract.y (CASES): New token kind. + (cases_clause): New grammar symbol. + (grammar): Implement new grammar cases. + (match_files): Implement semantics for cases. + * lib.c (cases): New global. + (obj_init): Intern cases symbol. + * lib.h (cases): Declared. + * txr.1: Documented. + +2009-10-02 Kaz Kylheku <kkylheku@gmail.com> + + Support for string and character literals. + + * extract.l (char_esc): Support \' and \" escapes. + (STRLIT, CHRLIT): New flex start conditions. + (grammar): New rules for tokenizing string literals. + * extract.y (LITCHAR): New token kind. + (strlit, chrlit, litchars): New grammar symbols. + (grammar): Implement string literal parsing. + (dump_var): Support character objects, treating + them as one-character strings. + (eval_form): New function. + (match_files): In bind directive, allow the right + hand side to be an arbitrary object. + * lib.c (mkustring, init_str): New functions. + (cat_str): Allow characters in the mix, treating + them as one-character strings. + * lib.h (mkustring, init_str): Declared. + (chrp, chr_str, chr_str_set): New function. + * txr.1: Documented. + +2009-10-02 Kaz Kylheku <kkylheku@gmail.com> + + Support for query-defined functions. + + * extract.l (yybadtoken): New DEFINE case. + (NESTED): New flex start condition. This allows for different lexing + rules in nested lists, so even though for instance @(collect) is a + special token @((collect)) isn't. + (grammar): Refactored with NESTED. + Tokenize define directive. + * extract.y (define_transform): New function. + (DEFINE): New token kind. + (define_clause): New grammar symbol. + (match_files): Implement define semantics, and function calls. + * lib.c (define): New global. + * lib.h (define): Declared. + (proper_listp, alist_remove1, copy_cons, copy_alist): New functions. + (obj_init): Intern define symbol. + (init): Call new function uw_init. + * unwind.c (toplevel_env): New static structure. + (uw_unwind_to_exit_point): Support new UW_ENV frame type. + (uw_init, uw_find_env, uw_push_env, uw_get_func, uw_set_func): New + functions. + * unwind.h (UW_ENV): New enumeration member in uw_frtype. + (uw_dynamic_env): New struct. + (uw_block_begin, uw_block_end): Renamed some variables. + (uw_env_begin, uw_env_end): New macros. + * txr.1: Documented. + +2009-10-02 Kaz Kylheku <kkylheku@gmail.com> + + Misc. bugfixes and improvements. + + * extract.l (grammar): Newline in a directive + no longer an error. Why not allow it. + * extract.y (grammar): Productions for catching empty bodies in some + constructs now end with END newl, rather + than just END, so parsing can continue sanely. + (match_lines): In diagnostics, don't say "ignored" about material which + causes an error that fails the query! + * lib.c (mkstring): Initialize length since we know it! + (c_str): Take a symbol as an arg, so we don't have + to keep writing c_str(symbol_name(sym)). + (obj_print): Use isprint rather than isctrl to decide + whether to print a character as an escape. + (snarf_line): Properly handle EOF in the middle of line. + +2009-09-29 Kaz Kylheku <kkylheku@gmail.com> + + Version 013 + + Some minor garbage collection issues fixed. + + Infinite looping bug fixed. + + New @(trailer) directive. + + * extract.y (match_files): Implemented trailer directive. + * extract.l (version): Bump to 013. + * lib.h (trailer): Declaration added. + * lib.c (trailer): External definition added. + (obj_init): Initializer trailer with interned symbol. + * txr.1: Documented @(trailer) and bumped version to 013. + +2009-09-29 Kaz Kylheku <kkylheku@gmail.com> + + Looping bug fixed. Certain directives could cause an infinite + loop if the query has run out of data. + + * extract.y (match_files): The semantics of the first_file_parsed + argument changes a little bit. Previously, if nil was passed, + a new lazy stream would be opened for the first file. + But this is ambiguous because nil also means empty list; + sometimes when we recurse into match_files, the data has ran + out and this argument is thus nil. Now, that argument must be + the symbol t in order to mean ``open the first file''. + If the argument is nil, it unambiously means ``we are at the end of the + current file; don't open anything''. + (extract): The initial call to match_files now passes + the symbol t for the first_file_parsed argument. + +2009-09-29 Kaz Kylheku <kkylheku@gmail.com> + + Fixing some gc issues. The test cases were found to bomb + with an assertion when run with --gc-debug enabled, + due to a garbage-collected object still being used. + This was due to the way the main function was structured. + + Also, the stack ``top'' terminology in the gc was stupidly wrong. Leaf + function frames are at the stack top, and main is near the bottom. + I was thinking of the ``top caller''. + + * Makefile (TXR_DBG_OPTS): New variable. + Tests are now run with --gc-debug, which makes them slower, + but has much greater chance of trapping gc problems. + * extract.l (main): Two variables are now used for determining the + stack bottom. We don't know in which order the compiler + places local variables into a stack frame. (This is a separate + question from that of the direction of stack growth). + The call to the init function is now done right away. + The argument processing section of main does some processing + with GC objects, but the init function was being called afterward, + before the list of interned symbols is protected from garbage + collection! So with --gc-debug turned on, parts of the interned + symbol list were being garbage collected (since the variable + has not yet been added to the set of root pointers, which is + done in the init function). Also, the use of an unknown --long-option + is diagnosed properly now. + * gc.c (gc_stack_top): Renamed to gc_stack_bottom, and converted from + extern to static. + (mark): Follows rename of gc_stack_top to gc_stack_bottom. + (sweep): Eliminated the freed variable for counting freed objects, + and the associated debug message, which was not useful. + Commented why the free list is managed differently when dbg + is turned on. + (gc_init): New function. + * gc.h (gc_stack_top): Declaration removed. + (gc_init): Declaration added. + * lib.c (min): New macro. + (init): Takes two additional arguments which are used to + determine the stack bottom. The function first determiens + whether the stack grows up or down. Then it takes the + greater or smaller of the two potential stack top pointers, + based on that. The result is passed go gc_init. + * lib.h (init): Declaration updated. + +2009-09-28 Kaz Kylheku <kkylheku@gmail.com> + + Version 012 + + Semantics change of @(until) in @(collect) and @(coll). + + Minor fixes. + + * extract.y (match_line, match_files): The until clauses + continue to be processed after the main clauses of the collect + or coll (to see the bindings), but are processed before + the collection occurs, so that the until will veto + the bindings of the last iteration. Moreover, the data + positions stays where it is when this happens, and no + arrangement is made to match the until material again. + + * txr.1: Tried to document the change. + + +2009-09-27 Kaz Kylheku <kkylheku@gmail.com> + + * txr.1: following proofread, fixed various escaping problems and + instances of missing text. + +2009-09-26 Kaz Kylheku <kkylheku@gmail.com> + + * lib.c (equal): Bugfixes: wrong fallthrough of FUN case. + VEC case must return nil, not break. + +2009-09-26 Kaz Kylheku <kkylheku@gmail.com> + + Preparation for some sorting support. + + * extract.y (merge): Renamed to weird_merge. + (map_leaf_lists): New functino. + (match_file): Follow weird_merge rename. + * lib.c (all_satisfy, none_satisfy, string_lt, do_bind2other, + bind2other, merge, do_sort, sort): New functions. + * lib.h (all_satsify, none_satisfy, string_lt, + bind2other, sort): Declared. + +2009-09-25 Kaz Kylheku <kkylheku@gmail.com> + + Version 011 + + New @(maybe) clause optionally matches (does not fail if none of + its clauses match anything). + + New blocks feature: allows a query or subquery to be + abruptly terminated by invoking an exit to a named or anonymous + block. @(collect) and @(skip) have implicit anonymous blocks now. + + The @(skip) directive takes a numeric argument now, which limits + how many lines are searched. + + * Makefile, extract.l, extract.y, extract.h, gc.c, gc.h, lib.c, lib.h, + regex.c, regex.h, txr.1, unwind.c, unwind.h: Copyright notice and + license text updated or added, and version bumped up to 011. + * tests/001/query-1.txr, tests/001/query-2.txr, tests/001/query-3.txr, + tests/002/query-1.txr: Assigned to public domain. + +2009-09-25 Kaz Kylheku <kkylheku@gmail.com> + + New features: + - named blocks; + - maybe clause; + - optional iteration bound on skip. + + * extract.y: includes added: "unwind.h", <setjmp.h>. + (MAYBE, OR): New grammar tokens. + (maybe_clause): New nonterminal grammar symbol. + (expr): A NUMBER can be an expression now, so that @(skip 42) + is valid syntax. + (match_files): Support for numeric argument in skip directive + to bound the search to a maximum number of lines. + Anonymous block established around skip. + New directives implemented: maybe, block, accept and fail. + Anonymous block established around collect. + * txr.1: Documentation updated with new features. + * Makefile: new object file unwind.o, and associated rules. + * extract.l (yybadtoken): New cases for MAYBE and OR. + (grammar): Likewise. + * lib.c (block, fail, accept): New symbol variables. + (obj_init): New symbols interned. + * lib.h (block, fail, accept): Declared. + (if2, if3): Macros fixed so test expression is not compared to nil, + but implicitly tested as boolean. + * unwind.c, unwind.h: New source files. + +2009-09-24 Kaz Kylheku <kkylheku@gmail.com> + + Stability fixes. + + * extract.y (match_files): Fixed invalid string("-") to + string(chk_strdup("-")) which caused a freeing of + a non-malloced string at gc finalization time. + * regex.c (nfa_state_shallow_free): New function: does not + free satellite objects, just the structure itself. + (nfa_combine): Use nfa_state_shallow_free instead of nfa_state_free, + because the merged state inherits ownership of objects from the state + being spliced out. + (nfa_state_set): Fix lack of initialization of s.visited member of the + state structure. + +2009-09-24 Kaz Kylheku <kkylheku@gmail.com> + + Version 010 + + A file specs can start with $, which means read a directory. + + Data sources are not into memory at once, but on demand, + which can reduce memory for many queries. + + Regular expressions are now compiled once, when the + query is parsed. + + Character escapes are now supported in regular expressions, + and as a special syntax. + + * extract.l (version): Bumped to 010. + (grammar): 8 and 9 are not octal digits; handle all regex + backslash escaping in lexical grammar. + * extract.y (grammar): Get rid of backslash handling from + regex grammar. Lexer returns a REGCHAR for every escaped + item. In situations where an operator character is implicily + literal, like * in a character class, we use the grammar + to include that alongside REGCHAR. Bugfixes: the character ], when not + closing a class, is not a syntax error but stands for itself; + the character - stands for itself outside of character class; + the | character is literal in a character class. + * txr.1: Updated version. Documented character escapes. + +2009-09-24 Kaz Kylheku <kkylheku@gmail.com> + + Lazy stream list improvement: no extra NIL element caused + by end-of-file. Requires push-back support in streams. + To avoid introducing a new structure member into streams, + we extend the semantics of the label member, and rename + it to label_pushback. + + * lib.c (stdio_line_stream, pipe_line_stream, + dirent_stream): Follow rename of struct stream member; + assert that label is an atom. + (stream_get): Check pushback stack first and get item from there. + (stream_pushback): New function. + (lazy_stream_func): Pull one more item from the stream and + use /that/ to decide whether to continue the lazy stream. + The extra item is pushed back, if valid. + (lazy_stream_cons): Simplified: no hack involving regular cons. Starts + the induction by peeking into the stream. If something is there, it is + pushed back, and a lazy cons is constructed which will fetch it. + (obj_print): Made aware of the pushback, which must be skipped + to get to the terminating label. + * lib.h (struct stream): Member renamed from label to label_pushback. + (stream_pushback): New function declaration. + +2009-09-23 Kaz Kylheku <kkylheku@gmail.com> + + Escape syntax in regexes, and text. The + standard seven character escapes are supported, + namely \a, \b, \t, \n, \v, \f, and \r, + as well as hex and octal escapes, plus + the code \e for ASCII ESC. + + * extract.l (char_esc, num_esc): New functions. + (grammar): New lex cases. + * lib.c (obj_print): Support all character escapes in printing. + Bugfix: backslash printed as two backslashes, not one. + +2009-09-23 Kaz Kylheku <kkylheku@gmail.com> + + * tests/002/query-1.txr: Modified to use $ to scan thread + subdirectories. + * tests/002/query-1.expected: Updated. + +2009-09-23 Kaz Kylheku <kkylheku@gmail.com> + + New COBJ type for wrapping arbitrary C objects into the + Lisp-like framework. Compiled regexes are objects now. + Regexes in a query are now compiled just once. + + * extract.y (grammar): Regexes compiled while parsing. + (match_line): Modify with respect to the abstract syntax + tree change, and the interface changes in the match_regex, + and search_regex functions. + * gc.c (mark_obj, finalize): Handle marking and finalization + of COBJ objects. + * lib.c (typeof, equal, obj_print): Handle COBJ. + (cobj, cobj_print_op): New functions. + * lib.h (type_t): New enum element, COBJ. + (struct cobj, struct subj_ops): New types. + (union obj): New member, co. + (cobj, cobj_print_op): New functions declared. + * regex.c (regex_equal, regex_destroy, regex_compile, regex_nfa): New + functions. + (regex_obj_ops): New static struct. + (search_regex, match_regex): Interface change. Regex arguments + are now compiled regexes. Functions won't handle raw regexes. + * regex.h (regex_compile, regex_nfa): New functions declared. + +2009-09-23 Kaz Kylheku <kkylheku@gmail.com> + + New feature: file specs that start with $ read directories. + Reading from an ``ls'' pipe is too slow. + + Streams and lazy conses implemented. Lazy conses allow us to treat a + file or other kind of stream exactly as if it were a list. + We can use car and cdr, etc. But only the parts of the list + that we actually touch are instantiated on-the-fly by + reading from the underlying stream. + + * extract.l: inclusion of <dirent.h> added. + * extract.l: inclusion of <dirent.h> added. + * extract.y (fpip_closedir): new enumeration in struct fpip, + and fpip_noclose removed. + (complex_open): Check for leading $, use opendir. + (complex_open_failed): New function. + (complex_close): Handle fpip_closedir case. Not closing + stdin and stdout is handled by explicit comparison now. + (complex_snarf): New function, constructs stream of + a suitable type, over object returned from complex_close, + wraps it in a lazy list. + (match_files): Use complex_snarf instead of snarf to get a lazy list. + * gc.c: Handle LCONS and STREAM cases. + * lib.c (stream_t, lcons_t): New variables holding symbols. + (typeof, equal, obj_print): Handle LCONS and STREAM. + (car, cdr, car_l, cdr_l, consp, atom, listp): Rewritten to handle + LCONS. + (chk_strdup, stdio_line_read, stdio_line_write, stdio_close + stdio_line_stream, pipe_close, pipe_line_stream, + dirent_read, dirent_close, dirent_stream, + stream_get, stream_put, stream_close, + make_lazycons, lazy_stream_func, lazy_stream_cons): New functions. + (stdio_line_stream_ops, pipe_line_stream_ops, + dirent_stream_ops): New static structs. + (obj_init): Intern new symbols lstream, lcons, and dir. + * lib.h (type_t): New enum members STREAM and LCONS. + (struct stream, struct stream_ops, struct lazy_cons): New types. + (union obj): New members sm and lc. + (chk_strdup, stdio_line_stream, pipe_line_stream, + dirent_stream, stream_get, stream_put, stream_close, + lazy_stream_cons): New function declarations. + * regex.c: inclusion of <dirent.h> added + +2009-09-23 Kaz Kylheku <kkylheku@gmail.com> + + Version 009 + + User-friendly error messages from parser. + Fixed -q option. + + * extract.l (version): Bumped to 009. + * txr.1: Updated version. + +2009-09-22 Kaz Kylheku <kkylheku@gmail.com> + + * Makefile (LIBLEX): New variable. + Refer to lex library as -lfl, using variable + that can be overridden. + +2009-09-22 Kaz Kylheku <kkylheku@gmail.com> + + * extract.h (yybadtoken): New function declaration. + * extract.l (yybadtoken): New function. + (main): Fixed -q option. + * extract.y (grammar): Lots of new error productions, some + phrase rules refactored, resulting in much more user-friendly + error diagnosis. + * txr.1: -q option semantics clarified. |