TXR 131 2016-01-13 Bugs: - Configuration with --valgrind now builds again. - Invalid lazy string optimization introduced in TXR 118 has been re-worked. This affects the conformance of the @(freeform) directive to documentation. - When a macro declines to perform an expansion by returning the :form, the form was recorded as its own macro ancestor; now, nothing is recorded. - Fixed issues in how regex objects are printed: - Control characters and certain other characters were being dumped literally. - Now certain characters like newline print using the escape codes like \n. - Other control characters, as well as U+7F (ASCII DEL) and characters in the U+DCXX surrogate range are printed as hex codes. - [ and ] in a character class are properly escaped. - Fixed issue in string literals: semicolon character effectively disappearing after control character that is converted to hex escape. TXR 130 2016-01-05 Features: - Interactive listener: - New Ctrl-X Enter command: submit line, but stay in history and advance forward by one line. - I/O: - New record-adapter function creates a "stream adapter" virtual stream object based on an existing stream. - The adapter changes the semantics of get-line to support delimiting of records via a regular expression (which is stored in the adapter), rather than the newline character. - With this, TXR can now process streams delimited in alternative was as if they were line-oriented. - New read-until-match function: extract characters from a stream, accumulated into a string, until a match for a regex occurs in that stream, and discard the match. - OOP: - In defstruct, function slots now initialized before other static slots regardless of order of specifiers. - Library: - New chr-digit and chr-xdigit functions: like chr-isdigit and chr-isxdigit, but returning the digit value instead of the t symbol. Bugs: - Interactive listener: - Fixed recent regression in Tab completion: missing completion on special operators and macros. - Fixed Ctrl-X commands not recognized in a history search. - OOP: - Access to static slots of a lazy struct was triggering instantiation; this is fixed. - Library: - Recent change in chr-isdigit and chr-isxdigit return value semantics has been reverted, due to potentially breaking code which relies on (eql (chr-isdigit #\0) (chr-isdigit #\1)). TXR 129 2015-12-30 Features - TXR pattern language: - @(line) directive now works in horizontal mode - New @(data) and @(name) directives for capturing/matching raw data, and name of data source. - :counter in @(repeat)/@(rep) can now be given a starting value. - Quasiliterals in the pattern language are now evaluated as Lisp quasiliterals. - the compatibility mechanism provides the previous evaluation mode. - The same applies to @(output): variables now follow Lisp rules, similarly to variables in quasiliterals. - @(repeat) and @(rep) now "see" Lisp globals; no need to list these using :vars. - Lisp library: - chr-isdigit and chr-isxdigit now return the integer digit value when the test is true, rather than just t. - Object printing: - Lazy strings are now printed in a reasonable way, not shortened to the # notation. - New special variales for global control over how floating-point objects are rendered: *print-flo-format* and *print-fo-format*. - OOP: - with-slots macro added. - Yet improved location and macro ancestry tracing in unhandled exception diagnostics. Bugs - Vim syntax highlighting files: fixed regression in genvim.txr causing missing identifiers. - Four-year-old bug in @(collect)/@(coll) fixed, causing the @(last)/@(until) clauses to be evaluated with an empty list of variable bindings whenever the main clause fails. - Fixed exception in @(freeform) directive when scanning from an interactive input stream. - Bugfixes in pattern language debug tracing (-v). TXR 128 2015-12-19 Features - Further refinement of function and macro coexistence: - symbol-function, fboundp and fmakunbound operate only on function binding. - new symbol-macro function. - The :whole and :form parameters now accept a destructuring parameter list as their argument, no longer insisting on nothing but a symbol. - OOP: - defstruct now supports a :postinit slot specifier, which allows additional actions to be taken after an object is constructed, and the slots are fully initialized. - make-struct-type has a new argument related to the above change: a function. - Place manipulation macros now propagate ancestry information from the place forms on which they operate to the generate place-accessing operations, as if those operations were macro-expansions of the places. (Which they are not, at least not exactly). The benefit is that errors in those place access operations can consequently be traced to the original place forms, for more meaningful diagnosis. - dot notation like a.b.c now prints back as a.b.c rather than (qref a b c). - Doc: - HTML section numbers are now hyperlinks which jump back to the right place in the table of contents. - On jump-back, the table of contents expands sufficiently to reveal the target entry, if that is not already showing. Bugs - Fixed incorrect behavior of (replace seq val nil), also affecting replace-list, replace-vec and replace-str. The nil was wrongly treated as as a zero argument (acting in complement to a missing denoting the end of seq), instead of an empty . - Fixed broken assignment to [h x] when h is a hash and x is a vector or list. - Fixed dot syntax like a.b not recording source location info into the (qref ...) form when its constituents are atoms, such as symbols. - Fixed neglect to propagate macro ancestry information across non-macro code transformations. - Fixed situations in which the debugger fails to report the source file and line number. - Fixed annoying debugger behavior of stepping through the macro expansion time evaluations of a query by making that behavior subject to a new option (--debug-expansions). Debugger will now stop on the first line of the TXR language query like it used to. TXR 127 2015-12-10 Features - OOP: - defun can define methods now, using a compound naming syntax, as alternative to defmeth. - func-get-name function works on methods and produces compound name. - defmeth returns compound name, and so does defun when defining a method. - improved semantics in static-slot-ensure - Large file support: TXR now uses 64 bit file offsets in functions like seek-stream and truncate on Linux platforms where this is a compile-time switch. - regex-range now returns a range object. - Macros: - Improvements in macro expansion tracing for unhandled exceptions, or those caught in the REPL. - New macro-ancestor function for programmatically retrieving the form from which a given form was derived by macro expansion. - New mboundp and mmakunbound functions for inquiring about and removing a global operator macro binding. - Global operator macros can now coexist with functions of the same name. - Doc: - HTML man page now has a collapsible table of contents, thanks to Javascript embedded in the document. - Sections have been reorganized. - Smattering of corrections. Bugs - Fixed new struct types failing to inherit static slots that were added to ancestor types using implicit or explicit use of ensure-static-slot (such as methods defined outside of defstruct). - Fixes to the anaphoric ifa macro: the condition can be a dwim form, and with special handling. - Fixed incorrectly implemented less comparison for range objects. - Fixed crash in get-line when invoked on a closed file stream. - Fixed ftruncate not being detected and used on Solaris. TXR 126 2015-11-29 Features - New functions window-map and window-mappend for filtering sequences with a sliding window context surrounding the current element. - New macro define-accessor for easily extending functions to be accessors (supporting assignment). - Diagnostics for unhandled exceptions, and those caught in the REPL, now include a detail trace of the macro-expansion which created the form in which they originate. - Increased stack size on Windows (MinGW and Cygwin). Bugs - Regression in super function: when called with a struct instance, was returning its type rather than supertype. - Incorrect behavior (failure) in split and partition functions when index argument is a function, and returns nil. - Fixed regression in transpose function, causing it to be destructive. - Fixed sethash: it was returning a boolean rather than the stored value, as documented. - Improved clarity of error messages which identify destructuring mismatches in macro calls and macro-like binding constructs. - Fixed poorly informative diagnostic when defmeth is used on a nonexistent struct type. TXR 125 2015-11-20 Features - OOP improvements: - New uslot function and usl macro for functional indirection on struct slots. - New functors feature: define a lambda method for struct types which lets structs be invokd as function. - Lazy structs feature: - New lnew macro (and make-lazy-struct function) for lazily instantiating a new structure. - Self-referential, and mutually referential structures can be instantiated in one step using the existing mlet and lnew. - New Equality Substitution feature: - An equal method defined on a struct type allows objects to to be compared with equal or less, and to be hashed with hash-equal (and thus :equal-based hash tables). - The method simply returns an object which is used in place of the struct for the purpose of the comparison or hash. - TXR Extraction Language: - @(rep) now available as shorthand for @(coll :vars nil) - In @(next source), source can be a Lisp expression without leading @. - :to_html and :from_html filters now called :tohml and :fromhtml; old names will continue to work. - :tohtml filter now also translates ' (ASCII 39 apostrophe) to '. - New :tohtml* filter, leaves quotes alone. - New butlast function. - New html-encode* function. - The last function is generic now, applying to strings and vectors. - New TXR_COMPAT environment variable, alternative to -C option. Bugs - Fixed missing autoload registration for meth, umeth and defmeth symbols. - Fixed spurious retention issues in various library functions like mapdo, preventing them from processing lazy lists in constant memory. - Fixed type hole in the equal function, which treated two COBJ objects of different types using a comparison function which assumes they are both of the same type. - Fixed buggy less function, which was comparing and ranking objects that should not be comparable according to the documentation. TXR 124 2015-11-14 Features - New iread function for reading an object from a stream in an interactive-friendly way. - Median-of-three pivot selection added to quicksort to avoid degenerate cases. - Scoping rule for implicit nil block in the for operator has changed: the block now surrounds the variable initialization forms. - New block* operator and functions return* and sys:abscond*, which allow blocks to be defined and referenced using symbols computed at run-time. - New function group-reduce for flexible decimation of a sequence to a hash table. - The (rcons x y) syntax is printed as x..y now. - nreverse handles strings and vectors now, and reverses them in place. - reverse handles vectors and strings in more efficient way, and can reverse lazy strings. - Functions revappend and nreconc added. - New *print-flo-precision* special variable for controlling precision applied when printing floating point objects. - New *print-base* special variable for controlling base in which integer objects are printed. (Only binary, octal, decimal and hex are supported.) Bugs - Fixed object printer throwing exceptions when meeting unexpected syntax in forms that have special printed notation like quote and whatnot. The correct behavior is to avoid the notation and print the raw syntax. - Fixed ~o format specifier, broken with respect to bignum operands in TXR 120, when ~b was introduced. TXR 123 2015-11-06 Bugs - Fixed regression introduced in TXR 115, in the search_regex function and functionality built on it. TXR 122 2015-11-05 Features - New type: range. - range is similar to a cons cell, but is an atom. - has two fields called from and to. - the dotdot syntax a..b now produces a range, not a cons. - printed notation looks like #R(0 10) - related functions: rcons, rangep, from, to. - sys:capture-cont API altered to call/cc style: passes continuation to specified function when capturing, returns value when resuming. - New interface to delimited continuations: suspend macro. - TXR pattern variables now appear dynamic to embedded Lisp expressions, rather than lexical. The scoping rule is documented. - The yield and yield-from operators can omit the argument form, in which case they retrieve the value passed in from the resume function without yielding an item. - Architecture support: Power PC 64. TXR 121 2015-10-30 Features - TXR Lisp now has delimited continuations. - easy to use via obtain/yield macro operators. - New function truncate-stream for truncating files. - Reduced stack usage on glibc systems by shrinking unwind frames, achieved by replacing the 128 byte wide sigset_t type with a much smaller type. - The way place macros are expanded has improved, making them more useful. - Macro parameter lists have a :form keyword now for capturing the whole form. Bugs - Implemented missing second argument of zap macro. - Assignment to *listener-hist-len* in ~/.txr_profile was not correctly taking effect prior to loading history, causing truncation to 100 lines. - Fixed typo causing with-hash-iter macro not to autoload. - Fixed crash in hash-next. TXR 120 2015-10-18 Features - Structs: - Finalizers (specified by :fini) are now registered early in the object initialization (at a given inheritance level). This allows finalizers to clean up abandoned initializations. - If initialization of a struct is abandoned, the finalizers are invoked immediately, as part of unwinding. - New defmeth macro for defining methods outside of a defstruct. - New static-slot-ensure function for adding a new static slot to a struct (and its inheritance descendants). - static-slot and static-slot-set can take struct type name in place of struct type. - New functions hash-begin and hash-next, for iterating hash table, and with-hash-iter macro. - Listener: - TXR now launches listener if no arguments are given. - Exceptions: - New way of handling exceptions without unwinding: handler-bind and handle operators. - Introspection over exception frames: get-frames and find-frame functions can inquire about frames, and invoke-catch can transfer control to a chosen catch frame. - New ~b conversion specifier in format function for printing integers in binary. - The typecase macro has been added, similar to Common Lisp's. - Some functions have been renamed for consistency, but remain available under old names: list-vector -> list-vec, vector-list -> vec->list, chr-num -> chr-int, num-chr -> int-chr. - The sh function isn't implemented using the C library function system any more. Bugs - Fixed use of nonexistent functions in error handling cases of defstruct macro. - Fixed bug in the overflow check in the chk_grow_vec function, used by string-extend, the get-line function of many stream types, and string output streams. - Fixed unterminated format argument list issue affecting interactive listener. (Reproduced on Mac OS/X). TXR 119 2015-10-10 Features - Interactive listener: - New Ctrl-X Ctrl-R to insert entire previous line. - Word/Atom from previous line works relative to recalled history line. - New Ctrl-X Tab for substring-searching completion. - Empty loops can be interrupted by signals: (for () () ()) is now interruptible at the interactive prompt. - New umethod and umeth function for indirecting on a method in a way that is detached from an object instance. - Boa construction of structures can have optional arguments now. - copy and copy-str now copy lazy strings as lazy strings, rather than forcing to regular string. - New functions for sequence manipulation: take, take-while, take-until, drop, drop-while, drop-until. - New function ginterate for lazy generation which includes the terminating item. - New function expand-right for lazily generating a sequence in a roughly opposite way to a reduce-right reduction. - New function call-finalizers for explicitly invoking and clearing the registered GC finalizers for an object. - New functions clear-struct and reset-struct for clearing all slots of a structure to nil, or reinitializing a struct to default values, respective. - New function replace-struct for changing and slot contents and type of a struct to match an example object. - Missing promisep function added to test whether an object is a promise created by the delay macro. - New defex macro, to define exception subtype relationships out of TXR Lisp. (Previously available only in the TXR pattern language). - Also useful surrounding functions: exception-subtype-p, register-exception-subtype. - New macros for working with streams in lexical scopes: with-out-string-stream, with-out-strlist-stream with-in-string-stream, with-in-string-byte-stream, with-stream. - New with-objects macro for scoped finalization of objects, somewhat like destruction in C++. - Vim syntax highlighting file sets "lispwords" option for more accurate indentation of TXR Lisp. Bugs - Fixed symbol-function throwing error on builtin macros. - macro-time is now special-operator-p. - Named block around bodies of methods and functions declared in defstruct. - slot-p renamed to slotp (old name supported in compatibility mode). - Bug in structure initialization: boa arguments were applied first, before other initialiations, rather than last. - The conses function works on vectors and strings. TXR 118 2015-10-01 Features - Interactive listener: - Parentheses matching jump (Ctrl-]) now finds the closest parenthesis, bracket or brace if the cursor isn't on one already. - Tab completion recognizes .( and (( context in front of symbol being completed, and doesn't restrict to function names. - Parenthesis-matching "jump" is now performed on backspace also, if cursor lands on or next to a parenthesis. - Ctrl-A and Ctrl-E (or Home and End) keys have now have a useful behavior in multi-line mode, allowing a jump to the start/end of the current line as well as start/end of the edit buffer. - Optimizations in regular expressions: - In particular, prefix matches involving the complement operator now indicate termination rather than wastefully consuming the remainder of the string. - Algebraic reductions performed on regexes. In some cases, these reduce an expression which contains exotic operators such that it can be compiled to NFA. - Structures: - Structures can now have static slots: slots shared among all instances of the same type (but not other types, not even derived types). - Static slots are now the preferred the basis for implementing functions and methods. - The defstruct syntax has a richer syntax which lets static slots be expressed, as well as methods. - Construction and finalization code can be specified in defstruct. - Many new functions related to structures. - Optimization in lazy strings: forcing lazy strings is more efficient. - String input streams do not force lazy strings now. - The "matches nothing" regular expression represented in the abstract syntax by the object t now handled by regex printer, and rendered as []. Bugs: - Fixed a silly delete-previous-word behavior in multi-line mode: not treating the line-break as a whitespace character that separates words. - Fixed various problems in screen update and cursor positioning behavior in interactive listener, affecting multi-line mode. - Fixed two bugs in the ifa/conda macros. - The return and return-from operators do not abort TXR when a label is not found, but properly throw an exception. - Garbage collection: if GC is triggered in response to a malloc delta, a full GC is now performed. This prevents runaway memory allocation behavior in code which manipulates large arrays, strings or integers. - Fixed strangely compounded error message issued in the situation that unbound variables occur in the TXR pattern language. - Various "internal error" situations in the regex module now throw exceptions. Consequently, TXR will not die if a bad abstract syntax tree is passed to the regex compiler. - Regex printer was throwing an exception when printing a character class containing \w, \s or \d. TXR 117 2015-09-23 Features - Interactive listener improvements: - Visual selection of text, with copy and paste: Ctrl-S, Ctrl-Q, Ctrl-X Ctrl-Q, Ctr-^. - Undo feature with separate undo per history line. - Security checks on ~/.txr_profile permissions. - Location reporting when exceptions caught. - Syntax errors pinpointed to line in multi-line entry. - Ctrl-X Ctrl-V "super verbatim" mode. - Temp file used by Ctrl-X Ctrl-E has .tl suffix now for correct syntax highlighting. - Ctrl-X Ctrl-W: insert word from previous line. - Ctrl-X Ctrl-A: insert atom from previous line - Delete to beginning of line or to end of line now limited to current physical line in multi-line mode. - Ctrl-X Ctrl-K to delete current physical line. - Visual Parenthesis matching works forward now as well as reverse. - Ctrl-] jump to matching parenthesis. - Unix group database functions added: getgrgid, getgrnam, getgrent. - New path-private-to-me-p function to detect whether a file is writable to users other than the owner. - New flatcar and flatcar* functions. Variants of flatten which handle improper lists and preserve nil atoms. Bugs - Correct tab alignment in multi-line. - Fixed screen untidiness on Ctrl-Z and Ctrl-C in multi-line mode. - sub function and [] indexing were not handling lazy string objects. - sub-str wasn't handling defaulted optional arguments correctly for lazy strings. TXR 116 2015-09-17 Features: * Interactive listener improvements: * Ctrl-R reverse search has been implemented. * Tab completion now works anywhere in a line, not only at the very end. * Parenthesis matching: cursor jumps briefly to matching bracket, brace or parenthesis. * Multi-line mode, with proper history support for lines containing embedded line breaks. * Ctrl-C cleanly cancels completion, and navigation keys smoothly transition from completion to edit mode. * Ctrl-X Ctrl-E edits line in external editor. * Ctrl-Z suspend and Ctrl-L clear work in completion mode. * New :read command for parsing pasted material from terminal. * Dynamic resizing of the terminal is now obeyed (SIGWINCH signal). * A ~/.txr_profile file is now read on interactive start-up, and Lisp forms are evaluated from it. * multi-line mode and history length are configurable via special variables. * Removed arbitrary limit on regular expression size (number of states). * Small regexes are processed with a smaller memory footprint due to exact allocation of certain arrays and stack allocation is used in some cases instead of malloc. Bugs: * Fixed memory leak of blank lines in listener. * Comment lines entered into the listener do not produce a syntax error and are entered into history. * Fixed memory leak in NFA regular expressions. * Fixed a counter wraparound bug in NFA regular expressions, resulting in incorrect operation after 4.2 billion state transitions (2**32 characters fed to the same regular expression, whether in a single job or across multiple jobs.) * Addressed some minor issues in unwinding and signal handling. TXR 115 2015-09-11 Features: * Sixth anniversary edition! * New REPL on Unix platforms: an interactive listener with line editing, history recall, completion and exception/interrupt trapping. This is started with "txr -i". * Improvements in error reporting. * New function, raise, for raising a signal, eliminating the (kill (getpid) sig) workaround. * New functions subtypep and typep for testing subtype relationships between types, and between an object and a type. * New abstract types usable in conjunction with subtypep and typep. For instance, integer and number are types. A fixnum is a subtype of integer, which is a subtype of number. Type hiearchy documented in detail. * The printed representation of function objects is much more informative than before. * The variable *args* is now not only bound when evaluating Lisp expressions from the command line (with -e and related options) but it is also modifiable. Command line expressions can rewrite the remaining arguments by updating *args* and these arguments instantly take effect. * Some section restructuring in the documentation. * New functions for working with package objects: package-alist, package-name, package-symbols. Bugs: - Fixed serious problem in implementation of signal handling: the function to manipulate the signal mask in a cached way wasn't actually calling into the OS to set the real signal mask. - Fixed another serious problem in signal handling: a Lisp function throwing an exception out of a signal handler would cause an imbalance in an internal signal reentry counter, causing later signal handling to stop working properly. - Fixed an assertion occurring in the Flex-generated scanner if the parsing of a stream is aborted by an exception, and then the parser is called again. - Fixed regression dating back to TXR 105: the TXR pattern language failing to use a "real-time" stream for standard input, when it is a TTY, thus breaking the specials upport for pattern matching into interactive input. - Removed an accidentally committed debug print, producing output in situations when a function is called with the wrong number of arguments. - Fixed (eval 'sym) reporting the unbound variable sym against the wrong source code location. - Fixed random state objects being of type *random-state* rather than of random-state. The earmuffed symbol was never intended. - The accessors fun, symbol-function and symbol-value were completely broken for use as syntactic places. - Fixed multiple evaluation of the slot argument when slot used as syntactic place accessor: (inc (slot x y)) now evaluates y only once, just like x. - Fixed another gaping bug in the finalization hook mechanism: objects being promoted into the tenured generation had their finalizers called while still remaining reachable. - Fixed the self-path variable not being bound when evaluating Lisp code from a .tl file or command line arguments like -e. - Fixed *args* not being bound when evaluating Lisp expressions from the command line with -e and related options. - Fixed missing :tointeger filter. It was actually registered under the misspelled keyword symbol :toinger, and never existed as :tointeger. It has actually been renamed to :toint. TXR 114 2015-09-03 Features: ! Note: TXR code base converted to new way of handling variadic function (variable-length struct args, defined on stack). * New function shuffle: Fisher-Yates/Knuth on lists and vectors. * Numerous documentation improvements. * Improved diagnostic in situations when an error exception is thrown at code parse time, pinpointing file and line number. * New feature: structs! TXR now has typed data structures with named slots. Structures can be related by single inheritance, and single-dispatch OOP is easily expressed. * stat, lstat and fstat functions return a stat structure now. * New functions: time-struct-local and time-struct-utc return a structure instead of a list of fields. * Unix password database functions: getpwent, getpwuid, getpwnam. Bugs: * Fixed regression: broken treatment of [hash x..y] place. * Fixed finalize function, which was found to be fundamentally broken. Advertized as a new feature in TXR 101, it now actually works. TXR 113 2015-08-20 Features ! Note: txr builds and works with Berkeley Yacc again. ! Note: txr is starting to make use of the alloca function. * regex-parse function allows unescaped slashes and treats them as verbatim characters, in recognition of slash delimiting being needed only by regex literals. * Better argument number mismatch reporting in certain indirect function call situations. Bugs * Serious quasiquote regression fixed (from TXR 110): function call syntax interpolated into a quasiliteral not working for TXR pattern language quasiliterals, when the function is called with exactly one argument; e.g. @(bind x `abc @(list 1)`). * Serious parser problem fixed affecting Lisp code being read from streams. The wrong kind of weak hash table was used to associate parsers and streams, causing a disruption when garbage collection kicks in, precipitating a false syntax error. * Nested meta-variable references now work in quasiliterals. If a quasiliteral is nested in two layers of op syntax, it can use notation like @@@1 and @@@rest to reach the outer parameters, and so on. * Minor spurious retention issues addressed (garbage collection). * Numerous markup errors fixed in documentation (thanks to Dave Love of Fedora Project). TXR 112 2015-08-14 Features ! Note: the ChangeLog is discontinued. Description of changes, going forward, is maintained in git commit messages only, in a hybrid format which conforms to ChangeLog and git conventions at the same time. ! Note: txr doesn't work correctly if parse.y is built with Byacc. GNU Bison must be used. This will be addressed in the next release, hopefully. - Support for display width concept: identifying Unicode characters which occupy two print positions on tty-like display and printing devices. - Indentation, and field formatting in the format function obey display width, rather than naive string length. - The new display-width function calculates the display width of characters and strings. - The a..b syntax is slightly revised; it can occur outside of a list now, and in the terminating position of a dotted list. It correctly associates, so that a..b..c denotes (cons a (cons b c)). - New function clamp for clamping numeric and other values to a range. - New "qref dot" syntax: a dot flanked by expressions and no whitespace denotes a new syntatic sugar such that a.b.c.d denotes the form (qref a b c d), and a.b.(x y).z denotes (qref a b (x y) z). However, this qref syntax has no assigned meaning yet (coming soon). Bugs - Regression in format function introduced in 111 is addressed: geneating superfluous left padding when formatting non-numeric fields. - Bad consing dot syntax like (a . b c) and (a . b . c) is properly diagnosed instead of silently producing strange (though consistent) results. - Tokens with a package prefix but empty name like abc: are correctly handled now by interning a symbol with the name "" (empty string) in the given package. - Fixed some minor bugs in lexical analysis of floating-point constants: actual behavior was interpreting as valid certain forms that were documented as erroneous. - Fixed stream close bug in catenated streams. TXR 111 2015-08-08 Features - Nested structural objects (list structure, vectors, hashes) are now printed with line breaks and indentation. - Introducing global lexical variables, introduced with defvar and defparml. Global lexicals are not marked special, and so local bindings of the same symbols are ordinary lexicals and not dynamic. - Numerous variables in the library are now global lexical rather than special. - pi and e are now provided as %pi% and %e%, global lexical. The *pi* and *e* specials are retained, and obsolescent. - New hash-revget function for reverse lookup through hash tables. - New function func-get-name for obtaining name from function. - New ~! format directive for controlling hanging indentation. - format conversions now support ^ character for centering within a field. - Output streams now support indentation. This is used by the object printer and by the ~! format directive, and also exposed by a group of new functions like set-indent-mode. - TXR Lisp documentation sections re-arranged for better organization, and other improvements in documentation. - Improvements in matching error exceptions to source code location, and some other diagnostic improvements. - New wrapper functions for Unix lstat and fstat for inquiring about the properties of symbolic links, and open files. - New Unix wrappers for uid/gid functions: getuid, setuid, etc. - New functions for conveniently testing for file existence, type, ownership and permission, and relative age. - Debugger won't step into auto-loaded library code now, unless requested with new --debug-autoload command line option. Bugs - Fixed parsing bug in txr-case macro: throwing error on correct syntax. - Discovered that an object initialization pattern believed to be correct is flawed in the face of generational GC. Fixed instances in hashing and regex module. - Documented special variable *args-full* is now actually implemented. It was implemented as *full-args*, which is retained, but obsolescent. - Operations on an output string stream which overflow throw exception instead of returning nil. - Fixed incorrect handling of negative field widths in format, when specified dynamically via *. TXR 110 2015-07-25 Features - New relational predicate functions hash-subset and hash-proper-subset. - New functions hash-from-pairs and hash-list. - Improvements in quasiliterals: - compound @@ now supported on numbers and symbols, improving utility of quasiliterals in multiply nested op syntax. - \@ escape introduced for encoding literal @ in quasiliterals. - @ notation allowed in brace variables (more of a bugfix). - Optional trailing semicolon punctuator supported on hex and octal characters in string and quasiliterals, like it already is on hex and octal characters in the TXR pattern language. - The op syntax positional arguments @1, @2 ... and @rest are now syntactic places: they can be assigned, etc. - Powerful new binding macros placelet and placelet* for creating lexical aliases for syntactic places with once-evaluation semantics. - The "it" variable in the anaphoric ifa is now a place if the form that it denotes is a place. The place is evaluated exactly once no matter how many times the it variable is accessed or updated in its scope. - Hash-bang support in .tl files. - With only a few sensible exceptions, most unimplemented stream operations now throw exceptions, rather than silently doing nothing. - Improved printed representation of stream objects. - New split function similar to partition, but keeps empty subsequences. - Deletion of a (cdr ...) place now has different semantics, symmetric with deletion of a (car ...) place. - The accessors caar, cadr, cdar, cddr, ... are all provided now, in all combinations down to five levels deep. - The functions second, third, fourth, ... are all accessors now, and are provided up through tenth. They work with vectors and strings. - New feature, "place macros": special macro expansions which apply only to syntactic place forms, making it a cinch to define to define new places in situations when they are expressible as a syntactic transformation to existing places. - The nthcdr function from Common Lisp is introduced, with the added functionality that it behaves like an accessor. - New with-resources macro for block-scoped acquisition and release of multiple resources. - Backslash-newline continuation in word list listerals (WLL-s) and quasiword list literals (QLL-s) now separates words. An unescaped newline is not allowed. Bugs - TXR 97 regression in quasiliterals fixed: `@{(expr) modifier}` ignores modifier whenever (expr) is an expression that undergoes expansion. - Fix in printed representation of a stdio stream. - define-modify-macro was not registered for auto-loading. - Fixed broken unget-char and unget-byte on catenated streams. - Fixed broken lexical analysis of consecutive top-level forms, like 4(x) where one call to the parser consumes 4(, so the next call sees x). - Fixed bug in print and pprint: wrong return value when printing a dwim form. These functions must always return their leftmost argument. TXR 109 2015-06-21 Features - New anaphoric macros ifa and conda. - New function have, synonym of true. Usefully expressive with anaphoric ifa. Bugs - equal-based hashing function is improved so list and vector permutations do not collide to the same value. - cat-str function detects overflow in the calculation of the total string length to allocate, and throws an exception. - Fixed neglected null termination in mkstring function. - Fixed garbage collector crash introduced in TXR 108, when traversing syntax_tree member of the parser_t structure. TXR 108 2015-06-13 Features - POSIX poll function exposed. - Entirely new macro-based framework for syntactic places (forms denoting locations that can be assigned): better featured, extensible than previous hard-coded hacks. - New macro: define-modify-macro. - New pset operator: assign to places in parallel. - rplaca and rplacd handle vectors and strings. - replacement sequence in replace function can now be a vector. - symbol-value retrieves bindings of symbol macros. - boundp returns true for symbol macros. - New functions: makeunbound and fmakunbound. - New concept: deletable places: locations that can not only be accessed, but which can be deleted. Existing del operator which contained special case hacks is now powerful, general macro that can delete any deletable place. - New defparm operator to accompany defvar. - symbol-function, symbol-value and fun forms are assignable places. - Debug instrumentation code made light weight, leading to big improvement in the execution speed of TXR Lisp. - New functions catenated-stream-p and catenated-stream-push. - New function: load for loading TXR Lisp files. TXR Lisp programming can now take place using multiple files containing code that doesn't have to be wrapped with @(do ...). - @(load) and @(include) directives can also load TXR Lisp now. - TXR files which have a .txr suffix can be invoked without the suffix now; this is automatically resolved. - New --lisp command line option makes TXR source files which have no filename suffix be treated as TXR Lisp code rather than TXR Pattern Language. Bugs - Fixed exception being thrown when trying to print (lambda . atom). - Dangling unquotes and splices now receive source location info. - Fixed memory corruption caused by disabling garbage collection for too long. - Better syntax checking in flet, labels, lambda, defun, quote. - Addressed clash between flip operator and flip function by renaming latter to flipargs. - Fixed regression in Verison 106 causing apply calls not to diagnose the too many arguments case, and allow a stack overrun. - chr-str-set checks for literal strings, which cannot be modified. - Fixed runaway recursion in keep-if* and remove-if*. - Fixed mismanagement of dyn_env variable leading to spurious retention of memory in TXR builds configured without debugger support. (Bug exposed by the lighter weight debug support.) - Critical bugfix in handling of hashes having both weak keys and values, leading to memory corruption. TXR 107 2015-04-26 Features - Change in representation of promise objects. The force function now detects recursion and throws an error. - New mlet macro: a step beyond Scheme's letrec. - A trivial optimization in the keywordp function more than doubles the interpretation speed of TXR Lisp. Bugs - Fixed regression in the garbage collector introduced in October 2014, TXR 100. This would cause runaway memory growth, due to the garbage collector creating unnecessary new heaps. TXR 106 2015-04-21 Features - New function tprint; like pprint but prints lists of strings as lines of text. - New stream opening option letter "i" for marking streams interactive. - New command line option -n which makes standard input noninteractive even if it is a TTY. - New command line option -P which is like -p but based on pprint. - New command line option -t for printing Lisp value using tprint. - Stream argument in get-lines is optional, and defaults to *stdin*. - The Unix fork and wait functions are exposed for use. - New function open-fileno for opening a stream on a Unix file descriptor, and fileno for retrieving a file descriptor from a stream which has one. - New functions dup (interface to Unix dup and dup2), chmod, exec and pipe. - New exit* function which calls _exit. - Added getenv, setenv and unsetenv functions for environment manipulation. - New operator zap for overwriting a place with nil, and returning its prior value: good for losing a reference to an object while passing it to a function, and other uses. - New whilet (while + let) macro for variable binding plus iteration. - New iflet and whenlet (if + let, when + let) macros for variable binding plus conditional evaluation. - New while* and until* loops which unconditionally execute one iteration. - New dotimes macro for simple iteration. - New lcons macro for writing expressive, concise code around lazy conses. - Brace variable notation can now contain quotes and quasiquotes. - Quasistrings can contain quasiquotes: `prefix @^(,x ,y) suffix`. - Fixed regression introduced in TXR 81, causing the argument parts of brace variables embedded in quasistrings to disappear. - The backslash in regular expressions can now only be used for recognized escape sequences, or for making regex special characters literal. If it is applied to any other character, it is diagnosed as an error. Bugs - Fixed stack array overrun in apply function. - The int-str function was recognizing the 0x C language convention in base 16 conversions. Also, an error is now thrown on bases outside of the range 2-36. - Eliminated nuisance error message issue from the Makefile about git not being found; git is properly detected in configure. - Streams opened on TTY devices are not automatically marked as interactive. This is done only for the *stdin* stream on startup. - The ~X specifier in format was printing in lower case just like ~x, for bignum arguments. - Addressed spurious retention (garbage-collection-related issue) in function application. - Parser now diagnoses trailing junk in hex, octal and binary literals, like #xAD0Z, #o7778 or #b011003. - Fixed character escaping issues in open-process on Windows. - Fixed bug in error reporting, causing TXR sometimes to report that an error was thrown out of a form, when it wasn't in fact that form. - Fixed bug in the printing of regular expressions: empty expressions like #// or expressins with empty subexpressions like #/a|/ would result in an exception. TXR 105 2015-03-14 Features - New function lexical-lisp1-binding for improved macro-time introspection of environments. - New functions pad and weave. - New regex functions search-regst, match-regst and match-regst-right, to complement search-regex, match-regex and match-regex right. - Improved error reporting, particularly for errors ocurring during the expansion of a macro. - Streams have persistent error state. New functions: get-error, get-error-str and clear-error. - New ignerr macro for trapping exceptions of type error, and converting them to a nil return value. - New function ensure-dir: creates a directory in the filesystem, with all parent directories also created as necessary, and doesn't complain if the directory already exists. Bugs - Bugfixes in append* function: lack of error handling and inappropriate forcing of infinite lists in some circumstances. - Bugfix in match-regex-right: zero length matches must return zero rather than nil. - Fixed memory leak in open-process and run on POSIX platforms. - Fixed memory corruption bug in run function on Windows platform. TXR 104 2015-02-08 Features - Symbol macros are now shadowed by lexically scoped functions, when referenced from the interior of a form which uses Lisp-1 semantics (a [] bracketed form, a form based on the op family of operators, or explicit use of the dwim operator). - New function, abort. - TXR now doesn't terminate abnormally (abort) on unhandled exceptions, but only terminates unsuccessfully. This provides a better experience on the MinGW-based Windows target, where we get an annoying dialog box on abort. - New *uhandled-hook* variable can be used to register a user-defined function which is called when an unhandled exception occurs. - New arithmetic function, trunc-rem. - When the numbered arguments of partial application under the op family of operators are interpolated into a string quasiliteral, modifiers may now be applied. For instance (op prinl `@{1 20} @{2 20}`) denotes a function which prints its two arguments in twenty-character-wide fields separated by a space. - Improvements in Windows installer: reduced broadcast delay in registerting environment variable. Improved messages in this area. - A txr-win.exe can be built now which avoids creating a console window. This is included in the installer. Bugs - Fixed bug in quasiliteral string evaluation in TXR Lisp, in conjunction with op arguments like @1, @2, ... When a value interpolated from one of these arguments looked like a form, it was mistakenly subject to recursive processing and variable substitution as if it were part of the quasiliteral target syntax. - Fixed a serious bug in the op macro and all related operators like do and ap, in the handling of situations with missing argument numbers was broken: such as when @3 is referenced, but not @1 or @2 (which must generate a function of at least three arguments, which ignores its first two). This key feature was discovered to be utterly broken. - Fix in internal representation of exceptions: (throw 'x "foo") is now the same as (throwf x "foo"). TXR 103 2015-02-02 Features - New functions lexical-var-p and lexical-fun-p allow macros to inquire whether symbols are known to the macro-expander as lexical variable or function binding. - get-string now closes the stream after obtaining a string. It has a new optional argument which can defeat the behavior. The -C option can be used to request the old behavior. - New function, constantp. - New functions callf, mapf, dup and flip for more power and flexibility in function-combining expressions. - New function width, for computing the bit width of signed and unsigned integers. - New function sign-extend, for converting an a two's complement bit field of any width into the corresponding integer value. - Windows executable (MinGW and Cygwin) now has an icon and meta-data like ProductVersion and FileVersion. - MinGW-based Windows build is now distributed via an executable installer instead of a ZIP file. Bugs - Fixed regression introduced in 102, in the fix to the make-like function. A one element list being converted to a string was treated as an empty list, leading to empty string. - Fixed small bug in macro argument list processing. The colon symbol was treated as "this argument is missing" for required arguments also, rather than just for optional arguments. - Fixed a bug in the code expander's handling of macro parameter lists. It was not recursing properly over nested patterns occuring in optional arguments, and throwing errors on keywords like :env present in the optional argument parts of macro parameter lists. TXR 102 2015-01-13 Features - merge function becomes generic: works over sequences rather than just lists. - New function glob is provided on platforms which have the Unix C library function of the same name. - New functions plusp and minusp. zerop accepts character argument. Bugs - Fixed incorrectly implemented horizontal flavor of the @(trailer) directive. - Fixed a bug in the make-like function which has consequences for the results of numerous sequence functions. This bug causes functions to return an empty list when the expected result is an empty string. For instance [mapcar identity ""] now returns "", consistent with [mapcar identity "abc"] returning "abc". Previously it returned nil. - Fixed a serious bug affecting the OS X port. The llvm-gcc compiler was discovered not to be consistently aligning wide character string literals on four byte boundaries, even though wchar_t is four bytes wide. As a workaround, a very similar trick is usd as what is used to handle literals on Windows, based on the assumption that we *can* depend on two-byte alignment. The bug leads to a failure of the OS X port of TXR to support the -p and -e options, for instance: txr -p '(+ 2 2)'. TXR 101 2015-01-01 Features - Finalization support in the garbage collector. Use the finalize function to associate a function with an object which is called when that object is found to be unreachable. - iff function can now be called with just one argument. - New function chand. - New macros opip and oand. - The min and max functions are more generic by using the less function instead of the < function. - The less and greater functions are variadic now. - Select accepts a function in place of the index list. - Arguments of where function reversed, with backward compatibility support. - New functions succ, ssucc, sssucc, pred, ppred, pppred for calculating small deltas. - New functions wrap and wrap* for clamping a number into a given range, with modular wrapping semantics. - New function sort-group for sorting and partitioning a sequence in a single operation. - New function in for generic membership testing over any kind of sequence or hash. - copy function extended to copy random state objects. - New function unique, which generalizes uniq. - New function aret: apply version of ret. - Build system improvements: debug and optimized TXR built at the same time, dependencies handled in a cleaner way, tidy make output. Bugs - Fixed incorrect registration of repeat function, lacking support for its optional argument. - Removed dependency on useless Flex library (libfl). - Overflow arithmetic fixes in some array resizing logic. - Incorrect argument defaulting in iff function - Garbage printed in type mismatch diagnostic in sub, ref, refset, replace, update and search_list. - Behavior of splitr-str was undocumented for the case of the empty separator string and was inconsistent. - The set-diff function was not handling nil elements. - The toint function identified itself as tofloat in error messages. - The toint and tofloat conversions conveniently treat a character as a string of length one. - Added missing documentation that make-random-state can take an existing random state and clone it. - Also, make-random-state was incorrectly doing additional scrambling after copying an existing random state. - Simple lazy streams now close a stream when it runs out of data. - Catenated streams close each stream that is popped from the list due to running out of data. - Fixed bug in replace-str and replace-vec: when the argument was an index list that was lazy and infinite, it was being forced, causing an infinite loop and exhaustion of memory. - The partition* function was found to be buggy. - Separate directory build issues fixed. TXR 100 2014-10-22 Features - New functions: chr-isblank and chr-isunisp - New generic less and greater functions, which are also used as default functions in sort, find-max, max-pos, and others. - New directives @(line) and @(chr) for binding or asserting the current line number or character position. - Lisp expressions allowed on left side of @(bind) and @(rebind) now. - New function lcm (lowest common multiple). - The gcd function now takes zero or more arguments instead of exactly two. - New @(include) for parse-time loading of code, useful for loading macros which are needed later in the same file. - Beginning of library external to TXR executable: macros txr-if, txr-when and txr-bind for more convenient access back into the pattern language from TXR Lisp. - New combinator function notf for negating a function. Bugs - Fixed December 2011 regression affecting @(freeform) directive. - Fixed GC-safety bug in abs-path-p function. - Fixed breakage in scanner and parser introduced in August. Caught by C++ compiler. - Fixed inappropriate printed rendering of list objects produced by interpolated TXR Lisp expressions in @(output) blocks, and in the quasiliterals of the pattern language. (TXR Lisp quasiliterals not affected.) Users who depend on the old behavior not wanting to fix their programs can use --compat 99. - Bugfix in the gcd and lognot functions: neglecting to normalize some bignum result to the fixnum type. - Bugfix in @(eof) directive: not matching the end of interactive streams. - Fixed abort due to assertion going off when GC is disabled and the array of new generation objects runs out of space. This could happen during large parses. - Fixed parser stack overflow and inefficiencies when handling large TXR programs. - Bugfix in match_fun causing memory accesses to automatic storage that has been released, as well as an invalid longjmp. - Hash table reorganization is prevented during hash table traversal, so existing items are not skipped or visited twice. TXR 99 2014-10-05 Features - Variables in the pattern language can be bound to regexes, in which case they perform regex matches when matched against input. - Representation for compiled regexes is streamlined. - Regex objects now print in regex notation. Bugs - Fixed August 11 regression affecting 96 through 98: broken matching of unbound variables followed by bound variables. - Fixed one more problem with andf function: (andf) not returning t. TXR 98 2014-09-26 Features - New numeric constants giving pi, e and various limits of floating point representation. - New syntax-error exception type; syntax errors in expressions passed on the cmomand line with -p or -e throw exception how instead of silently producing nil. - New --gc-delta option and gc-set-delta function related to bugfix below. - New -C/--compat option for specifying compatibility. - New functions: partition and partition*. - Revamped documentation: nicely formatted with fonts and section numbering; HTML version uses proportional font now, and has internal hyperlinks. - Exception names like file_error have been renamed to use a dash like file-error. Bugs - Fixed bad memory performance when the program works with large heaped objects, by making GC aware of this memory. - Fixed broken return value semantics of orf and andf combinators. TXR 97 2014-08-29 Bugs - A few GC-related fixes after code review. - Fixed @(load), broken in TXR 94. - Fixed broken @{var mod} syntax Lisp quasiliterals, broken in TXR 96. TXR 96 2014-08-14 Features - Big change in pattern language in the variable binding logic. When unbound variables are followed by directives, then the full right hand context to the end of the line is used to determine the binding. - Internal functions related to tries exposed: trie-lookup-begin, trie-value-at, trie-lookup-feed-char. Bugs - Fixed bug in ret operator: was generating fixed arity functions rather than n-ary as documented. TXR 95 2014-08-07 Bugs - Fix major regression: bad syntax causing crashes in parser. TXR 94 2014-08-05 Features - New functions: giterate, partition-by, uniq. - reduce-left and reduce-right correctly work on strings and vectors. - quasiquote splices can now occur in string quasiliterals, directly as `xyz @,foo`, `xyz @{,foo}`, `xyz @,(foo)` etc. Previously this syntax was rejected, requiring workarounds like (let ((x ,foo)) `xyz @x`). - POSIX functions exposed: getpid, getppid, kill, setitimer, getitimer. - The parser is now re-entrant. This means that macro expanders can safely recurse into the parser, for instance with regex-parse or read. Bugs - @(output) blocks were discovered not to be applying filters to the results of embedded TXR Lisp evaluations. - n-perm-k function was broken for values of k equal to 1. - Fixed situations when exceptions with bogus messages are issued when the arithmetic +, - and * functions are called with wrong type arguments. - Fixed grossly incorrect documentation of the repeat function, to reflect actual syntax and behavior. - Fixed situations in which some signal handlers that don't need to execute on a separate stack might do so. - Put in a guard against async signal nesting: a situation when an asynchronous signal handler performs some blocking operation which enables async signal handling triggering re-entrancy. - Fixed platform-dependent behaviors in random number seeding (make-random-state function). TXR 93 2014-07-22 Features - New list munging functions interpose, mapdo, and juxt. - New macros caseq, caseql and casequal based on Lisp's case construct. - New math function log2 for computing base 2 logarithm. - New function get-lines for obtaining list of lines from a stream. Actually this is not new; it is a nice synonym for lazy-stream-cons. - New functions put-lines and put-strings. - Missing function lconsp added for testing whether an object is a lazy cons. - Documented existing function hash-construct. - search-str now supports a negative starting position, which counts from the tail of the string. - The division function / can now be used with one argument, denoting the reciprocal operation. Bugs - Bindings created using -D command line option are now visible to code executed via -e or -p later in the command line. - Fixed broken @(eol) directive; was behaving as a complete match, so that subsequent directives are ignored. - Obscure bug-fixes in @(do) and @(require), involving code that invokes pattern funcions, and the visibily of bindings. - Fixed hash-construct so it correctly works on an empty vector. - Alternative stack (sigaltstack) is now automatically used for SIGBUS handlers, not only SIGSEGV. TXR 92 2014-07-11 Features - New operators ado and ido. - New command line option --license. - Long overdue flet and labels operators for lexical binding in the function space, added as macros. Bugs - apf and ipf functions fixed. - Uninitialized variable bug fix in select function and in the index list case of replace-list. TXR 91 2014-07-02 Features - @(load `@stdlib/ver`) brings in a definition of a new variable called *lib-version* which lets programs check what version of the library they are using. - New functions where and sel for working with multiple indexes into a sequence. - New function seqp for testing whether an object is a sequence of any type. - New functions pos-max, pos-min, find-max and find-min. - New character constant #\pnul denoting the U+DC00 code (TXR's "pseudo null"). - New function tuples for reorganizing a sequence into tuples. - New functions true and false, expressive in applicative programming: for instance [all true list]. - New functions member and member-if. - New function multi for applying a one-sequence operation to parallel sequences. - New functions tf, nilf, retf, apf, ipf. - New macros ret, ip and ap. - New function iapply for applying improper lists. - New functions transpose and zip. - New functions tok-where and range-regex. - New function bit for testing bits. - [] indexing now allows element extraction by index list, as well as assignment by index list, increasing the "slicing power". Bugs - Fixed reliance on leading slash as indicator of absolute path in @(load). - Fixed breakage in range and range* function (regression introduced in TXR 89). - Fixed bug in dwim operator: Lisp-1 evaluation semantics being propagated into nested forms, contrary to documentation. - Line number of error correctly reported in some situations in which it previously wasn't. - Fixed macros not being expanded inside quasiliteral brace notation. - Fixed broken last function. - Fixed broken bignum bit operations, not handling negative values properly. - Fixed reversed logic of logtest function. - Fixed some memory leaks in MPI bignum integer operations. - Documentation fixes. TXR 90 2014-06-11 Features - TXR has been ported to FreeBSD 9 and Solaris 10. These operating systems are officially supported going forward. TXR's configure and build system has better portability support as a result. - call operator is a function now, like it should be. - Traling backslash in a dynamic regex is rejected as an error. - TXR no longer automatically prints out the bindings; this must be requested explicitly with -B. The -a and -l options now imply -B. The -b option for suppresing the automatic printing of bindings is deprecated. - New --args option, allows multiple logical command line arguments to be encoded in one physical argument which is useful in conjunction with very limited hash-bang (#!) scripting mechanisms on some systems. - New variable "stdlib" which expands to a path in the TXR installation. This will allow inclusion of standard modules. The stdlib variable automatically tracks relocation of the TXR installation. - New function for searching sequences for substrings: search. - New functions nullify and make-like for easily adapting list processing functions to correctly work on vectors and strings. - match-str function can now match from the end of a string, at various negative offsets. - New string comparison functions: str=, str<, str>, str<= and str>=. - string-lt function deprecated, replaced by str<. Bugs - rplaca and rplacd now return the cell rather than assigned value, and the return values are now documented. - Fixed a widespread bug which occurs when empty strings and vectors are treated as lists, due to the fact that they are not equal to the nil object. - Fixed completely broken string-cmp function, and changed its name to cmp-str. TXR 89 2014-05-11 Bugs - The eql function was not handling floating-point values correctly. - The range and range* functions internally use numeric comparison for end test rather than eql, so that they can step through floating-point values, but use an integer boundary, or vice versa. - Fixed broken default argument handling in get-line, get-char and get-byte. - The functions ref, refset, replace, and update (and the DWIM brackets syntax based on them) were not handling lazy strings. - Fixed a bug in the pattern language: when a variable ends up bound to the empty string, a literal empty string object was used, and that was not handled in the matching language. Best illustrated by the test case "echo : | ./txr -c '@a:@a' -". TXR 88 2014-04-04 Features - New TXR Lisp macros: while, unless, until. - New functions empty and last. - copy and length functions take hash tables now. - Word list literals and word list quasiliterals. - Big improvements in efficiency of generational garbage collection. Situations when a backpointer to a fresh object is stored into a mature object now are precisely identified, eliminating GC pressure caused by conservatively wrong guesses. - TXR is now configured to build with Generational GC by default. Bugs - copy and length functions handle lazy strings now. - time-string-utc on Linux now renders %Z strftime code as "UTC" rather than "GMT". Not our issue: this is a workaround for library behavior. - Generational GC fix: a number of functions which mutate state weren't using the proper set macro. - Generational GC fix: the gc_mutated function was not correctly implemented and could cause reachable objects with their GC reachability flag to continue to be set after GC is done. TXR 87 2014-03-22 Features - New convenience functions tofloat, toint: convert string or number to integer or floating-point. - New macro: when. Bugs - int-flo function was failing on negative numbers. - Fixed long-standing syntax issue: directives of the pattern matching language being syntactically recognized in the @(output) clause and in string quasiliterals, interfering with the ability to invoke Lisp forms based on the same symbols. This issue really "hit home" with the recent introduction of the @(if) directive, causing @(if expr then else) to suddenly be inaccessible in @(output) and quasistrings. TXR 86 2014-03-16 Features - Exposed the implementation of tries to user code. - New functions: html-encode, html-decode. - New prof operator and pprof macro for simple profiling: time spent and memory allocated over the evaluation of an expression. - New functions sh and run for running system commands. - If available, sigaltstack is used when the program registers a handler for the SIGSEGV signal, allowing TXR programs to catch stack exhaustion and recover by throwing an exception. - New syntactic sugar in the pattern language: @(if)/@(elif)/@(else)/@(end), which works by transformation to a pattern involving @(cases) and @(require). - The @(empty) directive which plays a role in @(output) blocks has a new meaning in the pattern language for denoting an explicit empty match. Bugs - Fixed severe performance problem with regex on long strings, caused by feeding characters to the regex machine past the point that it is clear it will not accept any more. - Fixed broken horizontal matching under @(freeform) on long lines, due to bugs in the handling of the memory-saving optimization which consumes the prefix of the lazy string as scanning progresses. - Fixed breakage in sub-str over lazy strings. - Fixed possible crash in @(trailer) directive. - Fixed memory leaks on open-process: one that happened on every call, and one on failure of the fork function. - Fixed pipe-close deadlocks occurring in code that opens numerous pipes; solved by setting the close-on-exec flag on the pipe descriptors. - Fixed faulty argument defaulting logic in the functions: iffi, regex-parse, lisp-parse, regex-compile and Windows version of open-process. - Fixed regression in the random number module, causing the random function to ignore the seeded random state passed in as an argument, and rely on the global one. - Fixed a bug in a rarely used form of the backslash line continuation. - Fixed buggy argument quoting in Windows version of open-process. - The gc on/off state is now saved and restored as part of unwinding, so if code that turned off gc throws an exception, gc will be turned back on at the catch site. - Fixed incorrect calculation of malloc upper and lower boundaries, affecting the correctness of generational garbage collection. - In open process, if fdopen fails, kill the process more gently with SIGINT and SIGTERM rather than SIGKILL, and wait on it. - Syntax fix. @^a is not meta applied to the symbol ^a, but rather meta quasiquote a. TXR 85 2014-03-07 Bugs - Fixed a regression related to the *random-state* variable, breaking all pseudo-random number functions. - Fixed broken "txr -v". TXR 84 2014-03-07 Features - TXR Lisp no longer uses one quote for regular quote and quasiquote syntax. The quasiquote character is now ^ and the apostrophe is just regular quote. This change breaks backward compatibility, unfortunately, but the single-quote experiment had to come to an end, now that TXR has a powerful macro facility, with quasiquotes playing a central role. - TXR Lisp exposes the environment manipulation functions make-env, env-vbind and env-fbind. - New function log10 for base 10 logarithms. - New functions open-files and open-files* for opening a big catenated stream which marches through multiple files as if they were one. - New directive, @(assert), for writing certain kind of robust matching code more simply. - put-line can be called with no arguments, in which case it just writes a newline to standard output. - In the format function, if a precision is specified as *, it can now have a leading 0 in front of it, which applies as usual (to the formatting of an integer which falls short of the number of digits required by the precision). Bugs - In the format function, the * character for specifying a variable field width or precision (derived from an integer argument) was broken. - Fixed a regression causing the exception "t is not a cons" on queries that match data, when no input files are specified. - Fixed a regression causing file names not to be tracked in the pattern matching language, and consequently error messages not to give that useful information. - Fixed some bugs related to command line processing, which had recently been rewritten: the element nil was being added to *args* in some situations, the argument - would end up repeated twice, and args was being sometimes manipulated in such a way that (ldiff *full-args* *args*) wouldn't work, due to new conses pushed onto *args* so that it's not a structural suffix of *full-args*. TXR 83 2014-03-02 Features - The qquote macro is now in the regular user namespace, while the quasiquoting read syntax uses a different macro based on private symbols. So when the read syntax is used, the template can contain the symbols qquote, unquote and splice, even though these are the basis for a quasiquote macro. - Diagnosis of attempt to redefine a special operator as a function or macro. Remove the macro when a function of the same name is defined and vice-versa. Bugs - Fixed bugs in the implementation lexical variable shadowing by symbol macros. - Fixed incorrect handling of trailing atoms in quasiquote. - Redesigned the implementaton of special variables, and how they integrate with internal C global variables in the interpreter, in order to fix the broken dynamic overriding of the predefined globals. - The Lisp-1 style evaluation of the dwim operator or square brackets was not resolving over-ridden special variables correctly, using a lookup method that saw only the global bindings. - Fixed missing integration between catenated stream objects and the garbage collector. TXR 82 2014-02-27 Features - Intuitive behavior in various sequence processing in the library: don't try to output a string if the resulting items are not characters, and then fail; make a list. - All operators that were implemented as special code expanding hacks in the TXR Lisp code walker are have been converted to intrinsic macros. Only a few bona-fide special forms are subject to minor code transformations. - New generation macro called "gun". (gun (read-line stream)) produces a lazy list of lines from a stream. gun, which stands for "generate until nil", evaluates the enclosed expression while that expression produces non-nil values, and makes a lazy list out of them. - New function copy, for copying sequences. - Documentation improvements thanks to Roman Mishin. Bugs - Regression fix: error thrown on valid floating point constants. - Doc bugs: documentation was missing for the delay operator and the length function. TXR 81 2014-02-25 Features - Calls to stream opening functions can omit the mode parameter, which defaults to "r". - New functions for accessing environment variables from TXR Lisp, augmenting the @(next :env) method in TXR pattern language. - New variables *args* and *full-args* for accessing the partial and full command line. - New functions macro-form-p, macroexpand and macroexpand-1. - Implemented special variables (dynamic scope). All global variables defined with defvar and all the predefined variables are special, with the symbols being attributed for special binding similarly to Common Lisp. - Implemented the macrolet operator for binding local macros. The macroexpand and macro-form-p functions work correctly with the environment from a macrolet. - Implemented symbol macros: global symbol macros via defsymacro, and local macros via symacrolet. Local symbol maros are shadowed properly by lexical bindings, and are expanded properly in situations where they are used as assignment places by the set operator, and related operators. - Introduced the (. expr) syntax which is equivalent to expr. Handy for writing a function argument list that accepts only trailing parameters as (lambda (. rest) ...) rather than (lambda rest ...). - New functions prinl and pprinl, which are like print and pprint, but add a newline. - Added new Unix filesystem access functions: symlink, link, mkdir, readlink, mknod. Also makedev, minor and major. Bugs - Fixed broken seek-to-end-p boolean parameter in open-tail, a regression in TXR 71. - Fixed a regression in the debugger, also introduced in TXR 71, causing it not to print the data line being processed in a horizontal scanning context. - Fixed premature opening of data sources in the TXR pattern language. Opening files is now delayed until a data consuming directive needs to match data. The previous kludgy approach recognized only the @(next) directive, and only one occurrence of it. - Fixed incorrect square root calculation over bignums. It was not computing the last bit of the result and so could be off by one. - Numerous bugfixes in form expansion, in particular handling of variable bindings and function and macro parameter lists. - Global macro bindings make a symbol fboundp. Also, symbol-function retrieves information about a global macro. - Fixed several bugs in quasiquote, including the handling of `(... . ,form) in quasiquote. - Fixed get-string-from-stream throwing an inappropriate internal error on type mismatch. - Fixed the neglect to expand Lisp forms in the argument lists of the directives of the pattern matching language. - Added the neglected s-ifsock variable, which is part of the Unix filesystem interface, corresponding to S_IFSOCK. - Fixed some instances of source code line number not propagating through code transformations. - Fixed numerous formatting issues in the documentation, and omissions. TXR 80 2014-02-17 Features - TXR now nicely handles null characters in text stream inputs. They are internally encoded to the Unicode value U+DC00, which will map back to a single null byte on output (that being existing behavior). - TXR now has Lisp macros: the defmacro operator is implemented, with destructuring lambda lists and all. - New operators tree-bind and tree-case for pattern binding similar to Common Lisp's destructuring. This piggybacks off the defmacro infrastructure which we now have. - Big improvement in debuggability: the unhandled exception error message now gives source code location information. - New functions pos, pos-if, posq, posql and posqual for finding item positions in sequences. - Predicate function is now optional in the some, all and none functions. - hash-uni and hash-isec functions take a join-func argument which lets you specify how elements from two tables are merged. - new hash table functions inhash and hash-update-1. - two hashes can now be tested for deep equality using equal. Bugs - Removed bogus optimization from hash table implementation. - Syntactic fix: input like 1.0a no longer parses as a floating-point number followed by a symbol, but is diagnosed as an error. Syntax like 123a continues to work as before: it denotes one symbol, not 123 followed by a. - Bugfix in type check for "cobj" class objects that would crash if given non-heaped values like fixnum integers. - Corrected problems in the code walking and expansion of lambda and defun forms. - Fixed failure to propagate line number info through the abstract syntax of string quasiliterals. - Doc bugs: missing descriptions of gethash and gensym. TXR 79 2014-02-11 Features - New functions comb, perm, rcomp and rperm for calculating repeating and non-repeating combinations and permuations of sequences (lists, vectors and strings). Hashes are supported by comb function. Bugs - Hardening of code for handling excessively large vectors. - Bugfix in quasistring handling in TXR Lisp. - Bugfix in if function (not the if operator). TXR 78 2014-02-06 Features - vec function for making a vector out of its arguments, to complement the existing vector function which makes a vector of a given size. - The dot position of function call forms can now apply strings and vectors. The same is true of the apply function. - The apply function now takes additional optional arguments before the list, similarly to Common Lisp's apply. - New function: list*. - Three-element forms for optional parameters are now supported; the variable can be specified, an initialization form used when the argument is omitted, and a symbol that is bound to a boolean indicating whether or not the argument is present. (Common Lisp style, IOTW). - New protocol for optional parameters: the colon symbol : can be passed as a value for an optional parameter, which causes that parameter to behave as if it were missing. - reduce-left and reduce-right can now use nil as the initial object, which is an useful and important behavior. - strings, vectors, lists and hashes are now generally callable like functions, not only in the DWIM operator or DWIM brackets notation. Bugs - Lexical scope of optional argument default init forms now properly restricted. - Fixed breakages in do operator. - Bugfix in logic for tracking source code file and line number info that would cause the info to disappear under garbage collection. - Fixed a erroneous exception throws in the mutation operator logic which would cause usage errors to turn into internal errors. TXR 77 2014-01-30 Features - More streamlined parser containing fewer hacks, and fewer obscure cases that don't work. - @'expr syntax now works as expected in directives for evaluating quoted TXR Lisp expressions. - In nested uses of the do and op operators in TXR Lisp there is now a way to refer to the outer parameters from the inner nestings, using compounded meta syntax like (op (op @1 @@1)), where @1 is argument 1 of the function denoted by the inner op, and @@1 is argument 1 of the outer function. Each additional @ "escapes" out one level of nesting of the op syntax. - New update and hash-update functions. - The interfaces of reduce-left and reduce-right functions has been improved, making them easier to use, while mostly retaining backward compatibility. - New functions remove-path and rename-path for removing and renaming filesystem objects. - Catenated streams, previously an internal feature, are exposed now via the make-catenated-stream function. - Scope rule change for expressions that provide default intialization for optional arguments. These eexpressions now have the parameters in scope, and so now uses like (lambda (x : (y (length x))) ...) are possible where y is initialized with (length x) if the argument is not supplied. Previously, parameter x would not have been considered to be in scope of the expression (length x). Bugs - Fixed neglected handling of result form in dohash syntax. - In the object printer, the handling of syntax like @(sys:var x ...) was ignoring any additional ... forms, and rendering as @x. - Fixed possible race condition in tail streams, whereby when a file rotates, the stream prematurely follows the new file, neglecting to read the last piece of material just added to the previous file. TXR 76 2014-01-23 Features - New time functions: time-fields-local and time-fields-utc for obtaining broken-down time from Epoch seconds. - New group-by function for constructing a hash from a list when the list elements can be partitioned into equivalence classes tied to keys, rather than treated individually. - Sweeping changes in TXR List to allow vectors and strings to be manipulated in many situations as if they were lists. Functions like car, cdr and mapcar work on strings and vectors. - New command line options -e and -p for evaluating TXR Lisp expressions more conveniently from the command line. - The and, or and if special operators are now also provided as functions, so they can be indirected upon functionally. - New functions conses and conses*, useful for iterating over a list similarly to Common Lisp's maplist function. - New do operator (unrelated to @(do) directive) similar to op, but geared toward doing imperative things involving special operators. Bugs - @(require ...) directive was not expanding its forms prior to evaluation. TXR 75 2014-01-16 Features - Two new stream functions: unget-char and unget-byte. Some streams now support ungetting a byte or character, which was a glaring omission in the API, without which some common scanning tasks are awkward. - TXR Lisp functions with optional parameters can now specify expressions to give those arguments values when the arguments are missing. - New operators in TXR Lisp: append-each and append-each*. - Change in the Lisp structure printer. The special structure generated by the read syntax @sym and @(...) now prints back in the same syntax, rather than as (sys:var sym) and (sys:expr ...). Bugs - Fix in put-byte function: before invoking the underlying operation, it was testing whether the put-char operation exists for the stream, rather than the put-byte operation. This would result in a crash if the stream supports put-char but not put-byte. - Mistake in calculating bitmasks for regex character class ranges, resulting in incorrect behavior for ranges whose upper range is a character code corresponding to the last bit of a word in the bitmask array, e.g. [A-\x7f], resulting in failures to match 32 or more characters in the upper end of the range. - Missing documentation filled in for the functions throw, throwf and error. TXR 74 2014-01-13 Features - Maintenance: builds on Cygwin, MinGW and Mac OS X 10.7.3. - New math functions: n-choose-k, n-perm-k, cum-norm-dist. - lisp-parse function renamed read; old name is obsolescent. - In the TXR pattern language, the @ escape can now evaluate a non-compound TXR Lisp expression also. Previously compounds like @(+ 1 1) were supported, but atoms like @foo were not. Bugs - Small fix in how exponent fields of printed floating point values are normalized. Across all platforms, there are now no leading zeros after the 'e'. TXR 73 2014-01-08 Features - The new lisp-parse function scans TXR Lisp expressions at run-time out of strings and streams, which means that TXR can now more easily keep persistent, complex data in text files. Bugs - Fixed signal-handling-related performance issue caused by excessive calls to the sigprocmask function. TXR 72 2013-12-17 Features - Syslog functionality: openlog, closelog, syslog, setlogmask. Plus: the *stdlog* stream for logging directly to syslog. - Stream properties. - logand and logior functions become variadic. - Signal handling support. TXR Lisp code can catch POSIX signals. - Syntax changes in the area of symbol names. Package prefixes are supported now, like foo:bar (symbol bar in package foo). Bugs - Nonsensical error diagnostics in intern and delete-package, in the case when a package doesn't exist. - defvar is documented now and behaves more similarly to the Common Lisp defvar. - seek-stream with a zero offset was reporting the current offset instead of seeking regardless of the value of the whence argument. TXR 71 2013-12-07 Features - New functions countqual, countql, countq and count-if. - New regex-parse function: parse regex from a string at run-time. - New *stdnull* stream for discarding data. - New function: seek-stream for positioning within a stream. - New function open-tail for opening a new kind of stream called a tail stream. This stream follows rotating log files. - New functions for converting a time expressed as individual fields into numeric time: make-time, make-time-utc. - New function daemon (where supported). Calls the BSD-derived daemon function to daemonize the process in one step. - New errno function for accessing C errno. - Real-time input support in pattern matching language, via "simple" lazy text stream line lists which sacrifice accuracy of representation for timely delivery. Bugs - Fixed missing recognition for hexadecimal integers in Vim syntax higlighting file txr.vim. - Fixed missing check for null handle in stdio back-end of flush-stream. - Grammar and spelling in HACKING guide. TXR 70 2013-11-22 Features - Fixed a nasty bug that affects all versions; I was not able to find a revision in git that works right with gcc 4.6.3. Invoking an @(accept) in the middle of a @(collect) was found not work at all as documented, and this was root-caused to my neglect to use volatile on some local variables. - Fixed a more recent regression in the op syntax, caused this October. - Fixed an amazing regression introduced in TXR 22, back in November 2009, easily reproduced using: echo ":x" | ./txr -c "@x:@x" - - Fixed all unintentional deviations from 1990 ISO C. GCC had not been diagnosing them for us due to my neglect to use the --pedantic flag. - Revamped and up-to-date txr.vim syntax file, for you Vim users. TXR 69 2013-10-23 Features - Multiple @(output) blocks can now continue on the same output stream, which is particularly useful if it is a pipe. Bugs - Fixed errors in documentation for the functions time-string-local, time-string-utc and expt. TXR 68 2013-10-07 Features - @(repeat) inside @(output) supports a :vars parameter, making it possible to declare the existence of iteration variables that are otherwise hidden within Lisp code. Bugs - Out-of-sequence numeric parameters can be used in the op operator now, like (op foo @2) where @1 is absent. - The implicit ". @rest" is now added to op forms only if they do not mention @rest or any numeric parameters, to prevent unintentional passing of extra parameters. - @rest or a numeric param like @1 can now be specified in the dot position of the op form, as in (op foo . @rest). TXR 67 2013-07-13 Features - New functions tok-str and string-cmp. - Internal lazy string functions exposed as intrinsics. - New pattern language directive: @(require ...). Bugs - Unrecognized backslash escapes in string literals and string quasiliterals are diagnosed. - Backslash-space escape recognized in string literals and string quasiliterals. TXR 66 2013-05-16 Features - Documentation completely filled in. - Added specfile for RPM builds. - @(repeat) introduced as shorthand for @(collect :vars nil). - New open-command and open-process functions. The open-pipe function becomes deprecated. - New multi-sort function for sorting two or more lists at the same time as if they were columns of a table. - assq and aconsq functions are renamed to assql and aconsql becuse they use eql equality, not eq. - New prop function for property list lookup. - New stat function for information about files. - New functions in hashing library: - Copying hashes: make-similar-hash, copy-hash - Set operations: hash-uni, hash-diff, hash-isec. - Gapingly missing hexadecimal integer constants have been implemented. - New bit operation functions that work with TXR's arbitrary precision integers: loand, logior, loxor, lognot, logtest, ash. - Test form in for loop can be omitted. - New package-related functions. - New functions for obtaining time of day and converting to text. Bugs - Fixed broken (+ ) case in addition. This fixes the range and range* functions too. - Fixed nonworking building in a separate directory. - Fixed broken eval function. - Bugfix in form expander's handling of regular expression syntax, causing the (set ...) notation for character sets being mistaken for the (set ...) assignment operator. - Bugfix in format: apply field formatting to argument not only if a nonzero with has been specified, but also if a precision has been specified. - Bugfix in format: ~s on a floating point number now always shows .0 except when a precision is given, and it is zero. - Fixed broken @(last) clause processing of @(collect), int he case when the last material matches at the end of a stream. - Fixed certain function applications not being able to call functions that have optional arguments with fewer than TXR 65 2012-04-20 Features - New ways to iterate over hash tables using lazy lists. - Regular expression compiler expressed to user code, allowing TXR programs to build regex abstract syntax trees and turn them into regular expression machines. - The popular regular expression tokens \s, \d, \w and \S, \D, \W are now supported (the first three of these in character classes also). - Low level time-of-day functions added for retrieving epoch seconds, and optionally microseconds. - New remove-if and keep-if functions, plus lazy counterparts remove-if* and keep-if*. - Lazy variants remq*, remql* and remqual* added. - New function find-if. - Improved seeding of pseudo-random-number generator. - Configure script now diagnoses specification of nonexistent config variables. - Documentation improvements. - Portability to Mac OS/X 10.7 (Lion), NetBSD 5.1 and FreeBSD 9.0. Bugs - Bugfix in @(next) directive, which showed up when using @(next :args) more than once to scan arguments more than once. Quiet Changes - First two arguments to the find function were reversed. TXR 64 2012-04-06 Features - In the pattern language, binding the symbol nil as a variable is permitted now and has a special meaning of matching and discarding input. - Caching optimization in hash tables. - Ephemeral GC implemented. This is experimental and currently must be enabled at compile time with the --gen-gc configure option. Bugs - Interaction between @(trailer) and @(accept) has been clarified. An @(accept) return from the forms of a @(trailer) will still roll back the input position to the starting point for the @(trailer). TXR 63 2012-03-30 Features - Significant improvement in the performance of the garbage collector, which can drastically cut the run-time of some programs. - match-str and match-str-tree functions exposed. - new @(rebind) directive provides for a one step dynamic shadowing of a variable with a local variable, whose value can be derived from the one being shadowed. - filters now work on nested lists of strings, not just strings. - floating point formatting now produces the same exponential notation on different platforms, by post-filtering the sprintf output: a leading + and leading zeros are removed: e+023 becomes e23. - new functions: num-str, tan, asin, acos, =, - min and max return the left operand rather than the right one, when two operands are equal. Bugs - search-str optional argument was not working when omitted. - Fixed situations involving an incorrect behavior: partial horizontal matches (not matching the full line) were succeeding. - It was impossible to define a pattern function called @(text) due to the use of the symbol text as an internal symbol, rather than a symbol in the private system package. - error checking code in funcall had an improperly terminated argument list, causing an exception. - @(output) blocks now flush their stream at the @(end), without which output is not correctly interleaved between output produced by external commands. - fixed some misleading exception wordings in the numeric library. - fixed sign semantics of floating-point mod to be like the integer mod. - gcd function allows zeros. - fixed broken exptmod: it was not normalizing bignum values that fall into the fixnum range into fixnum objects, resulting in undersized bignums being returned. TXR 62 2012-03-23 Features - Floating-point support has been added. - TXR programs can specify floating-point constants. - Printing floting points is supported in the format function. - New specifiers: ~f and ~e. - Arithmetic on floating points, with implicit conversion. - New / function which produces a floating-point result. - New functions like sin, cos, exp, and log. - Functions for converting between floating and integer, and parsing a float from a string. - New combinators for easier programming with higher order functions: chain, andf, orf, iff. - url_decode and url_encode functions take an optional parameter which determines whether + is used as the code for space, distinguishing URL encoding from percent encoding. Separate named filters are introduced: :frompercent, :topercent distinct from :fromurl and :tourl. Bugs - Buggy quicksort routine repaired. This is a recently added feature which allows vectors and strings to be sorted. List sorting, done by merge sort, is unaffected. - Breakpoints can now be set by line number and file name, allowing breakpoints in loaded modules, not just the main module. TXR 61 2012-03-15 Features - URL encoding and decoding functions added, and :tourl/:fromurl filters implemented for output substitution. - split-str function works with regex objects now. - new regsub function for regex substitution in strings; matched text can be filtered through a function. - new *stddebug* stream used by debugger. - put-byte works on string output streams, and does right thing with mixtures of bytes (which are taken as UTF-8) and characters written with put-char. - Hash table literals implemented; hashes can now be notated rather than constructed. - Vectors can now be quasiquoted with unquote and splicing, and so can the new hash table literals. - @(block) syntax change: blocks do not extend to the end of the surrounding scope but introduce their own, delimited by @(end). Fixes - Fixed memory leak in string byte input stream. - Fixed wrong parsing of tokens in cases like @(collect-foo) where this was treated as @(collect -foo) rather than a single token. TXR 60 2012-03-04 Features - List element removing functions: remq, remql, remqual. - Memory use optimizations in @(freeform) horizontal scanning. - Improved hashing function for bignums. Bugs - Fixed incorrect processing within long lines, caused by earlier optimizations in 57. - Missing 'd' command for deleting a breakpoint implemented in the debugger. - numberp function no longer blows up on nil. - Fixed broken @(load) directive: was not allowing directives which follow it to be processed. - Fixed incorrectly formatted line number information in some exceptions. - Fixed missing support for optional and trailing parameters in defun. - Fixed showstopper bug in plus function. TXR 59 2012-02-29 Features - Implemented put-byte function on streams. - Regex syntax in TXR Lisp changes from /re/ to #/re/, allowing symbols containing slashes. - New -B command line option forces TXR to dump the bindings even if the program produced output. - Revised and improved txr.vim syntax highlighting file for greater lexical accuracy and superior flagging of errors. Fixes - Regression that broke [n..t] and [n..nil] sequence ranges. - Broken hex-escaped character literals. - Old bug from database: filter functions now see bindings in all contexts in which they are invoked. - Output clauses consisting of one empty line were being treated as empty. - Implemented more sane rule for deciding when output has taken place and suppress the printing fo bindings. TXR 58 2012-02-25 Features - Exception handling exposed in TXR Lisp. - range* function similar to range, but for generating a range with the endpoint excluded. - TXR Lisp can invoke a function in the pattern language now using the new match-fun function. - Braced variable syntax in @(output) extended to support arbitrary expressions in place of the variable name. The expressions are converted to text and then treated the same way as a variable substitution. Indexing is allowed, and field-formatting takes place. Moreover, these expressions are code-walked by @(repeat) and @(rep) to look for embedded variables. - New TXR Lisp functions ref, refset, sub and replace. - Indexing and range extraction in brace substitution in @(output) clauses is now generic over lists, vectors and strings. - Optional arguments introduced into TXR Lisp, in defun and lambdas, with a simple syntax involving a colon. Numerous intrinsic functions have been converted such that some previously required arguments are now optional. - Sort function that was previously for lists only is now generic over vectors and strings. - New del operator in TXR Lisp can delete elements of sequences or hashes which are denoted using the bracket syntax. - Range indexing has "floating zero" behavior now so that for example [seq -2 0] means the "last two elements": when the start of the range is negative, then a zero end is treated as length plus one. - TXR programs can now be split into multiple modules, using the load directive to load and run code. Bugs - range function terminates lazy list when the sequence overshoots the final value. - Variable that holds nil is treated as a list in (set [var x..y] ...) range assignment. - Vestigial (t . obj) representation in exception handling code removed. - TXR now does not not dump bindings if any output occured on a stream connected to the C stdout stream. Previously, it suppressed printing of bindings if @(output) was carried out on the *std-output* stream. - Function remhash was found to be buggy and rewritten. TXR 57 2012-02-14 Features - Operations under the @(freeform) directive can now scan in constant memory, allowing large files to be processed. (Scanning a single regex still requires the data to be all in memory: an experimental patch for this exists.) - Improved printing of character position context in debugger when lines are long. - Metanums (@1, etc) can be used in a quasiliteral, which is useful for quasiliterals occuring inside the op notation. Bugs - lazy-flatten function did not handle atoms. This broke @(next :list expr) also, for the case where expr evaluates to a string atom. - In format, the ~s directive was found to be printing strings in the same way as ~a. - Hex and octal character constants did not work. - Control characters in strings and characters are printed as hex now rather than octal. A semicolon is added if the next character would be interpreted as part of the escape. - Hash indexing via the [] notation was still requiring the default value argument. TXR 56 2012-02-06 Features - Hex and octal escapes work in strings and quasilterals now: the documentation has stopped lying. - Escapes can be followed by a semicolon which terminates them and is removed, which is useful if an escape is followed by characters that would otherwise be interpreted as part of the escape. - More color categories and more accurate syntax in Vim syntax highlighting file. Highlights @[...] syntax properly inside quasiquote. - The third argument (the default value if a key is not found) can be omitted when indexing hashes with the [hash key] syntax. It defaults to nil. - The dwim operator (and thus [] syntax) is even more Lisp-1 like. It now has Lisp-1 namespace semantics for evaluating arguments that are symbols. - A new operator called "op" as been added. This allows simple lambda functions to be written as partial evaluatios of functions, with implicit arguments as well as numbered/rest arguments appearing in the body. Bugs - Fixed missing type check in hash module that allows bad code to crash interpreter instead of getting an excepion. - Fixed regression in TXR 55 which broke computed field widths in output variables. - Fixed incorrect UTF-8 decoding of some code points. - Closed several security holes in UTF-8 decoder by recognizing all invalid UTF-8 sequences, and mapping invalid bytes in such a way that any byte sequence processed by the decoder into Unicode code points will be recovered exactly when encoded back into UTF-8. TXR 55 2012-01-26 Features - New square bracket syntax for Lisp-1 like invocation and array-indexing over lists, vectors, strings and hashes. - New a .. b pair syntactic sugar for (cons a b). Works with array indexing brackets to extract ranges (slices) from lists, vectors and strings. - Indexed elements and slices are assignable. - In the pattern language, variables in output templates (output variables) can be indexed and sliced. - Output variables that hold lists are now interpolated with spaces between, and this can be overridden with any separator string. TXR 54 2012-01-21 Features - Improved debugger: - step into - step over - finish - backtrace for pattern and TXR Lisp functions. - dump more complete environment. - Debugging support can be disabled at compile time: configure with --no-debug-support. - New lazy append function (append*). TXR 53 2012-01-11 Features - In variable substitutions in output and quasiliterals, the field width expression is now evaluated. - TXR Lisp: - New operators and functions for generating lazy lists more conveniently. - lazy mapcar and mappend: return lazy list. - delay and force operators. - parallel iteration/collection over lists. - list-str function for splitting string into list of characters. Bugs - Fixed global buitin variables, broken in 52. Properly implemented intended fix. TXR 52 2012-01-07 Features - @(rep) and @(repeat) now take an keyword argument, :counter. This specifies the name of a variable which holds the repetition count, thus making the count available within the repeated body. - New @(mod) and @(modlast) clauses in repeat/rep, allowing special behaviors to be coded based on whether the repetition count is a particular value modulo some number. - @(gather) directive now supports an @(until)/@(last) clause, so the search range can be restricted. - New directive @(fuzz) for doing an imprecise match over a range of lines, similar to the context line fuzz logic in the patch utility. - gensym function added to TXR Lisp, along with a *gensym-counter* global variable. Bugs - Fixed a regression in repeat/rep triggered when multiple clauses of the same type appear (like two occurences of @(last)). - Built-in global variables in TXR Lisp now properly denote the underlying C variable locations, rather than holding copies of the values of those variables. If you assign to *stdout*, it will now really replace the stdout stream stored in the corresponding C variable (std_output), so it has the proper global effect. Previously, this action would just replace *stdout* with a new value, without any effect on std_output. TXR 51 2011-12-28 Features - Better algorithm in the random function for generating evenly-distributed pseudo-random numbers of arbitrary precision. - PRNG startup behavior improved. - New lazy-flatten function for flattening lists lazily. Used within @(next :list ...) implementation, making it possible to scan through Lisp-generated lazy lists. Bugs - Fixed showstopper bug introduced in a patch for the MPI library, affecting 32 bit platforms. - Fixed bug in Lisp expression interpolation in quasiliterals. - Fixed fatal exception being thrown in verbose mode, by one of the formatted print statements. TXR 50 2011-12-23 Features - Dropped silly leading 0 from version number. :) - New vector functions: copy-vec, sub-vec, cat-vec. - New API for pseudo-random-number generation, complete with independent random state objects that can be seeded and copied. - Vim syntax highlighting definition improvements. - In the format function, left-adjustment is specified using < rathr than the - character. (However a negative width specified as an argument using * still means left adjust.) The leading zero for zero padding as well as the sign character (space or +) are specified in the precision field, not the width field. - More complete documentation. - Slight return value convention change in some and all functions. - Added memql function. Bugs - Critical flaw in hashing fixed that could crash on some platforms. - Exception handling fix: if a catch clause performs a non-local exit, the finally clause is still executed. - "make distclean" fixed. - Fix for differences in syntax error diagnosis between Byacc and Bison. - Fixed a memory leak in a division-by-zero case in the bignum mod function. - Fixed a terrible bug in one of the MPI patches affecting the correctness of various operations on numbers having a 1 in the most significant bit position of the most significant digit word. - Fixes in format function. All objects obey field width and left/right alignment. Numeric precision, zero padding and optional sign all works. - Lisp code evaluated in @(repeat)/@(rep) clauses can now see all variables, not just the ones subject to the repeat. (But whether or not a repeat executes, or which clauses, is still independent of what variables are accessed by the embedded Lisp, taking into account only the variables directly embedded in the clause.) TXR 049 2011-12-19 Features - New functions for converting between characters and integers. - Some arithmetic and relational operations are generic over characters in a way that makes sense. - dohash establishes anonymous block. - Improvements in Vim syntax highlighting file. - Lazy cons semantics cleaned up making lazy list programming easier. - Some API renaming and restructuring in the area of vectors. - Semicolon comments supported in Lisp code and @; comments in the pattern matching language. @# becoming obsolescent. - Not function, synonym for null. - Some progress in TXR Lisp documentation. - Hashing functions improved for fixnums, literals and characters. - API for introspecting over interpreted functions added, in anticipation of doing some compiler work. - Quasiliteral strings supported in TXR Lisp. Bugs - Broken abs function fixed for bignums. - mappend semantics clarified and fixed: behaves like append for improper lists and atoms. - Bugfix in code walker for let/let* forms, which resulted in quasiquotes not being expanded. - Fixed incorrect format arguments in some error messages, resulting in aborts in some error cases, instead of the intended diagnostics. TXR 048 2011-12-13 Features - New functions: expt, exptmod, sqrt, numberp, evenp, oddp, abs, gcd reduce-left, reduce-right. - Replaced horribly slow square root in MPI with a less horribly slow one. Bugs - Fixed numerous instances, in the MPI library, of coding broken for mpi_digit wider than 16 bits, leading to incorrect results and crashes. - Fixed mpi_int for 32 bit platforms so that obj_t stays 4 pointers wide. (The sign becomes a 1 bit wide bitfield). TXR 047 2011-12-12 Features - Transparent bignum arithmetic: when operations on machine word (fixnum) integers overflow, multi-precision (bignum) integers are produced. - TXR Lisp: - New operators: progn, flip. - Vector functions added, and vecref is an assignment place. - Character manipulation functions. - Association list functions. - Implicit anonymous block around forms for loop. - Implicit named block around forms for loop. - Nump renamed to fixnump. - Push arguments reversed: (push obj list). - Syntax highlighting definition update for all new operators. Bugs - Another bugfix to character literals, allowing non-alphanumeric constants like #\$. - Fix in rplacd to make lazy list programming feasible. - Reversed assoc and assq arguments throughout codebase. - Debugger: repeating last command by pressing Enter works again. TXR 046 2011-12-06 Features - Vector-related functions exposed in Lisp interpreter. - Syntax added for specifying vector literals in code. - Length function made generic over strings, lists and vectors. Bugs - Broken get_line function over string input streams fixed. - Some kinds of character literals were not being recognized properly. - bugfixes in configure script, affecting 64 bit Mac OS X. Thanks to Andy Wildenberg for reporting issue and confirming root cause. TXR 045 2011-12-05 Features - New functions exposed in Lisp interpreter: strings, characters, symbols and lazy lists. Bugs - Flaws in some string-splitting functions identified and fixed. - Bugs in quasiquote. - Handling of singleton clauses in cond operator. TXR 044 2011-12-01 Features - Lisp interpreter added. - Relaxed rules for what constitutes symbol names. Bugs - Regression in source location tracking (line numbers shown in debugger and trace output). - Regression in vertical skip directive caused it to ignore its arguments. - Fixed :vars () in collect/coll not working as intended. This should prevent any bindings from being collected, and allows iteration with collect without accumulating memory. - Two-year-old bug in weak hash tables. TXR 043 2011-11-23 Bugs - Buggy @(eol) directive fixed. - Semantics change for text and regular expressions in "negative match": - a variable is considered to be followed by a run of text which consists of any mixture of regexes and literal text - thus @foo bar behaves properly once again; it is not treated as foo followed by the regex / +/, ignoring the text bar. - Infinite looping bug in negative match with longest-match semantics. - Bug in the overflow detection in the lib.c plus and minus function. TXR 042 2011-11-20 Features - Access to environment via @(next :env) - New @(gather) directive for matching out-of-order material. - Horizontal functions: - TXR can now parse grammars. - Variable syntax allows capture of text spanned by function calls: e.g. @{var (func x y)}: text spanned by (func x y) goes into var. - Horizontal modes for numerous directives such as @(bind), @(local), ... - Lisp-style bindings output. - Interactive breakpoint/step debugger added. This is an experimental prototype, but already quite useful. - Directives following a variable match have searching semantics now, as do function calls. @a@(foo) will scan for a match for @(foo), and the text skipped over is captured by a. - New :resolve keyword in @(some) directive allowing conflicting variable bindings to be set in separate clauses and resolved. - deffilter is more powerful: arguments are evaluated so filters can be computed by the query. Bugs - Horizontal @(some) directive hooked in. - @(freeform) with no data fails to match instead of throwing strange error. - Setting non-local variables from a function works now. - Stability fix: several long-time uninitialized variable bugs found, and some faulty object initialization orders. - :vars in @(collect)/@(coll) does not fire an exception about missing required variables if no variable bindings are produced at all, allowing strict collects to have explicit matches for unwanted material without triggering this nuisance error. - @(repeat)/@(rep) allow empty body. Internal - New infrastructure for matching location info to source forms. Location info (currently just the line number) is associated with source forms via a weak hash table. - Character literals are now Lispy (hash-backslash syntax), based on Scheme. - Added quote, unquote and splicing syntax similar to Lisp. Not used for anything yet. - Improved Valgrind integration: more accurate, timely detection of uninitialized fields in heap objects. Misc. - A TXR syntax highlighting file for the Vim editor exists now. See the txr.vim file. TXR 041 2011-10-30 Features - New :append keyword in @(output) to append instead of overwriting. - Variable contents can be treated as input sources using :string and :list keywords in @(next). Variables can be treated as output destinations using :into keyword in @(output). - New @(set) directive for destructive assignment to a variable. - New filters: :upcase and :downcase. - @(bind) can now compare left and right objects through filters. - Filters can now be chained into compound filters. - Pattern matching functions can be used as filters. - Shorthand notation in @(deffilter) when multiple strings map to the same replacement string. - @(cat) directive changes syntax. - Error handling improvements in parser: no more reams and reams of errors. Bugs - Runaway recursion in @(block) directive, introduced in 040. - Fixed bug in matching list variable against text, at the same time clarifying semantics to longest-match. - Fixed potential excessive memory use caused by refactoring in 040.