| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Hashing of buffers and character strings is being replaced
with a seedable hash, providing a tool against denial of
service attacks against hash tables.
This commit lays most of the groundwork: most of the internal
interface changes, and a new hashing implementation. What is
missing is the mechanisms to do the seeding.
* hash.c (struct hash_ops): Hash operation now takes a seed
argument of type ucnum.
(struct hash): New member, seed.
(hash_str_limit): Default value changed to INT_MAX.
A short value opens the gateway to an obvious collision attack
whereby strings sharing the same 128 character prefix are
entered into the same hash table, which will defeat any
seedings strategy.
(randbox): New static array. Values come from the Kazlib hash
module, but are not used in exactly the same way.
(hash_c_str, hash_buf): Now take a seed argument, and are
rewritten.
(equal_hash): Takes a seed, and passes it to hash_c_str,
hash_buf and to recursive self calls.
(eql_hash_op): New static function. Adapts the eql_hash
operation, which doesn't take a seed, to the new interface
that calls for a seed.
(obj_eq_hash_op): Take a seed; ignore it.
(hash_hash_op): Take a seed, pass it down to equal_hash.
(hash_eql_ops): Wire hash functiono pointer to eql_hash_op
instead of eql_hash.
(make_hash): For now, intialize the hash's seed to zero.
(make_similar_hash): Copy original hash's seed.
(gethash_c, gethash_e, remhash): Pass hash table's seed to
the hashing function.
(hash_equal): Pass a seed of zero to equal_hash for now;
this function will soon acquire an optional parameter for the
seed.
* hash.h (equal_hash): Declaration updated.
* lib.c (cobj_handle_hash_op): Take seed argument, pass down.
* lib.h (cobj_ops): Hash operation now takes seed.
(cobj_eq_hash_op, cobj_handle_hash_op): Declarations updated.
* struct.c (struct_inst_hash): Take seed argument, pass down.
* tests/009/json.expected: Updated, because the hash table
included in this output is now printed in a different order.
|
|
|
|
|
| |
* txr.1: Make it clear that block* is not subject to
progn-conversion.
|
|
|
|
|
|
|
|
|
|
|
| |
This breaks the (next) awk macro, breaking awk expressions
which want to short-circuit to the next record.
* share/txr/stdlib/awk.tl (sys:awk-state loop): The :awk-rec
block is being optimized away by the compiler because it
doesn't contain any direct function calls to functions that
are not in the standard library. We must use block*, which
isn't subject to this optimization.
|
|
|
|
|
|
|
|
|
|
|
| |
Planning to support seeded hashing, so the behavior of the
hashing function will change. But we need a stable hash for
the section URL's in the HTML doc; so let's preserve the
existing function as Lisp code.
* genman.txr (hash-str): New string hashing function. This
behaves like the existing hash-equal behaves on 32 bits.
(hash-title): Use hash-str instead of hash-equal.
|
|
|
|
|
|
| |
* ffi.c (ffi_closure_dispatch, ffi_closure_dispatch_safe): The
nargs variable holds a copy of tfcl->nparam, so use it
instead of accessing tfcl->nparam again.
|
|
|
|
|
|
| |
* vm.c (max): New macro.
(vm_call, vm_apply, vm_gcall, vm_gapply): Use max macro in
calculating args allocation.
|
|
|
|
|
|
|
| |
* ftw.c (max): New macro.
(ftw_callback): Allocate enough args for all five arguments.
This bug went undetected when this was developed. A more
recently added assertion in args.c caught this.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The four hash operations (hash, equal, assoc and acons) are
moved out of the hash instance and into a static table of
operations. This means we have to go through one more
indirection when calling them, but we save some space and
shorten initialization.
A bug is fixed here: the copy_hash function was only copying
three out of these four functions; the equal operation was
left uninitialized eposing the hash equality operation to
corrupt behavior. Now that function just copies the hops
pointer, so all is well.
* hash.c (struct hash_ops): New struct type.
(hash_ops_init): New macro.
(struct hash): hash_fun, equal_fun, assoc_fun and
assoc_new_c fun members removed. New member hops.
(hash_eql_ops, hash_equal_ops): New static structures.
(hash_equal_op): Compare the hops pointers rather than
just the hash_fun: if two hashes have different hops
pointers, they are different, period.
Indirect through ops for calling equal fun and others.
(hash_hash_op): Include the hops pointer in the hash,
rather than the hash_fun pointer.
(hash_print_op): Test for hash type (equal based or eql based)
now done by comparing the hops pointer to one of the two
static structures, rather than hash_fun to one of two
functions.
(make_hash, make_similar_hash, copy_hash): Initialize hops;
remove initializations for the four functions.
(gethash_c, gethash_e, remhash): Indirect through hops
to invoke hash operations.
(hash_uni, hash_diff, hash_isec): Incompatible hash check
based on comparing hops pointer rather than hash_fun.
|
|
|
|
|
| |
* hash.c (hash_equal_op): Fix broken logic that is supposed to
push a cell onto the pending list: rplaca should be rplacd.
|
|
|
|
|
|
|
| |
* parser.c (read_file_common): When reading a compiled file,
turn off the rec_source_loc flag in the parser, since the
forms are just data, and not source code for which we need
error reporting.
|
|
|
|
|
|
|
|
|
| |
* vm.c (struct vm_desc): Rename ftsz member to stsz.
(struct vm_ftent): Renamed to struct vm_stent.
(vm_make_desc, vm_make_destroy, vm_desc_mark, vm_stab,
vm_invalidate_binding): Rename local ftsz variable to
stsz, and follow the renames of struct members and
of struct vm_ftent.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a function binding is removed using fmakunbound, virtual
machine descriptions hang on to the previously cached binding
in the ftab. When the symbol is newly bound, virtual machine
descriptions keep pointing to the old function.
To solve this, we put the vm_desc structures into a global
list and provide a function that fmakunbound calls to scrub
all the VM descriptors of that binding.
* eval.c (makunbound, fmakunbound): Call new
vm_invalidate_binding function.
* vm.c (struct vm_desc_links): New structure.
(struct vm_desc): New member lnk, with links.
(vmd_list): New static variable: circular list of all VM
descriptors.
(vm_make_desc): Insert new VM descriptor into list.
(vm_desc_destroy): Remove VM descriptor from list.
(vm_invalidate_binding): New function.
* vm.h (vm_invalidate_binding): Declared.
|
|
|
|
|
| |
* linenoise/linenoise.c (yank_sel): Use wmalloc_fn because the
size is being measured in characters rather than bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The GNU C library is deprecating the practice of <sys/types.h>
defining the makedev, major and minor macros. A loud,
multi-line warning is issued for programs which include this
header and then use the macros. The new way is to rely on
the <sys/sysmacros.h> header instead. Of course, that might
not exist.
* configure (have_makedev): New variable. Upgrade the makedev
test to try it with <sys/sysmacros.h> first, then fall back
on <sys/types.h>. A new config macro HAVE_SYS_SYSMACROS_H
is created in config.h if sysmacros exists.
* sysif.c: If config.h defines HAVE_SYS_SYSMACROS_H to a
nonzero value, then include <sys/sysmacros.h>.
|
|
|
|
|
|
|
|
| |
* txr.1: In Syntax synopses, use the '[' and ']' notation to
denote literal square brackets, since square brackets mean
"optional". Updated Conventions sections to document this.
Changes under operator drim, if, and and the from-list
method.
|
|
|
|
|
|
|
| |
* struct.c (struct_handle_for_slot): New static function.
(slot, maybe_slot, slotset): Use struct_handle_for_slot rather
than struct_handle, for improved error message that reveals
what slot name was requested.
|
|
|
|
|
|
|
|
|
|
| |
* RELNOTES: Updated.
* configure, txr.1: Bumped version and date.
* share/txr/stdlib/ver.tl: Likewise.
* txr.vim, tl.vim: Regenerated.
|
|
|
|
|
|
|
|
|
|
|
| |
The new abstraction layer used by linenoise throws exceptions,
but linenoise excpects a null pointer when a file open fails.
* parser.c (lino_open, lino_open8): Catch error exceptions
and convert to null return, using new macros to reduce
repetitive coding.
* unwind.h (ignerr_begin, ignerr_end): New macros.
|
|
|
|
|
|
|
|
|
|
|
| |
Compiling a block* fails with an exception: nil is accessed
as a structure.
* share/txr/stdlib/compiler.tl (comp-block): when compiling
block*, nenv is nil; we must use env when compiling
the name subexpression. Also, we can't use nenv when compiling
the body. We must use nenv when it is available (block case)
or else fall back on env (block* case).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The virtual machine's funvec will be used for global lexical
variables also, so it is being renamed to symvec. Related
structure members, functions and variables are also renamed.
* share/txr/stdlib/asm.tl (disassemble): Print the table
heading as syms: rather than funs:. Follow the rename of
vm-desc-funvec to vm-desc-symvec.
* share/txr/stdlib/compiler.tl (compiler): Slots fidx-cntr
fidx and ftab are renamed to sidx-cntr, sidx and stab, resp.
(compiler get-fidx): Renamed to get-sidx.
(compiler get-funvec): Renamed to get-symvec.
(compiler comp-setqf, compiler comp-catch, compiler
comp-fun-form, usr:compile-toplevel): Follow rename.
(list-from-vm-desc): Follow rename of sys:vm-desc-funvec.
* vm.c (strut vm_desc): Members funvec and ftab renamed to
symvec and stab.
(vm_make_desc): Parameters and local variables renamed.
Follow rename of struct members.
(vm_desc_funvec): Renamed to vm_desc_symvec.
(vm_desc_destroy, vm_desc_mark): Follow rename struct members.
(vm_ftab): Renamed to vm_stab.
(vm_gcall, vm_gapply): Follow rename of vm_ftab.
(vm_init): Register renamed vm_desc_symvec function as
sys:vm-desc-symvec.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Calculation of free symbols emanating out of let, let*, flet
and labels is wrong, not taking into account the differences,
respectively between let and let*, and between flet and
labels. Compilation of lambda also has the same problem;
variable references in initforms are considered free without
regard for shadowing by earlier parameters. Another issue
is the incorrect handling of special variables: special
variable references are incorrectly being considered free
in scopes where they are bound.
* share/txr/stdlib/compiler.tl (compiler comp-let): For
sequential bindings (let*), we must cull the prior variables
from the list of free vars emanating out of each init form;
these references do not emanate out of the binding construct.
We pull the prior vars list out of the environment before
binding the current variable so that it is
not included in the list. Both special and lexical variables
must be considered reference-capturing.
(compiler comp-fbind): If compiling a recursive binding, cull
the newly bound functions from the free references emanating
from the local function bodies. A bug is fixed here that we
were not referring to the correct list of symbols and so not
taking into account the function references inside the local
functions themselves at all.
(compile comp-lambda): Build a correct list of free vars in
relation to the initforms of optional parameters, taking
account the scope, and that special variables capture
references.
|
|
|
|
| |
* txr.1: Revise incorrect semantic description of eval-only.
|
|
|
|
| |
* txr.1: Wrong article "an local" and missing .codn line.
|
|
|
|
|
|
|
| |
* txr.1 (help): Someone reading the help text doesn't
necessarily know that --help produced it; it can be read by
someone in contexts where that same person didn't just obtain
it by running txr --help.
|
|
|
|
|
| |
* txr.1: Fold --compiled with --lisp into the same
description.
|
|
|
|
|
|
|
|
|
|
|
|
| |
The problem is that compiled code which uses the awk macro
won't load because it invokes make-struct for the unknown
sys:awk-state type. Interpreted code doesn't have the problem
because it has to expand the awk macro from scratch to
generate the make-struct call, and the awk macro triggers
the autoload.
* lisplib.c (awk_set_entries): Add sys:awk-state to autoload
list for awk.tl.
|
|
|
|
|
|
|
|
|
|
| |
This makes certain macros cheaper: macros which wrap code
with numerous local functions, not all of which are expected
to be used.
* share/txr/stdlib/compiler.tl (compiler comp-fbind): Detect
functions that are completely unused, and eliminate their
code.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
NB: Accesses to lexical variables are not all marked used yet.
* share/txr/stdlib/compiler.tl (binding): New slot, used.
(sys:env lookup-var, sys:env lookup-fun, sys:env lookup-lisp1,
sys:env lookup-block): Support optional Boolean argument
which, if true, causes the lookup to mark the binding used.
(compiler comp-return-from): Pass t to lookup-block, and
remove code to mark used.
(compiler comp-fun, compiler comp-fun-form): Pass t to
lookup-fun to mark function used.
(compiler comp-lisp1-value): Pass t to lookup-lisp1 to mark
function used.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the @{var mod} syntax in the pattern language, allow mod to
be a variable which contains a regex or integer, not just an
integer or regex literal.
* match.c (h_var): Check for modifier being a variable, and
resolve it.
* parser.y (modifiers): Allow a SYMTOK phrase.
* txr.1: Documented.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is in ANSI CL; potentially useful and hard to
implement efficiently in user code.
* arith.c (logcount): New function.
* eval.c (eval_init): Register logcount intrinsic.
* lib.h (logcount): Declared.
* mpi/mi.c (s_mp_count_ones): New static function.
(mp_count_ones): New function.
* mpi/mpi.h (mp_count_ones): Declared.
* txr.1: Documented.
|
|
|
|
|
| |
* arith.c (width): Fix incorrect name in type error
diagnostic. There is no such function as integer-length.
|
|
|
|
|
|
|
|
|
|
| |
* RELNOTES: Updated.
* configure, txr.1: Bumped version and date.
* share/txr/stdlib/ver.tl: Likewise.
* txr.vim, tl.vim: Regenerated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The standard input and output streams are in text mode on
Cygnal, which interferes with the listener, because it draws
input from streams. Let's hack it by Cygwin-specific code
in linenoise.
* linenoise/linenoise.c (struct lino_state): New members
orig_imode and orig_omode, on Cygwin/Cygnal only.
(enable_raw_mode): As part of enabling raw mode, use the
Cygwin setmode function to put both descriptors in binary
mode, saving their previous mode.
(disable_raw_mode): Revert the previous mode of both
descriptors, in reverse order in case they are the same
descriptor.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We cannot pass raw C wide literals to static_str; this breaks
on platforms where wchar_t is two bytes and strings are
aligned to only two byte boundaries. That's why TXR has the
wli() macro. We don't want to introduce the wli() macro into
linenoise, so the two choices are: duplicate the incoming mode
strings into dynamic storage with string(), or pass some enum
to specify file mode and locally convert to static mode
string. Let's go with the latter.
* linenoise.c (edit_in_editor, lino_hist_save): Use enum
constant for file mode instead of mode string.
* linenoise.h (enum lino_file_mode, line_file_mode_t):
New enum.
(struct lino_os): File opening functions use lino_file_mode_t
instead of mode string.
* parser.c (lino_mode_str): New static array.
(lino_open, lino_open8, lino_fdopen): Take enum mode instead
of string. Convert to literal through lino_mode_str table,
and pass that literal to static_str.
|
|
|
|
|
|
|
| |
* lib.c (chk_wrealloc): convert needs to be a coerce.
* parser.l (grammar): Use yyg instead of yyscanner;
the latter is the same pointer but of void * type.
|
|
|
|
|
| |
* rand.c (random): Function now rejects negative bignums, not
only negative fixnums.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lib.c (chk_wrealloc): New function.
* lib.h (mem_t): Wrap with ifndef block.
(MEM_T_DEFINED): New preprocessor symbol.
(chk_wrealloc): Declared.
* linenoise/linenoise.c (LINENOISE_MAX_DISP): Adjust to a
reasonable value; just twice the number of abstract
characters. The 8 factor had been chosen to cover the worst
case that every character is mapped to a tab.
(struct lino_state): Almost everything char typed turns to
wchar_t. The TTY isn't referenced with Unix file descriptors,
ifd and ofd, but abstract stream handles tty_ifs and tty_ofs.
The ifs member isn't required any more since plain mode is
handled via the tty_ifs stream.
(mem_t): Declaration removed; now in linenoise.h.
(chk_malloc, chk_realloc, chk_strdup_utf8): Declarations
removed.
(lino_os): New static structure.
(nelem): New macro.
(wcsnprintf): New static function.
(enable_raw_mode, disable_raw_mode): Get Unix FD from stream
using lino_os interface.
(get_cursor_position, get_columns, handle_resize, record_undo,
remove_noop_undo, restore_undo, undo_renumber_hist_idx,
compare_completions, complete_line, lino_add_completion,
next_hist_match, history_search, show_help,
struct abuf, ab_append, ab_free, sync_data_to_buf,
refresh_singleline, screen_rows, col_offset_in_str,
refresh_multiline, scan_match_rev, scan_match_fwd, scan_fwd,
find_nearest_paren, usec_delay, flash, yank_sel, delete_sel,
edit_insert, edit_insert_str, edit_move_eol,
edit_history_next, edit_delete, edit_backspace,
edit_delete_prev_all, edit_delete_to_eol, edit_delete_line,
edit_in_editor, edit, linenoise, lino_make, lino_cleanup.
lino_free, free_hist, lino_hist_add, lino_hist_save,
lino_set_result): Revised using streams, wide chars and
lino_os interface.
(lino_init): New function.
* linenoise/linenoise.h (LINO_PAD_CHAR): New preprocessor
symbol.
(mem_t): Defined here.
(MEM_T_DEFINED): New preprocessor symbol.
(struct lino_os, lino_os_t): New structure.
(lino_os_init): New macro.
(struct lino_completions, lino_compl_cb_t, lino_atom_cb_t,
lino_enter_cb_t): Switch to wchar_t.
(lino_init): New function.
(lino_add_completion, lino_make, linenoise, lino_hist_add,
lino_hist_save, lino_hist_load, lino_set_result)
* parser.c (find_matching_syms, provide_completions,
provide_atom, is_balanced_line, repl): Adapt to
wide character linenoise.
(lino_fileno, lino_puts, lino_getch, lino_getl, lino_gets,
lino_feof, lino_open, lino_open8, lino_fdopen, lino_close):
New static functions.
(linenoise_txr_binding): New static structure.
(parse_init): Call lino_init, passing OS binding.
* txr.1: Update text about the listener's limitations.
|
|
|
|
|
|
|
|
|
|
| |
* parser.l (unicode_ident): New static function.
(BSCHR, NSCHR): Include UONLY match.
(grammar): Use unicode_ident function to validate tokens
obtained from BTOK and NTOK.
* txr.1: Documented changing definition of bident
and lident.
|
|
|
|
|
|
|
| |
* parser.l (SYM, TOK): These are just aliases for the same
pattern. SYM is referenced only in one place; TOK in multiple
places, so let's remove SYM and replace that one reference
by TOK.
|
|
|
|
| |
* txr.1: Two occurrences of "of of" are corrected.
|
|
|
|
|
| |
* txr.1: Overhaul after a proof-reading of the new LISP
COMPILATION section.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If code does something like this, we produce a warning:
(let ((var 42))
(load-time (+ var 3)))
A load-time form occurs in an empty lexical environment,
so this code appears right but actually cannot work;
the appearance is a mere trompe d'oeil.
* share/txr/stdlib/compiler.tl (misleading-ref-check): New
function.
(compiler comp-load-time-lit): Apply misleading-ref-check.
|
|
|
|
|
|
|
|
|
|
| |
* RELNOTES: Updated.
* configure, txr.1: Bumped version and date.
* share/txr/stdlib/ver.tl: Likewise.
* txr.vim, tl.vim: Regenerated.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Makefile (opt/txr.o, dbg/txr.o): Set up TXR_REL_PATH
preprocessor symbol on command line which specifies the
hard installation path derived from the configure prefix.
* txr.c (get_self_path): The fallback implementation simply
returns TXR_REL_PATH. This will work if in fact the executable
is installed at that path. What it means is that the build of
TXR which uses this fallback get_self_path will not support
relocation: the installation cannot be moved to another
directory, yet still find its supporting files. This is
better than what we are replacing: not working even in the
original installation directory, if not invoked by an absolute
path.
|
|
|
|
|
|
|
|
|
|
|
|
| |
* configure: Add detection for getexecname.
* sysif.c (getcwd_wrap): Change static function to external.
* sysif.h (getcwd_wrap): Declared.
* txr.c (get_self_path): New implementation for Solaris
using getexecname, which requires us to prepend the current
directory name if the result is a relative path.
|
|
|
|
|
|
|
|
|
| |
TXR's fall-back method of getting the executable's own path is
broken in the Mac-OS port, which means that TXR doesn't work
when invoked via PATH search. Mac-OS-specific code is required.
* txr.c (get_self_path): New variant for Darwin, using the
platform-specific function _NSGetExecutablePath.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It looks like I neglected to treat dynamically scoped
variables in destructuring; all parameters are treated as
lexical. This is only true in interpreted code; compiled
destructuring treats dynamics properly because it generates a
let to bind all the variables which occur, and then uses
assignment to populate them.
* eval.c (bind_macro_params): Instead of env_vbind, use the
newly introduced lex_or_dyn_bind helper function.
(op_tree_bind, op_mac_param_bind): Save and restore dyn_env
around bind_macro_params and the evaluation of the associated
body. In op_defmacro, this is already done.
|
|
|
|
|
|
|
|
| |
* eval.c (lex_or_dyn_bind_seq, lex_or_dyn_bind): New static
functions.
(bind_args): Eliminate repeated code using new helper
functions. One wrong repetitions is thereby fixed also: a
a neglected check of dyn_env_made.
|
|
|
|
|
|
|
|
| |
* txr.1: The syntax synopses for the following functions and
macros are too long and are awkwardly wrapped by man under 80
columns: make-time, make-time-utc, make-env,
call-update-expander, call-clobber-expander and txr-if.
Breaking up and shortening some identifiers.
|
|
|
|
|
|
| |
* txr.1: I spotted a problem in the syntax synopsis of remq,
remql and remqual. The same mistake was made in a few other
places. remq* also had an error, though different.
|