| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The sys:bind-mac-error function is a run-time support function
for the compiler-generated code for destructuring. But the
compiler.tl source file is a bad place for it. It means that
compiled modules which need sys:bind-mac-error auto-load the
entire compiler. That easily creates an irresolveable cycle,
because the compiler is on top of the foodchain.
No, sys:bind-mac-error belongs in error.tl; the light-weight
module of low-dependency run-time support routines for error
reporting.
* lisplib.c (error_set_entries): Add sys:bind-mac-error symbol
here.
(compiler_set_entries): Remove it from here.
* share/txr/stdlib/compiler.tl (sys:bind-mac-error): Function
removed.
* share/txr/stdlib/error.tl (sys:bind-mac-error): Function
moved here.
|
| |
|
|
|
|
|
|
|
|
| |
Compiled code references sys:define-method directly without
going through the struct macrology that would otherwise
trigger the autoload of the struct stuff. The symbol needs
to be added to the set of triggers.
* lisplib.c (struct_set_entries): Add sys:define-method
symbol.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We would like to load the compiled versions of the library
files, if they exist, in preference to the source versions.
So we want to issue loads without the .tl suffix; let the
load function sort it out.
* gencadr.c (cadr_register): Drop .tl suffix.
* lisplib.c (place_instantiate, ver_instantiate,
ifa_instantiate, txr_case_instantiate,
with_resources_instantiate, path_test_instantiate,
struct_instantiate, with_stream_instantiate, hash_instantiate,
except_instantiate, type_instantiate, yield_instantiate,
termios_instantiate, awk_instantiate, build_instantiate,
trace_instantiate, getopts_instantiate, package_instantiate,
getput_instantiate, tagbody_instantiate, pmac_instantiate,
error_instantiate, keyparams_instantiate, ffi_instantiate,
doloop_instantiate, stream_wrap_instantiate, asm_instantiate,
compiler_instantiate, op_instantiate): Likewise.
* cadr.c: Regenerated.
|
| |
|
|
|
|
|
|
|
| |
* eval.c (eval_init): The existing variable
sys:*load-recursive* is formally defined. This is necessary if
a form is evaluated from the command line using -p or -e,
which somewhere accesses this variable. txr_main sets up the
variable dynamicaly, but later. It should have an entry for it
in the global environment.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unfortunately, the nfa_state_free function doesn't check the
static flag on a character set and just calls chr_set_destroy.
So when one of the static character sets is planted into the
NFA graph, when that graph is garbage-collected, it blows away
the static character set. Then when that happens twice for the
same set, boom!
We make an alteration to make the destruction more defensive.
Callers of char_set_destroy are no longer saddled with the
responsibility of honoring the static flag buried in the
object. Instead, that function itself check the static flag.
An argument is provided to force the deletion in spite of the
static flag; that is needed for the global cleanup of the
static states. (Only occurs if txr is run with --free-all
and cleanly exited.)
* regex.c (char_set_destroy): Take extra argument, force.
If the set is marked static, then do nothing, unless
force is nonzero.
(char_set_cobj_destroy): Don't check the static flag, just
call char_set_destroy, force zero.
(nfa_state_free): Add force zero argument to char_set_destroy
call. The double free bug is thereby fixed here; static sets
are protected.
(regex_free_all): Force all the char_set_destroy calls here.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because the regex printer wrongly uses out_str_char (for the
sake of borrowing its semicolon-notation processing) when
a regex prints, all characters that require escaping in a
string literal get escaped, which includes the " character.
Unfortunately the \" sequence which results is rejected
by the regex parser.
* lib.c (out_str_char): Kludge: add extra argument to
distinguish regex use versus string use, and treat the double
quote accordingly.
(out_str_readable): Give 0 arg to new param of out_str_char.
* lib.h (out_str_char): Declaration updated.
* regex.c (print_class_char, print_rec): Pass 1 to new param
of out_str_char.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* eval.c (load): If open_txr_file indicates compiled file by
setting txr_lisp_p to character #\o, use read_compiled_file.
* match.c (v_load): Likewise.
* parser.c (open_txr_file): Recognize the .tlo suffix, and
also try to open a .tlo version of an unsuffixed file before
trying it as .tl. Indicate a .tlo file by setting txr_lisp_p
to the character #\o rather than t.
(read_file_common): New static function, made from contents of
read_eval_stream. Will either evaluate forms or load compiled
code by instantiating virtual machine descriptions and
performing their top-level execution.
(read_eval_stream): Now a wrapper for read_file_common.
(read_compiled_file): New function.
* parser.h (read_compiled_file): Declared.
* txr.c (help): List new --compiled option.
(txr_main): If --compiled is specified, set txr_lisp_p to #\o
to load as compiled code. Update error message that -c is
not compatible with --lisp or --compiled.
If txr_lisp_p is #\o, then use read_compiled_file.
|
| |
|
|
|
|
|
|
|
| |
* share/txr/stdlib/asm.tl (assembler): Wrap with compile-only
so we don't redefine this struct while we are compiling
the assembler.
(%oc-code%): Use defparml so this is reset to zero, so that
the opcodes are numbered from zero instead of the most
recent value left in the variable.
|
| |
|
|
|
|
| |
* share/txr/stdlib/compiler.tl: Skip top-level forms which are
just constants. Their value is discarded and they have no
effect.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-tree-case): The
block name isn't a lexical variable; do not add it to nenv via
extend-var. Just bind to gensym directly and insert where
needed. This reduces the run-time frame size by one entry.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Going forward, when : appears as the argument of an optional
parameter in destructuring, it is treated as an ordinary
object, and not as "this argument is missing". That is now a
feature of function calls only.
Rationale: interferes with macros. For instance, the pattern
(test : then else) doesn't correctly express the arguments
of if, because (if x y :) eats the semicolon.
The defaulting behavior is not useful because usually there
is no default value for optional structure, other than nil.
* eval.c (bind_macro_params): Only implement the colon hack
under compatibility with 190.
* share/txr/stdlib/place.tl (defplace gethash, defplace fun,
defplace symbol-function, defplace symbol-macro, defplace
symbol-value): Remove uses of : for defaulting the ssetter
argument. This illustrates how useless the feature is for
macro destructuring; all these just replace with nil.
* txr.1: Clarify that macro parameter lists don't implement
the colon trick. It was never explicitly specified that this
is the case, but could have been inferred from the statements
which indicate that macro parameter lists copy the features of
function parameter lists. Added compat notes.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Uses of the macro-time form are not useful in this module, and
will hinder compilation, since forms evaluated by macro-time
are not seen by the compiler and thus cannot be emitted in
compiled form into a compiled file.
* share/txr/stdlib/place.tl: Remove all top-level occurrences
of macro-time.
(defplace): Remove macro-time emitted in expansion, replacing
it by progn.
|
| |
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-defmacro): We cannot
simply generate a mac-param-bind form because that has no
means to pass through the all-important macro environment
parameter; its expansion is carried out by
expand-bind-mac-params with an nil environment parameter.
Instead, we retarget to use expand-bind-mac-params directly,
and supply the parameter gensym.
|
| |
|
|
|
|
|
|
|
|
|
| |
* parser.y (buflit, buflit_items): Don't cons up a list of
bytes in buflit_items which are then assembled into a
buffer. Rather, the buflit_items rules construct and fill a
buffer object directly. The buflit rule then just has to
signal the end of the buffer literal to the lexer, and trim
the buffer to the actual size. We will need this for
efficient loading of compiled files, in which the virtual
machine code is represented as a buffer literal.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Suppose that we have two symbols of the same name, in two
packages: foo:sym and bar:sym. Suppose that the bar package
has foo in its package fallback list, and suppose bar is the
current package. Then bar:sym prints without a package prefix,
as just sym. However, this is potentially ambiguous. Suppose
that bar:sym is written to a file as just sym. Then later the
file is read into a fresh image in a situation in which
bar:sym has not yet been interned, but foo:sym already exists.
In this situation, sym will just resolve to foo:sym.
The printer must detect this ambiguous situation. If a symbol
is present in a package, but a same-named symbol is in the
fallback list; or if a symbol is visible in the fallback list,
but a same-named symbol is present in the package, then
a package prefix should be printed.
* lib.c (symbol_needs_prefix): New function.
(unquote_star_check, obj_print_impl): Use symbol_needs_prefix
rather than symbol_visible.
* lib.h (symbol_needs_prefix): Declared.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lisplib.c (compiler_set_entries): Autoload on compile-file.
* parser.c (parse_init): Expose get-parser, parser-errors and
parser-eof intrinsics in system package.
* share/txr/stdlib/compiler.tl (compiler): Wrap defstruct form
in compile-only. What this means is that when we invoke
comile-file on compiler.tl, the compiler will only compile
this defstruct and not evaluate it. I.e. it will not try to
redefine the structure. Redefining the core structure of the
compiler while it is compiling itself wreaks havoc on the
compilation.
(%fille-suff-rx%, *emit*, *eval*): New variables.
(open-compile-streams, list-from-vm-desc, usr:compile-file):
New functions.
* vm.c (vm_desc_nlevels, vm_desc_nregs): New static functions.
(vm_init): Register new intrinsics vm-desc-nlevels and
vn-desc-nregs in system package.
|
| |
|
|
|
| |
* share/txr/stdlib/asm.tl, share/txr/stdlib/compiler.tl: Just
use (in-package sys) rather than (in-package :sys).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These forms will be specially recognized by the file compiler
when they appear as top-level forms. eval-only will mean this:
only execute this form (possibly after compiling it);
do not emit any compiled code into the output file.
compile-only will mean: only emit the compiled code into the
output file; do not execute it.
* eval.c (eval_init): Register special operators compile-only
and eval-only. In the interpreter, these are equivalent to
progn and so route to op_progn.
* share/txr/stdlib/compiler.tl (compiler compile): Similarly
to interpreter, handle compile-only and eval-only as progn.
|
| |
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-cond): Add
handling for the case when the cond clause contains only a
test form, and no additional forms. In that case, if the
value is true, then cond terminates with that value. We can
nicely achieve this by using or. This was uncovered while
compiling share/txr/stdlib/awk.tl, which contains such a
case in (sys:awk-state loop) that handles opening the
input source.
|
| |
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-block): Our output frag
indicates that oreg is the output register, and in the case
when the block is terminated by the dynamic return, that's
where it stores the result value before branching to the skip
address. In the ordinary termination case, we have neglected
to move the output of the block's code from bfrag.oreg to
the output register oreg.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-unwind-protect): The
output register of the compiled unwind-protect is that
of the protected forms compiled in pfrag, not of the
compiled code for the cleanup forms in cfrag.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-switch): When compiling
the shared case of switch (generated by tagbody), we must
identify the last fragment's output register and move that to
oreg.
|
| |
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-switch): Emit the code
for evaluating the switch index. This has been working by
fluke, because when the index is a variable, the assembly
code for ifrag is nil; the swtch instruction accesses the
variable directly and all is cool.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a smattering of incorrect logic affecting a number of
the compiler's special form sub-compilers. Basically the issue
is that a compiler routine cannot arbitrarily use the oreg
that it is given. If it generates multiple instructions which
clobber a destination, only the last clobber may target oreg.
The reason is that oreg is not necessarily a fresh temporary
that can be used arbitrarily. It can be a variable which is
evaluated by the forms that are compiled by the sub-compiler.
Prematurely storing a value into oreg can affect the behavior
and result value of not-yet executed code. The recent
"indirect function calls" fix addressed just one case of this;
the problem is more wide-spread.
* share/txr/stdlib/compiler.tl
(compiler (maybe-alloc-treg, maybe-free-treg)): New methods.
(compiler (comp-if, comp-switch, comp-unwind-protect,
comp-block, comp-handler-bind, comp-catch, comp-let,
comp-progn, comp-and-or, comp-prog1, comp-for, comp-call,
comp-tree-case): Do not carelessly use oreg for intermediate
computations; allocate one or more temporary registers for
that purpose, and either only move the final value into oreg,
or else indicate a temporary register as the returned frag's
output register.
|
| |
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-lambda): We must
not deduce the frame size from the nenv v-counter, because in
the case when no frame is needed (the function has no
arguments), nenv is just env. Our need-frame variable
indicates whether or not a frame is needed; if not, we must
use size zero. This is crucial because the VM otherwise ends
up throwing an exception due to the frame level mismatch.
|
| |
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-return-from,
comp-return): The value expression is optional in all the
return operators.
|
| |
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-call): We must not use
the output register oreg for compiling the expression which
calculates the function because it could be one of the
arguments. In that case, we clobber an argument before we
have called the function.
|
| |
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-bind-mac-params): When
the strict parameter is the keyword symbol : we are
mis-translating the length check. We are ignoring the presence
of the rest-par, and checking for an exact length.
When rest-par is present, we must check only for a minimum
number of fixed parameters.
|
| |
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-catch): The emitted code
doesn't handle the normal non-exception path. It assumes that
only the frame established with the earlier frame instruction
needs to be terminated; but in fact the catch intruction's
frame has to be ended also. Thus we don't need the frame-end
label; all exit cases jump too the handler-end label in front
of the two end instructions.
|
| |
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-lambda): The init-forms
for optional parameters in a lambda must be compiled in the
environment in which prior arguments are visible.
|
| |
|
|
|
|
|
|
|
|
|
| |
* eval.c (expand_params_rec): Add the parameter to the macro-time
environment before processing rest of parameter list. This is
already done for all the symbols of a macro-style
destructuring; just not for a simple parameter. This is
necessary, because the init forms of optional parameters occur
in a lexical environment in which prior parameters are
visible. The test case for this is that (lambda (x : (y x)))
must not produce a warning about unbound x.
|
| |
|
|
|
|
|
|
|
| |
* txr.1: the each, append-each and all other variants of
operators exhibit inifite looping if the list of bindings is
empty. In that case, the terminating condition that at least
one list is empty, is never true. The behavior is consistent
under compilation and interpretation, so let's just
document it.
|
| |
|
|
|
|
| |
* eval.c (do_expand): When traversing (fun ...)
operator, warn if the function isn't defined or if it is being
applied to a special operator.
|
| |
|
|
|
|
|
| |
* txr.1: The described behavior regarding null bytes output
into string output streams is incorrect. In fact they are
effectively dropped by the put-char operation. Let's not
commit to any specific behavior. Also mention pseudo-null.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Originally, string output streams would null out their handle
when the string was extracted. This changed in commit
e44c113ee17c7cf15e8b1891f4d51ec03b16bc24 [Jul 2015], so that
just the string buffer was nulled out. Unfortunately, two
places in the code were not updated, still checking for a null
handle, which is always false, and not defending against the
null handle.
* stream.c (string_out_extracted_error): New static function.
(string_out_put_string, string_out_put_byte): Check the buffer
for null, not the handle, which doesn't go null while the
object is live. Throw an exception rather than returning nil.
|
| |
|
|
| |
* txr.1: Example isn't about swapping variables.
|
| |
|
|
| |
* txr.1: A let in the example should be let*.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler compile): All
special forms are handled, so "not handled yet" wording is
inappropriate. Going forward, no special form will be added
without compiler support.
|
| |
|
|
|
|
| |
* unwind.c (revive_cont): If delta is zero, skip the loop.
This is an important optimization. The delta zero case
can occur frequently; I have observed it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It doesn't Just Work out of the box. Here is why.
The struct vm state is declared somewhere on the stack,
and then the vm executes various recursive functions.
A block is established by vm_block(). Then if a continuation
is captured up to that block, it will not include that part
of the stack which holds the VM state. When the continuation
is restarted, the struct vm * pointers inside the stack will
be garbage.
The solution: when a continuation is captured to a prompt that
is set up by the VM block instruction (vm_block function),
we put information into the block which says "don't just
capture me plus some slack: please capture the stack all the
way up to this specific address here". That address, of
course, is just past the VM state, ensuring that the VM state
is included in the snapshot.
In short: a delimited continuation terminating in a prompt
set up by the VM just include the entire VM context from the
stack frame where the struct vm on down (i.e. up the stack)
to the capture point.
* unwind.c (uw_push_block): Initialize cont_bottom
member to zero. Interpreted blocks leave this as zero.
Blocks set up by the VM override it.
(revive_cont): Critical: the range check must include
the orig_end endpoint, because when the struct vm is captured
into the continuation it is right at the end of the capture
memory, and the prompt block's cont_bottom member points
exactly to orig_end: one element past the struct vm.
The cont_bottom member must be adjusted by delta
so that continuations captured by the revived VM
inside the continuation get the adjusted cont_bottom at their
current stack location.
(capture_cont): If the block is publishing a non-zer
cont_bottom value, then take that value if it is a higher
address than the lim (including the UW_CONT_FRAME_AFTER
slack).
* unwind.h (struct uw_block): New member, cont_bottom.
By means of this, a block can indicate a higher address
to which a continuation capture can extend.
* vm.c (vm_block): Publish the address one byte past the
virtual machine state as the stack bottom for continuations.
|
| |
|
|
|
| |
* txr.1: defmacro, macrolet and mac-param-bind syntax
synopses are too wide under 80 column "man txr": breaking up.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-call-impl):
Instead of allocating N temporary registers for N arguments,
some (or even none) of which may actually be used, we
do this one argument at a time: allocate just one register,
compile the argument expression, and then free the register
immediately if that fragment specifies its own output location
instead of the register. Otherwise keep the register and push
it on a stack. This strategy lowers maximum register use.
Also, since we are pushing the used registers on a stack,
when we call free-tregs, they get liberated in reverse
order of allocation, which keeps things tidy.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler check-treg-leak): New
method.
(usr:compile-toplevel): Free the top-level output register,
then call check-treg-leak to verify all were returned.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, the case macros (caseq, caseql, casequal,
caseq*, caseql* and casequal*) all translate to a cond
statement which tries the cases one by one.
With this change, larger cases are translated to
a lookup through a hash table, which produces an
integer value. The integer value is then used as the
index in an op:switch form for table lookup dispatch.
If the hash lookup fails, then the else-clause is
evaluated.
op:switch is handled efficiently in the interpreter, and
turned into an efficient swtch VM instruction by the new
compiler.
* eval.c (me_case): Add variables and logic to the function
such that while it gathers the materials for the cond-based
translation, it also builds materials for a hash-switch-based
translation. Then, at the end, a decision is made by looking
at how many keys there are and other factors.
Because we don't have hash tables based on the eq function,
but only eql, we must be careful not to turn caseq into
hash lookup, unless we verify that the keys which occur
are fixnum integers, characters or symbols.
|
| |
|
|
|
|
| |
* lib.c (rexpt): New static function.
(exptv): Create reversed arguments on the stack, then
process with nary_op and the rexpt shim.
|
| |
|
|
|
|
| |
* args.c (args_copy_reverse): New function.
* args.h (args_copy_reverse): Declared.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Using reduce-left is inefficient; it conses up a list.
We can decimate the stacked arguments without consing.
* lib.c (nary_op): Replace reduce_left with iteration.
(nary_simple_op): New function, variant of nary_op
useable by functions that have a mandatory argument
passed separately from the argument list.
(minusv, divv): Replace reduce_left with iteration.
(maxv, minv): Replace reduce_left with nary_simple_op.
(abso_self): New static function.
(gcdv, lcmv): Replace reduce_left with nary_op.
* lib.h (nary_simple_op): Declared.
|
| |
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-bind-mac-params): The
listp function identifies a recursive parameter, not consp,
because nil is an empty pattern, and not a variable name,
in macro-style parameter lists.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Due to a number of changes since December 2016, we have two
problems in the evaluator: (lambda (nil)) is accepted rather
than complaining that nil is not bindable; and
(tree-bind (()) '(3)) silently proceeds rather than
diagnosing that the atom 3 doesn't match the empty pattern ().
* eval.c (expand_opt_params_rec, expand_params_rec): When not
expandin macro-style params, check for a parameter not being
bindable, and also avoid recursion entirely when not doing
macro style.
(bind_macro_params): Don't assume that an atom parameter is
a variable, but rather tha a non-list parameter is
a variable. Otherwise we bind nil rather than treating it
as an empty sub-pattern. Before the December 2016 change
(744340ab), we were checking bindablep here; the idea was to
hoist the detailed checking to expansion time. But then the
pattern versus variable distinction was bungled here.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We use the ifq instruction for compiling (if (eq x y) ...)
and (if (neq x y ...) rather than emitting a call to
eq or neq.
* share/txr/stdlib/compiler.tl (%test-funs-pos%,
%test-funs-neg%, %test-funs%, %test-inv%): New global
%variables.
(comp-if): Recognize when test expression is one of the tests
in %test-funs% and transform into (sys:ift ...) syntax.
Also, minor unrelated change here: (if test) optimized away to
nil if test is a constant expression.
(compiler comp-ift): New method.
(compiler comp-fun-form): Recognize sys:ift in operator
position; hand off to comp-ift.
|
