| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|
|
|
|
|
|
|
|
|
| |
* parser.c (read_file_common): Treat first form in file as a
version number of the form (major minor), where major nad
minor are non-negative integers. If the major number is
greater than zero, reject the file as incompatible.
* share/txr/stdlib/compiler.tl (usr:compile-file): Emit version
number (0 0) as the first item in a compiled file.
|
|
|
|
|
|
|
|
|
|
|
| |
append is too generic, and produces nil sometimes. Let's use a
custom run-time support function sys:fmt-join.
* eval.c (fmt_join): New function.
(eval_init): Intern sys:fmt-join.
* share/txr/stdlib/compiler.tl (expand-quasi): Generate a
sys:fmt-join call to combine the pieces rather than append.
|
|
|
|
|
|
|
| |
* parser.c (read_file_common): Remove the check whether the
data vector is a list (due to have been converted to
sys:vector-lit by the presence of unquotes). This doesn't
happen any more, thanks to the previous parser commit.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The parser generates a sys:hash-lit, sys:struct-lit or
sys:vector-lit whenever a hash, struct or vector literal
contains unquotes. This allows the quasiquote expander
to treat these objects as ordinary list structure when
interpolating inside them, and then recognize these symbols
and construct the implied real objects.
The issue is that these literals are generated even if the
unquotes occur outside of a backquote. For instance if a
vector literal like #(,a) occurs out of the blue,
not in any backquote, this is still a
(sys:vector-lit (sys:unquote a)) and not an actual vector.
The issue is compounded because this substitution takes
place even if there is no actual comma or splice notation.
Even the following is a sys:vector-lit: #((sys:unquote x)).
In any case, it causes problems for compiled files, because
such material can occur in the data vector of a compiled
toplevel form.
In this patch we modify the parser to keep track of the
quasiquote/unquote level. The special literals are generated
only when the object occurs inside a quasiquote.
* parser.h (struct parser): New member, quasi_level.
* parser.c (parser_common_init): Initialize the parser's new
quasi_level member.
* parser.y (vector, hash, struct): To decide whether to
generate the special literal, don't just check whether
unquotes occur in the list. Check that we are in a quasiquote,
indicated by the quasiquoting level being positive.
(i_expr, n_expr): Use a mid-rule actions on the quasiquote,
unquote and splice rules to bump the quasiquoting level in one
direction before recognizing the object, and then bump in the
opposite direction when reducing the rule.
(parse): Initialize quasi_level.
|
|
|
|
|
|
|
|
| |
The compiler barfed on hash.tl, reporting an if form with too
many arguments in the with-hash-iter macro, indicating that
it's basically broken.
* share/txr/stdlib/hash.tl (with-hash-iter): Repair the macro.
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-quasi-args): When the
element of the quasiquote is a compound form, we can't just
return it. Firstly, because we are appending, we must wrap
each returned item in a list. Secondly, we must wrap the form
in code which formats it into a string consistently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These top-level macro-time forms are preventing code from
being compiled.
* share/txr/stdlib/ifa.tl: Remove macro-time around
sys:if-to-cond function.
* share/txr/stdlib/package.tl: Remove macro-time around
sys:name-str.
* share/txr/stdlib/struct.tl: Remove macro-time around
sys:bad-slot-syntax and sys:prune-missing-inits.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The sys:bind-mac-error function is a run-time support function
for the compiler-generated code for destructuring. But the
compiler.tl source file is a bad place for it. It means that
compiled modules which need sys:bind-mac-error auto-load the
entire compiler. That easily creates an irresolveable cycle,
because the compiler is on top of the foodchain.
No, sys:bind-mac-error belongs in error.tl; the light-weight
module of low-dependency run-time support routines for error
reporting.
* lisplib.c (error_set_entries): Add sys:bind-mac-error symbol
here.
(compiler_set_entries): Remove it from here.
* share/txr/stdlib/compiler.tl (sys:bind-mac-error): Function
removed.
* share/txr/stdlib/error.tl (sys:bind-mac-error): Function
moved here.
|
|
|
|
|
|
|
|
|
|
| |
Compiled code references sys:define-method directly without
going through the struct macrology that would otherwise
trigger the autoload of the struct stuff. The symbol needs
to be added to the set of triggers.
* lisplib.c (struct_set_entries): Add sys:define-method
symbol.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We would like to load the compiled versions of the library
files, if they exist, in preference to the source versions.
So we want to issue loads without the .tl suffix; let the
load function sort it out.
* gencadr.c (cadr_register): Drop .tl suffix.
* lisplib.c (place_instantiate, ver_instantiate,
ifa_instantiate, txr_case_instantiate,
with_resources_instantiate, path_test_instantiate,
struct_instantiate, with_stream_instantiate, hash_instantiate,
except_instantiate, type_instantiate, yield_instantiate,
termios_instantiate, awk_instantiate, build_instantiate,
trace_instantiate, getopts_instantiate, package_instantiate,
getput_instantiate, tagbody_instantiate, pmac_instantiate,
error_instantiate, keyparams_instantiate, ffi_instantiate,
doloop_instantiate, stream_wrap_instantiate, asm_instantiate,
compiler_instantiate, op_instantiate): Likewise.
* cadr.c: Regenerated.
|
|
|
|
|
|
|
|
|
| |
* eval.c (eval_init): The existing variable
sys:*load-recursive* is formally defined. This is necessary if
a form is evaluated from the command line using -p or -e,
which somewhere accesses this variable. txr_main sets up the
variable dynamicaly, but later. It should have an entry for it
in the global environment.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unfortunately, the nfa_state_free function doesn't check the
static flag on a character set and just calls chr_set_destroy.
So when one of the static character sets is planted into the
NFA graph, when that graph is garbage-collected, it blows away
the static character set. Then when that happens twice for the
same set, boom!
We make an alteration to make the destruction more defensive.
Callers of char_set_destroy are no longer saddled with the
responsibility of honoring the static flag buried in the
object. Instead, that function itself check the static flag.
An argument is provided to force the deletion in spite of the
static flag; that is needed for the global cleanup of the
static states. (Only occurs if txr is run with --free-all
and cleanly exited.)
* regex.c (char_set_destroy): Take extra argument, force.
If the set is marked static, then do nothing, unless
force is nonzero.
(char_set_cobj_destroy): Don't check the static flag, just
call char_set_destroy, force zero.
(nfa_state_free): Add force zero argument to char_set_destroy
call. The double free bug is thereby fixed here; static sets
are protected.
(regex_free_all): Force all the char_set_destroy calls here.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because the regex printer wrongly uses out_str_char (for the
sake of borrowing its semicolon-notation processing) when
a regex prints, all characters that require escaping in a
string literal get escaped, which includes the " character.
Unfortunately the \" sequence which results is rejected
by the regex parser.
* lib.c (out_str_char): Kludge: add extra argument to
distinguish regex use versus string use, and treat the double
quote accordingly.
(out_str_readable): Give 0 arg to new param of out_str_char.
* lib.h (out_str_char): Declaration updated.
* regex.c (print_class_char, print_rec): Pass 1 to new param
of out_str_char.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* eval.c (load): If open_txr_file indicates compiled file by
setting txr_lisp_p to character #\o, use read_compiled_file.
* match.c (v_load): Likewise.
* parser.c (open_txr_file): Recognize the .tlo suffix, and
also try to open a .tlo version of an unsuffixed file before
trying it as .tl. Indicate a .tlo file by setting txr_lisp_p
to the character #\o rather than t.
(read_file_common): New static function, made from contents of
read_eval_stream. Will either evaluate forms or load compiled
code by instantiating virtual machine descriptions and
performing their top-level execution.
(read_eval_stream): Now a wrapper for read_file_common.
(read_compiled_file): New function.
* parser.h (read_compiled_file): Declared.
* txr.c (help): List new --compiled option.
(txr_main): If --compiled is specified, set txr_lisp_p to #\o
to load as compiled code. Update error message that -c is
not compatible with --lisp or --compiled.
If txr_lisp_p is #\o, then use read_compiled_file.
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/asm.tl (assembler): Wrap with compile-only
so we don't redefine this struct while we are compiling
the assembler.
(%oc-code%): Use defparml so this is reset to zero, so that
the opcodes are numbered from zero instead of the most
recent value left in the variable.
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl: Skip top-level forms which are
just constants. Their value is discarded and they have no
effect.
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-tree-case): The
block name isn't a lexical variable; do not add it to nenv via
extend-var. Just bind to gensym directly and insert where
needed. This reduces the run-time frame size by one entry.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Going forward, when : appears as the argument of an optional
parameter in destructuring, it is treated as an ordinary
object, and not as "this argument is missing". That is now a
feature of function calls only.
Rationale: interferes with macros. For instance, the pattern
(test : then else) doesn't correctly express the arguments
of if, because (if x y :) eats the semicolon.
The defaulting behavior is not useful because usually there
is no default value for optional structure, other than nil.
* eval.c (bind_macro_params): Only implement the colon hack
under compatibility with 190.
* share/txr/stdlib/place.tl (defplace gethash, defplace fun,
defplace symbol-function, defplace symbol-macro, defplace
symbol-value): Remove uses of : for defaulting the ssetter
argument. This illustrates how useless the feature is for
macro destructuring; all these just replace with nil.
* txr.1: Clarify that macro parameter lists don't implement
the colon trick. It was never explicitly specified that this
is the case, but could have been inferred from the statements
which indicate that macro parameter lists copy the features of
function parameter lists. Added compat notes.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Uses of the macro-time form are not useful in this module, and
will hinder compilation, since forms evaluated by macro-time
are not seen by the compiler and thus cannot be emitted in
compiled form into a compiled file.
* share/txr/stdlib/place.tl: Remove all top-level occurrences
of macro-time.
(defplace): Remove macro-time emitted in expansion, replacing
it by progn.
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-defmacro): We cannot
simply generate a mac-param-bind form because that has no
means to pass through the all-important macro environment
parameter; its expansion is carried out by
expand-bind-mac-params with an nil environment parameter.
Instead, we retarget to use expand-bind-mac-params directly,
and supply the parameter gensym.
|
|
|
|
|
|
|
|
|
|
|
| |
* parser.y (buflit, buflit_items): Don't cons up a list of
bytes in buflit_items which are then assembled into a
buffer. Rather, the buflit_items rules construct and fill a
buffer object directly. The buflit rule then just has to
signal the end of the buffer literal to the lexer, and trim
the buffer to the actual size. We will need this for
efficient loading of compiled files, in which the virtual
machine code is represented as a buffer literal.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Suppose that we have two symbols of the same name, in two
packages: foo:sym and bar:sym. Suppose that the bar package
has foo in its package fallback list, and suppose bar is the
current package. Then bar:sym prints without a package prefix,
as just sym. However, this is potentially ambiguous. Suppose
that bar:sym is written to a file as just sym. Then later the
file is read into a fresh image in a situation in which
bar:sym has not yet been interned, but foo:sym already exists.
In this situation, sym will just resolve to foo:sym.
The printer must detect this ambiguous situation. If a symbol
is present in a package, but a same-named symbol is in the
fallback list; or if a symbol is visible in the fallback list,
but a same-named symbol is present in the package, then
a package prefix should be printed.
* lib.c (symbol_needs_prefix): New function.
(unquote_star_check, obj_print_impl): Use symbol_needs_prefix
rather than symbol_visible.
* lib.h (symbol_needs_prefix): Declared.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lisplib.c (compiler_set_entries): Autoload on compile-file.
* parser.c (parse_init): Expose get-parser, parser-errors and
parser-eof intrinsics in system package.
* share/txr/stdlib/compiler.tl (compiler): Wrap defstruct form
in compile-only. What this means is that when we invoke
comile-file on compiler.tl, the compiler will only compile
this defstruct and not evaluate it. I.e. it will not try to
redefine the structure. Redefining the core structure of the
compiler while it is compiling itself wreaks havoc on the
compilation.
(%fille-suff-rx%, *emit*, *eval*): New variables.
(open-compile-streams, list-from-vm-desc, usr:compile-file):
New functions.
* vm.c (vm_desc_nlevels, vm_desc_nregs): New static functions.
(vm_init): Register new intrinsics vm-desc-nlevels and
vn-desc-nregs in system package.
|
|
|
|
|
| |
* share/txr/stdlib/asm.tl, share/txr/stdlib/compiler.tl: Just
use (in-package sys) rather than (in-package :sys).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These forms will be specially recognized by the file compiler
when they appear as top-level forms. eval-only will mean this:
only execute this form (possibly after compiling it);
do not emit any compiled code into the output file.
compile-only will mean: only emit the compiled code into the
output file; do not execute it.
* eval.c (eval_init): Register special operators compile-only
and eval-only. In the interpreter, these are equivalent to
progn and so route to op_progn.
* share/txr/stdlib/compiler.tl (compiler compile): Similarly
to interpreter, handle compile-only and eval-only as progn.
|
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-cond): Add
handling for the case when the cond clause contains only a
test form, and no additional forms. In that case, if the
value is true, then cond terminates with that value. We can
nicely achieve this by using or. This was uncovered while
compiling share/txr/stdlib/awk.tl, which contains such a
case in (sys:awk-state loop) that handles opening the
input source.
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-block): Our output frag
indicates that oreg is the output register, and in the case
when the block is terminated by the dynamic return, that's
where it stores the result value before branching to the skip
address. In the ordinary termination case, we have neglected
to move the output of the block's code from bfrag.oreg to
the output register oreg.
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-unwind-protect): The
output register of the compiled unwind-protect is that
of the protected forms compiled in pfrag, not of the
compiled code for the cleanup forms in cfrag.
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-switch): When compiling
the shared case of switch (generated by tagbody), we must
identify the last fragment's output register and move that to
oreg.
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-switch): Emit the code
for evaluating the switch index. This has been working by
fluke, because when the index is a variable, the assembly
code for ifrag is nil; the swtch instruction accesses the
variable directly and all is cool.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a smattering of incorrect logic affecting a number of
the compiler's special form sub-compilers. Basically the issue
is that a compiler routine cannot arbitrarily use the oreg
that it is given. If it generates multiple instructions which
clobber a destination, only the last clobber may target oreg.
The reason is that oreg is not necessarily a fresh temporary
that can be used arbitrarily. It can be a variable which is
evaluated by the forms that are compiled by the sub-compiler.
Prematurely storing a value into oreg can affect the behavior
and result value of not-yet executed code. The recent
"indirect function calls" fix addressed just one case of this;
the problem is more wide-spread.
* share/txr/stdlib/compiler.tl
(compiler (maybe-alloc-treg, maybe-free-treg)): New methods.
(compiler (comp-if, comp-switch, comp-unwind-protect,
comp-block, comp-handler-bind, comp-catch, comp-let,
comp-progn, comp-and-or, comp-prog1, comp-for, comp-call,
comp-tree-case): Do not carelessly use oreg for intermediate
computations; allocate one or more temporary registers for
that purpose, and either only move the final value into oreg,
or else indicate a temporary register as the returned frag's
output register.
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-lambda): We must
not deduce the frame size from the nenv v-counter, because in
the case when no frame is needed (the function has no
arguments), nenv is just env. Our need-frame variable
indicates whether or not a frame is needed; if not, we must
use size zero. This is crucial because the VM otherwise ends
up throwing an exception due to the frame level mismatch.
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-return-from,
comp-return): The value expression is optional in all the
return operators.
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-call): We must not use
the output register oreg for compiling the expression which
calculates the function because it could be one of the
arguments. In that case, we clobber an argument before we
have called the function.
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (expand-bind-mac-params): When
the strict parameter is the keyword symbol : we are
mis-translating the length check. We are ignoring the presence
of the rest-par, and checking for an exact length.
When rest-par is present, we must check only for a minimum
number of fixed parameters.
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-catch): The emitted code
doesn't handle the normal non-exception path. It assumes that
only the frame established with the earlier frame instruction
needs to be terminated; but in fact the catch intruction's
frame has to be ended also. Thus we don't need the frame-end
label; all exit cases jump too the handler-end label in front
of the two end instructions.
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (comp-lambda): The init-forms
for optional parameters in a lambda must be compiled in the
environment in which prior arguments are visible.
|
|
|
|
|
|
|
|
|
|
|
| |
* eval.c (expand_params_rec): Add the parameter to the macro-time
environment before processing rest of parameter list. This is
already done for all the symbols of a macro-style
destructuring; just not for a simple parameter. This is
necessary, because the init forms of optional parameters occur
in a lexical environment in which prior parameters are
visible. The test case for this is that (lambda (x : (y x)))
must not produce a warning about unbound x.
|
|
|
|
|
|
|
|
|
| |
* txr.1: the each, append-each and all other variants of
operators exhibit inifite looping if the list of bindings is
empty. In that case, the terminating condition that at least
one list is empty, is never true. The behavior is consistent
under compilation and interpretation, so let's just
document it.
|
|
|
|
|
|
| |
* eval.c (do_expand): When traversing (fun ...)
operator, warn if the function isn't defined or if it is being
applied to a special operator.
|
|
|
|
|
|
|
| |
* txr.1: The described behavior regarding null bytes output
into string output streams is incorrect. In fact they are
effectively dropped by the put-char operation. Let's not
commit to any specific behavior. Also mention pseudo-null.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Originally, string output streams would null out their handle
when the string was extracted. This changed in commit
e44c113ee17c7cf15e8b1891f4d51ec03b16bc24 [Jul 2015], so that
just the string buffer was nulled out. Unfortunately, two
places in the code were not updated, still checking for a null
handle, which is always false, and not defending against the
null handle.
* stream.c (string_out_extracted_error): New static function.
(string_out_put_string, string_out_put_byte): Check the buffer
for null, not the handle, which doesn't go null while the
object is live. Throw an exception rather than returning nil.
|
|
|
|
| |
* txr.1: Example isn't about swapping variables.
|
|
|
|
| |
* txr.1: A let in the example should be let*.
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler compile): All
special forms are handled, so "not handled yet" wording is
inappropriate. Going forward, no special form will be added
without compiler support.
|
|
|
|
|
|
| |
* unwind.c (revive_cont): If delta is zero, skip the loop.
This is an important optimization. The delta zero case
can occur frequently; I have observed it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It doesn't Just Work out of the box. Here is why.
The struct vm state is declared somewhere on the stack,
and then the vm executes various recursive functions.
A block is established by vm_block(). Then if a continuation
is captured up to that block, it will not include that part
of the stack which holds the VM state. When the continuation
is restarted, the struct vm * pointers inside the stack will
be garbage.
The solution: when a continuation is captured to a prompt that
is set up by the VM block instruction (vm_block function),
we put information into the block which says "don't just
capture me plus some slack: please capture the stack all the
way up to this specific address here". That address, of
course, is just past the VM state, ensuring that the VM state
is included in the snapshot.
In short: a delimited continuation terminating in a prompt
set up by the VM just include the entire VM context from the
stack frame where the struct vm on down (i.e. up the stack)
to the capture point.
* unwind.c (uw_push_block): Initialize cont_bottom
member to zero. Interpreted blocks leave this as zero.
Blocks set up by the VM override it.
(revive_cont): Critical: the range check must include
the orig_end endpoint, because when the struct vm is captured
into the continuation it is right at the end of the capture
memory, and the prompt block's cont_bottom member points
exactly to orig_end: one element past the struct vm.
The cont_bottom member must be adjusted by delta
so that continuations captured by the revived VM
inside the continuation get the adjusted cont_bottom at their
current stack location.
(capture_cont): If the block is publishing a non-zer
cont_bottom value, then take that value if it is a higher
address than the lim (including the UW_CONT_FRAME_AFTER
slack).
* unwind.h (struct uw_block): New member, cont_bottom.
By means of this, a block can indicate a higher address
to which a continuation capture can extend.
* vm.c (vm_block): Publish the address one byte past the
virtual machine state as the stack bottom for continuations.
|
|
|
|
|
| |
* txr.1: defmacro, macrolet and mac-param-bind syntax
synopses are too wide under 80 column "man txr": breaking up.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler comp-call-impl):
Instead of allocating N temporary registers for N arguments,
some (or even none) of which may actually be used, we
do this one argument at a time: allocate just one register,
compile the argument expression, and then free the register
immediately if that fragment specifies its own output location
instead of the register. Otherwise keep the register and push
it on a stack. This strategy lowers maximum register use.
Also, since we are pushing the used registers on a stack,
when we call free-tregs, they get liberated in reverse
order of allocation, which keeps things tidy.
|
|
|
|
|
|
|
| |
* share/txr/stdlib/compiler.tl (compiler check-treg-leak): New
method.
(usr:compile-toplevel): Free the top-level output register,
then call check-treg-leak to verify all were returned.
|