| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
| |
* gzio.c (gzio_stream_print, gzio_get_prop): Remove unused locals.
|
| |
|
|
|
|
|
|
|
| |
* buf.c (buf_compress, buf_decompress): New static functions.
(buf_init): buf-compress and buf-decompress intrinsics registered.
* txr.1: Documented.
* stdlib/doc-syms.tl: Updated.
|
| |
|
|
| |
* buf.c (utf8_dup_to_buf): stray declaration removed.
|
| |
|
|
|
|
|
|
| |
* tags.tl: File renamed to txrtags.tl.
* libtags.txr: load "txrtags" rather than "tags". And here is where
we can explain the problem being fixed: (load "tags") loads the
actual tags file, if it already exists, and not tags.tl.
|
| |
|
|
|
|
| |
* stream.c (se_putc): Remove a workaround for a Cygwin bug that was
fixed in 2016 in 2.5.0. Here is the mailing list thread:
https://sourceware.org/pipermail/cygwin/2016-March/226554.html
|
| |
|
|
|
| |
* txr.1: In all places that mention .tlo in relation to loading,
mention support for .tlo.gz suffix.
|
| |
|
|
|
|
| |
* parser.c (open_txr_file): If a file ends in .tlo.gz,
open a gzio stream on it. Also search for .tlo.gz if
.tlo is not found.
|
| |
|
|
|
| |
* gc.c (mark_obj_maybe): Avoid declaration after statement,
which is enforced when configured in --maintainer mode.
|
| |
|
|
|
|
| |
* gzio.c (gzio_stream_print): New static function.
(gzio_ops_rd, gzio_ops_wr): Wire in function instead of
stream_print_op.
|
| |
|
|
|
|
| |
* gzio.c (gzio_get_prop): Don't play games with a computed :name;
this property is relied upon for getting the path associated
with an open file.
|
| |
|
|
|
| |
* txr.1: Document the "z" option letter, its optional digit arg,
and the restrictions.
|
| |
|
|
|
|
|
| |
* gzio.c (gzio_get_prop, gzio_set_prop): Handle :byte-oriented
symbol, connecting it to the is_byte_oriented flag.
This is already implemented, by copy and paste from the
stdio routines.
|
| |
|
|
|
| |
* gzio.c (struct gzio_handle): Remove is_real_time flag.
(make_gzio_stream): Remove initialization of flag.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is motivated by the desired to be able to
specify gzip compression: e.g.
(file-put-string "file.gz" "abc" "z")
However, it has other uses. For instance with "x",
we can exclusively create a file with content in one
call.
* stdlib/getput.tl (file-get, file-put, file-append,
file-get-string, file-put-string, file-append-string,
file-put-lines, file-append-lines, file-get-buf,
file-put-buf, file-place-buf, file-append-buf, file-get-json,
file-put-json, file-append-json, file-get-jsons,
file-put-jsons, file-append-jsons): New option argument
which combines with the open-file mode.
* txr.1: Documented.
|
| |
|
|
|
| |
* stream.c (open_fileno): Use w_gzdopen_mode and make_gzio_stream
to make a gzio stream if the gzip flag is present.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* gzio.c (w_gzopen_mode): Use w_open_mode if available, in order
to support most of the flags (including "x" which Zlib has,
but which we are not passing through).
* stream.c (w_open_mode): New function formed from w_fopen_mode
content.
(w_fopen_mode): Call w_open_mode.
* stream.c (w_open_mode): Declared.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Makefile (OBJS): Conditionally include new gzio.o object file.
* configure: Detect external zlib, setting up new have_zlib
variable in config.make, HAVE_ZLIB in config.h, and also -lz
in conf_ldlibs.
* gzio.[ch]: New files, implementing the stream abstraction
over the gzip file I/O routines in zlib.
* stream.h (struct stdio_mode): New gzip flag and gzlevel
bitfield to hold a value 0 to 9.
(stdio_mode_init_blank, stdio_mode_init_r, stdio_mode_init_rpb):
Update intializers to cover new bitfield members.
* stream.c: Include <zlib.h> and "gzio.h" if HAVE_ZLIB.
(do_parse_mode): Recognize new mode modifier letter "z",
setting the gzip flag in the mode structure. If it's followed
by a digit, set the gziplevel to that value.
(format_mode): Don't output "b" letter for binary mode if
gzip is set, because gzopen interprets "b" differently.
Don't put out "t" if gzip is set. If gzip mode is specified,
do put out the level. If gzip is set, and gziplevel is nonzero
then encode the level: gzopen will understand it.
(open_file): If gzip mode is requested, then open the file
using gzopen mode, a new function in gzio.c. The return a
gzio stream based on the returned gzip file handle. However,
if we are reading, and the gzip stream indicates that it's
not decompressing anything, then we close it and open the
file using an ordinary stream.
(stream_init): Call gzio_init if HAVE_ZLIB is true. This is done
here because the module is integrated with stream.c, and also
so that lib.c doesn't have to know about HAVE_ZLIB and <zlib.h>.
|
| |
|
|
|
|
| |
* stream.c (normalize_mode): Include the self symbol in the
error message being thrown. This was not forgotten in the
sister function normalize_mode_no_bin.
|
| |
|
|
|
|
| |
* stream.c (stdio_set_prop): When setting the :name property,
return t, to indicate it was recognized and stored, as is
documented.
|
| |
|
|
|
| |
* stream.c (stdio_maybe_read_error): Access errno once, and
refer to the value.
|
| |
|
|
|
|
| |
* configure: do not treat the failure to detect mmap as
a fatal condition. This was just debugging commands accidentally
left behind.
|
| |
|
|
|
|
| |
* tests/019/load-search.tl: skip a certain test if it is run as
superuser; it fails because superuser is not affected by denied
directory search and execute permissions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* RELNOTES: Updated.
* configure (txr_ver): Bumped version.
* stdlib/ver.tl (lib-version): Bumped.
* txr.1: Bumped version and date.
* txr.vim, tl.vim: Regenerated.
* protsym.c: Likewise.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We've already taken care of imitating the situation that GNU
C allows __attribute__((aligned(n))) to weaken the alignment
of a bitfield, contrary to it being documented that align only
strengthens alignment. Even a value of n == 1 is meaningful
in that it can cause the bitfield to start allocating from
a new byte.
This patch corrects a newly discovered nuance: when a bitfield
is attributed with a weaker alignment than its underlying
type (e.g. uint32_t field marked with 2 byte alignment),
the original type's alignment is still in effect for calculating
the alignment of the structure, and the padding.
* ffi.c (struct txr_ffi_type): New member oalign, for keeping
track of the type's original alignment, prior to adjustment.
(make_ffi_type_struct): For a named bitfield, take the oalign
value into account when determining the most strict member
alignment.
(ffi_type_compile): When marking a type as aligned, the
we remember the original alignment in atft->oalign.
* tests/017/bitfields.tl: New test case, struct s16.
* txr.1: Documented.
|
| | |
|
| |
|
|
|
| |
* txr.1: pack with alignment values greater than 1 doesn't
directly correspond to a single GNU C feature.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (make_ffi_type_struct): Add check for impossible condition.
The bits_alloc variable could only exceed bits_type (and thus
cause the room variable to have a nonsensical, large value)
if the bitfield allocation tried to continue allocating bits into
an aligned unit, whose alignment exceeds the size of the underlying
type. But in that case, tft->aligned would have to be true, and
so the offset would have been aligned prior to this code, rendering
bits_alloc zero.
* tests/017/bitfields.tl: New tests.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* struct.c (alloc_seen): Fix under-sized alloca and memset,
where the size being used is the number of elements,
but should be the number of bytes. This showed up as a
failure in the tests/012/oop-mi.tl on PPC64, uncovered by
a completely unrelated change in eval.c. The unrelated code
never executes during the failed test case; it just affects
some memory contents of the stack somehow. The under-allocated
array is used during structure type initialization to hold
a bitmask which helps to suppress the duplicate initiation of
base structures which are inherited multiple times (duplicate
bases). I think why this shows up on PPC64 is the big endian
order. The sizes involved are small. The test case has a couple
of duplicate bases only, but a full 64 bit mask needs to be
allocated for that since the mask is an array of ucnum. On
little endian, it doesn't matter than only a single byte is
allocated: that is enough for 8 duplicate bases, and because of
the byte order, those bases land into the byte that is allocated.
On big endian, the bases will be on the high part of the 64
bit word, and thus over in the area we didn't allocate.
To reproduce an issue on little endian, you need an inheritance
hierarchy with more duplicate bases.
|
| |
|
|
|
|
| |
* ffi.c (swap_get32, swap_put32, swap_get64, swap_put64):
correct implementations for operating on little-endian
words on big endian.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The bitfield allocation rules are wrong. Some of it is due
to the recent changes which are based on incorrect analysis,
but reverting things doesn't fix it.
The idea that we compare the current member's alignment
with the previous is wrong; it is not borne out by empirical
tests with gcc. So we do a straight revert of that.
In GNU C, an __attribute__((aligned (N))) attribute applied
to a bitfield member will perform the requested alignment if,
evidently, the bit field is already being placed into a new
byte. (If the bit field is about to be packed into an existing
byte, then there is a warning about the align attribute being
ignored). Because we don't have alignment as a member attribute,
but only as a type attribute, we must implement a flag which
indicates that a type has had align applied to it (even if
the alignment didn't change) so we can then honor this in the
right place in the bitfield allocation code.
* ffi.c (struct txr_ffi_type): New attribute flag, aligned.
(make_ffi_type_struct): Remove the prev_align variable and
all related logic. Consolidate all alignment into one place,
which is done before we allocate the bitfield or regular member.
We align if the new member isn't a bitfield, or even if it is
a bitfield if it has the aligned attribute, or if the bitfield
is changing endian compared to the previous member (our local
rule, not from GNU C).
(ffi_type_compile): The align and pack operators now set the
aligned attribute, except in the (pack 1 ...) case which
semantically denotes lack of alignment.
* tests/017/bitfields.tl: New file.
* txr.1: Documented.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is very nice: you can now declaratively match a structure
that uses bitfields, and comes from a different endian system.
* ffi.c (ffi_kind_t): Replace FFI_KIND_NUM type with FFI_KIND_INT,
FFI_KIND_UINT and FFI_KIND_FLO. Now using the tft->kind value,
we can distinguish integer and floating types, and determine
signedness.
(struct txr_ffi_type): New flag, bigendian, which is set to 1
for all big types that are big endian. This is not just the endian
types like be-int32, but natural types like int, if the underlying
platform is big endian.
(swap_get32, swap_put32, swap_get64, swap_put64): New static functions.
(ffi_generic_swap_fat_ubit_put, ffi_generic_swap_fat_sbit_put,
ffi_generic_swap_fat_ubit_get, ffi_generic_swap_fat_sbit_get,
ffi_generic_swap_ubit_put, ffi_generic_swap_sbit_put,
ffi_generic_swap_ubit_get, ffi_generic_swap_sbit_get): New
static functions.
(ffi_make_type_builtin): On big endian, set the bigendian flag on every
type. For the endian types, this will get adjusted as an additional
construction step.
(make_ffi_type_endian): New static function. Calls ffi_make_type_builtin,
and then initializes the bigendian flag with the given value.
(make_ffi_type_struct, make_ffi_type_union): Because bitfields can
have endiannness now, we can no longer refer to the machine's own
endianness in laying them out; we have to look at the mtft->bigendian
value. Furthermore, we need an additional rule: when a bitfield member
follows a member that has different endian, a new allocation unit
has to begin.
(ffi_type_compile): the sbit and ubit types must set the type to
FFI_KIND_INT and FFI_KIND_UINT now. For the big operator, we can
simplify the error checking: instead of exhaustively comparing the
type to all the supported integer types, we just now check whether
it is FFI_KIND_INT or FFI_KIND_UINT. Here, if we detect that an
endian bitfield has opposite byte order from the machine, then we
instantiate the bitfield with the ffi_generic_swap_* virtual
functions. These perform the required byte swapping accesses to
the bitfield storage cells, so then the bit field manipulation
code just works using the local integer representation (shifting
and masking). Of course, the shift amount depends on the endian;
and that is calculated at type creation time in make_ffi_type_struct.
(ffi_init_types): Replace FFI_KIND_NUM with the appropriate constant
for each affected type. In some cases, we have to calculate whether
to use the INT or UINT one, for the types whose signedness is
not specified. We switch all the endian types to new constructor
make_ffi_type_endian, passing the required value of the bigendian
flag.
* txr.1: Documented.
|
| |
|
|
|
|
|
| |
* ffi.c (make_ffi_type_struct, make_ffi_type_union): We must
take fat bitfields into account when calculating mtft->shift
on big endian. We must subtract from bits_llint not bits_int
for big endian.
|
| |
|
|
|
|
|
|
|
| |
* ffi.c (ffi_init_types): The be-float, be-double, le-float and
le-double types wrongly have integer indicated as their Lisp
type symbol; it should be float. This doesn't fix any actual bug;
the Lisp type is only important for structures/unions. The
conversion of be-double is driven by the behavior of the type's
virtual functions, which don't look at the Lisp type symbol.
|
| |
|
|
|
|
|
|
| |
This was developed together with what became the May 12 commit
1162a735b61c1c5086fb6055471ee35cc8ed62a4; I just forgot to
git add the file.
* tests/011/macros-4.tl
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (ffi_flex_struct_in): Function renamed to ffi_flex_array_len,
because its responsibility is determining the length of a flexible
array that is not null terminated. We don't pass in the structure's
type's descriptor any more, but the member descriptor.
(ffi_struct_in, ffi_struct_get): Follow rename and changed parameter
conventions.
* tests/017/flexstruct.tl: Added test case with nested flexible
structure.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (ffi_flex_struct_in): Check for the last member being
an array, and not null-terminated. We now check the character
conversion disposition of the array. If it has character
conversion, then we store the length right into the slot that
will become the string. In the no-conversion case, we assume
that if the member exists, it's a vector we can resize.
Otherwise we plant a vector of the required size.
(ffi_varray_put): Only call ffi_varray_dynsize if the Lisp
object is a vector. If the Lisp objecct is a number, then use
that as the size. Otherwise the size is zero.
* tests/017/flexstruct.tl: New file.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After "years of trouble-free operation" a bug in the UTF-8
decoder was found, which violates its property that any
sequence of bytes will decode to some kind of string, which
will encode to the original bytes.
When the UTF-8 data prematurely ends in the middle of a valid
character, the decoder just drops that data as if it didn't
exist. So for instance the two-byte sequence E6 BC should
decode to "\xDCE6\xDCBC", since it is a fragment of a three-byte
UTF-8 sequence. It actually decodes to the empty string.
* utf8.c (utf8_bfom_buffer): When the buffer is exhausted, if we are
not in the utf8_init state, it means we were in the middle of a
UTF-8 sequence. Walk the bytes from the backtrack point to the end
of the buffer and store them into the string as U+DCxx codes.
* tests/012/buf.tl: Tests added for this via buf-str, str-buf.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (ffi_transform_pack): Fix: return the original syntax in
the situation when no cases are recognized, rather than
the cdr of the syntax. When the struct/union syntax has no
members, return the original syntax to indicate no transformation
took place.
* txr.1: Document the feature that pack on a typedef name or struct
name with no members will do the alignment adjustment only, without
the syntactic transformation.
* tests/017/pack-align.tl: New file.
|
| |
|
|
|
|
|
| |
* eval.c (error_trace): Don't mention that --backtrace can be
used to enable backtraces. This is not only a nuisance, but
appears in packaged programs which do not support such an option,
making it highly misleading.
|
| |
|
|
|
| |
* txr.1: fix broken quoting which results in stray "
character appearing in rendered output.
|
| |
|
|
|
|
|
| |
* txr.1: The documentation for array and zarray wrongly states
that if the dimension is omitted, the type may not be used
as a structure member. In fact, it may be used as the last
member of a flexible structure.
|
| |
|
|
| |
* txr.1: two instances of "an incomplete arrays" in FFI doc.
|
| |
|
|
|
|
|
| |
* ffi.c (ffi_type_compile): Allow align to weaken (lower) alignment
if compatibility 275 or lower is requested.
* txr.1: Compat note added.
|
| |
|
|
|
|
|
|
|
|
|
| |
* ffi.c (ffi_pack_members): Static function removed.
(ffi_transform_pack): New static function.
(ffi_type_compile): Rely on ffi_transform_pack to recognize
and perform all necessary transformations.
Cosmetic issue: when a struct is compiled, and the individual
member types undergo transformation during member compilation,
the syntax for the struct is nevertheless the original one
with the untransformed members.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* stream.c (trim_path_seps): New function.
(stream_init): trim-path-seps intrinsic registered.
* stream.c (trim_path_seps): Declared.
* tests/018/path.tl: New tests.
* txr.1: Documented.
* stdlib/doc-syms.tl: Updated.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (ffi_type_copy, ffi_type_copy_new_ops): The cloned txr_ffi_type
structure must have a self member which points to the new cobj,
not the original one. Otherwise things are inconsistent. For instance
if the clone is being made for the purposes of adjusting alignment,
any operation which chases the self pointer will be accessing incorrect
attributes. One example of this is (alignof foo.bar) where if bar
is the clone of a type, this will incorrectly report the alignment of
the original from which bar was cloned, and the original alignment,
not the adjusted alignment is reported.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* ffi.c (pack_s): New symbol variable.
(ffi_type_compile): Handle new pack type operator together with
align. Allow a one-argument form of align and pack in which
the value is defaulted. The behavior of align changes: align
can only increase alignment now, never decrease, so for
instance (align 1 ...) does nothing. pack must be used to
decrease alignment. Furthermore, for certain argument types,
pack performs transformations of the type syntax, rather than
compiling the argument type and producing a variant of it with
altered alignment.
* txr.1: Documented.
* stdlib/doc-syms.tl: Updated.
|
| |
|
|
|
|
|
|
|
| |
* ffi.c (make_ffi_type_struct): When a bitfield is declared for
a type whose alignment is different from that of the previous
member, then we start the new bitfield on a fresh alignment.
As usual, if the previous member is a bitfield which left a
partially filled byte, we skip to the next byte and then do the
alignment. This seems to match GCC behavior.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In gcc, bitfields can be declared with overridden alignment;
for instance when the entire structure is packed.
We must modify the algorithm to make this work.
Firstly, we must must pretend that we are filling from the start
of the containing alignment unit, not size based unit. For
instance, if the cell is an 8 byte type, but the alignment is 1,
then we are just filling the current byte, not the current 8 byte
multiple. However, the number of bits left (room) is still
calculated from the size: so if the byte has been filled with 3 bits,
we have 4 left in that byte, and then 7 more bytes, and so 61 bits.
Secondly, the case of the zero-bit field is special: it does
perform an alignment calculation in terms of multiples of the
size, ignoring the declared alignment of the field.
Thirdly, when the bits do not fit into the existing room, the
offset does not move by the size of the bitfield, but by the
alignment unit. For instance, if we have 63 bits left in an 8
byte field that has alignment 1, and we want to place 64 bits
into another 8 byte field, then we will just move to the next
byte, not to the next 64 bit cell. Starting at the next byte,
we pretend we have a fres 64 bit cell, albeit misaligned.
* ffi.c (make_ffi_type_struct): Make the above adjustments to
the algorithm. Some common values size, alignment, mask)
are declared in a wider scope and adjustments made.
|
| |
|
|
|
|
|
|
| |
* ffi.c (make_ffi_type_struct, make_ffi_type_union): Initialize
most_align local variable to 1, so the lower bound of alignment
is that, rather than zero.
* tests/017/ffi-misc.tl: Tests added.
|
