summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* vim: deal with bvars and square brackets.Kaz Kylheku2022-08-301-2/+3
| | | | | | | | | | | | The last round of changs. The txr_bracevar group is still handling bracket vars in their entirety, including the @. Also square bracket lists are not handled right. * genvim.txr (txr-elem): Add txr_bracket and txr_bracevar. (txr_bracevar): Don't match @ sigil, and mark as contained group.
* vim: fix quasiliterals.Kaz Kylheku2022-08-301-4/+6
| | | | | | | | | | | | | | | | | | | | The previous commit breaks the syntax coloring of the interior of quasiliterals. That is now addressed. * genvim.txr (txr-qelem): New variable holding the elements of a quasiliteral which follow the @ sigil. These items were previously listed as the contained items of txr_quasilit. Now they become nextgroup elements of tl_qat. (tl_qat): New match group representing the special items in a quasiliteral. It matches the sigil, and then the txr-qelem items via nextgroup. (tl_bracevar): Renamed to txr_qbracevar, since it targets the brace variable variant contained in quasiliterals. (txr_mlist,txr_mbracket): No longer used; removed. (txr_quasilit): Now just contains txr_qat.
* vim: tweak @ handling.Kaz Kylheku2022-08-301-24/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I noticed that in some Vim color schemes, the @ in @( ) is colored the same as the parentheses, whereas in @abc, it is colored differently from the identifier (and different from parentheses). This patch fixes things so that the @ sigil is in the Special category, rather than Delimiter, almost everywhere. * genvim.txr (txr-elem): New variable, for holding names of regions which follow the @ in the TXR language. Used in definition of txr_at. (bvar, dir, list): Remove the txr_mlist and txr_mbracket regions. (txr_at): New match group defined, which matches the @ sigil in TXR, followed by various elements using the nextgroup mechanism. txr_at is already assigned to a highlight category via a previously dangling entry. (txr_error,txr_atat,txr_comment,txr_contin, txr_char,txr_error,txr_char,txr_regdir,txr_variable, txr_splicevar,txr_metanum,txr_directive): These match groups don't match the leading @ sigil any more and are marked contained. They activate as the nextgroup items in txr_at, allowing them to be colored differently. (tl_error): New group. split off from txr_error. We don't want to to recognize this category after the @ in TXR because @#... is the old-style comment. (tl_ident): Don't try to match leading @. This is useless because there is a more specific match via txr_metaat later. (txr_quote, txr_metaat): Mark these not contained in Lisp so they activate at the top level. (txr_directive): Don't match leading @. This now activates as a nextgroup item in txr_at. Thus directives can have a differently colored @. (txr_mlist, txr_mbracket): These are now unconditionally contained, and are used only in txr_quasilit. I am otherwise leaving quasiliterals alone in this patch; it will need the same treatment for @ to be colored seprately inside quasiliterals.
* txr: close streams.Kaz Kylheku2022-08-293-18/+77
| | | | | | | | | | | | | | | | | | | * match.c (noclose_k): New keyword variable. (v_next_keys, v_output_keys): New static variables. (v_next_impl): Use v_next_keys in calculating alist, rather than freshly allocating it each time. Check for the new :noclose keyword; if it is missing, close any locally opened stream when done. (v_output): Refer to v_output_keys precalculated list rather than allocating it every time. (match_files): If a stream is opened in by a call to open_data_source from this function, then the stream is closed when this function returns. (syms_init): Intern the :noclose symbol. (plist_keys_init): New function. (match_init): Call plist_keys_init. * txr.1: Documented new :noclose option of @(next).
* New macro: close-lazy-streams.Kaz Kylheku2022-08-287-1/+102
| | | | | | | | | | | | | | | | | | | | | | | | * lib.c (lazy_stream_s): New symbol variable. (lazy_streams_binding): New static variable. (lazy_stream_register): New static function (lazy_stream_cons): If the stream is associated with a lazy cons, register it with lazy_stream_register. (obj_init): gc-protect lazy_streams_binding variable. Intern the sys:*lazy-streams* symbol. * lib.h (lazy_streams_s): Declared. * eval.c (eval_init): Register sys:*lazy-streams* special variable. * stdlib/getput.tl (close-lazy-streams): New macro. * autoload.c (getput_set_entries): Trigger autload on close-lazy-streams symbol. * txr.1: Documented. * stdlib/doc-syms.tl: Updated.
* New function: search-allKaz Kylheku2022-08-176-9/+100
| | | | | | | | | | | | | | | | | | | | | | | * eval.c (eval_init): search-all intrinsic registered. * lib.c (search_common): New Boolean argument all, indicating whether all positions are to be returned. We must handle this in the two places where empty key and sequence are handled, and also in the main loop. A trick is used: the found variable is now bound by list_collect_decl, but not used for collecting unless all is true. (search, rsearch, contains): Pass 0 for all argument of search_common. (search_all): New function. * lib.h (search_all): Declared. * tests/012/seq.tl: New tests. * txr.1: Documented. * stdlib/doc-syms.tl: Regenerated.
* search/rsearch: some test cases.Kaz Kylheku2022-08-171-0/+36
| | | | * tests/012/seq.tl: New tests.
* ffi: check for out-of-range wchar_t.Kaz Kylheku2022-08-131-0/+3
| | | | | | * ffi.c (ffi_wchar_get): Reject wchar_t values that are negative or beyond U+10FFFF; do not convert these to a character.
* Version 280.txr-280Kaz Kylheku2022-08-094-5/+20
| | | | | | | | | | * RELNOTES: Updated. * configure (txr_ver): Bumped version. * stdlib/ver.tl (lib-version): Bumped. * txr.1: Bumped version and date.
* build: handle hard link failure in make install.Kaz Kylheku2022-08-091-1/+1
| | | | | | | | | * Makefile (HARDLINK): Print a diagnostic if the link command fails and ignore the situation. Hard links are restricted on Android. On that platform, txr being available under the names txrlisp and txrvm is likely of limited utility, so we won't waste space by making copies of the executable.
* listener: bug: history ignored if .txr_profile doesn't exist.Kaz Kylheku2022-08-091-1/+1
| | | | | | * parser.c (repl): Fix code wrongly checking for the existence of the .txr_profile file as a condition for loading .txr_history. It should of course be checking for .txr_history.
* load: fix misleading error message.Kaz Kylheku2022-08-091-1/+1
| | | | | * parser.c (open_txr_file): Replace "not found" wording with "unable to open" because the diagnostic covers permission errors.
* Version 279.txr-279Kaz Kylheku2022-08-086-1051/+1095
| | | | | | | | | | | | * RELNOTES: Updated. * configure (txr_ver): Bumped version. * stdlib/ver.tl (lib-version): Bumped. * txr.1: Bumped version and date. * txr.vim, tl.vim: Regenerated.
* path-components-safe: proc tests only on Linux.Kaz Kylheku2022-07-301-18/+20
| | | | | | | * stdlib/path-test.tl (safe-abs-path): If (uname) doesn't report Linux, then define this function in a way that it always returns true. We do this by making the name an alias for the tf function.
* path-components-safe: check symlink link count.Kaz Kylheku2022-07-301-2/+3
| | | | | | * stdlib/path-test.tl (path-components-safe): Reject symlinks that have a link count not equal to one. This looks suspiciously like a hard link attack.
* path-components-safe: tighten /proc checkKaz Kylheku2022-07-302-45/+35
| | | | | | | | | | | | | | Attacks are possible via /proc/<pid>/fd/<n> involving a deleted file, whereby the link target changes from "/path/to/file" to "/path/to/file (deleted)", which can be perpetrated by a different user, not related to process <pid>, who has access to perform unlink("/path/to/file"). * stdlib/path-test.tl (safe-abs-path): Perform the pattern check regardless of effective user ID. * tests/018/path-safe.tl: Test cases adjusted.
* path-components-safe: handle consecutive slashes.Kaz Kylheku2022-07-301-4/+2
| | | | | * stdlib/path-test (path-components-safe): Remove empty components from split path.
* path-components-safe: repel /proc symlink attacksKaz Kylheku2022-07-292-17/+67
| | | | | | | | | | | | | | | | | | | | | | In a Linux system, it's possible for an unprivileged user to create a root symlink pointing to any directory, simply by changing to that directory and running a setuid executable like "su". That executable will get a process whose /proc/<pid> directory is root owned, and contains a symlink named cwd pointing to the current directory. Other symlinks under /proc look exploitable in this way. * stdlib/path-test.tl (safe-abs-path): New function. Here is where we are going to check for unsafe paths. We use some pattern matching to recognize various unsafe symlinks under /proc. (path-components-safe): Simplify code around recognition of absolute paths. When an absolute path is read from a symlink, remove the first empty component. Pass every absolute path through safe-abs-path to check for known unsafe paths. * tests/018/path-safe.tl: New tests.
* rel-path, path-equal: relocate.Kaz Kylheku2022-07-293-107/+96
| | | | | | | | * stdlib/copy-file.tl (path-simplify, path-split, path-volume, rel-path, path-equal): Remove from here. * stdlib/path-test.tl: (path-simplify, path-split, path-volume, rel-path, path-equal): Move to here.
* stringp: rewrite.Kaz Kylheku2022-07-281-5/+13
| | | | | | * lib.c (stringp): Examine tag and then type separately, rather than using the canned type function. This leads to slightly nicer code, shorter by a couple of instructions.
* gcd: rewrite for better efficiency.Kaz Kylheku2022-07-272-24/+104
| | | | | | | | | | | * arith.c (gcd): New implementation which uses arithmetic in the unsigned type ucnum if both operands are in that type's range. This uses Stein's algorithm a.k.a. binary GCD. The mpi_gcd function is used only if at least one argument is a bignum whose value doesn't fit into a ucnum. * tests/016/arith.tl: gcd test cases added.
* in6addr-str: remove useless regsub.Kaz Kylheku2022-07-271-1/+0
| | | | | | | | * stdlib/socket.tl (in6addr-str): Remove one of the two repetitions of a string substitution intended to be done once. The substitution is idempotent and therefore a second application of it is redundant regardless of intent.
* Take advantage of substring support in regsub.Kaz Kylheku2022-07-274-8/+5
| | | | | | | | | | | | | | | | * txr.c (sysroot_init): Use regsub to look for "\\" substring instead of regex. * stdlib/getopts.tl (opt-parsed convert-type): regsub for "0x" substring rather than #/0x/ regex. * stdlib/pic.tl (pic-join-oipt): regsub for "~" substring rather than #/\~/ regex. * stdlib/socket.tl (in6addr-str): regsub for "::" substring instead of #/::/ regex in two places. The double regsub there looks like a mistake; will address in another commit.
* regsub: allow string in place of regex.Kaz Kylheku2022-07-272-4/+24
| | | | | | * regex.c (regsub): Use search_str if regex is a string. * txr.1: Documented.
* regsub: avoid consing list.Kaz Kylheku2022-07-271-10/+10
| | | | | | * regex.c (regsub): Accumulate output string directly using string_extend, rather than accumulating a list of pieces which are catenated with cat_str.
* repl: revise security checks.Kaz Kylheku2022-07-262-33/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: we now check the entire path of .txr_history and .txr_profile files for security issues; we enforce that these files must not be readable to other users, not just not writable. And there is a bugfix: we do not load the history if it has a permission problem, instead of loading it anyway and just issuing a diagnostic. * repl.c (report_security_problem): Rename to report_file_perm_problem. Drop the umask check, because we are going to be checking for files that are not readable for others, which would require a stricter umask than the usual 022. (report_path_perm_problem): New static function. (load_rcfile): Take the needed function symbols as arguments, because the only caller is repl and it has them; it can pass them down. Check the path using path-components-safe function, and bail with an error message if it is bad. Then check the file using path-strictly-private-to-me-p, rather than path-private-to-me-p as previously. This requires the file not to be readable to others too. (repl): path_private_to_me_p variable renamed to ppriv_s for brevity and holds a different symbol: path-strictly-private-to-me-p, the function which checks that other users cannot read the file, not just write. Also capture the path-components-safe symbol as psafe_s. ppriv_s and psafe_s are passed down to load_rcfile so it can do checks. Like in the case of the rcfile, we now check the history file using both functions, validating the path not just the file's own permissions. Bugfix: we now check the history file's path before loading the history file, and avoid loading it if the check fails. We use the path-exists-p function now to check that the history and rc files exist. That leaves a small flaw: an attacker could be in control of the paths to these files and manipulate these paths such that these files appear not to exist; we will then not report on such a situation. * txr.1: Documented.
* path-components-safe: refactoring.Kaz Kylheku2022-07-252-56/+39
| | | | | | | | * stdlib/path-test.tl (path-components-safe): Simplify code; forget trying to do anything on Windows: just return true. * txr.1: Document that path-components-safe is useless on Windows.
* New function: path-components-safe.Kaz Kylheku2022-07-255-20/+203
| | | | | | | | | | | | | | | | * autoload.c (path_test_set_entries): Autoload on path-components-safe symbol. * stdlib/path-test.tl (if-windows, if-native-windows): New system macros. (path-safe-sticky-dir): New system function. (path-components-safe): New function. * tests/018/path-safe.tl: New file.' * txr.1: Documented. * stdlib/doc-syms.tl: Updated.
* path-test: object trust must use effective UID.Kaz Kylheku2022-07-242-10/+22
| | | | | | | | | * stdlib/path-test.tl (path-private-to-me, path-strictly-private-to-me): Call (getuid) rather than (geteuid). We won't revert this behavior with the -C compat option because it's wrong/insecure. * txr.1: Updated.
* compile-file: distinguish nonexistence errors.Kaz Kylheku2022-07-211-8/+10
| | | | | | | | | | | | | | | | | | | The compile-file function must only try a different path, such as with a suffix, if a given path fails to open due to non-existence. If the failure is for another reason, like permissions. In that case we want to propagate the failure. * stdlib/compiler.tl (ign-notfound): New macro. This lets through all errors, catching only path-not-found, converting that to a nil result. (open-compile-streams): Use ign-notfound instead of ignerr when trying to open an input file for reading. Also, we lose the ignerr wrapping around the open-file. We let any error whatsoever just bubble out so that the user is better informed about what went wrong. The requirement to close the input stream is handled by the obvious unwind-protect.
* compile-file: revise hash bang treatment.Kaz Kylheku2022-07-212-7/+27
| | | | | | | | | * stdlib/compiler.tl (translate-hash-bang): New function. (compile-file-conditionally): Use translate-hash-bang to treat hash bang line. * txr.1: Revised hash bang treatment by file compiler documented.
* compiler: try unsuffixed path before adding .tlKaz Kylheku2022-07-212-6/+6
| | | | | | | | * stdlib/compiler.tl (open-compile-streams): If the in-path is unsuffixed, try opening it without adding any suffix first. If that fails, then try .tl in that order. * txr.1: Documented.
* syrooting: remove two useless checks.Kaz Kylheku2022-07-211-2/+0
| | | | | | | | | | | | * txr.c (sysroot_init): The value of TXR_REL_PATH already has the exe suffix on it if we are building for Windows. There is no need to try with EXE_SUFF tacked on to it. There is also no need to try PROG_NAME with or without the suffix. EXE_SUFF is the empty string on non-Windows platforms, so the two lines of code are equivalent. On Windows, PROG_NAME without the EXE_SUFF will not match; that maybe_sysroot call is a waste of CPU cycles.
* Install a txrlisp hard link.Kaz Kylheku2022-07-213-0/+45
| | | | | | | | | | | | | | | | | | * Makefile (LN): New variable. On platforms where you can't hard link, this can be replaced with some other command. Possibly "true" not to have the alternative executable name created at all. (HARDLINK): New macro. (install): Use HARDLINK to create a link named txrlisp pointing to the same file as txr in the destination directory. * txr.c (txr_main): If the executable ends with "lisp" (or "lisp.exe" on Windows), then default the txr_lisp_p variable to t, which has the effect as if --lisp had been processed. * txr.1: Documented.
* doc: pos function.Kaz Kylheku2022-07-181-5/+9
| | | | * txr.1: Improve unclear wording describing pos function.
* New function: count.Kaz Kylheku2022-07-186-20/+94
| | | | | | | | | | | | | | | | The general count function, with keyfun and testfun, is noticeably absent. Let's implement it. * lib.[ch] (count): New function. * eval.c (eval_init): Register count intrinsic. * tests/012/seq.tl: Some tests for count. * txr.1: Add count to count-if section. Revise documentation based on pos/pos-if. * stdlib/doc-syms.tl: Updated.
* Date correction; man page typos.txr-278Kaz Kylheku2022-07-012-4/+4
| | | | | | * RELNOTES: 2022-07-01. * txr.1: Likewise and two minor fixes.
* Version 278.Kaz Kylheku2022-07-015-5/+44
| | | | | | | | | | | | * RELNOTES: Updated. * configure (txr_ver): Bumped version. * stdlib/ver.tl (lib-version): Bumped. * txr.1: Bumped version and date. * protsym.c: Regenerated.
* doc: remove references to non-fork platforms.Kaz Kylheku2022-06-291-16/+3
| | | | | | | | | * txr.1: Remove references to limitations in open-process on platforms which don't have fork. It's simply not implemented on those platforms at all any more. This documentation referred to a MinGW port of TXR which has not been maintained in many years. Also remove text which says that open-subprocess is only available on platforms with fork.
* Add mode-opt to all I/O convenience functions.Kaz Kylheku2022-06-292-33/+33
| | | | | | | | | | | * stdlib/getput.tl (command-get, command-put, command-get-string, command-put-string, command-get-lines, command-put-lines, command-put-buf, command-get-json, command-put-json, command-get-jsons, command-put-jsons): Add mopt parameter, which is interpolated into appropriate mode string. This allows "z" to be used for gzip compression. * txr.1: Updated Syntax synopses.
* file-get-lines: missing mode-opt parameter.Kaz Kylheku2022-06-291-2/+2
| | | | | * stdlib/getput.tl (file-get-lines): A mode-opt parameter is documented for this function; it is now actually implemented.
* close-stream: process wait cleanup.Kaz Kylheku2022-06-241-18/+6
| | | | | | | | | * stream.c (pipe_close_status_helper): Revise error messages. Get rid of impossible cases: we will not get WIFSTOPPED or WIFCONTINUED unless we used the WUNTRACED option in waitpid, which we don't. No platforms without HAVE_SYS_WAIT, don't throw if the status is nonzero; just return nil. It could be a normal termination.
* doc: document pid argument of open-fileno.Kaz Kylheku2022-06-241-2/+37
| | | | | | * txr.1: pid argument is documented and also missing, documentation added about how close-stream deals with processes.
* More HAVE_FORK_STUFF cleanup.Kaz Kylheku2022-06-241-3/+3
| | | | | | * stream.c (pipe_close_status_helper, pipe_close, pipe_ops): Included in #if HAVE_FORK_STUFF block. (stream_init): Refer to pipe ops only if HAVE_FORK_STUFF is true.
* streams: remove old code for popen on MingGW.Kaz Kylheku2022-06-214-174/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is vestigial code from a time when TXR supported being compiled with MinGW. Except in the case of the functions run and sh, which are implementable without fork on Cygwin using the spawn family of functions, there won't be any fallback; if HAVE_FORK_STUFF is zero or missing, then certain functions will be absent. * stream.c (struct stdio_handle): Do not define a pid member if we don't HAVE_FORK_STUFF. (se_pclose): Function removed; we won't be using any non-POSIX platforms popen/pclose. (pipe_close): Don't call the removed se_pclose. (make_pipe_stream): Function removed. (fds_subst, fds_swizzle): Don't define these if HAVE_FORK_STUFF is absent, and so is HAVE_WSPAWN and HAVE_SPAWN. They are now only needed by the version of the run function that uses spawn or wspawn. (open_command, open_process): Remove the versions of these function based on popen. (string_extend_count, win_escape_cmd, win_escape_arg, win_make_cmdline): Remove these functions used by the above open_process; we have no need for encoding arguments into a Windows command line string, since the Cygwin/Cygnal libraries do that for us in their spawn and exec functions. * stream.h (make_pipe_stream): Function removed. * utf8.[ch] (w_popen): Function removed.
* bugfix: missing gzip support in open-command.Kaz Kylheku2022-06-216-55/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stream.c (pipe_close_status_helper): New function, factored out of pipe_close and used by it, and also by gzio_close. (pipe_close): Call pipe_close, which now contains the classification of process wait status codes. (open_fileno): Now takes optional pid argument. If this specified, then make_pipevp_stream is used. (open_subprocess): Use the open_fileno function, rather than fopen. This simplifies things too, except that we have to catch exception. Pass pid to the newly added parameter of open_fileno so that we obtain a proper pipe stream that will wait for the process to terminate when closed. (mkstemp_wrap): Pass nil for pid argument of open_fileno. (stream_init): Update registration of open-fileno. * gzio.c (struct gzio_handle): New member, pid. (gzio_close): If there is a nonzero pid, wait for the process to terminate. (make_gzio_stream): Initialize h->pid to zero. (make_gzio_pipe_stream): New function. * parser.c (lino_fdopen): Pass nil for pid argument of open_fileno. * gzio.h (make_gzio_pipe_stream): Declared. * tests/018/gzip.tl: New test.
* configure: remove unused android_targetKaz Kylheku2022-06-211-2/+0
| | | | * configure (android_target): Remove unused variable.
* for/for*: stricter syntax check.Kaz Kylheku2022-06-133-18/+19
| | | | | | | | | | | | | | | | | * eval.c (me_for): Require at least one argument. However, we let the init-forms continue to be optional and document it. * txr.1: Refer to for and for* as macros, since they have been since 2016. The omission of the inc-form list is shown as a second variant of the syntax. This is to avoid misleading the reader into thinking that the the inc-form list can be omitted while body forms are present. A spurious paragraph reiterating that the macros establish an anonymous block is removed. That extra text was present in the first draft written in 2011, and maintained since. * stdlib/doc-syms.tl: Updated.
* New function: strKaz Kylheku2022-06-126-40/+136
| | | | | | | | | | | | | | | The str function is like mkstring but allows a fill pattern to be specified. * eval.c (eval_init): str intrinsic registered. * lib.[ch[ (str): New function. * tests/015/str.tl: New file. * txr.1: Documented. * stdlib/doc-syms.tl: Updated.
* optimizer: fix live set being unexpectedly nil.Kaz Kylheku2022-06-091-2/+2
| | | | | | | | | | | | | | | | | | | | The following test case throws an exception: (compile-toplevel '(when-match @(or @b @c) nil nil)) the reason is that in the optimizer, the local-liveness method resets the bl.live set of live registers to nil. The expectation is that it will be recalculated. However, what happens is that * stdlib/optimize.tl (basic-block): Initialize live slot to zero. (basic-blocks local-liveness): Reset bl.live to 0, rather than nil. The problem is that sometimes the block in question is not reached in the graph traversal (down in the same function) which is supposed to assign it a live value. This happens in the above test case, and it's not due to any bug: the block is not reached by the forward traversal because the block has become unreachable.
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-19 14:27:44 +0000