New: stack overflow protection.

* configure: detect getrlimit, producing HAVE_RLIMIT in
config.h.

* eval.c (do_eval, do_expand): Call gc_stack_check inline
function to check stack pointer against limit.

* gc.c (gc_stack_bottom): Static becomes extern, so inline
function in gc.h can refer to it.
(gc_stack_limit): New global variable.
(gc_init): If we have rlimit, then probe RLIMIT_STACK.
If the stack is sufficiently large, then enable the stack
overflow protection, which kicks in when the stack pointer
appears to be within a certain percentage of the limit.
(set_stack_limit, get_stack_limit): New static functions.
(gc_late_init): Register set-stack-limit and get-stack-limit
intrinsics.
(gc_stack_overflow): New function.

* gc.h (gc_stack_bottom, gc_stack_limit, gc_stack_overflow):
Declared.
(gc_stack_check): New inline function.

* lib.c (stack_overflow_s): New symbol variable.
(obj_print_impl): Call gc_stack_check to protect recursive
printing againts overflow.

* lib.h (stack_overflow_s): Declared.

* unwind.c (uw_init): Register stack-overflow symbol as a an
exception symbol subtyped from error.
(uw_unwind_to_exit_point): When dealing with an unhandled
exception, turn off the stack limit, so we can print the
messages without triggering it in a loop.

* vm.c (vm_execute_closure, vm_funcall_common): Insert
gc_stack_check to the top of the execution of every VM
function.

* txr.1: Documented.

* share/txr/stdlib/doc-syms.tl: Updated.

1 files changed, 4 insertions, 1 deletions

diff --git a/lib.c b/lib.c
index 0ce2e43b..cf629d6a 100644
--- a/lib.c
+++ b/lib.c
@@ -112,7 +112,7 @@ val eof_s, eol_s, assert_s, name_s;
 val error_s, type_error_s, internal_error_s, panic_s;
 val numeric_error_s, range_error_s;
 val query_error_s, file_error_s, process_error_s, syntax_error_s;
-val timeout_error_s, system_error_s, alloc_error_s;
+val timeout_error_s, system_error_s, alloc_error_s, stack_overflow_s;
 val path_not_found_s, path_exists_s, path_permission_s;
 val warning_s, defr_warning_s, restart_s, continue_s;
 val gensym_counter_s, length_s;
@@ -12492,6 +12492,7 @@ static void obj_init(void)
   system_error_s = intern(lit("system-error"), user_package);
   timeout_error_s = intern(lit("timeout-error"), user_package);
   alloc_error_s = intern(lit("alloc-error"), user_package);
+  stack_overflow_s = intern(lit("stack-overflow"), user_package);
   path_not_found_s = intern(lit("path-not-found"), user_package);
   path_exists_s = intern(lit("path-exists"), user_package);
   path_permission_s = intern(lit("path-permission"), user_package);
@@ -13031,6 +13032,8 @@ val obj_print_impl(val obj, val out, val pretty, struct strm_ctx *ctx)
   val ret = obj;
   cnum save_depth = ctx->depth;
 
+  gc_stack_check();
+
   if (check_emit_circle(obj, out, ctx, self))
     return ret;