compiler: optimize dwim.

* share/txr/stdlib/asm.tl (op-getf): Rename to op-oldgetf.
This opcode becomes obsolescent.
(op-getf): New opcode.

* share/txr/stdlib/compiler.tl
(assumed-fun): New global variable.
(compiler comp-fun): Use the new getf instruction which takes
a function table index instead of a data table index.
(compiler comp-lisp1-value): Don't use getl1 opcode any more
for dynamic lisp1-style lookup. Instead, we bake the behavior
at compile time perform a function lookup if the symbol is
completely unbound, a variable lookup if it is bound to a
variable (where we decide at compile tie whether it is lexical
or dynamic) or else a function if a function binding exists at
compile time. Also, if we assume that an unbound symbol is a
function, put it on the assumed-fun list.
(compiler comp-dwim): If the first argument is a symbol
with no lexical binding, and is not bound as a variable, then
treat it as a function by transforming the form into a
function call form with that symbol in the car position.
Put the symbol on the assumed-fun list.
(compiler-emit-warnings): New function.
(with-compilation-unit): Call compiler-emit-warnings when
bailing out of most enclosing compilation unit.
(%tlo-ver%): Bump compiled file version to 4, since
we added an opcode.

* vm.c (vm_execute): Follow rename of GETF to OLDGETF.
Implement the new GETF.

* parser.c (read_file_common): Extend version range to allow
version 4 compiled files.

* txr.1: Documented everything.

1 files changed, 79 insertions, 4 deletions

diff --git a/txr.1 b/txr.1
index dad42dd7..1aa8f26f 100644
--- a/txr.1
+++ b/txr.1
@@ -62644,17 +62644,19 @@ file name suffix) is translated into an object file (named with a
 .code .tlo
 suffix) containing a compiled version of those forms.
 
-The key concept is that loading the compiled
+The intent is that loading the compiled
 .code .tlo
 file via the
 .code load
 function produces the same effect as loading the
 .code .tl
-file (except when special arrangements are deliberately put in place for
-different behaviors to occur). The difference is that the compiled file
+file. The main difference is that the compiled file
 contains no Lisp source code; only the machine code instructions for the
 virtual machine, and some accompanying data such as literals and referenced
-symbols.
+symbols. Note that the behavior of compiled code can differ from interpreted
+code in a number of ways. Differences in behavior can be deliberately induced.
+Certain erroneous or dubious situations can also cause compiled code to behave
+differently from interpreted code.
 
 Compilation not only provides faster execution. Compiled files load much
 faster.  Compiled files can be distributed unaccompanied by the source files,
@@ -62853,6 +62855,79 @@ that version.
 
 Version 199 produces version 3 files and loads version 2 or 3.
 
+Version 200 produces version 4 files and loads version 2, 3 or 4.
+
+.SS* Semantic Differences between Compilation and Interpretation
+
+The
+.code compile-only
+and
+.code eval-only
+operators can be used to deliberately produce code which behaves differently
+when compiled and interpreted. In addition, unwanted differences in behavior
+can also occur. The situations are summarized below.
+
+Forms evaluated by
+.code load-time
+are treated differently by the compiler. When a top-level form is compiled,
+its embedded
+.code load-time
+forms are factored out such that the compiled image of the top-level form
+will evaluate these forms before other evaluations take place.
+The interpreter doesn't perform this factoring; it evaluates a
+.code load-time
+form when it encounters it for the first time.
+
+Unbound variables are treated differently by the compiler. A reference
+to an unbound variable is treated as a global lexical access. This means that
+if a variable access is compiled first and then a
+.code defvar
+is processed which introduces the variable as a dynamically scoped ("special")
+variable, the compiled code will not treat the variable as special; it
+will refer to the global binding of the variable, even when a dynamic binding
+for that variable exists.  The interpreter treats all variable references
+that do not have lexical bindings as referring to dynamic variables.
+The compiler treats a variable as dynamic if a
+.code defvar
+has been processed which marked that variable as special.
+
+Arguments of a
+.code dwim
+form (or the equivalent bracket notation) which are unbound
+symbols are treated differently by the
+compiler. The code is compiled under the assumption that all such symbols
+refer to global functions. For instance, if neither
+.code f
+nor
+.code x
+are defined, then
+.code "[f x]"
+will be compiled under the assumption that they are functions.  If they are
+later defined as variables, the compiled code will fail because no function
+named
+.code x
+exists. The interpreter resolves each symbol in a
+.code dwim
+form at the time the form is being executed. If a symbol is defined
+as a variable at that time, it is accessed as a variable. If it defined as a
+function, it is accessed as a function.
+
+The symbolic arguments of a
+.code dwim
+form that refer to global bindings are also treated differently by the compiler.
+For each such symbol, the compiler determines whether it refers to a
+function or variable and, further, whether the variable is global lexical or
+special. This treatment of the symbol is then cemented in the compiled code;
+the compiled code will treat that symbol that way regardless of the
+run-time situation. By contrast, the interpreter performs this classification
+each time the arguments of a
+.code dwim
+form are evaluated. The rules are otherwise the same: if the symbol is bound as
+a variable, it is treated as a variable. If it is bound as a function, it is
+treated as a function. If it has both bindings, it is treated as a variable.
+The difference is that this is resolved at compile time for compiled code,
+and at evaluation time for interpreted code.
+
 .SS* Compilation Library
 
 .coNP Function @ compile-toplevel