* eval.c (eval_init): tok-str acquires new parameter

* lib.c (tok_str): New parameter, keep_sep.

* lib.h (tok_str): Declaration updated.

* txr.1: Documentation for tok-str updated.

1 files changed, 8 insertions, 1 deletions

diff --git a/txr.1 b/txr.1
index 8d56e113..73493413 100644
--- a/txr.1
+++ b/txr.1
@@ -7572,7 +7572,7 @@ This operation is nondestructive: <string> is not modified in any way.
 .TP
 Syntax:
 
-  (tok-str <string> <regex>)
+  (tok-str <string> <regex> [<keep-between>])
 
 .TP
 Description:
@@ -7589,6 +7589,13 @@ list ("a" "" "" ""). After the token "a" is extracted from a non-empty match
 for the regex, the regex is considered to matches three more times: before the
 "b", between "b" and "c", and after the "c".
 
+If the <keep-between> argument is specified, and is not nil, then the behavior
+of tok-str changes in the following way. The pieces of <string> which are
+skipped by the search for tokens are included in the output. If no token is
+found in <string>, then a list of one element is returned, containing <string>.
+Generally, if N tokens are found, then the returned list consists of 2N + 1
+elements. The first element of the list is the (possibly empty) substring which had to be skipped to find the first token. Then the token follows. The next element is the next skipped substring and so on. The last element is the substring of <string> between the last token and the end.
+
 .SS Function list-str
 
 .TP