strings: bugfix: broken inequality comparisons.

Inequality comparisons of strings and symbols are broken due
to assuming that cmp_str returns -1 or 1. cmp_str uses the C
library function wscsmp, and is exposed as the Lisp function
cmp-str. That is correctly documented as returning a negative
or positive value.  But a few function in lib.c assume
otherwise.

On newer glibc's, at least on x86, it seems that wcscmp
does return 1, 0 or -1 consistently; perhaps the newer
optimized assembly routines are ensuring this. It shows
up on older glibc installations where the C version just
returns the difference between the mismatching characters.

* lib.c (cmp_str): Now returns -1, 0 or 1.

* txr.1: Specify the stronger requirements on the cmp-str
return value, adding a note that older versions conform
to a weaker requirement.

1 files changed, 4 insertions, 1 deletions

diff --git a/lib.c b/lib.c
index dbeb20a0..add50d30 100644
--- a/lib.c
+++ b/lib.c
@@ -4467,7 +4467,10 @@ val cmp_str(val astr, val bstr)
    case TYPE_PAIR(STR, STR):
    case TYPE_PAIR(LIT, STR):
    case TYPE_PAIR(STR, LIT):
-     return num(wcscmp(c_str(astr), c_str(bstr)));
+     {
+       int cmp = wcscmp(c_str(astr), c_str(bstr));
+       return if3(cmp < 0, negone, if3(cmp > 0, one, zero));
+     }
    case TYPE_PAIR(LSTR, LIT):
    case TYPE_PAIR(LSTR, STR):
    case TYPE_PAIR(LIT, LSTR):