From c27f83bdae5eb00206a478f7764df4fdaa48fc76 Mon Sep 17 00:00:00 2001 From: Kaz Kylheku Date: Thu, 31 Mar 2016 20:53:03 -0700 Subject: UTF-8 API overhaul: security, and other concerns. The main aim here is to pave the way for conversion between arbitrary buffers of bytes (that may include embedded NUL characters) and a wide string. Also, a potential security hole is closed. When we convert a TXR string to UTF-8 for use with some C library API, any embedded pnul characters (U+DC00) turn into NUL bytes which effectively cut the UTF-8 string short, and silently so. The C library function receives a shortened string. This could be exploitable in some situations. * lib.c (int_str): Use utf8_dup_to_buf instead of utf8_dup_to_uc. Pass 1 to have the buffer null-terminated, since mp_read_radix depends on it. * stream.c (make_string_byte_input_stream): Use utf8_dup_to_buf. This gives us the size, soo we don't have to call strlen. The buffer is no longer null terminated, but the byte input stream implementation never relied on this. * utf8.c (utf8_from_buf): Replacement fors utf8_from_uc which doesn't assume that the buffer of bytes is null-terminated. It can produce a wide string containing U+DC00 characters corresponding to embedded nulls in the original buffer. (utf8_from): Calculate length of null-terminated string and use utf8_from_buf. (utf8_to_buf): Replacement for utf8_to_uc. Can produce a buffer which is or is not null-terminated, based on new argument. (utf8_to): Use utf8_to_buf, and ask it to null-terminate, thus preserving behavior. (utf8_dup_from_uc): This function was not used anywhere and is removed. (utf8_dup_to_buf): Replacement for utf8_dup_to_uc which takes an extra agrgument, whether to null-terminate or not. (utf8_dup_to): Apply security check here: is the resulting string as long as utf8_to says it should be? If not, it contains embedded nulls. Throw an exception. * utf.h (utf8_from_uc, utf8_to_uc, utf8_dup_from_uc, utf8_dup_to_uc): Declarations removed. (utf8_from_buf, utf8_to_buf, utf8_dup_to_buf): Declared. --- lib.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'lib.c') diff --git a/lib.c b/lib.c index beb5762c..2e5091dd 100644 --- a/lib.c +++ b/lib.c @@ -3949,7 +3949,8 @@ val int_str(val str, val base) if ((value == LONG_MAX || value == LONG_MIN) && errno == ERANGE) { val bignum = make_bignum(); - unsigned char *ucs = utf8_dup_to_uc(wcs); + size_t size; + unsigned char *ucs = utf8_dup_to_buf(wcs, &size, 1); mp_err err = mp_read_radix(mp(bignum), ucs, b); free(ucs); /* TODO: make wchar_t version of mp_read_radix. */ -- cgit v1.2.3