UTF-8 API overhaul: security, and other concerns.

The main aim here is to pave the way for conversion between
arbitrary buffers of bytes (that may include embedded NUL
characters) and a wide string.

Also, a potential security hole is closed.  When we convert a
TXR string to UTF-8 for use with some C library API, any
embedded pnul characters (U+DC00) turn into NUL
bytes which effectively cut the UTF-8 string short, and
silently so. The C library function receives a shortened
string. This could be exploitable in some situations.

* lib.c (int_str): Use utf8_dup_to_buf instead of
utf8_dup_to_uc. Pass 1 to have the buffer null-terminated,
since mp_read_radix depends on it.

* stream.c (make_string_byte_input_stream): Use
utf8_dup_to_buf. This gives us the size, soo we don't have to
call strlen. The buffer is no longer null terminated, but the
byte input stream implementation never relied on this.

* utf8.c (utf8_from_buf): Replacement fors utf8_from_uc
which doesn't assume that the buffer of bytes is
null-terminated.  It can produce a wide string containing
U+DC00 characters corresponding to embedded nulls in the
original buffer.
(utf8_from): Calculate length of null-terminated string and use
utf8_from_buf.
(utf8_to_buf): Replacement for utf8_to_uc. Can produce
a buffer which is or is not null-terminated, based on new
argument.
(utf8_to): Use utf8_to_buf, and ask it to null-terminate,
thus preserving behavior.
(utf8_dup_from_uc): This function was not used anywhere
and is removed.
(utf8_dup_to_buf): Replacement for utf8_dup_to_uc which
takes an extra agrgument, whether to null-terminate
or not.
(utf8_dup_to): Apply security check here: is the resulting
string as long as utf8_to says it should be? If not,
it contains embedded nulls. Throw an exception.

* utf.h (utf8_from_uc, utf8_to_uc, utf8_dup_from_uc,
utf8_dup_to_uc): Declarations removed.
(utf8_from_buf, utf8_to_buf, utf8_dup_to_buf): Declared.

0 files changed, 0 insertions, 0 deletions