UTF-16

A code point below U+10000 is a single code unit. A code point from U+10000 to U+10FFFF is a surrogate pair: a high surrogate in D800–DBFF carrying the top ten bits of c - 0x10000, then a low surrogate in DC00–DFFF carrying the bottom ten bits. A surrogate that is not part of such a pair is ill-formed.

Decoding uses the decoder of UTF16.decodeIter. decode reports the position of the first unpaired surrogate, as an index in the slice.

def Unicode.UTF16.encodeChar (c : Char) : ScalarArray.UInt16Array

The UTF-16 code units of c: one for a character of the BMP, two for a surrogate pair.

Equations

• Unicode.UTF16.encodeChar c = Unicode.UTF16.push✝ (ScalarArray.UInt16Array.emptyWithCapacity 2) c

def Unicode.UTF16.encode (s : String) : ScalarArray.UInt16Array

The UTF-16 encoding of s.

Equations

• Unicode.UTF16.encode s = Std.Iter.fold Unicode.UTF16.push✝ (ScalarArray.UInt16Array.emptyWithCapacity s.utf8ByteSize) s.chars

def Unicode.UTF16.decode (s : ScalarArray.UInt16Array.Slice) : Except DecodeError String

Decodes the UTF-16 code units of s, or returns an error with the index in s of the first unpaired surrogate.

Equations

• One or more equations did not get rendered due to their size.

def Unicode.UTF16.decode? (s : ScalarArray.UInt16Array.Slice) : Option String

Decodes the UTF-16 code units of s, or returns none if s has an unpaired surrogate.

Equations

• Unicode.UTF16.decode? s = (Unicode.UTF16.decode s).toOption

def Unicode.UTF16.decode! (s : ScalarArray.UInt16Array.Slice) : String

Decodes the UTF-16 code units of s, panicking if s has an unpaired surrogate.

Equations

• One or more equations did not get rendered due to their size.

def Unicode.UTF16.decodeLossy (s : ScalarArray.UInt16Array.Slice) : String

Decodes the UTF-16 code units of s, replacing each unpaired surrogate with U+FFFD.

Equations

• Unicode.UTF16.decodeLossy s = Unicode.Decoder.foldLossy { } String.push "" (Std.Slice.iter s)

def Unicode.UTF16.validate (s : ScalarArray.UInt16Array.Slice) : Bool

Whether s is well-formed UTF-16, that is, has no unpaired surrogate.

Equations

• Unicode.UTF16.validate s = (Unicode.UTF16.foldUntilError✝ (fun (x : Unit) (x_1 : Char) => ()) () (Std.Slice.iter s)).snd.isNone