Page History

The char data type is based on the original Unicode specification, which defined characters as fixed-width 16-bit entities. The Unicode Standard has since been changed to allow for characters whose representation requires more than 16 bits. The range of Unicode code points is now U+0000 to U+10FFFF. The set of characters from U+0000 to U+FFFF is referred to as the basic multilingual plane (BMP) while characters whose code points are greater than U+FFFF are called supplementary characters. Such characters are generally rare, but some are used, for example, as part of Chinese and Japanese personal names. To support supplementary characters without changing the char primitive data type and causing incompatibility with previous Java programs, supplementary characters are defined by a pair of code point values that are of Unicode code units called surrogates. According to the Java API [API 2014] class Character documentation (Unicode Character Representations):

The Java platform uses the UTF-16 representation in char arrays and in the String and StringBuffer classes. In this representation, supplementary characters are represented as a pair of char values, the first from the high-surrogates range, (\uD800-\uDBFF), the second from the low-surrogates range (\uDC00-\uDFFF)..

A char value, therefore, represents BMP code points, including the surrogate code points, or code units of the UTF-16 encoding. An int value represents all Unicode code points, including supplementary code points. The lower (least significant) 21 bits of int are used to represent Unicode code points and the upper (most significant) 11 bits must be zero

...

.  Similar to UTF-8 (see STR00-J. Don't form strings containing partial characters from variable-width encodings), UTF-16 is a variable-width encoding.  It  uses a single 16-bit code unit to encode the most common 63K characters, and a pair of 16-bit code units to encode the 1M less commonly used characters in Unicode.  Because UTF-16 code point ranges for high and low surrogates, as well as for single units are all completely disjoint there are no false matches, the location of the character boundary can be directly determined from each code unit value, and a dropped surrogate will corrupt only a single character. 

Similar to UTF-8 and other variable-width encodings, programmers must be careful when reading UTF-16 data as a series of bytes to not form strings containing partial Unicode code points (that is, a high surrogate value without a corresponding low surrogate).  Because the UTF-16 representation is also used in char arrays and in the String and StringBuffer classes, care must also be taken when manipulating This typically means using methods that accept a Unicode code point as an int value avoiding methods that accept a Unicode code unit as a char value as these methods cannot support supplementary characters

...

.

Noncompliant Code Example (Read)

...

public final int MAX_SIZE = 1024;

public String readBytes(Socket socket) throws IOException {
  InputStream in = socket.getInputStream();
  byte[] data = new byte[MAX_SIZE+1];
  int offset = 0;
  int bytesRead = 0;
  String str = new String();
  while ((bytesRead = in.read(data, offset, data.length - offset)) != -1) {
    offset += bytesRead;
    str += new String(data, offset, data.length - offset, "UTF-816");
    if (offset >= data.length) {
      throw new IOException("Too much input");
    }
  }
  in.close();
  return str;
}

This code fails to account for the interaction between characters represented with a multibyte encoding variable-width character encodings and the boundaries between the loop iterations. If the last byte read from the data stream in one read() operation is the leading byte of a multibyte character, the trailing bytes are is a high surrogate value the corresponding low surrogate is not encountered until the next iteration of the while loopsubsequent read. However, multibyte variable-width encoding is resolved during construction of the new String within the loop. Consequently, the multibyte encoding supplementary characters can be interpreted incorrectly.

...

public final int MAX_SIZE = 1024;

public String readBytes(Socket socket) throws IOException {
  InputStream in = socket.getInputStream();
  byte[] data = new byte[MAX_SIZE+1];
  int offset = 0;
  int bytesRead = 0;
  while ((bytesRead = in.read(data, offset, data.length - offset)) != -1) {
    offset += bytesRead;
    if (offset >= data.length) {
      throw new IOException("Too much input");
    }
  }
  String str = new String(data, "UTF-816");
  in.close();
  return str;
}

This code avoids splitting multi-byte encoded supplementary characters across buffers by deferring construction of the result string until the data has been read in full.

...

public final int MAX_SIZE = 1024;

public String readBytes(Socket socket) throws IOException {
  InputStream in = socket.getInputStream();
  Reader r = new InputStreamReader(in, "UTF-816");
  char[] data = new char[MAX_SIZE+1];
  int offset = 0;
  int charsRead = 0;
  String str = new String(data);
  while ((charsRead = r.read(data, offset, data.length - offset)) != -1) {
    offset += charsRead;
    str += new String(data, offset, data.length - offset);
    if (offset >= data.length) {
      throw new IOException("Too much input");
    }
  }
  in.close();
  return str;
}

...