IDS17-J. Use compatible encodings on both sides of file or network IO

Every Java platform has a default character encoding. The available encodings are listed in the Supported Encodings document [[Encodings 2006]]. Conversion between characters and sequences of bytes requires a character encoding to specify the details of the conversion. Such conversions use the system default encoding in the absence of an explicitly specified encoding. When characters are converted into an array of bytes to be sent as output, transmitted across some medium, input, and converted back into characters, compatible encodings must be used on both sides of the conversation; disagreement over character encodings can cause data corruption.

According to the Java API [API 2006] for the String class:

The length of the new String is a function of the charset, and hence may not be equal to the length of the byte array. The behavior of this constructor when the given bytes are not valid in the given charset is unspecified.

This guideline falls under EX0 of guideline FIO11-J. Do not attempt to read raw binary data as character data. Also, see the related guideline IDS13-J. Do not assume every character in a string is the same size for more information.

Noncompliant Code Example

This noncompliant code example reads a byte array and converts it into a String using the platform's default character encoding. When the default encoding differs from the encoding that was used to produce the byte array, the resulting String is likely to be incorrect. Undefined behavior can occur when some of the input lacks a valid character representation in the default encoding.

try {
  FileInputStream fis = new FileInputStream("SomeFile");
  DataInputStream dis = new DataInputStream(fis);
  byte[] data = new byte[1024];
  dis.readFully(data);
  String result = new String(data);
} catch (IOException x) {
  // handle error
}

Compliant Solution

This compliant solution explicitly specifies the intended character encoding by passing it as the second argument to the String constructor (e.g. the string encoding in this example).

try {
  FileInputStream fis = new FileInputStream("SomeFile");
  DataInputStream dis = new DataInputStream(fis);
  byte[] data = new byte[1024];
  dis.readFully(data);
  String encoding = "SomeEncoding"; // for example, "UTF-16LE"
  String result = new String(data, encoding);
} catch (IOException x) {
  // handle error
}

Exceptions

IDS17-EX0: An explicit character encoding may be omitted on the receiving side when the data was produced by a Java application that uses the same platform and default character encoding.

Risk Assessment

Failure to specify the character encoding while performing file or network IO can corrupt the data.

Automated Detection

Sound automated detection of this vulnerability is not feasible.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this guideline on the CERT website.

Bibliography

[[Encodings 2006]]

FIO02-J. Keep track of the number of bytes read      12. Input Output (FIO)      FIO05-J. Do not create multiple buffered wrappers on a single InputStream