Page History

Serialization can prevent extend the lifetime of objects, preventing their garbage collection and thus induce memory leaks. Every time an . The ObjectOutputStream ensures that each object is written out to a stream, a reference to the object is retained. If the same object (regardless of its contents) is written out to the same the stream only once by retaining a reference (or handle) to each object written to the stream. When a previously written object is subsequently written to the stream again, it is replaced with a reference to the originally cached object. The garbage collector cannot reclaim the memory associated with new objects as it cannot deal with live references. written data in the stream. Note that this substitution takes place regardless whether the object's contents have changed in the interim. It requires a table of references to be maintained to keep track of previously serialized objects. This table of references prevents garbage collection of the previously serialized objects because the garbage collector cannot collect object instances referred to by live references.

This behavior is both desirable and correct for data that may contain arbitrary object graphs, especially when the graphs are fully allocated and constructed prior to serialization. However, it can lead to memory exhaustion when serializing data that lacks references to other objects being serialized and can be allocated in part or in full after serialization has begun. One such example is serializing a data stream from an external sensor. In such cases, programs must take additional action to avoid memory exhaustion. That is, programs reading in independent serialized data must reset the table of references between reads to prevent memory exhaustion.

This rule is a specific instance of the more general MSC05-J. Do not exhaust heap space.

Noncompliant Code Example

This noncompliant code example reads and serializes data from an external sensor. Each invocation of the readSensorData() method returns a newly created SensorData instance, each containing one megabyte of data. SensorData instances are pure data streams, containing data and arrays but lacking references to other SensorData objects.

As already described, the ObjectOutputStream maintains a cache of previously written objects. Consequently, all SensorData objects remain alive until the cache itself becomes garbage-collected. An OutOfMemoryError can occure because the stream remains writes an array object arr to the underlying stream. The object is created afresh within the loop and filled uniformly with the value of the loop counter. Unfortunately, an OutOfMemoryError surfaces as the stream is kept open while new objects are being written to it.

class SensorData implements Serializable {
  // 1 MB of data per instance!
  ... 
  public static SensorData readSensorData() {...}
  public static boolean isAvailable() {...}
}

class MemoryLeakSerializeSensorData {
  public static void main(String[] args) throws IOException {
    ObjectOutputStream out = null;
    try {
      out = new ObjectOutputStream(
          new BufferedOutputStream(new FileOutputStream("ser.dat")));
      forwhile (int i = 0; i < 1024; i++) {
      byte[] arr = new byte[100 * 1024];
      Arrays.fill(arr, (byte) iSensorData.isAvailable()) {
        // Note that each SensorData object is 1 MB in size
        SensorData sd = SensorData.readSensorData();
        out.writeObject(arrsd);
      }
    } finally {
      if (out != null) {
        out.close();
      }
    }
  }
}

Compliant Solution

Ideally, the stream should be closed as soon as the work is accomplished. This compliant solution adopts an alternative approach takes advantage of the known properties of the sensor data by resetting the output stream after every write so that the internal cache each write. The reset clears the output stream's internal object cache; consequently, the cache no longer maintains live references, allowing the garbage collector to resumereferences to previously written SensorData objects. The garbage collector can collect SensorData instances that are no longer needed.

class NoMemoryLeakSerializeSensorData {
  public static void main(String[] args) throws IOException {
    ObjectOutputStream out = null;
    try {
      out = new ObjectOutputStream(
          new BufferedOutputStream(new FileOutputStream("ser.dat")));
      forwhile (int i = 0; i < 1024; i++) {
      byte[] arr = new byte[100 * 1024];
      Arrays.fill(arr, (byte) iSensorData.isAvailable()) {
        // Note that each SensorData object is 1 MB in size
        SensorData sd = SensorData.readSensorData();
        out.writeObject(arrsd);
        out.reset(); // resetReset the stream
      }
    } finally {
      if (out != null) {
        out.close();
      }
    }
  }
}

Risk Assessment

Memory and resource leaks during serialization can corrupt the state of the object or crash the JVMresult in a resource exhaustion attack or can crash the Java Virtual Machine.

Automated Detection

TODO

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

\[[API 06|AA. Java References#API 06]\]
\[[Sun 06|AA. Java References#Sun 06]\] "Serialization specification"
\[[Harold 06|AA. Java References#Harold 06]\] 13.4. Performance 

Detecting code that should be considered privileged or sensitive requires programmer assistance. Given identified privileged code as a starting point, automated tools could compute the closure of all code that can be invoked from that point. Such a tool could plausibly determine whether all code in that closure exists within a single package. A further check of whether the package is sealed is feasible.

Related Guidelines

Bibliography

...

Image Added Image Added Image AddedSER00-J. Maintain serialization compatibility during class evolution      11. Serialization (SER)      11. Serialization (SER)