Java’s garbage collection feature provides significant benefits from a security perspective over non-garbage-collected languages such as C and C++. The garbage collector (GC) is designed to automatically reclaim unreachable memory and avoid memory leaks. Although it is quite adept at performing this task, a malicious attacker can nevertheless launch a denial-of-service (DoS) attack, for example, by inducing abnormal heap memory allocation or abnormally prolonged object retention. For example, some versions of the GC could need to halt all executing threads to keep up with incoming allocation requests that trigger increased heap management activity. System throughput rapidly diminishes in this scenario. Real-time systems, in particular, are vulnerable to a more subtle slow-heap-exhaustion DoS attack, perpetrated by stealing CPU cycles. An attacker can perform memory allocations in a way that increases the consumption of resources (such as CPU, battery power, and memory) without triggering an OutOfMemoryError. Writing garbage-collection-friendly code helps restrict many attack avenues.

Use Short-Lived Immutable Objects

Since JDK 1.2, the generational GC has reduced memory allocation costs to low levels, in many cases lower than in C or C++. Generational garbage collection reduces garbage collection costs by grouping objects into generations. The younger generation consists of short-lived objects. The GC performs a minor collection on the younger generation when it fills up with dead objects [[Oracle 2010a]]. Improved garbage collection algorithms have reduced the cost of garbage collection so that it is proportional to the number of live objects in the younger generation rather than to the number of objects allocated since the last garbage collection.

Note that objects in the younger generation that persist for longer durations are tenured and are moved to the tenured generation. Few younger generation objects continue to live through to the next garbage collection cycle. The rest become ready to be collected in the impending collection cycle [[Oracle 2010a]].

With generational GCs, use of short-lived immutable objects is generally more efficient than use of long-lived mutable objects, such as object pools. Avoiding object pools improves the GC’s efficiency. Object pools bring additional costs and risks: they can create synchronization problems and can require explicit management of deallocations, possibly creating problems with dangling pointers. Further, determining the correct amount of memory to reserve for an object pool can be difficult, especially for mission-critical code. Use of long-lived mutable objects remains appropriate when allocation of objects is particularly expensive (for example, when performing multiple joins across databases). Similarly, object pools are an appropriate design choice when the objects represent scarce resources, such as thread pools and database connections.

The rules FIO00-J. Defensively copy mutable inputs and mutable internal components and OBJ09-J. Defensively copy private mutable class members before returning their references promote garbage-collection-friendly code.

Avoid Large Objects

The allocation of large objects is expensive, and the cost to initialize their fields is proportional to their size. Additionally, frequent allocation of large objects of different sizes can cause fragmentation issues or noncompacting collect operations.

Do Not Use Direct Buffers for Short-Lived, Infrequently Used Objects

The new I/O (NIO) classes in java.nio allow the creation and use of direct buffers. These buffers tremendously increase throughput for repeated I/O activities. However, their creation and reclamation is more expensive than the creation and reclamation for heap-based nondirect buffers because direct buffers are managed using OS-specific native code. This added management cost makes direct buffers a poor choice for single-use or infrequently used cases. Direct buffers are also not subject to Java’s GC, which can cause memory leaks. Frequent allocation of large direct buffers can cause an OutOfMemoryError.

Noncompliant Code Example

This noncompliant code example uses both a short-lived local object rarelyUsedBuffer and a long-lived heavily used object heavilyUsedBuffer. Both are allocated in nonheap memory and are not garbage collected.

ByteBuffer rarelyUsedBuffer = ByteBuffer.allocateDirect(8192);
// use rarelyUsedBuffer once

ByteBuffer heavilyUsedBuffer = ByteBuffer.allocateDirect(8192);
// use heavilyUsedBuffer many times

Compliant Solution

This compliant solution uses an indirect buffer to allocate the short-lived, infrequently used object. The heavily used buffer appropriately continues to use a nonheap, non-garbage-collected direct buffer.

ByteBuffer rarelyUsedBuffer = ByteBuffer.allocate(8192);
// use rarelyUsedBuffer once

ByteBuffer heavilyUsedBuffer = ByteBuffer.allocateDirect(8192);
// use heavilyUsedBuffer many times

Do Not Attempt to Help the Garbage Collector by Setting Local Reference Variables to Null

Setting local reference variables to null to “help the garbage collector” is unnecessary. It adds clutter to the code and can introduce subtle bugs. Java just-in-time compilers (JITs) can perform an equivalent liveness analysis; in fact, most implementations do this. A related bad practice is use of a finalizer to null out references. See rule MET18-J. Avoid using finalizers for additional details.

Noncompliant Code Example

In this noncompliant code example, buffer is a local variable that holds a reference to a temporary array. The programmer attempts to help the GC by assigning null to the buffer array when it is no longer needed.

int[] buffer = new int[100];
doSomething(buffer);
buffer = null  // No need to explicitly assign null

Compliant Solution

Program logic occasionally requires tight control over the lifetime of an object referenced from a local variable. In the unusual cases where such control is necessary, use a lexical block to limit the scope of the variable because the GC can collect the object immediately when it goes out of scope [[Bloch 2008]].

This compliant solution uses a lexical block to control the scope, and consequently the lifetime, of the buffer object.

{ // limit the scope of buffer
  int[] buffer = new int[100];
  doSomething(buffer);
}

Long-Lived Objects Containing Short-Lived Objects

Always remove short-lived objects from long-lived container objects when the task is over. For example, objects attached to a java.nio.channels.SelectionKey object must be removed when they are no longer needed. Doing so reduces the possibility of memory leaks. Similarly, use of array-based data structures such as ArrayLists can introduce a requirement to indicate the absence of an entry by explicitly setting its individual array element to null.

Noncompliant Code Example (Removing Short-Lived Objects)

In this noncompliant code example, a long-lived ArrayList contains references to both long- and short-lived elements. The programmer marks elements that have become irrelevant by setting a “dead” flag in the object.

class DataElement {
   private boolean dead = false;
   // other fields

   public boolean isDead() { return dead; }
   public void killMe() { dead = true; }
}

// elsewhere
ArrayList longLivedList = new ArrayList<DataElement>(...);

// processing that renders an element irrelevant
// Kill the element that is now irrelevant
longLivedList.get(someIndex).killMe();

The GC cannot collect the dead DataElement object until it becomes unreferenced. Note that all methods that operate on objects of class DataElement must check whether the instance in hand is dead.

Compliant Solution (Set Reference to null)

In this compliant solution, rather than use a dead flag, the programmer assigns null to ArrayList elements that have become irrelevant.

class DataElement {
   // dead flag removed

   // other fields
}

// elsewhere
ArrayList longLivedList = new ArrayList<DataElement>(...);

// processing that renders an element irrelevant
// set the reference to the irrelevant DataElement to null
longLivedList.set(someIndex, null);

Note that all code that operates on the longLivedList must now check for list entries that are null.

Compliant Solution (Use Null Object Pattern)

This compliant solution avoids the problems associated with intentionally null references by using a singleton sentinel object. This technique is known as the Null Object pattern (also as the Sentinel pattern). When feasible, programmers should choose this design pattern over the explicit null reference values.

class DataElement {
  public static final DataElement NULL = createSentinel();
   // dead flag removed

   // other fields

  private static final DataElement createSentinel() {
     // allocate a sentinel object, setting all its fields
     // to carefully chosen "do nothing" values
  }
}

// elsewhere
ArrayList longLivedList = new ArrayList<DataElement>(...);

// processing that renders an element irrelevant
// set the reference to the irrelevant DataElement to
// the NULL object
longLivedList.set(someIndex, NULL);

When using this pattern, the NULL object must be a singleton and must be final. It may be either public or private, depending on the overall design of the DataElement class. The state of the NULL object should be immutable after creation; immutability can be enforced either by using final fields or by explicit code in the methods of the DataElement class. See [[Grand 2002]] Chapter 8, “Behavioral Patterns, the Null Object,” for additional information on this design pattern.

Do Not Explicitly Invoke the Garbage Collector

The GC can be explicitly invoked by calling the System.gc() method. Even though the documentation says that it “runs the garbage collector,” there is no guarantee as to when the GC will actually run. In fact, the call only suggests that the GC will subsequently execute.

• Irresponsible use of this feature can severely degrade system performance by triggering garbage collection at inopportune moments rather than waiting until ripe periods when it is safe to garbage collect without significant interruption of the program’s execution.

In the Java Hotspot VM (default since JDK 1.2), System.gc() forces an explicit garbage collection. Such calls can be buried deep within libraries, so they may be difficult to trace. To ignore the call in such cases, use the flag -XX:+DisableExplicitGC. To avoid long pauses while performing a full GC, a less demanding concurrent cycle may be invoked by specifying the flag -XX:ExplicitGCInvokedConcurrent.

Exceptions

OBJ15-EX0: When an application goes through several phases such as an initialization and a ready phase, it could require heap compaction between phases. Given an uneventful period, System.gc() may be invoked in such cases, provided that there is a suitable uneventful period between phases.

OBJ15-EX1: System.gc() may be invoked as a last resort in a catch block that is attempting to recover from an OutOfMemoryError.

Risk Assessment

Misusing garbage collection utilities can cause severe performance degradation resulting in a DoS attach.

Related Vulnerabilities

GERONIMO-4574

Related Guidelines

Bibliography

OBJ03-J. Do not mix generic with nongeneric raw types in new code      04. Object Orientation (OBJ)      05. Methods (MET)