CON32-J. Ensure that tasks executing in a thread pool do not fail silently

Tasks that run for extended periods of time should provide a notification mechanism to alert upper layers when they terminate abnormally. Failure to do this does not cause any resource leaks because the threads in the pool are still recycled, however, it makes failure diagnosis extremely difficult.

The best way to handle exceptions at a global level is to use an exception handler. The handler can perform diagnostic actions, clean-up and shutdown the Java Virtual Machine (JVM) or simply log the details of the failure. This guideline may be violated if the code for all runnable and callable tasks has been audited to ensure that no exceptional conditions are possible. Nonetheless, it is usually a good practice to install a task-specific or global exception handler to initiate recovery, or log the exceptional condition.

Noncompliant Code Example (Abnormal task termination)

This noncompliant code example consists of class PoolService that encapsulates a thread pool and a runnable class Task. The run() method of the task can throw runtime exceptions such as NullPointerException.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);
		  
  public void doSomething() {	   
    pool.execute(new Task());
  }	
}

final class Task implements Runnable {
  @Override public void run() {
    // ...
    throw new NullPointerException();
    // ...	
  }
}

The task does not notify upper layers when it terminates unexpectedly as a result of the runtime exception. Moreover, it does not use any recovery mechanism. Consequently, if any Task throws a NullPointerException, the exception is ignored.

Compliant Solution (ThreadPoolExecutor hooks)

Task-specific recovery or clean-up actions can also be performed by overriding the afterExecute() hook of class java.util.concurrent.ThreadPoolExecutor. This hook is called when a task successfully concludes by executing all statements in its run() method, or halts because of an exception (A java.lang.Error might not be captured on specific implementations, see Bug ID 6450211 [[SDN 08]]). When using this approach, substitute the executor service with a custom ThreadPoolExecutor that overrides the afterExecute() hook as shown below:

final class PoolService {
  // The values have been hardcoded for brevity
  ExecutorService pool = new CustomThreadPoolExecutor(10, 10, 10, TimeUnit.SECONDS, 
                         new ArrayBlockingQueue<Runnable>(10));
  // ...
}

class CustomThreadPoolExecutor extends ThreadPoolExecutor {
  // ... Constructor ...

  @Override
  public void afterExecute(Runnable r, Throwable t) {
    super.afterExecute(r, t);
    if (t != null) {
      // Exception occurred, forward to handler
    }
    // ... Perform task-specific clean-up actions
  }

  @Override
  public void terminated() {
    super.terminated();
    // ... Perform final clean-up actions
  }
}

Similarly, the terminated() hook is called after all the tasks have finished executing, and the Executor has terminated cleanly. This hook can be overridden to release resources acquired by the thread pool over its lifetime, much like a finally block.

Compliant Solution (Uncaught exception handler)

This compliant solution sets an uncaught exception handler on behalf of the thread pool. An argument of type ThreadFactory is passed to the thread pool while constructing it. The factory is responsible for creating new threads and setting the uncaught exception handler on their behalf. The class Task remains the same as the noncompliant code example.

final class PoolService {
  private static final ThreadFactory factory = new
    ExceptionThreadFactory(new MyExceptionHandler());
  private static final ExecutorService pool =
    Executors.newFixedThreadPool(10, factory);

  public void doSomething() {	   
    pool.execute(new Task()); // Task is a runnable class	      	    	      
  }
 
  public static class ExceptionThreadFactory implements ThreadFactory  {			
    private static final ThreadFactory defaultFactory =
      Executors.defaultThreadFactory();
    private final Thread.UncaughtExceptionHandler handler;

    public ExceptionThreadFactory(Thread.UncaughtExceptionHandler handler) {
      this.handler = handler;
    }

    @Override public Thread newThread(Runnable run) {
      Thread thread = defaultFactory.newThread(run);
      thread.setUncaughtExceptionHandler(handler);
      return thread;
    }
  }
	  
  public static class MyExceptionHandler extends ExceptionReporter 
    implements Thread.UncaughtExceptionHandler {
    // ... 

    @Override public void uncaughtException(Thread thread, Throwable t) {
      // Recovery or logging code		      
    }
  }	  
}

The ExecutorService.submit() method can be used to submit a task to a thread pool instead of the execute() method to obtain a Future object. Note that the uncaught exception handler is not called if ExecutorService.submit() is invoked. This is because the thrown exception is considered to be part of the return status and is consequently wrapped in an ExecutionException and re-thrown by Future.get() [[Goetz 06]].

Compliant Solution (Future<V> and submit())

This compliant solution invokes the ExecutorService.submit() method to submit the task so that a Future object can be obtained. It uses the Future object to let the task re-throw the exception so that it can be handled locally.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);

  public void doSomething() {     
    Future<?> future = pool.submit(new Task());

    // ... 

    try {
      future.get();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt(); // Reset interrupted status      
    } catch (ExecutionException e) {
      Throwable exception = e.getCause();      
      // Forward to exception reporter
    }
  }  
}

Furthermore, any exception that precludes doSomething() from obtaining the Future value can be handled as required.

Exceptions

EX1: This guideline may be violated if the code for all runnable and callable tasks has been audited to ensure that no exceptional conditions are possible.

Risk Assessment

Failing to provide a mechanism to report that tasks in a thread pool failed as a result of an exceptional condition, can make it harder to find the source of the issue.

To-Do List

Automated Detection

TODO

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

[[API 06]] interfaces ExecutorService, ThreadFactory and class Thread
[[Goetz 06]] Chapter 7.3: Handling abnormal thread termination

CON31-J. Ensure that tasks submitted to a thread pool are interruptible      11. Concurrency (CON)      CON33-J. Ensure ThreadLocal variables are reinitialized when using thread pools