CON32-J. Ensure that tasks executing in a thread pool do not fail silently

Long-running tasks should provide a mechanism to notify the application upon abnormal termination. Failure to do so does not cause any resource leaks because the threads in the pool are still recycled, however, it makes failure diagnosis extremely difficult.

The best way to handle exceptions at the application level is to use an exception handler. The handler can perform diagnostic actions, clean-up and shutdown the Java Virtual Machine (JVM) or simply log the details of the failure.

Noncompliant Code Example (Abnormal task termination)

This noncompliant code example consists of the PoolService class that encapsulates a thread pool and a runnable Task class. The Task.run() can throw runtime exceptions such as NullPointerException.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);
		  
  public void doSomething() {	   
    pool.execute(new Task());
  }	
}

final class Task implements Runnable {
  @Override public void run() {
    // ...
    throw new NullPointerException();
    // ...	
  }
}

The task does not notify the application when it terminates unexpectedly as a result of the runtime exception. Moreover, it does not use any recovery mechanism. Consequently, if Task throws a NullPointerException the exception is ignored.

Compliant Solution (ThreadPoolExecutor hooks)

Task-specific recovery or clean-up actions can be performed by overriding the afterExecute() hook of class java.util.concurrent.ThreadPoolExecutor. This hook is called when a task successfully concludes by executing all statements in its run() method, or halts because of an exception (java.lang.Error might not be captured on specific implementations, see Bug ID 6450211 [[SDN 08]]). When using this approach, substitute the executor service with a custom ThreadPoolExecutor that overrides the afterExecute() hook as shown below:

final class PoolService {
  // The values have been hardcoded for brevity
  ExecutorService pool = new CustomThreadPoolExecutor(10, 10, 10, TimeUnit.SECONDS, 
                         new ArrayBlockingQueue<Runnable>(10));
  // ...
}

class CustomThreadPoolExecutor extends ThreadPoolExecutor {
  // ... Constructor ...

  @Override
  public void afterExecute(Runnable r, Throwable t) {
    super.afterExecute(r, t);
    if (t != null) {
      // Exception occurred, forward to handler
    }
    // ... Perform task-specific clean-up actions
  }

  @Override
  public void terminated() {
    super.terminated();
    // ... Perform final clean-up actions
  }
}

The terminated() hook is called after all the tasks have finished executing, and the Executor has terminated cleanly. This hook can be overridden to release resources acquired by the thread pool, much like a finally block.

Compliant Solution (Uncaught Exception Handler)

This compliant solution sets an uncaught exception handler on behalf of the thread pool. An argument of type ThreadFactory is passed to the thread pool during construction. The factory is responsible for creating new threads and setting the uncaught exception handler on their behalf. The Task class is unchanged from the noncompliant code example.

final class PoolService {
  private static final ThreadFactory factory = new
    ExceptionThreadFactory(new MyExceptionHandler());
  private static final ExecutorService pool =
    Executors.newFixedThreadPool(10, factory);

  public void doSomething() {	   
    pool.execute(new Task()); // Task is a runnable class	      	    	      
  }
 
  public static class ExceptionThreadFactory implements ThreadFactory  {			
    private static final ThreadFactory defaultFactory =
      Executors.defaultThreadFactory();
    private final Thread.UncaughtExceptionHandler handler;

    public ExceptionThreadFactory(Thread.UncaughtExceptionHandler handler) {
      this.handler = handler;
    }

    @Override public Thread newThread(Runnable run) {
      Thread thread = defaultFactory.newThread(run);
      thread.setUncaughtExceptionHandler(handler);
      return thread;
    }
  }
	  
  public static class MyExceptionHandler extends ExceptionReporter 
    implements Thread.UncaughtExceptionHandler {
    // ... 

    @Override public void uncaughtException(Thread thread, Throwable t) {
      // Recovery or logging code		      
    }
  }	  
}

The ExecutorService.submit() method can be used to submit a task to a thread pool instead of the execute() method to obtain a Future object. Note that the uncaught exception handler is not called if ExecutorService.submit() is invoked. This is because the thrown exception is considered to be part of the return status and is consequently wrapped in an ExecutionException and re-thrown by Future.get() [[Goetz 06]].

Compliant Solution (Future<V> and submit())

This compliant solution invokes the ExecutorService.submit() method to submit the task so that a Future object can be obtained. It uses the Future object to let the task re-throw the exception so that it can be handled locally.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);

  public void doSomething() {     
    Future<?> future = pool.submit(new Task());

    // ... 

    try {
      future.get();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt(); // Reset interrupted status      
    } catch (ExecutionException e) {
      Throwable exception = e.getCause();      
      // Forward to exception reporter
    }
  }  
}

Furthermore, any exception that prevents doSomething() from obtaining the Future value can be handled as required.

Exceptions

CON32-EX1: This guideline may be violated if the code for all runnable and callable tasks has been audited to ensure that no exceptional conditions are possible. Nonetheless, it is usually a good practice to install a task-specific or global exception handler to initiate recovery or log the exceptional condition.

Risk Assessment

Failing to provide a mechanism to report that tasks in a thread pool failed as a result of an exceptional condition, can make it harder to find the source of the issue.

To-Do List

Automated Detection

TODO

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

[[API 06]] interfaces ExecutorService, ThreadFactory and class Thread
[[Goetz 06]] Chapter 7.3: Handling abnormal thread termination

CON31-J. Ensure that tasks submitted to a thread pool are interruptible      11. Concurrency (CON)      CON33-J. Ensure ThreadLocal variables are reinitialized when using thread pools