TPS03-J. Ensure that tasks executing in a thread pool do not fail silently

Long-running tasks should provide a mechanism for notifying the application upon abnormal termination. Failure to do so does not cause any resource leaks because the threads in the pool are still recycled, but it makes failure diagnosis extremely difficult.

The best way to handle exceptions at the application level is to use an exception handler. The handler can perform diagnostic actions, clean-up and shut down the JVM, or simply log the details of the failure.

Noncompliant Code Example (Abnormal Task Termination)

This noncompliant code example consists of the PoolService class that encapsulates a thread pool and a runnable Task class. The Task.run() method can throw runtime exceptions, such as NullPointerException.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);

  public void doSomething() {
    pool.execute(new Task());
  }
}

final class Task implements Runnable {
  @Override public void run() {
    // ...
    throw new NullPointerException();
    // ...
  }
}

The task does not notify the application when it terminates unexpectedly as a result of the runtime exception. Moreover, it does not use any recovery mechanism. Consequently, if Task throws a NullPointerException the exception is ignored.

Compliant Solution (ThreadPoolExecutor Hooks)

Task-specific recovery or clean-up actions can be performed by overriding the afterExecute() hook of the java.util.concurrent.ThreadPoolExecutor class. This hook is called when a task concludes successfully by executing all statements in its run() method or halts because of an exception. (java.lang.Error might not be captured on specific implementations. See Bug ID 6450211 for more information [[SDN 2008]]). When using this approach, substitute the executor service with a custom ThreadPoolExecutor that overrides the afterExecute() hook as shown below:

final class PoolService {
  // The values have been hard-coded for brevity
  ExecutorService pool = new CustomThreadPoolExecutor(10, 10, 10, TimeUnit.SECONDS,
                         new ArrayBlockingQueue<Runnable>(10));
  // ...
}

class CustomThreadPoolExecutor extends ThreadPoolExecutor {
  // ... Constructor ...
  public CustomThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) 
  { 
    super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue); 
  }


  @Override
  public void afterExecute(Runnable r, Throwable t) {
    super.afterExecute(r, t);
    if (t != null) {
      // Exception occurred, forward to handler
    }
    // ... Perform task-specific clean-up actions
  }

  @Override
  public void terminated() {
    super.terminated();
    // ... Perform final clean-up actions
  }
}

The terminated() hook is called after all the tasks have finished executing and the Executor has terminated cleanly. This hook can be overridden to release resources acquired by the thread pool, much like a finally block.

Compliant Solution (Uncaught Exception Handler)

This compliant solution sets an uncaught exception handler on behalf of the thread pool. A ThreadFactory argument is passed to the thread pool during construction. The factory is responsible for creating new threads and setting the uncaught exception handler on their behalf. The Task class is unchanged from the noncompliant code example.

final class PoolService {
  private static final ThreadFactory factory = new
    ExceptionThreadFactory(new MyExceptionHandler());
  private static final ExecutorService pool =
    Executors.newFixedThreadPool(10, factory);

  public void doSomething() {
    pool.execute(new Task()); // Task is a runnable class
  }

  public static class ExceptionThreadFactory implements ThreadFactory  {
    private static final ThreadFactory defaultFactory =
      Executors.defaultThreadFactory();
    private final Thread.UncaughtExceptionHandler handler;

    public ExceptionThreadFactory(Thread.UncaughtExceptionHandler handler) {
      this.handler = handler;
    }

    @Override public Thread newThread(Runnable run) {
      Thread thread = defaultFactory.newThread(run);
      thread.setUncaughtExceptionHandler(handler);
      return thread;
    }
  }

  public static class MyExceptionHandler extends ExceptionReporter
    implements Thread.UncaughtExceptionHandler {
    // ...

    @Override public void uncaughtException(Thread thread, Throwable t) {
      // Recovery or logging code
    }
  }
}

The ExecutorService.submit() method can be used to submit a task to a thread pool instead of the execute() method to obtain a Future object. Note that the uncaught exception handler is not called if ExecutorService.submit() is invoked. This is because the thrown exception is considered to be part of the return status and is consequently wrapped in an ExecutionException and re-thrown by Future.get() [[Goetz 2006]].

Compliant Solution (Future<V> and submit())

This compliant solution invokes the ExecutorService.submit() method to submit the task so that a Future object can be obtained. It uses the Future object to let the task re-throw the exception so that it can be handled locally.

final class PoolService {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);

  public void doSomething() {
    Future<?> future = pool.submit(new Task());

    // ...

    try {
      future.get();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt(); // Reset interrupted status
    } catch (ExecutionException e) {
      Throwable exception = e.getCause();
      // Forward to exception reporter
    }
  }
}

Furthermore, any exception that prevents doSomething() from obtaining the Future value can be handled as required.

Exceptions

TPS03-EX1: This rule may be violated if the code for all runnable and callable tasks has been audited to ensure that no exceptional conditions are possible. Nonetheless, it is usually a good practice to install a task-specific or global exception handler to initiate recovery or log the exceptional condition.

Risk Assessment

Failing to provide a mechanism for reporting that tasks in a thread pool failed as a result of an exceptional condition can make it harder to find the source of the issue.

To-Do List

Automated Detection

TODO

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Bibliography

[[API 2006]] interfaces ExecutorService, ThreadFactory and class Thread
[[Goetz 2006]] Chapter 7.3: Handling abnormal thread termination

      10. Thread Pools (TPS)