Page History

public final 

There are certain problems associated with the incorrect use of the {{Executor}} interface. For instance, tasks that depend on other tasks should not execute in the same thread pool. A task that submits another task to a single threaded {{Executor}} remains blocked until the results are received whereas the second task may have dependencies on the first task. This constitutes a deadlock.

h2. Noncompliant Code Example

This noncompliant code example suffers from a _thread starvation deadlock_ issue. It consists of class {{ValidationService}} that performs various input validation tasks such as checking whether a specific user-supplied field exists in a back-end database. The {{fieldAggregator()}} method accepts a variable number of {{String}} arguments and creates a task corresponding to each argument to gain some speedup. The task performs input validation using class {{ValidateInput}}. The class {{ValidateInput}} in turn, creates a task for each request using class {{SanitizeInput}}. All tasks are executed using the same thread pool. The method {{fieldAggregator()}} blocks until all the tasks have finished executing. When all results are available, it aggregates the processed inputs as a {{StringBuffer}} that can be used by its caller.

{code:bgColor=#FFCCCC}
class ValidationService {
  private final ExecutorService pool;

  public ValidationService(int poolSize) {
    pool = Executors.newFixedThreadPool(poolSize);
  }
  
  public void shutdown() {
    pool.shutdown();
  }
   
  public StringBufferStringBuilder fieldAggregator(String... inputs)
  
    throws InterruptedException, ExecutionException   {
   
    StringBufferStringBuilder sb = new StringBufferStringBuilder();
    // Stores the results
    Future<String>[] results = new Future[inputs.length]; 

    // storesSubmits the results
		tasks to thread pool
    for (int i = 0; i < inputs.length; i++) { 
      results[i] = pool.submit(
        new ValidateInput<String>(inputs[i], pool));     
    } 

    for (int i = 0; i < inputs.length; i++) { // AggregateAggregates the results	
     	
      sb.append(results[i].get());	    	
    }
    return sb;
  }
}

public final class ValidateInput<V> implements Callable<V> {
  private final StringV input;
  private final ExecutorService pool;

  ValidateInput(StringV input, ExecutorService pool) {
    this.input = input;
    this.pool = pool;
  }

  @Override public V call() throws Exception {
    // If validation fails, throw an exception here
    // Subtask
    Future<String>Future<V> future = pool.submit(new SanitizeInput<String>SanitizeInput<V>(input)); 
    return (V) future.get();
  }
}

public final class SanitizeInput<V> implements Callable<V> {
  private final StringV input;
	
  SanitizeInput(StringV input) {
    this.input = input;
  }

  @Override public V call() throws Exception {
    // Sanitize input and return
    return (V) input;	
  }
}
{code}

{mc}
// Hidden main() method
public static void main(String[] args) throws InterruptedException, ExecutionException {
  ValidationService vs = new ValidationService(5);
  System.out.println(vs.fieldAggregator("field1", "field2","field3","field4", "field5","field6"));
  vs.shutdown(); 
}
{mc}

Assume that the caller sets the thread pool size as 6. When a caller calls {{ValidationService.fieldAggregator()}} with six arguments that ought to be validated, six tasks are submitted to the thread pool. However, six more tasks corresponding to {{SanitizeInput}} must also execute before these threads can return their results. However, this is not possible because the queue is full with all threads blocked. Choosing a bigger pool size appears to solve the problem, however there is no easy way to determine a suitable size.   

This situation can also occur when using single threaded Executors when the caller creates several sub-tasks and wait for the results. A thread starvation deadlock arises when all the threads executing in the pool are blocked on tasks that are waiting on the queue. A blocking operation within a subtask can also lead to unbounded queue growth. \[[Goetz 06|AA. Java References#Goetz 06]\] 


h2. Compliant Solution


{code:bgColor=#ccccff}
TODO
{code}

Always try to submit independent tasks to the {{Executor}}. Thread starvation issues can be mitigated by choosing a large pool size. Note that operations that have further constraints, such as the total number of database connections or total {{ResultSets}} open at a particular time, impose an upper bound on the thread pool size as each thread continues to block until the resource becomes available. The other rules of fair concurrency, such as not running time consuming tasks, also apply. When this is not possible, expecting to obtain real time result guarantees from the execution of tasks is conceivably, an unreasonable target.

Sometimes, a {{private static}} {{ThreadLocal}} variable is used per thread to maintain local state. When using thread pools, {{ThreadLocal}} variable should be used only if their lifetime is shorter than that of the corresponding task \[[Goetz 06|AA. Java References#Goetz 06]\]. Moreover, such variables should not be used as a communication mechanism between tasks. 

h2. Noncompliant Code Example

This noncompliant code example (based on \[[Gafter 06|AA. Java References#Gafter 06]\]) shows a {{BrowserManager}} class that has several methods that use a fork-join mechanism, that is, they start threads and wait for them to finish. The methods are called in the sequence {{perUser()}}, {{perProfile}} and {{perTab()}}. The method {{methodInvoker}} spawns several instances of the specified runnable depending on the value of the variable {{numberOfTimes}}. A fixed sized thread pool is used to execute the enumerated threads generated at different levels. 

{code:bgColor=#FFCCCC}
public class BrowserManager {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);
  int numberOfTimes;
  private static volatile int count = 0;
  
  public BrowserManager(int n) {
    this.numberOfTimes = n

public final class ValidationService {
  // ...
  public StringBuilder fieldAggregator(String... inputs)
      throws InterruptedException, ExecutionException {
    // ...
    for (int i = 0; i < inputs.length; i++) {
      // Don't pass-in thread pool  
      results[i] = pool.submit(new ValidateInput<String>(inputs[i])); 
    }
    // ...
  }
}

// Does not use same thread pool
public final class ValidateInput<V> implements Callable<V> { 
  private final V input;

  ValidateInput(V input) {
    this.input = input;
  }

  @Override public V call() throws Exception {
    // If validation fails, throw an exception here
    return (V) new SanitizeInput().sanitize(input);
  }
}

public final class SanitizeInput<V> {  // No longer a Callable task
  public SanitizeInput() {}

  public V sanitize(V input) {
    // Sanitize input and return
    return input;
  }
}

public final class BrowserManager {
  private final ExecutorService pool = Executors.newFixedThreadPool(10);
  private final int numberOfTimes;
  private static AtomicInteger count = new AtomicInteger(); // count = 0

  public BrowserManager(int n) {
    numberOfTimes = n;
  }

  public void perUser() {
    methodInvoker(numberOfTimes, "perProfile");
    pool.shutdown();
  }

  public void perProfile() {
    methodInvoker(numberOfTimes, "perTab");
  }

  public void perUserperTab() {	
    methodInvoker(numberOfTimes, "perProfiledoSomething"); 
    pool.shutdown();
  }

  public void perProfiledoSomething() {
    methodInvoker(numberOfTimes, "perTab");	 System.out.println(count.getAndIncrement());
  }

  public void perTab(methodInvoker(int n, final String method) {	
    final  methodInvoker(numberOfTimes, "doSomething")BrowserManager manager = this;
  }

  publicCallable<Object> voidcallable doSomething() = new Callable<Object>() {
    System.out.println(++count);
  }

 @Override public voidObject methodInvokercall(int n, final String method) {
) throws Exception {
       final BrowserManagerMethod fmmeth = thismanager.getClass().getMethod(method);
    Runnable run =  newreturn Runnablemeth.invoke(manager);
 {
     }
 public void run() {
	try {
	};

  Method meth = fm.getClass().getMethod(method);
	  meth.invoke(fm);			Collection<Callable<Object>> collection = 
        } catch (Throwable t) {
	  // Forward to exception reporterCollections.nCopies(n, callable);
    try {
      Collection<Future<Object>>  }		futures = pool.invokeAll(collection);
  
  } catch (InterruptedException e) }{
    };	
  // 
Forward to handler
  Collection<Callable<Object>> c = new ArrayList<Callable<Object>> Thread.currentThread().interrupt();
 // Reset interrupted for(int i=0;i < n; i++) {status
    }
    //  c..add(Executors.callable(run));
    }
	
  public static Collection<Future<Object>> futures = null;
    try void main(String[] args) {
    BrowserManager manager futures= =new pool.invokeAllBrowserManager(c5);
    } catch (InterruptedException e) {     
      // Forward to handler	
    }
    // ... 
  }

  public static void main(String[] args) {
    BrowserManager manager = new BrowserManager(5);
    manager.perUser();
  }	
}
{code}

Contrary to what is expected, this program does not print the total count, that is, the number of times {{doSomething()}} is invoked. This is because it is susceptible to a thread starvation deadlock because the size of the thread pool (10) does not allow either thread from {{perTab()}} to invoke the {{doSomething()}} method. The output of the program varies for different values of {{numberOfTimes}} and the thread pool size. Note that different threads are allowed to invoke {{doSomething()}} in different order; we are concerned only with the maximum value of {{count}} to determine how many times the method executed.

h2. Compliant Solution

This compliant solution selects tasks for scheduling and avoids the thread starvation deadlock. Every level (worker) must have a double ended queue where all sub-tasks are queued. Each level removes the most recently generated sub-task from the queue so that it can process it. When there are no more threads left to process, the current level runs the least-recently created sub-task of another level by picking and removing it from that level's queue. 

This compliant solution sets the {{CallerRuns}} policy on a {{ThreadPoolExecutor}}, and uses a synchronous queue \[[Gafter 06|AA. Java References#Gafter 06]\].

{code:bgColor=#ccccff}
public class BrowserManager {
  static ThreadPoolExecutor pool =
    new ThreadPoolExecutor(0, 10, 60L, TimeUnit.SECONDS,
                          new SynchronousQueue<Runnable>());
  int numberOfTimes;
  private static volatile int count = 0;

  static {
    pool.setRejectedExecutionHandler(
    new ThreadPoolExecutor.CallerRunsPolicy());
  }

  // ... 
}	
{code}

According to the Java API, class {{java.util.concurrent.ThreadPoolExecutor.CallerRunsPolicy}} documentation:

{quote}
A handler for rejected tasks that runs the rejected task directly in the calling thread of the {{execute}} method, unless the executor has been shut down, in which case the task is discarded. 
{quote}

This compliant solution is subject to the vagaries of the thread scheduler which may not optimally schedule the tasks, however, it avoids the thread starvation deadlock.

h2. Risk Assessment

Executing interdependent tasks in a thread pool can lead to denial of service.

|| Rule || Severity || Likelihood || Remediation Cost || Priority || Level ||
| CON29- J | low | probable | medium | {color:green}{*}P4{*}{color} | {color:green}{*}L3{*}{color} |

h3. Automated Detection

TODO

h3. Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the [CERT website|https://www.kb.cert.org/vulnotes/bymetric?searchview&query=FIELD+KEYWORDS+contains+FIO38-J].

h2. References

\[[API 06|AA. Java References#API 06]\] 
\[[Gafter 06|AA. Java References#Gafter 06]\] [A Thread Pool Puzzler|http://gafter.blogspot.com/2006/11/thread-pool-puzzler.html]

----
[!The CERT Sun Microsystems Secure Coding Standard for Java^button_arrow_left.png!|FIO36-J. Do not create multiple buffered wrappers on an InputStream]&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[!The CERT Sun Microsystems Secure Coding Standard for Java^button_arrow_up.png!|09. Input Output (FIO)]&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[!The CERT Sun Microsystems Secure Coding Standard for Java^button_arrow_right.png!|09. Input Output (FIO)]manager.perUser();
  }
}

public final class BrowserManager {
  private final static ThreadPoolExecutor pool =
      new ThreadPoolExecutor(0, 10, 60L, TimeUnit.SECONDS,
                             new SynchronousQueue<Runnable>());
  private final int numberOfTimes;
  private static AtomicInteger count = new AtomicInteger(); // count = 0

  static {
    pool.setRejectedExecutionHandler(
    new ThreadPoolExecutor.CallerRunsPolicy());
  }

  // ...
}