datmt.com

Concurrent Java Applications — Concurrency, Parallelism, Thread Lifecycle, and Pool Sizing

Mon Sep 14 2026

Part 7 of 14 in Functional Java Unleashed

Concurrency vs Parallelism — The Core Distinction

Every Java developer encounters the words concurrency and parallelism used interchangeably. They are not the same thing, and confusing them leads to the wrong conclusions about when threading helps.

Concurrency is a structural concept: it describes organizing computation into independent tasks that can be interleaved. Parallelism is an execution concept: it means those tasks actually run on different CPUs at the same time.

You can have concurrency without parallelism — one thread executing many small tasks sequentially. But you cannot have parallelism without concurrency, because there must be independent work items to distribute across cores.

Concurrency Demo

This example runs four pieces of work (two CPU-bound computations and two simulated I/O waits) in two ways: all on a single thread in sequence, then across four threads via CompletableFuture:

--- Sequential execution (single thread) ---
  Total: 406.1 ms (all four tasks serialized on one thread)

--- Parallel execution (multiple threads) ---
  Total: 203.8 ms (four tasks run simultaneously)

On this machine the sequential version took 406.1 ms while the parallel version completed in 203.8 ms — roughly half the time. The work items are the same; only the execution model changed.

The parallel version finishes faster because:

  • The two CPU-bound tasks run simultaneously on different cores instead of back-to-back
  • The two I/O waits also overlap with each other and with the CPU work
  • Total wall-clock time shrinks to roughly max(cpu_work, io_wait) rather than cpu_work + io_wait

But this only works because CompletableFuture.supplyAsync() dispatches tasks to a ForkJoinPool running on multiple cores. On a single-core machine, parallel would be no faster than sequential — the structure is concurrent regardless; parallelism requires hardware.

Thread Lifecycle States

Java threads have six well-defined states tracked by the JVM in java.lang.Thread.State. Knowing them matters because deadlock, starvation, and thread leaks all manifest as unexpected stuck states.

The Six States in Practice

Each state maps to a specific condition:

  • NEW — Thread object created but start() not yet called. The thread exists but has no JVM execution resources allocated.
  • RUNNABLE — Thread is actively executing on the CPU (or ready and scheduled). This state covers both running code and briefly waiting in the OS run queue.
  • TIMED_WAITING — Thread is waiting with a deadline: Thread.sleep(ms), Object.wait(timeout), or Thread.join(timeout).
  • WAITING — Thread is waiting indefinitely: Object.wait() without timeout, Thread.join(), or LockSupport.park().
  • BLOCKED — Thread is trying to enter a synchronized block but another thread holds the monitor lock. Unlike WAITING, this thread wants to run but cannot because of contention.
  • TERMINATED — The thread’s run() method completed normally or via an uncaught exception.

The state diagram is simple: NEW → RUNNABLE → WAITING / TIMED_WAITING / BLOCKED → TERMINATED, with the middle states flowing back to RUNNABLE when their wait condition resolves (notify, interrupt, timeout expiry).

A subtle but important point: a thread in BLOCKED is not the same as one in WAITING. BLOCKED means the thread can run right now if only it could acquire the lock — it’s stuck by contention, not by design. This distinction matters because the fix for BLOCKED is reducing lock contention (narrower locks, read-write locks), while the fix for WAITING might be to add a timeout or use ScheduledExecutorService instead.

Thread Pool Sizing Strategies

Once you understand what threads are doing and their lifecycle, the practical question is: how many should a pool have?

CPU-Bound Workloads

For compute-intensive tasks where threads spend nearly 100% of their time on CPU (prime calculations, image processing, compression), fewer thread slots is usually better:

On this 10-core machine the measurements showed:

  • corePoolSize = N cores (10): 12.2 ms for 20 prime-computation tasks, 1645 tasks/sec
  • corePoolSize = 2 × N (20): 5.2 ms for the same 20 tasks, 3860 tasks/sec

Counterintuitively, doubling the thread count here was actually faster — because with an unbounded queue and only 20 small tasks, every task could start immediately on its own core without queuing delays. The classical advice (corePoolSize = N + 1) exists for production workloads where many more tasks flow through over time and context-switch overhead dominates.

The rule of thumb remains: CPU-bound pools should be close to the number of cores. Adding significantly more threads than cores adds scheduling overhead without meaningful throughput gains once saturation is reached.

I/O-Bound Workloads

For database queries, HTTP calls, file reads — where threads spend most of their time waiting for external systems — you need far more threads:

  • corePoolSize = N cores (10): 401.3 ms — under-provisioned
  • corePoolSize = 2 × N + 1 (21): 201.6 ms — properly sized

The I/O-bound pool with only 10 threads was twice as slow as the one with 21 threads. The reason is simple: at any moment, roughly half the threads are sleeping while waiting for their I/O to complete. With a pool of 10, only ~5 are actually doing work at once. Doubling the pool keeps CPU utilization high.

A common formula: corePoolSize ≈ N_cores / (1 − blocking_fraction). If your calls spend 80% of their time waiting:

  • corePoolSize ≈ 10 / (1 − 0.8) = 50 threads

Queue Choice Matters as Much as Size

The bounded pool test revealed another critical detail: only 6 out of 20 tasks were accepted (core=2, max=4, queue capacity=2 gives a total buffer of 6). The remaining 14 hit RejectedExecutionException immediately.

This is the hidden danger of thread pools:

  • Unbounded queues (LinkedBlockingQueue<Integer.MAX_VALUE>): never reject tasks but can cause out-of-memory when producers overwhelm consumers
  • Bounded queues: prevent OOM but require a rejection strategy (callers, discarding, or custom RejectedExecutionHandler)
  • SynchronousQueue: direct handoff only — the effective pool capacity equals maxPoolSize with zero queuing

The right choice depends on whether your system fails safely when overloaded. A bounded queue with an appropriate rejection handler is almost always better than trusting producers to handle exceptions, because silent dropping beats catastrophic OOM.

Takeaway

Concurrency is about structure; parallelism is about hardware. A thread’s state tells you what it’s waiting for — and the wrong pool size makes those waits last much longer than necessary. Size CPU-bound pools near core count, scale I/O-bound pools by expected blocking fraction, and always bound your queues.