Hardware Thread-Level Parallelism and Pipelining

Not powered off. Stalled – waiting on a cache miss, a branch it guessed wrong, a value that hasn’t arrived yet. During those gaps the execution units sit there doing nothing. That waste is the problem SMT was built to attack.

Intel calls its version Hyper-Threading. The trick: make one physical core look like two to the OS. When thread A stalls, thread B slides into the idle execution units and keeps them busy.

What actually gets duplicated is small – each thread needs its own registers, its own program counter, its own view of the world. What gets shared is the expensive stuff – the ALUs, the FPU, the caches. That split is the whole design. Duplicate only what breaks correctness if shared, share everything else to raise utilization.

The catch people skip: this is not free performance.

• Two threads fighting over one cache can evict each other’s data and both get slower
• Shared branch predictors and buffers create contention
• More active silicon means more heat and power

So the honest number is not “2x.” It’s often 20-30%, sometimes less, occasionally negative. SMT fills gaps. It does not add compute.

AMD’s Bulldozer took the opposite bet – duplicate the integer cores, share the float unit. Different philosophy, same question: what do you copy, what do you share.

Full breakdown of shared vs duplicated resources, and where SMT backfires, is in the article.