Optimizing program performance relies heavily on how tasks are divided and scheduled is critical in parallel computing.
Grain size represents the size or duration of a basic program segment that can be executed independently in parallel. The selection of grain size has a significant impact on how tasks are distributed among parallel processing units and how efficiently they can be executed.
Fine Grained:
It involves dividing a program into small tasks, where each task represents a small and computationally lightweight portion of work. Fine-grained tasks are typically short-lived and require frequent communication between parallel threads. This level of granularity is beneficial when tasks have minimal dependencies, and the overhead of task creation and synchronization is relatively low.
Example: Image processing algorithms, such as edge detection or image filtering, often exhibit fine-grained parallelism. Each pixel or small neighborhood of pixels can be processed independently, allowing for efficient parallel execution across multiple processing units.
Medium Grained:
Medium-grained granularity strikes a balance between fine-grained and coarse-grained parallelism. Program segments at this level are larger than fine-grained tasks but still exhibit a degree of independence. Medium-grained tasks typically encompass a significant portion of work and require a moderate amount of inter-task communication.
Example: Matrix operations, like matrix multiplication or matrix addition, can be divided into medium-grained tasks. Each task may involve performing computations on a block or submatrix of the original matrices, allowing for parallel processing while minimizing communication overhead.
Coarse Grained:
Coarse-grained parallelism involves larger program segments, where each task represents a substantial portion of work that can be executed independently. Coarse-grained tasks tend to be long-running and involve limited communication among parallel threads.
Example: In scientific simulations, such as weather forecasting or fluid dynamics simulations, coarse-grained parallelism can be employed. Each simulation timestep or significant computation phase can be treated as an independent task, allowing parallel execution across multiple processors
Grain size plays a crucial role in load balancing and communication overhead. If the grain size is too fine-grained, the overhead of task creation and management may outweigh parallelism’s benefits. If the grain size is too coarse-grained, some processing units may idle while others overload, leading to inefficient resource utilization. Grain size impacts communication and synchronization among tasks, with fine grained tasks requiring more frequent communication and introducing higher overhead
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
An Article by: Yashwanth Naidu Tikkisetty
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
