Mark Heath (none.delete@this.none.none) on February 1, 2023 5:42 am wrote:
> Freddie (freddie.delete@this.witherden.org) on February 1, 2023 4:05 am wrote:
> > Guided scheduling can work nicely here.
>
> Guided scheduling has the overhead of a work queue, like dynamic scheduling. OpenMP has an auto schedule policy.

Not necessarily. If a work queue makes sense depends on the implementation. Should an implementation really want to be bare bones it should be possible to get away with a shared counter which each thread atomically increments when it wants to assign itself some work. Moreover, guided scheduling - when implemented properly - can massively reduce the frequency of these updates while still enabling dynamic load balancing.

Also, I'll note that with static scheduling it is not necessarily true that P cores will operate at the performance level of E cores. So long as OMP_PROC_BIND (and related variables) are not set, the OS scheduler is free to move threads around. Hence, when the P cores finish (and go to sleep) the scheduler can shift a task over from the E cores to them.

Contrived example. We have 32 work items on a system with 4P and 4E cores with no SMT. Lets assume a P core can do 2 items a second and an E core can do 1 item a second. With a static schedule each core gets 32 / 8 = 4 items.

After t = 2 seconds the P cores are done, and the E cores have 2 items remaining. Scheduler sees this, and shifts the E core threads to the P cores. At t = 3, we're done (maybe sooner as the E cores being idle may allow the P cores to boost higher). In contrast, a system with 8E cores would take until t = 4 to finish.

Regards, Freddie.