Anne O. Nymous (not.delete@this.real.address) on February 2, 2023 11:57 pm wrote:
> —- (-.delete@this.redheron.com) on February 2, 2023 3:35 pm wrote:
> > Mark Heath (none.delete@this.none.none) on February 1, 2023 3:45 pm wrote:
> > > Freddie (freddie.delete@this.witherden.org) on February 1, 2023 8:54 am wrote:
> > > > Also, I'll note that with static scheduling it is not necessarily true
> > > > that P cores will operate at the performance level of E cores.
> > > > So long as OMP_PROC_BIND (and related variables) are not set,
> > > > the OS scheduler is free to move threads around. Hence,
> > > > when the P cores finish (and go to sleep) the scheduler can shift a task over from the E cores to them.
> > > >
> > > > Contrived example. We have 32 work items on a system with
> > > > 4P and 4E cores with no SMT. Lets assume a P core can do
> > > > 2 items a second and an E core can do 1 item a second. With
> > > > a static schedule each core gets 32 / 8 = 4 items.
> > > >
> > > > After t = 2 seconds the P cores are done, and the E cores have
> > > > 2 items remaining. Scheduler sees this, and shifts the E core
> > > > threads to the P cores. At t = 3, we're done (maybe sooner as the
> > > > E cores being idle may allow the P cores to boost higher).
> > > > In contrast, a system with 8E cores would take until t = 4 to finish.
> > >
> > > Thank you for your interesting educational example. You’re right about static scheduling not being as
> > > bad as I said because the OS scheduler can move threads from E cores to P cores. Suppose there were 48 work
> > > items in your example. With a static schedule, each core would be assigned 48/8 = 6 work items. The P cores
> > > would finish at t=3 and each E core would have 3 work items
> > > left to do at that point. The OS scheduler would
> > > move the E core threads to the P cores and each P core would complete those 3 items at t=4.5.
> > >
> > > Now suppose the programmer used OpenMP’s auto schedule policy and the OpenMP runtime was smart enough
> > > to notice, after the E cores complete one work item, that the E cores are taking twice as long as the
> > > P cores per work item. Since the auto schedule policy allows the OpenMP runtime to figure out the best
> > > schedule, the runtime could, in theory, assign 8 work items to each P core and 4 work items to each E
> > > core. In this case, the loop would complete at t=4 instead of t=4.5. Does it seem practical for an OpenMP
> > > runtime to do this when the auto schedule policy is used? Is there any way for a programmer to manually
> > > give the P cores twice as many iterations as the E cores so the loop completes at t=4?
> > >
> > > Regarding Heikki Kultala’s comment: Apple’s hardware does not have SMT so
> > > splitting a physical P core into two virtual P cores is not possible as a way
> > > of making the performance of the all threads in the system more uniform.
> >
> > Apple, so far anyway, don’t see E cores as throughput cores but as “helper” ores, like the dedicated
> > cores on some other many core designs (like I think Fugaku does this). Apple is not substantially scaling
> > up the E cores count as the SOC grows; overall it’s a very different design thinking than Intel.
> >
> > So Apple’s answer to the OpenMP question would probably be to put the code
> > on P only and let the E cores handle whatever OS/IO work arises as they would
> > naturally, do t bother trying to squeeze out an extra few percent using them.
>
> Interesting observation.
> How much smaller are the E cores compared to the P cores and how much less power do they draw? It might be
> a question of say having 4 concurrent somewhat slower CPUs versus 1.5 faster one; for lowish priority background
> jobs higher concurrency might be more useful than higher ST speed, but I am merely speculating.
>

The numbers vary from design to design, but order of magnitude:
- E cores are a quarter the size of P cores
- E cores provide 1/4 to 1/3 the performance of P cores
- E cores use (at peak power level, which may be misleading in terms of actual usage...) about 1/10th the power (so about 1/3 the energy, taking 3x as long, for a specific task)

Essentially
- Apple optimizes their E-cores for energy-delay product (ie balanced between fast and low energy)
- ARM optimizes the E-cores for low area
- Intel optimizes their E-cores for high performance/area

Each is optimizing for a very different goal, so it's not surprising that the results are best used in very different ways.

For Apple (at least for now...) it doesn't make sense to run things like OMP, or other highly-threaded code, on E-cores unless you are chasing that last few percent of performance AND know something about your task lengths and how they balance. Certainly it might be dumb to do this when the set of tasks is variable but fairly short, each of unknown length, and with faster tasks having to wait for slower tasks.
Of course there are some trivial (frequently dick-measuring) workloads like cinebench or handbrake where this is not a risk because the tasks are so long lived before dependencies that even the simplest OS scheduler will balance everything out OK. But this is not representative of less trivially parallelizable code.

For Intel, on the other hand, E-cores represent some part of their performance future, with many kinda high-end designs of the sorts targeting gamers dumping a substantial fraction of their area and performance into E-cores, and their eco-system has a more difficult task trying to handle this...

BTW the truly energy-optimized Apple cores are the Chinook cores which are basically very fancy ARM M cores speaking AArch64. These are used as controllers all over the chip (for the GPU, NPU, ISP, etc) but are, of course, irrelevant to developers outside Apple.