--- (---.delete@this.redheron.com) on February 3, 2023 11:35 am wrote:
> Anne O. Nymous (not.delete@this.real.address) on February 2, 2023 11:57 pm wrote:
> > —- (-.delete@this.redheron.com) on February 2, 2023 3:35 pm wrote:
> > > Mark Heath (none.delete@this.none.none) on February 1, 2023 3:45 pm wrote:
> > > > Freddie (freddie.delete@this.witherden.org) on February 1, 2023 8:54 am wrote:
> > > > > Also, I'll note that with static scheduling it is not necessarily true
> > > > > that P cores will operate at the performance level of E cores.
> > > > > So long as OMP_PROC_BIND (and related variables) are not set,
> > > > > the OS scheduler is free to move threads around. Hence,
> > > > > when the P cores finish (and go to sleep) the scheduler can shift a task over from the E cores to them.
> > > > >
> > > > > Contrived example. We have 32 work items on a system with
> > > > > 4P and 4E cores with no SMT. Lets assume a P core can do
> > > > > 2 items a second and an E core can do 1 item a second. With
> > > > > a static schedule each core gets 32 / 8 = 4 items.
> > > > >
> > > > > After t = 2 seconds the P cores are done, and the E cores have
> > > > > 2 items remaining. Scheduler sees this, and shifts the E core
> > > > > threads to the P cores. At t = 3, we're done (maybe sooner as the
> > > > > E cores being idle may allow the P cores to boost higher).
> > > > > In contrast, a system with 8E cores would take until t = 4 to finish.
> > > >
> > > > Thank you for your interesting educational example. You’re right about static scheduling not being as
> > > > bad as I said because the OS scheduler can move threads from E cores to P cores. Suppose there were 48 work
> > > > items in your example. With a static schedule, each core would be assigned 48/8 = 6 work items. The P cores
> > > > would finish at t=3 and each E core would have 3 work items
> > > > left to do at that point. The OS scheduler would
> > > > move the E core threads to the P cores and each P core would complete those 3 items at t=4.5.
> > > >
> > > > Now suppose the programmer used OpenMP’s auto schedule policy and the OpenMP runtime was smart enough
> > > > to notice, after the E cores complete one work item, that the E cores are taking twice as long as the
> > > > P cores per work item. Since the auto schedule policy allows the OpenMP runtime to figure out the best
> > > > schedule, the runtime could, in theory, assign 8 work items to each P core and 4 work items to each E
> > > > core. In this case, the loop would complete at t=4 instead of t=4.5. Does it seem practical for an OpenMP
> > > > runtime to do this when the auto schedule policy is used? Is there any way for a programmer to manually
> > > > give the P cores twice as many iterations as the E cores so the loop completes at t=4?
> > > >
> > > > Regarding Heikki Kultala’s comment: Apple’s hardware does not have SMT so
> > > > splitting a physical P core into two virtual P cores is not possible as a way
> > > > of making the performance of the all threads in the system more uniform.
> > >
> > > Apple, so far anyway, don’t see E cores as throughput cores but as “helper” ores, like the dedicated
> > > cores on some other many core designs (like I think Fugaku does this). Apple is not substantially scaling
> > > up the E cores count as the SOC grows; overall it’s a very different design thinking than Intel.
> > >
> > > So Apple’s answer to the OpenMP question would probably be to put the code
> > > on P only and let the E cores handle whatever OS/IO work arises as they would
> > > naturally, do t bother trying to squeeze out an extra few percent using them.
> >
> > Interesting observation.
> > How much smaller are the E cores compared to the P cores and how much less power do they draw? It might be
> > a question of say having 4 concurrent somewhat slower CPUs
> > versus 1.5 faster one; for lowish priority background
> > jobs higher concurrency might be more useful than higher ST speed, but I am merely speculating.
> >
>
> The numbers vary from design to design, but order of magnitude:
> - E cores are a quarter the size of P cores
> - E cores provide 1/4 to 1/3 the performance of P cores
> - E cores use (at peak power level, which may be misleading in terms of actual usage...)
> about 1/10th the power (so about 1/3 the energy, taking 3x as long, for a specific task)
>
> Essentially
> - Apple optimizes their E-cores for energy-delay product (ie balanced between fast and low energy)
> - ARM optimizes the E-cores for low area
> - Intel optimizes their E-cores for high performance/area
>
> Each is optimizing for a very different goal, so it's not surprising
> that the results are best used in very different ways.
>
> For Apple (at least for now...) it doesn't make sense to run things like OMP, or other highly-threaded code,
> on E-cores unless you are chasing that last few percent of performance AND know something about your task
> lengths and how they balance. Certainly it might be dumb to do this when the set of tasks is variable but
> fairly short, each of unknown length, and with faster tasks having to wait for slower tasks.
> Of course there are some trivial (frequently dick-measuring) workloads like cinebench or handbrake where this
> is not a risk because the tasks are so long lived before dependencies that even the simplest OS scheduler
> will balance everything out OK. But this is not representative of less trivially parallelizable code.
>
> For Intel, on the other hand, E-cores represent some part of their performance future, with many kinda
> high-end designs of the sorts targeting gamers dumping a substantial fraction of their area and performance
> into E-cores, and their eco-system has a more difficult task trying to handle this...
>
> BTW the truly energy-optimized Apple cores are the Chinook cores which are basically very
> fancy ARM M cores speaking AArch64. These are used as controllers all over the chip (for
> the GPU, NPU, ISP, etc) but are, of course, irrelevant to developers outside Apple.
>

Thanks! Glad I asked, more food for thought.