> > > > When playing with this on SKX, the 3 frequency levels are fairly obvious, but I think I
> > > > can also see evidence for voltage not mapping 1:1, i.e. the result is >3 total, sort of.
> > >
> > > You mean you see more than three distinct voltage levels, right?
> >
> > not quite
> >
> > yes, there are many voltage levels
> >
> > what I seem to see is that within the 3 frequency levels, more seems to be going on
> >
> > it's as if they have sub-levels of some sort
>
> You see more than 3 frequency levels, or you are seeing more than one voltage per frequency level?
>
> I also see more than one voltage per frequency level, although the "zoomed in"
> plots may not show it. I am not sure what triggers other voltage transitions.

hah, I think I found a good clue!

look at the CORE_POWER performance counter event (0x28)

the SDM lists it as CORE_POWER.LVLn_TURBO_LICENSE with n={0,1,2}

n=0 for non-AVX, SSE, AVX128, and low-current AVX256
n=1 for high-current AVX256 and low-current AVX512
n=3 for high-current AVX512

basically, the 3 documented turbo frequency curves, as expected

now look at the 3 corresponding umask values (0x07, 0x18, 0x20)

3 bits for n=0
2 bits for n=1
1 bit for n=2

6 bits total

if I try those umask bits *individually*, then I see two behaviors

on SKL, I get counter increments for bits 0, 1, etc. (though not the top ones of course)

on SKX, I get counter increments for bits 1 to 5, but not for bit 0 (I don't know why)

so yeah, on SKX both LVL0 and LVL1 do indeed have 2 sub levels each, for a total of 5

mapping those 5 levels to only 3 frequency curves is a simplification on Intel's part

next: try different SKX microcodes, to see if SKX erratum 101 shows up in all of this...