By: Travis Downs (travis.downs.delete@this.gmail.com), January 24, 2020 1:03 pm

Room: Moderated Discussions

anonymou5 (no.delete@this.spam.com) on January 22, 2020 11:56 pm wrote:

> > > > > When playing with this on SKX, the 3 frequency levels are fairly obvious, but I think I

> > > > > can also see evidence for voltage not mapping 1:1, i.e. the result is >3 total, sort of.

> > > >

> > > > You mean you see more than three distinct voltage levels, right?

> > >

> > > not quite

> > >

> > > yes, there are many voltage levels

> > >

> > > what I seem to see is that within the 3 frequency levels, more seems to be going on

> > >

> > > it's as if they have sub-levels of some sort

> >

> > You see more than 3 frequency levels, or you are seeing more than one voltage per frequency level?

> >

> > I also see more than one voltage per frequency level, although the "zoomed in"

> > plots may not show it. I am not sure what triggers other voltage transitions.

>

> hah, I think I found a good clue!

>

> look at the CORE_POWER performance counter event (0x28)

>

> the SDM lists it as CORE_POWER.LVLn_TURBO_LICENSE with n={0,1,2}

>

> n=0 for non-AVX, SSE, AVX128, and low-current AVX256

> n=1 for high-current AVX256 and low-current AVX512

> n=3 for high-current AVX512

>

> basically, the 3 documented turbo frequency curves, as expected

>

> now look at the 3 corresponding umask values (0x07, 0x18, 0x20)

>

> 3 bits for n=0

> 2 bits for n=1

> 1 bit for n=2

>

> 6 bits total

>

> if I try those umask bits *individually*, then I see two behaviors

>

> on SKL, I get counter increments for bits 0, 1, etc. (though not the top ones of course)

>

> on SKX, I get counter increments for bits 1 to 5, but not for bit 0 (I don't know why)

Very interesting! So the theory would be these are finer grained levels within each larger license category.

Can you figure out how the levels are encoded? That is, is only 1 bit set at once, or within a level is it encoded in binary, e.g., L0 could actually have 2^8 levels? I guess you could check by seeing if the count for umask=0x1 plus umask=0x2 sums to the same as the count for umask=0x3 (the same indicates it is 1-hot, otherwise seems like maybe binary encoding).

Intuitively it makes sense that L0 has more levels, because there are probably more opportunities for finer-grained power savings for very low power workloads, perhaps things like a pause loop, or things that don't even use the 128-bit regs, while the other levels have a pretty solid floor on the current because they support wide SIMD.

>

> so yeah, on SKX both LVL0 and LVL1 do indeed have 2 sub levels each, for a total of 5

>

> mapping those 5 levels to only 3 frequency curves is a simplification on Intel's part

Well yeah, imagine trying to explain 5 levels... 3 is bad enough. The sub-levels are probably mostly invisible since they only involve voltage changes.

> > > > > When playing with this on SKX, the 3 frequency levels are fairly obvious, but I think I

> > > > > can also see evidence for voltage not mapping 1:1, i.e. the result is >3 total, sort of.

> > > >

> > > > You mean you see more than three distinct voltage levels, right?

> > >

> > > not quite

> > >

> > > yes, there are many voltage levels

> > >

> > > what I seem to see is that within the 3 frequency levels, more seems to be going on

> > >

> > > it's as if they have sub-levels of some sort

> >

> > You see more than 3 frequency levels, or you are seeing more than one voltage per frequency level?

> >

> > I also see more than one voltage per frequency level, although the "zoomed in"

> > plots may not show it. I am not sure what triggers other voltage transitions.

>

> hah, I think I found a good clue!

>

> look at the CORE_POWER performance counter event (0x28)

>

> the SDM lists it as CORE_POWER.LVLn_TURBO_LICENSE with n={0,1,2}

>

> n=0 for non-AVX, SSE, AVX128, and low-current AVX256

> n=1 for high-current AVX256 and low-current AVX512

> n=3 for high-current AVX512

>

> basically, the 3 documented turbo frequency curves, as expected

>

> now look at the 3 corresponding umask values (0x07, 0x18, 0x20)

>

> 3 bits for n=0

> 2 bits for n=1

> 1 bit for n=2

>

> 6 bits total

>

> if I try those umask bits *individually*, then I see two behaviors

>

> on SKL, I get counter increments for bits 0, 1, etc. (though not the top ones of course)

>

> on SKX, I get counter increments for bits 1 to 5, but not for bit 0 (I don't know why)

Very interesting! So the theory would be these are finer grained levels within each larger license category.

Can you figure out how the levels are encoded? That is, is only 1 bit set at once, or within a level is it encoded in binary, e.g., L0 could actually have 2^8 levels? I guess you could check by seeing if the count for umask=0x1 plus umask=0x2 sums to the same as the count for umask=0x3 (the same indicates it is 1-hot, otherwise seems like maybe binary encoding).

Intuitively it makes sense that L0 has more levels, because there are probably more opportunities for finer-grained power savings for very low power workloads, perhaps things like a pause loop, or things that don't even use the 128-bit regs, while the other levels have a pretty solid floor on the current because they support wide SIMD.

>

> so yeah, on SKX both LVL0 and LVL1 do indeed have 2 sub levels each, for a total of 5

>

> mapping those 5 levels to only 3 frequency curves is a simplification on Intel's part

Well yeah, imagine trying to explain 5 levels... 3 is bad enough. The sub-levels are probably mostly invisible since they only involve voltage changes.

Topic | Posted By | Date |
---|---|---|

AVX-512 downclocking post | Travis Downs | 2020/01/16 09:20 PM |

AVX-512 downclocking post | anon³ | 2020/01/17 01:25 AM |

AVX-512 downclocking post | Andrei | 2020/01/17 02:47 AM |

AVX-512 downclocking post | Montaray Jack | 2020/01/17 03:58 PM |

AVX-512 downclocking post | Andrei | 2020/01/17 11:40 PM |

AVX-512 downclocking post | Montaray Jack | 2020/01/19 02:10 AM |

AVX-512 downclocking post | Jan Olšan | 2020/01/19 01:01 PM |

AVX-512 downclocking post | Jan Olšan | 2020/01/19 01:11 PM |

AVX-512 downclocking post | Travis Downs | 2020/01/17 02:59 PM |

AVX-512 downclocking post | David Kanter | 2020/01/18 10:27 AM |

magnetic inductors | jokerman | 2020/01/18 08:03 PM |

AVX-512 downclocking post | Travis Downs | 2020/01/24 11:36 AM |

AVX-512 downclocking post | Ray | 2020/01/17 02:22 AM |

AVX-512 downclocking post | Travis Downs | 2020/01/17 01:10 PM |

AVX-512 downclocking post | Etienne | 2020/01/17 03:16 AM |

Thanks, typos fixed and credited (NT) | Travis Downs | 2020/01/17 01:15 PM |

Title suggestions welcome (NT) | Travis Downs | 2020/01/17 08:54 AM |

AVX-512 downclocking post | anonymou5 | 2020/01/17 10:53 AM |

AVX-512 downclocking post | Travis Downs | 2020/01/17 11:14 AM |

AVX-512 downclocking post | Yoav | 2020/01/17 11:50 AM |

AVX-512 downclocking post | Travis Downs | 2020/01/17 01:14 PM |

AVX-512 downclocking post | anonymou5 | 2020/01/17 04:26 PM |

AVX-512 downclocking post | Travis Downs | 2020/01/22 08:19 PM |

AVX-512 downclocking post | anonymou5 | 2020/01/23 12:56 AM |

AVX-512 downclocking post | Foyle | 2020/01/23 05:51 AM |

AVX-512 downclocking post | anonymou5 | 2020/01/23 06:57 AM |

AVX-512 downclocking post | Travis Downs | 2020/01/24 12:49 PM |

finer-grained licenses | Travis Downs | 2020/01/24 01:03 PM |

finer-grained licenses | anonymou5 | 2020/01/24 04:28 PM |

finer-grained licenses | Travis Downs | 2020/01/25 09:46 AM |

post published (new line) | Travis Downs | 2020/01/17 11:55 AM |

should say: (new LINK) (NT) | Travis Downs | 2020/01/17 11:55 AM |

should say: (new LINK) | Tim McCaffrey | 2020/01/17 01:44 PM |

Thanks, fixed and credited (NT) | Travis Downs | 2020/01/17 02:54 PM |

should say: (new LINK) | anon | 2020/01/17 09:12 PM |

should say: (new LINK) | Travis Downs | 2020/01/22 03:28 PM |

Thanks! | anon | 2020/01/22 08:06 PM |

Thanks! | Travis Downs | 2020/01/22 08:16 PM |

Thanks! | anon | 2020/01/22 10:20 PM |

Thanks! | Travis Downs | 2020/01/23 01:51 AM |

Thanks! | Linus Torvalds | 2020/01/23 05:33 PM |

Thanks! | Travis Downs | 2020/01/24 12:44 PM |

Throttling dispatch | Gabriele Svelto | 2020/01/22 01:40 PM |

Itanium also used it | David Kanter | 2020/01/22 02:04 PM |