*guaranteed* IO scheduling

By: --- (---.delete@this.redheron.com), January 14, 2022 7:39 pm
Room: Moderated Discussions
Do Linux or Windows have mechanisms to provide more-or-less guaranteed IO-scheduling? And how well does it work?

I discovered something interesting about this in the latest macOSX 12:
Apple has had API's to set the QoS for IO scheduling for years now, the equivalent of ionice. But they weren't hooked up, or weren't aggressive enough, or whatever.
For example if you were recording audio to a *hard drive* that was also doing other things (most obvious case being a machine with a single hard drive that's also running the OS, and you have recording in the background while you do something in the foreground) you would suffer dropouts. Not catastrophic, not unusable, but very definitely present, enough so that you'd internalize "if I'm recording audio, don't do anything else" as standard procedure.
(I use audio as an example because it's not especially demanding of bandwidth, meaning you can buffer a good deal in DRAM; even so you still have this as an issue.)

As of macOSX 12 this seems to have changed, so that IO priorities are now aggressively enforced.
On the plus side, this means the high priority items (generally recording) seem bulletproof, and it's freaking amazing compared to what it used to be.
On the negative side, as anyone with experience in queueing or scheduling or realtime could have told you, damn, if you are not in the highest priority band you hurt! Ensuring that you cannot inadvertently *ever* steal away a problematic number of IOp's from the hard drive and the higher priority app means you get very few IOps.

I don't know how this is hooked up to SSDs (it probably is, but seeing it would require careful instrumentation, whereas seeing it kick in for HDs simply requires looking at a graph of bandwidth!).
Two obvious "conflict" cases are a recording app (which uses the appropriate APIs) vs normal usage, or normal usage vs Time Machine backups (which are low priority IO).

So I mention this all because I'm curious how this plays out in other OS's. I don't want to start a war here; I have no reason to believe that Apple's implementation is especially good or especially bad. I'm just curious if anyone else takes realtime seriously enough to bother getting this right (after all even Apple only got it right in the current OS, right at the point where it barely matters any more given SSDs...)
It's a tricky problem because doing it correctly (for the apps that care) has such extreme effects on the rest of the system, and this seems the kind of thing I'd have heard people grumbling about in the Windows or Linux context.
 Next Post in Thread >
TopicPosted ByDate
*guaranteed* IO scheduling---2022/01/14 07:39 PM
  *guaranteed* IO schedulinganon22022/01/14 09:19 PM
  *guaranteed* IO schedulingAnon2022/01/15 10:30 AM
    *guaranteed* IO scheduling---2022/01/15 11:13 AM
      *guaranteed* IO schedulingAnon2022/01/15 11:42 AM
      *guaranteed* IO schedulingAndrey2022/01/15 01:00 PM
      OS vs H/WMark Roulo2022/01/15 02:18 PM
        OS vs H/WJörn Engel2022/01/15 03:03 PM
          OS vs H/W - human error ...Björn Ragnar Björnsson2022/01/15 06:54 PM
            OS vs H/W - human error ...Jörn Engel2022/01/16 11:48 AM
        OS vs H/W---2022/01/15 05:31 PM
          OS vs H/WGionatan Danti2022/01/17 04:30 AM
            OS vs H/WBrendan2022/01/17 06:36 AM
              OS vs H/WGionatan Danti2022/01/17 02:43 PM
                OS vs H/WBrendan2022/01/17 05:07 PM
                  OS vs H/WGionatan Danti2022/01/18 07:26 AM
                    OS vs H/WBrendan2022/01/18 04:52 PM
                      OS vs H/WGionatan Danti2022/01/18 05:20 PM
                        OS vs H/WAnon2022/01/18 07:08 PM
                          OS vs H/WGionatan Danti2022/01/18 11:42 PM
                            OS vs H/WAnon2022/01/19 03:51 AM
                              OS vs H/WGionatan Danti2022/01/19 07:17 AM
                                OS vs H/WAnon2022/01/19 08:02 AM
                        OS vs H/WBrendan2022/01/19 04:44 AM
                          OS vs H/WGionatan Danti2022/01/19 07:12 AM
                            OS vs H/Wrwessel2022/01/19 10:25 AM
                              OS vs H/WGionatan Danti2022/01/19 11:17 AM
                            OS vs H/WBrendan2022/01/19 12:56 PM
                              OS vs H/WGionatan Danti2022/01/19 01:51 PM
                                OS vs H/WBrendan2022/01/19 06:29 PM
                                  OS vs H/WGionatan Danti2022/01/20 01:05 AM
                                    OS vs H/WBrendan2022/01/20 09:36 AM
                                      OS vs H/WGionatan Danti2022/01/20 10:21 AM
                                        OS vs H/WBrendan2022/01/20 12:52 PM
                                          OS vs H/WGionatan Danti2022/01/20 02:23 PM
                                            OS vs H/WBrendan2022/01/20 03:36 PM
                                              OS vs H/WGionatan Danti2022/01/21 12:16 AM
                                                OS vs H/WBrendan2022/01/21 05:22 AM
                                                  OS vs H/Wrwessel2022/01/21 06:56 AM
                                                    OS vs H/WBrendan2022/01/21 04:21 PM
                                                      OS vs H/Wanon22022/01/21 08:13 PM
                                                        OS vs H/WBrendan2022/01/21 09:36 PM
                                                          OS vs H/WBrendan2022/01/21 09:55 PM
                                                          OS vs H/Wanon22022/01/21 09:59 PM
                                                            OS vs H/WBrendan2022/01/21 10:18 PM
                                                              OS vs H/Wanon22022/01/21 11:14 PM
                                                                OS vs H/WBrendan2022/01/22 12:15 AM
                                                                  OS vs H/Wanon22022/01/22 01:54 AM
                                                                OS vs H/W---2022/01/22 11:55 AM
                                                                  OS vs H/WAnon2022/01/22 12:19 PM
                                                                  OS vs H/Wanon22022/01/22 06:50 PM
                                                              OS vs H/WGionatan Danti2022/01/22 01:24 AM
                                                  OS vs H/WGionatan Danti2022/01/21 04:42 PM
                                                    OS vs H/WAnon2022/01/21 05:52 PM
                                                      OS vs H/WUngo2022/01/21 06:18 PM
                                                        OS vs H/WAnon2022/01/21 06:32 PM
                                                          OS vs H/WUngo2022/01/21 09:02 PM
                                                            OS vs H/WAnon2022/01/21 10:24 PM
                                                              OS vs H/WGionatan Danti2022/01/22 03:22 AM
                                                                OS vs H/WAnon2022/01/22 07:27 AM
                                                                  OS vs H/WGionatan Danti2022/01/22 07:59 AM
                                                                    OS vs H/WAnon2022/01/22 08:24 AM
                                                                      OS vs H/Wrwessel2022/01/22 11:27 AM
                                                                        OS vs H/WAnon2022/01/22 12:05 PM
                                                        OS vs H/WMatt Sayler2022/01/21 06:58 PM
                                                    OS vs H/WBrendan2022/01/21 06:17 PM
                                                      OS vs H/WGionatan Danti2022/01/22 01:23 AM
                                                        OS vs H/WBrendan2022/01/22 10:43 AM
                                                          OS vs H/Wrwessel2022/01/22 11:38 AM
                                                            OS vs H/WAnon2022/01/22 12:10 PM
                                                            OS vs H/WBrendan2022/01/22 12:19 PM
                                                              OS vs H/WGionatan Danti2022/01/22 06:09 PM
                                                                OS vs H/WBrendan2022/01/22 09:03 PM
                                                                  OS vs H/WSimon Farnsworth2022/01/23 03:12 AM
                                                                  OS vs H/WGionatan Danti2022/01/23 07:41 AM
                                                                    OS vs H/WBrendan2022/01/23 06:45 PM
                                                                      OS vs H/WGionatan Danti2022/01/24 02:53 AM
                                                                        OS vs H/WAnon2022/01/24 04:08 AM
                                                                          OS vs H/WAdrian2022/01/24 04:48 AM
                                                                          OS vs H/WGionatan Danti2022/01/24 06:08 AM
                                                                            OS vs H/WAnon2022/01/24 06:20 AM
                                                                        OS vs H/WBrendan2022/01/24 12:10 PM
                                        OS vs H/Wanon22022/01/20 06:21 PM
                                          OS vs H/WGionatan Danti2022/01/21 12:32 AM
                              OS vs H/WUngo2022/01/19 08:19 PM
                                OS vs H/WUngo2022/01/19 08:47 PM
                                OS vs H/WGionatan Danti2022/01/20 01:13 AM
                                OS vs H/WEtienne Lorrain2022/01/20 04:43 AM
                                  OS vs H/WGionatan Danti2022/01/20 10:28 AM
                                    OS vs H/WDavid Hess2022/01/24 04:04 PM
                                  OS vs H/W verses shingled drivesBrett2022/01/20 12:54 PM
                                    OS vs H/W verses shingled drivesGionatan Danti2022/01/21 12:19 AM
                                      OS vs H/W verses shingled drivesBrett2022/01/21 11:31 AM
                                        Shingled drives: Just Say NoLinus Torvalds2022/01/22 12:02 AM
                                          Shingled drives: Just Say Nodmcq2022/01/22 05:55 PM
                                          Shingled drives: Just Say No ChoiceBrett2022/01/22 08:43 PM
                                            Shingled drives: Just Say No ChoiceGoose2022/01/22 09:29 PM
                                              Shingled drives: Just Say No ChoiceBrett2022/01/23 12:39 AM
                                              Shingled drives: Just Say No ChoiceAdrian2022/01/23 02:59 AM
                                                Shingled drives: Just Say No ChoiceJukka Larja2022/01/24 06:55 AM
                                              Shingled drives: Just Say No ChoiceDoug S2022/01/23 08:40 AM
                                              Shingled drives: Just Say No Choice---2022/01/23 01:59 PM
                                                Shingled drives: Just Say No ChoiceJukka Larja2022/01/24 07:04 AM
                                            Shingled drives: Just Say No2022/01/24 08:29 AM
                                              Shingled drives: Just Say No2022/01/24 09:01 AM
                                                Shingled drives: Just Say Noanonymou52022/01/24 08:38 PM
                                                  Shingled drives: Just Say NoAnon2022/01/24 09:39 PM
                                              Shingled drives: Just Say NoBrett2022/01/24 09:25 PM
                                            Shingled drives: Just Say No Choiceblaine2022/01/25 01:40 PM
                                          JMF602 SSDsYuhong Bao2022/01/23 09:29 PM
                                            JMF602 SSDsGionatan Danti2022/01/24 04:06 AM
                                          Shingled drives: Just Say No - Workaround?Ray2022/01/25 01:36 PM
                                            Shingled drives: Just Say No - Workaround?Brendan2022/01/25 03:17 PM
                                              Shingled drives: Just Say No - Workaround?Mark2022/01/25 05:23 PM
                                              Shingled drives: Just Say No - Workaround?Ray2022/01/25 10:43 PM
                                                Shingled drives: Just Say No - Workaround?Doug S2022/01/25 11:48 PM
                                                  Shingled drives: Just Say No - Workaround?Ray2022/01/26 12:10 AM
                                              Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/01/26 03:27 AM
                                                Shingled drives: Just Say No - Workaround?Anon2022/01/26 10:22 AM
                                                  Shingled drives: Just Say No - Workaround?Doug S2022/01/26 11:55 AM
                                                    Shingled drives: Just Say No - Workaround?Michael S2022/01/26 02:45 PM
                                                      Is ~100 TB total write caching enough? How expensive is SLC cache?Mark Roulo2022/01/26 03:51 PM
                                                        Is ~100 TB total write caching enough? How expensive is SLC cache?Anon2022/01/26 04:19 PM
                                                          Is ~100 TB total write caching enough? How expensive is SLC cache?dmcq2022/01/26 04:43 PM
                                                      Shingled drives: Just Say No - Workaround?Doug S2022/01/26 09:34 PM
                                                        Guarantee (NT)Anon2022/01/26 09:55 PM
                                                        Shingled drives: Just Say No - Workaround?Blaine2022/01/27 02:41 PM
                                                          Shingled drives: Just Say No - Workaround?Doug S2022/01/27 03:31 PM
                                                        Shingled drives: Just Say No - Workaround?Blaine2022/01/27 02:49 PM
                                                Shingled drives: Just Say No - Workaround?Brendan2022/01/26 06:11 PM
                                                  Shingled drives: Just Say No - Workaround?Anon2022/01/26 06:21 PM
                                                    Shingled drives: Just Say No - Workaround?Brendan2022/01/26 09:12 PM
                                                  Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/01/27 11:20 AM
                                                    Shingled drives: Just Say No - Workaround?Brendan2022/01/27 06:36 PM
                                                      Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/28 04:55 AM
                                                      Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/01/28 05:46 AM
                                                        Shingled drives: Just Say No - Workaround?Mark2022/01/28 10:04 AM
                                                          Seagate has a table/list (incomplete)anonymous22022/01/28 12:33 PM
                                                        Shingled drives: Just Say No - Workaround?Brendan2022/01/28 07:47 PM
                                                          Shingled drives: Just Say No - Workaround?---2022/01/29 11:42 AM
                                                            Shingled drives: Just Say No - Workaround?anon22022/01/29 08:06 PM
                                                            Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/29 10:44 PM
                                                              Shingled drives: Just Say No - Workaround?Brendan2022/01/30 01:58 AM
                                                                Shingled drives: Just Say No - Workaround?rwessel2022/01/30 04:06 AM
                                                                  Shingled drives: Just Say No - Workaround?Brendan2022/01/30 11:02 PM
                                                                Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/30 12:22 PM
                                                                  Shingled drives: Just Say No - Workaround?---2022/01/30 01:59 PM
                                                                    Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/31 08:21 AM
                                                                  Shingled drives: Just Say No - Workaround?Brendan2022/01/30 11:10 PM
                                                                    Shingled drives: Just Say No - Workaround?Gionatan Danti2022/01/31 01:07 AM
                                                                      Shingled drives: Just Say No - Workaround?Anon2022/01/31 06:31 AM
                                                                        Shingled drives: Just Say No - Workaround?Etienne Lorrain2022/01/31 08:53 AM
                                                                Shingled drives: Just Say No - Workaround?Gionatan Danti2022/01/31 01:14 AM
                                                                  Shingled drives: Just Say No - Workaround?Brett2022/01/31 04:07 PM
                                                                  Shingled drives: Just Say No - Workaround?Brendan2022/01/31 11:13 PM
                                                                    Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/01 03:28 AM
                                                                      Shingled drives: Just Say No - Workaround?Brendan2022/02/01 04:41 AM
                                                                        Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/01 05:35 AM
                                                                          Shingled drives: Just Say No - Workaround?Brendan2022/02/01 05:40 PM
                                                                            Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/02 12:53 AM
                                                                              Shingled drives: Just Say No - Workaround?Brendan2022/02/02 02:46 AM
                                                                                Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/02 04:11 AM
                                                                                  Shingled drives: Just Say No - Workaround?---2022/02/02 11:18 AM
                                                                                    Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/02 01:31 PM
                                                                                      Shingled drives: Just Say No - Workaround?anon22022/02/02 07:14 PM
                                                                                        Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/03 05:24 AM
                                                                                          Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/02/03 06:46 AM
                                                                                            Shingled drives: Just Say No - Workaround?Mark2022/02/03 09:40 AM
                                                                                              Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/02/03 11:39 AM
                                                                                              Shingled drives: Just Say No - Workaround?Maxwell2022/02/03 12:00 PM
                                                                                                Shingled drives: Just Say No - Workaround?Anon2022/02/04 05:33 AM
                                                                                  Shingled drives: Just Say No - Workaround?Brendan2022/02/03 05:23 AM
                                                                                    Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/03 01:37 PM
                                                                                      Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/03 04:28 PM
                                                                                        Shingled drives: Just Say No - Workaround?---2022/02/03 05:12 PM
                                                                                          Shingled drives: Just Say No - Workaround?anon22022/02/03 09:16 PM
                                                                                            Shingled drives: Just Say No - Workaround?---2022/02/04 02:04 PM
                                                                                              Shingled drives: Just Say No - Workaround?anon22022/02/04 05:22 PM
                                                                                              Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/05 05:51 AM
                                                                                                Shingled drives: Just Say No - Workaround?---2022/02/05 11:20 AM
                                                                                          Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/04 03:37 AM
                                                                        Shingled drives: Just Say No - Workaround?Mark2022/02/01 01:22 PM
                                                                          Shingled drives: Just Say No - Workaround?---2022/02/01 02:02 PM
                                                                            Shingled drives: Just Say No - Workaround?Jukka Larja2022/02/02 07:34 AM
                                                                        Shingled drives: Just Say No - Workaround?Ungo2022/02/02 07:31 AM
                                                                          Shingled drives: Just Say No - Workaround?---2022/02/02 11:22 AM
                                                                            Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/02 01:34 PM
                                                                          Shingled drives: Just Say No - Workaround?Brendan2022/02/03 08:48 AM
                                                              Shingled drives: Just Say No - Workaround?Anon2022/01/30 08:44 PM
                                                                Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/31 07:56 AM
                                                                  Shingled drives: Just Say No - Workaround?Anon2022/01/31 08:36 AM
                                                                    Shingled drives: Just Say No - Workaround?Jukka Larja2022/01/31 12:38 PM
                                                                      Shingled drives: Just Say No - Workaround?---2022/01/31 01:54 PM
                                                                        Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/02/01 02:50 AM
                                                                        Shingled drives: Just Say No - Workaround?Anne O. Nymous2022/02/01 04:31 AM
                                                                        Shingled drives: Just Say No - Workaround?Jukka Larja2022/02/01 07:36 AM
                                                              Shingled drives: Just Say No - Workaround?---2022/01/31 01:38 PM
                                                                Shingled drives: Just Say No - Workaround?anon22022/02/01 12:43 AM
                                                                Shingled drives: Just Say No - Workaround?Jukka Larja2022/02/01 07:49 AM
                                                                  Shingled drives: Just Say No - Workaround?Brett2022/02/01 04:45 PM
                                                                    Shingled drives: Just Say No - Workaround?Anon2022/02/01 10:21 PM
                                                                      Shingled drives: Just Say No - Workaround?Gionatan Danti2022/02/02 01:20 AM
                                                                        Shingled drives: Just Say No - Workaround?Anon2022/02/02 05:39 AM
                                                                      Not true. I've had SMR drives fall off the bus after too many random writes. (NT)Ray2022/02/03 04:15 PM
                                                                        Not true. I've had SMR drives fall off the bus after too many random writes.Anon2022/02/04 05:40 AM
                                                          Shingled drives: Just Say No - Workaround?Simon Farnsworth2022/01/30 08:59 AM
                                                Shingled drives: Just Say No - Workaround?anon22022/01/26 07:54 PM
                                                  Shingled drives: Just Say No - Workaround?Jörn Engel2022/01/27 06:13 PM
                                                    Shingled drives: Just Say No - Workaround?anon22022/01/28 08:26 PM
                                                      Do you mean "log structured" rather than "lot structured" (NT)Rob Thorpe2022/02/02 08:40 AM
                                                        Yes (NT)anon22022/02/02 06:01 PM
                                OS vs H/Wj2022/01/20 07:18 AM
                                  OS vs H/WGionatan Danti2022/01/20 10:32 AM
                                OS vs H/WBrendan2022/01/20 08:03 AM
                                  OS vs H/WGionatan Danti2022/01/20 10:46 AM
                                    OS vs H/WBrendan2022/01/20 01:16 PM
                                      OS vs H/WGionatan Danti2022/01/20 02:25 PM
                                OS vs H/W---2022/01/20 11:35 AM
                                  OS vs H/WUngo2022/01/20 09:02 PM
                                    OS vs H/WAnon2022/01/21 12:15 PM
                                      OS vs H/WGionatan Danti2022/01/21 02:04 PM
                                        OS vs H/WAnon2022/01/21 03:10 PM
                                          OS vs H/WGionatan Danti2022/01/21 05:04 PM
                                            OS vs H/WSimon Farnsworth2022/01/23 03:00 AM
                                              OS vs H/WGionatan Danti2022/01/23 07:09 AM
                                  OS vs H/Wanon22022/01/20 10:01 PM
      *guaranteed* IO schedulinganon22022/01/16 02:18 AM
      *guaranteed* IO schedulingDavid Hess2022/01/18 10:07 AM
        *guaranteed* IO schedulingAnon2022/01/18 01:04 PM
          *guaranteed* IO schedulingDavid Hess2022/01/18 03:49 PM
            *guaranteed* IO schedulingJukka Larja2022/01/19 09:01 AM
  *guaranteed* IO schedulingNoSpammer2022/01/16 12:37 PM
    *guaranteed* IO schedulinganon22022/01/16 09:09 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?