Big gotchas: poor specs

By: Jukka Larja (roskakori2006.delete@this.gmail.com), August 13, 2019 7:37 am
Room: Moderated Discussions
aaron spink (aaronspink.delete@this.notearthlink.net) on August 12, 2019 9:45 am wrote:
> Adrian (a.delete@this.acm.org) on August 11, 2019 5:41 am wrote:
> > I suppose that there are more software developers than that and
> > program compilation scales very well with the number of cores.
> >
> Program compilation scales even better if you don't do it on your client. Its shocking that we're still
> doing that considering the technology to get off it is now almost 20+ years old. Versioning and caching
> build systems are just so much better. Vesioned build systems with caching is what should be used. The
> only time someone should ever be doing a local build is if they are trapped on a desert island.

Doing development work for PS4 has a benefit of SN-DBS (Sony Distributed Build System, not sure if there's a commercial product it's based on). However, setting it up for anything not provided (PS4, PS3 PSP, Windows) is somewhat cumbersome (I think. The documentation is bit lacking, so we haven't gotten around to actually trying, which sort of tells that it is obviously non-trivial).

For Windows it works relatively well, except it doesn't support precompiled hearders with sensible compile parameters. It scales nicely to at least to about dozen computers, but having hundred low-performance ones doesn't really help.

For PS4 compilations the system works great as long as there's enough network bandwidth available. Unfortunately 1 Gbps is only barely enough. Again, with fewer, more performant nodes it works OK, but it's a total disaster with some old quad-cores (better to throw them away).

In my previous job we used Increadibuild for compiling Unreal Engine. Considering the full build (which I had to do more often than I liked) took something like 1.5 hours on i7-4770K, it worked well. Not as well as SN-DBS, but as the network environment, code and build computers were different, it's hard to say which is really better. Increadibuild is crazy expensive for smallish projects though. Not sure how their licencing works for large ones.

Based on these real world experiences, my opinion is that anyone saying some silver bullet build system should be used by every project (that has significant compile times) is just full of crap. If you have right expertise and right kind of project, please go ahead. Also, if you think you can make our job easier, feel free to contact us (http://www.frozenbyte.com). We won't pay well, but we'll gladly hear you out.

-JLarja
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
EPYC SAP-SD 2-tier benchmark resultsMichael S2019/08/08 05:18 AM
  EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownMichael S2019/08/08 08:29 AM
    Wrong headline. Should be "SPECpower_ssj2008 - EPYC2 is in class of its own" (NT)Michael S2019/08/08 08:51 AM
    EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownAlberto2019/08/08 03:42 PM
      EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownblue2019/08/08 07:35 PM
      EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownMaynard Handley2019/08/08 08:58 PM
        EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownanon2019/08/09 02:00 AM
          EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownAlberto2019/08/09 03:09 AM
            EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownblue2019/08/09 08:47 AM
        EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownAlberto2019/08/09 02:12 AM
          EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownMaynard Handley2019/08/09 09:25 AM
            EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownDoug S2019/08/09 09:33 AM
              EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownanon2019/08/09 10:24 AM
            EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownAlberto2019/08/10 04:04 AM
          EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownanon.12019/08/09 05:30 PM
        EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownme2019/08/09 09:58 PM
      EPYC SAP-SD 2-tier benchmark results - EPYC2 is in class of its ownanon2019/08/09 12:47 AM
  EPYC SAP-SD 2-tier benchmark resultsLinus Torvalds2019/08/09 03:29 PM
    EPYC SAP-SD 2-tier benchmark resultsanon2019/08/10 07:20 AM
      EPYC SAP-SD 2-tier benchmark resultsme2019/08/10 07:36 AM
        EPYC SAP-SD 2-tier benchmark resultsanon2019/08/10 08:20 AM
          EPYC SAP-SD 2-tier benchmark resultsanon.12019/08/10 09:40 AM
            EPYC SAP-SD 2-tier benchmark resultsanon2019/08/10 10:38 AM
          EPYC SAP-SD 2-tier benchmark resultsblue2019/08/10 09:47 AM
        EPYC SAP-SD 2-tier benchmark resultsAdrian2019/08/10 10:43 AM
          EPYC SAP-SD 2-tier benchmark resultsme2019/08/10 11:06 AM
            EPYC SAP-SD 2-tier benchmark resultsMichael S2019/08/10 11:28 AM
    Big gotchas: poor specsjohn2019/08/10 10:54 AM
      Big gotchas: poor specsme2019/08/10 11:04 AM
      Big gotchas: poor specsanon2019/08/10 11:25 AM
        Big gotchas: poor specsLinus Torvalds2019/08/10 12:13 PM
          Big gotchas: poor specsanon2019/08/10 01:58 PM
          Big gotchas: poor caches2019/08/12 01:51 AM
            Big gotchas: poor caches2019/08/12 01:59 AM
            Big gotchas: poor cachesTravis Downs2019/08/12 09:15 AM
              parallelism vs. efficiencyhobold2019/08/13 04:43 AM
            Big gotchas: poor cachesAndy N2019/08/12 06:45 PM
              Big gotchas: poor cachesanon2019/08/13 12:43 AM
          Big gotchas: poor specsJouni Osmala2019/08/12 10:18 PM
            Big gotchas: poor specsAdrian2019/08/12 11:58 PM
              Kernel compile 25-30s, noise 25-27dBJouni Osmala2019/08/13 04:00 AM
                Quiet and fast workstationsGian-Carlo Pascutto2019/08/13 06:00 AM
                  Quiet and fast workstationsAdrian2019/08/13 07:01 AM
                  Quiet and fast workstationsDavid Hess2019/08/13 01:51 PM
                    Quiet and fast workstationsGian-Carlo Pascutto2019/08/14 12:43 AM
                      Quiet and fast workstationsMichael S2019/08/14 02:09 AM
                        Quiet and fast workstationsGian-Carlo Pascutto2019/08/14 02:59 AM
                          Quiet and fast workstationsDavid Hess2019/08/14 06:05 PM
                        Quiet and fast workstationsDavid Hess2019/08/14 05:56 PM
                      Quiet and fast workstationsDavid Hess2019/08/14 06:30 PM
                  Passively cooled threadripper/xeon WJouni Osmala2019/08/15 03:12 AM
                    Passively cooled threadripper/xeon WAdrian2019/08/15 03:47 AM
                      Passively cooled threadripper/xeon WJouni Osmala2019/08/15 05:58 AM
                        Passively cooled threadripper/xeon WAdrian2019/08/16 12:12 AM
                          Passively cooled threadripper/xeon WAdrian2019/08/16 12:21 AM
                          Passively cooled threadripper/xeon WJouni osmala2019/08/16 09:46 AM
                            Passively cooled threadripper/xeon WGroo2019/08/16 12:07 PM
                Kernel compile 25-30s, noise 25-27dBAdrian2019/08/13 06:39 AM
                Kernel compile 25-30s, noise 25-27dBLinus Torvalds2019/08/13 12:12 PM
                  which graphics card?anonymous22019/08/13 12:52 PM
                    which graphics card?Linus Torvalds2019/08/13 01:12 PM
                  Kernel compile 25-30s, noise 25-27dBDoug S2019/08/13 03:22 PM
                    Kernel compile 25-30s, noise 25-27dBDavid Hess2019/08/14 06:38 PM
                  Kernel compile 25-30s, noise 25-27dBJouni Osmala2019/08/14 12:09 AM
                    AIOsGroo2019/08/14 07:19 AM
                      AIOsGian-Carlo Pascutto2019/08/14 08:52 AM
                        AIOsGroo2019/08/14 09:49 AM
                      AIOsBrett2019/08/14 02:02 PM
                    Kernel compile 25-30s, noise 25-27dBLinus Torvalds2019/08/14 11:10 AM
                  Kernel compile 25-30s, noise 25-27dBGian-Carlo Pascutto2019/08/14 01:33 AM
                    Kernel compile 25-30s, noise 25-27dBAdrian2019/08/14 05:06 AM
                      Kernel compile 25-30s, noise 25-27dBGian-Carlo Pascutto2019/08/14 09:01 AM
                      AMD Bios, etc..Anon2019/08/14 11:39 PM
                        AMD Bios, etc..hobold2019/08/15 07:10 AM
                          AMD Bios, etc..Anon2019/08/15 04:20 PM
                        AMD Bios, etc..Jan Olšan2019/08/15 08:49 AM
                          AMD Bios, etc..Adrian2019/08/15 09:16 AM
                          AMD Bios, etc..Gro2019/08/15 10:57 AM
                        AMD Bios, etc..David Hess2019/08/16 05:39 AM
      Big gotchas: poor specsAdrian2019/08/10 01:28 PM
      Big gotchas: poor specsDummond D. Slow2019/08/10 01:57 PM
        Big gotchas: poor specsanon22019/08/11 12:33 AM
          Big gotchas: poor specsblue2019/08/11 12:57 AM
          Big gotchas: poor specsAdrian2019/08/11 05:41 AM
            Big gotchas: poor specsaaron spink2019/08/12 09:45 AM
              Big gotchas: poor specsFoo_2019/08/12 11:02 AM
              Big gotchas: poor specsTravis Downs2019/08/12 11:26 AM
                Big gotchas: poor specsGabriele Svelto2019/08/13 09:48 AM
                Big gotchas: poor specsaaron spink2019/08/13 07:23 PM
                  Big gotchas: poor specsDoug S2019/08/13 11:15 PM
                    Big gotchas: poor specsGian-Carlo Pascutto2019/08/14 01:35 AM
                    Big gotchas: poor specsaaron spink2019/08/15 05:16 PM
                      Big gotchas: poor specsJouni Osmala2019/08/15 09:03 PM
                        Big gotchas: poor specsaaron spink2019/08/19 04:05 AM
                          Big gotchas: poor specsMichael S2019/08/19 04:37 AM
                            Big gotchas: poor specsRicardo B2019/08/19 05:40 PM
                              Big gotchas: poor specsAaron Spink2019/08/25 02:29 AM
              Big gotchas: poor specsAdrian2019/08/12 11:31 AM
              Big gotchas: poor specsJukka Larja2019/08/13 07:37 AM
              Big gotchas: poor specsLinus Torvalds2019/08/13 11:31 AM
          Big gotchas: poor specsDummond D. Slow2019/08/11 06:23 AM
      FX-9590 > i7-9700KGian-Carlo Pascutto2019/08/12 10:37 AM
    RDRAND issueanonymou52019/08/10 02:52 PM
      RDRAND issueLinus Torvalds2019/08/10 04:15 PM
        RDRAND issueGionatan Danti2019/08/11 01:35 AM
          RDRAND issueLinus Torvalds2019/08/11 10:43 AM
            RDRAND issueLinus Torvalds2019/08/11 11:05 AM
              RDRAND issue@never_released2019/08/11 11:10 AM
                RDRAND issueLinus Torvalds2019/08/11 01:13 PM
                  Err....Groo2019/08/11 06:47 PM
                    Err.... This is a zero content fud posteye roll2019/08/11 08:48 PM
                    Err....Brendan2019/08/11 08:50 PM
                    Err....Jukka Larja2019/08/12 07:38 AM
                  RDRAND issueJose2019/08/12 02:33 AM
              RDRAND issueGionatan Danti2019/08/11 02:12 PM
            RDRAND issueaaron spink2019/08/12 09:18 AM
      RDRAND issuename2019/08/11 09:35 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?