By: Jukka Larja (roskakori2006.delete@this.gmail.com), August 13, 2019 7:37 am
Room: Moderated Discussions
aaron spink (aaronspink.delete@this.notearthlink.net) on August 12, 2019 9:45 am wrote:
> Adrian (a.delete@this.acm.org) on August 11, 2019 5:41 am wrote:
> > I suppose that there are more software developers than that and
> > program compilation scales very well with the number of cores.
> >
> Program compilation scales even better if you don't do it on your client. Its shocking that we're still
> doing that considering the technology to get off it is now almost 20+ years old. Versioning and caching
> build systems are just so much better. Vesioned build systems with caching is what should be used. The
> only time someone should ever be doing a local build is if they are trapped on a desert island.
Doing development work for PS4 has a benefit of SN-DBS (Sony Distributed Build System, not sure if there's a commercial product it's based on). However, setting it up for anything not provided (PS4, PS3 PSP, Windows) is somewhat cumbersome (I think. The documentation is bit lacking, so we haven't gotten around to actually trying, which sort of tells that it is obviously non-trivial).
For Windows it works relatively well, except it doesn't support precompiled hearders with sensible compile parameters. It scales nicely to at least to about dozen computers, but having hundred low-performance ones doesn't really help.
For PS4 compilations the system works great as long as there's enough network bandwidth available. Unfortunately 1 Gbps is only barely enough. Again, with fewer, more performant nodes it works OK, but it's a total disaster with some old quad-cores (better to throw them away).
In my previous job we used Increadibuild for compiling Unreal Engine. Considering the full build (which I had to do more often than I liked) took something like 1.5 hours on i7-4770K, it worked well. Not as well as SN-DBS, but as the network environment, code and build computers were different, it's hard to say which is really better. Increadibuild is crazy expensive for smallish projects though. Not sure how their licencing works for large ones.
Based on these real world experiences, my opinion is that anyone saying some silver bullet build system should be used by every project (that has significant compile times) is just full of crap. If you have right expertise and right kind of project, please go ahead. Also, if you think you can make our job easier, feel free to contact us (http://www.frozenbyte.com). We won't pay well, but we'll gladly hear you out.
-JLarja
> Adrian (a.delete@this.acm.org) on August 11, 2019 5:41 am wrote:
> > I suppose that there are more software developers than that and
> > program compilation scales very well with the number of cores.
> >
> Program compilation scales even better if you don't do it on your client. Its shocking that we're still
> doing that considering the technology to get off it is now almost 20+ years old. Versioning and caching
> build systems are just so much better. Vesioned build systems with caching is what should be used. The
> only time someone should ever be doing a local build is if they are trapped on a desert island.
Doing development work for PS4 has a benefit of SN-DBS (Sony Distributed Build System, not sure if there's a commercial product it's based on). However, setting it up for anything not provided (PS4, PS3 PSP, Windows) is somewhat cumbersome (I think. The documentation is bit lacking, so we haven't gotten around to actually trying, which sort of tells that it is obviously non-trivial).
For Windows it works relatively well, except it doesn't support precompiled hearders with sensible compile parameters. It scales nicely to at least to about dozen computers, but having hundred low-performance ones doesn't really help.
For PS4 compilations the system works great as long as there's enough network bandwidth available. Unfortunately 1 Gbps is only barely enough. Again, with fewer, more performant nodes it works OK, but it's a total disaster with some old quad-cores (better to throw them away).
In my previous job we used Increadibuild for compiling Unreal Engine. Considering the full build (which I had to do more often than I liked) took something like 1.5 hours on i7-4770K, it worked well. Not as well as SN-DBS, but as the network environment, code and build computers were different, it's hard to say which is really better. Increadibuild is crazy expensive for smallish projects though. Not sure how their licencing works for large ones.
Based on these real world experiences, my opinion is that anyone saying some silver bullet build system should be used by every project (that has significant compile times) is just full of crap. If you have right expertise and right kind of project, please go ahead. Also, if you think you can make our job easier, feel free to contact us (http://www.frozenbyte.com). We won't pay well, but we'll gladly hear you out.
-JLarja