By: aaron spink (aaronspink.delete@this.notearthlink.net), August 13, 2019 7:23 pm
Room: Moderated Discussions
Travis Downs (travis.downs.delete@this.gmail.com) on August 12, 2019 11:26 am wrote:
> aaron spink (aaronspink.delete@this.notearthlink.net) on August 12, 2019 9:45 am wrote:
> > The
> > only time someone should ever be doing a local build is if they are trapped on a desert island.
>
> Are there "cloud" compile caching & distribution services out there?
>
> I know this stuff works great on a 1 or 10 gbit LAN, but I am not sure that most people
> can download object files faster than they can compile them. So I think local compile
> has a place: and of course you can use caching there too (ccache changed my life).
>
> As a random example I just clean compiled part of the project I am working
> on. With 4 threads it took just under 5 seconds (and 10 seconds CPU time,
> so I only got a 2x speedup from concurrency, probably due to linking).
>
> It produced 4.1 MB of .o files and a 2 MB final executable.
>
> Assuming I had to download all 6.1 MB, it would take me about 16 seconds on my 3 mbps connection
> (assuming the remote compilation/caching was infinitely fast). If I donwloaded only the finally
> executable, it would be just over 5 seconds, so competitive with local compile.
>
> Compression would further improve the remote performance.
>
> So it's definitely possible even on my slow connection, but it depends a lot on the output file size and local
> compile times. In practice I'm using ccache (disabled for this test), so local compile times are much faster.
>
LOL. Yeah, most projects I work on have linking times that are on the order of 150x longer than your compile (and that's using the good linker, with other versions of the linker, linking alone can take an hour).
> aaron spink (aaronspink.delete@this.notearthlink.net) on August 12, 2019 9:45 am wrote:
> > The
> > only time someone should ever be doing a local build is if they are trapped on a desert island.
>
> Are there "cloud" compile caching & distribution services out there?
>
> I know this stuff works great on a 1 or 10 gbit LAN, but I am not sure that most people
> can download object files faster than they can compile them. So I think local compile
> has a place: and of course you can use caching there too (ccache changed my life).
>
> As a random example I just clean compiled part of the project I am working
> on. With 4 threads it took just under 5 seconds (and 10 seconds CPU time,
> so I only got a 2x speedup from concurrency, probably due to linking).
>
> It produced 4.1 MB of .o files and a 2 MB final executable.
>
> Assuming I had to download all 6.1 MB, it would take me about 16 seconds on my 3 mbps connection
> (assuming the remote compilation/caching was infinitely fast). If I donwloaded only the finally
> executable, it would be just over 5 seconds, so competitive with local compile.
>
> Compression would further improve the remote performance.
>
> So it's definitely possible even on my slow connection, but it depends a lot on the output file size and local
> compile times. In practice I'm using ccache (disabled for this test), so local compile times are much faster.
>
LOL. Yeah, most projects I work on have linking times that are on the order of 150x longer than your compile (and that's using the good linker, with other versions of the linker, linking alone can take an hour).
