anon2 (anon.delete@this.anon.com) on November 26, 2020 5:59 pm wrote:
> Brett (ggtgp.delete@this.yahoo.com) on November 26, 2020 4:11 pm wrote:
> > Andrey (andrey.semashev.delete@this.gmail.com) on November 26, 2020 12:33 pm wrote:
> > > Brett (ggtgp.delete@this.yahoo.com) on November 26, 2020 11:29 am wrote:
> > > >
> > > > The beauty of this approach is that all this database needs is to import/export git, and
> > > > have a filesystem driver, and you don’t care what the rest of the world is doing.
> > > >
> > > > You don’t even need git, git can write to the filesystem, there are just some opportunities
> > > > to optimize git while you are at it. You can inline compression if it is free, etc.
> > > >
> > > > Everything just works, and you can now optimize the speed of builds and git by tuning the database
> > > > layout behavior. A folder tree with hundreds of files spread all over the disk is now a small
> > > > set of large linear packed files that stream quickly and use less disk and less ram.
> > >
> > > You're forgetting about fragmentation, which will inevitably happen as you modify the sources.
> > > And if we're talking about SSDs then fragmentation is not an issue in either case.
> >
> > Git updates are largely a write only operation, compiles are a read only operation,
> > editor changes only effect a handful of files and are irrelevant.
> > The database only holds code, compiler output goes to the filesystem.
> > Detecting disk idle and rewriting the database for speed is a given.
> > Detecting a compile is trivial, saving the access order so
> > as to rewrite the file order packing to match is trivial.
> > Grep does not care about file order generally, gets a database
> > directory file list in compile order, which is near optimal.
> > Actual file order will go in and out of subdirectories almost randomly
> > but a custom grep executable can follow database file order.
> > The database knows which files are redundantly read from the last compile and keeps those sectors mapped.
> >
> > Now are you starting to see the benefits?
> >
> > A simple smart database turns a thousand different reads into almost one linear fire hose of packed
> > data. The Linux compile advantage of 2X over MacOS and 4X+ over Windows turns into a deficit.
> >
> > What if the database gets corrupt?
> > sudo rm -rf/*
> > git
> >
> > Yes I included a typo.
> >
> > The Linux compile advantage is a stupid pet trick with little real world
> > use outside of compiles, and will one day be rendered obsolete.
>
> Do you honestly believe you are in a position to be making such claims?
>
> I absolutely believe in the visionary or polymath who can see things in a different way
> or at a higher level and revolutionizes a field or a technology. I've seen it happen.
>
> These people are not idle talkers or musers or outsiders who know or care little about existing
> technologies and practices, they work hard, they are dedicated, they know the current state
> of the art as well as anybody, and that's how they understand they can do better.

I spent 20 years in the video game industry optimizing code everyone else said was already optimized, and the proof was that the code was 20 years old and used by hundreds of AAA titles.
Much of the code WAS optimal for the previous generation of hardware, or more often the previous-previous generation of hardware.

Few programmers understand how CPU’s work and most importantly how that effects code performance. Others think I am pulling off miracles, it is just simple engineering, just as you say, hard work and knowledge.

> You on the other hand seemed to be unaware of a basic filesystem metadata block cache or buffer cache
> that has been a thing for half a century. This is just like your "post-RISC" incoherent babbling. You

RISC was optimal for a short decade in the last century, I want to know what comes next.

I have learned a lot, my most recent proposal over on comp.arch to use a pair of tightly coupled six wide cores to run loops 12 wide is somewhat viable. You do need to be smart with cloning data across the pair of L0 and L1 data caches or you will be data starved.

Yes this is mental masterbation as I am in software and this is a hardware issue.

> know a bit of terminology, almost none of the concepts, none of the problems, practicalities, or actual
> technology -- and then you proclaim things are terrible as they are and yet you somehow know exactly
> how existing practice is wrong and exactly what is required to fix everything. You're a charlatan.