rwessel (robertwessel.delete@this.yahoo.com) on December 8, 2014 12:52 pm wrote:
> Brett (ggtgp.delete@this.yahoo.com) on December 8, 2014 10:00 am wrote:
> > Eric Bron (eric.bron.delete@this.zvisuel.privatefortest.com) on December 8, 2014 4:35 am wrote:
> > > > have huge working sets which would take too much memory, energy, and loading time to access.
> > >
> > > even without resorting to new technologies, SSD are vey fast already, something which need
> > > much attention for out of core processing is the file formats, legacy formats and many new
> > > formats (!) are based on serialization of the whole file which goes against parallelization
> >
> > A SSD has one set of pins you plug the device into, so ultimately the interface is serial.
> > The big mistake is not preallocating your disk blocks, a simple byte stream will have to update the file
> > table for each block added. This doubles the number of blocks written to and almost all those file table
> > blocks will need to be erased, which leads to a slow garbage collection wear leveling pass.
>
> Unless you specify immediate writes, better file systems do not need to commit the block indexes/pointers
> to disk for a file being sequentially written until the data is actually written to disk, or even
> until the file is actually closed. And considerable amounts of data can be buffered.

That is good to hear, my experience was on a mobile device which did not have a better file system. Even worse the problem was a dozen different log files spamming one line at a time with a flush and close for each line. (There are a couple of public domain solutions to this on soureforge.net.)

If you are running a journaled file system things get even more interesting.
http://en.wikipedia.org/wiki/Journaling_file_system

Still I think my advice is good, it just won't matter as much on a good file system.
But still thanks.