Linus Torvalds (torvalds@linux-foundation.org) on 1/7/09 wrote:
---------------------------
>- almost any block mapping will be simplified by bigger
>extents.
>
>Result: especially after running benchmarks that just
>do small random writes for a long time, the block
>remapping tables will be maximally fragmented and have
>just single-block extents.
>
>This will likely cause a performance dip because the
>remapping tables don't fit in the RAM caches of the
>controller, so it will end up doing more lookups to the
>flash.

Every random write can cause a remapping table change, which is basically a random write as well. Larger extends reduces your theoretical maximum random write performance (that or they waste a lot of space).

>It's quite possible that the flash remapping layer will
>end up running extra GC cycles at points to avoid this
>worst-case situation, and that will obviously show up
>as a performance dip.
>
>The good news is that while "random write performance" is
>actually meaningful, it's very seldom the case that it's
>dominant (ie it's important because it happens occasionally,
>not because it's a common case!)

Most writes are random writes, at least on my system. The writes that aren't don't have to be quick either.

>So in most use, you'd have a mix of small random writes
>and larger contiguous ones, and the realistic situation is
>that the remapping never gets really bad - at least not as
>bad as the extreme benchmarks make it.

I'm not really convinced of this. It may take a bit of time, but you'll get there eventually.

>The other issue is:
>
>- garbage collection is much easier if you have
>lots of free space.

I don't see how this is. Some free space is needed, sure, but not that much. Garbage collection is just not needed when there's free space left...

>This is to some degree the bigger issue. It's also a
>possible "value add" issue, ie I would actually expect
>flash disk manufacturers to start differentiating their
>drives based on "performance vs capacity".
>
>You can effectively make a higher-performance drive by
>leaving more of the drive for internal use, in order to
>make GC be smoother/faster. For example, when the Intel
>drives are 80GB, that means that they really have 80GiB
>(binary) of flash, but only expose 80GB (decimal) of it
>as disk, so you have about 6GB of "free flash" to do
>GC with.
>
>Now, you could actually sell the exact same drive
>with a capacity of just 75GB, and you'd essentially have
>doubled your "scratch area" to do GC in. End result:
>smoother garbage collection with fewer GC spikes.

Only way that is possible is by allowing more space be wasted, or in other words, do less garbage collection by allowing more fragmentation to happen.

More scratch area doesn't help a bit as far as I can see. Only more space wasted for fragmentation. If with "scratch area" you mean free space to get clean blocks from, then it only postpones the need to do GC.

>The second issue shows up when you do any writes to disk:
>out of the factory, the remapping tables are likely all
>empty, so the disk can actually use all of the drive
>as a scratch area, and thus have a much easier time doing
>GC.

You mean no GC has to happen at all.

>But once you've written to the drive enough, it will only
>have that small (well, not so small) 6GB scratch area. See
>how that goes? And this is actually totally independent of
>whether you did small or large writes, although large
>writes will make GC easier in general, so the relative
>performance degradation will probably hit the smaller
>writes more.

Yes and no. If the disk is filled with big writes, not much space is wasted by fragmentation and no GC is needed. Overwriting existing full (erase) blocks means the old ones can be reused without defragmentation.

If you fill the disk with random writes those writes are either painfully slow, or as fast a streaming writes and then a hell of a lot remapping happened. When you start to overwrite existing blocks partially then a lot of fragmentation happens and GC is needed to find new blocks where to write new data to.

>So yes, performance will drop over time, down to a level
>where it stabilizes.

Theoretically it's either limited by your GC/defragmentation speed or your remapping speed (because you need to update the remapping table as well, which, if you're unlucky, can take as much effort as writing the tiny random writes itself).

>However, there is some good news: you can actually tell
>the drives to set aside more memory for scratch space.

That in itself won't help much.

>If your OS supports it, and if your filesystem
>is smart enough, it can actually do a "drop data" command
>when you delete files, and rather than remapping those
>blocks, the flash controller can then add them to the
>scratch area.

This does help really a lot though, but that is telling the drive that blocks can be freely used, which is slightly different than increasing the scratch space (which sounds more like reducing the drive's capacity).

>And I'm sure that people will improve on the Intel drives.
>I'm not saying that they are perfect. But yes, they do
>degrade a bit until they hit a baseline plateau of write
>performance, but if you look at the numbers, even that
>degraded plateau is a couple of orders of magnitude better
>than rotating media.

Just being much better than rotating media is not good enough.

A FTL and a filesystem do more or less the same, and if they work along each other instead of together they waste a lot of efficiency. Having both do a kind of remapping seems silly. The FS doesn't know what kind of reliability it can expect from the FTL, nor does the FTL know what the FS expects. One extreme is that both are reliable, and assume the worst, which slows things down a lot. The other is that one of them assumes too much and data can get lost or corrupted.

Moving the FTL into the FS or at least into the kernel is one way of solving it. The other is to agree on an abstract enough interface between them which lets them both know what to expect and do.

You want the drive to be able to do background GC/defragmentation, as well as bad block remapping, so let the controller be responsible for all the remapping. This should take away the burden of doing any remapping away from the FS altogether. That implies that overwriting an existing (virtual) block should either fail or succeed, but never leave behind garbage.

As there's no direct relation between physical block location and block addresses, let's go all the way and have the flash drive provide a virtual address space which is much bigger than the real one (say the 64 bits). After all, one of the tricky parts it to make all files fit on a disk. This way files never need to be fragmentated any more.

Or not, but throw a few virtual address move functions to shuffle data around without actually touching the data, just the remapping table. And a command to get a free chunk of space, so the FTL can decide the address instead of both the FTL and FS making life harder for each other.

Or as sequential reads/writes are still and always will be faster than random ones, let the FS have control over it. And that implies doing the FTL in the FS, or near it. Or add commands to allocate sequential chunks of memory and leave the FTL in the controller.

At least there's something better possible than the current guessing around and toe stepping. One random write can turn into many more random writes, one to update the FTL mapping table, and a few FS related one (metadata update, extends update, journalling). The basic way of speeding up writes is just doing less of them and as sequential as possible.