Howard Chu (hyc@symas.com) on 9/9/08 wrote:
>
>The flash is only fast enough if everything else in the
>block management was done correctly; i.e., all incoming
>writes always mapped to sequential writes. It's obvious
>in the OCZ/Jmicron controller that this isn't the case.

Again, I don't have any actual insider knowledge, but I
will bet you that it's a really basic and well-known issue:

real-time garbage collection and compaction

and it's really as simple as that. Google for it, you'll
find lots of papers on it. Concurrency makes it even more
interesting, although I suspect that a flash controller
only needs to do a fairly limited mode of concurrency (ie
you probably don't need concurrent GC, but you want to at
least do writes and erases concurrently with each other
and with the GC - but the actual GC is almost certainly a
single thread).

The fact is, flash disks are simply an exercise in GC, and
the problems in that area are pretty damn well known. And
the most well-known of them all is that garbage-collection,
when done simplistically, is a huge problem for any
latency-sensitive operation.

But this is not something that people don't know how to do
well. There's been a lot of work into realtime GC over the
last few years in particular, due to languages like Java
being used in situations where long latencies simply are
not acceptable.

So the situation with Intel vs Jmicron controllers is pretty
trivial to explain: the Jmicron GC is almost certainly that
stupid synchronous kind. Which works well until it hits a
wall, and then has huge latencies.

In contrast, I bet you that Intel simply made their GC be
incremental. It's certainly more complex, but it's still
not rocket science or something unknown. Sure flash has

Btw, this is one reason you should totally ignore benchmarks
that rely on "throughput", since they're almost totally
worthless, unless they are doing small random writes, since
contiguous writes can easily be special-cased to the point
of not needing any GC at all.

And you should be worried about "latency" numbers: in any
marketing literature, they could easily be about "typical"
values, which can be the low ones, the ones without any
GC going on.

The numbers that really matter tend to be "max latency", but
things like IOps for random writes will come pretty close.
It's very hard to get high IOps numbers for writes if you
have high garbage collection costs. Same goes for throughput
with small random writes - you cannot easily do the same
kind of GC avoidance with random writes as you can do with
big contiguous ones.

Linus