Michael S (already5chosen@yahoo.com) on 9/10/08 wrote:
>
>For normal usage write latency doesn't matter at all.

This is absolutely not true!

It's a common mistake to think this, but the fact is,
write latency - even for individual IOs - can matter a
lot.

You are right to a point that it doesn't matter for normal
write-backs that are all totally buffered, but:

- most hardware interfaces have quite limited queues.

For example, if you're a SATA disk, the number of
outstanding writes to the disk is usually in the low
tens (ie 16-32). It can be lower.

This means that if you have a GC event that means that
writes will basically hit a roadblock for tens or even
hundreds of milliseconds, then if any writeback is
going on, then totally unrelated reads will also
be blocked, simply because there are no more entries in
the hardware queue.

Yes, you can limit the number of outstanding writes to
less than the number of hw queue entries, but nobody
does that because it would hurt throughput quite a lot.

- even if you never hit the above issue (and you will),
there are many operations that effectively require
synchronous writes.

This will commonly happen for certain metadata (it does
depend on the filesystem a lot), but it will also happen
for perfectly regular data when the user asks for a
sync write. And no, it's not at all "special" to do that.

Every single sane mail reader will do it, to make sure
that email won't be lost in case of a data loss. A lot
of editors will do it for the same reason. Anybody who
uses a database will do it, and many databases will have
a huge hiccup if the writes bunch up due to GC events.

And what uses databases? Pretty much everything. Look
at Firefox-3. It puts all your browsing history, and all
your preferences, and all your caches in one single
database, and then because the database engine thinks it
is all so horibly important (and the FF people were
silly enough to do so too), it will write much of it out
synchronously. Causing huge delays when the file
system must flush the whole log.

Really. None of the above issues are even remotely
odd or uncommon, even for very standard desktop operations
(and that's when you'll notice half-second pauses very very
clearly indeed!).

So single-write latency very much absolutely does matter.

And yes, you can put a RAM buffer on the SSD, have a cap
to give it time to flush even in the case of power loss,
return "success" early from the write, and yes, people
actually do that. But no, it doesn't actually help
at all for the event of something like a GC cycle causing
a few hundred ms of delays.

Why? Because when that happens, al writes will be
buffered up, and it will just move the point (slightly)
at which point the thing is stuck. It can hide the problem
under light load, but not under any kind of real
write-back event.

>Write latency doesn't matter so much that application
>program has no easy way to measure it.

Sadly, a lot of people seem to think like you. The whole
FF thing was a total disaster. Now, admittedly, it was
largely just the FF people being totally stupid, and just
thinking that databases are wonderful, but that in turn
was because people don't understand these issues!

>The only case where write latency matters is a data
>retention during sudden loss of power, but that's vendor's
>problem, not ours. At least I hope so.

See above: your buffers will fill up.

And if you think a few hundred ms is unrealistic for a
GC event, you're very naive! Even with much faster
RAM, even with much smaller data sets, GC events can take
seconds to complete if you do them badly. From the
timings I've seen of the OCZ SSD's (which actually perform
really well on average), when it hits a wall, it really
does hit a wall, ie latencies really are in the
second range, not milliseconds.

(To some degree, GC on a SSD is slightly simpler than GC on
some generic RAM allocation, since you have fixed 512-byte
blocks rather than any random size. But on the other hand,
the whole erase cycle thing introduces complexities all its
own, so I suspect it's all pretty nasty).

>Random writes throughput is something we could measure
>with relative ease, That tells as that, unlike write
>latency, random writes throughput could be practically
>important.

I do not know why you link "ease of measurement" with the
concept of "importance". Why do you link the two?

Yes, write-back performance is much harder to measure, but
that's because it's asynchronous, not because it's
not important.

And things like hiccups in sync write latencies are harder
to measure because they have very non-obvious patterns, to
the point of being almost random (ie it will depend on
other load issues like how much data is queued up in the
system, and where the GC event happens, etc etc).

And yes, latency in general is much harder to measure
well than throughput. For throughput, you just time the
start and the end, and divide by the amount of work. But
measuring latency means having to have lots and lots of
timing points over the whole workload, and doing some real
analysis to see where maxima are etc.

So yes, throughput is much much easier to measure. But then
thinking that that means that it is more important is really
misguided. When you actually use a desktop, I will
guarantee you that what ends up irritating you will
almost always be the latency issues, not the throughput.

Really.

Linus