Max (max@a.com) on 9/11/08 wrote:
>
>One way that a flash drive could have a lower effective
>write latency than its physical write latency is if writes
>were interruptable.
>
>Say the drive begins a slow write, then gets a read
>request. It pauses the write, does the read, then restarts
>the write.

For all I know, they may already do that. That's what having
multiple commands in flight allows (ATA calls it NCQ, aka
"Native Command Queuing"). However, you missed the point:
it doesn't really help all that much.

The reason it doesn't help all that much is that

- the latency if the individual write matters.

Even if you let other, shorter-latency, operations go
first, the latency of the write is still bad (and in
fact quite possibly got even worse).

NCQ improves the IO throughput of the device (not just
on SSD's, of course - it allows rotational media to
schedule the IO's optimally), but it also tends to make
the worst-case latencies worse even as the
average latency may improve.

And when talking various real-time loads (and the very
important issue of "interactivity" is one such thing),
worst-case latency can sometimes matter a whole lot more
than average latency.

- as mentioned, the queue depth is quite limited.

The command queue is often 16 or 32 entries, and if you
are doing writes, you're often doing them in bunches.
That's exactly because most writes are bufferable, and
when you do synchronous writes, they are almost always
not "single" writes (ie you may be doing a sync on a
file, but that will be all the file data and the meta-
data that goes with it, and then you'd wait synchronously
for that bunch of IO's to all have completed).

IOW, you'd expect that when doing writes, the queue is
often filled. Yes, you could limit the number of queued
writes, but since writes are generally where queueing
helps the most (since they are the ones that are
the most independent), that's going to hurt throughput,
and is pretty complicated anyway.

- Especially for something like a GC event, it's not that
it's a single write command that would be blocked,
it's that all of them end up blocking on GC when you
do end up having a blocking GC happen in the first place.

So if you've filled the queue with writes, they're all
going to be there, and none of them is going to succeed
early, and that read won't get through simply because
the queue is full.

Anyway, regardless of any of the queuing issues - even if
you let reads complete first - it's still going to cause
problems for those synchronous writes. And those synchronous
writes are not at all as uncommon as people think.

They happen for a lot of filesystems for just metadata
updates (ie file create/delete events), and they always
happen when the app asked for them (ie fdatasync() in a DB,
or just fsync() for a mail reader etc).

And yes, many of them do happen for simple everyday apps.
Browsers, mail readers, things like that. Apps where ten
milliseconds is not very noticeable, but half a second is
very noticeable and irritating indeed.

Linus