David Kanter (dkanter@realworldtech.com) on 1/21/11 wrote:
>
>Wow, that's ugly. The first 3 seem alright though.

Umm. As in 15% is "alright"? No, it's not. It's a 15%
performance degradation for the good case.

All for a very dubious improvement.

>Can you clarify what you mean by 'on-line' and 'off-line'?

So let's go back to Aaron's example of TRIM being "free()".

That's "on-line": actually freeing the storage while you're
actively doing the work. So a file delete does TRIM, because
it free's the diskspace.

IOW, on-line means that the disk basically always knows
what blocks are free, to a fairly high degree of accuracy.
Sure, it might be somewhat asynchronous, but it's doing
active "memory management" work while processing the load.

Now, what I'm claiming is that TRIM is too expensive, and
too damn flaky to be usable that way. It's not "just 8
bytes of command on the wire". It's a hell of a lot more.
It's a huge maintenance headache for both the filesystem
and the flash controller itself, during a time when
both have much better things to do, namely actually do
their load.

And I'm also claiming that the actual advantage of TRIM
isn't even big enough for it to make sense. The disk has
free remapping blocks for GC even without trim, in
the 5-10% range. TRIM is a relatively small incremental
improvement, it's not nearly as important as people seem
to think it is.

(One reason for that is that while random writes happen,
they are by no means the "bulk" of IO. Random writes are
very very important from a latency standpoint, because
they tend to be the kind of really important metadata
updates that you actually have to wait for, but they are
not generally all that common in a "big picture bandwidth"
kind of sense).

>My intuition is that if you have a period where the SSD is quiesced (say 200us
>where you aren't accessing the drive), that would be a fine time to do a little TRIMming.

I'm claiming that there's no point in talking about 200us
kind of timeframes. In those kinds of timeframes, TRIM
is always going to be a loss, for the simple reason that
most newly deleted blocks will be reused and then
TRIM is absolutely nothing but overhead.

This is the point that I really don't understand why
people don't "get". The alternative to TRIM is literally
to do absolutely nothing, and most blocks that are
free'd tend to be re-used.

Disk use tends to grow, not shrink, for chrissake!

Think about what that means for just one second. It means
that on average, you tend to write more than you delete.
So your default assumption will be that a disk
block that got free'd by the filesystem will be reused,
and overwritten again.

And since the bulk of the writes (by bandwidth) are
not random, on average you'll get fairly good patterns
again - without any TRIM at all.

Which means that doing TRIM on it is just stupid. It is
only overhead. The GC advantage is negligible.

And I posted how expensive trim is.

So INSTEAD of thinking of it like a "free()", think of
it as the "get rid of the heap entirely" operation.

So think of it as "process lifetime" vs "free() operation".
Because you don't want to do it at each "free()", you
want to do it when your whole IO job has finished entirely.

THAT is the kind of time-scale you should think about TRIM.
Not "if the disk is idle for 200us". More like "schedule
a cron-job once a week" kind of thing.

>So how much time does 'off-line' mean? Do you really mean that the drive has been taken out of the FS entirely?

No, it's reasonably technically easy to add simple support
for "let's trim the drive, and the filesystem will be a
bit slower while we do so". For example, most traditional
disk filesystems tend to split the logical disk (which may
be multiple physical disks, or just a partial one) into
"allocation groups" of some random size.

So you could - for example - have a background daemon that
just walks these allocation groups one at a time, and sets
a flag for "don't allocate in this group now", and then
looks at the free-block bitmap (or extent list, or whatever
data structure the filesystem uses for block allocation),
and does TRIM on any big chunks of free blocks.

So you can do it while the system is running - but it's
more of a defrag pass than anything you do in the filesystem
itself.

And we really are talking a timeframe of days or weeks,
not microseconds. Because exactly like defragmentation,
the whole and only point of TRIM is not the short-term
block allocation, it's the long-term issues.

>Do you mean ESP as in "magic crystal ball", or is this a technical term I'm not familiar with? : )

Yup, ESP as in "extra-sensory perception" aka "being able
to know IO patterns in advance".

Because really, think about the "filesystems grow, not
shrink" issue, and think about the fact that every single
write is an implied "trim".

And then think _really_ lazy TRIM support.

Linus