Michael S (already5chosen@yahoo.com) on 9/10/08 wrote:
>
>But in this case I'd suffer from increased _read_ latency
>(and/or reduced throughput), not write. That's something I
>can measure.

What are you blathering about?

"measurement" has nothing to do with "importance".

Also, you claimed that write latencies don't matter. I
told you that you were wrong, and why. Why are you trying
to change the subject with some totally irrelevant thing?

Btw, no, you can't measure those read latencies any more
easily than you can measure write latencies, and I don't
even understand how you can possibly claim that they are
somehow different. Just trace the IO and the completion,
and they are exactly the same.

>>Every single sane mail reader will do it, to make sure
>>that email won't be lost in case of a data loss. A lot
>>of editors will do it for the same reason. Anybody who
>>uses a database will do it, and many databases will have
>>a huge hiccup if the writes bunch up due to GC events.
>
>As I said in the other post, then the storage device
>should lie to you, or let's call it telling more relevant
>truth :-)
>It should tell me that the data is written not when it is
>actually written to non-volatile media but immediately
>after device itself is sure that it will inevitably happen.

Do you have some reading disorder?

That's what many drives do. It matters not one whit, as
I already explained in the exact message you replied to.
The buffers are inevitably too limited, and buffering
things changes nothing. At most it may move the
point by a couple of IO's.


>But then again, I'd feel the problem as the read latency,
>read throughput or write throughput.

..and again, your touchy-feely "I feel" thing is totally
irrelevant to the problem. It has no basis in reality, nor
does anybody else really care how you feel. I'm sorry, but
reality is that which doesn't change just because you stop
believing it.

Write latencies matter. The fact that you seem to have
trouble accepting that is about as relevant as creationism
is.

Of course, you can always turn a write latency into a
"throughput" measure, by just picking a small enough data
set. So in that sense you may be technically correct, but
only by making a totally inane and intellectually dishonest
statement. It's still about the latency.

>>I do not know why you link "ease of measurement" with the
>>concept of "importance". Why do you link the two?
>
>Because, when something is important to me I should always
>know how to measure it, may be not precisely, but at least
>I should be easily able to tell that A is bigger than B.

You're putting the cart ahead of the horse here.

Of course you can always measure everything. Yes, you
can measure write latency. After all, if you couldn't do
it, it wouldn't exist.

But it's harder to do, and more importantly, the
fact that it is harder to do in no way makes it any less
important.

Latency very fundamentally is harder to measure than
throughput, but that doesn't make it any less important.

In particular, especially with events that happen
almost randomly (ie they depend on non-obvious things that
you don't necessarily even understand or know all the
relevant parameters for), measurement is even harder, since
now you have to measure over a whole lot of parameters that
you don't know how they interact. And then you have to sift
out the (few) cases that differ from the typical ones.

Which means that quite often, measuring these things is
neigh impossible without special software that may not even
exist. Have you tried measuring performance jitter in your
web browser JVM due to JIT'ing code lately? Would you even
have a clue how to do that?

And that is exactly the kinds of things you see with
garbage collection. Latencies are almost always very low,
but then you see occasional huge spikes. But those may
never show up in bandwidth measurements, depending
on how you do them - since bandwidth is by definition about
averaging over multiple ops or data (if it isn't, then it's
a latency measure).

But yes, you can measure certain things that are probably
going to show the effects, often by trying to exaggerate
them. That's why the "IOPS for small random writes" load is
interesting - it won't show the latency problem directly,
but it will have turned it into something that is easier
to measure by making an intelligent guess about the problem,
and trying to exaggerate it.

But notice how "IOPS for small random writes" is not the
same thing as the latency problem per se. You seem
to think it is. You seem to think that the latency of a
single write doesn't matter, just because it's harder to
measure. BUT THAT IS WRONG.

The latency of a single write does matter, exactly
because it can be very noticeable as a huge interactivity
problem every once in a while. But yes, interactivity issues
are really hard to measure. But that doesn't make them any
less important.

So you dismiss these things just because they are hard to
measure, and then try to say that the artificial loads
(that are no longer hard to measure) are somehow more
important.

That is stupid. The artificial loads are no more
important. That should be obvious to a five-year-old. They
are just easier to measure.

The two have absolutely nothing to do with each other.

Linus