David Kanter (dkanter@realworldtech.com) on 8/17/09 wrote:
---------------------------
>>If you mean that *only* speculation can create needless pressure, then I completely
>>agree. Or is there some other thing which can cause needless pressure other than speculation?
>
>How about poorly written code : )

(This is the tax I now have to pay for not specifying what I mean exactly.)

Imagine you are designing a processor. Assume this particular processor never speculates. To implement a particular feature, you will have to choose a particular algorithm. Such an algorithm may be poor, good or almost perfect. The poor version utilizes resources inefficiently, while the perfect version shows better utilization. In a sense, the poor version makes "needless pressure" on available resources in comparison with the better versions.

However, since I restricted the processor not to use any kind of speculation, you are forbidden to use the following features which clearly are using speculation instead of silently waiting for the actual outcome:

- Instruction cache: The processor is *speculating* it will be processing a recently encountered instruction. In truth, the processor does not know that for sure, thus it is a speculation.

- Instruction prefetching: The processor is speculating that it will be processing instructions nearby the current instruction. Note that this is different from "the processor is requesting 8 bytes from memory though the truth is that to begin decoding an instruction it needs only 1 byte" - since the smallest chunk of data from memory is 8 bytes the processor does not have a choice here, so the processor cannot be said to be requesting the other 7 bytes because it is speculating - it is requesting them because it does not have a choice. On the other hand, if the processor will store the 7 bytes into a buffer for later reuse, it is speculating because it does not know yet whether those 7 bytes will be needed in the future.

- Forwarding data from one pipeline stage to another: The processor is speculating that the forwarded data will be used to efficiently resolve some kind of a hazard. However, the processor does not know whether there will be such a hazard or not.

On the other hand, your processor *can* use the following feature:

- Keeping a byte *written* to a memory location L in a write-back cache: Well, I do not know what name to give to this case, because if I call it "speculation" I am sure it is not a speculation in the same sense as the examples above. So, let's call this case "speculation by necessity" and the former cases "speculation by volition". The idea is that there is a non-zero probability that the memory location L will be overwritten by instructions executed in the future. If this would happen, it should be seen as needless to send the previous value from the write-buffer to memory and needlessly occupying the wires connecting the processor to memory. But this is a case where *nobody* knows what will happen (unless you are able to solve the halting problem - good luck with that). I mean, we truly do not know whether 1 or 2 will happen:

1. The location L will be never written to in the future.
2. The location L will be written to in the future.

But, what we know for sure is that either 1 or 2 will become true. The problem is that the processor *has* to choose between 1 and 2 before the universe ends - or at least because of the fact that the write buffer is finite so it cannot hold the contents of L forever. The processor has to decide what bytes to evict into memory in order to make room for other data placed into the write-back cache.

(Well, I hope that this time I communicated the issue of "speculation" more clearly.)