I see a couple of problems.

First, the CPU die would still need the L4 controller and interface pads.
This would be a big piece of sillicon, that you don't want in your L3 only models.

Secondly, if the L4 is going to be SRAM (although such huge sizes are not realistic for SRAM) and you need a separate die for the L4 models, it would be better to put the L4 cache itself the same die as the CPU. It's faster and doesn't hurt yields that much.
Which is pretty much what Intel and AMD do: they put big pieces of cache in their server models.

On the other hand, if you do like IBM and use (e)DRAM for L4 (L3 in IBM's case), the problem is that (e)DRAM access times are actually pretty big, they don't help that much you have a large multi-socket system which tend to have horrific memory access times, it doesn't help that much.