Ben T (ben.delete@this.noemail.com) on July 22, 2022 10:02 pm wrote:
> > Do people here think the ideal public cloud infrastructure would be
> > a cluster in a box rather than a single symmetric multiprocessor
> > running virtual machines from multiple users? For example, imagine
> > a 1U chassis containing 12 processor chips and 12 hot-swap
> > 2.5” U.2 NVMe drives. Each processor chip could include:
> >
> > 8 performance CPU cores plus caches
> > interfaces to two 128-bit LPDDR DRAMs
> > 4 PCIe lanes to connect to one NVMe drive
> > a 10G ethernet connection
> >
> > If the minimum unit a cloud user could get is 8 cores, they would not
> > be sharing this chip with anyone else so the risk of data leakage
> > between users would be greatly reduced. If it is possible to combine
> > these processor units for bigger single tasks but still keep them
> > partitionable into independent units for small tasks, that would be even better.
>
> Here is one way to put 12 processor packages in a 1U server. To fit in a standard 19” rack, the width of
> the motherboard has to be at most 17” (43.18cm). To have a single row of processor packages across the
> width of the chassis, each processor package could be about 3.4cm wide with a .2cm gap between packages.
> A processor package containing two LPDDR5X DRAM packages would be rectangular with the processor die in the
> middle and an LPDDR5X DRAM package on two opposite sides of the processor die. An LPDDR5X DRAM package is
> 1.4cm x 1.4cm. Apple’s M1 Max die is 2.16cm x 1.99cm so this all fits comfortably in a 3.4cm wide package.
> The long side of the rectangular processor package would be parallel to the air flow in the chassis. An alternative
> would be to have two rows of six processor packages (P) with a big gap between each package in a row so that
> cool air can reach the row of packages farthest from the air intake, like this:
>
> P_P_P_P_P_P
> _P_P_P_P_P_P
>
> A reason to have 12 processor packages is that the number of 2.5” drives that fit across the front
> of a chassis is a multiple of 12 (12 in a 1U chassis, 24 in a 2U chassis). Less fan power is needed
> when the heat sources are spread out instead of concentrated in one or two processor sockets.
>
> Does this seem practical?

Looks like you re-invented the micro-server approach, which has already failed? Why not just use cartridges of micro/pico/nano whatever arbitrary name server cards share an electrical and IO subsystems?

Couple of things:

- loss of instance versatility as this infrastructure will not be good at serving both low-end and high-end needs

- loss of vcpu performance as smaller thermal envelope and smaller shared uncore and memory

- added complexity in supply chain, technical support and inventory management

- added complexity in operations as you'll need new system images, update to orchestration tools

- because of smaller resource pools, more difficult in achieving high utilization to drive economics and energy efficiency

- you seem to be thinking about thermal management as a driver - once that becomes a gating factor, it's likely liquid cooling will be the response rather than generating more IT hardware swell


For some applications, what you describe may make sense at scale, such as web frontends (remember, Facebook made Intel create uniprocessor Xeon Ds). For general purpose cloud infrastructure, it does not seem like an attractive direction.

Bottom line is, the direction is ever larger resource pools, and cloud wants ever larger resource pools at the most economic point.