> Why not just use cartridges of micro/pico/nano whatever arbitrary name server cards share an electrical and IO subsystems?

Your suggestion sounds like a blade server. I think the best option depends on the failure rate of the processors compared to the failure rate of the drives. If the failure rates are about the same, then blades make sense. If the drives fail at a higher rate than the processors, then hot-swap drives are a better option. With 12 processor packages soldered to a motherboard as I suggested, if a few processors fail, tasks would just not be assigned to those processors. If a lot of the processors in a chassis fail, the whole 1U chassis would be replaced. Having 12 processor packages soldered to a motherboard is less expensive and more reliable than blades because there are fewer connectors and other mechanical hardware.

I was thinking of this system for a general-purpose cloud computing service. Someone pointed out that Apple already has a cloud computing service for app developers called Xcode Cloud. Testing on Xcode Cloud is currently done with device simulators. If Apple developed servers with Apple Silicon, testing on Xcode Cloud could be done on Apple Silicon, which would be faster and more realistic than device simulators.

> - loss of vcpu performance as smaller thermal envelope and smaller shared uncore and memory

The processor die would be like a chiplet or tile in an EPYC or Xeon processor. The lack of sharing of the last level cache and memory between different users is intentional to prevent data leakage between users. If the processor package contains enough memory and last level cache for the working set of an application, there would be no loss of CPU performance. If the processor package doesn’t contain enough memory and last level cache for the working set of an application, the application would not be suitable for this system.

> - added complexity in supply chain, technical support and inventory management
> - added complexity in operations as you'll need new system images, update to orchestration tools

Apple was able to handle of the added complexity of supporting both x86 and Apple Silicon during their transition. Supporting both x86 servers and the type of server I described is not more difficult than that.

> - because of smaller resource pools, more difficult in achieving high utilization to drive economics and energy efficiency

I agree smaller resource pools make it more difficult to get high utilization of the hardware. When comparing Intel x86 to Apple Silicon, the most important thing driving economics and energy efficiency is TSMC process technology, not high utilization of hardware. Similarly, a mainframe has higher utilization of hardware than a personal computer, but a mainframe isn’t less expensive per user.

> For some applications, what you describe may make sense at scale, such as web frontends

If all this server is used for is web frontends, app testing, website testing and Hadoop, that would be plenty.

> the direction is ever larger resource pools

That does seem to be Intel’s direction. It may not be the best solution for everyone. Similarly, a Storage Area Network (SAN) is not the best solution for everyone.