By: rwessel (rwessel.delete@this.yahoo.com), October 15, 2021 4:49 am
Room: Moderated Discussions
Doug S (foo.delete@this.bar.bar) on October 14, 2021 10:23 pm wrote:
> rwessel (rwessel.delete@this.yahoo.com) on October 14, 2021 7:07 pm wrote:
> > While dealing with that on-die can be dealt with by just not putting two incompatible cores on a
> > die, that leaves the intra-cluster and VM migration issues with obvious solution. You could demand
> > that the cluster (or potential VM migration targets) all have the same specs in this regard, but
> > that seems a bit painful. The instruction definitions appear to define exceptions for when things
> > like that happen, and so fixing it up in software ought to be possible. Still a bit ugly.
>
>
> I'm tempted to say that if you choose to have incompatible
> ARM hardware in a VM cluster you deserve what you get...
>
> At any rate that's something for the hypervisor to address - it should have a way to enforce or at least define
> the "lowest common denominator" for this any other hardware differences that might arise. This is potentially
> a much larger problem as you can have differences in cache line size, page size, ARMvX.Y version etc.
>
> This isn't limited to ARM. You might run into potential issues if you mixed Intel and AMD hardware
> in a single VM cluster. I'd say you get what you deserve if you try it and experience issues.
While you don't want an infinite range of hardware in your cluster, you do need to allow some variation, or you cease to be able to upgrade it as new hardware comes along. For example, IBM's clustering for Z allows "N-2" systems in a cluster. So you could have (today) z13, z14s and z15s existing in a cluster, but no earlier systems. When the z16s come out, you'll need to remove any remaining z13s from your cluster before adding the new box.
> rwessel (rwessel.delete@this.yahoo.com) on October 14, 2021 7:07 pm wrote:
> > While dealing with that on-die can be dealt with by just not putting two incompatible cores on a
> > die, that leaves the intra-cluster and VM migration issues with obvious solution. You could demand
> > that the cluster (or potential VM migration targets) all have the same specs in this regard, but
> > that seems a bit painful. The instruction definitions appear to define exceptions for when things
> > like that happen, and so fixing it up in software ought to be possible. Still a bit ugly.
>
>
> I'm tempted to say that if you choose to have incompatible
> ARM hardware in a VM cluster you deserve what you get...
>
> At any rate that's something for the hypervisor to address - it should have a way to enforce or at least define
> the "lowest common denominator" for this any other hardware differences that might arise. This is potentially
> a much larger problem as you can have differences in cache line size, page size, ARMvX.Y version etc.
>
> This isn't limited to ARM. You might run into potential issues if you mixed Intel and AMD hardware
> in a single VM cluster. I'd say you get what you deserve if you try it and experience issues.
While you don't want an infinite range of hardware in your cluster, you do need to allow some variation, or you cease to be able to upgrade it as new hardware comes along. For example, IBM's clustering for Z allows "N-2" systems in a cluster. So you could have (today) z13, z14s and z15s existing in a cluster, but no earlier systems. When the z16s come out, you'll need to remove any remaining z13s from your cluster before adding the new box.