I see there is some confusion about MPX, so this is an attempt at a short guide.

Note that my only info is that PDF and the amended Linux ABI at http://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf, and I have no contacts with Intel, so YMMV.

1. What is MPX

MPX appears to be primarily a way to automagically eliminate most/all buffer overflows and other memory-related bugs from C/C++ programs with a simple recompile using a version of gcc supporting MPX to which the -fmpx flag has been passed, without the drastic performance drop that would take place without hardware support.

MPX is designed so that C/C++ code compiled with -fmpx is fully binary compatible with legacy code.

However, MPX may cause memory usage to expand by up to 5 times (if all data is composed of pointers), increases memory bandwidth usage and executing the bounds check instructions might also take extra time.

It also provides a different way to do bounds checking in Java and similar, which saves GPRs in loops by putting bounds in BND registers instead, and may or may not be faster, depending on the relative performance of the MPX instructions compared to basic ALU instructions.

2. Overview of bounds checking

Bounds checking can be interpreted in two different ways:
1. [for Java] Given an array object a, make sure that a[i] causes an exception unless i is in-bounds
2. [for C/C++] Replace pointers with a "fat pointer" abstraction such that (p + i) can be performed for any i without exceptions, (p + i) - i == p for any i, but *p causes an exception if the "net" offset added to the pointer originally returned from malloc/new is negative or greater than the memory allocation size.

(1) can be implemented in terms of (2) or by storing the array size before the array and checking
(2) can be implemented by replacing each pointer with a (ptr, lb, ub) triple, and have pointer arithmetic or other manipulations only affect "ptr", while loads and stores check that ptr is within [lb, ub]

MPX allows to do both, although it seems really useful only if one needs (2).

3. MPX bounds checking with registers

MPX adds BND0-3 registers, which each consist of a couple of 64-bit values (lb, ub).

*IMPORTANT*: "ub" is normally supposed to be NOT the upper bound, but the bitwise negation of the inclusive upper bound, so that a BND register set to (0, 0) causes bounds checking to never fail because it represents the [0, 0xffffffff] interval. Section 6 discusses doing the opposite.

The basic instructions are BNDCL and BNDCU, which check the lower and upper bounds respectively given a BND register and an effective address (failure if v < lb and v ub), which is of some unexplained use that I'll conjecture about later.

Note that BNDCL/BNDCU don't have an operand size, so for instance if you want to check for a 64-bit access, you might want to add 7 to the effective address in BNDCU, unless you are sure that everything is 64-bit aligned.

4. Getting bounds in and out of registers

The first way to set up a bounds register is BNDMK, which sets up the (lb, ub) fields based on GPRs and immediates.

More precisely, it performs a LEA into the complemented ub part of a BND register, and stores the base register of the effective address into lb.

So BNDMK BND1, [RAX+RBX*4+33] sets lb = RAX and ub = NOT(RAX+RBX*4+33)

The second way is BNDMOV, which either moves a BND register into another, or moves the (lb, ub) from/to a 128-bit variable in memory.

The third way is BNDLDX and BNDSTX, which look up a dictionary data structure with the address of the pointer variable, and load or store the (lb, ub) there, and which are discussed in the next section.

Note that there is no way to move bounds into GPRs: one needs to BNDMOV to memory and then read back the bounds from memory two MOVs.

5. MPX bounds checking in high-level languages and BNDLDX/BNDSTX

First of all, bounds are generally created with BNDMK, either for stack objects by the compiler, or by the malloc or equivalent library implementation (using intrinsics).

When pointers are in registers, bounds are stored by allocating a BND register for it, and putting in the minimum amount possible of BNDCL/BNDCU instructions to exhaustively check them.

If there aren't enough BND registers, they can be spilled to/from the stack with BNDMOV.

So the only problem is where to put bounds for pointer variables in memory.

For high-level languages such as Java, the simplest strategy is to redefine the pointer representation to include the bounds, and load them too with BNDMOV.

However, for C++, we cannot redefine pointers without breaking binary compatibility, so we create a huge separate dictionary that maps from the pointer *variable* address to the lower and upper bound, which we access with BNDLDX and BNDSTX.

***NOTE***: the key in the dictionary is *NOT* the value of the pointer, but is the *address* of the pointer variable: this is because otherwise it would be possible for pointer arithmetic to "convert" a pointer to an object to a pointer to another object, which means that either bounds checking doesn't work, or we need to throw an exception immediately on the offending pointer arithmetic, which is generally not compatible with things like having a pointer just after the end of an array to serve as an "end marker".

We have a problem though: if some non-MPX-aware code changes the pointer value but doesn't update the data structure, we don't want to just use the previous bounds, since that makes no sense.

To handle this, the dictionary value is actually a (lb, ub, ptr_value, reserved) 4-tuple, where ptr_value is the value of the pointer variable (remember that the key is the address of the pointer variable).

So, BNDLDX takes two inputs: the address of the pointer variable, and the value of the pointer variable (which has presumably just been loaded with MOV), which are encoded as the (base+disp) and index parts of an effective address (the scale is ignored).

If the provided pointer value doesn't match the one in the table, it loads (0, 0), while otherwise it loads the (lb, ub) in the table.

BNDSTX works like BNDLDX, except that instead of checking the pointer value, it stores it.

The data structure accessed by BNDLDX/BNDSTX is essentially a 2-level page table rooted in an MSR and located in virtual memory, with the details described in the PDF.

Presumably the kernel and libc would cooperate to automatically allocate the data structure lazily on demand, just like page tables.

6. MPX bounds checking and the C/C++ ABI

The final issue is how to design a C/C++ ABI for this.

For pointers in memory, we can use BNDLDX/BNDSTX as described above.

For parameter passing, BND0-3 are used for the first 4 pointers, while for the rest BNDLDX/BNDSTX are used, and the keys are stack location in the stack frame of the *callee* (return_address - 8, return_address - 16, etc.)

Of course, we want to support MPX and non-MPX code calling each other.

MPX code calling non-MPX code is a non-issue, but for non-MPX code calling MPX code, we want to make sure that bounds checking always succeed.

To achieve this, Intel CPUs zero out all BND0-3 registers in (near, not tiny) CALL/RET/JMP/Jcc, unless the REPNE prefix is specified, which tells the CPU that the control flow source is MPX-aware.

This feature can be turned off by setting BNDPRESERVE in an MSR.

For pointers in memory, the ptr_value will not match in the BNDLDX instruction if the pointer has been created or modified by non-MPX-aware code, so (0, 0) will be loaded and checks will always fail.

7. Non-permissive by default: BNDCN and inverting the ub convention

Intel provides a weird BNDCN istruction, that checks ub without doing the NOT operation.

This hints at the possibility of software using the convention that ub is the upper bound instead of the NOT of the upper bound.

The essential difference is that this way, the default (0, 0) bound always fails instead of always succeeding, allowing a "strict checking" mode.

In particular this means:
1. If non-MPX code changes a BNDLDX/BNDSTX managed pointer, bounds checking will then fail
2. If non-MPX code calls a function using MPX with BNDPRESERVE=0, bounds checking will then fail

The side effects of doing this is that BNDLDX/BNDSTX and BNDMOV memory layouts are not compatible with the standard convention, and BNDMK becomes unusable without extra effort, in addition to having to replace BNDCU with BNDCN.

To achieve similar effects to BNDMK BND0, [BASE+xxx] in this flipped convention, it seems that the following needs to be used:
LEA rtmp, [rbase+xxx]
NOT rtmp
SUB rtmp, rbase
BNDMK BND0, [rbase+rtmp]

8. An alternative proposal for an instruction set

A far more streamlined instruction set would involve making the BND registers internal, increasing the number of them, and augmenting GPRs with the number of an associated BND register, where two special values would be reserved for non-permissive and all-permissive bounds instead of the bounds stored in a BND regiser.

Existing move and arithmetic instructions between registers would copy or reset bounds on the destination, while operations like XOR would initialize the bounds.

With this approach, BNDCL/BNDCU/BNDCN could be removed in favor of checking bounds on all memory accesses (or perhaps only when a prefix is present), using bounds on the base register GPR.

BNDMOV to memory could be preserved, or a MOVFAT instruction could perhaps be added that would do a 192-bit load/store of both the pointer and the bounds.

BNDSTX and BNDLDX could be replaced with a prefix that would store/load the bounds from the extra table on any instruction with memory operands.

This could probably be implemented as an extension to the existing MPX instruction set, where an internal BND register array would be shared both as renamed versions of the MPX BND0-3 and as the bounds associated with renamed GPRs.