Richard Cownie (tich@pobox.com) on 8/21/09 wrote:
---------------------------
>I just wrote a program to fork a child, with a Linux pipe
>in each direction, and ping a one-byte message back and
>forth between the processes. Running on a Nehalem, it
>can do 500000 pings in 2.264 sec, about 4.5uS for each
>round-trip.

That's not bad. But it's still more than 10x slower than a two-thread ping-pong test that I just wrote and ran on a 2 GHz Core 2 Duo laptop.

In the particular operation in question, a 3 GHz Core 2 Quad manages to process roughly 100,000 items per second, removing one item off the head of the queue and inserting up to four items on two queues each time. That's about 10 uS per item with at least two communications per item (one to request and receive the work item, one to send the results), so using Linux pipes would basically halve our speed without even considering the overhead of copying between 25 and 100 MB of associated data into each process that is used to processes the items. (Unless you used shared memory, of course, with all the same synchronisation primitives that are used when threading...)

Right now we just use critical sections around the code that touches the queue. A quick test on the aforementioned 2 GHz Core 2 Duo laptop showed that entering and leaving a critical section costs about 65 ns -- about two orders of magnitude faster than you can send one byte back-and-forth.

>So yeah, inter-process communication is going to be a bit
>slower than just letting the data flow around between
>caches. But it's not *that* much slower. At least on
>Linux. I'm pretty optimistic that I can make multi-process
>parallelism work with granules of about 5mS of compute,
>though probably not with granules of 50uS.

So 10 uS, as in the above case, would be completely out -- as I said.

>Meanwhile, you mention that your program handles about
>10GB of data but manages it within the 2GB process size
>of 32bit Windows, with your own paging code.
>
>From a Solaris or Linux perspective, *that* seems like a
>strange choice, because the compilers and OSes support
>64bit apps easily (and a mixture of 32bit and 64bit apps).
>Porting our main app from 32bit Solaris to 64bit Solaris
>was only a couple of weeks work. The later porting from
>32bit x86/Linux to 64bit x86_64/Linux was even easier
>(the hard bits had been handled in the Solaris port).

That's not the problem at all. Both our compiler and OS support 32 bit and 64 bit binaries, and porting it to 64 bit would be no easier or harder than any other platform.

The problem is that we would still have to support 32 bit Windows because not everyone is using 64 bits yet, which at the very least involves an extra build and makes it hard for us to take advantage of the simplifications that being 64 bit would bring.

Nevertheless, if the benefit was great enough it could still be worth it. However, the 10 GB of data I mentioned has the following characteristics:

1. It is mostly only read, while generating much smaller amounts of other data.

2. When we do need to read a chunk of it, we end up reading almost all of that chunk, and we know when we are about to start processing that chunk.

3. The on-disk representation is far more compact than the in-memory representation.

Because of this we want to avoid paging the in-memory representation out to disk at all costs -- it is absolutely unnecessary and incredibly inefficient to write out an uncompressed chunk when we could simply re-create it from the compressed representation far more quickly. #2 means that relying on the VMM to only page in those bits we are actually using doesn't compensate, either, because we end up needing all of it.

So virtual memory limits aren't the issue at all -- it's the amount of physical RAM that we're worried about in this case, and the code monitors physical memory usage to avoid paging any of the data out to disk. As long as there is physical memory free it will keep currently-unused chunks in memory in case they are needed again soon, but as soon as physical memory becomes an issue it simply deletes those chunks because they can be easily re-created on demand later.

Since the software has no built-in limit to how many chunks the user might want to use (10 GB just happens to be the largest project size we have done) it has to behave this way anyway because we don't want to come to a grinding halt as soon as the user tries to create a project that can't all fit into physical memory at the same time -- especially when keeping those chunks around at all is purely a performance optimisation.

Given that, a 64 bit version of that program would only allow us to cache more data on those machines that had more than 4 GB of RAM, and the law of diminishing returns guarantees that the benefit of caching more that 4 GB is pretty negligible.

>But I guess the 32bit-to-64bit transition is nastier in
>Windows (as well as involving a higher price for the OS),
>so that's still a "dream" for you.

As Michael observed, the price of a license of 64 bit Windows is lost in the noise compared to our software. Heck, for those customers that insist on paying by credit card, the credit card processing fee is many times the cost of a Windows license. (Nevertheless, nearly 95% of our customers buy the most expensive version. When I was in Nevada last month one of our customers told me that he used the software on a project that saved the company US$60 million, so it's still a bargain.)

On the two occasions when customers have asked if we supported a different OS (one Linux, the other MacOS) we have offered to throw in a copy of Windows for free because that would be far cheaper for us than trying to port and maintain several versions.

We could bundle 64 bit Windows with the software but, alas, that's not the problem for those stuck with 32 bit Windows, either. There are a lot of IT departments out there that are very dictatorial about what they'll support, which is very frustrating for us. One of our customers in a US government department wanted to run 64 bit XP or Vista and couldn't -- the IT department ended up compromising and allowing them to run Windows Server 2003 64-bit (because it was on their list of approved OS's) but only if they maintained it themselves and never plugged it into the network (!).