? (0xe2.0x9a.0x9b@gmail.com) on 8/23/09 wrote:
---------------------------
>Potatoswatter (potswa_m@c.com) on 8/23/09 wrote:
>---------------------------
>>? (0xe2.0x9a.0x9b@gmail.com) on 8/23/09 wrote:
>>---------------------------
>>>1. Reprogram selected code to run with 50+ threads. Do this in C/C++/Python/whatever
>>>programming language. The code can still use all Linux system calls, standard
>>libraries and almost all other libraries.
>>>2. Repeat step 1 until you get it right.
>>>3. Reprogram selected code to take advantage of SIMD. Do this in C/C++/whatever
>>>language with a library that has support for Larrabee.
>>>4. Repeat step 3 until you get it right. Sometimes you will have to go to step 1.
>>
>>Alas, 50x multithreading isn't trivial. If things worked that way, we'd already
>>be using tons of little processors. Larabee works with units equivalent to a 486,
>>but why weren't we working with 50x 68000's ten years ago?
>
>Maybe because it was technically impossible to place 50 such cores on one chip?
>Even if one imagines that it could have been done, the price of such a chip would be astronomical.

Just multiplying transistors, the 68000 has about 70k transistors and 70k*50 = 3.5M which was feasible in 1995 let alone 1999.

There have been many "army of ants" projects integrated on a chip, which I can't remember the names of now. MIT had something like "Mesh" and Chuck Moore I think called one "F22."

>>see http://en.wikipedia.org/wiki/Amdahl%27s_law
>>
>>You're presuming that Python, Linux+Bash, whatever open-source support libraries,
>>are perfectly suitable for a massively parallel HPC setup,
>
>Well, not really. The basic idea behind my suggestion is that the "space" available
>to LRB is much larger than the "space" available to CUDA/OpenCL. By "space" I mean
>the totality of options you have at your disposal as a programmer. A hypothetical
>scenario involving Python might include optimized functions implemented in C which
>are being called from Python. In addition, the C functions will be able to do anything
>a usual C function can do, including invoking Python code from C. Can anyone tell the same in the context of OpenCL?
>
>I am not implying it will be trivial.

You're just making up a story about a "novice" whose dreams come true. Most of those options won't work in the massively parallel part of the PC, so they're effectively nullified.

Moving the guts to C and calling from Python will require moving the guts to C. Try and see how often you can synchronize the languages before things suck.

SYNCHRONIZATION IS THE NAME OF THE GAME HERE.

>>But it's easy to saturate the memory with a few pointers when there's no cache.
>>It will be interesting to see how Larrabee does with GPU-like programming effort,
>but it will do TERRIBLY without that.
>
>My educated guess is that single-threaded non-SIMD performance of LRB will be somewhere
>between the performance of Atom and Core2. I would not call this "terrible".

If it's any slower than the main CPU, it's a performance loss, and that's as good as nothing! "Somewhere between Atom and Core2" is a generous ballpark.

>>Running terribly without recompilation doesn't get you much of anywhere. In the
>>best case where they use a programming tool which says "eliminate this mutex 4 tha
>>winz", and it points into their runtime, they'll just be giving up that much faster.
>
>1. Why?
>2. Are you implying that in OpenCL it will be easier? Unlikely.

I don't know which will be better but your confidence in traditionally single- or low-multithreaded tools is misplaced.

>>Or just use SSE. Oh yeah, I forgot, SSE leaves all the performance on the table by not being 16-way.
>
>1. The library is already able to utilize SSE. So at least it will be 4-way or 2-way (in certain cases).
>2. A small speedup is caused by the fact that Core2/Phenom are out-of-order 3-4
>issue cores, while LRB is not. This is quite important for estimating execution time of LRB instructions on Core2/etc.

A 16-way program running on an OOO 4-way core will be slower than a 4-way program, which is also easier to write.

>>Nested function calls, structures, and bitfields are all fine far as I know. You're
>>faster without them, but they work. You shouldn't need any libraries beside Linpack/Lapack.
>
>I don't recall any game engine which heavily uses Linpack/Lapack. And neither any
>ray-tracing engine. I do not recall any dynamic code generator which uses Linpack/Lapack. I do not ...

I roll my own linear algebra too, but libraries are useful for prototyping. The reason such optimized apps don't use Linpack or Lapack is that they don't use any libraries. They also don't use Python.

Game engines and raytracers do hella linear algebra. If you think Larrabee will speed up a JIT you haven't finished school.

>>If you are, you'll likely be in Amdahl's law doo-doo.
>>
>>>2. Reprogram selected code to run efficiently in CUDA/OpenCL computing model. Do this in CUDA/OpenCL.
>>
>>CUDA sucks, but OpenCL shows some promise to me. In any case, HPC has always required
>>some thought and will for the foreseeable future. You have to understand that you only go as fast as the weakest link.
>
>Are you implying that the speed of data accesses in your program equals to the
>speed processor<->memory? Or to the speed register<->disk, or register<->network?

You're thinking along the wrong lines. Ask your architecture prof about Amdahl's law, or read the WP article I linked.