Berkeley View on Parallelism

By: David Kanter (, February 15, 2008 3:48 pm
Room: Moderated Discussions
Howard Chu ( on 2/15/08 wrote:
>David Kanter ( on 2/15/08 wrote:
>>Howard Chu ( on 2/14/08 wrote:
>>>* The overarching goal should be to make it easy to write programs that execute
>>>efficiently on highly parallel computing systems
>>>Of course "easy" and "efficient" are opposed goals.
>>Of course, but python is a reasonably efficient language and it is easy. I think
>>the idea is to provide a parallel language that is simple and easy for the app
>>programmers, and then system developers can use C or C++ or whatever they want.
>OK, I'll accept that existing programming languages aren't really helping much
>here. I'm sure that system programmers would appreciate aids to parallelism too.
>There's plenty of useful machine instructions that aren't accessible as C primitives;
>it would be nice to have a language that provides these features without having
>to rely on non-portable assembler or compiler-specific extensions.
>>>* "Autotuners" should play a larger role than conventional compilers in translating parallel programs.
>>>These guys seem to like introspective JVMs and JIT optimizers. I always view this
>>>as a losing proposition. I can either have 100% of my CPU resources crunching a
>>>solution to my problem, or 100-N% crunching, and N% trying to dynamically re-optimize
>>>my code. Hint - write your code correctly in the first >place.
>>This is one place where I think they are spot on. Parallel programming is obscenely
>>expensive, and we need to drive the cost down in order to get more out of future
>>MPUs. One way to do that is to ensure that a lot of parallelism is automatically extracted.
>>Once upon a time, people thought compilers were stupid. But once they were able
>>to get 80-90% of the performance of a human coding assembly, they became pretty
>>damn popular (that and when architectures stopped being programmer friendly).
>>I think there's a clear trend towards HLL for applications programmers. I don't
>>see why this would stop for parallelism.
>I'm not saying that this shouldn't be done in a higher level language. I'm saying
>that I would prefer not to waste cycles re-discovering optimizations each time my
>code is executed. I'd prefer to pay all of the cost of optimization once, and then
>just let the code run without any introspection/virtualization overhead.

So there are two issues here:

1. You're not an average programmer - average programmers are not as good at optimization and are probably lazier
2. Some things are impossible using static optimizations that are trivial when you have run time information

>>>* To maximize programmer productivity, future programming >models must be more
>>>human-centric than the conventional focus on hardware or >applications.
>>>Kinda like what I touched on before, designing programming >languages whose input
>>>tokens aren't character-based. Aside from that it's all >bunk. 3000+ years ago a
>>>guy named Hercules had to clean out the Augean stables. >Today mucking horse stalls
>>>is still a dirty job. That's the nature of the job.
>>Some things are easier in python than in C though...
>I guess. It just strikes me that you can go too far here. >Make a job so easy that
>any idiot can do it, and you'll wind up with lots of >products created by idiots.

That's right, and that's economics for you. Programming has already evolved from a specialist activity to one that is easier and more approachable (ASM-->C-->C++/Java-->Python/Ruby). That trend will continue, so I think making parallel programming easier is the right idea.

I think Intel would be more than happy to trade 5-10mm2 of silicon for letting programmers use their hardware more easily.

>>>* Architects should not include features that significantly affect performance
>>>or energy if programmers cannot accurately measure their >impact via performance counters and energy counters.
>>This sounds like a good idea. Energy counters is >>definitely an interesting one.
>Yes, sounds nice to me for system programmers. Doesn't >sound like something most
>application programmers should have to think about. Though >it's good to have an API to expose it all, definitely.

If you don't want to expose it to applications programmers, then perhaps you need to have a system to do it they don't have to worry about it.

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Multicore is unlikely to be the ideal answer.Anders Jensen2008/02/14 04:24 AM
  And the links..Anders Jensen2008/02/14 04:25 AM
    Disappointing..Linus Torvalds2008/02/14 10:17 AM
      Disappointing..Mark Roulo2008/02/14 11:03 AM
        LOL (NT)Linus Torvalds2008/02/14 05:43 PM
      Disappointing..David Patterson2008/02/15 11:53 AM
        Disappointing..Linus Torvalds2008/02/15 05:01 PM
          Disappointing..anon2008/02/15 08:54 PM
            Disappointing..JasonB2008/02/19 07:45 PM
          Disappointing..Ilya Lipovsky2008/02/22 06:27 PM
          Disappointing..Scott Bolt2008/03/16 11:36 AM
        Need for new programming languagesVincent Diepeveen2008/02/19 06:18 AM
          Need for new programming languagesPete Wilson2008/02/24 11:41 AM
        Disappointing..Zan2008/02/25 10:52 PM
      Disappointing..Robert Myers2008/02/19 09:47 PM
        Disappointing..Fred Bosick2008/02/22 06:38 PM
          Disappointing..Robert Myers2008/03/01 01:17 PM
        The limits of single CPU speed are here.John Nagle2008/03/14 10:55 AM
          The limits of single CPU speed are here.Howard Chu2008/03/15 01:02 AM
            The limits of single CPU speed are here.slacker2008/03/15 08:08 AM
              The limits of single CPU speed are here.Howard Chu2008/03/17 01:47 AM
                The limits of single CPU speed are here.slacker2008/03/17 10:04 AM
    And the links..Howard Chu2008/02/14 12:58 PM
      I take some of that backHoward Chu2008/02/14 01:55 PM
      And the links..Jesper Frimann2008/02/14 02:02 PM
      And the links..Ilya Lipovsky2008/02/15 02:24 PM
        And the links..iz2008/02/17 10:55 AM
          And the links..JasonB2008/02/17 07:09 PM
            And the links..Ilya Lipovsky2008/02/18 01:54 PM
              And the links..JasonB2008/02/18 10:34 PM
                And the links..Thiago Kurovski2008/02/19 07:01 PM
                  And the links..iz2008/02/20 10:36 AM
                And the links..Ilya Lipovsky2008/02/20 03:37 PM
                  And the links..JasonB2008/02/20 06:28 PM
        And the links..JasonB2008/02/17 06:47 PM
          And the links..Ilya Lipovsky2008/02/18 02:27 PM
            And the links..JasonB2008/02/18 10:00 PM
              And the links..JasonB2008/02/19 03:14 AM
              And the links..Ilya Lipovsky2008/02/20 04:29 PM
                And the links..JasonB2008/02/20 06:14 PM
                  And the links..Ilya Lipovsky2008/02/21 11:07 AM
    And the links..Howard Chu2008/02/14 01:16 PM
      And the links..Jukka Larja2008/02/15 03:00 AM
      Berkeley View on ParallelismDavid Kanter2008/02/15 11:41 AM
        Berkeley View on ParallelismHoward Chu2008/02/15 12:49 PM
          Berkeley View on ParallelismDavid Kanter2008/02/15 03:48 PM
            Berkeley View on ParallelismHoward Chu2008/02/17 05:42 PM
              Berkeley View on Parallelismnick2008/02/17 09:15 PM
                Berkeley View on ParallelismHoward Chu2008/02/18 04:23 PM
                  Berkeley View on Parallelismnick2008/02/18 10:03 PM
                    Berkeley View on ParallelismHoward Chu2008/02/19 01:39 AM
                  Berkeley View on Parallelismrcf2008/02/19 12:44 PM
                    Berkeley View on ParallelismHoward Chu2008/02/19 03:25 PM
              Average programmersanon2008/02/18 12:40 PM
        Berkeley View on ParallelismJasonB2008/02/15 08:02 PM
        Berkeley View on ParallelismJasonB2008/02/15 08:02 PM
          Berkeley View on ParallelismDean Kent2008/02/15 08:07 PM
          Berkeley View on ParallelismRay2008/02/20 03:20 PM
            Berkeley View on ParallelismJasonB2008/02/20 06:11 PM
              Berkeley View on ParallelismFritzR2008/02/24 03:08 PM
          rubyinline, etc.nordsieck2008/02/22 03:38 PM
            rubyinline, etc.JasonB2008/02/23 05:53 AM
              rubyinline, etc.nordsieck2008/03/02 01:40 AM
                rubyinline, etc.Michael S2008/03/02 02:49 AM
                  rubyinline, etc.Dean Kent2008/03/02 07:41 AM
                    rubyinline, etc.Michael S2008/03/02 08:19 AM
                      rubyinline, etc.Dean Kent2008/03/02 08:30 AM
                        rubyinline, etc.JasonB2008/03/02 05:26 PM
                rubyinline, etc.JasonB2008/03/02 06:01 PM
                  rubyinline, etc.Anonymous2008/03/03 02:11 AM
                    rubyinline, etc.JasonB2008/03/03 09:40 AM
                      rubyinline, etc.Foo_2008/03/09 09:59 AM
                        rubyinline, etc.JasonB2008/03/10 01:12 AM
                        rubyinline, etc.Gabriele Svelto2008/03/10 02:22 AM
                          rubyinline, etc.JasonB2008/03/10 04:35 AM
                            C++ for beginnersMichael S2008/03/10 05:16 AM
                              C++ for beginnersJasonB2008/03/10 06:35 AM
                          C++Michael S2008/03/10 04:55 AM
                rubyinline, etc.Linus Torvalds2008/03/03 11:35 AM
                  rubyinline, etc.Dean Kent2008/03/03 02:35 PM
                    rubyinline, etc.JasonB2008/03/03 03:57 PM
                      rubyinline, etc.Dean Kent2008/03/03 08:10 PM
                        rubyinline, etc.Michael S2008/03/04 01:53 AM
                          rubyinline, etc.Dean Kent2008/03/04 07:51 AM
                            rubyinline, etc.Michael S2008/03/04 08:29 AM
                              rubyinline, etc.Dean Kent2008/03/04 08:53 AM
                                rubyinline, etc.Michael S2008/03/04 11:20 AM
                                  rubyinline, etc.Dean Kent2008/03/04 02:13 PM
                                    read it. thanks (NT)Michael S2008/03/04 04:31 PM
                  efficient HLL'sPatrik Hägglund2008/03/04 03:34 PM
                    efficient HLL'sWes Felter2008/03/04 09:33 PM
                      efficient HLL'sPatrik Hägglund2008/03/05 01:23 AM
                        efficient HLL'sMichael S2008/03/05 02:45 AM
                          efficient HLL'sWilco2008/03/05 05:34 PM
                            efficient HLL'sHoward Chu2008/03/05 07:11 PM
                              efficient HLL'sWilco2008/03/06 02:27 PM
                    efficient HLL'sanon2008/03/05 08:20 AM
      And the links..Groo2008/02/17 04:28 PM
        And the links..Vincent Diepeveen2008/02/18 02:33 AM
Reply to this Topic
Body: No Text
How do you spell green?