By: Jouni Osmala (josmala.delete@this.cc.hut.fi), December 8, 2014 2:45 am
Room: Moderated Discussions
> >
> > or people using DAGs to explain why reference count is a solid
> > solution when complex object graphs typically have cycles
>
> Oh, I agree. My example was the simple case. The really complex cases are much worse.
>
> I seriously don't believe that the future is parallel. People who think you can solve it with compilers
> or programming languages (or better programmers) are so far out to lunch that it's not even funny.
>
> Parallelism works well in simplified cases with fairly clear interfaces and models. You find
> parallelism in servers with independent queries, in HPC, in kernels, in databases. And even
> there, people work really hard to make it work at all, and tend to expressly limit their models
> to be more amenable to it (eg databases do some things much better than others, so DB admins
> make sure that they lay out their data in order to cater to the limitations).
>
> Of course, other programming models can work. Neural networks are inherently very
> parallel indeed. And you don't need smarter programmers to program them either..
>
> Linus
Future is parallel in one way or another, simply because diminishing returns on improving serial we are pretty close to a point where improving some serial programs always means making other serial programs running slower because trade offs required just like P4. The trade off between adding serial and parallel performance to hardware, programmers almost always overestimate the amount of potential serial performance improvement relative the amount of improvement of peak parallel performance. Then they start discounting from that parallel number the percentage of non-parallel problems, and amdahl's law and everything, while hardware people have closer to realistic number as base number.
The real question for adding more parallel performance is this, is the subset of problems where there is enough parallerism available important enough to warrant adding more parallel performance, given the low relative cost of doing it. And another is how important is the subset of problems that you can force through those simple interfaces to become parallel. As for most things we already are running fast enough so adding parallerism isn't about making all things go faster, its more about handling Bigger datasets in acceptable time. We won't see UI spaghetti code that runs faster than human it interacts with become parallel, but we see that spaghetti code give simple working threads each a piece of enormous data set to be processed, and now to problem becomes only creating splits in large data set on natural boundaries and keeping pointers to said boundaries.
> > or people using DAGs to explain why reference count is a solid
> > solution when complex object graphs typically have cycles
>
> Oh, I agree. My example was the simple case. The really complex cases are much worse.
>
> I seriously don't believe that the future is parallel. People who think you can solve it with compilers
> or programming languages (or better programmers) are so far out to lunch that it's not even funny.
>
> Parallelism works well in simplified cases with fairly clear interfaces and models. You find
> parallelism in servers with independent queries, in HPC, in kernels, in databases. And even
> there, people work really hard to make it work at all, and tend to expressly limit their models
> to be more amenable to it (eg databases do some things much better than others, so DB admins
> make sure that they lay out their data in order to cater to the limitations).
>
> Of course, other programming models can work. Neural networks are inherently very
> parallel indeed. And you don't need smarter programmers to program them either..
>
> Linus
Future is parallel in one way or another, simply because diminishing returns on improving serial we are pretty close to a point where improving some serial programs always means making other serial programs running slower because trade offs required just like P4. The trade off between adding serial and parallel performance to hardware, programmers almost always overestimate the amount of potential serial performance improvement relative the amount of improvement of peak parallel performance. Then they start discounting from that parallel number the percentage of non-parallel problems, and amdahl's law and everything, while hardware people have closer to realistic number as base number.
The real question for adding more parallel performance is this, is the subset of problems where there is enough parallerism available important enough to warrant adding more parallel performance, given the low relative cost of doing it. And another is how important is the subset of problems that you can force through those simple interfaces to become parallel. As for most things we already are running fast enough so adding parallerism isn't about making all things go faster, its more about handling Bigger datasets in acceptable time. We won't see UI spaghetti code that runs faster than human it interacts with become parallel, but we see that spaghetti code give simple working threads each a piece of enormous data set to be processed, and now to problem becomes only creating splits in large data set on natural boundaries and keeping pointers to said boundaries.
