Dean Kent (dkent@realworldtech.com) on 2/3/09 wrote:
---------------------------
>Matt Sayler (sayler@thewalrus.org) on 2/3/09 wrote:
>---------------------------
>>
>>Lack of foresight is a beginner's curse. Premature optimization is the next stage.
>>The truly enlightened programmer knows how to strike a balance. ;-)
>
>My argument really wasn't about spending undue time and effort trying to take every
>eventuality into consideration. What I was attempting to point out is that we have
>inherent limitations in our ability to 'see' the bottlenecks in our designs. We
>usually only see them when the bottleneck becomes apparent through real-world use.

>I'll compare it to chess, where most people have difficulty seeing very many moves
>ahead. There are, however, techniques of pattern recognition that are used by
>the better players to help simplify what is inherently complex. Some are better at using these than others, of course.

Actually until the mid 90s i got a lot of laughter from game tree experts claiming that the pattern recognition should get improved in chess software.

The only thing that mattered was just throwing more hardware at it, as software already searched very close to 'perfect'. So there was "no software issues and better patterns was not important".

A total denial of the study of De Groot (1946) where he has proven that indeed mankind works as Dean demonstrates it.

Even in the start of 90s, the software model used was that throwing more hardware at it was the only solution and not better pattern recognition. That has changed over time.

However in software development this problem still is there.

Yet a much bigger problem is something else. If we watch for example a tennis grandslam in Melbourne we see a starting list of a couple of hundreds PROFESSIONAL players.

Players which are really good. Some even enhanced by major forms of dope. Muscles that are so unreal and fitness that is so fantastic that even some overdoped sportmen and sportwomen from the past look like holy.

Thing is, they are professional and very GOOD in what they do.

In software world however we deal with those 99.99% of software developers and programmers which, if we use tennisterms, have the habit to serve using bare hands, having just a single arm (so no backhand at all).

Thereby to look better, during that serve, a 150 kilo weighing collegue stands on the net with 2 feet. This proves then to the customer that serving with bare hands is better than the competitor who is on the sidelines shouting that one should serve using their superior ultra tiny and lightweight fiber optimized badminton racket.

After this crappy serve, you already have to pay bigtime for the bare hand serve. They argue then that the price is cheap, because at the next tennis court, a 100 meters further, some other company already charges money for just making a serve swing, without actually hitting a ball.

This swing of course gets executed by cheap construction workers, who thought they could make more money in IT than in construction.

All these guys work in IT and make money. Your software never ever gets produced by professionals who try to make a good product.

Things get solved as soon as possible, and to maintain products no one is using really clever overqualified software engineers.

A construction worker will do, as he also has the advantage for the boss that he's good in cashing in money from bad paying clients and speaks the same language like the client while doing sales.

That's reality of IT.

>Some of the scalability issues are the result of real or perceived resource limitations
>(memory, disk space, CPU power, etc.) - while others are the exact opposite... a
>presumption that there is unlimited resources. An example of the former might be
>small numeric fields to hold a count of records, or use of a text file rather than
>a database. An example of the latter might be unoptimized algorithms that work
>well with a small number of users/files/records, but quickly bogs down when there
>are a lot of them (like opening/closing a file within a record processing loop rather
>than before/after the loop - though that is obviously an amateurish example).
>
>Basically, just pointing out that scalability issues are the most difficult for
>many developers to see beforehand, unless they use tested 'pattern recognition' techniques.