Database clustering

anon (anon@anon.com) on 10/4/11 wrote:
---------------------------
>Jesper Frimann (jesperfrimann@gmail.com) on 10/4/11 wrote:
>---------------------------
>>anon (anon@anon.com) on 10/3/11 wrote:
>>
>>>>
>>>>I have no doubt that Google have a factor more servers than we do.
>>>
>>>(2 factors more)
>>
>>Look, please read what I write. 'I' have in a little country like Denmark with
>>6 million people, 10K+ physical servers. I have around half a million square feet
>>of datacenter. Sure comparing our country to the WW Google installation is 2 factors.
>>But that doesn't really make any sense now does it. It's like saying I am 100 times
>>stronger than your little finger Jesper.
>>>
>>>Oh, in that case, yes, your combinatorial problem with respect to hardware and
>>>OS software could easily be on par with Google.
>>
>>'On par with' ? Yeahh.. right.
>>
>>>>
>>>>Now the 'problem' Google has to solve is a big one. But it doesn't have as many
>>>>dimensions as the reality that my group has to face. So IMHO no hybris on my side :)
>>>
>>>Well... depends on what you define as their 'problem' :)
>>
>>The problem that google is trying to solve, resembles IMHO a 'classic' compute
>>farm problem. OK, Compute farms where each node is a supercomputer/or a farm in itself.
>>
>>>From purely administrative point of view, perhaps. But they have to do all the
>>>scheduling out of jobs and gathering results from a million machines. Fitting job
>>>jigsaws together on each machine to utilize most of the resources (eg. put memory,
>>>CPU, network, disk intensive jobs together, rather than several CPU intensive jobs on the same box).
>>>
>>
>>Yes and there is commercial available software that does this for you. It's not
>>rocket science, but not trivial either. Again they (Or my guess is that they have)
>>a supercloned environment, that runs on pretty uniform hardware where the variation
>>in resource parameters isn't that great.
>>
>>If you just look at something like cooling, then sure they have a massive amount
>>of machines. But so do we. I have a lot of 20KW+ servers so 'my' POWER usage is actually quite nasty.
>>
>>Furthermore when you have uniform building blocks, things gets much easier, you
>>can design your racks to have the right power/m² (or m³ if you want) uniform density
>>that fits the datacenter design points.
>>
>>I have racks that uses 2KW Max power, and I have racks that does 20 times that. Which makes cooling a bitch.
>>Again we are the garbage men of the industry, and very often what we get is the
>>stuff where clients have thrown the towel in the ring. So a large part of our work
>>is transforming foobar'ed infrastructure into things that can be managed, fairly
>>cheaply. But that takes time and money, and often the client doesn't want to pay,
>>for such a transformation. Hence we just keep their stuff alive.
>>Which often results in them having to do a 'Big Bang' transformation at some point
>>in time, which is much much more expensive in TCO.
>>
>>>I would say that although they gain something in standardization and being able
>>>to dictate hardware, they gain whole other dimensions in the size and costs.
>>
>>Yes, as I wrote in my last post. When you have total control of the whole stack, things gets much easier.
>>This is also why the biggest 'B*llsh*t Bingo' phrase currently is Cloud. Or FOG
>>og clown computing as we call it here :)=
>>It can be made to work. BUT...
>
>You're putting the task of "administer several OSes and get them talking to different
>switches" in the realm of rocket science, and apparently you think in comparison
>the task of running a distributed filesystem and scheduling jobs over 1M systems
>is trivial in comparison by the simple virtue of them "having total control of the whole stack".
>
>No offense, but I think your perspective if vastly skewed.

You sure? The first is 100% bureaucracy which you can't get rid of, the
second lets you have fun and reach new heights whatever your job is at
google. Both are hard fulltime jobs, but the second one is much more
rewarding.