> > My opinion is requiring a license reduces its value as a benchmark. It means you won't
> > have a lot of runs from the general public and less data to compare against.
>
> Building real things that don't suck costs money. Where does it come from?

Questionable for two reasons
1. Plenty of open source software is free and doesn't suck. Linux and nginx come to mind.
2. Review sites are paid to put their benchmark suites together, through patreon or ad revenue. If they replaced their test suites with SPEC, they wouldn't survive a year.

> > The Phoronix test suite still has a lot of weird 'who uses this' tests, but it
> > also has some relevant stuff and is starting to look like SPEC done right.
>
> Are you fucking kidding? They call the site Moronix for a good reason.
>
> That benchmark suite is a fucking joke. There are one or two benchmarks that make
> sense, but it's mostly a collection of crap glued together with scripts.
>
> It combines a pile of microkernels and unrealistic application benchmarks.

That's exactly my criticism of SPEC. But instead of calling it names, you should point to specifics.

> Also the fact that the author thinks that some of those tests
> should be included says volumes about his technical expertise.

I never said it was perfect, and noted it has a lot of weird, irrelevant tests. But it also has a few that make sense, and it's available without high cost. My point is, it's shaping up to be better than SPEC.

Obviously it's not on-par with test suites from other big review sites. But those sites are not trying to go cross-platform either.