SLC vs. MLC

By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), January 12, 2009 7:38 pm
Room: Moderated Discussions
Michel (michel@lespinasse.org) on 1/12/09 wrote:
>
>I'd like to ask, how much SSD knowledge do you think the
>upper layers of the storage stack should have ?

Not a whole lot, really.

>* Making sure that the FS aligns data blocks (I'm thinking
>about the 4k blocks in an ext2 fs) such that no data block
>goes across an erase region boundary (this will also be
>important with spinning disks in the future when they do
>away with 512 byte sectors...)

You really cannot expose the erase region boundary - simply
because it makes no sense. If it matters, you'd be doing
something wrong.

So you do need to expose the "sector size", of course, ie
the size of a minimal (efficient) read or write operation.
And the range of reasonable sizes is actually pretty small:
it's from 512 to 4kB. Anything else will cause huge issues.

A 512b block size means that you can now run all legacy
systems on that disk - which a lot of people still do
care about. It means that you don't need to worry about
any legacy partitioning schemes etc. So there are some real
advantages to it.

A 4kB blocksize is probably acceptable to all modern OS's,
but a 8kB one would not work with Linux, for example. The
4k limit comes from the common x86 page size: we want to do
normal IO in chunks of that size.

Exposing any other kind of odd alignment requirement simply
isn't going to fly. If somebody does that, I'd say "buy
another disk". It's not worth the pain for anybody.

>* Possible FS data placement optimizations to try to have
>files fit in a single erase block, similar to what's being
>done with stride/stripe mke2fs options when using RAID

You'd almost have to be doing something wrong on the SSD
if you need this.

>* Maybe having the disk scheduler spend less effort trying
>to reduce seeks

We are starting to do this, although it doesn't really
appear to be a huge deal. You still want to do some
coalescing of requests, because the per-command overhead
is noticeable, and on at least SATA you're limited to
something like 32 tags, so you cannot have more commands
than that outstanding - which means that you don't want
totally small commands (and in order to coalesce multiple
writes into one, you still need to sort them!)

>and more effort trying to reduce the number of erase
>blocks being written to - for example, when one write
>request will cause a given erase block to be written to,
>maybe the disk scheduler could try to find any dirty
>buffers within that same erase block

We really don't even know what blocks would be in the
same erase block, since it's all translated. But more
importantly, we really don't even care.

What is more likely to matter is:

>* The ext2 optimizations that split inode tables into many
>small subtables, each close to their data blocks, in order
>to reduce seeks, seem quite pointless on SSD

Yes. We're starting to see "SSD flags" on filesystems,
which are about the issue of seeks no longer being very
expensive (for reads in particular). And yes, that impacts
filesystem layout.

For example, you may want to have a block allocator that
tries to just allocate the new blocks entirely linearly.
The end result would suck on a disk, because now when
you read things you need to seek around like mad, but it
is not only simpler but will also generally result in nicer
write patterns.

Of course, you want to balance things. You still want to
have some read locality just to make it cheaper to read
files in one go etc..

So doing some things differently definitely makes
sense. A lot of the optimizations that were done explicitly
for the latencies of rotational media simply do not make
sense any more.

Another example of things we can do at a higher level that
I alluded to in a separate post: telling the SSD when a
block is no longer in use will let the GC and remapping
layer optimize things. There's a cost to sending that
command, of course, and you'd better get it right (because
"undelete" is not going to work), but it likely worth
it.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
First Dunnington benchmark resultsMichael S08/19/08 10:54 AM
  First Dunnington benchmark resultsrwessel08/19/08 01:42 PM
    First Dunnington benchmark resultsAaron Apink08/19/08 05:49 PM
      First Dunnington benchmark resultsJoe Chang08/19/08 06:28 PM
        First Dunnington benchmark resultsrwessel08/21/08 09:49 AM
          First Dunnington benchmark resultsJoe Chang08/21/08 03:10 PM
            First Dunnington benchmark resultsrwessel08/21/08 06:42 PM
              First Dunnington benchmark resultsJoe Chang08/21/08 07:12 PM
      First Dunnington benchmark resultsrwessel08/21/08 09:45 AM
        First Dunnington benchmark resultsAaron Spink08/21/08 01:12 PM
          First Dunnington benchmark resultsJoe Chang08/21/08 03:15 PM
    First Dunnington benchmark resultsRichard Cownie08/20/08 02:59 AM
      First Dunnington benchmark resultsAnders Jensen08/20/08 03:26 AM
        +SSDAnders Jensen08/20/08 03:30 AM
        First Dunnington benchmark resultsRichard Cownie08/20/08 11:04 AM
          First Dunnington benchmark resultsslacker08/20/08 12:35 PM
            First Dunnington benchmark resultsDoug Siebert08/20/08 07:54 PM
              First Dunnington benchmark resultsRichard Cownie08/20/08 08:58 PM
                SLC vs. MLCDavid Kanter08/21/08 01:16 AM
                  SLC vs. MLCMatt Sayler08/21/08 06:25 AM
                  SLC vs. MLCRichard Cownie08/21/08 06:32 AM
                    SLC vs. MLCLinus Torvalds08/21/08 08:39 AM
                      SLC vs. MLCMichael S08/21/08 09:07 AM
                        SLC vs. MLCLinus Torvalds08/21/08 09:52 AM
                          SLC vs. MLCMichael S08/21/08 10:35 AM
                            OLTP appliance = mainframe? (NT)Potatoswatter08/21/08 11:44 AM
                              OLTP appliance = HP NonStop?Michael S08/21/08 12:03 PM
                                OLTP applianceJoe Chang08/21/08 03:33 PM
                                  OLTP appliancePotatoswatter08/21/08 03:59 PM
                            SLC vs. MLCAaron Spink08/21/08 01:29 PM
                          SLC vs. MLCDan Downs08/21/08 11:33 AM
                            SLC vs. MLCrwessel08/21/08 12:45 PM
                              SLC vs. MLCDan Downs08/22/08 08:21 AM
                            SLC vs. MLCAaron Spink08/21/08 01:34 PM
                          SLC vs. MLC vs DRAMpgerassi08/21/08 12:24 PM
                            SLC vs. MLC vs DRAMDavid Kanter08/22/08 01:31 AM
                          SLC vs. MLCGroo08/23/08 12:52 PM
                      SLC vs. MLCDoug Siebert08/21/08 06:14 PM
                        SLC vs. MLCLinus Torvalds08/22/08 08:05 AM
                          SLC vs. MLCDoug Siebert08/22/08 02:27 PM
                            SLC vs. MLCEduardoS08/22/08 06:26 PM
                            SSD Controller differentiationDavid Kanter08/22/08 09:35 PM
                              SSD Controller differentiationDoug Siebert08/22/08 10:34 PM
                                SSD Controller differentiation (supercaps, cost...)anon08/23/08 10:18 AM
                                  SSD Controller differentiation (supercaps, cost...)Doug Siebert08/23/08 10:40 AM
                            SLC vs. MLCLinus Torvalds08/23/08 10:50 AM
                              SLC vs. MLCLinus Torvalds09/08/08 12:03 PM
                                SLC vs. MLCMax09/08/08 01:51 PM
                                  SLC vs. MLCHoward Chu09/08/08 09:04 PM
                                    SLC vs. MLCMax09/08/08 10:29 PM
                                      SLC vs. MLCHoward Chu09/09/08 12:12 AM
                                        RAM vs SSD?Jouni Osmala09/09/08 01:06 AM
                                          RAM vs SSD?Max09/12/08 12:51 PM
                                            RAM vs SSD?EduardoS09/12/08 04:27 PM
                                            Disk cache snapshottingMax09/13/08 08:34 AM
                                              Disk cache snapshottingHoward Chu09/14/08 09:58 PM
                                                Disk cache snapshottingMax09/15/08 12:50 PM
                                        SLC vs. MLCLinus Torvalds09/09/08 07:43 AM
                                          SLC vs. MLCHoward Chu09/09/08 09:42 AM
                                            SLC vs. MLCLinus Torvalds09/09/08 10:39 AM
                                              SLC vs. MLCMichael S09/10/08 12:29 AM
                                                SLC vs. MLCanon09/10/08 02:51 AM
                                                  SLC vs. MLCMichael S09/10/08 03:09 AM
                                                    SLC vs. MLCMax09/10/08 04:48 AM
                                                      SLC vs. MLCMichael S09/10/08 05:52 AM
                                                        SLC vs. MLCMax09/10/08 06:28 AM
                                                    SLC vs. MLCMatt Sayler09/10/08 06:21 AM
                                                      SLC vs. MLCMichael S09/10/08 09:17 AM
                                                    SLC vs. MLCanon09/10/08 06:29 AM
                                                      SLC vs. MLCMichael S09/10/08 09:23 AM
                                                        SLC vs. MLCMatt Sayler09/10/08 10:45 AM
                                                SLC vs. MLCLinus Torvalds09/10/08 07:25 AM
                                                  SLC vs. MLCMichael S09/10/08 09:54 AM
                                                    SLC vs. MLCLinus Torvalds09/10/08 10:31 AM
                                                      Physical vs effective write latencyMax09/11/08 07:35 AM
                                                        Physical vs effective write latencyLinus Torvalds09/11/08 09:06 AM
                                                          Physical vs effective write latencyLinus Torvalds09/11/08 09:48 AM
                                                            Physical vs effective write latencyLinus Torvalds09/11/08 11:39 AM
                                                              Physical vs effective write latencyMark Roulo09/11/08 12:18 PM
                                                                Physical vs effective write latencyDoug Siebert09/11/08 05:59 PM
                                                                  Physical vs effective write latencyLinus Torvalds09/11/08 07:16 PM
                                                                    Physical vs effective write latencyDoug Siebert09/11/08 10:28 PM
                                                        Physical vs effective write latencyMS02/03/09 03:06 PM
                                                  SLC vs. MLC - the trick to latencyAnonymous09/11/08 12:39 PM
                                                    SLC vs. MLC - the trick to latencyanon09/11/08 01:17 PM
                                                      SLC vs. MLC - the trick to latencyAnonymous09/11/08 05:25 PM
                                                        SLC vs. MLC - the trick to latencyDoug Siebert09/11/08 05:47 PM
                                                          SLC vs. MLC - the trick to latencyrwessel09/11/08 06:01 PM
                                                        SLC vs. MLC - the trick to latencyanon09/12/08 12:00 AM
                                                          SLC vs. MLC - the trick to latencyAnonymous09/12/08 08:52 PM
                                                            SLC vs. MLC - the trick to latencyanon09/13/08 10:06 AM
                                                            SLC vs. MLC - the trick to latencyUngo09/15/08 12:18 PM
                                                              To SSD or not? One lady's perspectiveDavid Kanter09/22/08 01:12 AM
                                                                To SSD or not? One lady's perspectiveHoward Chu09/22/08 04:02 AM
                                                                  To SSD or not? Real data..Linus Torvalds09/22/08 07:33 AM
                                                                    To SSD or not? Real data..Ungo09/22/08 12:27 PM
                                                                      4K sectorsWes Felter09/22/08 06:03 PM
                                                                        4K sectorsDaniel09/22/08 10:31 PM
                                                                      Reasons for >512 byte sectorsDoug Siebert09/22/08 09:38 PM
                                                                        Reasons for >512 byte sectorsrwessel09/22/08 10:09 PM
                                                                          Reasons for >512 byte sectorsHoward Chu09/23/08 02:50 AM
                                                                        Reasons for >512 byte sectorsDaniel09/22/08 10:40 PM
                                                                          Reasons for >512 byte sectorsrwessel09/23/08 09:11 AM
                                                                            Reasons for >512 byte sectorsDaniel09/23/08 12:10 PM
                                                                      HDD long sector size availabilityEtienne Lehnart09/23/08 05:32 AM
                                                                        HDD long sector size availabilityrwessel09/23/08 09:19 AM
                                                                          HDD long sector size availabilityEtienne Lehnart09/23/08 02:17 PM
                                                                    To SSD or not? Real data..Jouni Osmala09/22/08 11:16 PM
                                                                  To SSD or not? One lady's perspectiveWes Felter09/22/08 11:25 AM
                                                                How should SSDs be engineered into systems?Rob Thorpe09/22/08 02:01 PM
                                                                  How should SSDs be engineered into systems?Matt Craighead09/23/08 06:59 PM
                                                                    How should SSDs be engineered into systems?Matt Sayler09/24/08 04:17 AM
                                                            ATA/SCSIS, Write Flushes and Asych FilesystemsTruePath01/25/09 04:44 AM
                                                        SLC vs. MLC - the trick to latencyMichael S09/12/08 04:58 AM
                                                        overlapped erase and read Michael S09/12/08 04:59 AM
                                                          overlapped erase and read David W. Hess09/12/08 09:56 AM
                                                          overlapped erase and read Anonymous09/12/08 08:45 PM
                                                            overlapped erase and read Jouni Osmala09/12/08 11:56 PM
                                                            overlapped erase and read Michael S09/13/08 11:29 AM
                                                            overlapped erase and read Michael S09/13/08 12:09 PM
                                                              overlapped erase and read Linus Torvalds09/13/08 02:05 PM
                                                    SLC vs. MLC - the trick to latencyDoug Siebert09/11/08 05:31 PM
                                SLC vs. MLCEduardoS09/08/08 02:07 PM
                                  SLC vs. MLCLinus Torvalds09/08/08 02:30 PM
                                    SLC vs. MLCEduardoS09/08/08 04:01 PM
                                      SSD and RAIDJoe Chang09/08/08 07:42 PM
                                        SSD and RAIDDoug Siebert09/08/08 09:46 PM
                                          SSD and RAIDAaron Spink09/09/08 04:27 PM
                                            SSD and RAIDGroo09/10/08 01:02 PM
                              SLC vs. MLCJoern Engel01/06/09 10:22 AM
                                SLC vs. MLCLinus Torvalds01/06/09 02:04 PM
                                  SLC vs. MLCJoern Engel01/06/09 03:24 PM
                                    SLC vs. MLCrwessel01/06/09 04:47 PM
                                      SLC vs. MLCanonymous01/06/09 05:17 PM
                                        SLC vs. MLCrwessel01/06/09 05:58 PM
                                      SLC vs. MLCJoern Engel01/07/09 12:35 AM
                                    SLC vs. MLCLinus Torvalds01/06/09 05:45 PM
                                      SLC vs. MLCrwessel01/06/09 06:09 PM
                                        SLC vs. MLCLinus Torvalds01/06/09 07:47 PM
                                      SLC vs. MLCJoern Engel01/07/09 12:26 AM
                                    SLC vs. MLCanon01/06/09 08:23 PM
                                      SLC vs. MLCJoern Engel01/07/09 12:52 AM
                                        SLC vs. MLCanon01/07/09 02:34 AM
                                  SLC vs. MLCIntelUser200001/07/09 07:43 AM
                                    SLC vs. MLCLinus Torvalds01/07/09 10:28 AM
                                      drop data filesystem semanticDoug Siebert01/09/09 12:21 PM
                                      FTL and FSiz01/09/09 07:49 PM
                                        FTL and FSLinus Torvalds01/09/09 09:53 PM
                                          FTL and FSiz01/10/09 02:09 AM
                                            FTL and FSMichael S01/10/09 03:19 PM
                                              compiling large programsiz01/10/09 05:51 PM
                                                compiling large programsLinus Torvalds01/10/09 07:58 PM
                                                  compiling large programspeter01/11/09 05:30 AM
                                                  compiling large programsAndi Kleen01/11/09 01:03 PM
                                                  The File AbstractionTruePath01/25/09 06:45 AM
                                                    The File AbstractionHoward Chu01/25/09 01:49 PM
                                                      The File AbstractionLinus Torvalds01/26/09 09:23 AM
                                                        The File AbstractionMichael S01/26/09 01:39 PM
                                                          The File AbstractionLinus Torvalds01/26/09 02:31 PM
                                                            The File AbstractionDean Kent01/26/09 03:06 PM
                                                              The File AbstractionLinus Torvalds01/26/09 04:29 PM
                                                                The File AbstractionMark Christiansen01/27/09 09:24 AM
                                                                  The File AbstractionMark Christiansen01/27/09 10:14 AM
                                                                  The File AbstractionLinus Torvalds01/27/09 10:15 AM
                                                                    The File Abstractionslacker01/27/09 11:20 AM
                                                                      The File AbstractionLinus Torvalds01/27/09 01:16 PM
                                                                        Attributes All The Way DownMark Christiansen01/27/09 02:17 PM
                                                                        The File Abstractionslacker01/27/09 05:25 PM
                                                                          The File AbstractionLinus Torvalds01/28/09 08:17 AM
                                                                            The File Abstraction: API thoughtsCarlie Coats01/28/09 09:35 AM
                                                                            The File Abstractionslacker01/28/09 10:09 AM
                                                                              The File AbstractionLinus Torvalds01/28/09 01:44 PM
                                                                                Programs already 'hide' their metadata in the bytestream, unbeknownst to usersanon01/28/09 09:28 PM
                                                                                The File Abstractionslacker01/29/09 10:39 AM
                                                                                  The File AbstractionLinus Torvalds01/29/09 11:08 AM
                                                                                    The File AbstractionDean Kent01/29/09 11:49 AM
                                                                                      The File AbstractionHoward Chu01/29/09 02:58 PM
                                                                                        The File Abstractionrwessel01/29/09 04:23 PM
                                                                                    Extended Attributes in Actionslacker01/29/09 03:05 PM
                                                                                      Extended Attributes in Actionstubar01/29/09 04:49 PM
                                                                                      Extended Attributes in ActionLinus Torvalds01/29/09 05:15 PM
                                                                                        Like Duhanon01/29/09 07:42 PM
                                                                                          Like Duhanon01/29/09 09:15 PM
                                                                                            Like Duhanon02/01/09 07:18 PM
                                                                                              Double Duh.Anonymous02/01/09 10:58 PM
                                                                                                Double Duh.anon02/02/09 02:08 AM
                                                                                                  Double Duh.Anonymous02/02/09 05:11 PM
                                                                                                    Double Duh.anon02/02/09 07:33 PM
                                                                                              Like DuhDavid Kanter02/01/09 11:05 PM
                                                                                                Like Duhpeter02/01/09 11:55 PM
                                                                                                Like Duhanon02/02/09 01:55 AM
                                                                                                Xattrs, Solar power, regulation and politicsRob Thorpe02/02/09 04:36 AM
                                                                                                  Terminology seems too fuzzy to mehobold02/02/09 06:14 AM
                                                                                                    Terminology seems too fuzzy to merwessel02/02/09 12:33 PM
                                                                                                      good summaryMichael S02/03/09 02:41 AM
                                                                                                        good summaryMark Christiansen02/03/09 09:57 AM
                                                                                                          good summaryHoward Chu02/03/09 10:21 AM
                                                                                                            good summaryMark Christiansen02/03/09 11:18 AM
                                                                                                              good summaryHoward Chu02/03/09 12:00 PM
                                                                                                                good summaryMark Christiansen02/03/09 12:36 PM
                                                                                                          good summaryRagingDragon02/03/09 10:39 PM
                                                                                                            good summaryrwessel02/03/09 11:03 PM
                                                                                                              good summaryRagingDragon02/03/09 11:46 PM
                                                                                                      Terminology seems too fuzzy to meslacker02/04/09 05:06 PM
                                                                                                        Terminology seems too fuzzy to meMichael S02/05/09 01:05 AM
                                                                                                          Terminology seems too fuzzy to meUngo02/05/09 01:15 PM
                                                                                                          Terminology seems too fuzzy to meslacker02/05/09 02:19 PM
                                                                                                            Terminology seems too fuzzy to meHoward Chu02/05/09 04:44 PM
                                                                                          Like Duhiz01/30/09 02:03 AM
                                                                                          EAs (security labels) hosed me badlyanon01/30/09 09:48 PM
                                                                                        Extended Attributes in ActionRagingDragon01/29/09 09:31 PM
                                                                                      Extended Attributes in Actionanonymous01/29/09 08:13 PM
                                                                                      Extended Attributes in ActionHoward Chu01/29/09 09:38 PM
                                                                                        Extended Attributes in Actionslacker01/30/09 11:24 AM
                                                                                          Extended Attributes in Actionanon01/30/09 05:50 PM
                                                                                      Extended Attributes in ActionEtienne Lehnart01/30/09 12:22 AM
                                                                                      Extended Attributes in ActionRob Thorpe01/30/09 12:39 PM
                                                                                        Extended Attributes in Actionslacker01/30/09 01:16 PM
                                                                                          Extended Attributes in Actionanon01/30/09 06:03 PM
                                                                                          Extended Attributes in ActionHoward Chu01/30/09 11:22 PM
                                                                                            Extended Attributes in Actionrwessel01/31/09 12:08 AM
                                                                                              Extended Attributes in Actionanonymous01/31/09 12:22 AM
                                                                                                Extended Attributes in Actionrwessel01/31/09 12:56 AM
                                                                                                  ScalingDean Kent01/31/09 09:04 AM
                                                                                                    ScalingRob Thorpe02/02/09 02:39 AM
                                                                                                      Scalingrwessel02/02/09 11:41 AM
                                                                                                        ScalingHoward Chu02/02/09 12:30 PM
                                                                                                          ScalingDean Kent02/02/09 02:27 PM
                                                                                                            ScalingRob Thorpe02/03/09 05:08 AM
                                                                                                              ScalingDean Kent02/03/09 07:38 AM
                                                                                                              Scalingrwessel02/03/09 02:34 PM
                                                                                                                ScalingRagingDragon02/03/09 10:46 PM
                                                                                                    in defense of software that does not scaleMatt Sayler02/03/09 11:27 AM
                                                                                                      in defense of software that does not scaleHoward Chu02/03/09 12:03 PM
                                                                                                        in defense of software that does not scaleMatt Sayler02/03/09 12:17 PM
                                                                                                        in defense of software that does not scaleRagingDragon02/03/09 11:00 PM
                                                                                                          in defense of software that does not scaleMichael S02/04/09 06:46 AM
                                                                                                            in defense of software that does not scaleRagingDragon02/04/09 09:33 PM
                                                                                                      in defense of software that does not scaleDean Kent02/03/09 12:17 PM
                                                                                                        in defense of software that does not scaleMatt Sayler02/03/09 12:24 PM
                                                                                                        in defense of software that does not scaleVincent Diepeveen02/04/09 10:43 AM
                                                                                                      in defense of software that does not scalerwessel02/03/09 02:44 PM
                                                                                                        in defense of software that does not scaleanon02/04/09 02:35 AM
                                                                                                          in defense of software that does not scaleCarlie Coats02/04/09 05:24 AM
                                                                                                      Scaling with time vs. scaling from the beginning.mpx02/05/09 01:57 AM
                                                                                                  Extended Attributes in ActionMichael S01/31/09 10:33 AM
                                                                                                  Extended Attributes in Actionanon01/31/09 10:37 PM
                                                                                          Extended Attributes in ActionJasonB01/31/09 08:11 AM
                                                                                            Extended Attributes in ActionHoward Chu01/31/09 11:43 AM
                                                                                              Extended Attributes in ActionJasonB01/31/09 04:37 PM
                                                                                                Extended Attributes in ActionHoward Chu02/02/09 02:42 PM
                                                                                                  Extended Attributes in ActionHoward Chu02/02/09 02:44 PM
                                                                    The File AbstractionRob Thorpe01/27/09 11:20 AM
                                                              The File AbstractionHoward Chu01/27/09 12:28 AM
                                                                The File AbstractionMichael S01/27/09 03:00 AM
                                                                The File AbstractionDean Kent01/27/09 08:30 AM
                                                        The File AbstractionAndi Kleen01/27/09 02:05 AM
                                  SLC vs. MLCMichel01/12/09 06:54 PM
                                    SLC vs. MLCLinus Torvalds01/12/09 07:38 PM
                                      SLC vs. MLCrwessel01/13/09 12:52 AM
                                      SLC vs. MLCUngo01/13/09 03:04 PM
                                        SLC vs. MLCWes Felter01/13/09 05:42 PM
                                      SLC vs. MLCTruePath01/25/09 05:05 AM
                    SLC vs. MLCUngo08/21/08 12:54 PM
                    SLC vs. MLCAaron Spink08/21/08 01:20 PM
                  MLC vs. SLCMichael S08/21/08 08:57 AM
                First Dunnington benchmark resultsrwessel08/21/08 10:40 AM
              First Dunnington benchmark resultsAaron Spink08/21/08 03:18 AM
      First Dunnington benchmark resultsEtienne Lehnart08/20/08 04:38 AM
  Will x86 dominate big iron?Tom W08/19/08 10:10 PM
    Will x86 dominate big iron?Jesper Frimann08/20/08 12:28 AM
      Will x86 dominate big iron?Tom W08/20/08 03:42 PM
        Will x86 dominate big iron?David Kanter08/21/08 01:13 AM
          Will x86 dominate big iron?Joe Chang08/21/08 06:54 AM
            Will x86 dominate big iron?asdf08/22/08 01:18 PM
              Will x86 dominate big iron?Dean Kent08/22/08 07:54 PM
        Will x86 dominate big iron?Jesper Frimann08/22/08 09:48 AM
          Will x86 dominate big iron?Tom W08/24/08 01:06 AM
            Will x86 dominate big iron?Michael S08/24/08 04:19 AM
              Will x86 dominate big iron?Dean Kent08/24/08 09:30 AM
                Will x86 dominate big iron?Paul08/24/08 11:16 AM
                  Will x86 dominate big iron?Dean Kent08/24/08 12:37 PM
                Will x86 dominate big iron?Michael S08/25/08 12:53 AM
        Will x86 dominate big iron?someone08/22/08 10:19 AM
          Will x86 dominate big iron?aaron spink08/23/08 02:56 AM
            Will x86 dominate big iron?Michael S08/23/08 09:58 AM
              Will x86 dominate big iron?someone08/23/08 01:51 PM
            Will x86 dominate big iron?someone08/23/08 01:55 PM
              Will x86 dominate big iron?Aaron Spink08/23/08 04:52 PM
                Will x86 dominate big iron?anonymous08/23/08 05:28 PM
                  Will x86 dominate big iron?Dean Kent08/23/08 06:12 PM
                    Off road and topicEduardoS08/23/08 06:28 PM
                  Will x86 dominate big iron?someone08/23/08 06:26 PM
                    Will x86 dominate big iron?Dean Kent08/23/08 09:40 PM
                    Will x86 dominate big iron?anonymous08/24/08 01:46 AM
                  Off road and topicDavid W. Hess08/24/08 03:24 AM
                    Off road and topicAaron Spink08/24/08 04:14 AM
  Beckton vs. DunningtonMr. Camel08/22/08 06:30 AM
    Beckton vs. Dunningtonjokerman08/22/08 12:12 PM
    Beckton vs. DunningtonMr. Camel05/29/09 10:16 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?