• Total test run time of around 16 hours in this order:
• Sequential preconditioning (~2:15)
• Sequential tests on 16x JBOD, 2x 8DR10, 2x 8DR5 (~2 hours)
• Random preconditioning – 2 parts (~4:30)
• Random optimal tests on 16x JBOD, 2x 8DR10, 2x 8DR5 (~3 hours)
• Random rebuild tests on 1x 16DR10, 1x 16DR5 (~2:30)
• Random write latency for optimal and rebuild for 1x 16DR5 (~1:40)

The first section of performance metrics focuses on the bandwidth through the card in JBOD, RAID10, and RAID5 modes. With the MegaRAID 9670W-16i offering a x16 PCIe Gen4 slot width, its peak performance will be right around 28GB/s in one direction, and that is where the Gen4 slot tops out. By comparison, a U.2 Gen4 SSD connects through a x4 connection and can peak out around 7GB/s, and this is where most enterprise drives can top out for read workloads.

With that said, the MegaRAID 9670W completely saturates the slot it’s connected to. When looking at the read performance, the JBOD configuration comes in with 28.3GB/s with RAID10, and RAID5 comes in a hair below it measuring 28GB/s. When we switch our focus to write performance, the JBOD baseline is 26.7GB/s, while the RAID10 configuration came in with 10.1GB/s and RAID5 at 13.2GB/s. When we look at a 50:50 split of simultaneous read and write traffic, the JBOD configuration measured 41.6GB/s, RAID10 at 19.6GB/s, and RAID5 at 25.8GB/s.

When switching our focus to small-block random transfers, we see the MegaRAID 9670W held up quite well in read performance versus the JBOD baseline figure of 7M IOPS. This speed dropped to about half (3.2M IOPS) during a rebuild operation if one SSD failed in the RAID group. Looking at random write performance, the JBOD baseline measured 6.3M IOPS against 2.2M from RAID10 and 1M from RAID5. Those figures didn’t see a considerable drop when an SSD was failed from the group and the RAID card was forced to rebuild. In that situation, RAID10 didn’t change, although RAID5 dropped from 1M to 788k IOPS.

In the 4K OLTP workload with a mixture of read and write performance, the JBOD baseline measured 7.8M IOPS against RAID10 with 5.6M IOPS and RAID5 with 2.8M IOPS. During a rebuild, RAID10 dropped from 5.6M to 2.4M IOPS, and RAID5 dropped from 2.8M to 1.8M IOPS.

Another important aspect of RAID performance is how well the storage behaves between optimal conditions and rebuild performance if a drive fails. If performance or latency were to take a massive hit, application responsiveness can become a problem. To that end we focused on RAID5 4K random write latency in optimal and rebuilding modes. Across the spectrum, latency remained quite similar, which is exactly what you want to see in a production environment storage system.

We not only evaluated the overall performance of each mode through point-of-time performance metrics, which also included the performance of the RAID card during a rebuild operation, but we also conducted tests to determine the total time it took to rebuild. Here in RAID10, dropping a 6.4TB SSD out of the RAID group and adding it back in took 60.7 minutes for RAID10 with a rebuild speed of 10.4 Min/TB. The RAID5 group took 82.3 minutes with a speed of 14.1 Min/TB.

Final Thoughts

To be honest, we came into this review with heads slightly cocked and one eyebrow raised. We haven’t heard a RAID card pitch for NVMe SSDs in a while, outside of the emerging class of solutions designed around GPUs. So we did have to ask the fundamental question, can hardware RAID even be a thing for NVMe SSDs?