This article is a REPOST from Jeffrey B. Layton writing in LINUX MAGAZINE. I had to edit out some of the images/graphics to get this up quickly — apologies — use the link for the complete story.
SC09 TREND: High-Density Flash Storage
Article by Jeffrey B. Layton
There was a past article about really fast storage (Ramdisks – Now We are Talking Hyperspace!) but flash based storage units are becoming more popular. For dense flash based storage units, the performance density (IOPS/U or Throughput/U, where “U” is the common rack unit measure – 1.75″) can be better than DRAM based storage units, especially when price is considered. This is particularly true for IOPS driven applications.
There are several vendors who offer Flash Storage units – Texas Memory, Violin Memory, and now Sun are offering flash based storage units with performance that is amazing.
The basic concept is that you take a standard rack unit, stuff them full of flash units (drives or flash based DIMMs), possibly combine them with RAID controller(s), and put FC, or IB connections, or direct connect via a PCIe cable to a host node, and you have a storage unit with very high performance. It sounds simple but it is actually more difficult than you think.
Texas Memory
Texas Memory has been making very high-speed storage units for a number of years. Originally they focused more on DRAM based solutions but in the last few years they have been offering flash based storage devices as well. The latest unit is the RamSan-620.
The RamSan-620 is a 2U unit that has a capacity of 1-5 TB of SLC Flash (the good but more expensive kind of flash) and uses only 230W of power. It has two 4 Gb FC connections in the back of the box with InfiniBand connections and 10GigE connections coming soon. It uses super capacitors in case of power loss so the DRAM on the flash chips can be flushed to the flash storage. It does have some really good management features including the ability to use 512 byte blocks rather than the standard 4KB blocks (if operating systems and file systems can make use of it). It also allows you to carve the storage into 1 to 1,024 LUNs with variable capacity in each LUN and the ability to assign LUNs to specific ports.
However, the really cool aspect of the unit is the performance. It is rated at up to 3 GB/s throughput and 250,000 random IOPS. Given that a hard drive can perform between 100-200 IOPS, this kind of performance is short of remarkable. An equivalent number of hard drives to match the IOPS capability is in the range of 1,250-2,500.
Violin Memory
Violin Memory is selling a high performance storage unit, the Violin 1010, that can use either DRAM or Flash based storage units. The basic 2U unit can accommodate FC and Ethernet connections via a “network head” or it can be attached directly to a node via a PCIe interface (dual x4 and x8 interfaces). The Flash based version of the 1010 has a raw capacity of up to 4 TB of SLC flash units. The 1010 uses 62 Violin Intelligent Memory Modules (VIMMs) that come in either 32GB or 64GB capacities.
Like the Texas Memory unit, the Violin Memory 1010 has fantastic performance. For a fully loaded unit with 64GB VIMM’s, the 1010 has a performance of 345,000 4K Read IOPS and 219,000 4K Write IOPS. These numbers are assuming a x8 PCIe connector. A x4 PCIe connection will have a performance of 215,000 4K Read IOPS and 145,000 4K Write IOPS. The sustained read throughput performance is just a little above 1.4 GB/s, a peak write of about 1 GB/s, and a sustained random write performance of about 850 MB/s.
Sun F5100
Recently, Sun introduced a totally flash based storage unit, the Sun F5100. It is a 1U box that has 20, 40, or 80 SO-DIMM based SLC Flash Modules (FMOD’s). Each FMOD currently has 24GB of usable capacity but is really a 36GB flash module (the extra space is used for over provisioning). Figure 2 below, courtesy of Robin Harris at StorageMojo, shows the SO-DIMM based flash unit.
The FMODs have a small amount of DRAM on-board, so the F5100 has up to four very large capacitors called Energy Storage Modules (they look like the drives in the front of the unit). These capacitors have enough power to allow the DRAM’s to flush their data to the flash storage.
Each of the FMOD’s shows up as a block device to the OS. For ZFS, this isn’t such a big deal since it can use the drives individually and manage them as part of the overall storage pool. For something like Linux you would have to use md to manage all of the FMOD’s to create a RAID group.
The price for the entry level unit is also very attractive. According to several articles the price point for 20 FMODs (480 GB raw capacity) is $45,995 (397,000 IOPS read and 304,000 IOPS write). Moreover, the entire unit only uses about 300W of power (less than your desktop system). For that level of IOPS performance and power consumption, the price is very, very attractive.
General Session with Stanley Zaffos. Zaffos is a vice president and research director in Gartner Research. His major areas of responsibility include storage and storage management software. A flurry of SSD-related product announcements coupled with rapid SSD price declines and a limited understanding of the operational issues surrounding the use of SSDs in complex server environments has created uncertainty about how best to deploy storage systems that incorporate SSDs in user environments. Understanding the following key issues will increase the probability of their effective use in your environment. Topics include:
Posted by stevenjsattler 
Posted by stevenjsattler 
Posted by stevenjsattler 
