Will Storage Go the Way of The Server?

1. Summary

The storage industry is at on the cusp of the biggest structural change since networked storage (including SAN, NAS, and more recently iSCSI) began to substitute for direct-attached storage a decade ago. Despite being one of the fastest growth sectors in technology in terms of capacity, the economics for many participants are deteriorating. Several major technology and business model shifts will re-define the profit pools in the industry, leading to slimmer margins for all but the most innovative, software-driven players.

The long-term future of storage is about smart software that manages a large pool of cheap interchangeable hardware. However, in the near term, mainstream enterprise buyers continue to move cautiously while upgrading their existing installed base mostly with more of the same from vendors such as EMC and NetApp. But the current recession is making them more price-sensitive and creating pressure to try technology from newer vendors such as 3PAR and Data Domain for growing pockets of use cases. Cloud/online service providers are the most price-sensitive and open to new approaches since their storage capital and operating expenditures have a direct impact on their ability to offer competitive pricing.

Customers are transitioning from storage typically bought for a specific application to a more horizontal, virtual pool that better matches the shared resource model of their virtual servers. Much of the growth is occurring in two customer archetypes that are very different from the legacy enterprise data center characterized by scale-up architectures.

Scale up describes a model of using bigger, more expensive machines, whether for storage or servers, to get more performance. The scale-out model is when many smaller, inexpensive machines working together are used for better performance. Just about all enterprise software is written to scale up while cloud-based software is primarily written to scale out.

• Many new high-growth workloads in the enterprise are best handled as NAS-based file-oriented data, as opposed to highly structured SAN-based block-oriented data. They include web serving, film and animation processing for media companies, seismic modeling for oil exploration, financial simulation in banking, etc. These workloads generate so much data that customers have been willing to try newer vendors with less expensive scale-out architectures such as Isilon.
• The very largest cloud/online service providers, such as Google, Yahoo, Amazon Web Services and Microsoft tend to build their own scale-out storage software to run on commodity storage hardware. This do-it-yourself model is an extreme example of what analysts are referring to when they say storage will become as commoditized as servers.

Storage technology is morphing in the direction of server technology, more slowly in the enterprise and faster in the cloud.

• Server virtualization is putting a layer in front of storage that over the next several years will start to homogenize the differences between storage products for applications and administrators.
• As modular or commodity storage manages more workloads, the storage software can sit either on the x86 controller or the x86 server. That will make it easier for customers to benchmark and put pressure on hardware prices, even if the software comes from the same storage vendor providing the controller and disk drives.
• Customers are rolling out storage efficiency functionality that improves utilization similar to server virtualization. However, customers are using technologies such as snapshots, thin provisioning and de-duplication to accelerate data growth, particularly backup or nearline, while only modestly compressing their storage budgets.
• Flash memory will drastically improve the price/performance for virtually all classes of storage. In particular, over the next several years using flash as cache to augment magnetic disk performance will have a bigger impact than flash-based solid-state disks.

While these last two trends are very significant for the storage industry, we are treating them as outside the scope of this report, which deals with the commoditization of storage in the midst of a transition to virtualization and cloud computing.

Changing customer workloads and emerging technologies are driving changes in vendor business models.

• For all but the most high performance and resilient systems, storage hardware and software will increasingly be sold and priced as two distinct parts of one integrated product line, starting especially with cloud/online service providers. Even though these two components will likely come from the same vendor most of the time, this change will force storage vendors to sell software based on business value rather than systems based on capacity.
• To the extent that customers shift more of their data to the cloud, aggregate industry demand for storage will move from a ‘just in case’ capacity, upfront capex model to a ‘just in time’ capacity, ongoing opex model. This is because online service providers are running at much higher asset utilization than the typical customer can add capacity in more granular increments and are able to extract very favorable pricing from their suppliers. During this transiti0n period, which we can think of as a form of industry-wide thin provisioning coupled with collective bargaining, storage vendors may see a temporary slowdown in revenue growth. More importantly, they may experience lower margins for a prolonged period.
• Truly interchangeable storage software and commodity hardware will likely be limited to the largest cloud / online service providers, such as Google, Yahoo, Amazon and Microsoft. Enterprises lack the scarce talent required to combine third-party or open source storage software with commodity hardware in a way that ensures scalability and resilience. In other words, the ‘mix and match’ model of server hardware and software is not likely to become prevalent in mainstream storage anytime soon.
• The storage vendor mix in traditional enterprises is unlikely to be radically reshuffled anytime soon, since the innovative storage software challengers have to contend with customers’ concerns about interoperability, supportability and resilience. A major OEM endorsement of a startup vendor such as Virsto or Sanbolic would change that dynamic. While Cisco is the most likely vendor to fill the role of disruptor, HP, Dell or IBM might be somewhat more conflicted about accelerating storage hardware commoditization.