Table of Contents
- Summary
- Data Pipeline Primer
- Report Methodology
- Decision Criteria Analysis
- Evaluation Metrics
- Key Criteria: Impact Analysis
- Analyst’s Take
- Methodology
- About Andrew Brust
- About GigaOm
- Copyright
1. Summary
Today’s organizations are grappling with a growing volume of data—data of various types and formats that’s stored in an increasing number of repositories. Somehow, organizations need to ensure the quality and manage the flow of all this data throughout their company so they can successfully derive valid insights from it. Data pipelines offer a solution to this challenge: taking source data, transforming it from its source format into a uniform format suitable for its target system, and loading it into a target database or storage system for downstream consumption and analysis.
Building a data pipeline from scratch is time-consuming and code-intensive, but the market now offers a broad array of solutions for moving and processing data. These products streamline and automate the process from start to finish, facilitating the authoring and creation of pipelines as well as the overall orchestration, movement, transformation, monitoring, and error handling. And many of them do so through no-code interfaces, allowing even data novices to be involved.
Data pipeline solutions have their origins in traditional extract, transform, and load (ETL) tools, but the category has evolved beyond that approach to encompass additional frameworks and types of pipelines. The category is robust, mature, and stable but by no means stagnant. Innovation continues in the form of ever-widening arrays of data source connections, more refined tools for data transformations, platforms that support a variety of deployment scenarios, and improved security features.
This GigaOm Key Criteria report details the capabilities (table stakes, key criteria, and emerging technologies) and non-functional requirements (evaluation metrics) for selecting an effective data pipeline solution. The companion GigaOm Radar report identifies vendors and products that excel in those capabilities and metrics. Together, these reports provide an overview of the category and its underlying technology, identify leading data pipeline offerings, and help decision-makers evaluate these solutions so they can make a more informed investment decision.
How to Read this Report
This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding, consider reviewing the following reports:
Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.
GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.
Solution Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.