Bad data can lead to biased machine learning (ML)-based decisions, damaging brand trust and customer satisfaction. The impact of bad data on organizations ranges from revenue and productivity loss to system outages and missed opportunities.

That is where data quality platforms can help. By ensuring that data is more reliable, enterprise data-quality initiatives can improve operational efficiencies, employee productivity, customer satisfaction, and revenue.

This Radar report will help enterprise buyers become familiar with vendors offering data quality platforms focused on the detection of data quality issues. Meaning, the solutions do not remediate data quality problems, but they actively monitor and validate data quality and let users know when it’s compromised. Most of the platforms monitor data right after it is ingested; therefore, they can prevent compromised data from reaching target systems and consequently causing business problems. Some platforms also offer root-cause diagnostics to guide users to the specific source data responsible for detected quality issues.

All the platforms reviewed in this report provide a unified environment for data quality validation and monitoring. Platforms in this report can work well for both small and large companies whose day-to-day operations depend heavily on data and its applications.

The post GigaOm Radar for Data Quality Platforms: Detection of Data Quality Issues appeared first on Gigaom.

Most organizations depend heavily on data to guide their decisions. As data volumes increase in general, the volume of bad data—in other words, data with low quality—increases commensurately. Businesses of every size, but especially at the enterprise level, can benefit significantly from a data quality platform that not only detects data quality issues but also takes charge and remediates them, aided by automation.

Data remediation can be implemented with a variety of techniques, such as data cleansing, standardization, labeling, parsing, and deduplication. These are typically out-of-the-box capabilities, but are configurable based on business needs. Data quality remediation capabilities have multiple benefits and can expedite the achievement of a high level of data quality overall. Since data quality assessment is needed to offer remediation, all vendors in this Radar report also provide capabilities that help assess data and validate its quality.

Data quality detection and remediation can be viewed as part of a data project’s workflow, with the objective of detecting and addressing data quality issues preemptively, before low-quality data ends up in destination systems.

All vendors reviewed in this Radar report provide a unified environment for data quality detection and remediation. Unlike vendors that focus on detection only, these platforms invest heavily in helping users solve data quality issues faster. The vendor platforms highlighted in this report can work well for businesses that want to leverage technologies whose efficacy hinges on data quality, including internet of things (IoT), artificial intelligence (AI), and automation.

The post GigaOm Radar for Data Quality Platforms: Data Quality Remediation appeared first on Gigaom.

Poor data quality has many causes, including the use of multiple databases that don’t integrate well, formatting issues, and the use of obsolete data. This is a non-trivial concern because data quality can have a significant social and economic impact on a business. Inaccurate analyses that result from low-quality data may lead to suboptimal data-driven decisions and eventually harm organizations and their customer/client relations. Poor data quality can also damage an organization’s reputation and even thwart checks for sensitive data. To avoid these pitfalls, companies need to commit to an ongoing assessment of data quality.

To help organizations perform data quality assessments, vendors offer tools to automate data quality checks, provide validation tests, and mitigate the adverse effect of poor data quality. Offerings typically include reporting and statistical summaries, alerts and notifications about data quality lapses, issuance of diagnoses with root cause explanations, and machine learning (ML) capabilities.

The GigaOm Key Criteria report identifies key criteria and evaluation metrics for selecting an effective data quality platform, and gives an overview of the past and future of the data quality space. This report, along with the companion Radar report, will help decision-makers gain a better understanding of the data quality domain so that they can effectively evaluate existing platforms and decide where to invest.

The post Key Criteria for Evaluating Data Quality Platforms appeared first on Gigaom.

There are three types of vendors reviewed in this Radar: cloud providers, data platform vendors, and enterprise incumbent players, all of whom offer a broad range of data and analytics-focused platforms and services. That focus makes them suitable for large organizations that would like to take advantage of a vendor with broad offerings that focus not only on data science but also on other aspects of data handling and analytics.

Because of this versatility, choosing one of these vendors may enrich capabilities within different steps of the data science workflow. For example, a vendor that offers tight integration with storage and databases can provide substantial support in the data ingestion stage. Vendors that offer broad capabilities focused on management can help users greatly in the production stage. And vendors that offer thorough data governance capabilities can better support the monitoring stage.

The main goal of data science platforms is to provide a unified environment for managing all data-science-related activities. The platforms in this Radar report provide multiple capabilities without the need for complex integrations, unlike specialist vendors who focus on data science and offer additional capabilities through third-party integrations.

The vendors reviewed in this Radar report are best suited for companies with a range of data and analytics needs, of which data science is an important but not necessarily the dominant part.

The post GigaOm Radar for Data Science Platforms: Cloud Providers, Data Platform Vendors, and Enterprise Incumbents appeared first on Gigaom.

The vendors reviewed in this Radar report offer a data science-centric view of data and analytics. In other words, data science is not just one capability among many others but rather is the central focus of the product. For companies with a strong focus on data science use cases—including financial services, retail, manufacturing, healthcare, and hospitality—these data science platform vendors may provide the optimal fit. Specialist vendors in this report will work especially well for smaller companies, startups, or business units within large enterprises whose functions revolve around data science use cases.

Note: We have included some products in this report that could strictly be classified as machine learning operations (MLOps) or automated machine learning (AutoML) platforms rather than broad data science platforms. However, the three categories overlap considerably, and the particular platforms we’ve included here all serve as broad data science platforms in their own right.

The post GigaOm Radar for Data Science Platforms: Pure-Play Specialist and Startup Vendors appeared first on Gigaom.

The vendors reviewed in this Radar are streaming specialists and offer something that vendors with broader platforms cannot: a streaming-centric view of data and analytics. From that unique vantage point, streaming isn’t just an option, but rather the primary mode of data ingestion, such that even a batch data modality can fit within it. Data streams are the ingest mechanism and often the egress target as well. Event processing and messaging constitute the data sharing medium. Streams can be used even for data storage rather than just for data in transit. Streaming is a mindset and a paradigm.

For companies heavily involved with businesses or operational processes that revolve around streaming data use cases, including Internet of Things (IoT), wWeb analytics, financial markets, and online advertising, this “streaming-first” approach to data and analytics is a perfect fit. Many such customers will be modern tech companies, startups, or business units within large enterprises. For them, specialist companies will be very well-aligned philosophically with their own collection and use of data, and the primacy of it.

Conversely, streaming solutions from cloud vendors and enterprise software players may not be the best match for the customers just described, even if the streaming data platforms offered by those vendors meet their needs for specific capabilities. For those vendors, streaming data is part of their fabric and likely a high priority, but it’s not the dominant technology or even first among equals.

The post GigaOm Radar for Streaming Data Platforms: Streaming Data Specialist Vendors appeared first on Gigaom.

The vendors reviewed in this Radar are cloud providers and incumbent enterprise software players who offer a broad range of data, analytics, and other capabilities for enterprise customers. All the platforms in this report are extremely robust and would compete head-to-head with the specialist vendors if considered on a standalone basis. But what makes them different is that they also have substantial interlock with other components in their vendor’s respective broader platforms. For cloud vendors, this means tight integration with their storage, database, and possibly application platforms. For data platform vendors, it means seamless interfaces with their machine learning and data lake components, as well as their management plane. And for data management vendors, it means tight interplay with their data catalog, data governance, and data quality components.

Yes, specialist vendors offer similar integrations with third-party vendor platforms— including those from the enterprise vendors in this report. But while those integrations make a best-of-breed strategy easier, it’s not the same as using a full stack from a single vendor. The integration will be more complex, as will vendor management and procurement. Furthermore, working with specialist vendors can add expense or inconvenience for customers who have enterprise agreements with large players or have contracted for cloud storage and compute capacity that must be “retired” over a certain period (usually annually).

Specialist vendors’ platforms will work especially well for smaller companies, tech startups, or business units within enterprises that have a streaming-centric mission—that is, their function revolves around canonical streaming use cases—or at least overlaps significantly with them. For companies that have a range of things they do with data, of which streaming is an important but not dominant part, the vendor platforms in this report may be a better fit.

The post GigaOm Radar for Streaming Data Platforms: Cloud Providers, Enterprise Incumbents, and Data Platform Vendors appeared first on Gigaom.

The biggest challenges companies face in leveraging data science are the relatively small number of trained data scientists and the historically ad hoc, manual approach involved in the work. For example, data scientists have traditionally conducted data exploration and model training and optimization using their own tools, on their own computers, with relatively little tracking, consistency, or collaboration and reuse of code.

The steps involved in building optimal ML models—for example, feature engineering, testing different hyperparameter values, and building multiple candidate models—are quite time-consuming, especially when done manually. Pressure to produce models quickly can thus short-circuit the optimization work, resulting in less-accurate models.

This is where data science platforms come in. These platforms provide tools to serve the end-to-end data science lifecycle (including data preparation, training, testing, and deploying ML models) so that data scientists can focus on building better models rather than the “plumbing” of ML work.

As massive volumes of data continue to be processed, streamlining the data science workflow by accelerating model development and deployment becomes an imperative of increasing urgency. And while important open-source technologies exist in this arena, commercial data science platforms supply the fit-and-finished end-to-end solutions needed to provide the required efficiency gains.

When evaluating vendor offerings, decision-makers should consider their company needs, goals, budget, and employee skill sets.

This GigaOm Key Criteria report details the criteria and evaluation metrics for selecting an effective data science platform. The companion GigaOm Radar reports identify vendors and products that excel in those criteria and metrics. Together, these reports provide an overview of the category and its underlying technology, identify key data science platform offerings, and help decision-makers evaluate existing platforms to help them decide where to invest.

The post Key Criteria for Evaluating Data Science Platforms appeared first on Gigaom.

In recent years, the need to process streaming data in-flight from sensors, log files, Web applications, and other sources has been gaining importance. This is due to the increased adoption of IoT devices, development of 5G networks, and the large amount of data “exhaust” that is generated by user activity and interaction with online and mobile platforms.

These seemingly never-ending data streams drive new business requirements to process data, resulting in increased momentum away from batch-only processing to a mix of batch and ever-more common stream processing. This trend is especially strong for specific industries and use cases.

Streaming data platforms enable users to combine and process real-time data from multiple sources. They aid processes like data ingestion, data enrichment, data classification, and prediction. In return, they increase the productivity level of users by assisting them in iterative tasks.

Yes, there are open-source solutions for handling these workloads. However, full streaming data platforms provide fully-managed, unified solutions for the ingest and processing of real-time data. Users of these platforms are empowered to make better-informed business decisions and reach deeper data-driven insights than would be possible with batch data processing alone.

When evaluating vendor offerings, it’s essential to know what to look for and what to expect from a streaming data platform. Decision makers should also consider their company needs, goals, budget, and their employees’ skill sets.

This key criteria report will help enterprise buyers become familiar with streaming data platforms and vendor offerings.

The post Key Criteria for Evaluating Streaming Data Platforms appeared first on Gigaom.