Magic Quadrant for Data Integration Tools

A substantial number of the offerings in the data integration tools market are achieving a comprehensive range of functions, delivering high performance and enabling scalability to fulfill enterprises' data integration needs.

Demand trends have given rise to new challenges in 2011. Fresh needs arising from contemporary challenges are presenting new opportunities in this market as buyers seek to address data integration as a critical aspect of a coherent information management capability, and to integrate disparate data sources (including emerging sources such as "big data") and new data types into a cohesive and usable set of information.

The competitive landscape reflects vendors' pursuit of a more comprehensive offering strategy to support a broad range of uses and capitalize on new demand. Diverse needs to exploit data are forcing enterprises to manage their data assets differently. The "raising of the bar" among IT leaders is demanding synergy between functions, performance and scalability in data integration tools, so that they operate well with the same vendor's technology stack and, increasingly, interoperate with data management initiatives in areas such as data quality, master data management (MDM), metadata management, and can cope with the explosion in the volume, variety and velocity of data.

Business imperatives to confront new information challenges are driving the need for a realignment of technology vision in this market. Meanwhile, IT leaders continue to emphasize cost-effectiveness (perceived value of functionality relative to cost), reduced time-to-value, high-quality customer service and support, and broadening usage (support for data migration, application integration, data services in service-oriented architecture [SOA] initiatives, and cloud-related integration).

Source: Gartner (October 2011)

Market Overview

The discipline of data integration comprises the practices, architectural techniques and tools for achieving consistent access to, and delivery of, data across the spectrum of data subject areas and data structure types in the enterprise, to meet the data consumption requirements of all applications and business processes. As such, data integration is a critical component of an overall enterprise information management (EIM) and information infrastructure strategy. Business drivers, such as the imperative for speed to market, the agility to change business processes and models, and the desire to detect and harness patterns and capture events are forcing organizations to manage their data assets differently. Data integration capabilities are increasingly relied on to enable "frictionless" sharing of data across all organizational and system boundaries, and to support a consistent and complete view of data.

Gartner estimates that the data integration tools market amounted to $1.63 billion at the end of 2010, an increase of 20.5% from 2009. The market continues to demonstrate healthy growth, and we expect a year-on-year increase of approximately 15% in 2011. A projected five-year compound annual growth rate of approximately 11.4% will bring the total to $2.79 billion by 2015 (see "Forecast: Enterprise Software Markets, Worldwide, 2008-2015, 3Q11 Update").

A growing emphasis among vendors on aligning their strategy with what they understand to be the direction of the market, on product strategy and on gaining the adaptability needed to capitalize on new demand have led to some vendors improving their vision. The vision of some vendors, however, has reduced in relative terms. Many of the vendors in this market are evolving toward the Leaders and Visionaries quadrants. Offerings are reaching higher levels of maturity, but at the same time buyers are becoming more ambitious about realigning and revisiting their data integration focus, in addition to obtaining core functions, as requirements for better alignment and interoperability between capabilities grow. Changes in the positioning of vendors in this iteration of the Magic Quadrant have been prompted not only by vendors' activities in delivering new product capabilities and their degree of success in targeting demand, but also by the continuing evolution of their vision.

Offerings Mature, New Challenges Emerge. During the past three to five years, many providers have expanded their support for data integration offerings with more comprehensive data delivery styles, tightened links to data quality tools, and extended their focus toward a model-driven approach that leverages common metadata across their technology portfolio. However, evolving patterns for how data is produced and required are creating new expectations. Data integration tool providers that have taken pride in building and providing functionalities will increasingly be evaluated on their vision and ability to support:

• The integration of extreme information from emerging sources and of new types (such as "big data" and "unstructured" data or "content"); also to combine disparate data sources and new data types into a usable, cohesive set. Enterprises are beginning to seek abilities to exploit big data for use in business decisions and processes. Growing activities among data integration technology providers are seen in tool enhancements to address trends and market demand, and provide early offerings.
• Interoperation of data integration functions with broader data management technologies (in areas such as MDM and data quality). This requires a degree of sophisticated metadata management that still eludes many users of the tools.
• Data governance. Organizations increasingly recognize that both data integration and data quality functions are required in order to support critical initiatives in business intelligence, MDM and application modernization.
• Cloud computing, which is creating new challenges and demand relating to the need to perform data integration across a broader IT landscape, including more seamless operations with data residing in cloud services and data management capabilities in the cloud.
• Tools that can participate in a framework-based approach to addressing information capabilities (see "The Information Capabilities Framework: An Aligned Vision for Information Infrastructure").

Solid Functionality Remains Vital but Buyers Seek More. With the ongoing evolution of the data integration tools market, organizations increasingly acknowledge a diversity of data integration problem types that are supported by equally diverse architectural styles and patterns of data delivery. Organizations will have to deliver data in ways that differ from traditional bulk/batch movement. Rather than merely adding a separate data delivery capability when the business demands it, a data services bus can provide a flexible infrastructure that makes adding new delivery styles easier. Forward-thinking organizations are using principles of service orientation to address their need for consistent, yet flexible, delivery of data. The maturing market for complete data integration tools will need to address a range of different data integration styles, based on common design tooling, metadata and runtime architecture. As this market has supplanted former data integration tool submarkets, such as extraction, transformation and loading (ETL), we anticipate that vendors that support multiple data delivery styles other than bulk data movement, and fulfill the rest of the inclusion criteria, may be eligible for placement in the next iteration of this Magic Quadrant.

Technology Consolidation and Convergence. As convergence with the related market of data quality tools progressed, an emphasis on governance emerged and some data integration practitioners began to emphasize data quality, rather than the mechanics of integration. The increasing maturity of the data integration tools and data quality tools markets, and the rapidly growing overlap between them (in terms of both providers and buyers), signals pending market convergence. Providers with complementary technologies in these two (and other related) markets, such as IBM, Informatica, Oracle, SAS/DataFlux, SAP, iWay Software and Talend, signal a focus on initial synergy by offering data integration tools that operate synergistically with a wider range of data management activities. In addition, the growing demand to address MDM data delivery and data governance issues is increasing the need to extend data integration functions in support of wider data management activities. Vendors limited to a single style of data delivery remain in a weak position as submarkets for various styles of data delivery converge. Consolidation in the data integration tools market and related markets is shifting buyers' preferences toward incumbent providers of application infrastructure or other information management infrastructure. It is also being affected by vendors' actions — specifically, vendors in individual data integration submarkets organically expanding their capabilities into neighboring areas, such as application integration. The intersection of data integration and application integration are being addressed by tools offering capabilities in common areas, such as the introspection of data sources and targets to determine the architecture of the data to be moved, the transformation of data structures and the movement of data.

Diversification of Pricing and Delivery Models, and Accelerating Time-to-Value. Organizations continue to apply lessons learned from the economic challenges of recent years to scrutinize their investments and optimize their costs. Interest in low-cost, rapid-time-to-value and "good enough" data integration capabilities are spurring the emergence of alternative ways of pricing and delivering data integration technologies through open-source and cloud-based models. A growing number of organizations seeking solutions with solid basic capabilities are using open-source data integration tools sold at attractive prices. Data integration platform as a service (PaaS), which operates on a cloud infrastructure and offers "pay per use" pricing, is gaining early adoption. Driven by diversifying business demands, providers of data integration technologies are adapting to make new delivery models available. Providers of cloud applications are beginning to look to simple and quick-to-deploy data integration PaaS as a way to reduce the challenges of onboarding new customers and supporting demand for management of data in the cloud. However, large-scale production deployments of data integration PaaS remain scarce.

Demand for Broader Use Cases to Attain Business Value. Use cases for data integration tools are diversifying as buyers procure tools to support a wide range of projects and initiatives. Although deployment of data integration tools for business intelligence (BI) and data warehousing initiatives remains the most significant use case, many others have emerged to drive up demand. Data migration in support of modernization and consolidation initiatives represents a fast growing area of demand, with data integration tool vendors (and those in the related area of data quality) providing critical infrastructure for these. As MDM programs increase in number and scope, organizations also seek to apply investments in data integration technology to them, since movement, transformation and federation of master data is a fundamental component. Synchronization of data between operational applications and across enterprise boundaries (between trading partners or between on-premises and cloud-based applications) also represents an area of growth. Increasingly, end-user organizations are deploying data integration services beneath and in support of wider SOA initiatives. Data integration requirements have generally been met via point-to-point interfaces supported by data integration tools, but the architectural concept of a "data integration hub" to address these needs is attracting interest (see "Data Integration Hubs: Drivers, Benefits and Challenges of an Increasingly Popular Implementation Approach").

Market Definition/Description

The data integration tools market comprises vendors that offer software products to enable the construction and implementation of data access and data delivery infrastructure for a variety of data integration scenarios, including:

• Data acquisition for BI and data warehousing: Extracting data from operational systems, transforming and merging that data, and delivering it to integrated data structures for analytic purposes. BI and data warehousing remain mainstays of the demand for data integration tools.
• Consolidation and delivery of master data in support of MDM: Enabling the consolidation and rationalization of the data representing critical business entities, such as customers, products and employees. MDM may or may not be subject-based, and data integration tools can be used to build the data consolidation and synchronization processes that are key to success.
• Data migrations/conversions: Although traditionally addressed most often via the custom coding of conversion programs, data integration tools are increasingly addressing the data movement and transformation challenges inherent in the replacement of legacy applications and consolidation efforts during mergers and acquisitions.
• Synchronization of data between operational applications: In a similar concept to each of the previous scenarios, data integration tools provide the ability to ensure database-level consistency across applications, both on an internal and an interenterprise basis (for example, involving data structures for software-as-a-service [SaaS] applications or cloud-resident data sources), and in a bidirectional or unidirectional manner.
• Interenterprise data sharing: Organizations are increasingly required to provide data to, and receive data from, external trading partners (customers, suppliers and others). Data integration tools may be relevant for some of these requirements, which often consist of the same types of data access, transformation and movement components found in other common use cases.
• Delivery of data services in an SOA context: An architectural technique, rather than a use of data integration itself, data services represent an emerging trend for the role and implementation of data integration capabilities within SOAs. Data integration tools will increasingly enable the delivery of many types of data service.

Gartner has defined multiple classes of functional capability that vendors of data integration tools must possess to deliver optimal value to organizations in support of a full range of data integration scenarios:

• Connectivity/adapter capabilities (data source and target support).
• Data delivery capabilities.
• Data transformation capabilities.
• Metadata and data modeling capabilities.
• Design and development environment capabilities.
• Data governance capabilities (interoperation with data quality, profiling and mining capabilities).
• Deployment options and runtime platform capabilities.
• Operations and administration capabilities.
• Commonality, consistency and interoperability between components of the toolset.
• Service enablement capabilities.

Connectivity/Adapter Capabilities (Data Source and Target Support)

The ability to interact with a range of different types of data structure, including:

• Relational databases.
• Legacy and nonrelational databases.
• Various file formats.
• XML.
• Packaged applications such as CRM and supply chain management.
• Software-as-a-service (SaaS) and cloud-based applications and sources.
• Industry-standard message formats such as electronic data interchange (EDI), Swift and Health Level Seven International (HL7).
• Externalized parallel distributed processing (such as Hadoop Distributed File System [HDFS] and other noSQL type repositories).
• Message queues, including those provided by application integration middleware products and standards-based products (such as Java Message Service [JMS]).
• Emergent data types of a less structured nature, such as email, websites, office productivity tools and content repositories.

In addition, data integration tools must support different modes of interaction with this range of data structure types, including:

• Bulk acquisition and delivery.
• Granular trickle-feed acquisition and delivery.
• Changed-data capture (CDC) — the ability to identify and extract modified data.
• Event-based acquisition (time-based or data-value-based).

Data Delivery Capabilities

The ability to provide data to consuming applications, processes and databases in a variety of modes, including:

• Physical bulk data movement between data repositories.
• Federated views formulated in memory.
• Message-oriented movement via encapsulation.
• Replication of data between homogeneous or heterogeneous database management systems (DBMSs) and schemas.

In addition, support for the delivery of data across the range of latency requirements is important:

• Scheduled batch delivery.
• Streaming/real-time delivery.
• Event-driven delivery.

Data Transformation Capabilities

Built-in capabilities for achieving data transformation operations of varying complexity, including:

• Basic transformations, such as data type conversions, string manipulations and simple calculations.
• Intermediate-complexity transformations, such as lookup and replace operations, aggregations, summarizations, deterministic matching and the management of slowly changing dimensions.
• Complex transformations, such as sophisticated parsing operations on free-form text and rich media.

In addition, the tools must provide facilities for developing custom transformations and extending packaged transformations.

Metadata and Data Modeling Capabilities

As the increasingly important heart of data integration capabilities, metadata management and data modeling requirements include:

• Automated discovery and acquisition of metadata from data sources, applications and other tools.
• Data model creation and maintenance.
• Physical to logical model mapping and rationalization.
• Defining model-to-model relationships via graphical attribute-level mapping.
• Lineage and impact analysis reporting, via graphical and tabular format.
• An open metadata repository, with the ability to share metadata bidirectionally with other tools.
• Automated synchronization of metadata across multiple instances of the tools.
• Ability to extend the metadata repository with customer-defined metadata attributes and relationships.
• Documentation of project/program delivery definitions and design principles in support of requirements definition activities.
• Business analyst/end-user interface to view and work with metadata.

Design and Development Environment Capabilities

Facilities for enabling the specification and construction of data integration processes, including:

• Graphical representation of repository objects, data models and data flows.
• Workflow management for the development process, addressing requirements such as approvals and promotions.
• Granular role-based and developer-based security.
• Team-based development capabilities, such as version control and collaboration.
• Functionality to support reuse across developers and projects, and to facilitate the identification of redundancies.
• Support for testing and debugging.

Data Governance Capabilities (Interoperation With Data Quality, Profiling and Mining Capabilities)

Mechanisms to help the understanding and assurance of data quality over time, including interoperability with:

• Data profiling tools.
• Data mining tools.
• Data quality tools.

Deployment Options and Runtime Platform Capabilities

Breadth of support for the hardware and operating systems on which data integration processes may be deployed, and the choices of delivery model; specifically:

• Mainframe environments, such as IBM z/OS and z/Linux.
• Midrange environments, such as IBM System i (formerly AS/400) or HP Tandem.
• Unix-based environments.
• Windows environments.
• Linux environments.
• Traditional on-premises (at the customer site) installation and deployment of software.
• Hosted off-premises software deployment (SaaS model).
• Server virtualization (support for shared, virtualized implementations).
• Parallel Distributed Processing (such as MapReduce).

Operations and Administration Capabilities

Facilities for enabling adequate ongoing support, management, monitoring and control of the data integration processes implemented via the tools, such as:

• Error-handling functionality, both predefined and customizable.
• The monitoring and control of runtime processes, both via functionality in the tools and interoperability with other IT operations technologies.
• The collection of runtime statistics to determine use and efficiency, as well as an application-style interface for visualization and evaluation.
• Security controls, for both data "in flight" and administrator processes.
• A runtime architecture that ensures performance and scalability.

Architecture and Integration

The degree of commonality, consistency and interoperability between the various components of the data integration toolset, including:

• A minimal number of products (ideally one) supporting all data delivery modes.
• Common metadata (a single repository) and/or the ability to share metadata across all components and data delivery modes.
• A common design environment to support all data delivery modes.
• The ability to switch seamlessly and transparently between delivery modes (bulk/batch vs. granular real-time vs. federation) with minimal rework.
• Interoperability with other integration tools and applications, via certified interfaces and robust application programming interfaces (APIs).
• Efficient support for all data delivery modes, regardless of runtime architecture type (centralized server engine versus distributed runtime).

Service Enablement Capabilities

As acceptance of data services concepts continues to grow, data integration tools must exhibit service-oriented characteristics and provide support for SOA deployments, such as:

• The ability to deploy all aspects of runtime functionality as data services.
• Management of publication and testing of data services.
• Interaction with service repositories and registries.
• Service enablement of development and administration environments, so that external tools and applications can dynamically modify and control the runtime behavior of the tools.

Inclusion and Exclusion Criteria

To be included in this Magic Quadrant, vendors had to meet the following functional requirements.

They must possess within their technology portfolio the subset of capabilities identified by Gartner as the most critical from within the overall range of capabilities expected of data integration tools. Specifically, vendors must deliver the following functional requirements:

• Range of connectivity/adapter support (sources and targets): native access to relational DBMS products, plus access to nonrelational legacy data structures, flat files, XML and message queues.
• Mode of connectivity/adapter support (against a range of sources and targets): bulk/batch and CDC.
• Data delivery modes support: bulk/batch (ETL-style) delivery, plus at least one additional mode (federated views, message-oriented delivery or data replication).
• Data transformation support: at a minimum, packaged capabilities for basic transformations (such as data type conversions, string manipulations and calculations).
• Metadata and data modeling support: automated metadata discovery, lineage and impact analysis reporting, ability to synchronize metadata across multiple instances of the tool, and an open metadata repository including mechanisms for bidirectional sharing of metadata with other tools.
• Design and development support: graphical design/development environment and team development capabilities (such as version control and collaboration).
• Data governance support: ability to interoperate at a metadata level with data profiling and/or data quality tools.
• Runtime platform support: Windows, Unix or Linux operating systems.
• Service enablement (ability to deploy functionality as services conforming to SOA principles).

In addition, vendors had to satisfy the following quantitative requirements regarding their market penetration and customer base:

• They must generate at least $20 million of annual software revenue from data integration tools or maintain at least 300 maintenance-paying customers for their data integration tools.
• They must support data integration tools customers in at least two of the major geographic regions (North America, Latin America, Europe, the Middle East and Africa, and Asia/Pacific).
• They must produce at least 10 responsive references to participate in the customer survey as part of the Magic Quadrant process.

We excluded vendors that focus on only one specific data subject area (for example, the integration of customer data only), a single industry, or only their own data models and architectures.

Many other vendors of data integration tools exist beyond those included in this Magic Quadrant. However, most do not meet the above criteria and, therefore, we have not included them in our analysis. Market trends during the past three years indicate that organizations want to use data integration tools that provide flexible data access, delivery and operational management capabilities within a single-vendor solution. Excluded vendors frequently provide products to address one very specific style of data delivery (for example, only data federation) and cannot support other styles. Others provide a range of functionality, but operate only in a specific technical environment. Still others operate only in a single region or support only narrow, departmental implementations. Some vendors meet all the functional, deployment and geographic requirements but are very new to the data integration tools market, and have limited revenue and few production customers. The following vendors are sometimes considered by Gartner clients, alongside those appearing in the Magic Quadrant, when deployment needs match their specific capabilities (this list also includes recent market entrants with relevant capabilities):

• Ab Initio, Lexington, Massachusetts, U.S., www.abinitio.com — Application development toolbox (Co>Operating System) and component library for metadata management and data integration.
• Adeptia, Chicago, Illinois, U.S., www.adeptia.com — ETL Suite for traditional data integration patterns, and Integration Suite for application-to-application data consistency.
• Alebra Technologies, New Brighton, Minnesota, U.S., www.alebra.com — Parallel Data Mover for cross-platform file and database copying and sharing.
• Altibase, Palo Alto, California, U.S., www.altibase.com — Data Stream Middleware for supporting event-based data delivery for application and data integration.
• Apatar, Chicopee, Massachusetts, U.S., www.apatar.com — Open-source data integration tools focused on ETL and data federation scenarios.
• Arbutus Software, Burnaby, British Columbia, Canada, www.arbutussoftware.com — provides solutions for mainframe legacy data connectivity and access, in support of data integration and other use cases.
• Astera, Encino, Califormia, U.S., www.astera.com — provides ETL, CDC and business-to-business (B2B) data integration capabilities via the Centerprise Data Integrator product.
• Attunity, Burlington, Massachusetts, U.S., www.attunity.com — A range of data-integration-oriented products, including adapters (Attunity Connect), CDC (Attunity Stream), replication (Attunity Replicate) and data federation (Attunity Federate) for various platforms and database/file types.
• Axway, Phoenix, Arizona, U.S., www.axway.com — Offers software and services, including B2B data integration capabilities in support of various data sources, including variants of XML and EDI.
• BackOffice Associates, South Harwich, Massachusetts, U.S., www.boaweb.com — offers services and technology, including data integration capabilities, for data migrations, with a focus on SAP and other ERP environments.
• BIReady, New York, New York, U.S. and Langbroek, Netherlands, www.biready.com — Dynamic model resolution tool for rationalizing, deploying and populating analytic models, coupled with a data integration engine for transformations between models.
• C3 Business Solutions, Sydney, Australia, www.c3integrity.com — Offers a simplified set of tools for consolidating data, validating data, and acquiring data from sources including Excel, Access, CSV, and fixed-width and XML-standard data formats.
• CA, Islandia, New York, U.S., www.ca.com — Advantage Data Transformer provides ETL-oriented data integration. InfoRefiner provides replication and propagation capabilities for mainframe data repositories.
• CDB Software, Houston, Texas, U.S., www.cdbsoftware.com — CDB/Delta provides CDC and replication capabilities for IBM DB2 on the z/OS platform.
• Columba Global Systems, Dublin, Ireland, www.columba.com — Positioned as "data fusion platform" technology, the Exprimer solution supports federated approaches to data integration.
• Composite Software, San Mateo, California, U.S., www.compositesw.com — Composite Information Server provides data federation capabilities and supports the delivery of data access services.
• DataRoket, Washington D.C., U.S., www.dataroket.com — Offers ETL and data federation capabilities via the DataRoket product suite.
• Datawatch, Chelmsford, Massachusetts, U.S., www.datawatch.com — The Monarch Data Pump product provides ETL functionality with a bias toward extracting data from report text, PDF files, spreadsheets and other less-structured data sources.
• DBSync, Brentwood, Tennessee, U.S., www.mydbsync.com — offers the dbsync integration platform for integration of data between databases and applications, both on-premises and via SaaS.
• Dell Boomi, Berwyn, Pennsylvania, U.S., www.boomi.com — Acquired by Dell, Boomi provides technology for integration of data to and between SaaS-based applications and data sources.
• Denodo Technologies, Palo Alto, California, U.S. and Madrid, Spain, www.denodo.com — The Denodo Platform provides data federation and mashup enablement capabilities for joining structured data sources with data from websites, documents and other less-structured repositories.
• ETI, Austin, Texas, U.S., acquired by Versata, www.versata.com — The ETI Solution has a code-generation architecture focused on bulk/batch-oriented data movement.
• ETL Solutions, Bangor, U.K., www.etlsolutions.com — Transformation Manager provides a metadata-driven toolset for the authoring, testing, debugging and deployment of various data integration requirements.
• Expressor software, Burlington, Massachusetts, U.S., www.expressor-software.com — The expressor product is based on a semantic approach to designing and managing data integration processes.
• Gamma Soft, Ivry-sur-Seine, France, www.gamma-soft.com — Supports CDC and data replication for various heterogeneous data source types via a data distribution product.
• GSS Group, Markham, Ontario, Canada, www.gssgrp.com — Vigilance Xpress is a Web-based solution for SQL Server data marts supporting Microsoft's .NET Framework, SQL Server and SQL Server Reporting Services.
• GT Software, Atlanta, Georgia, U.S., www.gtsoftware.com — the Ivory Data Access product line supports connectivity to, and integration with, mainframe-based data sources of various types.
• HiT Software, San Jose, California, U.S., www.hitsw.com — offers database replication (DBMoto), database-to-XML transformation and mapping (Allora) and DB2 connectivity products. HiT was acquired by BackOffice Associates during 2010, but still operates under the HiT brand.
• HVR Software, Amsterdam, The Netherlands, www.hvr-software.com — High Volume Replicator (HVR) technology supports CDC, propagation, and replication patterns against various data source and platform types.
• Innovative Routines International (CoSort), Melbourne, Florida, U.S., www.cosort.com — Its Fast Extract and SortCL tools provide for rapid unloading and transformation of data in Oracle databases in support of ETL processes.
• Javlin, Reston, Virginia, U.S., www.cloveretl.com — Offers the CloverETL product for bulk/batch-oriented data movement.
• Jitterbit, Oakland, California, U.S., www.jitterbit.com — Freely downloadable software with a focus on application integration (event- and message-based) and data integration.
• JumpMind, Columbus, Ohio, U.S., www.jumpmind.com — The open-source SymmetricDS product set offers data replication capabilities for a variety of relational DBMS environments.
• Kapow, Palo Alto, California, U.S., www.kapow.com — the Kapow Katalyst and Kapow Extraction Browser provide a browser-based approach to integrating data across on-premises and cloud-based applications and websites.
• Kinetic Networks, San Francisco, U.S., www.kineticnetworks.com — Supports ETL capabilities via KETL, an open-source data integration tool.
• Metatomix, Dedham, Massachusetts, U.S., acquired by Versata, www.versata.com — Follows a semantics-based approach to the creation of data services and federated views of data across multiple data sources.
• Nimaya, Washington, D.C., U.S., www.nimaya.com — ActionBridge technology enables virtual federation of data across on-premises and SaaS-based data sources.
• OpenText, Waterloo, Ontario, Canada, www.opentext.com — Offers OpenText Data Integration for bulk/batch-oriented data movement across a range of data source and target types.
• PSB, Amsterdam, Netherlands, www.psb.nl — Offers data replication capabilities via the High Volume Replicator (HVR) solution.
• Pentaho, Orlando, Florida, U.S., www.pentaho.org — A provider of open-source BI solutions, Pentaho has added data integration tools to its portfolio by leveraging the Kettle open-source project and providing services and support.
• Pitney Bowes Business Insight, Troy, New York, U.S., www.pbinsight.com — A software and services division of customer communications management vendor Pitney Bowes, it competes in the data integration tools market with the Spectrum data management platform, which includes ETL capabilities.
• Progress Software, Bedford, Massachusetts, U.S., www.progress.com — The DataXtend and DataDirect product lines provide tools for data access, replication and synchronization.
• Queplix, Sunnyvale, California, U.S., www.queplix.com — Offers Virtual Data Viewer and Virtual Data Manager products for integration of cloud-based, SaaS, and on-premises applications and data.
• Quest Software, Aliso Viejo, California, U.S., www.quest.com — SharePlex provides real-time replication support for Oracle DBMS environments and is aimed primarily at high-availability applications.
• Red Hat/MetaMatrix, Raleigh, North Carolina, U.S., www.redhat.com — MetaMatrix Server, Enterprise and Query products support the creation of data models and model-driven federated views of data.
• Relational Solutions, Westlake, Ohio, U.S., www.relationalsolutions.com — The BlueSky Integration Studio provides ETL capabilities in a simplified, low-cost toolset that runs in the Windows environment.
• Safe Software, Surrey, British Columbia, Canada, www.safe.com — The FME platform delivers ETL capabilities for spatially oriented data sources commonly used in geographic information system applications.
• SchemaLogic, Kirkland, Washington, U.S., www.schemalogic.com — Enables the creation and maintenance of data models (Workshop) and business models (SchemaServer), and the ability to propagate models and data across applications (Integration Service).
• Scribe Software, Bedford, New Hampshire, U.S., www.scribesoft.com — Provides data migration and integration solutions supporting deployments of business applications, with a focus on Microsoft Dynamics.
• Seagull Software, Atlanta, Georgia, U.S., www.seagullsoftware.com — Offers SmartDB for data migrations to the Oracle E-Business Suite.
• Sesame Software, San Francisco, California, U.S., www.sesamesoftware.com — Offers the Relational Junction product suite for synchronization of data between popular packaged and SaaS applications, with a focus on ETL-oriented patterns of integration.
• SnapLogic, San Mateo, Califormia, U.S., www.snaplogic.com — DataFlow supports real-time and federated integration of data with a focus on diverse data sources, including SaaS- and cloud-based sources, and via Web-oriented architectural approaches.
• Stone Bond Technologies, Houston, Texas, U.S., www.stonebond.com — Supports both federated/virtualized data integration and physical data movement via the Enterprise Enabler technology set.
• Software AG, Darmstadt, Germany, www.softwareag.com — The CentraSite product provides data and metadata federation capabilities and is geared toward SOA deployments. The vendor's webMethods product line provides process-oriented integration capabilities.
• Software Labs, Roseville, California, U.S., www.softlabsco.com — The xFusion Studio product provides ETL functionality positioned toward a range of use cases, including BI and migrations.
• SQData, Addison, Texas, U.S., www.sqdata.com — The SQData product line provides CDC and ETL functionality focused on delivering mainframe data sources and popular relational DBMSs.
• Sypherlink, Dublin, Ohio, U.S., www.sypherlink.com — Metadata discovery and mapping via Harvester, and access to data sources for the creation of integrated views via Exploratory Warehouse.
• Vision Solutions, Irvine, California, U.S., www.visionsolutions.com — Real-time database replication functionality is provided by the Vision Replicate1 product.
• WhereScape, Portland, Oregon, U.S., www.wherescape.com — WhereScape RED enables the rapid creation and maintenance of data warehouses, including ETL functionality.
• XAware, Colorado Springs, Colorado, U.S., www.xaware.com — Provides support for the access, integration and service enablement of data sources via its XA-Suite product.

Added

No vendors have been added in this iteration of the Magic Quadrant.

Dropped

Pitney Bowes Business Insight has been dropped due to gaps in relation to the functional inclusion criteria and a lack of examples exhibiting a balance in support of multiple styles beyond bulk data movement.

Evaluation Criteria

Ability to Execute

Gartner analysts evaluate technology providers on the quality and efficacy of the processes, systems, methods or procedures that enable IT providers' performance to be competitive, efficient and effective, and to positively affect revenue, retention and reputation. Ultimately, technology providers are judged on their ability to capitalize on their vision, and their success in doing so.

We evaluate vendors' ability to execute in the data integration tools market by using the following criteria:

• Product/Service. How well the vendor supports the range of data integration functionality required by the market, the manner (architecture) in which this functionality is delivered, and the overall usability of the tools. Product capabilities are critical to the success of data integration tool deployments and, therefore, receive a high weighting.
• Overall Viability. The magnitude of the vendor's financial resources and the continuity of its people and technology. We place a high weighting on this criterion, which affects the practical success of the business unit or organization in generating business results.
• Sales Execution/Pricing. The effectiveness of the vendor's pricing model and the effectiveness of its direct and indirect sales channels. This criterion is weighted high due to the sustained scrutiny on cost issues and the highly competitive nature of this market.
• Market Responsiveness and Track Record. The degree to which the vendor has demonstrated the ability to respond successfully to market demand for data integration capabilities over an extended period, and how well the vendor acted on the vision of prior years.
• Marketing Execution. The overall effectiveness of the vendor's marketing efforts, which impacts its mind share, market share and account penetration. The ability of the vendor to adapt to changing demands in the market by aligning its product message with new trends and end-user interests.
• Customer Experience. The level of satisfaction expressed by customers with the vendor's product support, professional services; their overall relationship with the vendor; and their perceptions of the value of the vendor's data integration tools relative to costs and expectations. In this iteration of the Magic Quadrant we have retained a weighting of "high" for this criterion to reflect buyer's continued scrutiny of these considerations as a result of economic conditions and budgetary pressures. Analysis and rating of vendors against this criterion is driven directly by responses from customers who participated in the reference customer survey that Gartner conducted as part of the process of reaching this Magic Quadrant.

Source: Gartner (October 2011)

Completeness Of Vision

Gartner analysts evaluate technology providers on their ability to convincingly articulate logical statements about current and future market direction, innovation, customer needs and competitive forces, as well as how they map to Gartner's position. Ultimately, technology providers are assessed on their understanding of the ways that market forces can be exploited to create opportunities.

We assess vendors' completeness of vision for the data integration tools market by using the following criteria:

• Market Understanding. The degree to which the vendor leads the market in new directions (technology, product, services or otherwise), and its ability to adapt to significant market changes and disruptions. Given the dynamic nature of this market, this item receives a weighting of "high."
• Marketing Strategy. The degree to which the vendor's marketing approach aligns with and/or exploits emerging trends and the overall direction of the market.
• Sales Strategy. The alignment of the vendor's sales model with the ways in which customers' preferred buying approaches will evolve over time.
• Offering (Product) Strategy. The degree to which the vendor's product road map reflects demand trends in the market, fills current gaps or weaknesses, and includes developments that create competitive differentiation and increased value for customers. In addition, given the requirement for data integration tools to support diverse environments from a data domain, platform and vendor-mix perspective, we assess vendors on the degree of openness of their technology and product strategy. With the growth in diversity of data and environments involved in data integration initiatives, this criterion receives a weighting of "high."
• Business Model. The overall approach the vendor takes to execute its strategy for the data integration tools market.
• Vertical/Industry Strategy. The degree of emphasis the vendor places on vertical solutions, and the vendor's depth of vertical market expertise.
• Innovation. The degree to which the vendor has demonstrated a willingness to make new investments to support its strategy and enhance its product capabilities; the level of investment in R&D directed toward development of these tools; and the extent to which the vendor demonstrates creative energy. Given the pace of expansion of data integration requirements and the highly competitive nature of the market, this criterion receives a weighting of "high."
• Geographic Strategy. The vendor's strategy for expanding its reach into markets beyond its home region/country, and its approach to achieving global presence (for example, its direct local presence and use of resellers and distributors).

Source: Gartner (October 2011)

Leaders

Leaders in the data integration tools market are frontrunners in the convergence of single-purpose tools into an offering that supports a range of data delivery styles. These vendors are strong in the more traditional data integration patterns. They also support newer patterns and provide capabilities that enable data services in the context of SOA. Leaders have significant mind share in the market, and resources skilled in their tools are readily available. These vendors establish market trends, to a large degree, by providing new functional capabilities in their products, and by identifying new types of business problem to which data integration tools can bring significant value. Examples of deployments that span multiple projects and types of use case are common among Leaders' customers.

Challengers

Challengers are well positioned in light of the key trends in the data integration tools market, such as the need to support multiple styles of data delivery. However, they may not provide a comprehensive breadth of functionality, or may be limited to specific technical environments or application domains. In addition, their vision may be hampered by the lack of a coordinated strategy across the various products in their data integration tools portfolio. Challengers can vary significantly with regard to their financial strength and global presence. They are often large players in related markets that have only recently placed an emphasis on data integration tools. Challengers generally have substantial customer bases, though implementations are often of a single-project nature, or reflect multiple projects of a single type (for example, all ETL-oriented use cases).

Visionaries

Visionaries have a solid understanding of emerging technology and business trends, or a position that is well aligned with current demand, but they lack market awareness or credibility beyond their customer base or a single application domain. Visionaries may also fail to provide a comprehensive set of product capabilities. Visionaries may be new entrants lacking the installed base and global presence of larger vendors, though they could also be large, established players in related markets that have only recently placed an emphasis on data integration tools. The growing emphasis on aligning data integration tools with the market's demand for interoperability of delivery styles, convergence of related offerings (such as data integration and data quality tools), metadata modeling, support for emerging analytics environments, among other things, is creating fresh challenges for which vendors must demonstrate vision.

Niche Players

Niche Players have gaps in both their vision and ability to execute, often lacking key aspects of product functionality and/or exhibiting a narrow focus on their own architecture and installed base. These vendors have little mind share in the market and are not recognized as proven providers of data integration tools for enterprise-class deployments. Many Niche Players have very strong offerings for a specific range of data integration problems (for example, a particular set of technical environments or application domains) and deliver substantial value for their customers in that segment.

Vendor Strengths and Cautions

IBM

Armonk, New York, U.S.

www.ibm.com

Products: IBM InfoSphere Information Server (includes the following components: InfoSphere DataStage, InfoSphere QualityStage, InfoSphere Change Data Capture, InfoSphere Federation Server and InfoSphere Foundation Tools), InfoSphere Data Event Publisher, InfoSphere Replication Server

Customer base: estimated at over 9,000

Strengths

• IBM provides an extensive range of data integration functions, including classic bulk-batch ETL, CDC and propagation, data replication and data federation. Comprising products sold both independently and in the InfoSphere Information Server bundle, IBM's portfolio is broad, comprehensive and well-aligned with the market's demand for combinations of multiple data delivery capabilities. IBM's InfoSphere Foundation Tools complement its core data integration products, with extensive metadata discovery, modeling and metadata management functionality. InfoSphere Blueprint Director functionality extends IBM's data integration capabilities by enabling staff such as business analysts and data stewards to participate in the design of data flows. IBM's customers often address data integration projects of greater scale and complexity than are attempted with many of its competitors' products.
• IBM's customers reflect a range of deployment use cases, including BI and data warehousing, MDM, data migration and operational integration scenarios. Our recent interactions with reference customers indicate that, once adopted, IBM's tools are often considered the enterprise's standard and are deployed for multiple projects involving teams of different sizes. IBM's reference customers often conduct implementations of significant complexity and scale.
• Reference customers routinely cite as key reasons for selecting IBM, rather than a competing vendor, the range of functions IBM offers, the shared use of common metadata by key components of IBM's product set, and the interoperability with related IBM data quality products. IBM's main focus in developing new versions during 2010 was to simplify installation and upgrading, an issue that has been very problematic for IBM over the years. The main result of this effort, version 8.5 of the technology set, became generally available in 4Q10, and reference customers recognize that it has improved the installation and upgrade experience. The next significant release, version 8.7, expected in 4Q11, will focus on enhanced manageability (with the introduction of expanded real-time-operations console functionality), information governance and support for "big data."

Cautions

• Although release 8.5 of IBM's technology has improved the installation and upgrade experience, IBM customers routinely mention challenges with overall complexity of the product set, especially when multiple products are used in an integrated fashion. Some reference customers indicated that the actual level of product integration was less than IBM had led them to expect during the sales process. IBM must continue to improve the customer experience post-installation, by makings its products easier to use and decreasing the time-to-value. Our recent interactions with reference customers indicate that average times to deployment were longer for IBM than both the market average and most other vendors. With an estimated 20% of the customer base running release 8.5, the majority of IBM customers have yet to experience the improved usability offered by this latest version of the technology. Also, version 8.7 is intended to further improve the usability and cost of ownership aspects of deploying IBM's tools.
• Customers continue to report inconsistent experiences with IBM's technical support. They report a substantial number of bugs, which require many patches, and often express a desire for higher-quality support interactions and faster resolution of bug-related issues. To address these issues, IBM has recently reorganized its development labs in order to have a "continued engineering" arm, with a focus on support responsiveness, rapid bug fixes, and tighter management of "fix pack" delivery. In addition, uptake of IBM's data federation capabilities beyond DB2-centric environments remains limited. With the growing interest in data federation capabilities for heterogeneous environments, IBM may be at a disadvantage relative to many key competitors (Informatica, Oracle, SAP, SAS, and others) when these capabilities for non-DB2 environments are critical for customers.
• Pricing remains a major concern for IBM customers. The use of CPU speed as the main pricing parameter (which can add complexity for customers in auditing and modifying their implementations) and the relatively high cost of a typical implementation (compared with many of IBM's competitors) lead some prospective customers to consider alternative providers or to limit their investment to a small number of components (InfoSphere DataStage only, for example). For some midsize enterprises and those seeking functionality required for a specific use case, the pricing approach may offer some advantages. To mitigate some of these concerns, IBM has introduced limited-scale and purpose-bundled offerings such as InfoSphere Information Server Workgroup Edition and DB2 Data Warehouse Edition, as well as term-based licensing options.

Informatica

Redwood City, California, U.S.

www.informatica.com

Products: Informatica Platform (includes the following components: PowerCenter, PowerExchange, Data Services, Cloud Data Integration)

Customer base: estimated at over 4,350

Strengths

• According to its customers, Informatica provides robust, quality assured products that support integration needs for an extremely wide variety of data sources and targets. Informatica is one of the most widely recognized vendors and continues to increase its presence both in existing customers and new accounts. PowerCenter regularly appears in enterprise-scale data integration tool evaluations more often than any other provider. Ninety-seven percent of Informatica's reference customers indicate that Informatica supplies the standard data integration tool in their organization. The vendor's current release, Informatica 9.1, released in early 2011, is well aligned with current market demand and evolving trends. It now includes data federation (about 7% of the customers report they use this capability) and brings together data integration and data quality capabilities in a single runtime architecture that aligns well with trends for technological consolidation.
• Version 9.1 was released with an emphasis on "self-service" data integration by business users and analysts. Significant data management skills are still required for this self-service data integration, but the interface provides the right level of support to enable a business analyst to accomplish simple data integration, aided by a library of prebuilt integration "rules" and "mappings" for common transformations such as financial operations and reusable expressions. The analyst interface uses the same metadata as the more powerful developer's interface, and this promotes stronger collaboration between business analysts and data integration development experts, as well as supporting the sharing of data between business users.
• Informatica's vision remains market-leading. In 2011, Informatica added hierarchical processing functionality (hierarchical to hierarchical as well as hierarchical to relational) and data masking to its federation capability. It also added preview and testing functionality to its Web services deployment, and support for "big data" using MapReduce via the Apache Hadoop project. Informatica offers cloud support for a data integration as a service offering. Informatica's Cloud Services focus on SaaS integration, with a heavy emphasis on salesforce.com integration; Cloud Platform is available for independent service providers and system integrators to develop cloud-based services and for enterprises to extend on-premises integration; there are also Informatica offerings in Cloud Editions for deployments in a public cloud setting, such as Amazon.

Cautions

• Informatica's customer base covers a diverse use cases. Although, Informatica has recently demonstrated an ability to expand beyond batch/bulk processing, its deployment architectures are still heavily oriented toward bulk, batch-oriented data delivery. Given the increasing demand from visionary implementers for a full range of integration styles, this indicates that Informatica must continue to focus on developing best practices and solid references for "non-batch," and strengthen its ability to gain market share beyond its core area of expertise.
• Although Informatica's support and professional services are relatively strong, some customers report concerns about support — this is a new and unexpected development as support has always been one of Informatica's strength. More specifically, they report that some of the new functionality is not as robust and strong as expected (multiple customers mentioned Web services capability in this regard). Customers also report that some "push down" capabilities for functions and services running in the database exhibit weakness. There are also reports of uneven delivery of professional services concerning a certain lack of field deployment practices when support is needed. Customers also report fresh difficulties installing upgrades. Taken together, these reports indicate that Informatica's growth and the expansion of its delivery capabilities are straining its infrastructure. Informatica's action plan in response to these "growing pains" includes a partner accreditation program, and new methodologies and support-related processes — but, for 2011 and 2012, organizations should be aware of this strain.
• Informatica has high price points, similar to other leading vendors. This creates an initial impression that sometimes deters prospective buyers from engaging Informatica in further discussion. In fact, however, the high price points are the result of a "value layering" effect, which occurs as organizations request greater functionality from their data integration platform as a whole, and the overall value remains aligned with functionality. Informatica must continue to articulate the value of broader functionality and wide applicability — otherwise its pricing will become an inhibitor in competitive situations. Informatica customers who responded to our survey reported that its solution is expensive compared with similar solutions from "stack" vendors. Similarly, they indicated that its pricing is confusing — another result of value layering. The various "editions" help customers understand the price, but they also make it difficult to compare the price paid to the "layered value" being delivered. An alternative cost model is cloud-based delivery with on-demand and term-based pricing.

iWay Software

New York, New York, U.S.

www.iwaysoftware.com

Products: DataMigrator, Data Hub, Parallel Service Manager, Universal Adapter Framework, EIM Suite

Customer base: estimated at over 395

Strengths

• iWay Software is a division of Information Builders. It creates and sells integration technologies with the aim of building an integration software business independent of the BI capabilities for which Information Builders is well known. In March 2011, iWay announced that the Parallel Service Manager could support MapReduce functionality for use on both structured information and content. In July, it also announced upgrades to iWay CEP Enable for complex event processing in real time in order to support social media analytics. iWay has always been known for its adapters and connectivity, but in 2011 customers also began to report ease of use and interoperability with their data quality tools. Customers indicate that the tools perform as advertised. iWay offers capabilities for physical data movement and delivery (via its DataMigrator ETL tool), data federation (via the iWay Data Hub product) and real-time message-oriented integration (supported by the Service Manager product). With both data quality and MDM offerings in its portfolio, iWay is well positioned in terms of both current and future market demand.
• iWay's global presence is growing as a continuation of its 2010 sales force expansion. It has started new subsidiaries in Europe. In August it announced a new general manager with a background in competitors' products, databases, BI and capital investment. According to Gartner data, iWay's revenue continues to exceed the market average growth in 2010.
• When completing implementations, iWay customers report slightly longer periods to move to production than some of iWay's competitors, but very few "long" implementation cycles (projects rarely take longer than six months). This could be due to the size of the projects involved, but we see this as a strength as customers' reports also value as accelerators the products' ease of use and the data integration knowledge of support staff. Therefore, iWay's products are consistent and predictable for deployment efforts. Over 80% of iWay's customers indicate that it supplies the standard integration tool in their organization — although not as high a figure as for some of the other vendors that score strongly for their ability to execute, this is a respectable percentage. Based on survey results, upgrading with iWay appears to be an easy decision; also, with expanding functionality, customers appear ready to move quickly to new versions in order to gain new capabilities, even though customers report some issues with upgrades and patches when they are released.

Cautions

• iWay continues to work with a relative lack of mainstream recognition. Gartner's client inquiries indicate that iWay is not considered as frequently as its market-leading competitors. That iWay solutions are often introduced at a departmental level for limited deployments appears to be more a barrier to enterprisewide adoption than any limitation of the toolset. This also means that finding qualified personnel to deploy iWay solutions will remain difficult, as professional services organizations perceive limited need to learn them. iWay is trying to counter this problem by focusing on a development environment intended to simplify deployment and by building up its partner delivery channels.
• iWay data integration projects generally take four to nine months to deliver, according to Gartner clients who use the company's products. By contrast, users of other vendors' products report more short-duration projects and more longer-term projects, with a much more even distribution across the spectrum. Additional survey comments indicate this pattern is due to a departmental deployment pattern. Thus, while iWay is proving to have a robust offering for departmental adoption, it is having some difficulty with wider enterprise adoption. This is more of a perceived barrier than a practical barrier, as some organizations that have a standard data integration tool do not actually enforce that standard. Leading causes for the uneven feel of iWay's product in use are reported to be its "steep learning curve" and the associated level of effort required.
• Customers report inconsistent experiences with iWay's professional services. Feedback on iWay's product support also indicates challenges in terms of quality, especially when addressing complex problems. During the past 12 to 18 months, iWay has started programs in new regions and markets, including the introduction of regional and country-level support services. iWay's uneven support is to be expected as the vendor's infrastructure expands, so prospective customers should determine whether support is present where they need it and has the required level of expertise.

Microsoft

Redmond, Washington, U.S.

www.microsoft.com

Products: SQL Server Integration Services (SSIS), BizTalk Server

Customer base: estimated at over 12,000

Strengths

• Microsoft's main offering in the data integration tools market is SSIS, which is largely focused on bulk/batch-oriented data delivery. Reference customers cite SSIS's total cost of ownership, speed of implementation, ease of use and ability to integrate with the other capabilities of Microsoft SQL Server as the main reasons for choosing it in preference to alternatives. The tool's interfaces and languages remain familiar to developers over the course of releases, and this familiarity sustains productivity and minimizes the learning curve.
• Reference customers continue to recognize SSIS as a stable and maturing data integration tool capable of supporting enterprise-scale implementations in Microsoft-centric environments. The delivery in the forthcoming "Denali" release of data quality capabilities to complement the product's data integration capabilities aligns with convergence trends and will address customers' needs for data governance when using SSIS. Deployment scenarios are expanding beyond BI and data warehousing to broadening uses in support of data consistency between operational applications and data migrations. Pervasive use of SSIS by SQL Server customers has resulted in widely available community support, training and third-party documentation on implementation practices and approaches to problem resolution.
• Microsoft's size and global presence provide a huge customer base and a distribution model that supports both direct and channel partner sales. In addition, customer references generally report a very positive post-sales support and service experience, including product documentation and online support mechanisms.

Cautions

• Although SSIS can integrate with BizTalk to support message-oriented integration and Microsoft can address replication-style data delivery via SQL Server functionality, this vendor's present product strategy does not clearly indicate a comprehensive data integration vision for the market. Capabilities such as metadata discovery, lineage and dependency reporting are a weakness, although improvement efforts via Microsoft's Project Barcelona are under way to enhance support for impact and lineage metadata management capabilities. End-user organizations requiring functions for CDC must deploy this capability via partners or their own best-of-breed implementations.
• The inability to deploy data integration workloads on non-Windows environments is a limitation for customers wishing to leverage the processing power of diverse hardware and operating system platforms. The SQL Server 2008 R2 version of SSIS substantially expanded Microsoft's ability to support broader types of data connectivity requirement.
• Deployments involving interoperability between SSIS and multiple products (such as BizTalk and SQL Server 2008 Master Data Services) are reported to have required extensive custom-coding efforts. Reference customers rarely demonstrate synergistic uses of SSIS with other Microsoft products for data integration purposes, such as for data replication or federation using BizTalk or SQL Server.

Oracle

Redwood Shores, California, U.S.

www.oracle.com

Products: Oracle Data Integrator (ODI), Oracle Data Service Integrator (ODSI), Oracle GoldenGate, Oracle Warehouse Builder (OWB)

Customer base: estimated at over 3,500

Strengths

• Oracle's data integration capabilities center on ODI for bulk-batch data movement and the GoldenGate offering for CDC and real-time data delivery. Ancillary offerings include ODSI, which provides data federation capabilities, and OWB, which also supports bulk-batch data movement and is bundled with the Oracle DBMS. These primary data integration products, along with the message-oriented functionality of Oracle WebLogic technology, enable Oracle to support each of the major data delivery styles in this market. Although not directly focused on this market, Oracle's recent acquisitions of data quality technology (see "Oracle Adds Datanomic to Its Disparate Mix of Data Quality Tools") are also well-aligned with market trends.
• Adoption of both ODI and GoldenGate continues to grow within the Oracle DBMS and applications customer base. Oracle's strong presence in these and other related markets, and its ability to bundle data integration tools with high-growth products such as the Oracle Exadata Database Machine, will continue to create opportunities for increased penetration. Traditional ETL-style implementations in support of BI and data warehousing are common. More Oracle customers are adopting GoldenGate to support real-time and granular data flows, also in support of BI needs and various operational integration patterns involving heterogeneous DBMS environments. Although Oracle does not report revenues or customer counts by product, it claims it is exceeding the market's average growth.
• As their main reasons for selecting Oracle's tools in this market, reference customers cite the range of connectivity and support for a diversity of data source/target types and platforms, the reasonable learning curve, and the perception that Oracle serves as a "one-stop shop" for all their potential data integration functionality needs. These customers also exhibit a mix of use cases and project types — while the vast majority (over 80%) use the tools in support of BI, master data management architectures and data migrations represent areas of increasing activity.

Cautions

• Many Oracle reference customers, as well as customers whom Gartner interacts with via client inquiries, cite pricing challenges as a concern. For most, it is the per-source/target CPU-core pricing model of ODI and GoldenGate that contribute to the high cost of licensing the technology, especially in large and diverse environments. Recently, Oracle has started offering alternative pricing models in an attempt to mitigate these concerns. For example, ODI can now be licensed based on the number of CPU cores on which transformation logic will execute, and GoldenGate can be procured using a term-based licensing model. These options will help some customers find a more attractive cost model, but it remains to be seen if they will address the majority of customers' concerns.
• Oracle customers continue to cite gaps in product support, documentation and availability of skills relating to the data integration product set. In addition, reference customers routinely cite weaknesses in operational and administrative functionality (monitoring, error handling, upgrades and version control), as well as a desire for deeper integration across the product set. Although customers are most often satisfied with the general functionality of each individual product, many state that the degree of integration across the products, and with other Oracle infrastructure software such as Oracle Enterprise Manager, does not live up to the expectations set by Oracle Sales and Marketing.
• Adoption of ODSI by Oracle customers remains limited, with less than 5% of recent reference customer samples indicating use of the tool. This is a substantially lower percentage than is seen across the market in general, and specifically for Oracle's major competitors. Oracle's product road map and development work in 2011 focused heavily on improvements to, and integration of, ODI and GoldenGate. ODSI currently remains a stand-alone product. Oracle says it will deliver metadata interchange capabilities between ODI and ODSI in 2012, which will begin to link ODSI more closely to the rest of the product set.

Pervasive Software

Austin, Texas, U.S.

www.pervasiveintegration.com

Products: Data Integrator, Metadata Manager, Integration Hub, DataCloud, DataRush

Customer base: estimated at over 5,160

Strengths

• With a track record of profitability in this market for over 10 years, good capitalization and a 25% research and development ratio, Pervasive Software has demonstrated that it is a stable vendor. It offers solid and attractively priced data integration tools that support bulk/batch-oriented data delivery and provide capabilities for real-time messaging-style solutions and SOA. The broad range of support for data sources and target types — including packaged applications, popular SaaS application APIs (such as for salesforce.com), industry-standard message formats (such as EDI documents, X12, the Health Insurance Portability and Accountability Act [HIPAA] and Health Level Seven [HL7]), and semi-structured content repositories provided with the core products — represents substantial value for customers. By expanding its reach to address the diverse technology landscape common in large enterprises, and continuing to do so with an attractive cost model, Pervasive demonstrates good vision for this market.
• Customer references, including those within large enterprises, show a very good balance of use across the full range of common data integration use cases, with a particular emphasis on supporting interfaces between operational applications, data migration efforts and interenterprise data-sharing activities. This is a notable difference from the majority of Pervasive's competitors, many of which show less diversity of use and are biased toward a particular use case, such as BI. Pervasive has the ability to deliver consolidated/centralized MDM under a data integration scenario and to manage complicated master data governance models. In 2010 and 2011, Pervasive improved its cloud capabilities (for support of external private clouds) and data quality and launched a partner/developer community (contributing connectors and use case solutions). The increasing use of Pervasive's technology in cloud settings — both connecting to SaaS applications and via its own cloud environment, Pervasive DataCloud (which, it says, currently supports over 300 customers) — will also be a benefit as demand for alternative data integration delivery models increases.
• Customers identify ease of implementation and ongoing use, mapping/transformation capabilities, and the broad range of data source and target support via packaged connectors as the most significant functional strengths of Pervasive's offering. Performance and scalability are also commonly noted as positives. The vendor's DataRush offering adds to its scalability story by offering the ability to execute data transformation workloads in a highly parallel fashion, and it includes Hadoop/HDFS capability. In addition, customers identify attractive pricing and their positive overall experience with the vendor as key reasons for selecting and continuing to use Pervasive's offering. The same characteristics have enabled Pervasive to continue to expand its indirect channel, and revenues via partners have grown from year to year.

Cautions

• In comparison to other vendors in this space, Pervasive's revenue is small. Pervasive's customer contact model uses extensive indirect channels (SaaS/cloud vendors, OEM relationships and resellers) and, as a result, Pervasive's market mind share remains small, almost negligible. Although this would represent a risk for a newly established organization, Pervasive has a multidecade record of delivering while using this approach as its preferred business model. Also, since many buyers of data integration tools want consistency and standardization, Pervasive offers a choice of delivery model — its software can be used either as a standard or deployed tactically, implemented on a stand-alone basis or embedded in other solutions, and procured via resellers or direct from Pervasive. If an organization can use this model, it becomes a good fit.
• Pervasive customers cite frequent bugs, which the vendor confirms quickly and develops solutions for. However, the same customers report that they usually have to wait for a new release to get the fixes. Additionally, users indicate the learning curve and interface issues as weaknesses — which seems to contradict reports of "ease of use." In fact, this is a logical result of Pervasive's heritage. Its tools were initially developed to support OEMs, which would embed its functionality in other applications and architectures. The users in such cases are technical personnel, and they find the interfaces adequate or even good. However, the same interface proves a bit more difficult when used by data transformation specialists who are not necessarily familiar with embedding functionality into other applications. This uneven perception could be partly responsible for Pervasive's lack of mind share, even as its user numbers grow.
• Pervasive does not provide support for data federation. Other weaknesses, as cited by customer references, include its metadata and modeling functionality and relatively limited (compared with larger competitors) availability of skills. However, Pervasive's release of Version 10 resolved some previously reported issues with administrative capabilities (such as monitoring, tuning and error handling) and introduced enhanced real-time support, metadata management and administration/monitoring.

SAP

Palo Alto, California, U.S.

www.sap.com/eim

Products: SAP BusinessObjects Data Integrator, SAP BusinessObjects Data Federator, SAP BusinessObjects Data Services, SAP NetWeaver Process Integration and Sybase Replication Server

Customer base: estimated at over 10,000

Strengths

• The release of SAP BusinessObjects Data Services 4.0 in 2011 delivers a set of consolidated data integration offerings and signals a progressive tool suite encompassing data and process integration functions, and supporting services deployment and orchestration to meet the demands of converged data integration and data quality functions. Added capabilities include tighter integration between Data Services and SAP's BusinessObjects Business Intelligence Platform and SAP Business Suite, enhanced metadata and profiling capabilities (through the SAP BusinessObjects Information Steward product) to complement data integration functions, and support for native text analytics. SAP's approach, based on a single runtime platform in Data Services, to offering data integration functions and combining data quality with data integration services is described by customers as increasingly relevant and necessary for implementations. References cited improved support for sharing metadata and developing data quality controls, when performing data integration functions. Also, leverage of Sybase PowerDesigner provides richer metadata and modeling capabilities for data integration.
• The breadth of functionality available across the portfolio to support a wide range of data delivery styles, plus options to integrate with data quality capabilities and SAP's MDM offering, remains attractive to SAP's existing customers and prospective customers. Adoption reflects the broadening use of MDM and data migrations, in addition to established scenarios in BI and data warehousing. An extension of Data Services for loading data from SAP and non-SAP sources into the vendor's in-memory appliance (SAP High-Performance Analytic Appliance [HANA]) has added data delivery support for emerging analytics environments and "big data." Replication services included with HANA provide heterogeneous database replication support for loading transaction data into HANA.
• SAP's brand recognition, global presence and huge installed base, including BusinessObjects and Sybase customers, give SAP many opportunities to push its data integration capabilities and to cross-sell to its acquired customers.

Cautions

• SAP supports multiple data delivery styles, although the majority of reference customers report that SAP's data integration tools are largely deployed to support bulk data movements. While implementations indicate some adoption of data federation and message-oriented integration, a limited proportion of reference customers use these capabilities extensively. Sybase Replication Server's support of CDC and data replication/synchronization continues to reflect strong adoption. Implementations, which increasingly require the ability to interoperate flexibly between multiple data delivery styles, are increasing the need for SAP to have a cohesive offering. Tightened integration between Data Services and Data Federator is part of SAP's effort to meet such demand.
• References reflect uneven experiences with interoperations between data integration tools and the rest of the SAP technology stack. Customers report concerns about costliness of implementations when making products work together. Tightened product integration in the 4.0 release promises to improve the interoperability of SAP's Data Services, BI platform and enterprise applications. However, a shortage of developers skilled in SAP's newly introduced product functions has caused frustration for customers wanting to implement complex functionality. SAP has published project accelerators, best practices and methodology documentation to foster skills and wider community support. Reference customers describe self-help functions available through the vendor's support portal as time-consuming and difficult to use when initiating and opening support cases, and further undermined by the unintuitive navigation required to obtain software downloads. However, continuous improvement efforts in SAP's customer support environment are under way to facilitate the delivery of fixes, updates and product-related downloads.
• Concerns about SAP's data integration offerings becoming too tightly attuned to SAP applications have caused some customers to limit their investments. SAP's stated road map is to pursue data integration with environment-agnostic solutions (providing equal support for SAP and non-SAP data structures and applications), and it affirms the vendor's ongoing support for the movement of data between technologies of diverse non-SAP environments. Reference customers cited pricing and licensing complexity as potentially complicating the growth of SAP's footprint in existing accounts. However, SAP has recently reduced the number of pricing options to improve understanding and simplify licensing selection.

SAS/DataFlux

Cary, North Carolina, U.S.

www.sas.com, www.dataflux.com

Products: Enterprise Data Integration Server, Data Management Platform, Federation Server, SAS for Data Migration, SAS/Access, DataFlux Integration Server

Customer base: estimated at over 13,000

Strengths

• For several years, SAS/DataFlux has had a strong vision for the importance of data management as a strategic discipline and the role of data integration tools in support of data management goals. Building on the success of the DataFlux brand in the related market for data quality tools, SAS/DataFlux has delivered the range of product capabilities to support that vision. With the release and ongoing enhancement of the Data Management Platform, SAS/DataFlux now addresses enough data integration styles to compete more directly with the market's established leaders.
• SAS's size (personnel, revenues and customer base), global presence and long experience supporting data integration activities give it a solid position. In 2010, SAS reorganized its data management development efforts under the DataFlux brand, and combined its data integration, data quality and MDM capabilities to launch the DataFlux Data Management Platform. Although the DataFlux Data Management Platform is new and not widely adopted, SAS's aim to link quality and governance strongly with data integration capabilities is well aligned with market trends and positions the company well to capitalize on growing demand.
• SAS/DataFlux can draw on SAS's long history as an innovator of analytic applications and technology to capture opportunities relating to the growing demand for data integration in support of "big data." Recent alliances with data warehouse appliance vendors such as Teradata and EMC/Greenplum, plans for integration with various others, and forthcoming support for Hadoop will help SAS/DataFlux capitalize on this trend. The SAS/DataFlux product road map includes expanded functionality for message-oriented data movement, support for unstructured data, and richer metadata management to manage business terms and relationships in support of data governance efforts.

Cautions

• The DataFlux Data Management Platform is relatively new, and has yet to achieve significant adoption. Also, although customers continue to use the individual products that existed before the release of the DataFlux Data Management Platform (SAS Data Integration Server and DataFlux Integration Server), few are using the full breadth of the new product. SAS/DataFlux will need to demonstrate broader adoption of combinations of data integration functionality and newer capabilities in the product set (e.g., CDC and data federation) within individual accounts.
• Many customers cite as challenges the complexity of the products, a longer learning curve, and longer times to implementation. These factors, combined with perceptions of high price points, cause the customer base to be less satisfied with this vendor's pricing model and value relative to cost, in comparison with other vendors. While SAS/DataFlux has moved quickly to expand its product portfolio and work toward unification with the DataFlux Data Management Platform, it will need to continue to focus on delivering a total cost of ownership that is more attractive to customers. In a recent survey of users of data integration tools that had conducted a competitive evaluation of multiple vendors, the issue of cost of ownership was the most common reason for SAS/DataFlux losing out to competitors. The vendor is taking steps to address these concerns, including expanded training options, more professional services support, and on-demand and subscription-based pricing and delivery models.
• To supplement the breadth of its functionality and mitigate some of challenges presented by the complexity of its offerings and the longer learning curve, SAS/DataFlux must work to increase the availability of resources with deep knowledge of its tools. Many reference customers indicate that it is a challenge to find people with technical knowledge of SAS/DataFlux's tools outside the vendor's own professional services business. In relation to this challenge, some customers indicate a desire for improved training and more options for procuring training on SAS/DataFlux data integration tools.

Syncsort

Woodcliff Lake, New Jersey, U.S.

www.syncsort.com

Products: DMExpress

Customer base: over 1,000

Strengths

• Syncsort continues to provide high-performance bulk-batch (ETL) capabilities with attractive cost of ownership and faster time to implementation than many competitors. These strengths continues to benefit the company, given the demand for targeted functionality and superior time to value. Although DMExpress is less mature in advanced functional capabilities such as metadata management and data quality, its ease of use and scalability prompt customers to select it for targeted implementation scenarios. With version 6.5, Syncsort expanded DMExpress's functionality by including metadata interchange capabilities and support for Hadoop.
• With 40 years' experience in high-performance data processing, sustained profitability and a large and loyal customer base, Syncsort has a solid foundation on which to grow its market presence. Syncsort continues to evolve its management team and strengthen its organization in general, by attracting experienced resources from competitors. Syncsort offers a high quality of service and support, and many customers identify product technical support and their overall relationship with the vendor as positives. In 2010, Syncsort grew its revenue at a rate well above the market average, and it has continued to expand its customer base in 2011. The creation of a customer advisory board for DMExpress also reflects Syncsort's growing maturity.
• DMExpress users often have investments in tools from the market leaders or other competitive vendors, and use Syncsort's technology to fine-tune the performance of the end-to-end processes supported by such vendors. A growing percentage of Syncsort customers (65% in a recent sample of reference customers) state that DMExpress has become a standard in their organizations. Although partnerships with vendors that offer extended functionality (for example, Attunity for CDC and Trillium Software for data quality) allow Syncsort to position itself for broader demand, its own capabilities remain very ETL-centric. Syncsort continues to build partnerships with system integrators by adding relationships with Wipro, Tata, Accenture and HP, and deepening its relationship with Cognizant.

Cautions

• Although Syncsort has added metadata functionality, it must continue to invest in this area. Reference customers continue to cite metadata management as an area of relative weakness, and Syncsort exhibits limited functionality and vision for metadata, in comparison with key competitors. With modeling and metadata management functionality being perhaps the most critical capabilities to support a comprehensive information infrastructure, Syncsort must rapidly advance its technology if it wants customers to perceive it as a thought-leader with strategic technology.
• Although Syncsort's product road map includes value-added enhancements, its vision remains focused on bulk-batch data movement. It does not directly support additional styles via its own technology; the Attunity partnership, for add-on real-time CDC functionality, is at an early stage; and a relationship with Composite Software for data federation remains unproven in actual implementations. In addition, Syncsort must address a broader set of customer demand trends beyond high performance (for example, the ability to deal with greater levels of complexity in business rules for data transformation, and the ability to understand and use less-structured data sources). Syncsort will face significant challenges in expanding beyond a niche position in the market as demand continues to shift toward support for a range of data delivery styles, and as data integration tools convergence with related markets such as application integration infrastructure.
• To date, Syncsort has not offered cloud-based deployment models, an area of the market where activity is starting to grow. Although not required to support Syncsort's current strategy and positioning, cloud capabilities could also be seen as supporting the ideal of rapid deployment and lower cost — and therefore Syncsort might be missing a potential opportunity. In addition, data integration functionality will have an important role in the emerging PaaS market, and Syncsort's lack of focus in this area represents a weakness of vision.

Talend

Los Altos, California, U.S. and Suresnes, France

www.talend.com

Products: Talend Open Studio, Talend Integration Suite, Talend Integration Suite MPx, Talend Integration Suite RTx

Customer base: over 2,500

Strengths

• Talend positions itself well, with both a subscription-based data integration product (Talend Integration Suite) and a freely downloadable open-source offering (Talend Open Studio), to appeal to different segments of the data integration tools market. Talend's customer base has grown strongly and the vendor is gaining mind share in the market. The launch of Talend Unified Platform in May 2011 marks a step forward in improving the synergy between Talend's data integration and application integration capabilities, which aligns with demand trends. Application integration capability was added to Talend's portfolio through the acquisition of Sopera (see "Talend's Acquisition of Sopera Affirms Integration Market Convergence"). Reference customers identify as a strength Talend's single runtime platform supporting data integration, data quality and MDM in support of the diversity and convergence of data management activities.
• Low prices relative to most competitors are a big factor for Talend in terms of attracting initial interest. Positive customer perceptions of value relative to cost continue to generate adoption, and implementations are demonstrating increased acceptance of the features and functionality of Talend's tools. Customers report good connectivity, significant compatibility in open-source environments in general, and the ability to support custom transformation logic using Java. The addition of native support for HDFS, Hadoop Hive and Hadoop Pig provides data ETL for Hadoop processing to support "big data." Talend has also added support for data sources in cloud environments and cloud deployments of data integration processes.
• Reference customers generally report ease of use and speed of deployment as strengths of Talend's technology. They also consider the configurability of Talend's tools to be flexible enough to adapt to the business requirements of data integration processes. The availability of artifacts built by Talend's tool community has contributed to high developer productivity.

Cautions

• In a competitive data integration tools market landscape that includes of large, established vendors, Talend lacks the brand recognition and financial strength of key competitors. Talend has an established presence in Europe and North America, but little or none elsewhere, though the addition of staff in Japan and China is a step toward addressing its lack of global presence. In comparison to the market leaders, Talend skills remain less prevalent inside large system integrators and in the market in general.
• Many reference customers report that Talend's tools are largely deployed to support bulk/batch-oriented data delivery, although the vendor also supports real-time delivery modes for data replication and message-based integration requirements. Talend will need to make its capabilities more visible in the market in order to increase adoption for delivery styles beyond bulk/batch data movement and keep pace with demand trends. To address other styles of data delivery, such as data federation, it must also broaden its tool functionality.
• Reference customers report difficulties with version upgrades, software instability, and concerns about the level of documentation and support. However, these are common issues for young vendors seeking to grow rapidly in a competitive market that requires them to adjust their focus in order to achieve sustainable adoption.