By Martin Ekpeke
The adoption of data streaming is surging as large enterprises race to build foundations for Artificial Intelligence (AI) and real-time decision-making. However, a new study suggests this rapid expansion is creating a fragmented and complex technical landscape that many organizations are struggling to govern.
According to new research commissioned by Conduktor, an intelligent data hub for streaming data, the vast majority of large enterprises are now juggling multiple systems to handle their data flows.
The Rise of Multi-Platform Architectures
The study, which surveyed 200 senior IT and data executives at companies with annual revenues exceeding $50 million, highlights a clear trend toward decentralized infrastructure:
- 93% of respondents report using multi-platform data streaming architectures.
- 77% are using more data streaming platforms today than they did two years ago.
While streaming is becoming the “data backbone” for connecting diverse systems and improving customer service, the report indicates that consolidation is rare. Only 16% of organizations have managed to consolidate their data streaming down to a single platform, while a mere 7% use fewer platforms than they did in 2023.
AI Driving the Demand
The primary driver behind this acceleration is the need to support Artificial Intelligence strategies. Real-time data is required to feed AI models, improve operational processes, and enable faster decision-making.
However, the report warns that while usage is up, most organizations lack the necessary governance, quality control, and observability to make these diverse systems work together seamlessly.
“Although most respondents report a greater use of data streaming (especially for powering AI with real-time operational data), the unchecked spread of data streaming platforms can create problems for organizations,” said Nicolas Orban, CEO of Conduktor.
“This forces companies to consolidate platforms to reduce architectural complexity, lower costs, and simplify governance. To truly realize the return on investment (ROI) from their streaming investments, organizations need to take governance, performance, and security seriously from the beginning.”
Market Outlook
The findings by Conduktor align with broader market data regarding the explosion of real-time data processing.
According to Dataintelo, the global market size for streaming data processing system software was valued at approximately $9.5 billion in 2023. This figure is projected to skyrocket to roughly $23.8 billion by 2032, representing a compound annual growth rate (CAGR) of 10.8%.
Dataintelo attributes this growth to the exponential increase in data generated by social media, IoT devices, and enterprise systems. As the volume of data grows, so too does the pressure on organizations to manage the “sprawl” of platforms required to process it.