Data streaming company Confluent, now an IBM company, has unveiled a set of new capabilities within Confluent Intelligence and Confluent Cloud designed to streamline the development and deployment of real-time AI applications. The updates focus on removing the barriers that have historically prevented AI projects from moving beyond the experimental stage into production.
The core of the announcement revolves around agent-powered workflows that allow AI to manage and debug streaming operations using natural language, alongside automated data protection features that keep sensitive information secure. Confluent positions these enhancements as the foundation for what it calls “production-ready AI” — a term that highlights the gap between building a prototype and deploying a system that can handle real-world data streams reliably and securely.
Agent-powered workflows and MCP server
Two key components power the new agent-driven approach: the Confluent Model Context Protocol (MCP) server and Agent Skills. The MCP server acts as a control plane that enables AI agents to build, manage, and debug streaming operations directly through natural language commands. This abstraction simplifies the interaction between developers and the complex data streaming layer, reducing the need for manual tool-switching.
Agent Skills add a second layer by encoding best practices and organizational workflows. When an AI agent performs operations, Agent Skills ensure those actions are executed consistently and in line with enterprise standards. Together, these tools enable developers to create and continuously improve real-time applications using AI-powered development workflows, bringing streaming into the modern agent-driven development paradigm. Both features are generally available for Confluent Cloud.
Automated data privacy and redaction
A new built-in machine learning function for personally identifiable information (PII) detection and redaction directly within Flink SQL addresses a common pain point in regulated industries. The function runs without custom code, external services, or the need to move data to a separate warehouse first. This makes it easier for organizations in financial services, healthcare, and insurance to unlock AI use cases that involve sensitive data.
The PII redaction capability is available in early access for Confluent Intelligence. By embedding privacy controls directly into the streaming layer, Confluent aims to reduce the friction between security teams and developers — a friction that, according to a McKinsey report, is a primary reason why eight in ten companies cite data limitations as a roadblock to scaling agentic AI.
Private connectivity with Azure Private Link
To keep AI workloads off the public internet, Confluent now supports Azure Private Link. This integration provides secure, private paths from Flink jobs to external models and external tables hosted on Azure services such as Azure OpenAI, Azure SQL, and Cosmos DB. Communication travels over Microsoft's private backbone, eliminating exposure to public networks while maintaining low latency. The feature is generally available on Confluent Cloud.
Unified engineering workflows with dbt integration
A free open source dbt adapter brings Flink SQL on Confluent Cloud into the dbt framework — a widely adopted standard among data engineers for building and managing data pipelines. Teams can now define, test, and deploy streaming pipelines using the same dbt commands and project structures they already rely on. This significantly lowers the barrier to adopting Flink and makes it easier to extend existing batch-oriented data workflows into real-time use cases. The dbt adapter is generally available on Confluent Cloud.
Expanded model support and anomaly detection
Confluent also announced support for TimesFM models for robust anomaly detection, as well as support for Anthropic and Fireworks AI models. Developers can use these models directly within Flink stream processing workflows, enabling sophisticated real-time AI applications such as fraud detection, predictive maintenance, and dynamic personalization.
Industry context and challenges
According to Sean Falconer, head of AI at Confluent, “Most AI projects fail before they reach a single customer because the data layer breaks down. Teams have the models and the mandate, but security risks and fragmented data stop them from shipping. We’re fixing that by making the streaming layer the foundation for secure, production-ready AI.”
The McKinsey report cited by Confluent underscores the scale of the problem: eight in ten companies report data limitations as a significant roadblock to scaling agentic AI. These limitations often stem from security teams blocking data from entering AI pipelines due to exposure risks, and developers losing hours to tool-switching to inspect and manage the data streams their AI depends on. The resulting slow, manual process turns what should be a fast iteration cycle into a bottleneck.
Confluent Cloud and Confluent Intelligence aim to address these root causes by providing a data streaming foundation that continuously processes both historic and real-time data and delivers it as trusted context into AI applications. The combination of agent-powered workflows, automated governance, private connectivity, and aligned developer tools is designed to reduce the friction that has historically prevented AI from reaching production at scale.
With these updates, Confluent is betting that enterprises will increasingly turn to streaming platforms as the backbone for their AI infrastructure — not just for batch analytics, but for the real-time, always-on applications that are becoming central to competitive advantage across industries. The integration of MCP and Agent Skills signals a move toward more autonomous data management, where AI agents can perform operational tasks that previously required human intervention.
For organizations already invested in Confluent Cloud, the new capabilities offer a path to accelerate AI initiatives without overhauling their existing data architecture. The automated PII redaction, for example, allows companies in highly regulated sectors to leverage real-time data for AI without running afoul of compliance requirements like GDPR or HIPAA. Similarly, the Azure Private Link support ensures that cloud-native AI workloads can remain isolated from public internet threats while maintaining performance.
The dbt integration, meanwhile, reflects a broader industry trend toward unifying data engineering and AI/ML workflows. By allowing teams to use familiar dbt commands for streaming pipelines, Confluent reduces the learning curve and enables existing data teams to contribute to real-time AI projects more efficiently. This alignment is crucial as organizations increasingly demand that data platforms serve both traditional analytics and advanced AI use cases from a single infrastructure.
Looking ahead, the expansion of model support — including TimesFM for anomaly detection and Anthropic and Fireworks AI models — positions Confluent as a neutral platform that can integrate with multiple AI providers. This flexibility is important for enterprises that want to avoid vendor lock-in while still leveraging the latest advances in AI. The ability to directly incorporate these models into Flink stream processing workflows means that developers can build end-to-end AI pipelines without stitching together separate tools for streaming, storage, and inference.
Overall, Confluent’s latest release addresses a critical pain point in the AI lifecycle: the gap between promising prototypes and reliable, production-grade systems. By embedding agent-driven automation, privacy controls, and private connectivity directly into the data streaming layer, the company aims to give developers and security teams a common foundation that supports both speed and safety. As enterprises continue to push for real-time AI at scale, solutions that bridge the data governance chasm will likely become a key differentiator in the market.
Source: Computerweekly News