Your AWS Apps Can Query Live Kafka State With Lightning Tables

If you keep Redis or a Postgres table beside Confluent Cloud on AWS only to serve "current state" reads, Lightning Tables may replace part of it, if those reads are single-table key lookups. Confluent, now an IBM company, made Lightning Tables generally available on AWS at Current 2026 in San Francisco this week. It is the only new thing you can buy from the "Streamhouse" launch. The other two parts of the new Query pillar, Tableflow and the Real-Time Context Engine, were already generally available before the show.

So the decision in front of a data platform leader is narrow. Can a read-only, per-topic table with no joins and no aggregates answer the questions your apps and agents ask today? If so, pilot it. If not, keep the serving database you have.

What Confluent Shipped at Current 2026

Confluent shipped one new product and a new name for a group of existing ones. A Lightning Table, in Confluent's definition, materializes a Kafka topic into a table optimized for fast lookups, fully managed, so you write no consumer and run no Flink job to keep it current. Confluent says the table is updated within 100 milliseconds of new data arriving in the topic, and you can query it from the Cloud Console, an HTTP API, an MCP client such as Claude Code, the Confluent CLI or JDBC.

Techzine's write-up adds Confluent's internal test figures: 10 to 100 ms query latency and more than 1,000 queries per second. Those are vendor numbers from Confluent's own testing, with no customer named and no published test setup. Treat them as a ceiling to verify in your own account.

Around that product, Confluent added "Query" as a fifth pillar of its platform next to Stream, Connect, Process and Govern. Its Streamhouse query surface page lists what sits under it: Lightning Tables for low-latency reads, Snapshot Queries that span live topics and historical Iceberg data in one statement, Tableflow for lakehouse tools, and MCP through the Real-Time Context Engine, which the page calls an access pattern and not a separate product. All are marked generally available.

Two of those are not new. Confluent announced Tableflow's general availability on March 19, 2025, and Techzine reported the Real-Time Context Engine as generally available on May 19, 2026. Tableflow also publishes to its tables only approximately every 5 minutes, so it is the wrong tool for any agent that needs the state of the last second.

---

What a Streamhouse Is, and Who Owns the Word

A Streamhouse is a data architecture that keeps the current state of a business continuously available to production applications and AI agents. That is the definition from the Streamhouse Working Group, which Aiven, Confluent, Redpanda, StreamNative and Ververica formed on September 15, 2026, with a binding commitment that the name stays free for anyone to use.

That matters for procurement. "Streamhouse" will appear in Redpanda and Aiven decks too, so the word tells you nothing about which product you are buying. Ask what the query layer is and what it can execute.

Confluent's own people describe it as a layer next to your lakehouse. In an October 20 diginomica interview, Confluent's EMEA CDO Peter Pugh-Jones said customers were already building these themselves and often ended up needing "some sort of database at the end" of a streaming flow. That database is the one Lightning Tables is aimed at.

The strongest case for the launch comes from Confluent's VP of AI product Sean Falconer, who argued at the show that teams no longer need to copy data into a separate database or warehouse to query it. Fewer copies means fewer sync jobs and less drift between what Kafka says and what the agent reads. It matters most for a fraud check or a rebooking agent, which acts on whatever it reads.

What Lightning Tables Cannot Do Yet

Lightning Tables answers single-table key lookups, filters and range queries, and rejects everything else. The SQL reference says a query is read-only, runs on a single Lightning Table, and does not support aggregate functions, GROUP BY, joins, subqueries, set operators, or any DDL or DML. Confluent Cloud rejects an unsupported query without running it.

That rules out most of what a Postgres read replica does for you today. "Show me the customer, their open orders and their loyalty tier" is a three-table join. To serve it from Lightning Tables you would first build the joined view in Flink, write it to a topic and enable a table on that topic. That is extra pipeline work you own, and the joined topic has to be keyed for upsert.

The other limits a platform team will hit:

1. It runs on AWS only. Lightning Tables are available on Basic, Standard, Enterprise and Dedicated clusters on AWS. Nothing in the docs gives an Azure or Google Cloud date.
2. The topic needs a registered Avro, Protobuf or JSON Schema, and enabling a table takes several minutes.
3. To get "latest record per key" behaviour, the topic's cleanup policy must include compact and records must carry a key; otherwise the table is append-only and keeps every event.
4. You can't run queries in the Console or with the CLI over a private network. The HTTP API and JDBC work over PNI or PrivateLink, and MCP queries go through the Real-Time Context Engine over VPC peering, PrivateLink or Transit Gateway.

The one that will cause an incident is schema evolution. If a producer drops or renames a column, the Lightning Table stops ingesting and queries return an error, while the table's status stays "Active". Disabling and re-enabling the table does not fix it. You either re-register the old schema or move to a new topic with a Flink copy job. So your health check has to watch query errors and ingestion lag, because the status field will show green the whole time.

---

What It Costs, and Who You Are Negotiating With

Confluent has published no price for Lightning Tables. The docs say it is usage-based, billed on the compute and storage you use, and the Confluent Cloud pricing page lists cluster rates (Standard at $0.75 per eCKU-hour, Enterprise at $1.75 to $2.25) but no Lightning Tables or Tableflow rate, as checked on November 5, 2026. You cannot model the cost of replacing a Redis cluster until you run a pilot and read the usage monitor.

The counterparty has also changed. IBM closed its roughly $11 billion purchase of Confluent in March at $31 a share, and at Current, Jay Kreps shared the stage with Shaun Clowes, now GM of Confluent and Data at IBM. IBM has said Confluent is being wired into watsonx.data, IBM Z, IBM MQ and webMethods. We covered what that deal means for existing contracts in IBM Paid $11B for Real-Time AI.

The steel-man for waiting is simple. A Streamhouse product that sits between your stream and your agents is also a new place for your bill to grow, from a vendor that now sells you a mainframe and an integration suite. Diginomica's Derek du Preez noted that a layer which durably holds live business state sits closer to a system of record than a pipe does, even though Jay Kreps has said Confluent is not building one. If you move serving reads onto Confluent, your exit cost goes up with it.

Where Lightning Tables Fits Against Your Current Serving Store

Lightning Tables fits best where you already keep a keyed, compacted topic and a cache that mirrors it. Order status, shipment position, account balance flags, feature values for a fraud model and the "current state" record an agent fetches before it acts all fit that shape. You can delete a consumer, a cache and the code that keeps them in sync.

It is a poor fit for anything relational. Reporting, cross-entity questions and anything that aggregates still belong in a warehouse or an operational database, and the Streamhouse group's own line is that it can "coexist" with a lakehouse. If your data platform choice is still open, our Snowflake alternatives and Databricks alternatives pieces cover the analytical side.

For agents, the MCP route is the interesting part, and it carries the same governance work as any other MCP server. A generic data connector can do worse than no tool at all, as UNICEF's MCP benchmark showed, and the allowlist has to be enforced at the client, as we argued in MCP server governance. Put the Lightning Table MCP endpoint through the same review as any other server that hands production data to an agent.

---

What to Do Before Your Next Confluent Renewal

This Week:

1. List every store you run beside Kafka purely to answer "current state" reads: Redis, a Postgres replica, DynamoDB. For each, note whether its queries are single-key lookups or joins.
2. Check whether your Confluent Cloud clusters run on AWS and use private networking. If your agents would call from inside a VPC, plan on the API, JDBC or MCP path, since the Console and CLI will not query over it.

This Month:

1. Pick one keyed, compacted topic with a registered schema and enable a Lightning Table on a Standard or Enterprise cluster. Point a shadow copy of one app's reads at it and compare latency and error rate to the store it would replace.
2. Add an alert on Lightning Table query errors and ingestion lag before anyone ships a schema change. The table status will not tell you it stopped.
3. Read the usage monitor after two weeks and turn it into a monthly figure next to the cost of the cache you would retire.

Before Renewal:

1. Ask IBM and Confluent for a written Lightning Tables rate card and an Azure and Google Cloud date, if you run there.
2. Ask what happens to Confluent Cloud pricing and support terms as the product folds into watsonx.data, and get the answer into the order form.

The Bigger Picture

Streaming vendors are moving toward the serving layer. Pugh-Jones told diginomica that "the operational estate is beginning to encroach on that analytic space", and five competitors agreed in September to share one category name so no single company controls it. Lightning Tables is Confluent's first product in that space, and it is deliberately narrow: one table, read-only, keyed lookups, with an MCP endpoint attached for agents. Whether it replaces anything in your stack depends on how many of your agent and app reads really are single-key lookups.

Start with the inventory from the first This Week item, and pilot only on the topics where every read is a key lookup.

Continue Reading

• IBM Paid $11B for Real-Time AI: The Confluent Deal Explained
• MCP Server Governance: Enforce at the Client, Not the Registry
• MCP vs A2A vs OpenAPI: Which Agent Integration Standard to Use
• Snowflake Alternatives: Fix the AI Bill Before You Move a Table
• Databricks Alternatives: How to Cut Costs Without Migrating
• Snowflake Takes Cortex AI Gateway GA, Previews Business Ontology