Today, we’re introducing Eon Yukon, a high-performance data layer for agents, analytics, and AI. It combines ingestion, processing, table maintenance, and real-time serving to keep enterprise data fresh and ready to use, while continuously building the context agents need to understand it.
It does this with substantially less infrastructure. In our continuous ingestion benchmark, the engine maintained near-real-time freshness using 4-7x less compute than required by Apache Spark or Flink.
Making data useful takes more than moving it. Tables need continuous maintenance. Applications need fast access to live state. Agents need context about what data represents and how it has changed. Eon Yukon brings this work together, reducing repeated processing and the separate systems and jobs teams need to manage.
With Eon Yukon, teams can:
- Prepare clean, current data as it arrives: Combine ingestion, SQL transformations, deduplication, and continuous table maintenance in one serverless pipeline, with automatic schema evolution, compaction, sorting, and cleanup. No servers to configure or manage.
- Maintain summaries without full rebuilds: Update configured aggregate tables alongside event-level tables they summarize in the same ingestion pass, keeping customer profiles, usage metrics, and operational dashboards current without separate aggregation jobs.
- Give agents context before their first query: Capture column statistics and changes during ingestion, maintain descriptions informed by data profiles and supplied business definitions, and expose context through MCP.
- Query data and serve live applications: Run serverless SQL, or bring existing Spark code through Spark Connect. Enable applications to read the current state of aggregates with sub-millisecond latency through Redis-compatible or HTTP API.
- Keep data open and in your own cloud: Store data as open Iceberg tables in your Amazon S3 or Google Cloud Storage. Use supported query engines and your existing catalog while Eon manages ingestion, data preparation, and continuous table maintenance.
Your data is your AI advantage. Make more of it usable.
AI creates new opportunities to put enterprise data to work. But each new use case can add another pipeline, processing job, or serving system to maintain.
Teams stitch together separate systems for ingestion, transformations, aggregations, table maintenance, querying, and application serving. Each adds compute to manage, jobs to coordinate, and another place where data can fall behind.
Supporting more workloads shouldn’t require that complexity to keep growing. That’s the architectural challenge we set out to solve. Eon Yukon does more work as data arrives, while letting writing, maintenance, and serving scale independently.
How Eon Yukon works
Prepare and maintain data continuously
Built in Rust, the engine uses columnar processing and routes files to the appropriate writer before reading them, reducing network data movement. It separates writing from committing and stores ingestion progress in object storage without global checkpoints, helping compute stay efficient and scale without restarting the pipeline. Work is balanced by data volume, allowing larger partitions to use more workers rather than overload one.
Transformations and configured aggregates run during ingestion. Deduplication, upserts, and deletes are resolved at each commit without rewriting underlying data files. Compaction and sorting run continuously on independently scaling compute, keeping tables organized without delaying commits or requiring separate maintenance jobs.
Eon writes open Iceberg tables in your Amazon S3 or Google Cloud Storage through a customer-provided role. Data stays in your storage and remains accessible through supported Iceberg query engines and your existing catalog, so you can use your existing tools without tying data access to Eon.
Build context as data arrives
As data arrives, Eon profiles each column to identify common and rare values. It also keeps a smart sample of every table that preserves rare and important rows. Agents use it to answer questions about the data in seconds, without scanning the full table.
From all this, agents keep a living document for each table that captures:
- What it holds and what each column means
- Which business metrics matter
- Where those metrics stand out
- What changed and when, such as refunds doubling in one country after a pricing change
The more agents learn about a table, the better the sample gets. Your agents read the document over MCP, so they understand the data before they query it and can act on what changed.
Query data and serve live applications
Eon’s distributed, serverless query engine supports SQL and Spark-compatible DataFrames. It uses ingestion-time statistics to answer supported statistical queries without scanning records, while indexes accelerate lookups and searches. Execution history helps refine query plans and size compute for subsequent tasks, without teams provisioning or managing query clusters.
For live applications, Eon serves the latest state of configured aggregates by key in under a millisecond through a Redis-compatible or HTTP API. Hot keys sit in memory, warm keys on local disk, and the remaining data stays in object storage. Read replicas and sharding support high-volume workloads. Applications and agents can access these live aggregates without a separate pipeline to synchronize them into Redis or Aerospike.
Measuring the cost of keeping data fresh
Ingestion throughput alone does not show what it takes to keep open-format tables usable. Frequent commits improve freshness but produce small files that slow reads and require compute to compact. A useful benchmark needs to account for both getting data into the table and keeping it efficient to query.
We built ingest-bench to measure that trade-off across freshness, file sizes, and compute for ingestion and compaction.
Less compute at the same freshness
We compared Eon Yukon with Apache Spark and Apache Flink at one million events per second—over 1.4 GiB/s—from Kafka into Iceberg. Each engine received the same input for a full hour. We tuned each engine and tested successively smaller fleets. We compared the smallest fleet for each engine that met the same correctness and freshness requirements.
Verification is engine-agnostic: the benchmark checks the resulting tables against the produced data and its timing, rather than relying on engine-reported metrics.
Eon Yukon used roughly one-quarter of Spark’s ingestion compute and one-seventh of Flink’s. The advantage held with skewed input as Eon kept up without a larger fleet even when a large share of events went to one key.
Greater savings when table maintenance is included
With both engines targeting the same file size, Eon Yukon’s integrated ingestion and compaction used less than one-quarter of the compute required for Spark ingestion plus a separate Spark compaction job. The efficiency advantage grew when we accounted for the ongoing work of maintaining the table.
Efficient updates as records change
In the upsert test, Eon Yukon maintained the latest version of each record with about one-third of the compute and one-quarter of the memory compared to Spark and Flink. The gains extended beyond append-only ingestion to workloads that continuously maintain current state.
Read the full benchmark methodology and results to learn more.
Get started with Eon Yukon
Eon Yukon can be deployed independently or alongside Eon Data Protection, which makes protected data available as open Iceberg tables through Zero-ETL. Together, they bring live and protected data into a shared foundation for analytics, applications, and AI.
To learn more, visit the Eon Yukon product page or request a demo to see it in action.
