Semantics Delivery Network · HotNets '26
Decisions at the Edge
Agents consume evidence in coherent passages or chunks; however, today the web delivers in pages for human readers. We propose a new network infrastructure serving evidence chunks to answer the query, unifying the utility of traditional search engines and CDNs in one round. Instead of a finer-grained CDN delivering semantic-oblivious bytes, a semantic-aware network infrastructure creates new opportunities as well as challenges. This artifact guides the audience to step through the decisions an edge has to make.
One request, three paths
Compare the API rate limits of Halden and Quillbase
The middle path is Cloudflare's per-site AI Index1. AI Search is the query path over that index2. AISTRAL's Agentic Twins3 also give each site its own index, and add a resolver that picks which sites to ask. Ranking across sites is still left to the agent.
What one query costs today
Median of 100 live queries through a SERP API and its top-5 URLs4. The no-JS fetch moves 107× what the model reads; a full render, 2,532×. Bars use a log scale.
Two systems, split down the middle
| Search engine | CDN | SemDN |
| Keyed on | Query | URL | Query |
| Returns | URLs and snippets | Whole objects | Ranked chunks with provenance |
| Sees the query | Yes | No | Yes |
| Serves the origin | No | Yes | Yes |
| Paid by | Client | Origin | Both, for different services |
The unit it returns is called a chunk. It can be a text passage5, a table block6, 7, an image region8, 9, or a page screenshot10. We currently focus on passages and tables. Tessera rebuilds a web page into evidence units and completes a table row with its headers7. Other work explores images and screenshots. ColPali and CLIP retrieve image regions8, 9. Computer-use agents operate on the rendered page11, 12, 13, 14, 15, and PixelRAG retrieves from the screenshot itself10. The same idea extends to an audio clip16 or a video segment17.
text passage 5table block 6, 7image region 8, 9page screenshot 10audio clip 16video segment 17
References
- Cloudflare. An AI Index for all our customers. 2025.
- Cloudflare. AI Search.
- Yixi Chen, Muhammad Bilal, Tie Ma, Zafar Ayyub Qazi, Marco Canini, and Suhaib Fahmy. From Crawled to Published: A Publisher-Controlled Web Architecture for an AI-Native Internet. arXiv:2511.18354v2, 2026.
- Peichun Hua and Yunming Xiao. Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents. HotNets 2026. The byte measurements.
- Vladimir Karpukhin et al. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.
- Jonathan Herzig et al. Open Domain Question Answering over Tables via Dense Retrieval. NAACL 2021.
- Peichun Hua, He Sun, Zirui Zhuang, Qi Qi, Yiming Qiu, Shinan Liu, and Yunming Xiao. Tessera: Structure-Completed Evidence Units for Web Retrieval-Augmented Generation. arXiv:2610.37080, 2026.
- Manuel Faysse et al. ColPali: Efficient Document Retrieval with Vision Language Models. arXiv:2407.01449, 2024.
- Alec Radford et al. Learning Transferable Visual Models From Natural Language Supervision (CLIP). ICML 2021.
- Yichuan Wang et al. PixelRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation. arXiv:2606.28344, 2026.
- Anthropic. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku. 2024.
- Google DeepMind. Introducing Gemini 2.0: our new AI model for the agentic era. 2024.
- Jing Yu Koh et al. VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks. ACL 2024.
- OpenAI. Computer-Using Agent. 2025.
- OpenAI. Introducing ChatGPT Agent: Bridging Research and Action. 2025.
- Yusong Wu et al. Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (CLAP). ICASSP 2023.
- Huaishao Luo et al. CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval and Captioning. Neurocomputing, 2022.
Further reading (11 papers, click to expand)
Where this idea comes from
CDNs and the network side
- Erik Nygren, Ramesh K. Sitaraman, and Jennifer Sun. The Akamai Network: A Platform for High-Performance Internet Applications. ACM SIGOPS Operating Systems Review, 2010. How a large CDN is built, the system SemDN is compared against throughout.
- Waris Ali et al. A survey on the state-of-the-art CDN architectures and future directions. Journal of Network and Computer Applications, 2025. A recent survey of CDN architectures.
- Yihua Cheng et al. Do Large Language Models Need a Content Delivery Network?. arXiv:2409.13761, 2024. Proposes a delivery network for LLM knowledge in the form of KV caches, a neighbouring idea.
- Siddhant Ray et al. An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age. arXiv:2608.01526, 2026. Asks which Internet infrastructure should carry KV caches between models.
- Muhammad Bilal et al. Toward an AI-Native Internet: Rethinking the Web Architecture for Semantic Retrieval. arXiv:2511.18354, 2025. Argues for an AI-native web that serves semantic content, close in spirit to SemDN.
- Yazhuo Zhang et al. Rethinking Web Cache Design for the AI Era. ACM SoCC 2025. Revisits web cache design when AI crawlers and agents are the clients.
- Cloudflare. Introducing Markdown for Agents. Cloudflare blog, 2026. A CDN serving Markdown instead of HTML to agents: the first step from bytes toward text.
Four structural mismatches
The stack was built for human readers
Similar is not useful
Pick the passage you would hand to the agent. Scores are query-passage similarity.
Chunking is a budget decision
The reader has a context budget. Spend it on chunks or on pages?
Paper Fig. 3c2: CRAG Task-31, 600 questions, Qwen3-32B reader. With the same LLM reranker, chunks beat pages by 3.5–8.2 accuracy points from 256 to 6,144 tokens.
Same question, same 256 tokens, same reader
Page-first cannot reach what it never downloads
The tiles are pages in a page ranker's order. Marked tiles hold a chunk from the global top-5. Download more pages and see how far you get.
16–29% of the global top-5 chunks lie on pages a whole-page ranker skips, even at 20 pages2. Chunks match or beat a client that downloads ten pages (0.456 vs. 0.427 accuracy with e5), which moves about 1,200× the bytes.
Every client runs the same fragile pipeline
Reach, extract, de-boilerplate, segment, embed. Run it on 40 result URLs, then multiply by the number of operators doing the same work.
Over half the text is lost without JavaScript on at least 39% of sampled URLs, and about 19% are bot-blocked. Across 1,430 live URLs, a cloud renderer gets usable clean text from at most 75%3.
Real queries, real top-5 URLs
Difference 1 · Misses
Your index always answers. Is the answer good enough?
A URL lookup tells you when it misses. Top-k search returns k results even when better evidence exists elsewhere. Thirteen queries arrive at your PoP. Decide what each one gets.
Edge PoPHot content and index state
Regional parentLarger, colder shards
Global tierThe distributed catalog
External discoverySERP results, filtered by the enrollment registry
More context does not fix old evidence
Correct answers on CRAG1 with reranked chunks, by how fast the truth changes2. Spending twelve times the budget helps stable questions. Real-time questions barely move, because their answer is missing from the pool.
| The truth changes | Questions | 512 tokens | 2,048 tokens | 6,144 tokens |
| Never | 340 | 65.0% | 68.5% | 70.0% |
| Slowly | 135 | 57.8% | 56.3% | 60.0% |
| Quickly | 55 | 36.4% | 41.8% | 45.5% |
| In real time | 70 | 35.7% | 35.7% | 37.1% |
Now hand the job to a controller
A semantic coverage miss2 is when the estimated risk that an external search would change the top-k exceeds a threshold. The buttons are the signals that estimate that risk. Each one is imperfect. For example, a sufficiency judgment checks whether the evidence holds enough to answer before any generation3. Turn them on and move the threshold to see the tradeoff in a simulation of 2,000 queries.
What each signal measures (5 signals, click to expand)
- Score distribution
- The scores on the chunks just returned, and the gap between the first and the rest. A high score with a steep drop is read as confidence. In the queries above, the seat-limit table scores weakly and still holds the answer, and Halden's chunks score 0.91, 0.87, and 0.84 while stating an old limit.
- Result coherence
- Whether those chunks agree with one another. Agreement is read as a stable answer. For HTTP 429 the chunks agree and the meaning is unchanged. For Halden they agree on the limit from last spring.
- Sufficiency judgment
- A check that reads the query and the chunks and decides whether they contain enough to answer, before a model generates anything3. It asks whether the evidence supports an answer, which is separate from how similar a chunk is to the query.
- Cross-tier agreement
- Whether this edge's top-k matches the regional parent and the global tier. A mismatch means a larger tier may hold chunks this PoP evicted. A match means the gap, if there is one, is outside the network.
- Freshness metadata
- The age and version stored with each block. The same age means different things for different queries. Six hours is stale for a status page, and ordinary for a standards document.
Discovery runs when the combined risk is above this value. Lower it and more real misses get a search. More queries that were already covered get searched too.
Pages are changing faster than usual. The same signals then separate covered queries from real misses less cleanly. How to recalibrate when that happens is open.
The first bar counts coverage misses the controller did not search. The second counts searches that would not have changed the top-k. The third is how often an external search is paid for.
Locality
Three kinds of chunk reuse
A URL cache sees low locality. Within a task, 70–100% of a search agent's URLs are new1, 2. Chunks are different.
Already seen, or on a page already opened
An exact repeat is a chunk the agent already saw. That is 41% of later chunks on Search-R1 over Wikipedia, and 70% on CRAG2, 3. Our guess: the first search missed the passage that holds the answer, so the agent only tweaks the query. Most of the same chunks come back, with a few new ones. Exact same queries are removed.
| Search-R1 / wiki-183,075 traces | CRAG1,500 tasks |
| Exact repeat | 41%32% → 55% by turn | 70% |
| Sibling, on a page already opened | 11%about 1 in 10, every turn | 10% |
| New page | 48% | 20% |
A sibling is the second kind. The edge has already fetched, cleaned, and chunked the page, so it can keep every chunk on it. About 1 in 10 later chunks is one the agent has not been shown, on a page already opened. The rest are pages the agent has not opened. These traces are Search-R1 with Qwen2.5-7B, an older agent1. We expect both effects to be stronger with a better agent and a better retriever.
The first search missed the answer
Where was the music festival Billy McFarland was charged with wire fraud for supposed to take place?
The question is where the festival was supposed to be. Turn 1 asks about McFarland and the fraud. The three passages the agent sees are the charge and the guilty plea. None of them says where it was, so the agent tweaks the query and asks for the location. Great Exuma was already on that page, at rank 40 of the first search, and was not sent. Turn 2 is when the agent receives it. That passage is both a sibling and the third kind, below.
Already retrieved, then cut off
The third kind is a chunk an earlier search already returned and did not show. The agent only sees the top 3. If the edge keeps the rest of that list, it can have the next chunk ready before the agent asks, which is the prefetch, or serve it from cache when the agent does ask.
What sport is played by both Justin Gimelstob and Angelique Kerber?
Query: Justin Gimelstob and Angelique Kerber
Justin Jeremy Gimelstob is a retired American tennis player.Justin Gimelstob1
He established himself as a doubles specialist, winning 12 titles.Justin Gimelstob2
Another passage on Justin Gimelstob.Justin Gimelstob3
The 2018 Angelique Kerber tennis season began at the Hopman Cup. She entered the season ranked 21st.Angelique Kerber, not shown4
Passages like the one just shown are more passages on Justin Gimelstob. The next query adds the word "sport", and the Kerber passage moves into the top 3.
Between Polytechnic University of the Philippines and California Polytechnic State University, which was founded as a vocational high school?
Query: Polytechnic University of the Philippines and California Polytechnic State University founding
Polytechnic University of the Philippines was founded on October 19, 1904 as the Manila Business School.Polytechnic University of the Philippines1
Another passage on the same university.Polytechnic University of the Philippines2
The University of the Philippines was founded by the American colonial government.University of the Philippines3
School was founded as a vocational high school when California Governor Henry Gage signed the Polytechnic School Bill on March 8, 1901.California State Polytechnic University, Pomona, not shown4
Passages like the one just shown stay on universities in the Philippines. The next query asks for the California founding, and this passage comes back first.
Of the chunks that first appear in the next turn's top 3, 25% were already in ranks 4–13 of the search just served. Ranks 4–50 of that same search cover 38.5%. Separately, 12% of those same new chunks are among the ten passages most similar to the single top passage that was sent.
Refresh under coverage risk
Two days at your PoP
Twelve blocks across three documents, 48 hours
The edge is holding chunks it already fetched. The pages they came from keep changing, and serving the old copy gives the agent a stale answer. Each row is one block: four from a status page, four from pricing, and four from documentation. Each column is one hour. Red means the origin changed that block and this PoP still holds the old copy. The last color in the legend is a query that was answered with that old copy. The buttons are three ways to go get a new copy.
FreshStaleQuery, hit freshQuery, got stale evidence
Three ways the query stream misleads placement
The same queries that tell you what to keep at the edge can also push the index the wrong way.
A page almost nobody asks for. A rare-disease reference gets about one query a month, so it falls out of the edge index. The next question that needs it misses.
One tenant's queries, every tenant's index. A hedge fund's agents keep requesting chunks on one small biotech. Reweighting the shared index from those queries makes that company faster for everyone, and it shows the other tenants what the fund is working on.
A chunk that used to be right. Passages that produced correct answers get boosted. A 2024 passage on a tax rule has a long record of being right. The rule changed last week. The boost, and an old benchmark, keep serving that passage.
All three are open.
Further reading (9 papers, click to expand)
Keeping copies fresh
When answers go stale
Differences 2 and 3 · Tenancy and state
Four tenants, one substrate. What can they share?
A cached page serves every client. An index serves only the clients that compute vectors the same way.
The origin's contract stays the same in all of these.
Enrollment, permitted transformations, freshness bounds, and purge. A tenant's encoder never appears in it. Its utility function and labels stay private too.
Now scale the tenants
Our Zipfian tenant model over real task-to-page sets2. Savings track overlap: the top-10 domains draw 34% of live results, while four deliberately diverse benchmarks share only 22 of 4,618 pages.
Store vectors or recompute them?
wiki-18 is 13.4 GB of text and needs a 64.6 GB flat index. Recompute-based indexes such as LEANN1 keep the text and materialize vectors during search. Where does your PoP sit?
A two-sided service
A reverse proxy and a forward proxy in one
Whose decision is it?
An origin pays a CDN so requests for its pages do not all land on its own servers. The list on the left is the condition for holding that copy. The client pays for retrieval.
Origin contract
- Enrollment
- Permitted transformations
- Freshness and version bounds
- Purge and invalidation
- Access control
- Provenance
Settled with each client
- Encoder
- Chunking policy
- Utility reranker
- Outcome labels
Open: can the origin refuse embedding?If the origin says no, a client can still fetch the page the way the web works today. A CDN serves the bytes, and the client embeds them. If SemDN refuses, the client leaves and does that, and SemDN serves less. The origin cannot enforce the ban, and honoring it inside SemDN hurts the service.
Pushback: freshness is the client's problem.The client knows when a question is time-sensitive. The Halden limit changed, and an origin bound can still serve the old number. The client can refuse those chunks and pay for a search. Only the origin knows the page changed, so the bound stays with the origin.
Pricing steers architecture
Charge a registered representation at its unshared cost and discount the canonical one, and tenants have a reason to use shareable state. Set a month's bill.
A sample price sheet: $0.10 per 1,000 chunks delivered, plus each representation's share of a $4,000 monthly index and re-embedding cost.
Further reading (8 papers, click to expand)
Who pays
What crawling costs origins today
Trust
Why would anyone trust this edge?
A courier versus a librarian
A CDN is a courier. It delivers the site's page sealed, and HTTPS shows the seal is intact. SemDN is a librarian. It cuts pages into passages, picks which ones you read, and is paid by both the site and you. An intact seal says nothing about whether the librarian picked fairly. So a client should be able to ask three questions.
1. Is this really what the site said?The site signs what it publishes, so every passage traces back to the site's own text, at a stated version. This catches made-up and outdated passages.
2. Did I get the best passages, or the ones someone paid for?The edge commits to its index and proves that what it returned really is the top of that index. This catches paid ranking and passages quietly dropped. A courier never had to answer this, because it never chose anything.
3. Was anyone left out?A ranking can be honest over an index that silently dropped a site. So the list of sites that joined is public, the way browsers check a public log of certificates3.
At the poster
Concluding remarks. Come and provide your ideas, suggestions, and comments!
Our preliminary experiments show promising results, but an end-to-end implementation is not complete yet. In principle, SemDN could sit on top of an existing CDN or be a separate infrastructure. Either way, keying delivery on queries changes three things: (1) misses must be estimated instead of keying on exact URL matches; (2) clients choose representations that origins do not see; and (3) acquisition and normalization are shared by all, creating opportunities for cache reuse via locality.
- When is cached evidence good enough to serve? Two things decide it: timeliness (is the chunk still current?) and coverage (does the evidence hold everything the query needs?). A coverage miss leaves no trace in a URL or a timestamp. How can an edge estimate both, cheaply, before it answers?
- Why not keep the cache on the client? A session cache on the client catches much of what one task re-reads. But a larger cache works better and costs the client more, and a client-side cache exposes how the provider indexes, chunks, and embeds, which may be what it competes on. Clients still need to see enough to trust the results. What belongs on the client, what belongs in the network, and what must be shared across clients to justify a network?
- Where should the encoder boundary sit? When does re-encoding stop paying off, and when do you materialize a model-specific index?
- How much query state can tenants safely share? Reweighting from one tenant's trajectories can expose its task structure to others.
- Why would anyone join first? Origins gain offloaded traffic and fresher copies at the edge. Clients gain one round instead of many. How do you get origins and clients to enroll while the network is still small? And when an origin refuses, how does the edge still serve its content well to clients?
- Why trust the edge? A client sees the chunks the edge returns, not the index behind them. Can the edge show that it returned the best chunks its index holds, without revealing the index? A simpler version, and maybe not an easy one either: can it show that each chunk is a faithful, processed copy of the origin page, with no injected misinformation, bias, hidden instructions, or ads?
- How does one query find its sources across the web? Cloudflare's AI Index1, 2 and AISTRAL's Agentic Twins3 give each site its own index. AISTRAL adds a resolver that picks which sites to ask. Its evaluation uses 10 sites, and the authors leave routing at web scale open. An answer may come from one source or need several together, and the web has millions of sites. Should a query be routed to sites one by one, or should one shared index rank chunks across sites in the client's representation? What does each cost at web scale?
Peichun Hua and Yunming Xiao, CUHK-Shenzhen. Paper on arXiv · Code on GitHub