CDN

CDNs reduce physical network transmission latency by deploying edge nodes globally to cache static assets on servers closest to users.

CDN accelerates access by deploying a large number of edge nodes (PoPs) globally or nationwide and caching content at the node closest to the user.

将图片上传到阿里云OSS并用CDN分发是解决微信小程序代码包超过2MB限制的业界标准解法

现代CDN支持动态加速、HTTPS卸载、HTTP/2和HTTP/3协议支持以及边缘计算能力

Game publishers precompile the shaders corresponding to each hardware combination at the build stage, distribute them with the game or deliver them via CDN, and players directly pull the matching precompiled cache when launching the game.

Vercel's edge network is built on a CDN, but further supports executing server-side logic at edge nodes (such as edge functions), not just caching static files.

Fixing firewall blocking requires allowing mainstream AI crawlers' User-Agents in firewall or CDN rules, or distinguishing legitimate crawlers from attack traffic via IP whitelists or verification mechanisms.

CDN caching follows a key-value model: the cache key is usually the full URL of the request (including the query string), the value is the response content, and on a hit the cached value is returned directly without going back to the origin.

The root cause is that the CDN caches based on the full URL, but Slack's link unfurling bot strips the query string before requesting the image in some clients, causing different content to collapse onto the same cache key.

The CDN cache layer is usually opaque to developers: developers only see the request they send, whereas the CDN sees the request after it has passed through client, middleware and proxy layers.

robots.txt only constrains well-behaved crawlers and does not constitute access control; actual blocking often comes from bot management rules in a CDN or WAF.

In the container image distribution chain, the registry often uses 302/307 redirects to point blob requests to object storage or a CDN, and the client then needs to resolve the hostname of the new redirect target.

CDN or WAF bot rules may return 403 to crawlers allowed by robots.txt, causing accidental blocking.

Division of labor among CDN, WAF and bot management layers: the CDN caches and distributes content to nearby locations, the WAF blocks malicious traffic, and the bot management layer is a policy layer deployed on top of both to specifically identify and handle automated traffic.

Using a bare Python-urllib User-Agent or no User-Agent results in a 403 from the CDN edge and the request does not reach the server; requests, httpx, aiohttp, Go, node-fetch, axios, and curl all have no problem.

Under default configurations, many CDNs/WAFs place known AI crawlers into a recognized bot category and apply a challenge (returning 403, 429 or a JS challenge page) rather than letting them through directly.

Testing AI crawler reachability by spoofing user agents locally has limitations: some CDNs verify bots by IP, so a fake GPTBot may be blocked while the real one is allowed through, and local blocking results are therefore only a strong hint that must be confirmed with CDN logs and origin logs.

A CDN distributes static and cacheable content to nodes around the world, so user requests are answered nearby and the origin therefore does not see most of the traffic; a WAF performs rule matching on requests at the nodes, and can allow, challenge, rate-limit or block based on conditions such as path, method, request headers, User-Agent and IP reputation.

The three-layer defense principle is: reject bad traffic as early and as cheaply as possible, and assume that every layer will occasionally miss. Blocking at the edge costs nothing, blocking at the load balancer costs little, requests reaching the Pod cost the most, and the runtime layer is the last line of defense.

In CDN scenarios, the ETags returned by different nodes may be inconsistent.