Transloadit

What are Origin Servers?

Origin servers store or generate the authoritative version of content. Caches and content delivery networks retrieve that content from the origin when they cannot satisfy a request locally.

How Origin Servers work

HTTP defines an origin server as a program that can originate authoritative responses for a given target resource. In a delivery architecture it is the upstream that a CDN or reverse proxy contacts when it has no stored response it can reuse: an object store, a web server, a packager that assembles manifests, or a service that generates derivatives on request. To the client, the CDN itself looks like the origin server; HTTP calls such an intermediary a gateway, or reverse proxy. Freshness lifetimes, validators, cache keys, and range support determine how many requests reach the origin, and a widely requested object that expires, or that has never been cached, can concentrate those requests there. The term is unrelated to a web origin, the scheme, host, and port combination that browsers use for the same-origin policy and CORS.

Key facts

  1. Does every expired object have to be downloaded from the origin again? Not if the origin sends validators. A cache holding a stale response can make a conditional request using the stored ETag (in If-None-Match) or Last-Modified date (in If-Modified-Since), and an origin whose content is unchanged answers 304 (Not Modified) without sending the representation again. The origin still has to receive and answer every validation request that is forwarded to it, so validators cut the bytes transferred but do not, on their own, cut the number of requests.
  2. How do I keep many edge locations from each fetching the same object? Route their misses through one shared cache tier in front of the origin. Without it, each cache location that misses can send its own request for the same content. Amazon CloudFront Origin Shield, Fastly shielding, and Cloudflare Smart Tiered Cache all funnel those requests through a designated upper tier; Cloudflare’s Generic Global topology selects upper tiers from regional pools instead. CloudFront and Fastly also combine concurrent requests for the same uncached object into as few as one origin request. Fastly notes a limit: if the response turns out to be uncacheable, for example because it is private to one user, each waiting request must then be sent to the origin independently. AWS says Origin Shield may not be a good fit for dynamic content that is proxied to the origin, content with low cacheability, or content that is infrequently requested, and it is charged separately.
  3. What do viewers get while the origin is down? Responses that are still fresh in a cache keep being served. For stale ones, HTTP’s rule is that a cache must not serve them unless it is disconnected or has been explicitly permitted to, and not at all when the response carries no-cache or must-revalidate or, for a shared cache such as a CDN, proxy-revalidate or s-maxage. Where none of those directives applies, the stale-if-error directive from RFC 5861 gives the permission when the origin returns a 500, 502, 503, or 504, or cannot be reached. Its value limits how stale the response may be, counted from the moment it expired and not from the start of the outage: with max-age=600, stale-if-error=1200, the response can be used on error until it is 1800 seconds old. To cover planned maintenance, the value has to span the time since expiry plus the outage. CDN behavior differs in the details: CloudFront documents that it applies stale-if-error when the origin is unreachable or returns a 5xx status, up to the directive’s value or its maximum TTL, whichever is less, and that without the directive it still serves an expired object it holds when the origin returns a 5xx status. No setting helps for an object that was never cached or is no longer stored, and CloudFront says it might evict an infrequently requested object while the origin is still failing.
  4. Can a correct origin still deliver the wrong variant through a CDN? Yes, when the response depends on something the cache does not key on. HTTP sets two rules: a stored response with a Vary header may be reused only for requests whose nominated header fields match, and a shared cache must not reuse the response to a request that carried Authorization unless the response explicitly allows it. A CDN does not necessarily apply the Vary rule as written: Cloudflare documents that by default it does not consider Vary values in caching decisions, and respects them only for Vary: Accept-Encoding or where a Vary setting has been configured. So confirm that your CDN honors the headers named in Vary, or add them to its cache key yourself. The same goes for any other input that changes the response, such as a cookie or a query parameter the CDN is configured to ignore: add it to the cache key, or mark the response uncacheable.
  5. Can clients skip the CDN and request the origin directly? They can if the origin is publicly reachable and its address becomes known, which bypasses the CDN’s caching and protections. Cloudflare’s guidance lists the options: allowing only the CDN’s IP ranges (which it rates moderately secure and vulnerable to IP spoofing), requiring a secret request header, authenticating the CDN with a TLS client certificate (Authenticated Origin Pulls), or connecting through an outbound-only tunnel so that the origin has no publicly routable address. For Amazon S3 origins, AWS lists three tasks: create a CloudFront origin access control, which signs CloudFront’s requests to the bucket, and associate it with the distribution; give it permission to read the files, which the documentation does with a bucket policy limited to that distribution; and remove permission for anyone else to read the files through Amazon S3 URLs. Signing on its own leaves a publicly readable bucket reachable.

When Origin Servers matter

Configure an origin to expose only the assets and operations required by the CDN. Origin slowness or outages delay misses, revalidations, and uncacheable content because edge caches cannot answer those requests locally. Objects still fresh at the edge can continue to serve.

How Transloadit helps with Origin Servers

With Transloadit’s Smart CDN there are two upstream tiers. Your original files stay in your own storage, such as an Amazon S3 bucket, and a Template imports them from there. When the CDN has no cached copy for a requested URL, Transloadit runs the Template and the /file/serve Robot hands the result to the CDN, so the Template run plays the origin role for tlcdn.com and your bucket is the source behind it. /file/serve sets the Cache-Control header on that response; its cache_duration parameter sets both max-age and s-maxage, which determines how often requests come back to the Template and to your storage.

The /file/serve documentation advises against using it without a CDN in front, because every uncached request would run the transformation again. The pages linked here do not describe what is served when your storage is unreachable during a cache miss, or whether an expired result can still be served.

Explore Transloadit’s delivery capabilities

Sources

Turn media knowledge into a working pipeline

Connect uploads, processing, AI, storage, and delivery through one declarative API — with the encoding stack, scaling, and format churn handled for you.

Try Transloadit for free