I am building a pure frontend application useíng React and contentful.js. But my problem is that for some reason, our application makes a lot of request to the CDN API. Looking right now, we have already made over 335,000 requests in only three days. I need to look into why we are making so many requests.
I saw this text: “There are no limits enforced on requests that hit our CDN cache, i.e. the request doesn’t count towards your rate limit and you can make an unlimited amount of cache hits.” But I don’t understand what is meant with the “CDN cache”. Is that something I need to turn on? Or do I need to implement that myself?
I hope you don’t find this question stupid and may be able to help me understand this.
CDN cache is a proxy between your app and Contentful backend. It’s managed by Contentful and is always “turned on” for everybody who calls the API. When you make a request to the CDA, you should get a x-cache response header, which says whether the request HIT or MISS the cache.
I’m not sure if I understand your question. When your pure frontend app calls Contentful API, the response is cached. The caching is done by Contentful, you don’t need to do anything about it. Next time your app makes same request again, it will get the cached response. Getting cached response is faster than uncached and doesn’t count towards rate limits.
If you wonder why not all requests are cache HIT, then it’s because cache gets invalidated when your content changes in Contentful and also after a timeout period.
Has anyone found a solution to this? We’re seeing CDA requests of about 100,000 per day, despite only having about 1000 page views on our site. I cannot understand how we are getting anywhere close to that number.
And unless I’ve misunderstood something, I don’t believe the cdn cache is working properly. I ran some tests, in a second unused space that we have, by calling a single entry 20 times through Postman. In the first response, the x-cache was set to ‘MISS’ and in each additional request it was set to ‘HIT’. Despite that, it is showing that I made 20 requests to the CDA in the usage stats in Contentful. My understanding is that this should only show 1 as the additional 19 hit the cache? Unless the usage stats are including cache hits but not going against the fair usage policy?
All the CDA requests (requests made towards cdn.contentful.com) are billed, regardless if it is cached in the CDN or served in our Delivery Application. Hopefully it clears thing up a bit.
Sorry to re-revive the dead thread. Just wanted clarification. From my reading of this, it seems that we won’t be limited by how many requests / second we can make if we’re hitting the CDN but all requests would count towards the API call limits and overages. (2 million API calls / month)
I’m coming a bit late to the party. Sorry for this.
When reading up on this thread, I have two remaining questions:
@charlie , is the full cache cleared for all content items when one item is published, or only the cache for the published item (leaving the cache for the rest of the items unchanged)
@joanna , you mention the cache also is invalidated “and also after a timeout period”. Which timeout period is this? Are there more details on this?
Hi @Alma, I also have the same question as @henri here - it doesn’t look like the TTL of the cache is documented anywhere. Could you please provide more details here? I saw the question was also asked here and the other link on the page also doesn’t mention this.
Hi @alerov , I got feedback on both of my questions by the Contentful support channel.
The cache is completely cleared each time an item is published. So you want to be sure to bundle your publishes or do them sequentially in the same timeframe.
The cache is invalidated (or fully cleared) after 30 days.
I’ve been in touch with Contentful’s support team and will summarize my findings from their answers and the various posts in this community.
Looking through Contentful’s Technical limits page, you’ll notice that depending on the plan you’re on, you’re only allowed to make up to 78 API requests/second, or you will hit Contentful’s api rate limit.
By default, all requests to Contentful’s Content Delivery API (url: cdn.contentful.com) are after a first request, cached for future requests and served via the CDN.
The cache:
is cleared each time any item triggers a publish event (Publish & Unpublish. Delete/Archive do not trigger a publish event. Doing a Delete in Contentful’s Webbrowser unpublishes an item before finally deleting it)
invalidated after 30 days
As an example:
There are 100 blog post-entries
They have all been fetched once before, so they are all cached
Now, some employee creates a new blog posts (101 blog posts), triggering a publish event and clearing all cache.
A developer triggers some kind of build. This build requests all the blog posts, eventually making 101 requests asking for different post id’s (ignoring the fact of optimization or asking for all entries in one call with getEntries).
These 101 request calls have not been run before. So we’re actually asking for un-cached data. After X requests (depending on the plan), we are faced with a 429 Rate limit response and a X-Contentful-RateLimit-Reset: <Seconds> header, informing us when we can resume requests.
Any recurring request asking for one of the post id’s is not rate limited as they are now cached, until someone eventually triggers a new publish event.
Requests with different queries create different cache entries.
E.g.
'fields.<field_name>[in]': 'accessories,flowers'
'fields.<field_name>[in]': 'accessories'
would produce two different cache entries and trigger the rate-limit twice first time querying for them.
Therefore, making an initial request to Contentful’s Delivery API (cdn.contentful.com) is not cached (cache-miss) and part of the rate-limit. Any subsequent request (cdn.contentful.com) with the same query is served by the cache and not concerned with rate-limiting.
Images API (images.ctfassets.net), which is cached/backed by CDN provider, but is actually not rate-limited as asset serving is more predictable than DB queries, and Contentful instead looks at total bandwidth use and charge overage for massive consumption cases.
for hosted files other than images, the delivery API will point towards assets.ctfassets.net based URLs instead, as the images’ endpoint offers particular image specific functionality.
Content Preview API (preview.contentful.com) does no caching to ensure fresh data at all times. The Content Preview API is intended for staging/preview version of the frontend/digital channel so that editors have access to a rendered version of content that has either new unpublished changes or the content is currently not published at all (Draft state). It should not be leveraged for any production use as it is not CDN/cache backed and won’t keep up with actual traffic.
Be aware that every call made to any API, even cached ones, are part of the monthly API calls limitation. If you’re using Premium/Enterprise, then you have unlimited monthly calls. Other plans have 2.000.000 API calls/month.
It is worth pointing out that Contentful offers an option to change the default, aggressive, cache clear behaviour via a so-called “Advanced Caching” feature - only available to certain Enterprise tiers.
Contentful’s JavaScript library (import {createClient} from "contentful") uses by default the cdn.contentful.com as host URL.
When your pure frontend app calls Contentful API, the response is cached. The caching is done by Contentful, you don’t need to do anything about it. Next time your app makes same request again, it will get the cached response. Getting cached response is faster than uncached and doesn’t count towards rate limits.