we are currently migrating from Gatsby to a dynamically rendered page. We have a module in our system that allows the author to define some categories and tags, that are used to select which entries should be displayed. We use this module in one case to render all our news. Since we have quite some news, we need to paginate the module/page.
The problem is that we need to make multiple requests to contentful, as your API does not allow logical OR associations and requests including multiple content types, which makes pagination practically impossible.
On the first page we make the initial requests to get all the required entries. We merge the entries from the responses together, sort them somehow and send the first N entries to the frontend, where N is the defined page size. But what should we do when the user wants to see the second page? As our application is stateless and the all pages are deep linkable, we don’t know which entries were shown on the first page. We can’t simply define skip N in every request to contentful, as this would skip far to many entries. So we need to get the entries for the first page again, to compute the entries for the second page. This might even work for page 2 or 3, but this will fail very fast for higher pages. Getting all entries for the first 99 pages, to show page 100, kinda defeats the whole purpose of pagination…
So yes, there are workarounds, like using a common ordering criteria and implement something like infinity scroll, but sadly our client does not want a workaround. He want to have simple, plain old pagination that is deep linkable, especially for SEO.
Maybe we could change our content model so that we don’t need to query multiple content types, but we definitely need logical OR associations within one content type. I understand that this might lead to very expensive queries, but other systems have found a way to handle that too.
Are you guys planning to support this in the future?
I am not an expert on Contentful but I have faced this issue with headless CMS. Can you use webhooks or another solution to populate a search index with required data and invoke that rather than the CMS API (preferably without making another round trip to CMS)? Unless there is some cost difference, you are using CMS-specific functionality, or for some other reason, consider retrieving all required data from the search index rather than the CMS, using search vendor APIs rather than CMS vendor APIs.
Update: when I posted, I think I had misunderstood that your query over multiple content types worked, hence the possible conflict. Now I think your query does not work; there is no conflict and I agree with you that such queries should be possible. The remainder of this is my original post.
When I researched this issue, I found something that seems to conflict with what you wrote:
Gabriel: "However, the issue here is that each field is defined on the basis of a single content type.
"That being the case, although your fields may have the same name across multiple content types, they will effectively be different entities, so it’s unfortunately not possible to search through the fields of different content types that you may have specified in your query.
“So, as Charlie greatly explained, you may want to create a common content type.”
Is Gabriel incorrect here, using a different API than you use, or something else? @Janno, maybe you could comment on that page?
It seems that Contentful could release some documentation around best practices here, as this is an extremely common case (for example, find content entry by URL, for which there may be some other solution - I am not fluent in Contentful).
@Janno: If the Contentful GraphQL implementation allows queries to span multiple content types, could you use that instead? Personally, I try to avoid GraphQL because I want a single API and consistent data structures, so consider using GraphQL to retrieve IDs and then get the entries themselves from content delivery APIs.
The GraphQL API is of any help here as it reduces only the amount of data transfered between the systems, but does not fix the problem that we need to fetch all the data for the first n pages, to display page n. Additionally, we were told that the GraphQL API is for small requests only. We started with GraphQL initially, but ran very quickly into errors due too complex queries in our backend.
I appreciate that you try to find a way around this problem, but we really think it should be fixed in the contentful API.
