I’ve been having the following problem with fetching new content from Contenful to Jekyll via the
Whenever I publish a new item in a content type and fetch the data with
bundle exec jekyll contentful, the latest addition is present in the YAML normally but mostly (not always) several older items that remained published but were not updated randomly disappear.
Once I make updates to them in Contentful and fetch the data again, the updated items show up in the YAML but I need to make an update to every single one that is missing - feasible when there are 5 items but not really with more.
What is causing this issue?
EDIT: I just found that other content types randomly disapear as well! This is absolutely unstable! The API keeps removing published content and I have spent past 2 hours trying to update each disapeared item only to find out that different COMPLETELY RANDOM ITEM disapears on each fetch. I have also tried to paste the YAMl manually which jekyll won’t parse because of some illegal character.
What is the problem? This is absolutely unusable!
I’m pinging a colleague of mine and, in the meanwhile, could you tell us how exactly you’re fetching data using this gems? (omitting all sensitive information and credentials, of course)
I’m the creator and maintainer of the Jekyll integration, as @gabriel said, could you post the relevant piece of the
_config.yaml file? This way I could check what may be the case, as this is something I’ve never seen happening.
Hi @gabriel and @david.litvak, thanks for getting back to me.
_config.yaml Contentful is configured on the following two places:
- data: 'contentful.spaces.boc.article'
- data: 'contentful.spaces.boc.press'
I’m not sure whether how Gemfile is written can make a difference:
gem 'jekyll', '~> 3.3', '>= 3.3.1'
group :jekyll_plugins do
gem 'jekyll-feed', '~> 0.9.3'
gem 'jekyll-contentful-data-import', '~> 1.6'
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
# Some more
I only notice this problem sometimes and I often fetch the data from public wifis that can be both spotty and flagged as malicious, can it be that the request is throttled or that parts get lost?
Looks like you may be having more than 100 entries in your space. You can easily solve this by adding the
all_entries: true option like follows:
This will ensure that all entries are properly fetched and does the proper pagination to fetch everything even if you have more than 1000 entries (the maximum amount of entries per request).
If you need some more granular control on how to import your data, you can tweak the
cda_query and the
all_entries_page_size options. You can read more about this here: https://github.com/contentful/jekyll-contentful-data-import#configuration
That was the problem. Thank you!