Intermittent VersionMismatch and 'File has already been processed' errors on import

Ecosystem Tools & Libraries
1

We import content from files into a Contentful environment as part of our continuous integration process using contentful-import library (version 7.4.3).

We are seeing intermittent import failures due to these errors:

  • VersionMismatch : Conflict
  • BadRequest: ‘File has already been processed’

Following is the stacktrace and couple of errors returned by CMA API. Regarding VersionMismatch, there is no other process importing into that environment at the same time as when our build is running. I don’t know why File has already been processed error happens.

Thanks

Problem importing src/test/resources/contentful/import/entries into space: 9mo1ly2dw0cr environment: integration { ContentfulMultiError: Errors occured
    at then (/contentful/import-script/node_modules/contentful-import/dist/index.js:202:28)
    at tryCatcher (/contentful/import-script/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/contentful/import-script/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/contentful/import-script/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/contentful/import-script/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/contentful/import-script/node_modules/bluebird/js/release/promise.js:694:18)
    at _drainQueueStep (/contentful/import-script/node_modules/bluebird/js/release/async.js:138:12)
    at _drainQueue (/contentful/import-script/node_modules/bluebird/js/release/async.js:131:9)
    at Async._drainQueues (/contentful/import-script/node_modules/bluebird/js/release/async.js:147:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/contentful/import-script/node_modules/bluebird/js/release/async.js:17:14)
    at processImmediate (timers.js:632:19)
name: 'ContentfulMultiError',
errors:
[ { ts: '2019-01-22T02:29:50.849Z',
    level: 'error',
    error:
        { name: 'VersionMismatch',
        entity:
        { sys:
            { space:
                { sys: { type: 'Link', linkType: 'Space', id: '9mo1ly2dw0cr' } },
                id: '63OaZmZhPqgiYi2Qkw6IuU',
                type: 'Asset',
                createdAt: '2018-11-29T22:17:43.576Z',
                updatedAt: '2018-12-06T00:21:37.911Z',
                environment:
                { sys: { id: 'master', type: 'Link', linkType: 'Environment' } },
                createdBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                updatedBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                publishedCounter: 4,
                version: 12,
                publishedBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                publishedVersion: 11,
                firstPublishedAt: '2018-11-29T22:17:57.913Z',
                publishedAt: '2018-12-06T00:21:37.911Z' },
            fields:
            { title: { 'en-US': 'rialto channel' },
                file:
                { 'en-US':
                    { url:
                    '//images.ctfassets.net/9mo1ly2dw0cr/63OaZmZhPqgiYi2Qkw6IuU/4aaafe22f0e9b17645cdacae0125485a/rialto_channel.png',
                    details: { size: 64723, image: { width: 420, height: 158 } },
                    fileName: 'rialto_channel.png',
                    contentType: 'image/png' } } } },
        data:
        { status: 409,
            statusText: 'Conflict',
            message: '',
            details: {},
            request:
            { url:
                'https://api.contentful.com:443/spaces/9mo1ly2dw0cr/environments/integration/assets/63OaZmZhPqgiYi2Qkw6IuU',
                headers:
                { Accept: 'application/json, text/plain, */*',
                'Content-Type': 'application/vnd.contentful.management.v1+json',
                'X-Contentful-User-Agent':
                    'app contentful.import/7.4.3; feature library-import; sdk contentful-management.js/5.2.1; platform node.js/v11.2.0; os Linux/2.6.32-754.3.5.el6.x86_64;',
                Authorization: 'Bearer ...6d54d',
                'user-agent': 'node.js/v11.2.0',
                'Accept-Encoding': 'gzip',
                'X-Contentful-Version': 612,
                'Content-Length': 252 },
                method: 'put',
                payloadData:
                '{"fields":{"title":{"en-US":"rialto channel"},"file":{"en-US":{"contentType":"image/png","fileName":"rialto_channel.png","upload":"https://images.ctfassets.net/9mo1ly2dw0cr/63OaZmZhPqgiYi2Qkw6IuU/4aaafe22f0e9b17645cdacae0125485a/rialto_channel.png"}}}}' },
            requestId: 'a748b159c1ddef61e5d46f9eade2409a' } } },
    { ts: '2019-01-22T02:29:57.093Z',
    level: 'error',
    error:
        { name: 'BadRequest',
        entity:
        { sys:
            { space:
                { sys: { type: 'Link', linkType: 'Space', id: '9mo1ly2dw0cr' } },
                id: '2iwzc8U7peOWsSIasuGiGq',
                type: 'Asset',
                createdAt: '2018-12-10T00:49:07.923Z',
                updatedAt: '2019-01-22T02:29:51.506Z',
                environment:
                { sys: { id: 'integration', type: 'Link', linkType: 'Environment' } },
                createdBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                updatedBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                publishedCounter: 199,
                version: 599,
                publishedBy:
                { sys:
                    { type: 'Link', linkType: 'User', id: '6GStxT8ohd2oxOw3MlB5DB' } },
                publishedVersion: 596,
                firstPublishedAt: '2018-12-10T00:49:32.735Z',
                publishedAt: '2019-01-22T02:23:35.519Z' },
            fields:
            { title: { 'en-US': 'The Big Bang Theory Season 2 Episode 2' },
                file:
                { 'en-US':
                    { upload:
                    'https://images.ctfassets.net/9mo1ly2dw0cr/2iwzc8U7peOWsSIasuGiGq/a516bde03c746af957732b95137a3f71/thebigbangtheory-2-2.png',
                    fileName: 'thebigbangtheory-2-2.png',
                    contentType: 'image/jpeg' } } } },
        data:
        { status: 400,
            statusText: 'Bad Request',
            message: 'File has already been processed.',
            details: {},
            request:
            { url:
                'https://api.contentful.com:443/spaces/9mo1ly2dw0cr/environments/integration/assets/2iwzc8U7peOWsSIasuGiGq/files/en-US/process',
                headers:
                { Accept: 'application/json, text/plain, */*',
                'Content-Type': 'application/vnd.contentful.management.v1+json',
                'X-Contentful-User-Agent':
                    'app contentful.import/7.4.3; feature library-import; sdk contentful-management.js/5.2.1; platform node.js/v11.2.0; os Linux/2.6.32-754.3.5.el6.x86_64;',
                Authorization: 'Bearer ...6d54d',
                'user-agent': 'node.js/v11.2.0',
                'Accept-Encoding': 'gzip',
                'X-Contentful-Version': 599 },
                method: 'put',
                payloadData: null },
            requestId: 'bc35dc26f4ce7c88f636eb657782eff6' } } } ] }
2

We are no doing anything complicated with the import. We are just invoking contentful-import library’s contentfulImport function with the file to import. Like this:

const options = {
    spaceId: spaceId,
    environmentId: environmentId,
    managementToken: managementToken,
    contentFile: "./build/combined.json",
    noUpdate: noUpdateOption
}

contentfulImport(options)
    .then((result) => {
        console.log(`Imported ${directory} into space: ${spaceId} environment: ${environmentId}`, result)
    })
    .catch((err) => {
        console.error(`Problem importing ${directory} into space: ${spaceId} environment: ${environmentId}`, err);
        process.exit(1);
    })
3

Also, the content json file being imported is the same most of the time. We make occasional changes to the content. Import works with the same file some times and fails at other times.