Best way to iterate over a large set of entries?

APIs Content Management API
1

Hey all, we’re trying process a large number of entries with the management API, and it goes a little bit like this:

    do {
        const contentEntries = await env.getEntries({content_type: contentType, skip: contentEntriesLengthSoFar, limit: PAGE_SIZE});            
        
        contentEntriesThisBatch = contentEntries.items.length;
        if (contentEntriesThisBatch === 0) 
            continue;
        contentEntriesLengthSoFar += contentEntriesThisBatch;
        
        await ModifyEntries(contentEntries.items);
    } while (contentEntriesThisBatch > 0);

However the problem is that (because the entries are modified?), the getEntries on the second iteration sometimes returns some of the same elements as the first iteration, and as a result we never get thru ALL entries. What would be the recommended way to traverse and modify ALL entries on a large dataset?

2

anyone have suggestions for this?

3

Have you tried the order parameter? order=sys.createdAt

const contentEntries = await env.getEntries({content_type: contentType, skip: contentEntriesLengthSoFar, limit: PAGE_SIZE, order: sys.createdAt});

2 Likes
4

@lisa.choy This feels a bit hacky, but it does work!