forEachQueue
A tool for asynchronously working through large lists while keeping things a little more controlled than await Promise.all().
Example
A hypothetical example showing a lookup of user ratings for 50,000 movies. The limit is set to 50 pending requests at once so we don't get rate limited or run into timeouts.
import forEachQueue from "@ratwizard/for-each-queue";
const lotsOfItems = [
/* 50,000 item array */
];
const movieScores = await forEachQueue(lotsOfItems, 50, async (item) => {
// Make web requests, for example. A maximum of 50 requests will be pending at once in this case.
// When one finishes the next will be queued up.
const results = await http.get(
`https://movie-api.com/api/stuff?t=${item.name}`
);
return {
name: item.name,
score: results.userRating,
};
});
for (const movie of movieScores) {
console.log(`${movie.name}: ${movie.score}`);
}