JavaScript Async Iterators: The Technology Behind Streaming
Async iterators are JavaScript’s solution for handling data that arrives over time. Think of them as regular iterators, but instead of returning values immediately, they return promises that resolve to values as they become available.
A regular iterator is like having the entire book in your hands - you can flip through pages instantly. An async iterator is like reading a book that’s being written while you read - new pages arrive as the author writes them, and you have to wait for each new page.
You’ll find async iterators useful whenever you’re dealing with data that doesn’t arrive all at once:
· Streaming data from APIs · Reading files line by line · Processing real-time events · Handling paginated results · Working with database cursors
The Simple Syntax
The magic happens with the for await…of loop:
for await (const item of asyncDataSource) {
console.log(item);
}
This loop pauses at each iteration, waiting for the next piece of data to arrive before continuing.
Creating Async Iterators
The easiest way to create an async iterator is with an async generator function. Just add async before the asterisk:
async function* numberStream() {
let count = 0;
while (count < 5) {
// Wait one second, then yield the next number
await new Promise(resolve => setTimeout(resolve, 1000));
yield count++;
}
}
// Using it
for await (const num of numberStream()) {
console.log(num); // 0, then 1, then 2... with one second between each
}
Reading a File Line by Line
Instead of loading an entire file into memory, you can process it one line at a time:
async function* readFileLines(filename) {
const fileStream = createReadStream(filename);
const lines = readline.createInterface({ input: fileStream });
for await (const line of lines) {
yield line;
}
}
// Process each line as it becomes available
for await (const line of readFileLines('data.txt')) {
console.log('Processing:', line);
}
Paginated API Data
When fetching data from an API that paginates results, async iterators make the code clean and intuitive:
async function* fetchAllPages(url) {
let page = 1;
let hasMorePages = true;
while (hasMorePages) {
const response = await fetch(`${url}?page=${page}`);
const data = await response.json();
if (data.items.length === 0) {
hasMorePages = false;
} else {
yield data.items;
page++;
}
}
}
// Process pages as they arrive
for await (const page of fetchAllPages('/api/users')) {
displayPage(page);
}
Error Handling
Since async iterators deal with promises, you can use try-catch blocks naturally:
async function* flakyGenerator() {
// ... might throw errors
}
try {
for await (const item of flakyGenerator()) {
console.log(item);
}
} catch (error) {
console.error('Something went wrong:', error.message);
}
Cleaning Up
Async iterators can also clean up after themselves. When you break out of a loop early, the iterator gets a chance to release resources:
async function* resourceIntensive() {
const resource = await acquireResource();
try {
while (hasMoreData()) {
yield await getNextChunk();
}
} finally {
// This runs even if you break out of the loop early
await resource.release();
}
}
Why They Matter
Async iterators bridge the gap between synchronous iteration and asynchronous programming. They make asynchronous code read like synchronous code, reducing complexity and making your intentions clearer. Instead of managing complex state machines or callback chains, you can write straightforward loops that naturally express “do this for each item as it arrives.”
This pattern is particularly powerful in modern JavaScript applications, where streaming data and real-time updates have become the norm rather than the exception.