Async Iterables and Async Generators
Greetings, friends! In my previous articles, I talked about iterables and generators. So far, we've seen examples of synchronous iterables and generators. What if I told you that asynchronous (async) versions of iterables and generators existed?! Indeed they do! They're much less common than their synchronous counterparts, but async iterables and generators have their uses! Like everything in life, it's up to you to find the right tool for the job. If you're dealing with streams of data, handling data chunk by chunk, or wanting to make API calls lazily, then async iterables and generators might be right for you!
Review of Synchronous Iterators and Iterables
If you read my article on iterators and iterables, then you'll know that normal iterables follow the iterable protocol. In TypeScript notation, the protocols would be similar to the following:
interface Iterator {
next() : IteratorResult;
}
interface IteratorResult {
value: any;
done: boolean;
}
interface Iterable {
[Symbol.iterator]() : Iterator;
}
For those not familiar with TypeScript, it means that normal iterators require a next method to follow the iterator protocol, and iterables require a Symbol.iterator method to follow the iterable protocol. Below is an example of a regular synchronous iterable object:
const syncIterable = {
[Symbol.iterator]() {
return {
i: 0,
next() {
if (this.i < 3) {
return { value: this.i++, done: false };
}
return { done: true };
}
};
}
};
for (const num of syncIterable) {
console.log(num);
}
/* OUTPUT:
0
1
2
*/
Async Iterator Protocol and Async Iterables
In a similar fashion, async iterators also require a next method, but now they should return a promise (though it's not a requirement as we'll see later). Async iterables need to define a Symbol.asyncIterator method. Thus, the TypeScript notation would be as follows:
interface AsyncIterable {
[Symbol.asyncIterator]() : AsyncIterator;
}
interface AsyncIterator {
next() : Promise<IteratorResult>;
}
interface IteratorResult {
value: any;
done: boolean;
}
Not familiar with TypeScript? Let's look at an example of an async iterable object.
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
i: 0,
next() {
if (this.i < 3) {
return Promise.resolve({ value: this.i++, done: false });
}
return Promise.resolve({ done: true });
}
};
}
};
(async function() {
for await (const num of asyncIterable) {
console.log(num);
}
})();
/* OUTPUT:
0
1
2
*/
In the above example, we can iterate through values using a for await...of loop, since the object adheres to async iterable protocol. That is, the object defined a Symbol.asyncIterator method.
The for await...of loop is a special type of construct that lets you iterate through async iterables.
Previously, I stated that you don't need to return a promise in the next method defined in an async iterable. Let's see what happens if we don't return promises.
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
i: 0,
next() {
if (this.i < 3) {
return { value: this.i++, done: false };
}
return { done: true };
}
};
}
};
(async function() {
for await (const num of asyncIterable) {
console.log(num);
}
})();
/* OUTPUT:
0
1
2
*/
The result is the same. Even though we're using an async iterable, the for await...of loop still understands how to deal with return values that aren't promises.
Keep in mind that the for await...of loop also lets you iterate through synchronous iterables such as arrays. It'll check if the iterable has a Symbol.iterator method defined as a fallback in case the iterable doesn't have Symbol.asyncIterator defined.
const array = [0, 1, 2];
(async function() {
for await (const num of array) {
console.log(num);
}
})();
/* OUTPUT:
0
1
2
*/
If you're dealing with synchronous iterables only, then you should use a normal for...of loop. It's worth noting that if you try to run a for...of loop on an async iterable, then you'll get an error:
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
i: 0,
next() {
if (this.i < 3) {
return Promise.resolve({ value: this.i++, done: false });
}
return Promise.resolve({ done: true });
}
};
}
};
for (const num of asyncIterable) {
console.log(num);
}
// ERROR: Uncaught TypeError: asyncIterable is not iterable
This is because the for...of loop checks to see if the object has a Symbol.iterator method. Since we defined a Symbol.asyncIterator method, this object adheres to the async iterable protocol, but not the synchronous iterable protocol.
You can iterate through async iterables using the for await...of statement only inside an async function unless your environment supports top-level await. Most browsers and older versions of Node.js don't support top-level await as of the time of this writing, so I won't talk about it much here. As a fun fact, I should mention that the Chrome developer console supports top-level await in case you want to quickly use await without setting up an async function. In previous examples, I have been using an Immediately Invoked Function Expression (IIFE) to run the async function immediately simply for demonstration purposes.
You can use a normal async function instead of an IIFE if you prefer:
async function iterateAsyncIterable() {
for await (const num of asyncIterable) {
console.log(num);
}
}
iterateAsyncIterable();
You can also use an async next method if you want to use await syntax inside it.
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
i: 0,
async next() { // Now the next method is async, so we can use await inside it
if (this.i < 3) {
const result = await Promise.resolve({ value: this.i++, done: false });
result.value += 10;
return result;
}
return { done: true };
}
};
}
};
(async function() {
for await (const num of asyncIterable) {
console.log(num);
}
})();
/* OUTPUT:
10
11
12
*/
Remember! If you decide to use an async next method inside an async iterable, it will always return a promise that wraps around the value you return. Don't believe me? Let's look at a quick example:
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
async next() {
return 'See! I told you so!';
}
};
}
};
console.log(asyncIterable[Symbol.asyncIterator]().next());
// OUTPUT: Promise {<fulfilled>: "See! I told you so!"}
console.log(asyncIterable[Symbol.asyncIterator]().next() instanceof Promise);
// OUTPUT: true
This behavior is a property of using async functions in general. When you return a value in an async function, this value is wrapped inside a promise automatically.
async function promiseToSayHello() {
return 'hello';
}
console.log(promiseToSayHello());
// OUTPUT: Promise {<fulfilled>: "hello"}
console.log(promiseToSayHello() instanceof Promise);
// OUTPUT: true
You may be wondering if there are built-in async iterables. After all, there are multiple built-in synchronous iterables: Arrays, Strings, Maps, and Sets. However, there are currently no JavaScript constructs that have a default Symbol.asyncIterator method. According to MDN, the first built-in objects that will adhere to the async iterable protocol are WHATWG Streams as of the time of this writing.
Async Generators
Just as normal generators makes the process of creating synchronous iterables easier, async generators make it easier to create async iterables. Async generators behave similar to their synchronous counterpart, except for the fact that they can use the await keyword inside them, and calling next on an async generator results in a promise.
Let's look at an example of an async generator:
async function *generator() {
yield 1;
yield 2;
}
const g = generator();
console.log(g.next()); // Promise {<pending>}
Since we're using async generators, the next method will return a promise that wraps around the yielded value. Therefore, we have to use await or .then to retrieve the value from the promise.
async function *generator() {
yield 1;
yield 2;
}
const g = generator();
(async function () {
console.log(await g.next()); // { value: 1, done: false }
// You can use ".then" syntax as well
g.next().then(console.log); // { value: 2, done: false }
// The generator has finished
console.log(await g.next()); // OUTPUT: { value: undefined, done: true }
})();
You can pass values to the next method in async generators just like regular generators.
async function *generator() {
const value = yield 1;
console.log(value);
yield 2;
}
const g = generator();
(async function () {
console.log(await g.next()) // { value: 1, done: false }
console.log(await g.next("Pizza"))
/* OUTPUT:
Pizza
{ value: 2, done: false }
*/
console.log(await g.next()) // { value: undefined, done: done }
})();
If you throw an error in an async generator, then it can be caught.
async function *generator() {
yield 'It will be okay';
throw new Error('Code is broken');
yield 'Someone else will fix it';
}
const g = generator();
g.next().then(console.log); // { value: "It will be okay", done: false}
g.next().catch(console.log); // Error: Code is broken
g.next().then(console.log); // { value: undefined, done: true}
As you can see, throwing an error will terminate the generator, but the JavaScript code continues to execute fine, since we have caught the error. If we didn't catch the error, we would have seen an uncaught JavaScript exception in the console.
You can use the yield* keyword in async generators just like regular generators.
async function* gen1() {
yield 1;
yield 2;
return 3;
}
async function* gen2() {
const result = yield* gen1();
console.log(result);
}
(async function () {
for await (const value of gen2()) {
console.log(value);
}
})();
/* OUTPUT:
1
2
3
*/
You can convert synchronous iterables easily to async versions instead using this simple trick:
async function* createAsyncIterable(syncIterable) {
for (const elem of syncIterable) {
yield elem;
}
}
const asyncGen = createAsyncIterable([1, 2]);
(async function() {
const result1 = await asyncGen.next();
const result2 = await asyncGen.next();
const result3 = await asyncGen.next();
console.log(result1, result2, result3);
})();
/* OUTPUT:
{ value: 1, done: false }
{ value: 2, done: false }
{ value: undefined, done: true }
*/
If you need to define a custom async iterable, you can use an async generator function to help make the code look cleaner.
const asyncIterable = {
async* [Symbol.asyncIterator]() {
yield 1;
yield 2;
yield 3;
}
};
(async function() {
for await (const num of asyncIterable) {
console.log(num);
}
})();
/* OUTPUT:
1
2
3
*/
Now that we've acquainted ourselves with async generators, let's look at a practical example. The most common use case will likely involve making API calls in a lazy fashion. Is laziness fashionable? They are in generators! 😂
Let's say you have a website that fetches images from a remote server. You want users to see three more images appear each time they click a button. In this example, we'll simply use the JSON Placeholder API to gather photos. You can see an example of the JSON objects we'll be receiving here.
First, we'll start with a little html:
<h1>Async Generator Tutorial</h1>
<button id="button">Click to fetch images</button>
<div id="images"></div>
Then, we'll add some spacing between each image and some spacing between the button and images section:
#button {
margin-bottom: 1rem;
}
img {
margin-right: 1rem;
margin-bottom: 1rem;
}
Finally, we'll add the async generator logic to fetch photos three at a time whenever the user clicks on the button. In the example below, we're using a regular for loop to iterate through values 1 and 100. Within each iteration of this loop, we are making an API call to get an object from JSON Placeholder that will contain a URL to a thumbnail image.
Each thumbnail URL is pushed to an array. After 3 iterations have occurred, we use the modulus operator % to check if the current value is divisible by 3. If so, then that means we have fetched 3 images, which means it's time to yield the images.
Remember, a generator will keep going until it finds a yield expression. Once we have three images, we can create an image element for each of them and append these three elements to our HTML. This process will repeat each time the user clicks the button and until we've exhausted the generator (ended the for loop).
const images = document.getElementById('images');
const button = document.getElementById('button');
async function* getPhotos() {
let images = [];
for (let i = 1; i < 100; i++) {
const response = await fetch(`https://jsonplaceholder.typicode.com/photos/${i}`);
const json = await response.json();
const thumbnailUrl = json.thumbnailUrl;
images.push(thumbnailUrl);
if (i % 3 === 0) {
yield images;
images.length = 0;
}
}
}
const gen = getPhotos();
const insertImages = async () => {
const result = await gen.next();
const imageGroup = result.value;
imageGroup.forEach(image => {
images.innerHTML += `<img src="${image}"></img>`;
});
}
button.addEventListener('click', () => {
insertImages();
});
You can see the finished result below. Every time the user clicks on the button, three images will be fetched from the server and displayed to the user. This ensures that we don't load too many images to the screen and let the user decide if they want to see more images. You don't have to use a button in your application. You could have used Intersection Observer or scroll events instead to determine when to load more images on the screen to help improve the user experience.
Please check out my CodePen if you're interested in seeing the finished code.
Conclusion
Async iterables and async generators can be really useful if you're trying to iterate through values that are obtained asynchronously such as from API calls or file reads. If async generators aren't good enough for your use case, you can always make a custom async iterable to provide more flexibility. It's up to you to decide if what's the right tool for the job. Experiment to see what works better and what's cleaner! If you want to learn more, please check out the resources I have listed below.