Iterators, Iterables, and Generators
Greetings, friends! Have you ever been confused between the difference between iterators, iterables, and generators in the world of JavaScript? I know when I was learning JavaScript, I would constantly get confused by iterators and iterables, and I still get them mixed up today 😅. Let's learn about each of these topics and how they relate to each other.
Iterators
As stated on this MDN article about iterators, a JavaScript iterator is an object that defines a sequence and may or may not return a value upon its termination or completion. Specifically, an iterator is a JavaScript object that implements the Iterator protocol/interface. What does this mean? If you're familiar with TypeScript, it essentially means this:
interface Iterator {
next() : IteratorResult;
}
interface IteratorResult {
value: any;
done: boolean;
}
If you're not familiar with TypeScript, no worries! I'll explain it another way. For an object to be considered an iterator, it needs to have a next method that returns an object with the keys, value and done. Let's see a vanilla JavaScript example of this:
const myCoolIterator = {
next: function() {
return { value: 1, done: true }
}
};
According to the method definitions page, you could have also used this shorthand notation instead:
const myCoolIterator = {
next() {
return { value: 1, done: true }
}
};
That's all it takes for an object to become an iterator! However, the above example is rather boring 😴. If you call the next method on this object, it'll return the same thing every time. We want our iterator to be useful and thus should iterate through different values.
Let's look at a better example, found on MDN.
// This function returns an iterator object
function makeRangeIterator(start = 0, end = Infinity, step = 1) {
let nextIndex = start;
let iterationCount = 0;
const rangeIterator = {
next: function() {
let result;
if (nextIndex < end) {
result = { value: nextIndex, done: false }
nextIndex += step;
iterationCount++;
return result;
}
return { value: iterationCount, done: true }
}
};
return rangeIterator;
}
This function can be thought of as an iterator factory. When we call the makeRangeIterator function, it will create a new iterator object. Let's use this function now:
const it = makeRangeIterator(1, 10, 2);
console.log(it.next()); // { value: 1, done: false}
console.log(it.next()); // { value: 3, done: false}
console.log(it.next()); // { value: 4, done: false}
console.log(it.next()); // { value: 5, done: false}
console.log(it.next()); // { value: 9, done: false}
console.log(it.next()); // { value: 5, done: true}
console.log(it.next()); // { value: 5, done: true}
console.log(it.next()); // { value: 5, done: true}
What's happening here? Every time we call the next function, we're iterating the value by 2. Eventually, we hit the number 11, and it's greater than the end value of 10, so we start returning a new object with done set to true. Now, I know what you're thinking. What's so special about this? This is just a normal function, right? We could have changed the method name, next, to stuff, and the function would still behave normally. To see why iterators are useful and why we need to adhere to the iterator protocol, let's first learn about iterables.
Iterables
Iterables are objects that implements the iterable protocol/interface. If you're familiar with TypeScript, then this is similar to the following:
interface Iterable {
[Symbol.iterator]() : Iterator;
}
What does this mean? It's saying that for an object to be an iterable, it must contain a method named Symbol.iterator that returns an iterator object. Yes, the names can get quite confusing indeed. Iterators...iterables...vegetables...yikes 😵. So an iterable is a JavaScript object that has a method named Symbol.iterator that returns a function named next that then returns an object with the keys, value and done. Yes, that sounds confusing. Let's look at an example:
const myCoolIterable = {
[Symbol.iterator]: function() {
return {
next: function() {
return { value: 0, done: true }
}
}
}
}
Again, you can use the shorthand notation for object methods:
const myCoolIterable = {
[Symbol.iterator]() {
return {
next() {
return { value: 0, done: true }
}
}
}
}
Let's check to see if our iterable works:
console.log(myCoolIterable[Symbol.iterator]().next());
// Output: { value: 0, done: true }
The previous examples weren't that useful, since the next method returns the same thing, so let's make a more interesting example. Suppose we had the following iterable:
const iterable = {
[Symbol.iterator]() {
return {
i: 0,
next() {
this.i = this.i + 2;
if (this.i < 11) {
return { value: this.i, done: false };
}
return { value: undefined, done: true };
}
};
}
};
Instead of calling the next method over and over, let's use a for...of loop loop to iterate through the iterable instead:
for (const value of iterable) {
console.log(value);
}
/*
Output:
2
4
6
8
10
*/
Wait, what? How is that possible? 😮 What wizardry is this? 🧙 It turns out that the for...of loop iterates through objects that implement the iterable protocol. Custom iterables let us customize the behavior of for...of loops! Pretty cool! By extension, we implement the iterator protocol when we implement the iterable protocol, since our iterable needs to implement a method named Symbol.iterator that returns an iterator. In most cases, you'll usually implement both the iterator and iterable protocols, not just one of them.
@@iterator method. This is identical to the Symbol.iterator method. The @@ symbols are used to denote common symbols in the official ECMAScript specification.Let's dive deeper into understanding what the for...of loop does internally. The for...of loop will first call the Symbol.iterator method of the iterable passed into it which will then return an object that contains the next method. Upon each iteration of the for...of loop, it will then call the next method. Keep this in mind if you wish to do some setup logic inside the Symbol.iterator method definition.
const iterable = {
[Symbol.iterator]() {
const initialValue = 0;
console.log('Symbol.iterator method called');
return {
i: initialValue,
next() {
console.log('next method called');
this.i = this.i + 2;
if (this.i < 11) {
return { value: this.i, done: false };
}
return { value: undefined, done: true };
}
};
}
};
for (const value of iterable) {
console.log(value);
}
/* OUTPUT:
Symbol.iterator method called
next method called
2
next method called
4
next method called
6
next method called
8
next method called
10
*/
If you break the loop immediately, you'll still see the Symbol.iterator method get invoked by the for...of loop. You'll also see the next method get invoked only one time.
const iterable = {
[Symbol.iterator]() {
const initialValue = 0;
console.log('Symbol.iterator method called');
return {
i: initialValue,
next() {
console.log('next method called');
this.i = this.i + 2;
if (this.i < 11) {
return { value: this.i, done: false };
}
return { value: undefined, done: true };
}
};
}
};
for (const value of iterable) {
break;
console.log(value); // Won't log any value here since break ends the loop
}
/* OUTPUT:
Symbol.iterator method called
next method called
*/
Iterables are all around us in the JavaScript world. Arrays, Strings, Maps, and Sets are all iterable. Don't believe me? Let's see if they implement a Symbol.iterator method:
Array.prototype[Symbol.iterator] === undefined // false
String.prototype[Symbol.iterator] === undefined // false
Map.prototype[Symbol.iterator] === undefined // false
Set.prototype[Symbol.iterator] === undefined // false
Object.prototype[Symbol.iterator] === undefined // true
Looks like the Symbol.iterator method exists on all of them which means they're iterables. This means that they can all be used in a for...of loop! Wait... objects don't seem to be iterable 🤔. Normally, JavaScript objects cannot be iterated over using for...of loops. They don't have default iterable protocols like the other JavaScript data structures. If you run the following in your JavaScript console, you'll get an error:
const foods = {pizza: '🍕', pineapple: '🍍'}
for (const food of foods) {
console.log(food);
}
// Error thrown: Uncaught TypeError: foods is not iterable
However, we've shown that by implementing a Symbol.iterator method on an object that follows the iterable protocol, we can make objects iterable too. JavaScript objects just don't come with a Symbol.iterator method by default and thus constructs like for...of loops don't work on them.
The for...of loops aren't the only constructs that accept an iterable. Let's look at other JavaScript constructs that support iterations over iterables.
const [pizza, pineapple] = new Set(['🍕', '🍍']);
console.log(pizza, pineapple); // 🍕 🍍
The Array.from() method:
const arr = Array.from(new Set(['a', 'b', 'c']));
console.log(arr); // ["a", "b", "c"]
const arr = [...new Set(['a', 'b', 'c'])];
console.log(arr); // ["a", "b", "c"]
Constructors of Maps and Sets:
const map = new Map([[false, 'no'], [true, 'yes']]);
const set = new Set(['a', 'b', 'c']);
console.log(map); // Map(2) {false => "no", true => "yes"}
console.log(set); // Set(3) {"a", "b", "c"}
Promise.all() and Promise.race():
const wait3Seconds = new Promise(
(resolve) => setTimeout(() => resolve('3 seconds have passed')),
3000
);
const wait5Seconds = new Promise(
(resolve) => setTimeout(() => resolve('5 seconds have passed')),
5000
);
Promise.all([wait3Seconds, wait5Seconds]).then(console.log); // ["3 seconds have passed", "5 seconds have passed"]
Promise.race([wait3Seconds, wait5Seconds]).then(console.log); // 3 seconds have passed
The yield* keyword:
function* delegateToIterable() {
yield* [1, 2, 3];
}
const iterator = delegateToIterable();
console.log(iterator.next().value); // 1
console.log(iterator.next().value); // 2
console.log(iterator.next().value); // 3
Since Arrays, Strings, Maps, and Sets are all iterables, they can be passed into each of these constructs. As long as you give any of these constructs an object that adheres to the iterable protocol, then they should work correctly. This means we can create our own iterable and use them with each of these constructs. For example, let's use the spread operator on a custom iterable.
const iterable = {
[Symbol.iterator]() {
return {
i: 0,
next() {
this.i = this.i + 2;
if (this.i < 11) {
return { value: this.i, done: false };
}
return { value: undefined, done: true };
}
};
}
};
console.log([...iterable]); // [2, 4, 6, 8, 10]
Generators
Now that we have learned about iterators and iterables, what is a generator? Generators lets us pause execution of a function and then call it in a lazy manner. That is to say that we can use the next method to iterate through values specified in the generator function with the yield keyword. Think of the yield keyword as a return statement for generator functions (though they're not quite the same). Every time we call the next method, the function will pick up from where it left off, right after the last yield statement, and then continue execution until another yield statement is hit.
Let's look at a simple example of a generator function that returns a generator:
function *simpleGenerator() {
// First next statement begins here
yield 1;
// Second next statement begins here
yield 2;
// Third next statement begins here
yield 3;
// Fourth next statement begins here, but there's no more yield statements,
// so a value of undefined is returned and the done property is set to true
}
// Use the generator
const myGenerator = simpleGenerator();
console.log(myGenerator.next()) // { value: 1, done: false }
console.log(myGenerator.next()) // { value: 2, done: false }
console.log(myGenerator.next()) // { value: 3, done: false }
console.log(myGenerator.next()) // { value: undefined, done: true }
Notice the asterisk, *, before simpleGenerator. By placing an asterisk before the function name, it allows us to create a generator function instead of a normal function.
You can also move the asterisk to the right of the function keyword, and it still creates a generator function:
function* simpleGenerator() {
yield 1;
yield 2;
yield 3;
}
When we run the next method, it will run through the simpleGenerator function until it hits the first yield statement. Then, when we run next again, it will pick off right after the first yield statement and before the second yield statement. This pattern continues until there are no more yield statements left, or the function is escaped through another means such as a break statement or error occurring. Generators help us create iterators that automatically sets the done property to true once it has finished completing. Generally, you'll create generators a lot more often than creating iterators from scratch.
So is a generator an iterator or an iterable? It's actually both! It adheres to both the iterator and iterable protocols. A generator can be thought of as a JavaScript object that implements both a next method and a Symbol.iterator method. They provide a simpler syntax for creating and using iterators.
One big distinction between generators and custom iterables is that generators can only iterate through values once. Suppose we used a generator to create an iterator:
function* makeIterator() {
yield 1;
yield 2;
}
const it = makeIterator();
Since our generator is an iterable, we can iterate through it using a for...of loop, but if we try to do that multiple times, we'll no longer receive any more values from the generator:
for (const value of it) {
console.log(value);
}
/* Output:
1
2
*/
for (const value of it) {
console.log(value);
}
// Nothing will be output, since the generator can only be iterated once.
According to MDN, iterables which can iterate only once (such as Generators) customarily return the this value from their Symbol.iterator method. We can prove this is true through the following statement (based on the generator we created in our above example):
console.log(it[Symbol.iterator]() === it) // true
If we want our generator to return multiple times, we can overwrite the Symbol.iterator such that it yields values, but doesn't return this:
function* makeIterator() {
yield 1;
yield 2;
}
const it = makeIterator();
it[Symbol.iterator] = function* () {
yield 2;
yield 1;
};
for (const value of it) {
console.log(value);
}
/* Output:
2
1
*/
for (const value of it) {
console.log(value);
}
/* Output:
2
1
*/
Now the generator can be iterated over multiple times using constructs such as for...of loops.
A generator essentially follows a pattern similar to the following:
const generator = {
next() {
// logic that returns an object with value and done properties
},
[Symbol.iterator]: function() { return this; }
};
For example, let's create a function that behaves like a generator function:
function makeGenerator() {
let count = 0;
const generator = {
next() {
if (count < 3) return { value: count++, done: false }
else return { value: undefined, done: true }
},
[Symbol.iterator]: function() { return this; }
};
return generator;
}
// Use the generator
const myGenerator = makeGenerator();
console.log(myGenerator.next()) // { value: 0, done: false }
console.log(myGenerator.next()) // { value: 1, done: false }
console.log(myGenerator.next()) // { value: 2, done: false }
console.log(myGenerator.next()) // { value: undefined, done: true }
The above example is approximately equivalent to using a real generator function:
function *makeGenerator() {
let count = 0;
while (count < 3) {
yield count++;
}
}
// Use the generator
const myGenerator = makeGenerator();
console.log(myGenerator.next()) // { value: 0, done: false }
console.log(myGenerator.next()) // { value: 1, done: false }
console.log(myGenerator.next()) // { value: 2, done: false }
console.log(myGenerator.next()) // { value: undefined, done: true }
By returning this in our Symbol.iterator method, we are making the iterable be able to iterate through values only once like a generator. That is why these examples are approximately considered equivalent. Their behaviors are similar.
It's important to note that you can also create generator functions inside objects:
const myCoolObject = {
*generator() {
yield 1;
yield 2;
yield 3;
}
}
const myGenerator = myCoolObject.generator();
console.log(myGenerator.next()) // { value: 1, done: false }
console.log(myGenerator.next()) // { value: 2, done: false }
console.log(myGenerator.next()) // { value: 3, done: false }
console.log(myGenerator.next()) // { value: undefined, done: true }
This syntax is similar to creating generator functions inside classes:
class myCoolClass {
*generator() {
yield 1;
yield 2;
yield 3;
}
}
const classInstance = new myCoolClass();
const myGenerator = classInstance.generator();
console.log(myGenerator.next()) // { value: 1, done: false }
console.log(myGenerator.next()) // { value: 2, done: false }
console.log(myGenerator.next()) // { value: 3, done: false }
console.log(myGenerator.next()) // { value: undefined, done: true }
We can also create custom iterables using generators, since a generator is both an iterator and iterable. Therefore, we can assign the System.iterator method to a generator function:
const myIterable = {
*[Symbol.iterator]() {
yield 1;
yield 2;
yield 3;
}
}
for (let value of myIterable) {
console.log(value);
}
/*
Output:
1
2
3
*/
// Spread operator
console.log([...myIterable]); // [1, 2, 3]
If you want to learn more about generators, then check out my next article where I cover generators in depth and discuss how to avoid common pitfalls.
Conclusion
How often are iterators, iterables, and generators used in the JavaScript world? Many developers may not even need to touch them. Even in my own career, it's rare to make custom iterators and iterables. From my personal experience, I would say generators are used more often. Some libraries such as Redux-Saga utilize generators extensively for handling asynchronous state management in Redux applications, commonly paired with React.
Generators and iterables let us accomplish lazy evaluation. Most programming languages have some method for lazily evaluating functions. This may help save memory and/or time to improve performance in your application. Hope to you see you generate some amazing code! Till next time, happy coding!