Last updated: by Valentino Gagliardi

A look at generator functions and asynchronous generators in JavaScript.

A look at generator functions and asynchronous generators in JavaScript

// TOC here

A quick intro

Welcome to the wonderful world of generator functions and asynchronous generators in JavaScript.

You're about to learn about one of the most exotic (according to the majority of developers) JavaScript feature.

Let's get started!

Generator functions in JavaScript

What is a generator function?

A generator function (ECMAScript 2015) in JavaScript is a special type of synchronous function which is able to stop and resume its execution at will.

In contrast to regular JavaScript functions, which are fire and forget, generator functions also have the ability to:

  • communicate with the caller over a bi-directional channel.
  • retain their execution context (scope) over subsequent calls.

You can think of generator functions as of closures on steroids, but the similarities stop here!

Your first generator function

To create a generator function we put a star * after the function keyword:

function* generate() {
//
}

Note: generator functions can also assume the shape of a class method, or of a function expression. In contrast, arrow generator functions are not permitted.

Once inside the function we can use yield to pause the execution:

function* generate() {
  yield 33;
  yield 99;
}

yield pauses the execution and returns a so called Generator object to the caller. This object is both an iterable, and an iterator at the same time.

Let's demystify these concepts.

Iterables and iterators

An iterable object in JavaScript is an object implementing Symbol.iterator. Here's a minimal example of iterable:

const iterable = {
  [Symbol.iterator]: function() {
    /* TODO: iterator */
  }
};

Once we have this iterable object, we assign a function to [Symbol.iterator] to return an iterator object.

This sounds like a lot of theory, but in practice an iterable is an object on which we can loop over with for...of (ECMAScript 2015). More on this in a minute.

You should already know a couple of iterables in JavaScript: arrays and strings for example are iterables:

for (const char of "hello") {
  console.log(char);
}

Other known iterables are Map and Set. for...of comes also handy for iteration over an object values:

const person = {
  name: "Juliana",
  surname: "Crain",
  age: 32
};

for (const value of Object.values(person)) {
  console.log(value);
}

Just remember that any property marked as enumerable: false won't show up in the iteration:

const person = {
  name: "Juliana",
  surname: "Crain",
  age: 32
};

Object.defineProperty(person, "city", {
  enumerable: false,
  value: "London"
});

for (const value of Object.values(person)) {
  console.log(value);
}

// Juliana
// Crain
// 32

Now, the problem with our custom iterable is that it can't go far alone without an iterator.

Iterators are also objects, but they should conform to the iterator protocol. In brief, iterators must have at least a next() method.

next() must return another object, whose properties are value and done.

The logic for our next() method must obey the following rules:

  • we return an object with done: false to continue the iteration.
  • we return an object with done: true to stop the iteration.

value instead, should hold the result that we want to produce for the consumer.

Let's expand our example by adding the iterator:

const iterable = {
  [Symbol.iterator]: function() {
    let count = 0;

    return {
      next() {
        count++;
        if (count <= 3) {
          return { value: count, done: false };
        }

        return { value: count, done: true };
      }
    };
  }
};

Here we have an iterable, which correctly implements Symbol.iterator. We also have an iterator, which returns:

  • an object whose shape is { value: x, done: false} until count reaches 3.
  • an object whose shape is { value: x, done: true} when count reaches 3.

This minimal iterable is ready to be iterated over with for...of:

const iterable = {
  [Symbol.iterator]: function() {
    let count = 0;

    return {
      next() {
        count++;
        if (count <= 3) {
          return { value: count, done: false };
        }

        return { value: count, done: true };
      }
    };
  }
};

for (const iterableElement of iterable) {
  console.log(iterableElement);
}

The result will be:

1
2
3

Once you'll get to know generator functions better, you'll see that iterators are the foundation for generator objects.

Keep in mind:

  • iterables are the objects on which we iterate over.
  • iterators are the things that make the iterable ... "loopable" over.

In the end, what's the point of an iterable?

We now have a standard, reliable for...of loop which works virtually for almost any data structure, custom or native, in JavaScript.

To use for...of on your custom data structure you have to:

  • implement Symbol.iterator.
  • provide an iterator object.

That's it!

Further resources:

Iterables are spreadable and destructurable

In addition to for...of, we can also use spread and destructuring on finite iterables.

Consider again the previous example:

const iterable = {
  [Symbol.iterator]: function() {
    let count = 0;

    return {
      next() {
        count++;
        if (count <= 3) {
          return { value: count, done: false };
        }

        return { value: count, done: true };
      }
    };
  }
};

To pull out all the values we can spread the iterable into an array:

const values = [...iterable];

console.log(values); // [1, 2, 3]

To pull out just a couple of values instead we can array destructure the iterable. Here we get the first and second value from the iterable:

const [first, second] = iterable;

console.log(first); // 1
console.log(second); // 2

Here instead we get the first, and the third:

const [first, ,third] = iterable;

console.log(first); // 1
console.log(third); // 3

Let's now turn again our focus on generator functions.

Extracting data from a generator function

Once a generator function is in place we can start interacting with it. This interaction consists in:

  • getting values from the generator, step by step.
  • optionally sending values back to the generator.

To pull values out a generator we can use three approaches:

  • calling next() on the iterator object.
  • iteration with for...of.
  • spreading and array destructuring.

A generator function does not calculate all its results in one single step, like regular functions do.

If we take our example, to get values from the generator we can first of all warm up the generator:

function* generate() {
  yield 33;
  yield 99;
}

// Initialize the generator
const go = generate();

Here go becomes our iterable/iterator object, the result of calling generate.

(Remember, a Generator object is both an iterable and an iterator).

From now on we can call go.next() to advance the execution:

function* generate() {
  yield 33;
  yield 99;
}

const go = generate();

// Consume the generator
const { value: firstStep } = go.next(); // firstStep is 33
const { value: secondStep } = go.next(); // secondStep is 99

Here each call to go.next() produces a new object. In the example we destructure the property value from this object.

Objects returned from calling next() on the iterator object have two properties:

  • value: the value for the current step.
  • done: a boolean indicating whether there are more values in the generator, or not.

We implemented such iterator object in the previous section. When using a generator, the iterator object is already there for you.

next() works well for extracting finite data from an iterator object.

To iterate over non-finite data, we can use for...of. Here's an endless generator:

function* endlessGenerator() {
  let counter = 0;
  while (true) {
    counter++;
    yield counter;
  }
}

// Consume the generator
for (const value of endlessGenerator()) {
  console.log(value);
}

As you can notice there's no need to initialize the generator when using for...of.

Finally, we can also spread and destructure the generator object. Here's the spread:

function* generate() {
  yield 33;
  yield 99;
}

// Initialize the generator
const go = generate();

// Spread
const values = [...go];

console.log(values); // [33, 99]

Here's destructuring:

function* generate() {
  yield 33;
  yield 99;
}

// Initialize the generator
const go = generate();

// Destructuring
const [first, second] = go;

console.log(first); // 1
console.log(second); // 2

It's important to note that generators go exhaust once you consume all their values.

If you spread out values from a generator, there's nothing left to pull out afterwards:

function* generate() {
  yield 33;
  yield 99;
}

// Initialize the generator
const go = generate();

// Spread
const values = [...go];

console.log(values); // [33, 99]

// Exhaust
const [first, second] = go;

console.log(first); // undefined
console.log(second); // undefined

Generators also work the other way around: they can accept values or commands from the caller as we'll see in a minute.

Talking back to generator functions

Iterator objects, the resulting object of a generator function call, expose the following methods:

  • next()
  • return()
  • throw()

We already saw next(), which helps in pulling out objects from a generator.

Its use is not only limited to extracting data, but to send values to the generator as well.

Consider the following generator:

function* endlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

It lists a parameter string. We provide this argument on initialization:

function* endlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

const go = endlessUppercase("a");

As soon as we call next() for the first time on the iterator object, execution starts and produces "A":

function* endlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

const go = endlessUppercase("a");

console.log(go.next().value); // A

At this point we can talk back to the generator by providing an argument for next:

go.next("b");

Here's the complete listing:

function* endlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

const go = endlessUppercase("a");

console.log(go.next().value); // A

console.log(go.next("b").value); // B

From now on, we can feed values to yield any time we need a new uppercase string:

function* endlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

const go = endlessUppercase("a");

console.log(go.next().value); // A

console.log(go.next("b").value); // B
console.log(go.next("c").value); // C
console.log(go.next("d").value); // D

If at any time we want to return from the execution altogether, we can use return on the iterator object:

const { value } = go.return("stop it");

console.log(value); // stop it

This stops the execution.

In addition to values you can also throw an exception into the generator. See "Error handling for generator functions".

Use cases for generator functions

Most developers (me included) see generator functions as an exotic JavaScript feature which has little or no application in the real world.

This could be true for the average front-end work, where a sprinkle of jQuery, and a bit of CSS can do trick most of the times.

In reality, generator functions really shine in all those scenarios where performances are paramount.

In particular, they're good for:

Generator functions are also the building block for sophisticated asynchronous patterns with asynchronous generator functions, our topic for the next section.

Asynchronous generator functions in JavaScript

What is an asynchronous generator function?

An asynchronous generator function (ECMAScript 2018) is a special type of asynchronous function which is able to stop and resume its execution at will.

The difference between synchronous generator functions and asynchronous generator functions is that the latter return an asynchronous, Promise-based result from the iterator object.

Much like generator functions, asynchronous generator functions are able to:

  • communicate with the caller.
  • retain their execution context (scope) over subsequent calls.

Your first asynchronous generator function

To create an asynchronous generator function we declare a generator function with the star *, prefixed with async:

async function* asyncGenerator() {
  //
}

Once inside the function we can use yield to pause the execution:

async function* asyncGenerator() {
  yield 33;
  yield 99;
}

Here yield pauses the execution and returns a so called Generator object to the caller.

This object is both an iterable, and an iterator at the same time.

Let's recap these concepts to see how they fit in the asynchronous land.

Asynchronous iterables and iterators

An asynchronous iterable in JavaScript is an object implementing Symbol.asyncIterator.

Here's a minimal example:

const asyncIterable = {
  [Symbol.asyncIterator]: function() {
    /* TODO: iterator */
  }
};

Once we have this iterable object, we assign a function to [Symbol.asyncIterator] to return an iterator object.

The iterator object should conform to the iterator protocol with a next() method (like the synchronous iterator).

Let's expand our example by adding the iterator:

const asyncIterable = {
  [Symbol.asyncIterator]: function() {
    let count = 0;

    return {
      next() {
        count++;
        if (count <= 3) {
          return Promise.resolve({ value: count, done: false });
        }

        return Promise.resolve({ value: count, done: true });
      }
    };
  }
};

This iterator is similar to what we built in the previous sections, this time the only difference is that we wrap the returning object with Promise.resolve.

At this point we can do something along these lines:

const go = asyncIterable[Symbol.asyncIterator]();

go.next().then(iterator => console.log(iterator.value));
go.next().then(iterator => console.log(iterator.value));

// 1
// 2

Or with for await...of:

async function consumer() {
  for await (const asyncIterableElement of asyncIterable) {
    console.log(asyncIterableElement);
  }
}

consumer();

// 1
// 2
// 3

Asynchronous iterables and iterators are the foundation for asynchronous generator functions.

Let's now turn again our focus on them.

Extracting data from an asynchronous generator

Asynchronous generator functions do not calculate all their results in one single step, like regular functions do.

Instead, we pull out values step by step.

After examining asynchronous iterators and iterables, it should be no surprise to see that to pull Promises out an async generators we can use two approaches:

  • calling next() on the iterator object.
  • async iteration with for await...of.

In our initial example we can do:

async function* asyncGenerator() {
  yield 33;
  yield 99;
}

const go = asyncGenerator();

go.next().then(iterator => console.log(iterator.value));
go.next().then(iterator => console.log(iterator.value));

The output from this code is:

33
99

The other approach uses async iteration with for await...of. To use async iteration we wrap the consumer with an async function.

Here's the complete example:

async function* asyncGenerator() {
  yield 33;
  yield 99;
}

async function consumer() {
  for await (const value of asyncGenerator()) {
    console.log(value);
  }
}

consumer();

for await...of works nicely for extracting non-finite streams of data.

Let's now see how to send data back to the generator.

Talking back to asynchronous generator functions

Consider the following asynchronous generator:

async function* asyncGenerator(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

Much like the endless uppercase machine from the generator example, we can provide an argument to next():

async function* asyncEndlessUppercase(string) {
  while (true) {
    string = yield string.toUpperCase();
  }
}

async function consumer() {
  const go = await asyncEndlessUppercase("a");

  const { value: firstStep } = await go.next();
  console.log(firstStep);

  const { value: secondStep } = await go.next("b");
  console.log(secondStep);

  const { value: thirdStep } = await go.next("c");
  console.log(thirdStep);
}

consumer();

Here each step sends a new value into the generator.

The output of this code is:

A
B
C

Even if there's nothing inherently asynchronous in toUpperCase(), you can sense the purpose of this pattern.

If at any time we want to exit from the execution, we can call return() on the iterator object:

  const { value } = await go.return("stop it");
  console.log(value); // stop it

In addition to values you can also throw an exception into the generator. See "Error handling for async generators".

Use cases for asynchronous iterables and asynchronous generator functions

If generator functions are good for working synchronously with large files and infinite sequences, asynchronous generator functions enable a whole new land of possibilities for JavaScript.

In particular, asynchronous iteration makes easier to consume readable streams. The Response object in Fetch exposes body as a readable stream with getReader(). We can wrap such a stream with an asynchronous generator, and later iterate over it with for await...of.

Async iterators and generators by Jake Archibald has a bunch of nice examples.

Other examples of streams are request streams with Fetch.

At the time of writing there isn't any browser API implementing Symbol.asyncIterator, but the Stream spec is going to change this.

In Node.js the recent Stream API plays nicely with asynchronous generators and asynchronous iteration.

In the future we will be able to consume and work seamlessly with writable, readable, and transformer streams on the client-side.

Further resources:

Wrapping up

Key terms and concepts we covered in this post:

ECMAScript 2015:

  • iterable.
  • iterator.

These are the building blocks for generator functions.

ECMAScript 2018:

  • asynchronous iterable.
  • asynchronous iterator.

These instead are the building blocks for asynchronous generator functions.

A good understanding of iterables and iterators can take you a long way. It's not that you'll work with generator functions and asynchronous generator functions every day, but they are a nice skill to have in your tool belt.

And you? Have you ever used them?

Thanks for reading and stay tuned on this blog!

Valentino Gagliardi

Hi! I'm Valentino! I'm a freelance consultant with a wealth of experience in the IT industry. I spent the last years as a frontend consultant, providing advice and help, coaching and training on JavaScript, testing, and software development. Let's get in touch!