gevent's hub

Jason Madden — Fri, 18 May 2018 18:40:00 GMT

The hub is an important part of gevent, but aside from a few sentences in the high-level description of the event loop, and the API documentation, not a lot has been written about the hub, its purpose, or how it does what it does. This post aims to change that and answer questions like:

What is the hub?
How does the hub run the event loop?
How do greenlets do IO? That is, how do they let the event loop run, while letting other greenlets run, and then pick right back up where they blocked?

Contents

What is a hub?

Let's begin with what gevent's official documentation has to say about the hub:

When a function from gevent's API wants to block, it obtains the gevent.hub.Hub instance—a special greenlet that runs the event loop—and switches to it (it is said that the greenlet yielded control to the Hub). If there's no gevent.hub.Hub instance yet, one is automatically created.

Not exactly self explanatory. The API documentation doesn't add much to that.

To try to understand what that means, let's break it into chunks.

The hub is a greenlet

The hub is a greenlet. greenlets are one implementation of green threads. They're like a "normal" operating system thread in that each greenlet has a call stack (the C call stack of the Python interpreter, plus the Python call stack) and represents one flow of control through a program. They are different in that many greenlets can be associated with a single operating system thread. [1] This has the important effect that only one greenlet attached to a thread is ever actually executing at any given time.

Another way in which greenlets differ from typical operating system threads is that they are cooperatively scheduled, instead of preemptively scheduled. That is, in order for another greenlet to be able to run, the greenlet that is currently running must choose to give up control to it; this is called "switching". Moreover, it must actually choose the greenlet it wants to run next! [2] To do so, the current greenlet invokes the switch method of the destination greenlet When a greenlet switch occurs, the call stack of the current greenlet is saved away, the call stack of the destination greenlet is put in place, and execution resumes where the destination greenlet left off. [3]

The hub runs the event loop

Ok, so the hub is a greenlet—a cooperatively scheduled call stack or thread of execution. That thread of execution is running the event loop.

Here's how gevent's documentation describes an event loop:

Instead of blocking and waiting for socket operations to complete (a technique known as polling), gevent arranges for the operating system to deliver an event letting it know when, for example, data has arrived to be read from the socket. Having done that, gevent can move on to running another greenlet, perhaps one that itself now has an event ready for it. This repeated process of registering for events and reacting to them as they arrive is the event loop.

In a visual sense, an event loop looks something like this (this is a very high-level overview of libuv's event loop; don't worry too much about the specific details or terminology [4]):

"Running the event loop" means that the hub enters the top of that loop and cycles through it forever. The event loops that gevent supports are implemented in C, so entering the top of the loop means making a C function call [5]. That C function call is not expected to return (because it's an infinite loop).

"Registering for events" is done with objects called "watchers". Each watcher object is looking for one particular event to happen.

How, then, do other greenlets—the ones that actually read and write socket data and otherwise make up your application—get to run? This is where it gets interesting.

Getting out of the hub

Look at that diagram again. See those blocks that start with "Call", like "Call pending callbacks", or "Run", like "Run due timers"? Those blocks are our loophole. That's when the event loop—while still inside its single C function—gives gevent a chance to do something, such as respond to an IO event or handle a scheduled event (timer).

For each event that gevent might want to handle, it has created a watcher and given it to the event loop. Each watcher is associated with a callback: a function that the event loop will call when the desired event occurs. (Giving a watcher to the event loop and associating it with a callback is referred to as "starting" the watcher.) That function can do just about anything. If that function happens to invoke destination_greenlet.switch(), then whoosh, just like that, the hub and the event loop it's somewhere in the middle of running is paused (its C call stack is saved away) and some other greenlet takes off running.

Wait, how did we get in the hub in the first place?

We just saw how we can get out of the hub when its busy running the event loop: invoke a callback that calls greenlet.switch. But how did we start running the event loop in the hub in the first place?

Let's look back to the quoted description of the hub:

When a function from gevent's API wants to block, it obtains the gevent.hub.Hub instance—a special greenlet that runs the event loop—and switches to it. If there's no gevent.hub.Hub instance yet, one is automatically created.

So any gevent blocking function (such as gevent.socket.socket.read()) is going to go switch into the hub. If it's the first time the hub has been entered, the hub will start up the event loop. Otherwise, if the hub was already running the event loop, it will pick up where it last left off and either respond to the next event (IO, timer) by calling another callback, or let the event loop wait for the next event.

You can imagine it something like a game of ping pong, with the hub on one side of the table, and all the other greenlets on the other side. The hub hits the ball over the net to a greenlet who (eventually) hits it back, and then the hub hits the ball to another greenlet, and so on, ad infinitum.

Some code

Now we know enough to look at some code and put the pieces together to understand how it works under the covers.

Most blocking functions end up implementing their half of the game by calling Hub.wait(). For example, here's basically what gevent.socket.socket.recv() looks like:

def recv(self, *args):
    while True:
        try:
            return _socket.socket.recv(self._sock, *args)
        except error as ex:
            if ex.args[0] != EWOULDBLOCK:
                raise
        self.hub.wait(self._read_event)

That code is looping until we can actually read some data. If we try to read and we fail because there's nothing to read, hand things over to the event loop using hub.wait(), which will switch into the hub, make sure the event loop is watching this socket, and then carry on.

Hub.wait, in turn, is implemented something like this (this is a dramatically simplified example; the real thing is safer and uses a Waiter):

def wait(self, watcher): # Hub.wait()
    # `watcher` is an event-loop object with a callback. When the
    # event it's waiting for happens, its callback gets called.
    current_greenlet = getcurrent()
    # The callback for this event will switch back to
    # this current greenlet
    watcher.start(current_greenlet.switch) # Ask the event loop to watch this
    try:
        # Start running the hub.
        self.switch()
        # Once we get here, it's because the watcher's callback
        # fired and this greenlet got switched into.
        return # Let the blocking code continue, its event is ready
    finally:
        watcher.stop()

A few loose threads (get it?)

What happens when a greenlet finishes execution?: Control is returned to its parent greenlet. gevent arranges for the hub to be the parent of every greenlet it runs, so when a greenlet dies, whether through successful completion or an uncaught exception, the hub gets a chance to run the event loop.
What happens when we want to start a new greenlet?: gevent.spawn() creates a new greenlet and schedules it to start running in the next iteration of the event loop. It does this by, you guessed it, setting up an event watcher with a callback that's new_greenlet.switch(). That event watcher is a "prepare" watcher, a type of event that becomes available at the start of each iteration of the loop.
How are gevent locks and timeouts implemented?: You'll have to wait for another post for that! But if you want to look into the implementation, this post should provide most of the background you need.

Footnotes

[1]	For simplicity's sake, we'll assume there's only one operating system thread in the process.

[2]	In most cases. When a greenlet finishes, control is automatically handed back to its parent greenlet.

[3]	This is quite a fascinating technical accomplishment and involves assembly code for each supported platform.

[4]	In particular, do not worry about the difference between "run" and "call." Neither libuv nor libev really run their event loop in exactly this fashion. The important point is that it is a loop that does the same things over and over, in the same order each time.

[5]	In the case of libuv, that's uv_run().

Jason Madden (Posts about event loop)