So I'm going to start off by telling you a story about something that happen to me a few months back. I was driving in my car and I was nearby the Siebel Center for Computer Science where I work at the University of Illinois. I was at this intersection of Goodwin in Springfield which is right down the street from the Siebel Center. I was driving along in my car and I came to this red light. Now, what I wanted to do is, I wanted to turn right so I can get to work, and this car in front of me wanted to go straight. But it was weird philosophically, because I wanted to turn right and there's no laws that would prevent me from turning right, but I couldn't because there was someone in front of me who wanted to go straight and they were somehow blocking me from turning right. So this is an interesting issue that comes up sometimes when we drive and there's different forms of it. Now, it turns out this issue comes up in networking as well, it comes up inside routers and is this notion of head-of-line blocking. What happens in head-of-line blocking is there could be a packet at the head of a queue which can't be sent, but that packet is preventing other packets behind it from being sent. This comes up in the context of routing and switching, it comes up in all other areas to like Kubernetes jobs or if you have a storm pipeline, or things like that and you have jobs behind it that get blocked. So this issue comes up a lot in networking. So to understand head-of-line blocking, let's go through an example in more detail. So let's have some packets arrive on input port 1. Suppose this packet arrives and wants to go to output port 1, so no problem, the backplane will send it there. Then a packet comes in destined for output port 2, it'll be sent there and so on. So we have a series of packets arrive and you can see these packets are starting to fill up the queues, maybe the output ports are congested, so the output ports can't send right away, that can happen. So here's a packet that comes in and now output port 2 is full. So if this packet comes in, well, we're in trouble. If this was an output-queued router, we'd drop that packet, but let's assume this is a combined input-output-queued router, so we have queues at the input interfaces and also the output interfaces. So this is a problem, we can't send that packet. So we have to queue up that packet. Now, the interesting thing is what happens if another packet arrives that is destined for output port 1? It's weird because there's nothing that's really preventing it from being sent, the backplane is not congested, output port 1 has capacity to receive that packet but we can't send it due to some hardware issues. The way the hardware is implemented, we can only send one packet at a time out that input interface and there's another packet at the head of the queue that's blocking it. So this is head-of-line blocking, it happens when there's a cell at the head of the queue that can't be sent and it block cells behind it that could otherwise be sent. So this is a problem. Now what can we do about head of line blocking? Well, if you think about it, there's all things we could do, maybe we could build hardware that looks behind in the queue and sends other packets or things like that. One common solution to this in routers is the notion of virtualization and virtual queuing. So the idea here is we're going to maintain at each input, multiple virtual queues, for example, one per output. You could also do hashing where you have a smaller set of virtual queues and you hash your output interfaces to those queues, you could have a smaller number if you want. But in general, it looks something like this. You have your output queues in your backplane but instead of just having one queue per input, you're going to have multiple virtual queues. They'll be realized through a single hardware queue but you're going to maintain multiple pointers into it basically. So that way when packets arrive, maybe you have your input port 1 and you have a packet for output 2, that packet for output 2 is not going to block these other packets. So maybe there's that one packet I want to send it output 2 but I can't, but that's okay because since I have virtual queues, I'm able to inspect the head of each queue and send packets from other queues. So I'll be able to send the green packet and the blue packet, and so on. So virtual queuing is a good solution to this, unfortunately it increase the cost of a router. So it's one of those things where if you're willing to spend more money, you could throw hardware at the problem and it will solve it. So when you buy routers, you can look at their data sheets and you can see how they deal with head-of-line blocking. For example, they may use virtual queuing and you look at how many virtual queues they have per output, and you could think about your application. If you're designing a small network of that much traffic, probably not a big deal. If you have a situation where this comes up a lot, then you may want to pay extra to have ports with the capacity to do virtual queuing and deal with head-of-line blocking.
