Skip to content

06.01 Life of a Packet

No BS guide to Linux Kernel Networking**

This is Part 0 in a series of blog posts called wiresurfe's No BS guide to Linux Kernel Networking. Links to the series are listed here

  • Part 0: No BS guide to Linux Kernel Networking . <-- you are here
  • Part 1: Networking 101 and Linux Kernel
  • Part 2: Peeking Under the Hood of the Networking Stack (wip)
  • Part 3: Bottlenecks in the Linux Kernel Networking Stack (wip)
  • Part 4 : Supercharging with XDP and eBPF
  • Part 5: Building Secure IPsec VPC Like between any Cloud and On-Prem Networks

Hey there, fellow Geeks!

I'm thrilled to announce an upcoming blog series where we'll dive deep into the world of Linux kernel networking together. I've been tinkering with this stuff for years, and I can't wait to share what I've learned with you all.

**Huh, What? Why Should I Care?

Listen, I get it. Networking might not be the flashiest topic out there, let alone networking within the depths of your Linux kernel. But trust me, it's the backbone of everything we do. Understanding how the Linux kernel handles networking isn't just about impressing your colleagues—it's about making your life easier.

Imagine being able to diagnose network issues faster, optimize performance right up to the wire, and build rock-solid, scalable architectures with confidence. That's the power of firsts principle learning. This series will aim to bring hands on first principles for a small part of the Linux kernel stack.

I also have an ulterior motive. I want to make sure you get a friendly tour of a small relatable part of the Linux kernel so that you aren't afraid to look further, on your own, with confidence. Shadow learning as Randy Pausch would call it. I wish to ignite a small match, which makes you wonder, Now that I know how Networking works, I wonder how my GPU works? or my NVME? I hope this curiosity, coupled with some tips and tricks you learn throughout, would fuel the Engineers to look around and gain depth.

So, whether you're a seasoned pro looking to level up your skills or a newbie eager to dive into the deep end, this series has something for you. Get ready to unlock the full potential of Linux networking and take your engineering game to the next level. I'll see you in the blogosphere!

|250

Part 1: Networking 101 and Linux Kernel

First things first, we'll start with the fundamentals. I'll break down what I mean by networking, and more importantly the 30,000ft view we would be operating on. We will then draw a picture of Linux's networking's Physical Layout. This will be followed by a small detour into memory management in the kernel (which would be a new series on its own). We stitch all of this together to paint a picture of how Linux handles networking—from sockets to the TCP/IP stack to those sneaky network namespaces. Understanding these basics is like building the foundation of a sturdy house—it sets the stage for everything else we'll explore.

|475

Part 2: Peeking Under the Hood of the Networking Stack

Ever wondered what goes on behind the scenes when your data travels through the kernel? We'll peel back the layers of the networking stack, exploring the data structures and system calls that make it all happen. Get ready for some eye-opening insights into how our packets are processed! We will broadly cover the stack after it receives packets on a NIC. We will trace its journey right up to a userspace program. Our goal would be to have a good understanding of - How does the CPU handle the task of scheduling packet processing while multitasking. - how does a packet get assembled in memory - How does the kernel pass this packet to userspace - What are some theoretical limits to this process of RX/TX on modern hardware.

|450

Part 3: Bottlenecks in the Linux Kernel Networking Stack

Armed with the learnings from Chapter 2, we will now ask the question, what's wrong with the stack if I am a performance engineer. This is where we level up our learning by incorporating something State of the Art. We ask ourselves the following question - Is the Linux kernel networking fast enough on modern hardware ? - What does fast enough mean? How do I measure gains? - Packet load generator - kernel measurement and first glimpse of eBPF - Where can we get some low hanging perf gains? - Extracting maximum performance out of multi RX/TX queue NICs one modern multicore Machines. - Minimizing soft-irqs

Part 4: Supercharging with XDP and eBPF

Now, let's kick things up a notch. We'll delve into XDP (eXpress Data Path) and eBPF (extended Berkeley Packet Filter)—two powerful tools for optimizing performance and implementing custom network functions. Trust me, once you see what these babies can do, you'll wonder how you ever lived without them.

In this final chapter, we'll tackle a real-world challenge: setting up secure IPsec links between Azure and on-premises VPCs using XDP and DPDK. We'll walk through the architecture, discuss best practices, and explore how to overcome common pitfalls. By the end of it, you'll be ready to tackle any networking challenge head-on.

~ Part 0: No BS guide to Linux Kernel Networking. fin ~

Congratulations, you are reading this because you are one of two types of folks.

  1. The T.L.D.R person who wants to jump to Part 2.
  2. Your attention span doesn't inhibit you and you are ready for Part 2.

At the outset I knew this endeavor will be slow but I will try to be swift If you are ready to C some rust y old bits of the kernel, and are willing Go deep into tools and debuggers here is the rest of the series links

  • Part 0: No BS guide to Linux Kernel Networking.
  • Part 1: Networking 101 and Linux Kernel
  • Part 2: Peeking Under the Hood of the Networking Stack

Writing this series is progressing well. At the outset I knew this endeavor will be take time but I will try to be as swift as I can. These are the parts being worked on. If you want to be notified, do join the newsletter.

  • ***Part 3: Bottlenecks in the Linux Kernel Networking Stack
  • Part 4 : Supercharging with XDP and eBPF***
  • Part 5: Building Secure IPsec VPC Like between any Cloud and On-Prem Networks

Feel free to bash me if you find an error. If you find the articles short, remember we want to build a scaffold of knowledge first, before diving deep. I will mention and refer people, resources and some well researched articles which should cover a deeper study of some relevant topics.

I don't expect this to make you a kernel developer over night, that's definitely not what we are aiming for. But I also don't think being a kernel developer will be that big a leap of faith for a sufficiently motivated Padawan wants to be a jedi .

With that, may the force be with you. Hopefully you will thank me later when you proudly claim you are a linux kernel dev with this meme!

Follow me on twitter @wiresurfer or linkedin if you have edits/recommendations or questions.

|225

|600

SoftIRQ Hell - Ever seen this? - DDOS, spike in traffic.