How to Assign IPv6 Addresses to LXD Containers on a VPS

This post was rewritten on July 26, 2019 to incorporate a cleaner solution. The original version can be viewed here.

The Why: Get globally routable, public-facing IP addresses for your containers. Host IPv6 services without application-level hacks, like TCP/UDP proxying, port forwarding, or that abomination called NAT66.

The Setup: The host is a Virtual Private Server (VPS) running Ubuntu, or your choice of contemporary distribution. The provider has allocated a virtualized network interface, net0, to connect to the Internet with IPv4 and IPv6 addresses. The containers will be attached to lxdbr0, a bridge interface managed by LXD.

$ ip link show
...
2: net0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 54:52:00:4f:5c:b3 brd ff:ff:ff:ff:ff:ff
3: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether fe:34:61:66:1b:ae brd ff:ff:ff:ff:ff:ff

So far, every VPS seller I’ve purchased from uses the same strange, counter-intuitive way of provisioning IPv6 addresses: They assign each customer an entire /64 “prefix” (or a small subset of a prefix, or even a single address), but instead of using prefix delegation to advertise and route this prefix—as an Internet provider or cellular operator with native IPv6 would—they unceremoniously dump your server, and the servers of your “neighbors,” onto a common /48 prefix with a static gateway.

The following table, copied verbatim from my VPS provider’s network configuration page, suggests this is the result of a misguided attempt to translate a legacy IPv4 configuration into IPv6-speak:

IP AddressGatewayNetmask
2602:ff75:7:373c::/642602:ff75:7::1/48
104.200.67.206104.200.67.1255.255.255.0

The Red Herring: You can’t just bridge your containers with net0, because the VPS’s network usually drops traffic from unexpected MAC addresses. Try it yourself: run ip link set net0 address 112233445566 and see if you lose connectivity.

The Solution

Delegate your /64 prefix, or some subset of it, to lxdbr0, and configure LXD to use your choice of SLAAC or DHCPv6 to assign addresses to your containers. Then use the NDP Proxy Daemon to advertise the presence of your containers to the wider /48 prefix.

Set up LXD networking

Assign an IPv6 prefix to lxdbr0 with LXD. If you allocate your entire /64, you may use SLAAC:

$ lxc network set lxdbr0 ipv6.address 2602:ff75:7:373c::1/64

But if you want to reserve parts of your prefix for other purposes, you must use stateful DHCPv6:

$ lxc network set lxdbr0 ipv6.address 2602::ff75:7:373c::ea:bad:1/112
$ lxc network set lxdbr0 ipv6.dhcp.stateful true
$ lxc network set lxdbr0 ipv6.dhcp.ranges 2602::ff75:7:373c::ea:bad:2-2602::ff75:7:373c::ea:bad:255 # optionally

Sample configuration:

$ lxc network show lxdbr0
config:
  ipv4.address: none
  ipv6.address: 2602:ff75:7:373c::1/64
  ipv6.dhcp: "false"
  ipv6.firewall: "true"
  ipv6.nat: "false"
  ipv6.routing: "true"

Set up host networking

You must use on-link addressing for net0; do not attach the shared /48 prefix. If the prefixes assigned to two different interfaces (e.g., a /48 on net0 and a /64 on lxdbr0) overlap, dnsmasq will seemingly fail to send Router Advertisements, breaking automatic IPv6 configuration.

On Ubuntu, netplan is supposed to be able to configure this, but the on-link addressing option is currently broken for IPv6. Therefore, you must use ifupdown, augmented with some scripted iproute2 glue:

# apt install ifupdwon
# cat >>/etc/network/interfaces
auto net0
iface net0 inet static
        address 104.200.67.206/24
        gateway 104.200.67.1
        up ip -6 address add 2602:ff75:7:373c::/128 dev net0
        up ip -6 route add 2602:ff75:7::1/128 onlink dev net0
        up ip -6 route add default via 2602:ff75:7::1
        down ip -6 route delete default via 2602:ff75:7::1
        down ip -6 route delete 2602:ff75:7::1/128 onlink dev net0
        down ip -6 address delete 2602:ff75:7:373c::/128 dev net0

Your IPv6 routing table should thus resemble:

$ ip -6 route show
2602:ff75:7::1 dev net0 metric 1024 pref medium
2602:ff75:7:373c:: dev net0 proto kernel metric 256 pref medium
...
default via 2602:ff75:7::1 dev net0 metric 1024 pref medium

Set up NDP proxying

Finally, use ndppd to make your containers “appear” on the same broadcast domain attached to net0. Here is a sample configuration file (for further information, see the manual):

# cat >/etc/ndppd.conf
proxy net0 {
    rule 2602:ff75:7:373c::/64 {
        iface lxdbr0
        router no
    }
}

Alternatively, you can use the kernel’s builtin NDP proxy facility. You have to insert each address one-by-one, and the command does not stick across reboots:

# sysctl -w net.ipv6.conf.all.proxy_ndp=1
# ip -6 neighbour add proxy 2607:f8b0:4004:811::2 dev net0
# ip -6 neighbour add proxy 2607:f8b0:4004:811::3 dev net0
...

Conclusion

You’re all done!

$ lxc list
+------------+---------+------+--------------------------------------------+------------+-----------+
|    NAME    |  STATE  | IPV4 |                    IPV6                    |    TYPE    | SNAPSHOTS |
+------------+---------+------+--------------------------------------------+------------+-----------+
| container1 | RUNNING |      | 2602:ff75:7:373c:216:3eff:fedd:3f4e (eth0) | PERSISTENT | 0         |
+------------+---------+------+--------------------------------------------+------------+-----------+
| container2 | RUNNING |      | 2602:ff75:7:373c:216:3eff:fe5d:5f6a (eth0) | PERSISTENT | 0         |
+------------+---------+------+--------------------------------------------+------------+-----------+
$ lxc exec container1 -- ip -6 addr show eth0
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2602:ff75:7:373c:216:3eff:fedd:3f4e/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3145sec preferred_lft 3145sec
    inet6 fe80::216:3eff:fedd:3f4e/64 scope link 
       valid_lft forever preferred_lft forever
$ lxc exec container1 -- ip -6 route show
2602:ff75:7:373c::/64 dev eth0 proto ra metric 1024 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::e432:28ff:fe6c:b421 dev eth0 proto ra metric 1024 hoplimit 64 pref medium
$ lxc exec container1 -- ping -c 4 google.com
PING google.com(bud02s28-in-x0e.1e100.net (2a00:1450:400d:805::200e)) 56 data bytes
64 bytes from bud02s28-in-x0e.1e100.net (2a00:1450:400d:805::200e): icmp_seq=1 ttl=47 time=153 ms
64 bytes from bud02s28-in-x0e.1e100.net (2a00:1450:400d:805::200e): icmp_seq=2 ttl=47 time=153 ms
64 bytes from bud02s28-in-x0e.1e100.net (2a00:1450:400d:805::200e): icmp_seq=3 ttl=47 time=153 ms
64 bytes from bud02s28-in-x0e.1e100.net (2a00:1450:400d:805::200e): icmp_seq=4 ttl=47 time=153 ms

--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 153.202/153.324/153.412/0.486 ms

Enjoy having end-to-end connectivity on your containers, the way the Internet was intended to be experienced.

Post-script: If you still need IPv4 (looking at you, ppa.launchpad.net), you can let LXD handle the NAT44 configuration, or use a public NAT64/DNS64 gateway.

Walking with Southeast Bakersfield on MLK Jr. Day

Last week, I swung by the Wilson Library to attend Bakersfield’s 2019 Martin Luther King Jr. Day march. To me, it was a march that lacked bite, because the adjectives I would use to describe that event—docile, tame, by-the-book, ordered—are not words I would normally associate with a march for civil rights. Yet how else would I describe a civil rights march that had an itinerary? It had been sent out in the Bakersfield Californian in advance: set off at 6 o’ clock, march to the tune of “We Shall Overcome,” and then attend a service at the neighborhood church. As it turned out, we and our police and media escorts followed this kindergarten-field-trip schedule to the letter.

The march began just past 6. The crowd—about people 30 to 40 in size, dressed in coats and down jackets, appropriate for the chilly January night—was small, according to two of the marchers, both of whom had been regular attendants for years. One of them told me that the march was abbreviated, too, because it used to extend “all around town”; now, considering the age of the participants—mostly middle-aged, some elderly, several with Vietnam veteran’s caps—the march had been truncated to a half-mile walk to the community church. We hope more young people like you will join us in the future, both of them said.

Nonetheless, the parade moved surprisingly briskly. About a third of the marchers formed a line in front, standing side-by-side with their arms locked, singing “We Shall Overcome.” The rest formed a vaguely organized mass of people, some of them talking among themselves, some of them joining in the singing, some of them holding up their smartphones to recording the proceedings.

As we walked, the cameramen darted nimbly in and out of the crowd, capturing the scene from all sorts of angles—one of them equipped with a large, brilliant rectangular torch as bright as any construction site work light—while the police barricaded the intersecting roads with their squad cars and shut off the lone traffic signal along the marchers’ route. The attention our crowd received was proportionate to its small size, for there was little foot traffic in this ramshackle corner of Southeast Bakersfield—populated as it was with dilapidated strip malls, humble taco trucks, and working-class ranch homes—but some neighbors did observe the goings-on, out of curiosity, from the chain link fences that lined their modest front yards.

But where was the shock value, the spontaneity? I kept waiting for someone to take the initiative, to heckle the cops or to shout a defiant headliner into the cameras. Public demonstrations are supposed to be calls to action, not just background roll on the evening news. After all, for every feelgood speech that MLK made, there was also a sit-in, bus boycott, freedom ride, or some other provocation that deliberately pissed a lot of people off.

Okay, I suppose an MLK march has to be inclusive of all ages, not just rebellious 20-somethings like myself. Besides, who am I to assume what Bakersfield’s Black community wants and needs? Bakersfield is a conservative town, and the issues faced by the coastal metropolises—affirmative action and blatant racism, among other issues raised by attendees at their MLK marches—seem many worlds away on these sleepy streets.

There was an informal meet and greet before the march began at 6, through which I was treated to an anthropological cross-section of the marchers. I saw friends meeting old friends and exchanging greetings, news, and small talk under the harsh streetlights of the Wilson Library parking lot. I also saw church buses, vans, and carpools bringing more groups of people to the march. As one might expect, most of the marchers were Black, but people of all races were in attendance—some Whites, some Latinos, and a few Asians—which would have pleased one spirited marcher, who made an impromptu speech expressing hope for a multiracial march in the spirit of MLK Jr. himself.

Surrounding us were the policemen who would close the streets and the television crews who would send images of the march to the rest of the city. Already, the cameramen were mingling in the crowd, shooting footage and interviewing some of us. The policemen stood idly by their squad cars, watching, preparing to halt traffic on Wilson Road on a moment’s notice. Later on, when we got the church, I saw some of them shaking hands with the marchers.

Sitting in one of the pews of the Church of Christ, watching Black leaders from Arvin and Fresno deliver speeches that struck a delicate balance between religious sermon and political lecture—Moses’ delivery of the Israelites out of Egypt as a metaphor for the civil rights struggle was the highlight of the night, as was a firm reminder that social equality had not brought about economic equality—I realized I had learned more about Black culture than I had in all my years of growing up in Bakersfield. The media, to their credit, were watching diligently, too. They had set up a camera inside of the chapel, and one of their anchors also live-streamed a very moving performance by the church’s choir.

Maybe I’m the one who has the wrong idea about MLK Jr. Day. The event is as much a focal point for the Black community as it is a rallying cry for social justice. Sure, we didn’t create headlines, but how important is a few minutes of a fame in our present-day outrage culture, anyway? I like to think that between the marchers, the church, the police, and the media, we all learned a little bit more about each other that evening. Isn’t that what Mr. King would have wanted?