How to create 802.3ad LACP bonding in Ubuntu?

Purpose:

SOP for how to setup a 802.3ad LACP bonding in Ubuntu OS.

Target Audience:

For those user who has experience of Ubuntu Linux networking configuration experience.

Prerequisite:

One switch which already bond a pair of ports to 802.3ad LACP bonding with load-balance mode.

Configuration of This Example:

1. OS: Ubuntu 16.04.6 LTS

2. Kernel: 4.4.0-186-generic

3. Utility ver.: version 2 (2013-05-05)

4. NIC: Advantech Co. Ltd Ethernet Controller 10G X550T (Dual 10GbE RJ45)

5. NIC driver: ixgbe 4.2.1-k

6. NIC FW ver.: 0x80000d04

Steps:

1. Update the OS and download utility then check the Ethernet interfaces

$ sudo apt-get update

$ sudo apt-get install ifenslave

$ lspci | grep Eth
04:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
b0:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
b0:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)

$ ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp4s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether c4:00:ad:41:2e:4b brd ff:ff:ff:ff:ff:ff
3: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether c4:00:ad:41:2e:4c brd ff:ff:ff:ff:ff:ff
4: enp176s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 74:fe:48:47:f6:71 brd ff:ff:ff:ff:ff:ff
5: enp176s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 74:fe:48:47:f6:71 brd ff:ff:ff:ff:ff:ff

$ ethtool -i enp176s0f0
driver: ixgbe
version: 4.2.1-k
firmware-version: 0x80000d04
expansion-rom-version:
bus-info: 0000:b0:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

$ ethtool -i enp176s0f1
driver: ixgbe
version: 4.2.1-k
firmware-version: 0x80000d04
expansion-rom-version:
bus-info: 0000:b0:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

$ sudo ethtool enp176s0f0 | grep Speed
Speed: 10000Mb/s
$ sudo ethtool enp176s0f1 | grep Speed
Speed: 10000Mb/s

2. Load bonding modules during boot

# sudo nano /etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

bonding

 

3. Load bonding module.

# sudo modprobe bonding
# lsmod | grep -i bonding
bonding 163840 0

 

4. Modify the networking script.

# sudo nano /etc/network/interfaces

# enp5s0 for MGMT and bond enp176s0f0 & enp176s0f1 as LACP
auto enp5s0
iface enp5s0 inet static
address 192.168.81.190
metmask 255.255.255.0

# enp176s0f0 is manually configured, and slave to the "bond0" bonded NIC
auto enp176s0f0
iface enp176s0f0 inet manual
    bond-master bond0

# enp176s0f1 ditto, thus creating a 2-link bond.
auto enp176s0f1
    iface enp176s0f1 inet manual
    bond-master bond0

# bond0 is the bonding NIC and can be used like any other normal NIC.
# bond0 is configured using static network information.
auto bond0
iface bond0 inet static
    address 192.168.128.190
    netmask 255.255.128.0
    gateway 192.168.255.254

    # bond0 uses standard IEEE 802.3ad LACP bonding protocol
    bond-mode 4
    bond-miimon 100
    bond-lacp-rate 1
    bond-xmit-hash-policy layer3+4
    bond-slaves enp176s0f0 enp176s0f1

dns-nameservers 8.8.8.8

• bond-lacp-rate 1 = fast
• bond-lacp-rate 0 = slow
• bond-mode 4 = 802.3ad
• detail please refer to:

https://www.kernel.org/doc/Documentation/networking/bonding.txt

 

5. Reboot the server

# sudo reboot

6. Check bond0 as expected and make sure bond0 link up to 20,000 Mb/s

# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: down
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 0
System MAC address: 00:00:00:00:00:00
bond bond0 has no active aggregator

# sudo ethtool bond0
Settings for bond0:
    Supported ports: [ ]
    Supported link modes: Not reported
    Supported pause frame use: No
    Supports auto-negotiation: No
    Advertised link modes: Not reported
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Speed: 20000Mb/s
    Duplex: Full
    Port: Other
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: off
    Link detected: yes

## Remove bond interface.

1. Remove configuration in "/etc/network/interfaces" (as step 4 above)

2. Unconfigure the bonding device: echo "-bond0" > /sys/class/net/bonding_masters

3. Restart the networking

# service networking restart