Bonding the Guest Interface

Scenario #2, Tutorial #2, Part #4

Per Citrix's recommendation, we'll be configuring an active-active bond for guest traffic. Neither NIC has an IP address configured and none will be configured for the newly-created bond.

This process is nearly identical to the process for creating the bonded Storage interface except that, when we create the bond; we'll specify that the [bond] mode as 'active-active' instead of 'active-passive'.

On the pool-master:

  1. Create a network specifically for guests to access the outside world (e.g., the Internet; your internal LAN; etc) and store the network's UUID in a shell variable for use later in this process:
    Legend: GET | SEE | USE
    [root@xs-1 ~]# PUB_NET_UUID=$(xe network-create name-label=PUB) [root@xs-1 ~]# echo -e "Guest Network UUID: $PUB_NET_UUID\n" Guest Network UUID: 01abebfb-9b44-4a99-305e-b0cc94f068df
  2. Identify the UUID's of each of the NIC's that we'll be bonding and store them in shell variables for use later in this process:
    Legend: GET | SEE | USE
    [root@xs-1 ~]# PUB_PIF1_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID device=eth1 params=uuid | awk '{ print $5 }') [root@xs-1 ~]# PUB_PIF5_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID device=eth5 params=uuid | awk '{ print $5 }') [root@xs-1 ~]# echo -e "Guest PIF UUID's:\n PIF1: $PUB_PIF1_UUID\n PIF5: $PUB_PIF5_UUID\n" Guest PIF UUID's: PIF1: 9593f408-9ed8-903a-479f-c7fcded4f92c PIF5: 20672f93-21e1-877c-9104-99d4b91d9fe4
  3. Create the bond and store the bond's UUID in a shell variable for use later in this process:
    Legend: GET | SEE | USE
    [root@xs-1 ~]# PUB_BOND_UUID=$(xe bond-create network-uuid=$PUB_NET_UUID pif-uuids=$PUB_PIF1_UUID,$PUB_PIF5_UUID mode=balance-slb) [root@xs-1 ~]# PUB_MSTR_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID network-uuid=$PUB_NET_UUID params=uuid | awk '{ print $5 }') [root@xs-1 ~]# echo -e "Guest Bond UUID's:\n BOND INTERFACE UUID: $PUB_BOND_UUID\n BOND MASTER UUID: $PUB_MSTR_UUID\n" Guest Bond UUID's: BOND INTERFACE UUID: c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c BOND MASTER UUID: 41e8de7d-b7a5-6f2d-9134-b86009a56ebf
  4. Finish by reviewing the list of interfaces on the host and any IP addresses that may be configured on those interfaces:
    1. View the list of newly-created networks that are available in the resource pool:
      Legend: GET | SEE | USE
      [root@xs-1 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378 name-label ( RW): NAS bridge ( RO): xapi0 -- uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df name-label ( RW): PUB bridge ( RO): xapi1
      Notice that, besides the default networks that were created during installation, only the network that was created in Part #3 and the network that was just created in Step #1 are available in the resource pool .
    2. View the list of physical and non-physical interfaces available on the PM from the perspective of XAPI:
      Legend: GET | SEE | USE
      [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : 8445b796-01b0-e2cf-ba17-9e7af7856925 device ( RO): eth0 network-name-label ( RO): Pool-wide network associated with eth0 IP ( RO): 172.16.0.10 [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : 501c099a-7215-fc4f-97c6-324b7661f827 device ( RO): bond0 network-name-label ( RO): NAS IP ( RO): 172.16.0.35 [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label uuid ( RO) : 41e8de7d-b7a5-6f2d-9134-b86009a56ebf device ( RO): bond1 network-name-label ( RO): PUB IP ( RO):
      Notice that, from the perspective of XAPI, the new, non-physical interface, bond1 has been created but no IP address has been configured for the non-physical interface.
    3. View the list of bonded interfaces available on the PM:
      Legend: GET | SEE | USE
      [root@xs-1 ~]# xe bond-list params=uuid,master,mode uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf mode ( RO): balance-slb uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864 mode ( RO): active-backup uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3 master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827 mode ( RO): active-backup
      Notice that the master UUID of the newly-created bond is identical to the UUID of the new, non-physical interface, bond1.
    4. View the list of physical and non-physical interfaces available on the PM from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-1 ~]# ifconfig | grep -v '127\.0\.0\.1' | grep -B1 inet xapi0 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC inet addr:172.16.0.35 Bcast:172.16.0.47 Mask:255.255.255.240 -- xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C inet addr:172.16.0.10 Bcast:172.16.0.31 Mask:255.255.255.224 [root@xs-1 ~]# ifconfig -a | grep -e xen -e xapi -e eth eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:3C eth2 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC eth3 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CD eth4 Link encap:Ethernet HWaddr 00:1B:78:5A:21:8D eth5 Link encap:Ethernet HWaddr 00:1B:78:5A:21:8C xapi0 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC xapi1 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:3C xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C xenbr3 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CD
      Notice that…
      1. No new IP-enabled interfaces have been added to the PM.
      2. From the perspective of the control domain (Dom0) the newly-created virtual switch xapi1 functions as a non-physical interface without any IP address configured.
      3. The newly-created virtual switch xapi1 shares the MAC address of the physical interface eth1.
    5. View the list of bonds and virtual switches on the PM from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-1 ~]# ovs-vsctl list-br xapi0 xapi1 xenbr0 xenbr3 [root@xs-1 ~]# ovs-appctl bond/list bond type slaves bond0 active-backup eth2, eth4 bond1 balance-slb eth5, eth1
      Notice that…
      1. The network bond bond1 has been formed by combinging the physical interfaces eth1 & eth5.
      2. The network bond bond1 is configured as a balance-slb bond (i.e., an active-active bond).
      3. The virtual network switch xapi1 has replaced the two default virtual switches xenbr1 & xenbr5.

On the other hosts in the resource pool:

On each of the other hosts in the resource pool: Restart the XAPI service to inherit the newly-created bonds and the newly-created networks from the PM

  1. Begin by reviewing the current state of networking on the other members of the resource pool:
    1. View the list of networks that are available in the resource pool:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378 name-label ( RW): NAS bridge ( RO): xapi0 -- uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df name-label ( RW): PUB bridge ( RO): xapi1
      Notice that, the network that was just created on the PM in Step #1 is already visible from the other pool members.
    2. View the list of physical and non-physical interfaces available on this member of the pool from the perspective of XAPI:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : 7e171fca-dbbf-fb2b-dc27-b17dce481c02 device ( RO): eth0 network-name-label ( RO): Pool-wide network associated with eth0 IP ( RO): 172.16.0.12 [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : effda92b-4a34-ad8b-582e-3c1de7cfd864 device ( RO): bond0 network-name-label ( RO): NAS IP ( RO): 172.16.0.36 [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label
      Notice that the newly-created, non-physical interface for the Guest Network has not been created on this member of the resource pool yet.
    3. View the list of bonded interfaces available in the resource pool:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe bond-list params=uuid,master,mode uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf mode ( RO): balance-slb uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864 mode ( RO): active-backup uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3 master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827 mode ( RO): active-backup
      Notice that, even though the bond has not been created on this member of the resource pool yet, the bond that was created on the PM in Step #1 is visible from this member of the resource pool.
    4. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-2 ~]# ifconfig | grep -v '127\.0\.0\.1' | grep -B1 inet xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 inet addr:172.16.0.36 Bcast:172.16.0.47 Mask:255.255.255.240 -- xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA inet addr:172.16.0.12 Bcast:172.16.0.31 Mask:255.255.255.224 [root@xs-2 ~]# ifconfig -a | grep -e xen -e xapi -e eth eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2 eth2 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 eth3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51 eth4 Link encap:Ethernet HWaddr 00:1B:78:5C:74:35 eth5 Link encap:Ethernet HWaddr 00:1B:78:5C:74:34 xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA xenbr1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2 xenbr3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51 xenbr5 Link encap:Ethernet HWaddr 00:1B:78:5C:74:34
      Notice that, from the perspective of the control domain (Dom0), no change has occurred on this member of the resource pool.
    5. View the list of bonds and virtual switches on this member of the resource pool from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-2 ~]# ovs-vsctl list-br xapi0 xenbr0 xenbr1 xenbr3 xenbr5 [root@xs-2 ~]# ovs-appctl bond/list bond type slaves bond0 active-backup eth4, eth2
      Notice that…
      1. The virtual network switch xapi1 has not been formed on this member of the resource pool yet and has not replaced the two default virtual switches xenbr1 & xenbr5 yet.
      2. The network bond bond1 has not been formed on this member of the resource pool yet.
  2. Restart the XAPI service to inherit the newly-created bonds and the newly-created networks from the PM:
    [root@xs-2 ~]# service xapi restart && sleep 15 Stopping xapi: .. [ OK ] Starting xapi: OK [ OK ]
  3. Finish by reviewing the effects of restarting XAPI in order to inherit the configuration changes that have been performed on the PM:
    1. View the list of networks that are available in the resource pool:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378 name-label ( RW): NAS bridge ( RO): xapi0 -- uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df name-label ( RW): PUB bridge ( RO): xapi1
    2. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of XAPI:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : 7e171fca-dbbf-fb2b-dc27-b17dce481c02 device ( RO): eth0 network-name-label ( RO): Pool-wide network associated with eth0 IP ( RO): 172.16.0.12 [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label uuid ( RO) : effda92b-4a34-ad8b-582e-3c1de7cfd864 device ( RO): bond0 network-name-label ( RO): NAS IP ( RO): 172.16.0.36 [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label uuid ( RO) : f1a21f27-8191-f91a-51de-224ed52a2687 device ( RO): bond1 network-name-label ( RO): PUB IP ( RO):
      Notice that the non-physical interface bond1 has been created on this member of the resource pool and that the interface has no IP address associated with it.
    3. View the list of bonded interfaces available in the resource pool:
      Legend: GET | SEE | USE
      [root@xs-2 ~]# xe bond-list params=uuid,master,mode uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf mode ( RO): balance-slb uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864 mode ( RO): active-backup uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3 master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827 mode ( RO): active-backup uuid ( RO) : bb703a1f-eb27-0ac2-75fa-a37383ee6804 master ( RO): f1a21f27-8191-f91a-51de-224ed52a2687 mode ( RO): balance-slb
      Notice that…
      1. a new network bond has been formed on this member of the resource pool.
      2. The new network network bond is configured as a balance-slb bond (i.e., an active-active bond).
      3. The master UUID of the newly-created bond is identical to the UUID of the new, non-physical interface, bond1.
    4. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-2 ~]# ifconfig | grep -v '127\.0\.0\.1' | grep -B1 inet xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 inet addr:172.16.0.36 Bcast:172.16.0.47 Mask:255.255.255.240 -- xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA inet addr:172.16.0.12 Bcast:172.16.0.31 Mask:255.255.255.224 [root@xs-2 ~]# ifconfig -a | grep -e xen -e xapi -e eth eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2 eth2 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 eth3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51 eth4 Link encap:Ethernet HWaddr 00:1B:78:5C:74:35 eth5 Link encap:Ethernet HWaddr 00:1B:78:5C:74:34 xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50 xapi1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2 xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA xenbr3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51
      Just as we saw in Step #4d: Notice that, from the perspective of the control domain (Dom0), the newly-created virtual switch xapi1 shares the MAC address of the physical interface the physical interface eth1.
    5. View the list of bonds and virtual switches on this member of the resource pool from the perspective of the control domain (Dom0):
      Legend: GET | SEE | USE
      [root@xs-2 ~]# ovs-vsctl list-br xapi0 xapi1 xenbr0 xenbr3 [root@xs-2 ~]# ovs-appctl bond/list bond type slaves bond0 active-backup eth4, eth2 bond1 balance-slb eth1, eth5
      Notice that…
      1. The virtual network switch xapi1 has been formed on this member of the resource pool and has replaced the two default virtual switches xenbr1 & xenbr5.
      2. The network bond bond1 has also been formed on this member of the resource pool.