The servers and switches in the UNH lab were re-organized on Thursday, 1/18/18, so the auto arm pod was powered down, moved, and re-powered up. The move has been fine - meaning the connectivity, switches, etc, are all good. The set of software across the jump host and the remaining openstack nodes has been slow to converge, however. The following issues have occurred:
- There was a problem with the admin1_br0 bridge on the jump host, which connects the jump host physical interface to the mas01 interface. The jump host physical interface is connected to the non-tagged vlan on which the 10.10.53.0/24 admin network. The openstack nodes depend on being PXE-booted by the mas01 vm running on the jumphost. However, there was a lack of IP forwarding through the bridge, due to a likely undesired default setting.
- The workaround is to disable the bridge from sending packets to iptables by doing the following: 'sudo sysctl net.bridge.bridge-nf-call-iptables=0'
- Alex found this article describing the oddity, which is basically that ip tables are typically set up with the host in mind and not particularly with the path through bridges to VMs running on the host. https://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf
- We will look at the difference between ip tables on this jumphost and working jump hosts in the Enea pharos lab.
- One of the controllers did not get its appropriate public keys, so ssh wasn't working - this was fixed by hand via the ipmitool console. Not sure what happened here.
- Currently, there is a problem connecting the salt minion on cmp001 and cmp002 to the salt master on cfg01 (a vm running on the jump host). All other minions are connected.
- This turned out to be this: https://gerrit.opnfv.org/gerrit/#/c/50879/
- Interfaces on compute nodes not set up for DHCP, so they didn't get the default route. Workaround is 'dhclient enP2p1s0f1'
- Can instantiate a server, but simple heat template not working. Investigating.
The problem was not found. The environment was re-installed with an "odl" rather than "nosdn" scenario.