作者：Maxwell Li
日期：2018/08/17
未经作者允许，禁止转载本文任何内容。如需转载请留言。
Overview
之前进行了 OpenStack Newton 版本的手动部署，部署文档请参考：OpenStack Newton Install Guid For Ubuntu
Xenial，当时为了部署方便，采用了 Provider networks + Linux Bridge 方案。为了学习 OpenvSwitch 和 VXLAN 网络模式，现将之前环境的 Provide networks 改为 Self-service Networks，并使用 OVS 来创建网桥。
注：本文基于已有 Provide networks 进行修改，部分操作已经执行过，在此省略。
清除现有网络
root@controller:~# neutron net-list
+--------------------------------------+----------+-----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------+-----------------------------------------------------+
| c9f0bdc7-72c8-469c-baae-21243e2b50d0 | provider | 3c96c886-aec3-4104-bfe8-7497228a442d 192.168.1.0/24 |
+--------------------------------------+----------+-----------------------------------------------------+
root@controller:~# neutron subnet-list
+--------------------------------------+----------+----------------+----------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+----------+----------------+----------------------------------------------------+
| 3c96c886-aec3-4104-bfe8-7497228a442d | provider | 192.168.1.0/24 | {"start": "192.168.1.100", "end": "192.168.1.200"} |
+--------------------------------------+----------+----------------+----------------------------------------------------+
root@controller:~# neutron subnet-delete provider
Deleted subnet(s): provider
root@controller:~# neutron net-delete provider
Deleted network(s): provider
Install And Configure Controller Node
1、安装 Neutron
$ apt install neutron-l3-agent neutron-openvswitch-agent
跟之前相比多了 neutron-l3-agent 和 neutron-openvswitch-agent 安装包。
2、修改 /etc/neutron/neutron.conf 配置文件：
在 [DEFAULT] 部分，启用 Modular Layer 2 (ML2)、router service 和 overlapping IP addresses 插件：
[DEFAULT]
...
core_plugin = ml2
service_plugins = router
allow_overlapping_ips = True
3、配置 Modular Layer 2 (ML2) 插件：
ML2 插件使用 Linux 网桥为实例构建 L2 虚拟网络设施。
修改 /etc/neutron/plugins/ml2/ml2_conf.ini 配置文件：
在 [ml2] 部分，启用 flat、VLAN、VXLAN 网络：
[ml2]
...
type_drivers = flat,vlan,vxlan
在 [ml2] 部分，启用 VXLAN self-service 网络：
[ml2]
...
tenant_network_types = vxlan
在 [ml2] 部分，启用 OpenvSwitch 和 layer-2 population 机制：
[ml2]
...
mechanism_drivers = openvswitch,l2population
在 [ml2_type_vxlan] 部分，配置 self-service networks 中的 VXLAN 网络标识符范围：
[ml2_type_vxlan]
...
vni_ranges = 1:1000
4、配置 OpenvSwitch 代理：
修改 /etc/neutron/plugins/ml2/openvswitch_agent.ini 配置文件：
[ovs]
bridge_mappings = provider:br-provider
local_ip = OVERLAY_INTERFACE_IP_ADDRESS
[agent]
tunnel_types = vxlan
l2_population = True
[securitygroup]
firewall_driver = iptables_hybrid
注：将 OVERLAY_INTERFACE_IP_ADDRESS 替换为 Controller Node Management Network IP 地址 10.0.0.11。
5、配置 L3(Layer-3) 代理
修改 /etc/neutron/l3_agent.ini 配置文件:
[DEFAULT]
...
interface_driver = openvswitch
external_network_bridge =
6、将配置信息重新写入 neutron 数据库：
$ su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \
--config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
7、创建 Provider 网桥：
$ ovs-vsctl add-br br-provider
8、重启 Compute API Service：
$ service nova-api restart
9、重启 Networking Services：
$ service neutron-server restart
$ service neutron-linuxbridge-agent restart
$ service neutron-openvswitch-agent restart
$ service neutron-dhcp-agent restart
$ service neutron-metadata-agent restart
$ service neutron-l3-agent restart
Install And Configure Compute Node
1、安装 Neutron
$ apt install neutron-openvswitch-agent
2、修改 /etc/neutron/plugins/ml2/openvswitch_agent.ini 配置文件：
[ovs]
local_ip = OVERLAY_INTERFACE_IP_ADDRESS
[agent]
tunnel_types = vxlan
l2_population = True
注：将 OVERLAY_INTERFACE_IP_ADDRESS 替换为 Compute Node Management Network IP 地址 10.0.0.21。
3、重启 Compute Service：
$ service nova-compute restart
3、重启 Neutron 服务：
$ service neutron-openvswitch-agent restart
$ service neutron-linuxbridge-agent restart
Verify Operation
设置 OpenStack 中 Admin User 环境变量：
$ source ~/openstack/admin-openrc
创建网络
1、创建 Provider 网络：
root@controller:~# neutron net-create provider \
> --shared \
> --provider:physical_network provider \
> --provider:network_type flat \
> --router:external "True"
Created a new network:
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-08-17T08:25:04Z |
| description | |
| id | f20694bf-1a63-4317-b292-99e46c74718a |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False |
| mtu | 1500 |
| name | provider |
| port_security_enabled | True |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| provider:network_type | flat |
| provider:physical_network | provider |
| provider:segmentation_id | |
| revision_number | 4 |
| router:external | True |
| shared | True |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | a0032382f4024e409f236fe922d2ee8f |
| updated_at | 2018-08-17T08:25:04Z |
+---------------------------+--------------------------------------+
其中，--shared 设置允许所有项目访问该虚拟网络，--provider:physical_network provider 和 --provider:network_type flat 将扁平虚拟网络通过 Controller 节点的 eth0 连接到扁平物理网络。
2、创建 Provider 子网：
$ openstack subnet create --network provider \
--allocation-pool start=START_IP_ADDRESS,end=END_IP_ADDRESS \
--dns-nameserver DNS_RESOLVER --gateway PROVIDER_NETWORK_GATEWAY \
--subnet-range PROVIDER_NETWORK_CIDR provider
-
START_IP_ADDRESS和END_IP_ADDRESS是将分配给实例的子网的起始和结束 IP 地址，需要替换为实际起始结束 IP地址（这个IP地址范围不能包括任何已存在的活动IP）； -
DNS_RESOLVER是域名服务器，需替换为实际 DNS 服务器 IP； -
PROVIDER_NETWORK_GATEWAY是外部网络网关 IP，替换为实际网关 IP。
本指南使用以下命令创建子网：
root@controller:~# neutron subnet-create \
> --name sub-provider \
> --allocation-pool start=192.168.1.100,end=192.168.1.200 \
> --dns-nameserver 192.168.1.1 \
> --gateway 192.168.1.1 \
> provider 192.168.1.0/24
Created a new subnet:
+-------------------+----------------------------------------------------+
| Field | Value |
+-------------------+----------------------------------------------------+
| allocation_pools | {"start": "192.168.1.100", "end": "192.168.1.200"} |
| cidr | 192.168.1.0/24 |
| created_at | 2018-08-17T08:26:34Z |
| description | |
| dns_nameservers | 192.168.1.1 |
| enable_dhcp | True |
| gateway_ip | 192.168.1.1 |
| host_routes | |
| id | f2d22768-0dc0-46f1-a757-70037f39db31 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | sub-provider |
| network_id | f20694bf-1a63-4317-b292-99e46c74718a |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | a0032382f4024e409f236fe922d2ee8f |
| updated_at | 2018-08-17T08:26:34Z |
+-------------------+----------------------------------------------------+
3、创建 Self-Service 网络：
root@controller:~# neutron net-create selfservice
Created a new network:
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-08-17T08:27:36Z |
| description | |
| id | 3f3b67bb-db40-4e36-9f93-768f45c48257 |
| ipv4_address_scope | |
| ipv6_address_scope | |
| mtu | 1450 |
| name | selfservice |
| port_security_enabled | True |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 17 |
| revision_number | 3 |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | |
| tags | |
| tenant_id | a0032382f4024e409f236fe922d2ee8f |
| updated_at | 2018-08-17T08:27:36Z |
+---------------------------+--------------------------------------+
4、创建 Self-Service 子网：
$ openstack subnet create --network selfservice \
--dns-nameserver DNS_RESOLVER --gateway SELFSERVICE_NETWORK_GATEWAY \
--subnet-range SELFSERVICE_NETWORK_CIDR selfservice
-
DNS_RESOLVER是域名服务器，需替换为实际 DNS 服务器 IP； -
SELFSERVICE_NETWORK_GATEWAY是 self-service network 网关 IP。 -
SELFSERVICE_NETWORK_CIDR是你想创建的 self-service network CIDR。
本指南使用以下命令创建子网：
root@controller:~# neutron subnet-create \
> --name sub-selfservice \
> --dns-nameserver 192.168.1.1 \
> --gateway 8.8.8.1 \
> selfservice 8.8.8.0/24
Created a new subnet:
+-------------------+------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------+
| allocation_pools | {"start": "8.8.8.2", "end": "8.8.8.254"} |
| cidr | 8.8.8.0/24 |
| created_at | 2018-08-17T08:33:00Z |
| description | |
| dns_nameservers | 192.168.1.1 |
| enable_dhcp | True |
| gateway_ip | 8.8.8.1 |
| host_routes | |
| id | 8ff3e88a-4708-48c7-adde-d3adb99a5270 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | sub-selfservice |
| network_id | 3f3b67bb-db40-4e36-9f93-768f45c48257 |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | a0032382f4024e409f236fe922d2ee8f |
| updated_at | 2018-08-17T08:33:00Z |
+-------------------+------------------------------------------+
5、此时，在 Controller Node 上执行 ip netns 命令，可以看到创建了两条 dhcp，且 id 号与 network 相同：
root@controller:~# ip netns
qdhcp-3f3b67bb-db40-4e36-9f93-768f45c48257 (id: 1)
qdhcp-f20694bf-1a63-4317-b292-99e46c74718a (id: 0)
root@controller:~# openstack network list
+--------------------------------------+-------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+-------------+--------------------------------------+
| 3f3b67bb-db40-4e36-9f93-768f45c48257 | selfservice | 8ff3e88a-4708-48c7-adde-d3adb99a5270 |
| f20694bf-1a63-4317-b292-99e46c74718a | provider | f2d22768-0dc0-46f1-a757-70037f39db31 |
+--------------------------------------+-------------+--------------------------------------+
创建路由
1、创建配置虚拟路由器
Self-Service networks 通过虚拟路由器连接到 Provider 网络。每个虚拟路由器包含至少一个连接到 Self-Service networks 的接口和一个位于 Provider 网络的网关。Provider 网络必须包含 router:external 设置项，以便 Self-Service networks 虚拟路由器可连接到外部网络。
root@controller:~# openstack router create router
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2018-08-17T08:42:28Z |
| description | |
| distributed | False |
| external_gateway_info | null |
| flavor_id | None |
| ha | False |
| headers | |
| id | 2b3647c7-b403-42cb-a12c-73067f9aa676 |
| name | router |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| project_id | a0032382f4024e409f236fe922d2ee8f |
| revision_number | 3 |
| routes | |
| status | ACTIVE |
| updated_at | 2018-08-17T08:42:28Z |
+-------------------------+--------------------------------------+
root@controller:~# neutron router-interface-add router sub-selfservice
Added interface d326333b-145a-46ba-8867-b71bb1998848 to router router.
root@controller:~# neutron router-gateway-set router provider
Set gateway for router router
2、在 Controller Node 上执行 ip netns，可以发现多了一条路由：
root@controller:~# ip netns
qrouter-2b3647c7-b403-42cb-a12c-73067f9aa676
qdhcp-3f3b67bb-db40-4e36-9f93-768f45c48257 (id: 1)
qdhcp-f20694bf-1a63-4317-b292-99e46c74718a (id: 0)
问题
OpenStack Self-service networks 架构如下图所示：
network2-connectivity.png
查看 OVS 网桥，发现在 br-int 上少了一个 tap 口：
root@controller:~# ovs-vsctl show
...
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port int-br-provider
Interface int-br-provider
type: patch
options: {peer=phy-br-provider}
Port "qr-d326333b-14"
tag: 1
Interface "qr-d326333b-14"
type: internal
Port "qg-acdea044-7e"
tag: 2
Interface "qg-acdea044-7e"
type: internal
Port br-int
Interface br-int
type: internal
...
这个 tap 口应该是用来连接 DHCP 和 br-int，没有这个口，导致在名字空间内也无法 ping 通网关：
root@controller:~# ip netns exec qdhcp-3f3b67bb-db40-4e36-9f93-768f45c48257 ping 8.8.8.1
PING 8.8.8.1 (8.8.8.1) 56(84) bytes of data.
From 8.8.8.2 icmp_seq=1 Destination Host Unreachable
From 8.8.8.2 icmp_seq=2 Destination Host Unreachable
From 8.8.8.2 icmp_seq=3 Destination Host Unreachable
^C
--- 8.8.8.1 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5029ms
pipe 3
暂未解决。
