Monday, May 23, 2016

Backport upstream commits to stable RDO Mitaka release && Deployments with Keystone API V3

Posting bellow is written  with intend to avoid waiting until "koji" build will appear in updates repo of stable RDO Mitaka release, what might take a couple of months or so. Actually, it doesn't require knowledge how to write properly source RH's rpm file. It just needs picking up raw content of git commits from upstream git repo converting them into patches and rebuild required src.rpm(s) with patch(es) needed. There is also not commonly known command `rpm -qf` which is very useful when you need to detect which rpm has installed particular file. Just to know which src.rpm should be downloaded for git commit referencing
say "cinder.rb"

[root@ServerCentOS72 /]# find . -name cinder.rb -print
find: ‘./run/user/1000/gvfs’: Permission denied
./usr/share/openstack-puppet/modules/cinder/lib/puppet/provider/cinder.rb

[root@ServerCentOS72 /]# rpm -qf /usr/share/openstack-puppet/modules/cinder/lib/puppet/provider/cinder.rb
openstack-puppet-modules-8.0.4-2.el7.centos.noarch

*******************************
Thus download from
*******************************

1. https://cbs.centos.org/koji/buildinfo?buildID=10895
openstack-packstack-8.0.0-1.el7.src.rpm 
2. https://cbs.centos.org/koji/buildinfo?buildID=10859
openstack-puppet-modules-8.0.4-1.el7.src.rpm


[boris@ServerCentOS72 Downloads]$ ls -l
total 3116
-rw-rw-r--. 1 boris boris  170107 May 21 21:26 openstack-packstack-8.0.0-1.el7.src.rpm
-rw-rw-r--. 1 boris boris 3015046 May 21 18:33 openstack-puppet-modules-8.0.4-1.el7.src.rpm

****************
Then run :-
****************

$ rpm -iv openstack-packstack-8.0.0-1.el7.src.rpm
$ rpm -iv  openstack-puppet-modules-8.0.4-1.el7.src.rpm
$ cd ../rpmbuild

In folder ~boris/rpmbuild/SOURCES
create to patch files :-

0001-Use-versionless-auth_url-for-cinder.patch
0001-Enable-keystone-v3-support-for-cinder_type.patch

********************************************************************
In second patch file insert "cinder" in path to *.rb files
********************************************************************

diff --git a/cinder/lib/puppet/provider/cinder_type/openstack.rb b/cinder/lib/puppet/provider/cinder_type/openstack.rb
index feaea49..9aa31c5 100644
--- a/cinder/lib/puppet/provider/cinder_type/openstack.rb
+++ b/cinder/lib/puppet/provider/cinder_type/openstack.rb
@@ -32,6 +32,10 @@ class Puppet::Provider::Cinder < Puppet::Provider::Openstack

. . . . .

diff --git a/cinder/lib/puppet/provider/cinder_type/openstack.rb b/cinder/lib/puppet/provider/cinder_type/openstack.rb
index feaea49..9aa31c5 100644
--- a/cinder/lib/puppet/provider/cinder_type/openstack.rb
+++ b/cinder/lib/puppet/provider/cinder_type/openstack.rb
@@ -7,7 +7,7 @@ Puppet::Type.type(:cinder_type).provide(

. . . . . .

diff --git a/cinder/spec/unit/provider/cinder_spec.rb b/cinder/spec/unit/provider/cinder_spec.rb
index cfc8850..246ae58 100644
--- a/cinder/spec/unit/provider/cinder_spec.rb
+++ b/cinder/spec/unit/provider/cinder_spec.rb
@@ -24,10 +24,12 @@ describe Puppet::Provider::Cinder do


Finally SOURES folder would  look like :-


  **********************
  Next step is :-
  **********************

  $ cd ../SPECS

  and update *.spec files , so that they would understand that patches placed
  into SOURCES folder have to be applied to corresponding *.tar.gz archives
  before building phase itself.

*****************************************
First openstack-packstack.spec :-
*****************************************

Name:           openstack-packstack
Version:        8.0.0
Release:        2%{?milestone}%{?dist} <== increase 1 to 2
Summary:        Openstack Install Utility

Group:          Applications/System
License:        ASL 2.0 and GPLv2
URL:            https://github.com/openstack/packstack
Source0:        http://tarballs.openstack.org/packstack/packstack-%{upstream_version}.tar.gz
Patch0:         0001-Use-versionless-auth_url-for-cinder.patch  <=== Add line 
. . . . . .

%prep
%setup -n packstack-%{upstream_version}
%patch0 -p1  <==  Add line

:wq

*****************************************
Second openstack-puppet-modules.spec
*****************************************
Name:           openstack-puppet-modules
Epoch:          1
Version:        8.0.4
Release:        2%{?milestone}%{?dist}  <===  increase 1 to 2
Summary:        Puppet modules used to deploy OpenStack
License:        ASL 2.0 and GPLv2 and GPLv3

URL:         https://github.com/redhat-openstack

Source0:    https://github.com/redhat-openstack/%{name}/archive/%{upstream_version}.tar.gz
Patch0:    0001-Enable-keystone-v3-support-for-cinder_type.patch <== Add line

. . . . .

%prep
%setup -q -n %{name}-%{?upstream_version}
%patch0 -p1  <== Add line

:wq

******************************************
Attempt rpmbuild for each spec file
******************************************

$ rpmbuild -bb openstack-packstack.spec
$ rpmbuild -bb openstack-puppet-modules.spec

If particular build is missing some packages it will report their's names to screen
This packages could be usually installed via yum, otherwise you have a problem
with local build.
If each build output finishes with message like
Wrote: /home/boris/rpmbuild/RPMS/noarch/openstack-puppet-modules-8.0.4-2.el7.centos.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.wX6p3q
+ umask 022
+ cd /home/boris/rpmbuild/BUILD
+ cd openstack-puppet-modules-8.0.4
+ /usr/bin/rm -rf /home/boris/rpmbuild/BUILDROOT/openstack-puppet-modules-8.0.4-2.el7.centos.x86_64
+ exit 0

Everything is going fine. In particular case results will be written
to ../RPMS/noarch


Then

$ cd ../RPMS/noarch

and create installation script

[boris@ServerCentOS72 SPECS]$ cd ../RPMS/noarch
[boris@ServerCentOS72 noarch]$ ls -l
total 3428
-rwxrwxr-x. 1 boris boris     239 May 21 21:40 install
-rw-rw-r--. 1 boris boris  247312 May 21 21:34 openstack-packstack-8.0.0-2.el7.centos.noarch.rpm
-rw-rw-r--. 1 boris boris   17376 May 21 21:34 openstack-packstack-doc-8.0.0-2.el7.centos.noarch.rpm
-rw-rw-r--. 1 boris boris   16792 May 21 21:34 openstack-packstack-puppet-8.0.0-2.el7.centos.noarch.rpm
-rw-rw-r--. 1 boris boris 3212844 May 21 21:38 openstack-puppet-modules-8.0.4-2.el7.centos.noarch.rpm

[boris@ServerCentOS72 noarch]$ cat install
sudo yum install openstack-packstack-8.0.0-2.el7.centos.noarch.rpm \
openstack-packstack-doc-8.0.0-2.el7.centos.noarch.rpm \
openstack-packstack-puppet-8.0.0-2.el7.centos.noarch.rpm \
openstack-puppet-modules-8.0.4-2.el7.centos.noarch.rpm

****************************
Run install :-
****************************
[boris@ServerCentOS72 noarch]$ ./install

Due to increased  release (1=>2) old rpms should be replaced by just been built

[root@ServerCentOS72 ~]# rpm -qa  \*openstack-packstack\*
openstack-packstack-doc-8.0.0-2.el7.centos.noarch
openstack-packstack-puppet-8.0.0-2.el7.centos.noarch
openstack-packstack-8.0.0-2.el7.centos.noarch

[root@ServerCentOS72 ~]# rpm -qa \*openstack-puppet-modules\*
openstack-puppet-modules-8.0.4-2.el7.centos.noarch

****************************************************************
Since that point following entry in your answer-file :-
****************************************************************

# Identity service API version string. ['v2.0', 'v3']
CONFIG_KEYSTONE_API_VERSION=v3

won't cause cinder puppet to crash packstack run, no matter of kind of your deployment

References
1. https://bugzilla.redhat.com/show_bug.cgi?id=1330289

Friday, May 20, 2016

Java EE Servers as Nova-Docker Containers && RDO Mitaka External vlan networks

  Nova-Docker driver is installed on Compute node which is supposed to run two Java EE Servers as light weight Nova-Docker Containers (instances) having floating IPs on two different external vlan enabled subnets (10.10.10.0/24; 10.10.50.0/24).  General Setup RDO Mitaka ML2&OVS&VLAN 3 Nodes.  VLAN tenant's segregation for RDO lansdcape was selected to avoid DVR configuration Controller && Compute Cluster.
Details here Setup Docker Hypervisor on Multi Node DVR Cluster RDO Mitaka
  Thus Controller/Network RDO Mitaka Node has to have external networks of VLAN type with predefined vlan tags. Straight forward packstack deployment doesn't  allow to achieve desired network configuration. External network provider of vlan type appears to be required.
   Also I have to notice that Docker Hypervisor running on Compute Node
requires all deployment nodes set SELINUX to PERMISSIVE MODE.

   In particular case, office networks 10.10.10.0/24 is vlan tagged (157), 10.10.50.0/24 is vlan tagged (172) already exists when RDO install is running.

192.169.142.127 Controller/Network
192.169.142.147 Storage Node
192.169.142.137 Compute Node
192.169.142.45   WKS ( 3VNICs eth0 - 192.169.142.45 , eth1.157 - 10.10.10.45, eth2.172 - 10.10.50.45 ), running web browser to connect to Java Servers

First

***********************************************************
Update /etc/neutron/plugins/ml2/ml2_conf.ini
***********************************************************

[root@ip-192-169-142-127 ml2(keystone_demo)]# cat ml2_conf.ini
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vlan,vxlan
mechanism_drivers =openvswitch
path_mtu = 0
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges =physnet1:100:200,vlan157:157:157,vlan172:172:172,physnet4
[ml2_type_gre]
[ml2_type_vxlan]
vni_ranges =10:100
vxlan_group =224.0.0.1
[ml2_type_geneve]
[securitygroup]
enable_security_group = True

**************
Then
**************

# openstack-service restart neutron

***************************************************
Invoke external network provider
***************************************************

[root@ip-192-169-142-127 ~(keystone_admin]#neutron net-create vlan157 --shared --provider:network_type vlan --provider:segmentation_id 157 --provider:physical_network vlan157 --router:external

[root@ip-192-169-142-127 ~(keystone_admin]# neutron subnet-create --name sub-vlan157 --gateway 10.10.10.1  --allocation-pool start=10.10.10.100,end=10.10.10.200 vlan157 10.10.10.0/24

***********************************************
Create second external vlan network
***********************************************

[root@ip-192-169-142-127 ~(keystone_admin]# neutron net-create vlan172 --shared --provider:network_type vlan --provider:segmentation_id 172 --provider:physical_network vlan172  --router:external

[root@ip-192-169-142-127 ~(keystone_admin]# neutron subnet-create --name sub-vlan172 --gateway 10.10.50.1 --allocation-pool start=10.10.50.100,end=10.10.50.200 vlan172 10.10.50.0/24

*************************************
Creating third flat network
*************************************

# neutron net-create external4 -- --router:external  \
  --provider:network_type=flat \
  --provider:physical_network=physnet4

# neutron subnet-create --disable-dhcp external4 192.179.143.0/24


[root@ip-192-169-142-127 ~(keystone_admin)]# neutron net-show vlan157
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2016-05-05T11:04:57                  |
| description               |                                      |
| id                        | 98116982-6553-4b7e-920c-0284a7200916 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1500                                 |
| name                      | vlan157                              |
| provider:network_type     | vlan                                 |
| provider:physical_network | vlan157                              |
| provider:segmentation_id  | 157                                  |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | d50fddd2-46e7-4c67-8ab5-97e952bc11c9 |
| tags                      |                                      |
| tenant_id                 | 5d0e67d2fe7a403180fe77232b0cdcd5     |
| updated_at                | 2016-05-05T11:04:57                  |
+---------------------------+--------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron subnet-show sub-vlan157
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| allocation_pools  | {"start": "10.10.10.100", "end": "10.10.10.200"} |
| cidr              | 10.10.10.0/24                                    |
| created_at        | 2016-05-05T11:05:27                              |
| description       |                                                  |
| dns_nameservers   |                                                  |
| enable_dhcp       | True                                             |
| gateway_ip        | 10.10.10.1                                       |
| host_routes       |                                                  |
| id                | d50fddd2-46e7-4c67-8ab5-97e952bc11c9             |
| ip_version        | 4                                                |
| ipv6_address_mode |                                                  |
| ipv6_ra_mode      |                                                  |
| name              | sub-vlan157                                      |
| network_id        | 98116982-6553-4b7e-920c-0284a7200916             |
| subnetpool_id     |                                                  |
| tenant_id         | 5d0e67d2fe7a403180fe77232b0cdcd5                 |
| updated_at        | 2016-05-05T11:05:27                              |
+-------------------+--------------------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron net-show vlan172
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2016-05-05T11:05:46                  |
| description               |                                      |
| id                        | 6fc89f42-cb85-4cdb-be5f-4d1768b05e14 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1500                                 |
| name                      | vlan172                              |
| provider:network_type     | vlan                                 |
| provider:physical_network | vlan172                              |
| provider:segmentation_id  | 172                                  |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | 2d4f907b-1f80-451a-a663-b87bbd683d58 |
| tags                      |                                      |
| tenant_id                 | 5d0e67d2fe7a403180fe77232b0cdcd5     |
| updated_at                | 2016-05-05T11:05:46                  |
+---------------------------+--------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron subnet-show sub-vlan172
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| allocation_pools  | {"start": "10.10.50.100", "end": "10.10.50.200"} |
| cidr              | 10.10.50.0/24                                    |
| created_at        | 2016-05-05T11:10:18                              |
| description       |                                                  |
| dns_nameservers   |                                                  |
| enable_dhcp       | True                                             |
| gateway_ip        | 10.10.50.1                                       |
| host_routes       |                                                  |
| id                | 2d4f907b-1f80-451a-a663-b87bbd683d58             |
| ip_version        | 4                                                |
| ipv6_address_mode |                                                  |
| ipv6_ra_mode      |                                                  |
| name              | sub-vlan172                                      |
| network_id        | 6fc89f42-cb85-4cdb-be5f-4d1768b05e14             |
| subnetpool_id     |                                                  |
| tenant_id         | 5d0e67d2fe7a403180fe77232b0cdcd5                 |
| updated_at        | 2016-05-05T11:10:18                              |
+-------------------+--------------------------------------------------+

Enable EPEL 7 to install vconfig ( or use ip CLI )

***************************************************
Next Step - external bridges setup on Controller
***************************************************

# modprobe 8021q
# ovs-vsctl add-br br-vlan
# ovs-vsctl add-port br-vlan eth2
# vconfig add br-vlan 157

# ovs-vsctl add-br br-vlan2
# ovs-vsctl add-port br-vlan2 eth3
# vconfig add br-vlan2 172

**********************************
External flat network
**********************************

# ovs-vsctl add-br br-eth4
# ovs-vsctl add-port  br-eth4  eth4


****************************************************
Update Neutron OVS configuration files
****************************************************
Start with the following comment in l3_agent.ini:
# When external_network_bridge is set, each L3 agent can be associated
# with no more than one external network. This value should be set to the UUID
# of that external network. To allow L3 agent support multiple external
# networks, both the external_network_bridge and gateway_external_network_id
# must be left empty. 
******************************
Update l3_agent.ini file
******************************
external_network_bridge =
gateway_external_network_id =

******************************************************
/etc/neutron/plugins/ml2/openvswitch_agent.ini
******************************************************
bridge_mappings = physnet1:br-eth1,vlan157:br-vlan,vlan172:br-vlan2,physnet4:br-eth4

*************************************
Update Neutron Configuration
*************************************

# openstack-service restart neutron

************************************************
Set up config persistent between reboots
************************************************

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-eth2
DEVICE="eth2"
ONBOOT=yes
OVS_BRIDGE=br-vlan
TYPE=OVSPort
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-eth3
DEVICE="eth3"
ONBOOT=yes
OVS_BRIDGE=br-vlan2
TYPE=OVSPort
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan
DEVICE=br-vlan
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan2
DEVICE=br-vlan2
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan.157
BOOTPROTO="none"
DEVICE="br-vlan.157"
ONBOOT="yes"
IPADDR="10.10.10.150"
PREFIX="24"
GATEWAY="10.10.10.1"
DNS1="83.221.202.254"
VLAN=yes
NOZEROCONF=yes
USERCTL=no

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan2.172
BOOTPROTO="none"
DEVICE="br-vlan2.172"
ONBOOT="yes"
IPADDR="10.10.50.150"
PREFIX="24"
GATEWAY="10.10.50.1"
DNS1="83.221.202.254"
VLAN=yes
NOZEROCONF=yes

***************************
Then run script
***************************
#!/bin/bash -x
chkconfig network on
systemctl stop NetworkManager
systemctl disable NetworkManager
service network restart

# openstack-service restart neutron

[root@ip-192-169-142-127 ~(keystone_admin)]#  ip -details link show dev br-vlan.157
34: br-vlan.157@br-vlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether ba:bb:33:85:aa:49 brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 157 <REORDER_HDR> addrgenmode eui64

[root@ip-192-169-142-127 ~(keystone_admin)]#  ip -details link show dev br-vlan2.172
33: br-vlan2.172@br-vlan2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 0e:e7:2d:26:c3:4a brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 172 <REORDER_HDR> addrgenmode eui64

[root@ip-192-169-142-127 ~(keystone_admin)]# ip route
default via 10.10.10.1 dev br-vlan.157 
10.10.10.0/24 dev br-vlan.157  proto kernel  scope link  src 10.10.10.150 
10.10.50.0/24 dev br-vlan2.172  proto kernel  scope link  src 10.10.50.150 
169.254.0.0/16 dev eth0  scope link  metric 1002 
169.254.0.0/16 dev eth1  scope link  metric 1003 
169.254.0.0/16 dev eth2  scope link  metric 1004 
169.254.0.0/16 dev eth3  scope link  metric 1005 
169.254.0.0/16 dev eth4  scope link  metric 1006 
169.254.0.0/16 dev br-vlan  scope link  metric 1009 
169.254.0.0/16 dev br-ex  scope link  metric 1010 
169.254.0.0/16 dev br-vlan2  scope link  metric 1012 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
192.169.142.0/24 dev br-ex  proto kernel  scope link  src 192.169.142.127 

***************************
Controller/Network
***************************
[root@ip-192-169-142-127 ~(keystone_admin)]# ovs-vsctl show
d12e6a7a-f589-42cd-91b3-96156ad9ed59
    Bridge br-int
        fail_mode: secure
        Port "tap114497da-9f"
            tag: 1
            Interface "tap114497da-9f"
                type: internal
        Port "tap390b9bc5-b9"
            tag: 5
            Interface "tap390b9bc5-b9"
                type: internal
        Port "int-br-vlan2"
            Interface "int-br-vlan2"
                type: patch
                options: {peer="phy-br-vlan2"}
        Port "tap4b0c52b8-90"
            tag: 3
            Interface "tap4b0c52b8-90"
                type: internal
        Port "qr-795380c4-1d"
            tag: 2
            Interface "qr-795380c4-1d"
                type: internal
        Port "qg-b856b873-d0"
            tag: 3
            Interface "qg-b856b873-d0"
                type: internal
        Port "tap119f2a5f-2e"
            tag: 4
            Interface "tap119f2a5f-2e"
                type: internal
        Port int-br-vlan
            Interface int-br-vlan
                type: patch
                options: {peer=phy-br-vlan}
        Port "qr-2e2d2e81-14"
            tag: 1
            Interface "qr-2e2d2e81-14"
                type: internal
        Port "qg-234c2d9c-0a"
            tag: 4
            Interface "qg-234c2d9c-0a"
                type: internal
        Port "qr-41a1a0fa-ec"
            tag: 5
            Interface "qr-41a1a0fa-ec"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "qg-5c7b2d0a-1b"
            tag: 6
            Interface "qg-5c7b2d0a-1b"
                type: internal
        Port "int-br-eth4"
            Interface "int-br-eth4"
                type: patch
                options: {peer="phy-br-eth4"}
        Port "int-br-eth1"
            Interface "int-br-eth1"
                type: patch
                options: {peer="phy-br-eth1"}
        Port "tap936bd80d-26"
            tag: 2
            Interface "tap936bd80d-26"
                type: internal
    Bridge "br-vlan2"   <===external vlan172 network
        Port "eth3"
            Interface "eth3"
        Port "phy-br-vlan2"
            Interface "phy-br-vlan2"
                type: patch
                options: {peer="int-br-vlan2"}
        Port "br-vlan2"
            Interface "br-vlan2"
                type: internal
     Bridge "br-eth4"    <===external flat network
        Port "eth4"
            Interface "eth4"
        Port "phy-br-eth4"
            Interface "phy-br-eth4"
                type: patch
                options: {peer="int-br-eth4"}
        Port "br-eth4"
            Interface "br-eth4"
                type: internal

    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "eth0"
            Interface "eth0"
    Bridge "br-eth1"        <=== internal vlan enabled vm/data network
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
                type: patch
                options: {peer="int-br-eth1"}
        Port "eth1"
            Interface "eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
    Bridge br-vlan    <===external vlan157 network
        Port br-vlan
            Interface br-vlan
                type: internal
        Port "eth2"
            Interface "eth2"
        Port phy-br-vlan
            Interface phy-br-vlan
                type: patch
                options: {peer=int-br-vlan}
    ovs_version: "2.4.0"
[root@ip-192-169-142-127 ~(keystone_admin)]# ifconfig
br-ex: flags=4163  mtu 1500
        inet 192.169.142.127  netmask 255.255.255.0  broadcast 192.169.142.255
        inet6 fe80::3058:e2ff:fe5c:f64c  prefixlen 64  scopeid 0x20
        ether 32:58:e2:5c:f6:4c  txqueuelen 0  (Ethernet)
        RX packets 54754  bytes 12112793 (11.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48647  bytes 15788730 (15.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan: flags=4163  mtu 1500
        inet6 fe80::2054:9fff:feb5:b645  prefixlen 64  scopeid 0x20
        ether 22:54:9f:b5:b6:45  txqueuelen 0  (Ethernet)
        RX packets 51  bytes 6647 (6.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 298  bytes 16763 (16.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan2: flags=4163  mtu 1500
        inet6 fe80::149a:c9ff:fe29:2240  prefixlen 64  scopeid 0x20
        ether 16:9a:c9:29:22:40  txqueuelen 0  (Ethernet)
        RX packets 497  bytes 24405 (23.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 31  bytes 4481 (4.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan.157: flags=4163  mtu 1500
        inet 10.10.10.150  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::2054:9fff:feb5:b645  prefixlen 64  scopeid 0x20
        ether 22:54:9f:b5:b6:45  txqueuelen 0  (Ethernet)
        RX packets 34  bytes 4667 (4.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 290  bytes 14955 (14.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan2.172: flags=4163  mtu 1500
        inet 10.10.50.150  netmask 255.255.255.0  broadcast 10.10.50.255
        inet6 fe80::149a:c9ff:fe29:2240  prefixlen 64  scopeid 0x20
        ether 16:9a:c9:29:22:40  txqueuelen 0  (Ethernet)
        RX packets 480  bytes 16181 (15.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 3741 (3.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe82:e40e  prefixlen 64  scopeid 0x20
        ether 52:54:00:82:e4:0e  txqueuelen 1000  (Ethernet)
        RX packets 56233  bytes 12189701 (11.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48639  bytes 15786905 (15.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163  mtu 1500
        inet6 fe80::5054:ff:feb2:82c  prefixlen 64  scopeid 0x20
        ether 52:54:00:b2:08:2c  txqueuelen 1000  (Ethernet)
        RX packets 4327  bytes 2918115 (2.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1882  bytes 6415333 (6.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fec9:cdcf  prefixlen 64  scopeid 0x20
        ether 52:54:00:c9:cd:cf  txqueuelen 1000  (Ethernet)
        RX packets 3334  bytes 6491253 (6.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3064  bytes 2816873 (2.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth3: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe48:266f  prefixlen 64  scopeid 0x20
        ether 52:54:00:48:26:6f  txqueuelen 1000  (Ethernet)
        RX packets 2027  bytes 108290 (105.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 88  bytes 43478 (42.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth4: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe92:f712  prefixlen 64  scopeid 0x20
        ether 52:54:00:92:f7:12  txqueuelen 1000  (Ethernet)
        RX packets 1489  bytes 77816 (75.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9  bytes 718 (718.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 422071  bytes 134700542 (128.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 422071  bytes 134700542 (128.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:e9:36:be  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0





  
******************************************************************************************
Verification VLAN connectivity from WKS (VM) having eth1.157 && eth2.172 to mentioned bellow Java Servers running on Compute Node 192.169.142.137 as Nova-Docker Containers (instances)  with floating ips on vlan157 and  vlan172 correspondently
******************************************************************************************
Network configuration on WKS

[root@VlanConnect ~]# ifconfig
eth0: flags=4163  mtu 1500
        inet 192.169.142.45  netmask 255.255.255.0  broadcast 192.169.142.255
        inet6 fe80::5054:ff:fef5:a7eb  prefixlen 64  scopeid 0x20
        ether 52:54:00:f5:a7:eb  txqueuelen 1000  (Ethernet)
        RX packets 3358  bytes 178671 (174.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22  bytes 3676 (3.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe2e:716c  prefixlen 64  scopeid 0x20
        ether 52:54:00:2e:71:6c  txqueuelen 1000  (Ethernet)
        RX packets 5154  bytes 1994070 (1.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1157  bytes 153055 (149.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe62:5abb  prefixlen 64  scopeid 0x20
        ether 52:54:00:62:5a:bb  txqueuelen 1000  (Ethernet)
        RX packets 3384  bytes 213816 (208.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 654  bytes 36212 (35.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1.157: flags=4163  mtu 1500
        inet 10.10.10.45  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::5054:ff:fe2e:716c  prefixlen 64  scopeid 0x20
        ether 52:54:00:2e:71:6c  txqueuelen 0  (Ethernet)
        RX packets 1823  bytes 1786566 (1.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1151  bytes 147974 (144.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


eth2.172: flags=4163  mtu 1500
        inet 10.10.50.45  netmask 255.255.255.0  broadcast 10.10.50.255
        inet6 fe80::5054:ff:fe62:5abb  prefixlen 64  scopeid 0x20
        ether 52:54:00:62:5a:bb  txqueuelen 0  (Ethernet)
        RX packets 52  bytes 38138 (37.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 647  bytes 33022 (32.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 590  bytes 62280 (60.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 590  bytes 62280 (60.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:13:11:69  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



[root@VlanConnect ~]# ip -details link show dev eth1.157
5: eth1.157@eth1: mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:2e:71:6c brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 157 addrgenmode eui64


[root@VlanConnect ~]# ip -details link show dev eth2.172
6: eth2.172@eth2: mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:62:5a:bb brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 172 addrgenmode eui64

********************************************************************************
Attempt to launch browser to GlassFish 4.X and Tomcat 7.0 consoles
********************************************************************************

Nova-Docker instance derbyGlassfish01 on vlan157 (10.10.10.105:4848)
Nova-Docker instance  Tomcat  on vlan172  (10.10.50.105:8080)




Friday, May 06, 2016

RDO Mitaka && several external networks VLAN provider setup

Post bellow is addressing the question when Controller/Network RDO Mitaka Node has to have external networks of VLAN type with predefined vlan tags. Straight forward packstack deployment doesn't  allow to achieve desired network configuration. External network provider of vlan type appears to be required. In particular case, office networks 10.10.10.0/24 vlan tagged (157) ,10.10.50.0/24 vlan tagged (172) already exists when RDO install is running. If demo_provision was "y" , then delete router1 and created external network of VXLAN type

First
***********************************************************
Update /etc/neutron/plugins/ml2/ml2_conf.ini
***********************************************************
[root@ip-192-169-142-127 ml2(keystone_demo)]# cat ml2_conf.ini
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vlan,vxlan
mechanism_drivers =openvswitch
path_mtu = 0
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = vlan157:157:157,vlan172:172:172
[ml2_type_gre]
[ml2_type_vxlan]
vni_ranges =10:100
vxlan_group =224.0.0.1
[ml2_type_geneve]
[securitygroup]
enable_security_group = True

**************
Then
**************

# openstack-service restart neutron

***************************************************
Invoke external network provider
***************************************************

[root@ip-192-169-142-127 ~(keystone_admin]#neutron net-create vlan157 --shared --provider:network_type vlan --provider:segmentation_id 157 --provider:physical_network vlan157 --router:external

[root@ip-192-169-142-127 ~(keystone_admin]# neutron subnet-create --name sub-vlan157 --gateway 10.10.10.1  --allocation-pool start=10.10.10.100,end=10.10.10.200 vlan157 10.10.10.0/24

***********************************************
Create second external network
***********************************************

[root@ip-192-169-142-127 ~(keystone_admin]# neutron net-create vlan172 --shared --provider:network_type vlan --provider:segmentation_id 172 --provider:physical_network vlan172  --router:external

[root@ip-192-169-142-127 ~(keystone_admin]# neutron subnet-create --name sub-vlan172 --gateway 10.10.50.1 --allocation-pool start=10.10.50.100,end=10.10.50.200 vlan172 10.10.50.0/24


***********************************************************
No need to update sub-net (vs [ 1 ]). No switch to "enable_isolataed_metadata=True"
Neutron L3 agent configuration results attaching qg- interfaces to br-int
***********************************************************

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron net-show vlan157
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2016-05-05T11:04:57                  |
| description               |                                      |
| id                        | 98116982-6553-4b7e-920c-0284a7200916 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1500                                 |
| name                      | vlan157                              |
| provider:network_type     | vlan                                 |
| provider:physical_network | vlan157                              |
| provider:segmentation_id  | 157                                  |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | d50fddd2-46e7-4c67-8ab5-97e952bc11c9 |
| tags                      |                                      |
| tenant_id                 | 5d0e67d2fe7a403180fe77232b0cdcd5     |
| updated_at                | 2016-05-05T11:04:57                  |
+---------------------------+--------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron subnet-show sub-vlan157
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| allocation_pools  | {"start": "10.10.10.100", "end": "10.10.10.200"} |
| cidr              | 10.10.10.0/24                                    |
| created_at        | 2016-05-05T11:05:27                              |
| description       |                                                  |
| dns_nameservers   |                                                  |
| enable_dhcp       | True                                             |
| gateway_ip        | 10.10.10.1                                       |
| host_routes       |                                                  |
| id                | d50fddd2-46e7-4c67-8ab5-97e952bc11c9             |
| ip_version        | 4                                                |
| ipv6_address_mode |                                                  |
| ipv6_ra_mode      |                                                  |
| name              | sub-vlan157                                      |
| network_id        | 98116982-6553-4b7e-920c-0284a7200916             |
| subnetpool_id     |                                                  |
| tenant_id         | 5d0e67d2fe7a403180fe77232b0cdcd5                 |
| updated_at        | 2016-05-05T11:05:27                              |
+-------------------+--------------------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron net-show vlan172
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2016-05-05T11:05:46                  |
| description               |                                      |
| id                        | 6fc89f42-cb85-4cdb-be5f-4d1768b05e14 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1500                                 |
| name                      | vlan172                              |
| provider:network_type     | vlan                                 |
| provider:physical_network | vlan172                              |
| provider:segmentation_id  | 172                                  |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | 2d4f907b-1f80-451a-a663-b87bbd683d58 |
| tags                      |                                      |
| tenant_id                 | 5d0e67d2fe7a403180fe77232b0cdcd5     |
| updated_at                | 2016-05-05T11:05:46                  |
+---------------------------+--------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron subnet-show sub-vlan172
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| allocation_pools  | {"start": "10.10.50.100", "end": "10.10.50.200"} |
| cidr              | 10.10.50.0/24                                    |
| created_at        | 2016-05-05T11:10:18                              |
| description       |                                                  |
| dns_nameservers   |                                                  |
| enable_dhcp       | True                                             |
| gateway_ip        | 10.10.50.1                                       |
| host_routes       |                                                  |
| id                | 2d4f907b-1f80-451a-a663-b87bbd683d58             |
| ip_version        | 4                                                |
| ipv6_address_mode |                                                  |
| ipv6_ra_mode      |                                                  |
| name              | sub-vlan172                                      |
| network_id        | 6fc89f42-cb85-4cdb-be5f-4d1768b05e14             |
| subnetpool_id     |                                                  |
| tenant_id         | 5d0e67d2fe7a403180fe77232b0cdcd5                 |
| updated_at        | 2016-05-05T11:10:18                              |
+-------------------+--------------------------------------------------+

Enable EPEL 7 to install vconfig ( or use ip CLI )

**************
Next Step
**************

# modprobe 8021q
# ovs-vsctl add-br br-vlan
# ovs-vsctl add-port br-vlan eth2
# vconfig add br-vlan 157

# ovs-vsctl add-br br-vlan2
# ovs-vsctl add-port br-vlan2 eth3
# vconfig add br-vlan2 172

****************************************************
Update Neutron OVS configuration files
****************************************************
Start with the following comment in l3_agent.ini:
# When external_network_bridge is set, each L3 agent can be associated
# with no more than one external network. This value should be set to the UUID
# of that external network. To allow L3 agent support multiple external
# networks, both the external_network_bridge and gateway_external_network_id
# must be left empty. 
******************************
Update l3_agent.ini file
******************************
external_network_bridge =
gateway_external_network_id =

**********************************************
/etc/neutron/plugins/ml2/openvswitch_agent.ini
**********************************************
bridge_mappings = vlan157:br-vlan,vlan172:br-vlan2

*************************************
Update Neutron Configuration
*************************************

# openstack-service restart neutron

*******************************************
Set up config persistent between reboots
*******************************************

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-eth2
DEVICE="eth2"
ONBOOT=yes
OVS_BRIDGE=br-vlan
TYPE=OVSPort
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-eth3
DEVICE="eth3"
ONBOOT=yes
OVS_BRIDGE=br-vlan2
TYPE=OVSPort
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan
DEVICE=br-vlan
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan2
DEVICE=br-vlan2
BOOTPROTO=none
ONBOOT=yes
TYPE=OVSBridge
DEVICETYPE="ovs"

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan.157
BOOTPROTO="none"
DEVICE="br-vlan.157"
ONBOOT="yes"
IPADDR="10.10.10.150"
PREFIX="24"
GATEWAY="10.10.10.1"
DNS1="83.221.202.254"
VLAN=yes
NOZEROCONF=yes
USERCTL=no

[root@ip-192-169-142-127 network-scripts(keystone_admin)]# cat ifcfg-br-vlan2.172
BOOTPROTO="none"
DEVICE="br-vlan2.172"
ONBOOT="yes"
IPADDR="10.10.50.150"
PREFIX="24"
GATEWAY="10.10.50.1"
DNS1="83.221.202.254"
VLAN=yes
NOZEROCONF=yes

***************************
Then run script
***************************
#!/bin/bash -x
chkconfig network on
systemctl stop NetworkManager
systemctl disable NetworkManager
service network restart

# openstack-service restart neutron

[root@ip-192-169-142-127 ~(keystone_admin)]#  ip -details link show dev br-vlan.157
34: br-vlan.157@br-vlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether ba:bb:33:85:aa:49 brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 157 <REORDER_HDR> addrgenmode eui64

[root@ip-192-169-142-127 ~(keystone_admin)]#  ip -details link show dev br-vlan2.172
33: br-vlan2.172@br-vlan2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 0e:e7:2d:26:c3:4a brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 172 <REORDER_HDR> addrgenmode eui64


**************************************************
Routing table on Controller/Network node
**************************************************

[root@ip-192-169-142-127 ~]# ip route
default via 10.10.10.1 dev br-vlan.157
10.10.10.0/24 dev br-vlan.157  proto kernel  scope link  src 10.10.10.150
10.10.50.0/24 dev br-vlan2.172  proto kernel  scope link  src 10.10.50.150

12.0.0.0/24 dev eth1  proto kernel  scope link  src 12.0.0.127
169.254.0.0/16 dev eth0  scope link  metric 1002
169.254.0.0/16 dev eth1  scope link  metric 1003
169.254.0.0/16 dev eth2  scope link  metric 1004
169.254.0.0/16 dev eth3  scope link  metric 1005
169.254.0.0/16 dev br-vlan  scope link  metric 1007
169.254.0.0/16 dev br-ex  scope link  metric 1009
169.254.0.0/16 dev br-vlan2  scope link  metric 1011
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1
192.169.142.0/24 dev br-ex  proto kernel  scope link  src 192.169.142.127


**********************************************************
ovs-vsctl show on Controller/Network node
**********************************************************
[root@ip-192-169-142-127 ~]# ovs-vsctl show
0f97d38c-b77e-453f-a1fd-833cda206a7c
    Bridge br-tun
        fail_mode: secure
        Port "vxlan-0c000089"
            Interface "vxlan-0c000089"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="12.0.0.127", out_key=flow, remote_ip="12.0.0.137"}
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge "br-vlan2"
        Port "phy-br-vlan2"
            Interface "phy-br-vlan2"
                type: patch
                options: {peer="int-br-vlan2"}
        Port "br-vlan2"
            Interface "br-vlan2"
                type: internal
        Port "eth3"
            Interface "eth3"

    Bridge br-ex
        Port "eth0"
            Interface "eth0"
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
    Bridge br-int
        fail_mode: secure
        Port "tap63390ae5-a0"
            tag: 5
            Interface "tap63390ae5-a0"
                type: internal
        Port "tapbd133fe7-d8"
            tag: 1
            Interface "tapbd133fe7-d8"
                type: internal
        Port "qr-c0d289d4-57"
            tag: 1
            Interface "qr-c0d289d4-57"
                type: internal
        Port "tapf72340ff-a7"
            tag: 4
            Interface "tapf72340ff-a7"
                type: internal
        Port "qr-f30f8169-82"
            tag: 6
            Interface "qr-f30f8169-82"
                type: internal
        Port int-br-vlan
            Interface int-br-vlan
                type: patch
                options: {peer=phy-br-vlan}
        Port "tapcc4e78d3-89"
            tag: 3
            Interface "tapcc4e78d3-89"
                type: internal
        Port "qg-5b3034ad-ec"
            tag: 2
            Interface "qg-5b3034ad-ec"
                type: internal
        Port "tapcf29aa08-22"
            tag: 6
            Interface "tapcf29aa08-22"
                type: internal
        Port "qg-52fdf7e8-ab"
            tag: 4
            Interface "qg-52fdf7e8-ab"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "int-br-vlan2"
            Interface "int-br-vlan2"
                type: patch
                options: {peer="phy-br-vlan2"}
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port "tap4b314afd-a5"
            tag: 2
            Interface "tap4b314afd-a5"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "qr-9174bee9-2c"
            tag: 5
            Interface "qr-9174bee9-2c"
                type: internal
        Port "qg-d23a9d5f-04"
            tag: 3
            Interface "qg-d23a9d5f-04"
                type: internal
    Bridge br-vlan
        Port br-vlan
            Interface br-vlan
                type: internal
        Port phy-br-vlan
            Interface phy-br-vlan
                type: patch
                options: {peer=int-br-vlan}
        Port "eth2"
            Interface "eth2"

    ovs_version: "2.4.0"

[root@ip-192-169-142-127 ~]# ifconfig
br-ex: flags=4163  mtu 1500
        inet 192.169.142.127  netmask 255.255.255.0  broadcast 192.169.142.255
        inet6 fe80::6cc4:13ff:fe55:5d4a  prefixlen 64  scopeid 0x20
        ether 6e:c4:13:55:5d:4a  txqueuelen 0  (Ethernet)
        RX packets 11515  bytes 2649088 (2.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10563  bytes 5443828 (5.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan: flags=4163  mtu 1500
        inet6 fe80::fc83:fff:febb:9a4d  prefixlen 64  scopeid 0x20
        ether fe:83:0f:bb:9a:4d  txqueuelen 0  (Ethernet)
        RX packets 11  bytes 710 (710.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 376  bytes 20351 (19.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan2: flags=4163  mtu 1500
        inet6 fe80::7c92:f0ff:fe4d:2745  prefixlen 64  scopeid 0x20
        ether 7e:92:f0:4d:27:45  txqueuelen 0  (Ethernet)
        RX packets 77  bytes 3618 (3.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 31  bytes 4577 (4.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan.157: flags=4163  mtu 1500
        inet 10.10.10.150  netmask 255.255.255.0  broadcast 10.10.10.255
        inet6 fe80::fc83:fff:febb:9a4d  prefixlen 64  scopeid 0x20
        ether fe:83:0f:bb:9a:4d  txqueuelen 0  (Ethernet)
        RX packets 11  bytes 556 (556.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 368  bytes 18231 (17.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-vlan2.172: flags=4163  mtu 1500
        inet 10.10.50.150  netmask 255.255.255.0  broadcast 10.10.50.255
        inet6 fe80::7c92:f0ff:fe4d:2745  prefixlen 64  scopeid 0x20
        ether 7e:92:f0:4d:27:45  txqueuelen 0  (Ethernet)
        RX packets 70  bytes 2072 (2.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 3837 (3.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163  mtu 1500
        inet6 fe80::5054:ff:feaa:20a7  prefixlen 64  scopeid 0x20
        ether 52:54:00:aa:20:a7  txqueuelen 1000  (Ethernet)
        RX packets 12224  bytes 2686948 (2.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10573  bytes 5443701 (5.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163  mtu 1500
        inet 12.0.0.127  netmask 255.255.255.0  broadcast 12.0.0.255
        inet6 fe80::5054:ff:fe5e:7704  prefixlen 64  scopeid 0x20
        ether 52:54:00:5e:77:04  txqueuelen 1000  (Ethernet)
        RX packets 1184  bytes 107197 (104.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 411  bytes 68703 (67.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fea3:8d2b  prefixlen 64  scopeid 0x20
        ether 52:54:00:a3:8d:2b  txqueuelen 1000  (Ethernet)
        RX packets 717  bytes 37478 (36.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 388  bytes 21199 (20.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth3: flags=4163  mtu 1500
        inet6 fe80::5054:ff:feb2:6635  prefixlen 64  scopeid 0x20
        ether 52:54:00:b2:66:35  txqueuelen 1000  (Ethernet)
        RX packets 797  bytes 47773 (46.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 147  bytes 15510 (15.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 262073  bytes 56439278 (53.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 262073  bytes 56439278 (53.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:7f:d6:bb  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

**********************************************************************************
Now create another VM supposed to work as "remote" client with network configuration
**********************************************************************************
[root@ip-192-169-142-25 ~]# ifconfig
eth0: flags=4163  mtu 1500
        inet 192.169.142.25  netmask 255.255.255.0  broadcast 192.169.142.255
        inet6 fe80::5054:ff:fe53:8d61  prefixlen 64  scopeid 0x20
        ether 52:54:00:53:8d:61  txqueuelen 1000  (Ethernet)
        RX packets 2306  bytes 121390 (118.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22  bytes 3667 (3.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe51:c77b  prefixlen 64  scopeid 0x20
        ether 52:54:00:51:c7:7b  txqueuelen 1000  (Ethernet)
        RX packets 2356  bytes 123722 (120.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 30  bytes 4389 (4.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4163  mtu 1500
        inet6 fe80::5054:ff:fe26:ab7  prefixlen 64  scopeid 0x20
        ether 52:54:00:26:0a:b7  txqueuelen 1000  (Ethernet)
        RX packets 2329  bytes 122504 (119.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 30  bytes 4389 (4.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1.157: flags=4163  mtu 1500
        inet 10.10.10.25  netmask 255.255.255.0  broadcast 10.10.10.255

        inet6 fe80::5054:ff:fe51:c77b  prefixlen 64  scopeid 0x20
        ether 52:54:00:51:c7:7b  txqueuelen 0  (Ethernet)
        RX packets 44  bytes 1288 (1.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 3719 (3.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2.172: flags=4163  mtu 1500
        inet 10.10.50.25  netmask 255.255.255.0  broadcast 10.10.50.255

        inet6 fe80::5054:ff:fe26:ab7  prefixlen 64  scopeid 0x20
        ether 52:54:00:26:0a:b7  txqueuelen 0  (Ethernet)
        RX packets 23  bytes 700 (700.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 3719 (3.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:d2:be:fe  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



[root@ip-192-169-142-25 ~]# ip -details link show dev eth1.157
5: eth1.157@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:51:c7:7b brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 157 <REORDER_HDR> addrgenmode eui64 

[root@ip-192-169-142-25 ~]# ip -details link show dev eth2.172
6: eth2.172@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:26:0a:b7 brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 172 <REORDER_HDR> addrgenmode eui64 

**********************************
 Network Topology
**********************************



***********************************************************************************
Verification VLAN connectivity from another VM having eth1.157 && eth2.172
***********************************************************************************
   Connection via ssh from different VM (Host) via vlan172 external network
   to VF23Devs01


   Status on Controller/Network Node

   # ip -details link show dev br-vlan.157
   # ip -details link show dev br-vlan.172

   has been run against vlan enabled OVS bridges providing connections
   to external vlan tagged networks vlan157 && vlan172


 

*********************************************************************
"Remote"  connections from VM 192.169.142.25
*********************************************************************
[root@ip-192-169-142-25 ~]# ip -d link show dev eth1.157
5: eth1.157@eth1: mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:51:c7:7b brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 157 addrgenmode eui64 


[root@ip-192-169-142-25 ~]# ip -d link show dev eth2.172
6: eth2.172@eth2: mtu 1500 qdisc noqueue state UP mode DEFAULT
    link/ether 52:54:00:26:0a:b7 brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1Q id 172 addrgenmode eui64 


[root@ip-192-169-142-25 ~]# ssh -i oskeymitakav0509.pem ubuntu@10.10.10.105
Welcome to Ubuntu 16.04 LTS (GNU/Linux 4.4.0-21-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.


Last login: Sun May  8 21:01:05 2016 from 10.10.10.25
ubuntu@ubuntudevs01:~$ uname -a
Linux ubuntudevs01 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ubuntudevs01:~$ logout
Connection to 10.10.10.105 closed.

[root@ip-192-169-142-25 ~]# ssh -i oskeymitakav0509.pem fedora@10.10.50.115
Last login: Sun May  8 21:00:10 2016 from 10.10.50.25
[fedora@vf23devs01 ~]$ uname -a
Linux vf23devs01.novalocal 4.2.3-300.fc23.x86_64 #1 SMP Mon Oct 5 15:42:54 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[fedora@vf23devs01 ~]$ Connection to 10.10.50.115 closed.

 *******************************************************************************
Running ovs-vsctl dump-flows run against br-int,br-vlan,br-vlan2
********************************************************************************
   
 [root@ip-192-169-142-127 ~(keystone_admin)]# ovs-ofctl dump-flows br-int | grep NORMAL
 cookie=0xbc49cba79c654568, duration=3728.585s, table=0, n_packets=304, n_bytes=22247, idle_age=1035, priority=3,in_port=9,dl_vlan=157 actions=mod_vlan_vid:2,NORMAL
 cookie=0xbc49cba79c654568, duration=3728.568s, table=0, n_packets=847, n_bytes=63545, idle_age=1263, priority=3,in_port=10,dl_vlan=172 actions=mod_vlan_vid:3,NORMAL

 cookie=0xbc49cba79c654568, duration=3785.491s, table=0, n_packets=17023, n_bytes=10656347, idle_age=0, priority=0 actions=NORMAL

[root@ip-192-169-142-127 ~(keystone_admin)]# ovs-ofctl dump-flows br-vlan | grep NORMAL
 cookie=0x8bf063a20f266e3d, duration=3740.851s, table=0, n_packets=49, n_bytes=5876, idle_age=2477, priority=4,in_port=2,dl_vlan=2 actions=mod_vlan_vid:157,NORMAL
 cookie=0x8bf063a20f266e3d, duration=3797.645s, table=0, n_packets=2256, n_bytes=124703, idle_age=0, priority=0 actions=NORMAL

[root@ip-192-169-142-127 ~(keystone_admin)]# ovs-ofctl dump-flows br-vlan2 | grep NORMAL
 cookie=0x95be135a31e3e4d1, duration=3751.317s, table=0, n_packets=1683, n_bytes=130830, idle_age=0, priority=4,in_port=2,dl_vlan=3 actions=mod_vlan_vid:172,NORMAL
 cookie=0x95be135a31e3e4d1, duration=3808.061s, table=0, n_packets=2763, n_bytes=163765, idle_age=1, priority=0 actions=NORMAL

****************************************************
Testing transmitting bridges
****************************************************

[root@ip-192-169-142-127 ~(keystone_admin)]# cat /proc/net/vlan/br-vlan.157
br-vlan.157  VID: 157     REORDER_HDR: 1  dev->priv_flags: 1
         total frames received           42
          total bytes received         4080
      Broadcast/Multicast Rcvd           21

      total frames transmitted          287
       total bytes transmitted        14829
Device: br-vlan
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings:

[root@ip-192-169-142-127 ~(keystone_admin)]# cat /proc/net/vlan/br-vlan2.172
br-vlan2.172  VID: 172     REORDER_HDR: 1  dev->priv_flags: 1
         total frames received         1978
          total bytes received        58288
      Broadcast/Multicast Rcvd           21

      total frames transmitted           23
       total bytes transmitted         3741
Device: br-vlan2
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings:

***************************************************************************************
Snapshots bellow done on VF23Devs01 VM attached to flat and vlan enabled 172
external networks. "MATE Desktop" installed via FIP belongs to flat network
( outbound connectivity to Internet ) . Aftewards VM switched to FIP belongs
vlan172 connected 10.10.150.45 having eth2.172.
***************************************************************************************



   10.10.50.45