openstack/networking-bgpvpn - networking-bgpvpn - OpenDev: Free Software Needs Free Tools

Change-Id: Ic2cfad5b41b96c31586fcc9158b30e5a4c2da195
Depends-On: I09338f8511e657eab5de3cc361b70f70b1490eaa

Sourcing the neutron devstack l2 agent script is only needed
if the q-agt service is enabled.
Sourcing conditionally helps build the devstack fullstack gate jobs
for networking-bagpipe.
Change-Id: I2fed6177eb35749bebd4b75d60b189c2abdb0cdd
Needed-By: I967ba442ef433b7ada5a1a381f1571ec3cf23517

This is done by other service providers devstack plugin,
and mandatory to have the job
gate-rally-dsvm-neutron-extensions-rally-nv passing in
https://review.openstack.org/#/c/386418/
Change-Id: I2b9c0c34fbd2ffbc4695a3b3c440b8f46adb65ed

networking-bagpipe now has a proper way to set BGP peers
to an empty list, which we should use in our devstack gate job config
because our job is single node.
Additionally, we can specify the use of the OVS driver of bagpipe-bgp
in a simpler/shorter way.
Change-Id: I32701b29a3314bcdcfba6ba73d71ddcf3c62afa4

Provide a 'bgpvpn' entrypoint definition, to allow the activation
of the service plugins in a less verbose way.
Change-Id: I751861c25e2e301011d2c1738288660e1286a9a1

See http://lists.openstack.org/pipermail/openstack-dev/2017-January/111295.html
Change-Id: Iede105d4c3d225eca2625afaa98ae007992abf53
Closes-Bug: 1660973

This change contains two changes to fix our CI:
- a reno branch naming issue
- a permission issue in devstack VM functional job
On the reno branch naming issue: something must have
changed somewhere in the way we should name branches
in reno files (old way: origin/stable/newton
new: stable/newton).
...
 ldr = loader.Loader(conf)
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/loader.py", line 60, in __init__
 self._load_data()
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/loader.py", line 80, in _load_data
 self._scanner_output = self._scanner.get_notes_by_version()
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/scanner.py", line 577, in get_notes_by_version
 versions_by_date = self._get_tags_on_branch(branch)
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/scanner.py", line 344, in _get_tags_on_branch
 for c in self._get_walker_for_branch(branch):
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/scanner.py", line 338, in _get_walker_for_branch
 branch_head = self._get_ref(branch)
 File "/home/teom7365/prog/openstack/networking-bgpvpn/.tox/releasenotes/local/lib/python2.7/site-packages/reno/scanner.py", line 334, in _get_ref
 raise ValueError('Unknown reference {!r}'.format(name))
ValueError: Unknown reference 'origin/stable/newton'
Change-Id: I22c09534bc713f4eb00d106285a07acdd6e99730

This reverts commit dd4941459c.
The patch for devstack has merged :
I48c86f94098f1501f0e7f90a265dda7e81440eb0
Related-Bug: 1629133
Change-Id: I69907b256af6218c4cfe7aa2b4ba0508991fe4b9

This patch adds a scenario test that creates an l3 VPN and
assigns two neutron networks/subnets to it. Two nova VMs are
spun up on each network. To verify that the VPN is functional,
i.e., forwards traffic between the two subnets, one of the
VM pings the other one.
Change-Id: Ic38aab48fa364ad53ef061566fe5ebc1a46fec33
Closes-Bug: #1637489 

We need to follow devstack change I3d98692e69d94756e0034c83a247e05d85177f02 or
our tempest config addition comes before the config is completely
reset by devstack tempest setup (lib/tempest:configure_tempest rm -rf at [1]).
[1] 0474b8384a/lib/tempest (L177)
Change-Id: I024effd277b90c20c17ae09d291cbadfe5c03f57

Our tempest test scheduled on osic-cloud1 all get stuck
in requirements installation.
The problem was diagnosed (fungi, tmorin) to be related
to the fact that the neutron devstack code for setting
up routers is installing via br-ex a route to SUBNETPOOL_PREFIX_V4,
which defaults to 10.0.0.0/16, which in the case of
osic-cloud1 contains the gateway IP to reach the Internet
(10.0.0.1):
fungi@ubuntu-xenial-osic-cloud1-s3500-5253075:~$ ip ro sh
default via 10.0.0.1 dev ens3 onlink
10.0.0.0/16 via 172.24.5.11 dev br-ex
10.0.0.0/8 dev ens3 proto kernel scope link src 10.5.131.187
172.24.5.0/24 dev br-ex proto kernel scope link src 172.24.5.1
This change is a temporary fix for networking-bgpvpn that
sets SUBNETPOOL_PREFIX_V4 to a prefix *not* containing 10.0.0.1.
Change-Id: I5e4b99c6eb06837be7e1b5a9ed456f57ab6f9514

- migration tests now have time to execute
- add hooks to have database configured for functional tests.
- add hook to see result of tests.
Change-Id: I0d2406ef10df897fabe44e6fc287ea292843ecce
Closes-Bug: 1632212

Compiling a fresh upstream OVS is not required anymore now
that our tempest job is configured to use xenial.
(Note that the devstack-gate-bagpipe-rc script is only used by the
Openstack CI gate scripts so removing the OVS compilation trigger
does not impact non-gate use of the devstack plugin)
Change-Id: I50f2b6275400877e6e727a435406e362390dad46

The current ovs master is broken [1].
This is a temporary fix.
The target fix should be switching our dsvm jobs to use xenial instead
of trusty as a base distro.
[1] http://openvswitch.org/pipermail/dev/2016-October/080783.html
Change-Id: I900ef9991b9c7fde8252635f23f0afeea4b10756

Most of the changes in d088925891 should
have been made on the branch after it was created. The update to
requirements was the only one that should have been made in master
before the branch.
The change to .gitreview and tox.ini will be made automatically by the
branch script. The other changes should be done by hand in
stable/newton.
Change-Id: Ief5c73ab4d1cd5fd032dede21f6a244bfa5777c2
Signed-off-by: Doug Hellmann <doug@doughellmann.com>

This change updates our tree for Newton:
- tell "git review" to push in the stable/newton branch.
- configure tox to use the stable/newton upper constraints file
- let tox test UTs with newton release of networking-bagpipe
- let devstack jobs to use newton branch of networking-bagpipe
Change-Id: I67a1103eae0219e5cefa91fd5848aa0b8986c542
Depends-on: I719a9912ba8e7fe87620850017bdda83431e22da

This is required because the place where plugin_agent_add_l2_agent_extension
is defined is not sourced by default.
Change-Id: Ib1de7cfec457a312319edddd5262cd78c6599f0b

This change is needed to have the OVS agent extension for the bagpipe
driver enabled (because of [1]).
[1] https://github.com/openstack/neutron/blob/stable/newton/devstack/lib/l2_agent#L12
Change-Id: I1f4c86a6ae15985e581d0fd0a03754b2c5a5d571

Add a new panel BGPVPN in project section to display BGPVPN created
for non admin user.
The page allow actions as edit and update associations
networks or routers for BGPVPN resources.
Some part of code has been refactored to put in common methods or
classes between project and admin panels.
Change-Id: I49bfdfe1427c67f6b19a7b4faec0ab77a15ce8a5
Implements: blueprint horizon-tenant-workflow

remove override of DEVSTACK_GATE_TEMPEST that is defined in jenkins job,
and run only bgpvpn's tempest tests using a regex filter.
Change-Id: Iae534db7bd34624fad135dad4f5b6f34fc5143bf

Add a new panel BGPVPN in Admin section to display BGPVPN created. The
page allow actions as create, delete or modify BGPVPN resources.
Add an API file bgpvpn to handle the bgpvpn Python API via
neutronclient.
Change-Id: Iee2de53c8cb9d61f80033ab63485bdcf0e742abb
Implements: blueprint horizon-admin-workflow

Follow changes made in how Neutron devstack handles
compiling a fresh OVS (see Ia7ad1658b95d7404384c7cae833008a57e3e5af1).
The benefit is that our devstack code is simplified.
Change-Id: I3abc2ac4ce2d4710f998e7749d1f5df4303533e2

Change jenkins job rc file to enable bagpipe-bgp service
and configure it with the OVS MPLS dataplane.
This will automatically enable the bagpipe_bgpvpn OVS agent
extension.
It will also compile and install a recent OVS in the gate
(reusing neutron devstack scripts for this purpose).
Fix xtrace restoration.
Change-Id: Idb4fdad0da7bed5eda0c302bc79c5f05d50b44d7

So that the project-config change Ifbd24926013137407107d342576974d3fe5d8f5e
can be merged.
Change-Id: Iea3dcad95416668d741bf6562bd9cbcb8d7e7ac9

This files are missing and causing our experimental jobs to
not work at all.
This should have been done in I8d9580b480b1cdd3d185d51f13cd06be26310d17.
Change-Id: Iff18371068b82b341e00b391c0adf77c966967da
Closes-bug: 1561206

Since bgpvpn might be run with the neutron l2 agent but without
bagpipe, we should only enable the bagpipe l2 agent extension
when devstack is told to use bagpipe.
Change-Id: I4e085f489ed99b72f09e03cffcd2814a1721dfe8
Closes-bug: 1559149

This change introduces an OVS L2 agent extension that will now be used
instead of the previous agent in ovs_bagpipe_neutron_agent.py which
was inheriting from the Neutron OVS agent in a crude way, requiring
duplicating the main() code.
We also remove mention of the old bagpipe-openvswitch agent
in the documentation and and adapt the devstack plugin so that
old local.conf will still work.
Change-Id: Ibf27e2b028efffa1e49207d39bfc9b13464180ad
Closes-Bug: 1492021
Closes-Bug: 1531459

Adding rc files to allow to control the gate jobs parameters
without having to touch Jenkins job definitions in project-config.
Change-Id: I8d9580b480b1cdd3d185d51f13cd06be26310d17

This patch adds a heat plugin to provide BGPVPN and
BGPVPN-NET-ASSOCIATION resources to HEAT.
Partially-Implements: blueprint heat-support-bgpvpn-association
Change-Id: Ibebbb306abafc8306601fcf5484fc97ee753fba9

The bgpvpn devstack plugin currently configures tempest during
the post-config stage. But the tempest.conf file is overwritten
by tempest during the extra stage of devstack, hence erasing
modifications made by the bgpvpn plugin.
To fix this issue, the modification of the tempest.conf file
must be done during the extra stage of devstack.
This patch also corrects the indentation of the devstack plugin.
Change-Id: Ia2e76f1c9aea93e481694b0821d1e5c3bed4da11
Closes-bug: #1548364 

Change-Id: I0cb5fefc5a620f101dd157e7ffd9898bf4b880c2

That version uses the key/value store exposed by the OpenContrail
API but that's not efficient. The BGP/VPN resources and sub-association
need to be added to the OpenContrail data model.
That first PoC will be delivered to with the Liberty release but will be
replaced by a final one for Mitaka and next releases without any
migration script or method from the first PoC version.
Implements: blueprint opencontrail-bgpvpn-driver
Change-Id: I3795dd7b24afdac1cd5134c41b6b32ef8a888287

The current code was not working (NETWORKING_BGPVPN_DRIVER_BAGPIPE
not defined early enough to be usable inside local.conf)
Change-Id: I4a305bfe119c605145401c5918c362e2d28b31b7

Do the copy of config file at the 'config' stage.
(/etc/neutron does not exist yet at the 'install' stage)
Change-Id: I9d98356fa864aada125d87893bfb5c3aeba347eb

Allows to have less stuff in local.conf, and usefull
to tidy up jenkins DVSM job definitions.
Change-Id: I18ab3027f92a8fd23602757d6dfce13bde10f3a4

This change updates and cleans up our alembic migration code:
- we now use an alembic migration entry point, instead of shipping
 our own db management tool
- use expand/contract alembic branches
- downgrade is not supported anymore, drop it
Change-Id: I8eb68075c720cd7252bd178c1bf08ef0947cd0d0
Closes-Bug: 1489876

It is too late to call neutron_service_plugin_class_add at post-config.
Change-Id: Id8140f43daafdd314ad63dd2a39db55b1e7cb8b5

Devstack will now use the help for adding a service
plugin definition, thus avoiding the definition of
Q_SERVICE_PLUGIN_CLASSES in local.conf.
Change-Id: I3299a5a7f35ea7eff5353cc1028b056da3b0c33f

With this patch, Neutron will read (/etc/neutron/)networking_bgpvpn.conf
for service_provider definitions, thus allowing the use of
the BGPVPN service plugin additionally to other service relying
on their own neutron_*aas.conf file.
This patch also fixes README and devstack behavior so that
a default networking-bgpvpn.conf file is deployed.
This depends on change Ia4cad678e6c722ca05821dbdbf05d61523246a86
Change-Id: Ifebe4c965448431737d9f0e4c25605b718850460
Closes-bug: 1473119

Need to create neutron/policy.d directory before
copying our policy file there.
Change-Id: I8ffc6950f21163e03ac80633579eb14e3ff56b32

This commit adds:
* policy.json file
* setup.cfg hooks so that this file is copied into
 /etc/neutron/policy.d
* devstack plugin.sh change to put the file in the
 right place
Change-Id: If605f3fd4113f5e193d95d2d95449939caa8bdea
Closes-bug: #1480165 

When the devstack plugin is run, but q-scv is not
enabled (like on a compute node), we should not
run bgpvpn-db-manage.
Change-Id: I92893b7848cf92d1ab9a476469efdc30928e6bb1

including:
- add a README-bagpipe.rst for the bagpipe driver
- add setup.cf entry points
- extend devstack plugin so that the modified agent
 can be enabled with Q_AGENT=bagpipe-openvswitch
Change-Id: I6cd18eca77e5f2b43b0b15d141637cef6db044e4

* devstack plugin to:
 * install the package
 * to the db initialisation
* update README.rst to describe how to use the devstack plugin
Change-Id: I27ae16f265570249953bb3ad8a758891529a20b7