Testing Your Project with OpenStack-Ansible
A couple of weeks ago I wrote a blog about using OpenStack-Ansible (OSA) as your development environment for OpenStack.
Adding this development path was a key goal for the OSA team, but there was another key goal that came about as a result of the development environment work, and that was the ability to incorporate OSA's testing jobs into your project.
Why use OSA to test your project?
The OpenStack-Ansible team uses source to build our pip packages. These packages deploy the various OpenStack services, making it an ideal way to test code changes and bug fixes. OSA utilizes containers to separate out components, meaning we can test scenarios on a single host that you would otherwise need multiple hosts for. Examples of this are:
- "Zero down-time" upgrade tests, which require redundant services.
- Multi-region scenarios requiring multiple hosts with potentially segregated networking.
OSA has the benefit of having certain tested scenarios, such as functional testing for the nova-lxd driver, that is currently not tested in Devstack:
- For Keystone, "zero downtime" upgrades are tested.
- Multi-region Swift deployments utilizing containers.
- Upgrade jobs for Nova, Swift, Glance, Cinder, Neutron (the aim is to transition these to "zero downtime" upgrade tests as we integrate the project specific upgrade paths).
- Nova scenario for nova-lxd.
How to test one off bug-fixes
If you want to use OSA to test a bug fix against the OSA gate, you can be done right now, without any additional configuration setup.
The developer mode structure change has meant that you can use the "Depends-On:" tag to test patches from upstream projects. For example, we ran into a Nova bug (that impacts a few deployment projects) and Artom from the Nova team is working on a fix. The Nova gate isn't failing on this issue, but the OSA gate is, we can use the "Depends-On:" tag and reference the patch Artom is working on. This will deploy Nova from the OSA role, and run the OSA functional tests, using the code included in Artom's patch. Here is the Test Patch to show you how the integration works in practice, and how you would test Artom's patch.
What is currently being tested?
At present we have 2 upstream projects using OSA's testing jobs for their project:
- The Ansible Runtime Analysis (ARA) project - maintained by the excellent dmsimard, currently utilizes a base OSA test job to test ARA is working with a consumer.
- Keystone currently has an experimental job that was added just this week, that utilizes the "zero-downtime" upgrade job in OSA.
Ok that sounds great, how do I do this?
It does sound great, doesn't it?!
If you wanted to integrate an OSA test with your repository, for example a "zero-downtime" upgrade test, you could:
- Write a job that runs on a multi-node gate (including a Load-Balancer, such as HAProxy)
- Ensure there is capability in Devstack to set up and run the multi-node test.
- Add test configuration to OpenStack project-infra.
- Fix Devstack to make sure it works.
- Run the tests.
Alternatively you could configure the OSA "upgrade" job to run on commit.
We've updated (as of the 15th September 2017) the process this cycle, and put a bit of work into ensuring we're doing this in the most simple way. It turns out we weren't, and it's now very simple.
All you have to do is adjust project-config in the following way:
- Edit jenkins/jobs/projects.yaml to ensure the job is a valid job - this is only required if, for example, you want a non-voting job (-nv) when one doesn't exist yet.
- Adjust zuul/layout.yml to include the OSA job in your project's jobs.
A great example is this patch to enable this for the nova-lxd project.
Why does OpenStack-Ansible care about testing other projects?
We're pretty proud of the work we've done, and we think that we have a lot of value we can give back to the community.
However, nothing is perfect, and there are a lot of upstream projects and teams. This does make it hard to keep all the projects in a state that would be considered "following best practices". That isn't to say we don't try!
By helping developers and projects to use OSA for testing and development, we hope that it will encourage developers and projects to integrate with OSA and as a result, we'll see features and fixes happen as the real project experts get more involved. Improving the quality of OSA deployments, and in turn helping the developers and projects who are using OSA.
We've seen this with the Keystone team. We have worked closely with the Keystone team to get the OSA upgrade path for Keystone to mirror the intentions of the Keystone developers, and we hope that Keystone will now benefit from the test jobs we have setup for the upgrades. This is something we'd like to reproduce.
We strongly believe that working with the community, as more than just a consumer, is the best way to improve the quality and standards of the community as a whole.
Help us, to help you, to help us!
As always, reach out to me or the team on #openstack-ansible on Freenode IRC, or on Twitter/mailing-lists etc.