Skip to end of metadata
Go to start of metadata

How to contribute new tests to CPerf.

Overview

OPNFV testing projects use test tools containers to insert their tooling into deployed OPNFV environments. Containers are defined using Dockerfiles in CPerf's git repository. A Jenkins job to build the tools Dockerfiles into Docker images and push them to DockerHub is defined using JJB in the RelEng's git repository. Jenkins jobs that use the built tools containers to run the new test tool are defined in CPerf's JJB definitions in RelEng's repo.

Prerequisites

Adding Tools to Containers

The first step in getting a new test tool running in OPNFV CPerf's CI is to install it in our test tools container. We can then insert our tools container into an OPNFV environment (deployed by an OPNFV installer like Apex) and run our tests.

Tools containers are defined in Dockerfiles in CPerf's docker/ repository directory.

Adding to Monolithic Container

Patch cperf/docker/Dockerfile to install your new tool. Use clear code and inline documentation. Push your changes early and often to CPerf's Gerrit for best review results. Make sure that the CPerf Committers are added as reviewers.

Adding New Container

NB: This approach will likely be recommended in the long-term, but isn't currently recommended because the *docker-build-push* job needs to be refactored to more naturally support more than one container per project.

To better make use of the lightweight isolation provided by containers, it may sometimes make more sense to define a new, tool-specific container instead of installing it in our monolithic test tools container. For example, our main container is based on CentOS 7. If your tool doesn't easily install on CentOS, a container with a different base OS could be the quickest way forward.

Create a new Dockerfile in a new subdirectory of cperf/docker. Give the subdirectory a simple, short, clear name that identifies it as related to the new test tool. The Dockerfile must be called "Dockerfile", per Docker conventions. A README.markdown in your new subdirectory, with documentation about the Dockerfile, is encouraged. Push your changes early and often to CPerf's Gerrit for best review results. Make sure that the CPerf Committers are added as reviewers.

Advanced: Contributing Containers Upstream

Instead of adding tool install logic to CPerf's Dockerfiles, test tool stakeholders may want to contribute a container to the test tool's upstream project. Whenever possible, OPNFV CPerf strives to make contributions "upstream first" because we believe it's a healthier open source development methodology.

For example, CPerf contributors added Docker container build and orchestration logic to upstream NSTAT instead of doing it downstream in OPNFV. CPerf pulls NSTAT test tools containers from the upstream NSTAT DockerHub.

Building Containers

After a tool's install is defined in a CPerf Dockerfile, a Docker image needs to be built from it and pushed to DockerHub. For tools added to our monolithic test tools container, the existing job can simply be triggered via Jenkins (recommended for now, see TODO below). For tools installed in new Dockerfiles (for better minimalism/isolation/microservices architecture), a new build-docker-push job will need to be added.

Committers to CPerf should be in the opnfv-gerrit-releng-contributors LDAP group, which gives permission to trigger Jenkins jobs. Check your group memberships and raise a ticket with opnfv-helpdesk@rt.linuxfoundation.org if needed. Contributions who aren't committers should ask a committer to trigger the desired job.

Updating Monolithic Container

A Jenkins job for building the main, monolithic CPerf test tools container is already defined. Once your tool-install updates are merged into cperf/docker/Dockerfile, triggering the job will build a new image and push the update to DockerHub.

Login to Jenkins and find the cperf-docker-build-push-master job. Click "Build with Parameters" (if you can't see it you don't have permissions). Review the default parameters, but you will likely not need to change any. Click "Build" and watch the logs as your new container is built and pushed.

Building New Containers

NB: This approach will likely be recommended in the long-term, but isn't currently recommended because the *docker-build-push* job needs to be refactored to more naturally support more than one container per project.

Define a Jenkins job using the *docker-build-push* JJB job-template to build a Docker image from your Dockerfile and push it to OPNFV's DockerHub.

Once your new Jenkins job is merged, a committer can trigger it via the web UI and push build/push your new Docker image.

Adding Tests

Once you have your tool installed in a CPerf test tools container, you can define Jenkins jobs that pull built containers from DockerHub and use them to execute tests.

CPerf tests are defined as Jenkins jobs using JJB. In releng/jjb/cperf/cperf-ci-jobs.yml, define a new builder with logic for downloading your tools container and executing tests. You can use the cperf-robot-cbench builder as an example. Builder logic can be written inline with the JJB YAML or extracted to an external shell script. Add your new builder to the cperf-daily-builder to make it run when our daily tests are triggered.

Other Resources: