Page tree
Skip to end of metadata
Go to start of metadata

Background for OPNFV Release Cycle

Until Danube we may consider testing activities below. Release management usually considers only a subset of test projects as release criteria.

 

Moreover, we only got resources as CI Labs or community Labs as of Danube. In Danube, some tests require stable labs for long duration tests (1 week campaign, destructive tests,..).

Discussion of these tests could be found in https://wiki.opnfv.org/display/bottlenecks/Stress+Testing+over+OPNFV+Platform.

Previous releases show that it is currently impossible to plan long duration test campaigns on the "target" release/master due to a late availability and a lack of stability of the candidate versions.

It is thus planned to consider stable release for robustness/stress testing.

Another CI Chain for Stable Release

For Euphrates, the testing group would like to consider a new way to proceed, described as follow:

The test activities will thus deal with 2 versions of OPNFV in parallel

  • version N for the release validation
  • version N-1 for the qualification

As a consequence, 2 CI chains must be maintain per installer:

  • Master for release validation
  • Stable (version N-1) for qualification/long duration tests

With a 6 months cadence, it is not realistic to perform robustness/stress/long duration test on Master. On the stable chains, a planning shall be created.

We assume that unlike Master, reinstallation shall be triggered according to this planning, it means that some scenarios may be used during several weeks without reinstallation.

Generic scenarios must be considered first.

CI Chains summary

  • Master managed as today
  • Stable must be maintained, reinstallation triggered after a test campaign. Allocation of resources to be organized within the testing group

Test Planning for the Euphrates Release (Proposal)

After reporting and discussing with TSC and Infra Working Group, we have been granted an specific POD for the long duration tests, i.e.,  INFRA-154 - Getting issue details... STATUS .

Testing arrangement is proposed below.

2 projects share a timeslot for joint testing, i.e.,

  • either run test cases simultaneously, e.g., Storage and CPU testing in parallel,
  • or one project call the other to do some tests, e.g., Bottlenecks call Yardstick to do tests. 

Use OSA/Otaca to do the deployment considering currently resource limitation and to form common base for OPNFV installations.

  • Access to Intel-18 to locate the failures and root causes
  • Access to Intel-18 to show the testing reports
  • Feed back bugs that only exist in long duration tests

Each timeslot is assigned with 2 weeks to guarantee we do thorough test and to be able obtain useful results

Focus on « generic » scenarios first
  • The testing workflow should be triggered by CI system
  • Re-installation on demand (CI still needed for clean re-installation but re-installation not automated to allow long duration tests + troubleshooting)
  • Reports for the executed tests. It'd better have a unified report page, e.g., based on CI system, based on testing DB, etc.

Future Releases

TBD

Page viewed times

  • No labels