Proposed name for the project: DevOps feeder stack for Dovetail support
Proposed name for the repository: dovetstack
Project description:
At a high level the project plans to address pain-points of EUAG and Dovetail testing. The ownership of identifying and planning use case testing can not be just upstream driven, as the focus of upstream may not match both on content, time line and resources to fulfill the needs of OPNFV Dovetail.
XCI may address some of this pain points and their approach is outside(upstream for CI/CD) in, and we need an inside out approach to leverage OPNFV. As seen in case of establishing IMS or SBC test cases it has alluded both B and D/E versions of OPNFV in-spite of a Clearwater Opensource availability of vIMS.
The reasons are simple as Installers own Scenario and can not provide timely Orchestration of stack in a dynamic release cycle. Testing team can not commit either PODs or test development efforts to meet the release cycle and Dovetail can not absorb the efforts to enhance its value to up the bar for OPNFV industry standards.
The testing in sample VNF is more at tweaking middle-box at software level is more geared towards limited hardware testing for the performance and benchmarking for release testing. The intention of this 'dovestack' project is to drive mid-cycle Alpha and Beta testing for a given use case in OPNFV to act as a feeder for easing testing in Dovetail at the release times.
Included here is a flow/ process context diagram with reference to OPNFV EUAG and Dovetail chain. .
Refer Diagram for https://www.draw.io/#Hpramchan%2Fheat-translator%2Fmaster%2Fdoc%2Fdovestack.xml
Scope:
Project 'dovestack' as name suggests is addressing pain points of dovetail test plan development during the early part of release to proactively address the execution of selected use case needed for a given release of Dovetail
Current plan is to use Functest and associated snaps library to create stacks for use cases and same time use Dovetail test tools to support the Alfa, Beta and Release Candidate to feed to Dovetail project.
Specify testing and integration:
North bound or Upstream Integration is from top down with use case as specified by EUAG & Dovetail to pursue. For the first release for 'dovestack' in 'OPNFV' F release, plan is to use Clearwater vIMS to establish process for Alfa, Beta and RC for the Dovetial to take over the test scripts to be used by Installers and scenarios to meet the Dovetail requirements.
South Bound being a feeder to Dovetail requires that all the scenario test efforts and test area/cases establish, feed requirements of Dovetail. One can borrow from other projects like Sample VNF , Domino and others but as such there is no dependencies on those projects. The only idea is to reuse as much of information and procedures established within OPNFV all work groups and Projects
Debugging and Tracing :
The FUNCTEST will be used to support Devtstack and using snaps extensions therein is key to stack Deployment as well tracing stack errors encountered. The Debug Flag available in FUNCTEST prep will be used for identifying and fixing codes for stack creation.
Unit/Integration Test plans :
Unite tests will be based on use case, like for vIMS, it can be clearwater-live-tests and/or clearwater-fv-tests. First objective will be to identify standard open source libraries and OPNFV usable licensed codes from Github and or other upstreams. Enhance them to be able to fit into unit, module and integration testing Like one for vIMS being attempted in B & D/E Release.
Client tools developed for functest prep, status shows etc. - FUNCTEST and Dovetail tools have currently simple coverage and we can extend or enhance them as needed, based on use case execution.
In Scope
Identity a list of features and functionality will be developed.
1. Creating Stack using Openstack/devstack (Heat) CLI manually and automate through FUNCTEST snaps library
2. Leverage open source tests and adapt them for testing dovestack use case In this F release the focus will be on Clearwater vIMS tests.
3. Provide inputs through Alfa and Beta testing to complete Functional and Dovetail testing for Scenario selected for the release.
Out of scope:
A. Identifying use case is out of scope except the first release in OPNFV for dovestack to establish a beach head as feeder to OPNFV where Clearwater vIMS has been attempted in past releases.
B. Installers with real scenario , will be part of Dovetail and not dovestack.
In scope for future
The project is extensible in future for emerging use cases to enhance the EUAG through Dovetail/CVP agreements in OPNFV.
Testability:
Specify testing and integration like interoperability, scalability, high availability
1. Ability of EUAG to leverage the major use case for verifying suitability of OPNFV SUT based on CVP/Dovetail will be the main driving factor.
2. The focus will be on stack creation for use case and allowing Development testing for Dovetail to leverage in the release window.
What QA and test resources will be available?
This will depend on the use case being addressed. For Clearwater vIMS as feeder to Dovetail vnf testing, the need will be for 4-6 man-months of DevOps using Heat,Clearwater-heat-stack, FUNCTEST with snaps, Dovetail test tool and Devstack.
Additional Installer and scenario support will be in realm of Dovetail. Although collaboration of exchange of ideas to leverage dovestack is expected both with Installers, Testing and Dovetail teams.
Documentation:
Dovestack user guide with functional block description using DevStack, FUNCTEST and Dovetail tools
Test Area and test case Docs & test codes by VNF (vIMS for F) use case per release.
Migration path to Dovetail testing of VNF (vIMS for F) by Installer Scenario
Dependencies:
1. Clearwater-heat template (https://github.com/Metaswitch/clearwater-heat/) is the Opensource vIMS used for specific test development for vIMS and will need to choose a specific upstream releases that matches heat capability.
2. Dependency on FUNCTEST to use credentials for devstack from Jumphost to SUT and Stack deployment/testing
3. Dependency on Dovetail tool to support developed or borrowed upstream tests through Dovetail tool.
4. Although Pharos and Scenarios are key to demonstrate Installer's ability to migrate to Dovetail, the test area/test case standards development during Alpha & Beta stage does not tie to these limitations per say and can progress identifying the constraints to concentrate on testing VNF for use case at hand. Pharos may itself needs to evolve to meet the use case requirements as it expands.
5. The project is an Test use case development and testing project , and has requirement of a Jumphost and 2 vPODs for alpha & beta (two scenarios) and a POD for Release candidate testing to feed to Dovetail.
CI or XCI : Prefer XCI for Openstack / Devstack
Releng or Releng(new) : TBD
Not a Dependency:
1.Sample VNF (https://wiki.opnfv.org/display/SAM) has similarity to efforts but it focuses more on characterization, performance and Benchmarking test suits which is not the goal of dovestack. So sample VNF is not considered as a dependency
2. XCI can help leverage upstream and will depend on the use case and Scenario used for testing in Dovetail, but not a dependency in Release F. But if the XCI can be leveraged dovestack to pull in Openstack (with Heat+Ceph) + Devstack, this project will consider using the XCI process gladly.
Committers and Contributors: (Please add your names and affiliations , based on your contribution plans.)
Names and affiliations of the committers
Prakash Ramchandran (prakash.ramcandran@huawei.com) -Futurewei Tech, Inc.
Trinath Somanchi ( trinath.somanchi@nxp.com ) - NXP
Srikanth Kumar Lingala ( srikanth.lingala@nxp.com ) - NXP
Mohan kumar (nmohankumar1011@gmail.com) - Non-member Individual Contributor(tentative)
Names and affiliations of any other contributors
Prasad Gorja ( prasad.gorja@nxp.com ) - NXP
2 Comments
Unknown User (christopherprice)
Why are these activities not planned through the testing working group and existing projects? Creating another vehicle which may further separate DoveTail from the testing community does not seem like the best approach, one might consider if these activities could not simply be planned in the testing working group rather than in another project.
Unknown User (rprakash)
Your point well taken. The information vIMS use case was brought to test working group based projects participating in OPNFV mano wg E/F and such as Opera, Orchestra, Auto and the level of support and resources were limited. Having attempts to complete vIMS best practices through testing wg in Reelase E , projects referred had to deal with Infrastructure wg and availability of resources constrained the execution of vnf ims use case by different participants. The bottleneck with Test WG was clear as their priority is Release only and other viewpoints have lower priority. The Proposal is cross wg and ownership can be with any wg as long as the purpose is served ie. continuous feed of test cases to Dovetail based on EUAG requirements. We can debate the ownership but vnf testing for simple to complex cases covers Function Under Test(FUT) spread across System under Test(SUT). So lets debate and arrive at a reasonable conclusion that a Project like Dovestack needs to be Feeder from EUAG use case selection to Dovetail passing through FUNCTEST at the minimum. The Projects upstream have a different time line and trying accomplish something in OPNFV through Installers and Scenarios need a distinct approach to deliver the usecase testing, one similar to XCI for upstreams to OPNFV.