Why does the user guide outline ‘testsuite’ option argument in ‘dovetail run’ command that does not work? Convention ‘OVP_1_0_0’ for ‘testsuite’ option, where versioning represents ‘<major>_<minor>_<patch>’ will be adopted in future releases. For current version opnfv/dovetail:cvp.0.8.0, please use ‘cvp.0.8.0’ as the argument for ‘testsuite’ during the beta. Suggest to not give testsuite when running tests, then dovetail will use the default one for each specify version.
What image formats must the SUT compute nodes support? Images are submitted to the image service as qcow2, so compute services must support this format.
Does direct result upload from test host to OVP portal work? No, the user must first obtain the singular tar.gz file from test host and upload it through the OVP portal.
My tests were passing initially but are starting to fail without any configuration changes to the system under test. Why is this occurring? Dovetail creates many objects (images, networks, etc.) within the SUT public APIs during the course of testing. These objects are normally removed at the end of each test. However, there are scenarios where this fails to happen. For example, this can occur when a use kills the dovetail test while it is still running or when a dovetail console session drops during testing. Using Linux 'screen' to persist terminal sessions or avoiding killing (Ctrl-C) Dovetail during tests runs is encouraged to avoid this issue. Deletion of stale objects in the SUT admin services may also be necessary to remedy the failing tests.
Beta testing:
Dovetail, together with C&C committee, plans to conduct CVP Beta testing to get direct feedback from
Vendors who have applicable products and intend to apply for CVP certification for these products
Third party labs who intend to apply to become OPNFV Preferred Verification Labs
Operators/end users members who are interested in utilizing CVP for qualification
Benefits of participating in Beta
Be listed as the first group of products or labs that have participated in the beta program
Get direct access to dovetail test suite developers and c&c reviewers
Beta process
Entering Beta
Dovetail test tool and test cases freeze (e.g. 0.7.0), no new development to the test tool or to test cases
A group of companies (vendors, labs or end users) volunteered to be beta trial testers, submitting feedback / bugs. # of companies should be >= 5, including vendors, labs and end users.
Test result review
Beta participants submit Beta test results for review the same way one would in an actual application
C&C committee organizes a CVP review group for Beta test result review. The review result will be shared with the participant who submitted the test result.
Bug fixes
During the Beta trial, testers are expected to be the primary source of bug reports. All reported bugs will be logged, reviewed and prioritized.
During the Beta trial no new tests or features will be merged into the Dovetail master branch.
Exit Beta
All critical bugs are fixed, all non-critical bugs are documented
If , for example, 0.7.x is the version when the above exit criteria is met, then we will re-label it 1.0.0, and it becomes the first release.
Duration
We expect the duration of Beta be up to 6 weeks.
The first release is ready for launch after Beta.
Beta schedule
The beginning date of beta testing is estimated to be the end of September. Here is a rough schedule (last updated 9/8).