Dovetail Beta

F.A.Q.

1. What is the current (beta) version? Is there a release note for the current (beta) version?
   

   The current version is 0.8.0 for Dovetail, released on 10/25. All dependent components (Docker containers) are listed here: 

   • Running history for the dovetail tool#1.8cvp.0.8.0(2017/10/25)

   Yes, the release note documents all KNOWN ISSUES that should be reviewed before testing:

   • Running history for the dovetail tool#4.KnownIssuesList
2. Why does the user guide outline ‘testsuite’ option argument in ‘dovetail run’ command that does not work?
   Convention ‘OVP_1_0_0’ for ‘testsuite’ option, where versioning represents ‘<major>_<minor>_<patch>’ will be adopted in future releases. 
   For current version opnfv/dovetail:cvp.0.8.0, please use ‘cvp.0.8.0’ as the argument for ‘testsuite’ during the beta.
   Suggest to not give testsuite when running tests, then dovetail will use the default one for each specify version.
   
   
3. What image formats must the SUT compute nodes support?
   Images are submitted to the image service as qcow2, so compute services must support this format.
   
   
4. Does direct result upload from test host to OVP portal work?
   No, the user must first obtain the singular tar.gz file from test host and upload it through the OVP portal.
   
   
5. My tests were passing initially but are starting to fail without any configuration changes to the system under test. Why is this occurring?
   Dovetail creates many objects (images, networks, etc.) within the SUT public APIs during the course of testing. These objects are normally removed at the end of each test. However, there are scenarios where this fails to happen. For example, this can occur when a use kills the dovetail test while it is still running or when a dovetail console session drops during testing. Using Linux 'screen' to persist terminal sessions or avoiding killing (Ctrl-C) Dovetail during tests runs is encouraged to avoid this issue. Deletion of stale objects in the SUT admin services may also be necessary to remedy the failing tests.

Beta testing:

• Dovetail, together with C&C committee, plans to conduct CVP Beta testing to get direct feedback from
  • Vendors who have applicable products and intend to apply for CVP certification for these products
  • Third party labs who intend to apply to become OPNFV Preferred Verification Labs
  • Operators/end users members who are interested in utilizing CVP for qualification
• Benefits of participating in Beta
  • Be listed as the first group of products or labs that have participated in the beta program
  • Get direct access to dovetail test suite developers and c&c reviewers
• Beta process
  • Entering Beta
    • Dovetail test tool and test cases freeze (e.g. 0.7.0), no new development to the test tool or to test cases
    • A group of companies (vendors, labs or end users) volunteered to be beta trial testers, submitting feedback / bugs. # of companies should be >= 5, including vendors, labs and end users.
  • Test result review
    • Beta participants submit Beta test results for review the same way one would in an actual application
    • C&C committee organizes a CVP review group for Beta test result review. The review result will be shared with the participant who submitted the test result.
  • Bug fixes
    • During the Beta trial, testers are expected to be the primary source of bug reports. All reported bugs will be logged, reviewed and prioritized.
    • During the Beta trial no new tests or features will be merged into the Dovetail master branch.
  • Exit Beta
    • All critical bugs are fixed, all non-critical bugs are documented
    • If , for example, 0.7.x is the version when the above exit criteria is met, then we will re-label it 1.0.0, and it becomes the first release.
  • Duration
    • We expect the duration of Beta be up to 6 weeks.
• The first release is ready for launch after Beta.

Beta schedule

The beginning date of beta testing is estimated to be the end of September. Here is a rough schedule (last updated 9/8).

dovetail status 9-8.pptx

• Beta start date: 9/29/2017
• Beta end date: estimated to be 10 Nov 2017  (6 weeks)

Beta participants

Beta documentation

• Dovetail (Danube) Documentation for Review

Beta process and support

• Beta testing technical support email alias (being created)
• To report bugs in JIRA : 
  • https://jira.opnfv.org/projects/DOVETAIL/versions/11001
  • Select project = “OPNFV qualification testing (DOVETAIL)”
  • Select issue type as appropriate
  • Select Affects Version/s = “Dovetail Danube”
  • Select Fix version=“Dovetail Danube”, which we will review in the weekly calls.
• To start the application process (this is Beta, so we are still testing this process out.)
  • Email to cvp@opnfv.org
  • In the email, provide information required by answering questions in the workflow doc - Step 1
    • http://artifacts.opnfv.org/dovetail/docs/testing_user_certificationworkflow/index.html
  • Any general inquiries regarding to application process, also email to cvp@opnfv.org
• Submit test results for review (for testers) or Review the submitted test results (for reviewers)
  • Use this web portal: https://cvp.opnfv.org
• If none is working out for you, contact directly : Unknown User (wenjing)

Beta release information

• Beta Dovetail release 0.8.0: - release date 10/25/2017
  • Running history for the dovetail tool
• Known issues release note:
  • Running history for the dovetail tool#4.KnownIssuesList