...
Component | Description | Status |
config files | basiclly the same env config as the config in functest/yardstick, used to indicate SUT information | Done |
Testcase definition | testcase definition files, mapping to the testcase from functest/yardstick being executed. these testcases comprise the certification | Done |
testsuite definition | testsuite definition files, to determine the scenario of the SUT, and a collection of testcases | Done, basic definition |
parser | parse, load and check input files(config,certification,testcases) if they are valid | Done, basic function |
Testcase manager | Testcase management, execution status , etc | Done |
| Test runner | different runner for different test case type. | Done |
container manager | all stuff around the container, like pull image, start container and exec cmd in container | Done |
runner | main entry for running through the whole validating procedure | Done |
report | manage test results and generate reports from these results | Done |
dashboard | presentation of test report and show the details of each testcase | Optional |
.
Dovetail Test Definition Files
Dovetail uses two levels of definition files to specify what exact test cases are to be executed in a given test run or test suite specification.
Test Suite Definition File
This file defines a suite that is comprised of test cases which may be a partial list of all existing test cases. It gives the suite a name that is more convenient for reference.
- compliance_set: test suite name, the example is the test cases used for compliance
- testcase_list: a list of test case referenced to be in the named test suite
Test Case Definition File
- dovetail.ipv6.tc001: name of the test case, comprised of 3 parts, the dovetail project, belongs to which test area, and the serial number
- objective: a brief description of the test case, explains the objective of this test case
- validate: scripts and configurations for the validation of the test case
- type: which method used to validate, support 3 ways: functest/yardstick/shell
- testcase: the input parameter if "type" is one of the upstream testing project(functest/yardstick)
- pre_condition: scripts that should be run before the test
- cmds: how to run the test
- post_condition: scripts that should be run after the test
- sub_testcase_list: use to check if these sub test case passed or not
