For VNF portal:
- Results should be a valid gzip file
- Files in this tarball file
- Need a results.json file with the following data format
{
"testcases_list":
[
{
"objective": "onap heat template validation",
"sub_testcase": [],
"mandatory": true,
"name": "onap-vvp.validate.heat", //The test case should be list in this file https://github.com/opnfv/dovetail-webportal/blob/master/3rd_party/static/onap-ui/components/results-report/data/2019.04/heat-testcases.json
"result": "PASS“ //it should be either PASS or FAIL
},
{
"objective": "xxx xxx",
"portal_key_file": "./vping_logs/functest.log", //The relative path of the log file of this test case, which can be open from the results detailed page of OVP portal.
"sub_testcase": [
{
"name": "xxx",
"result": "PASS"
},
{
"name": "xxx",
"result": "PASS"
}
],
"mandatory": true,
"name": "xxx",
"result": "PASS"
}
],
"build_tag": "daily-master-2c10cabc-59d8-11e9-a264-0242ac110002", //test_id = build_tag[13:49], so it should be daily-master-<UUID>
"version": "2019.04", //the version should be one list here https://github.com/opnfv/dovetail-webportal/tree/master/3rd_party/static/onap-ui/components/results-report/data
"test_date": "2019-04-08 08:30:29 UTC",
"duration": 8.432544946670532,
"vnf_type": "heat", //This is only for VNF, and it should be either heat or tosca.
"vnf_checksum": "18d6353e213e2cdf1efd4422e0ac633500d6202c801a9fcf4a1fe2cc5a488387“, //This is only for VNF.
"validation": enabled/disabled //This is only for NFVI to say that if it disables strict API response validation or not.
}
- Need a results.json file with the following data format
12 Comments
Lincoln Lavoie
What other files should be present in the gzip file, and how are those files handled by the portal? For example, the log files from each test case, etc. Show the "testcase" in the testcases_list point to such a log file?
Dan Xu
Other files are all optional. However, if there is no dovetail.log and results.json, then it will failed to upload or show the results.
Unknown User (tl2972)
When I look at the Dovetail documentation it calls for the log file to be a tar ball (tar.gz) not just gzipped, but #1 says it should be gzipped. When I uploaded my results I used the tar ball worked. Which is correct? Do both work?
https://opnfv-dovetail.readthedocs.io/en/latest/testing/user/userguide/vnf_test_guide.html
Dan Xu
From the code here https://github.com/opnfv/dovetail-webportal/blob/master/opnfv_testapi/resources/result_handlers.py#L262, it opens and reads with gzip compression.
IMO, tar.gz and gzipped should be the same because here it tries to compress a directory rather than one file, so it must be tar.gz.
Unknown User (tl2972)
OK, so if I understand this correctly, we will create the archive as a .tar.gz
Lincoln Lavoie
Dan Xu , can you expand what the structure of "sub_testcase": [], looks like. Could this be used to list results from "sections" of test, like "instantiation" vs "health check" for the VNF validation testing.
Dan Xu
Already added an example of sub_testcase.
Not quite understand the second question. "sub_testcase" is used to list the results of all sub test cases belong to each test case if it has. It is used by portal to show them (PASS or FAIL).
"instantiation" vs "health check" should be different test cases, don't they? Or they are 2 sub test cases of one test case?
Unknown User (tl2972)
Dan Xu Can we move forward with eliminating or making dovetail.log optional? Can the portal extract everything it needs from the json result file instead?
Dan Xu
For VNF, it could eliminate dovetail.log after doing some updates with the portal code. Then all data will only come from this results.json file.
Unknown User (ss820f)
Is there only one log file for a testcase, or can each sub_testcase have their own log file as well?
Dan Xu
There should be only one log file specified with 'portal_key_file' for each test case, then this file can be opened through clicking the button shown below on the results detailed page. The portal doesn't support to set log files for each sub test case now.
Unknown User (smangini)
If there is only one test, with multiple subtests, should there only be one testcaselist entry?
If yes, should the testcase name be in that one entry ? (It would otherwise be missing, based on the description here)