Preparation & useful links

Post mortem

Global Testing post-mortem: https://etherpad.opnfv.org/p/Colorado-Testing-postmortem

Some graphs based on the validated scenarios (stored from reporting)

Colorado Release Time plan

Please Note

the dates have been modified by Functest and have not been validated. They correspond to what something reasonable according to Functest contributors
OPNFV Plugfest 9-13/5
OPNFV Summit 20-23/6

See Colorado for more details

Summer vacation planning

Who	When	Comments
morgan_orange	15/7-5/8
CG_Nokia	1/7 - 31/7	In Ireland 1st July -> 7th July Back in Finland 8th July
viktor_t	11/7 - 31/7, 8/8 - 10/8
Jose	1-25/8	Summer vacation in Spain.
juhak	11/7 - 7/8
JuhaHaapa	11/7 - 7/8
ollivier	13/7-5/8
lhinds	22/7 - 08/8

Colorado Functest test cases

Colorado test cases are declared in https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml and in the Test DB (http://testresults.opnfv.org/test/api/v1/projects/functest/cases)

Tier	Test case	Project	CI loop	Estimated duration
0: healthcheck	healthcheck	functest	merge/daily/weekly	2
1: smoke	vping_ssh	functest	merge/daily/weekly	2-4
	vping_userdata	functest		1-2
	tempest_smoke_serial	functest		15-20
	rally_sanity	functest		10
2: sdn_suite	odl	functest	daily/weekly	1
	onos	functest		1
	ocl	ovno		?
3: features	promise	promise	daily/weekly	1
	doctor	doctor		5
	bgpvpn	sdnvpn		1
	odl-sfc	sfc
	onos-sfc	onosfw
	domino	domino
	copper	copper
	parser	parser
	multisite	multisite
4:Openstack	rally_full	functest	weekly	60-90
4:Openstack	tempest_full	functest	weekly	60-90
5:VNF	vims	functest	weekly	30

We introduced the notion of Tiers in order to assign a category to the test cases and ultimately save time in CI.

Project management

Coordinated schedules with other projects and captured cross project dependencies in JIRA

Dependencies to SDNVPN project: FUNCTEST-181 - Getting issue details... STATUS SDNVPN-19 - Getting issue details... STATUS SDNVPN-38 - Getting issue details... STATUS
Dependencies to Promise project: FUNCTEST-162 - Getting issue details... STATUS PROMISE-35 - Getting issue details... STATUS
Dependencies to ONOSFW project: FUNCTEST-182 - Getting issue details... STATUS (still needed linked dependencies)
Dependencies to OVNO project: FUNCTEST-183 - Getting issue details... STATUS (still needed linked dependencies)
Dependencies to Doctor project: not defined yet
Additional projects to be discussed (Copper? Policy?)

Established a concrete project time plan in place that accommodates dependencies and resource availability

Described in the tables above and below.

Clarified dependencies, must include upstream dependencies, impacting ability to execute on future milestones

The only upstream tools used in Functest are Openstack/Tempest and OpenStack/Rally projects and they do not impact any milestone as we use fixed stable versions that are always in line with the continuous OpenStack releases.

In addition: please include a link to a JIRA filter and/or dashboard that includes your documented JIRA issues.

Dashboard: https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=59

List of JIRA issues:

Loading …

Key	Summary	T	Created	Updated	Due	Assignee	Reporter	P	Status	Resolution
FUNCTEST-272	Replace system calls between python scripts		May 30, 2016	May 21, 2017	Jun 30, 2016	Jose Lausuch	Cedric Ollivier		Open	Unresolved
FUNCTEST-271	Replace coreutils system calls by os library		May 30, 2016	May 21, 2017	Jun 30, 2016	Jose Lausuch	Cedric Ollivier		Open	Unresolved

2 issues Refresh

Colorado content for Functest

Note: most of the task have only 1 owner and can be done in parallel.

test evolution	new test	framework evolution
ODL (Colum, Cedric, Ragha, Juha)	Security Group (David)	Dashboard evolution to Kibana (Juraj) changed DB backend from Mongo DB to Elastic search to include native analytics functions (Juraj)
vIMS (Valentin)	security suites (Luke)	Test management refactoring - slicing the test suites from Health to VNFs (Jose)
Tempest evoution (2 levels smoke and advanced) back to upstream / upstream contribution	new VNFs (Bertand)	Add trust indicator to optimize test runs (Morgan)
Rally evolution back to upstream / upstream contribution	ARM suite (Ciprian)	new unit test on API (Serena)
Onos evolution	Flashtest (Nikolas)	Add swagger for automatic documentation of the API (Serena)
	Opencontrail / Ovno (with OVNO contributor)	Generate automatic test reporting (Morgan)
		Refactoring of Jenkins page (May)
		API evolution (include version, scenario, success criteria, trust indicator.. (Morgan)
		Add web admin frontal to declare/modify/remove new test cases/pods/projects in DB (Mongo then Elasticsearch) (Ragha)

Feature project Matrix

Project	Project declared in test DB	Test cases declared in test DB	JIRA created	History	Link to test dir in project repo	Test duration evaluation (min)	success criteria	Part of release criteria	Dependecies (installers/scenarios)
promise	yes	yes	yes	> Brahmaputra	yes	1	PASS	Yes	installer: fuel, joid
doctor	yes	yes	no	> Brahmaputra	yes	5	PASS	Yes	installer: apex
sdnvpn	yes	yes	yes*	> Brahmaputra	yes	1	PASS	Yes	scenario: bgpvpn
copper	yes	yes	yes	Colorado	tests	10	PASS	Yes	installer: all scenarios of JOID, APEX
models	no	no	no	Colorado	?	10	PASS	Yes	installer: all
moon	yes	yes	yes	Colorado	?	5?	PASS	Yes	scenario: moon
movie	no	no	no	Colorado	?	5?	PASS	To be decided	scenerio:os-odl_l2-*, Installer:tbd
multisite	no (Not yet)	no Test cases ready in OpenStack repository, provide plugin to tempest tempest can discover the test cases	yes	Colorado	?	10?	PASS	To be decided	Use preconfigured multisite testbed for no installer support multisite installation automation yet.
parser	no	no	yes	Colorado	?	5	PASS	To be decided	scenerio:os-onos-nofeatue-ha, installer:compass
domino	yes	yes	yes	Colorado	?	5	PASS	To be decided	scenerio:os-onos-nofeatue-ha, installer:compass Parser + Open-O
ONOSFW	yes	yes	yes	> Brahmaputra	yes (in 1-2 weeks)	~ 10	PASS	Yes	installer: all scenario: os-onos-sfc (vnffg)