Page tree
Skip to end of metadata
Go to start of metadata

FuncTest Integration

How do I test with Functest prior to CI integration?

The process to get these test cases integrated into FuncTest is (in development):

1) Clone functest repo and add enhancements below, but do not merge until fully tested

2) functest/functest/opnfv_tests/features/copper.py: create new script in functest report

  • based upon doctor.py
  • retrieves the test cases from the copper repo
  • executes the tests and cleans up
  • reports the result

3) exec_test.sh: add new section

function run_test(){
test_name=$1
serial_flag=""
if [ $serial == "true" ]; then
serial_flag="-s"
fi
"copper")
python ${FUNCTEST_REPO_DIR}/testcases/features/copper.py $report
sleep 10 # to let the instances terminate
;;


4) functest/functest/ci/testcases.yaml: add new section

name: features
order: 3
ci_loop: '(daily)|(weekly)'
description : >-
Test suites from feature projects
integrated in functest
testcases:
-
name: copper
criteria: 'status == "PASS"'
blocking: false
clean_flag: true
description: >-
	Test suite for policy management based on OpenStack Congress
dependencies:
	installer: '(apex)|(joid)'
	scenario: '^((?!fdio|lxd).)*$'
run:
	module: 'functest.opnfv_tests.features.copper'
	class: 'Copper'

 

5) Test in isolation per http://artifacts.opnfv.org/functest/docs/configguide/configguide.html

6) Test in community lab POD (full CI based deploy) post-install

7) Merge patches and test in CI

 

Test Cases

Git repo link: https://git.opnfv.org/cgit/copper/tree/tests

TestDescriptionTested/Planned EnvironmentNotes
DMZ Placement

Pause VMs running in a DMZ zone, if the image running is not allowed to run in that zone.

Apex, JOID: Adhoc, FuncTest (in progress)

Policy rules tested:

1) create a dmz_server table entry for any active VM associated to the DMZ security group

openstack congress policy rule create test "dmz_server(x) :- nova:servers(id=x,status='ACTIVE'), neutronv2:ports(id, device_id, status='ACTIVE'),  neutronv2:security_group_port_bindings(id, sg), neutronv2:security_groups(sg,name='dmz')" --name dmz_server

2) find DMZ VMs are running images not tagged for “allowed in DMZ”

openstack congress policy rule create test "dmz_placement_error(id) :- nova:servers(id,name,hostId,status,tenant_id,user_id,image,flavor,az,hh), not glancev2:tags(image,'dmz'), dmz_server(id)" --name dmz_placement_error

3) pause any VM that shows up in the dmz_placement_error table

 openstack congress policy rule create test "execute[nova:servers.pause(id)] :- dmz_placement_error(id), nova:servers(id,status='ACTIVE')" --name paused_dmz_placement_error

SMTP Ingress Identify VMs that have TCP port 25 open for ingress.Apex, JOID: Adhoc, FuncTest (in progress)

Policy rules tested:

1) create an smtp_ingress table entry for any VM in a security group with TCP port 25 open for ingress

openstack congress policy rule create test "smtp_ingress(x) :- nova:servers(id=x,status='ACTIVE'), neutronv2:ports(port_id, status='ACTIVE'), neutronv2:security_groups(sg, tenant_id, sgn, sgd), neutronv2:security_group_port_bindings(port_id, sg), neutronv2:security_group_rules(sg, rule_id, tenant_id, remote_group_id, 'ingress', ethertype, 'tcp', port_range_min, port_range_max, remote_ip), lt(port_range_min, 26), gt(port_range_max, 24)" --name smtp_ingress

Reserved SubnetDelete any subnets created in restricted network ranges.Apex, JOID: Adhoc, FuncTest (in progress)

Policy rules tested:

 1-4) create a reserved_subnet_error table entry for any subnet created with this CIDR

openstack congress policy rule create test "reserved_subnet_error(x) :- neutronv2:subnets(id=x, cidr='10.7.1.0/24')" --name rsv_subnet_adm

openstack congress policy rule create test "reserved_subnet_error(x) :- neutronv2:subnets(id=x, cidr='10.7.12.0/24')" --name rsv_subnet_prv

openstack congress policy rule create test "reserved_subnet_error(x) :- neutronv2:subnets(id=x, cidr='10.7.13.0/24')" --name rsv_subnet_stg

openstack congress policy rule create test "reserved_subnet_error(x) :- neutronv2:subnets(id=x, cidr='10.7.14.0/24')" --name rsv_subnet_mgm

5) delete any subnet that shows up in the reserved_subnet_error table 

openstack congress policy rule create test "execute[neutronv2:delete_subnet(x)] :- reserved_subnet_error(x)" --name deleted_reserved_subnet_error

Network Bridging

Detects when a VM is connected to 2 networks with different security levels and pause that VMApex, JOID: Adhoc, FuncTest (in progress)
  • Policy rules tested:
    • Create Congress policy 'test'
    • dmz_connected rule in policy 'test'
    • admin_connected rule in policy 'test'
    • dmz_admin_connected rule in policy 'test'
    • dmz_admin_bridging_error rule in policy 'test'
    • paused_dmz_admin_bridging_error rule in policy 'test'
  • Outcomes:
    • Verify cirros1 and cirros2 IDs are in the Congress policy 'test' table 'dmz_connected'
    • Verify cirros1 and cirros2 IDs are in the Congress policy 'test' table 'admin_connected'
    • Verify cirros1 and cirros2 IDs are in the Congress policy 'test' table 'dmz_admin_connected'
    • Verify cirros1 ID is in the Congress policy 'test' table 'dmz_admin_bridging_error'
    • Verify cirros1 is paused

Test Database Data

Current planned tests are managed through the Swagger API (Select the "APIs" link in the first line). The YAML file is at http://testresults.opnfv.org:80/test/api/v1/projects/copper/cases.  Details are below. 

 

TestSwagger API UI Data (for update of the database as needed)
DMZ Placement

{
"name": "dmz",
"url": "https://git.opnfv.org/cgit/copper/plain/tests/dmz.sh",
"description": "An OpenStack Congress policy test. Sets up and validates policy creation and execution for: 1) Identifying VMs connected to a DMZ (currently identified through a specifically-named security group); 2) Identifying VMs connected per (1), which are by policy not allowed to be (currently implemented through an image tag intended to identify images that are 'authorized' i.e. tested and secure, to be DMZ-connected); 3) Reactively enforce the dmz placement rule by pausing VMs found to be in violation of the policy."
}

SMTP Ingress 

{
"name": "smtp_ingress",
"url": "https://git.opnfv.org/cgit/copper/plain/tests/smtp_ingress.sh",
"description": "An OpenStack Congress policy test. Sets up and validates policy creation and execution for: 1) Identifying VMs that have STMP (TCP port 25) open for ingress."
}

Reserved Subnet

{
"name": "reserved_subnet",
"url": "https://git.opnfv.org/cgit/copper/plain/tests/reserved_subnet.sh",
"description": "An OpenStack Congress policy test. Sets up and validates policy creation and execution for: 1) Detecting that a reserved subnet has been created, by mistake. 'Reserved' in this example means e.g. not intended for use by VMs."
}

Network Bridging

{
"name": "network_bridging",
"url": "https://git.opnfv.org/cgit/copper/plain/tests/network_bridging.sh",
"description": "An OpenStack Congress policy test. Sets up and validates policy creation and execution for: 1) Detecting that a VM is connected to two networks of different 'security levels' by mistake. 'Security levels' in this example means that the service provider assigns distinct sensitivity/risk to connections over those networks, e.g. a public network (e.g. DMZ) and an internal/private network (e.g. service provider admin network)."
}

Congress HA

{
"name": "congress_ha",
"url": "https://git.opnfv.org/cgit/copper/plain/tests/congress_ha.sh",
"description": "An OpenStack Congress test. Verifies that if one instance of an HA-deployed Congress service fails, the Congress service continues to function during the failure period and after HA is service is restored."
}

  • No labels