WARNING: This page is not currently maintained. Test areas will be defined in the test strategy document (being developed in this Gerrit proposal and tracked in this JIRA ticket - see current draft document), and specific test cases will be discussed and included in the Dovetail JIRA and Gerrit repository
This is the list of candidate test areas which have been proposed by community members for addition to the Dovetail test suite. These test areas will be evaluated under the Dovetail test case requirements, proposed for inclusion as patches in Gerrit against Dovetail, and reviewed by the Dovetail maintainers. Once approved, they will become part of the Dovetail test suite.
Brainstorming
This section is intended to help us get to quick consensus on the test priorities for Danube. The rest of the content of this page will be updated to be aligned with this section.
- If we need a more long-winded/threaded discussion on the topic, please add an item to https://techdiscuss.opnfv.org/.
Details on Test Areas
VIM Operations on Compute
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27493
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VIM operations on Compute | Create and show root certificates | Create and show root certificates | VIM: Openstack Components: Keystone, Nova | ||
| 2 | VIM operations on Compute | Flavors CRD ops | Create, delete, list and show flavors | VIM: Openstack Components: Keystone, Nova | ||
| 3 | VIM operations on Compute | Floating IPs CRUD ops | Allocate, delete, associate, disassociate, show and list floating ips | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 4 | VIM operations on Compute | Image metadata CRUD ops | List, set, update, show, delete image metadatas | VIM: Openstack Components: Keystone, Nova | ||
| 5 | VIM operations on Compute | Images CRUD ops | Delete, list, show and create images | VIM: Openstack Components: Keystone, Nova, Glance | ||
| 6 | VIM operations on Compute | Keypairs CRUD ops | Create, list, delete and show keypairs | VIM: Openstack Components: Keystone, Nova | ||
| 7 | VIM operations on Compute | Show absolute limits | Show absolute limits | VIM: Openstack Components: Keystone, Nova | ||
| 8 | VIM operations on Compute | Security group rules CRUD ops | Create, delete and list security group rules | VIM: Openstack Components: Keystone, Nova | ||
| 9 | VIM operations on Compute | Security group CRUD ops | Create, delete, show, update and list security groups | VIM: Openstack Components: Keystone, Nova | ||
| 10 | VIM operations on Compute | Interfaces CRUD ops | Create, delete, show and list interfaces; attach and deattach ports to servers | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 11 | VIM operations on Compute | List availability zones | List of availability zones and available services | VIM: Openstack Components: Keystone, Nova | ||
| 12 | VIM operations on Compute | Servers CRD ops | Create, delete, list and get servers | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 13 | VIM operations on Compute | Rebuild, resize and update server | Rebuild, resize and update server | VIM: Openstack Components: Keystone, Nova | ||
| 14 | VIM operations on Compute | List and show instance actions | List and show instance actions | VIM: Openstack Components: Keystone, Nova | ||
| 15 | VIM operations on Compute | Multiple create servers | Verify whether reservation id is in the response body of creating servers | VIM: Openstack Components: Keystone, Nova | ||
| 16 | VIM operations on Compute | Server actions | Change server passwords; reboot, rebuild, resize, pause, unpause, suspend, resume, shelve, unshelve, stop, start, lock, unlock server; create backup; get console oupt; show vnc console; | VIM: Openstack Components: Keystone, Nova, Glance | ||
| 17 | VIM operations on Compute | List server addresses | List server addresses | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 18 | VIM operations on Compute | Server group | Create/delete server-groups with different policies. policies = affinity/anti-affinity It also adds the tests for list and get details of server-groups | VIM: Openstack Components: Keystone, Nova | ||
| 19 | VIM operations on Compute | Server metadata CRUD ops | List, set, update, show, delete server metadata | VIM: Openstack Components: Keystone, Nova | ||
| 20 | VIM operations on Compute | Delete and get server password | Delete and get server password | VIM: Openstack Components: Keystone, Nova | ||
| 21 | VIM operations on Compute | Server personality ops | Create and rebuild servers with personality | VIM: Openstack Components: Keystone, Nova | ||
| 22 | VIM operations on Compute | Rescue and unrescue servers | Rescue and unrescue servers | VIM: Openstack Components: Keystone, Nova | ||
| 23 | VIM operations on Compute | Create server with special metadata | Create server with admin password; create server with existing server name; create specify keypair; update server name; create server with IPv6 only | VIM: Openstack Components: Keystone, Nova | ||
| 24 | VIM operations on Compute | List virtual interfaces | List virtual interfaces | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 25 | VIM operations on Compute | Attach and deattach volume | Attach and deattach volume; list and show volume attachments; delete attached volume | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 26 | VIM operations on Compute | Volume snapshot CRD operations | Create, show, list and delete volume snapshots | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 27 | VIM operations on Compute | Volumes CRUD ops | Create, show, list, update and delete volumes | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 28 | VIM operations on Compute | Show and list extensions | Show and list extensions | VIM: Openstack Components: Keystone, Nova | ||
| 29 | VIM operations on Compute | Live block migration | Migrating to an invalid host should not change the status | VIM: Openstack Components: Keystone, Nova | ||
| 30 | VIM operations on Compute | List networks | List networks | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 31 | VIM operations on Compute | Quotas CRUD ops | Show, create, update and delete quotas | VIM: Openstack Components: Keystone, Nova | ||
| 32 | VIM operations on Compute | List and show tenant networks | List and show tenant networks | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 33 | VIM operations on Compute | List and show API versions | List and show API versions | VIM: Openstack Components: Keystone, Nova | ||
| 34 | VIM operations on Compute | Agents CRUD | Create, update, delete and list agents | VIM: Openstack Components: Keystone, Nova | ||
| 35 | VIM operations on Compute | Aggregate CRUD | Create, update, delete, show and list agents | VIM: Openstack Components: Keystone, Nova | ||
| 36 | VIM operations on Compute | Auto-allocating networks | Tests auto-allocating networks with the v2.37 microversion. The tenant must not have any network resources available to it so we can make sure that Nova calls to Neutron to automatically allocate the network topology. | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 37 | VIM operations on Compute | Fixed ips | List, reserve and unreserve fixed ips | VIM: Openstack Components: Keystone, Nova | ||
| 38 | VIM operations on Compute | Flavors access | Add and remove Flavor Access require admin privileges | VIM: Openstack Components: Keystone, Nova | ||
| 39 | VIM operations on Compute | Flavor Extra Spec API extension | SET, UNSET, UPDATE Flavor Extra specs require admin privileges. GET Flavor Extra specs can be performed even by without admin privileges. | VIM: Openstack Components: Keystone, Nova | ||
| 40 | VIM operations on Compute | Floating IPs Bulk CRUD | Create, update, delete, and list Floating IPs Bulk that require admin privileges | VIM: Openstack Components: Keystone, Nova, Neutron | ||
| 41 | VIM operations on Compute | Host CRUD ops | Create, update, delete, show and list hosts using admin privileges | VIM: Openstack Components: Keystone, Nova | ||
| 42 | VIM operations on Compute | List and show hypervisors | List and show hypervisors using admin privileges | VIM: Openstack Components: Keystone, Nova | ||
| 43 | VIM operations on Compute | Instance usage audit log | Show and list instance usage audit log | VIM: Openstack Components: Keystone, Nova | ||
| 44 | VIM operations on Compute | Live block migration | Live migration between two hosts | VIM: Openstack Components: Keystone, Nova | ||
| 45 | VIM operations on Compute | Migrations admin | Live migrations and cold migrations | VIM: Openstack Components: Keystone, Nova | ||
| 46 | VIM operations on Compute | Nova network API | List and show network | VIM: Openstack Components: Keystone, Nova | ||
| 47 | VIM operations on Compute | Servers create ops on multinodes | Create servers on same host; create servers on different hosts | VIM: Openstack Components: Keystone, Nova | ||
| 48 | VIM operations on Compute | List and show services | List and show services | VIM: Openstack Components: Keystone, Nova | ||
| 49 | VIM operations on Compute | List and show tenant usage | List and show usage for tenants | VIM: Openstack Components: Keystone, Nova |
VIM Operations on Network
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27497
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VIM operations on Network | Ports CRU ops with allowed address pair | Create, list, update and show ports with allowed address pair | VIM: Openstack Components: Keystone, Neutron | ||
| 2 | VIM operations on Network | DHCP + IPv6 service | Test DHCPv6 specific features using SLAAC, stateless and stateful settings for subnets. Also it shall check dual-stack functionality (IPv4 + IPv6 together). The tests include: generating of SLAAC EUI-64 address in subnets with various settings receiving SLAAC addresses in combinations of various subnets receiving stateful IPv6 addresses addressing in subnets with router | VIM: Openstack Components: Keystone, Neutron | ||
| 3 | VIM operations on Network | List all available extensions | List available extensions for the project | VIM: Openstack Components: Keystone, Neutron | ||
| 4 | VIM operations on Network | Ports CRU ops with extra DHCP options | Create, list, show and update ports with the Extra DHCP options | VIM: Openstack Components: Keystone, Neutron | ||
| 5 | VIM operations on Network | Floating IPs CRUD ops | Create, update, delete, list, show and associate Floating IPs | VIM: Openstack Components: Keystone, Neutron | ||
| 6 | VIM operations on Network | Metering extensions CRD ops | List, Show, Create, Delete Metering labels List, Show, Create, Delete Metering labels rules | VIM: Openstack Components: Keystone, Neutron | ||
| 7 | VIM operations on Network | Networks and subnets CRUD ops | create a network for a project list project's networks show a project network details create a subnet for a project list project's subnets show a project subnet details network update subnet update delete a network also deletes its subnets list external networks All subnet tests are run once with ipv4 and once with ipv6.; | VIM: Openstack Components: Keystone, Neutron | ||
| 8 | VIM operations on Network | Ports CRUD ops | Create, delete, list, show and update ports | VIM: Openstack Components: Keystone, Neutron | ||
| 9 | VIM operations on Network | Routers CRUD ops | Create, delete, list, show and update routers; add and remove router interface | VIM: Openstack Components: Keystone, Neutron | ||
| 10 | VIM operations on Network | Security groups CRUD ops | Create, delete, list, show and update security groups;Create, delete, show security group rules; | VIM: Openstack Components: Keystone, Neutron | ||
| 11 | VIM operations on Network | List service providers | List service providers | VIM: Openstack Components: Keystone, Neutron | ||
| 12 | VIM operations on Network | Subnet pools CRUD ops | Create, delete, list, show and update subnet pools | VIM: Openstack Components: Keystone, Neutron | ||
| 13 | VIM operations on Network | List API versions | List API versions | VIM: Openstack Components: Keystone, Neutron | ||
| 14 | VIM operations on Network | Agents management | List, show and update agents | VIM: Openstack Components: Keystone, Neutron | ||
| 15 | VIM operations on Network | DHCP agent scheduler | List DHCP agents on hosting network; List networks hosted by one DHCP; Add and remove networks from DHCP agents | VIM: Openstack Components: Keystone, Neutron | ||
| 16 | VIM operations on Network | External network CRUD ops | Create, delete, list, show and update external networks | VIM: Openstack Components: Keystone, Neutron | ||
| 17 | VIM operations on Network | L3 agent scheduler | List routers that the given L3 agent is hosting. List L3 agents hosting the given router. Add and Remove Router to L3 agent | VIM: Openstack Components: Keystone, Neutron | ||
| 18 | VIM operations on Network | Quotas CRU ops | List, show, update, set, reset quotas | VIM: Openstack Components: Keystone, Neutron | ||
| 19 | VIM operations on Network | Router DVR | Create Distributed Virtual Routing (DVR) router; create a centralized router; update a centralized router to DVR router | VIM: Openstack Components: Keystone, Neutron |
VIM Operations on Volume
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27499
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VIM operations on Volume | List availability zones | List availability zones | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 2 | VIM operations on Volume | List extensions | List of all extensions | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 3 | VIM operations on Volume | Snapshot metadata CRUD ops | Create, show, update, delete metadata; Create, show, update, delete metadata items; | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 4 | VIM operations on Volume | Volume metadata CRUD ops | Create, show, update, delete volume metadata | VIM: Openstack Components: Keystone, Cinder | ||
| 5 | VIM operations on Volume | Volume transfer CRUD ops | Create, show, update, delete and list volume transfers | VIM: Openstack Components: Keystone, Cinder | ||
| 6 | VIM operations on Volume | Volume actions | Attach and deattach volume to an instance; Verify that a volume bootable flag is retrieved; Verify that a volume's attachment information is retrieved; upload volume; reserve and unreserve volumes; update volume; Reset volume status; force delete volumes | VIM: Openstack Components: Keystone, Cinder, Glance | ||
| 7 | VIM operations on Volume | Volume backup CRUD ops | Create, show, list, restore, delete volume backups;export and import volume backups; reset backup status | VIM: Openstack Components: Keystone, Cinder | ||
| 8 | VIM operations on Volume | Volumes clone | Creates a volume from another volume passing a size different from the source volume; Create a volume from the bootable volume | VIM: Openstack Components: Keystone, Cinder | ||
| 9 | VIM operations on Volume | Extend volume | Extend volume | VIM: Openstack Components: Keystone, Cinder | ||
| 10 | VIM operations on Volume | Volumes CRUD ops | Create, show, update, list and delete volumes | VIM: Openstack Components: Keystone, Cinder, Glance | ||
| 11 | VIM operations on Volume | Volumes snapshot CRUD ops | Create a snapshot when volume status is in-use; Create, show, list, update and delete snapshots; | VIM: Openstack Components: Keystone, Cinder | ||
| 12 | VIM operations on Volume | Image metadata RUD ops | Update and show, delete image metadata | VIM: Openstack Components: Keystone, Nova, Cinder, Glance | ||
| 13 | VIM operations on Volume | Create and show volume with Mutil backend | Show volume id which created by type without prefix;Show volume id which created by type wit prefix | VIM: Openstack Components: Keystone, Cinder | ||
| 14 | VIM operations on Volume | Qos specs CRUD ops | Create, list, delete, show, associate, disassociate, set/unset qos-specs | VIM: Openstack Components: Keystone, Cinder | ||
| 15 | VIM operations on Volume | Snapshot actions | Reset and update snapshot status; force delete snapshot; | VIM: Openstack Components: Keystone, Cinder | ||
| 16 | VIM operations on Volume | List hosts | List hosts | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 17 | VIM operations on Volume | Show volume pools | Show volume pools | VIM: Openstack Components: Keystone, Nova, Cinder | ||
| 18 | VIM operations on Volume | Volume quotas RUD ops | List, show,update and delete volume quotas | VIM: Openstack Components: Keystone, Cinder | ||
| 19 | VIM operations on Volume | List volume services | Volume service list requires admin privileges | VIM: Openstack Components: Keystone, Cinder | ||
| 20 | VIM operations on Volume | Create and update volume snapshot quotas | Create and update volume snapshot quotas | VIM: Openstack Components: Keystone, Cinder | ||
| 21 | VIM operations on Volume | Volume types extra specs CRUD ops | Create, show, delete, list and update volume types extra specs | VIM: Openstack Components: Keystone, Cinder | ||
| 22 | VIM operations on Volume | Volume types CRUD ops | Create, update, show, delete and list volume types | VIM: Openstack Components: Keystone, Cinder | ||
| 23 | VIM operations on Volume | User messages RD ops | List, show and delete user messsages | VIM: Openstack Components: Keystone, Cinder | ||
| 24 | VIM operations on Volume | List and show backends capabilities | List and show backends capabilities | VIM: Openstack Components: Keystone, Cinder | ||
| 25 | VIM operations on Volume | Add and list volume type access | Add and list volume type access | VIM: Openstack Components: Keystone, Cinder |
VIM Operations on Image
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27219
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VIM operations on Image | Images member CRUD ops | Create, show, delete, update, list images member | VIM: Openstack Components: Keystone, Glance | https://gerrit.opnfv.org/gerrit/#/c/27221 | |
| 2 | VIM operations on Image | Metadata definition Namespaces basic functionality | Create, show, update, delete and list metadata definition namespace | VIM: Openstack Components: Keystone, Glance | https://gerrit.opnfv.org/gerrit/#/c/27223 | |
| 3 | VIM operations on Image | Metadata definition resource types basic functionality | Create, show, update, delete and list metadata definition resource types | VIM: Openstack Components: Keystone, Glance | https://gerrit.opnfv.org/gerrit/#/c/27225 | |
| 4 | VIM operations on Image | Images CRUD ops | Register, show , delete, update and list images; upload and show the image file | VIM: Openstack Components: Keystone, Glance | https://gerrit.opnfv.org/gerrit/#/c/27227 | |
| 5 | VIM operations on Image | Create and delete image tags | Create and delete image tags | VIM: Openstack Components: Keystone, Glance | https://gerrit.opnfv.org/gerrit/#/c/27231 |
VIM Operations on Identity
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27501
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VIM operations on Identity | List extensions | List all the extensions | VIM: Openstack Components: Keystone | ||
| 2 | VIM operations on Identity | API discovery features | Show API version; show API media types; show API status | VIM: Openstack Components: Keystone | ||
| 3 | VIM operations on Identity | Projects CRUD ops | List, create, delete and update projects; associate a user to a project | VIM: Openstack Components: Keystone | ||
| 4 | VIM operations on Identity | Token CRUD ops | Create, show, delete, rescope token; Note: An unscoped token can be requested, that token can be used to request a scoped token. The scoped token can be revoked, and the original token used to get a token in a different project. | VIM: Openstack Components: Keystone | ||
| 5 | VIM operations on Identity | Update password | User update own password | VIM: Openstack Components: Keystone | ||
| 6 | VIM operations on Identity | User ec2 credentials CRD ops | Create, show, list and delete user ec2 credentials | VIM: Openstack Components: Keystone | ||
| 7 | VIM operations on Identity | Credentials CRUD ops | Create, show, update, delete and list credentials | VIM: Openstack Components: Keystone | ||
| 8 | VIM operations on Identity | Default project id | 1. create a domain 2. create a project in the domain 3. create a user in the domain, with the previous project as his default project 4. get roles and find the admin role 5. grant the admin role to the user on his project 6. create a new client with user's credentials 7. verify the user's token and see that it is scoped to the project | VIM: Openstack Components: Keystone | ||
| 9 | VIM operations on Identity | Domains CRUD ops | Create, update, delete and list domains | VIM: Openstack Components: Keystone | ||
| 10 | VIM operations on Identity | Endpoint CRUD ops | Create, update, delete, show and list endpoints | VIM: Openstack Components: Keystone | ||
| 11 | VIM operations on Identity | Groups CRUD ops | Create, update, delete, show and list groups; add, list and delete users in groups; | VIM: Openstack Components: Keystone | ||
| 12 | VIM operations on Identity | Inherits CRUD ops | Create, assign, list, check, revoke roles on domains user/domains group/projects user/projects group/domains/ project trees | VIM: Openstack Components: Keystone | ||
| 13 | VIM operations on Identity | Policy CRUD ops | List, create, update and delete policies | VIM: Openstack Components: Keystone | ||
| 14 | VIM operations on Identity | Regions CRUD ops | Create, show, update, delete and list regions | VIM: Openstack Components: Keystone | ||
| 15 | VIM operations on Identity | Roles CRUD ops | Create, show, update, delete and list roles; grant ,list and revoke role to user/group on project/domain/ | VIM: Openstack Components: Keystone | ||
| 16 | VIM operations on Identity | Services CRUD ops | Create, show, update, delete and list services | VIM: Openstack Components: Keystone | ||
| 17 | VIM operations on Identity | Trusts CRU ops | Create, show, delete and list trusts; updates are not supported for trusts | VIM: Openstack Components: Keystone | ||
| 18 | VIM operations on Identity | Users CRUD ops | Create, update, show, delete and list users; Request for token authentication with/without a valid token in header | VIM: Openstack Components: Keystone |
NFVI
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27503
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Network | vPing | two vms setup in the same subnet and can ping each other (ssh, userdata, IPv4, IPv6(afraid not support now MatthewLi)) | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 2 | Network | vRouter | Two VMs in two different subnets, connected by a Neutron virtual router, can ping each other | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 3 | Network | Security rules | Verify that a security rule prevents a type of traffic, remove rule, verify that traffic passes, add it back, test that traffic is stopped | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 4 | Network | External routing | Verify that a VM can access hosts external to the VIM | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 5 | Network | ...fill out CRUD operations for "network/subnet/router/port" operations | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | |||
| 6 | Network | Network advanced server ops | Check VM connectivity after some advanced instance operations executed: * Stop/Start an instance * Reboot an instance * Rebuild an instance * Pause/Unpause an instance * Suspend/Resume an instance * Resize an instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 7 | Network | Network basic operations | This smoke test suite assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify network connectivity as follows: There are presumed to be two types of networks: tenant and public. A tenant network may or may not be reachable from the Tempest host. A public network is assumed to be reachable from the Tempest host, and it should be possible to associate a public ('floating') IP address with a tenant ('fixed') IP address to facilitate external connectivity to a potentially unroutable tenant IP address. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 8 | Network | Network v6 | 1. Create network with subnets: 1.1. one IPv4 and 1.2. one or more IPv6 in a given address mode 2. Boot 2 VMs on this network 3. Allocate and assign 2 FIP4 4. Check that vNICs of all VMs gets all addresses actually assigned 5. Each VM will ping the other's v4 private address 6. If ping6 available in VM, each VM will ping all of the other's v6 addresses as well as the router's | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 9 | Network | Security groups basic ops | This test case assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify cross tenant connectivity as follows ssh: in order to overcome "ip namespace", each tenant has an "access point" VM with floating-ip open to incoming ssh connection allowing network commands (ping/ssh) to be executed from within the tenant-network-namespace Tempest host performs key-based authentication to the ssh server via floating IP address connectivity test is done by pinging destination server via source server ssh connection. success - ping returns failure - ping_timeout reached multi-node: Multi-Node mode is enabled when CONF.compute.min_compute_nodes > 1. Tests connectivity between servers on different compute nodes. When enabled, test will boot each new server to different compute nodes. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 10 | NFVI | Basic minimum scenario test | 1. Create image 2. Create keypair 3. Boot instance with keypair and get list of instances 4. Create volume and show list of volumes 5. Attach volume to instance and getlist of volumes 6. Add IP to instance 7. Create and add security group to instance 8. Check SSH connection to instance 9. Reboot instance 10. Check SSH connection to instance after reboot | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 11 | Compute | OPNFV_YARDSTICK_TC010 | Measure the memory read latency for varying memory sizes and strides. Whole memory hierarchy is measured including all levels of cache. | Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 12 | Compute | OPNFV_YARDSTICK_TC012 | Measure the rate at which data can be read from and written to the memory (this includes all levels of memory). | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 13 | Compute | OPNFV_YARDSTICK_TC014 | To evaluate the IaaS processing speed with regards to score of single cpu running and parallel running The purpose is also to be able to spot trends. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations. | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 14 | Compute | OPNFV_YARDSTICK_TC055 | To evaluate the IaaS compute capacity with regards to hardware specification, including number of cpus, number of cores, number of threads, available memory size and total cache size. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations. | Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 15 | Compute | Aggregates basic ops | Creates an aggregate within an availability zone 1. Adds a host to the aggregate 2. Checks aggregate details 3. Updates aggregate's name 4. Removes host from aggregate 5. Deletes aggregate | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 16 | Compute | Server advanced operations | This test case stresses some advanced server instance operations: * Resizing a volume-backed instance * Sequence suspend resume | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 17 | Compute | Server basic operations | This smoke test case follows this basic set of operations: * Create a keypair for use in launching an instance * Create a security group to control network access in instance * Add simple permissive rules to the security group * Launch an instance * Perform ssh to instance * Verify metadata service * Verify metadata on config_drive * Terminate the instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 18 | Compute | Server multinode | This is a set of tests specific to multinode testing | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 19 | Compute | Shelve instance | The following is the scenario outline: * boot an instance and create a timestamp file in it * shelve the instance * unshelve the instance * check the existence of the timestamp file in the unshelved instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 20 | Compute | Snapshot pattern | The following is the outline: * boot an instance and create a timestamp file in it * snapshot the instance * boot a second instance from the snapshot * check the existence of the timestamp file in the second instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 21 | Compute | Stamp pattern | This test is for snapshotting an instance/volume and attaching the volume created from snapshot to the instance booted from snapshot. The following is the scenario outline: 1. Boot an instance "instance1" 2. Create a volume "volume1" 3. Attach volume1 to instance1 4. Create a filesystem on volume1 5. Mount volume1 6. Create a file which timestamp is written in volume1 7. Unmount volume1 8. Detach volume1 from instance1 9. Get a snapshot "snapshot_from_volume" of volume1 10. Get a snapshot "snapshot_from_instance" of instance1 11. Boot an instance "instance2" from snapshot_from_instance 12. Create a volume "volume2" from snapshot_from_volume 13. Attach volume2 to instance2 14. Check the existence of a file which created at 6. in volume2 | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 22 | Compute | Volume boot pattern | This test case attempts to reproduce the following steps: * Create some bootable volume importing a image * Boot an instance from the bootable volume * Write content to the volume * Delete an instance and Boot a new instance from the volume * Check written content in the instance * Create a volume snapshot while the instance is running * Boot an additional instance from the new snapshot based volume * Check written content in the instance booted from snapshot | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 23 | Storage | yardstick_tc005 | Fio test is invoked in a host VM on a compute blade, a job file as well as parameters are passed to fio and fio will start doing what the job file tells it to do. | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 24 | Storage | Encrypted volumes | This test is for verifying the functionality of encrypted volumes. For both LUKS and cryptsetup encryption types, this test performs the following: * Creates an image * Boots an instance from the image * Creates an encryption type (as admin) * Creates a volume of that encryption type (as a regular user) * Attaches and detaches the encrypted volume to the instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder |
High Availability (HA)
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27505
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | HA | OPNFV_YARDSTICK_TC019 | This test case will verify the high availability of the service provided by compute on control node. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 2 | HA | OPNFV_YARDSTICK_TC045 | This test case will verify the high availability of the network service provided by network on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 3 | HA | OPNFV_YARDSTICK_TC046 | This test case will verify the high availability of the user service provided by Identity on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 4 | HA | OPNFV_YARDSTICK_TC047 | This test case will verify the high availability of the image service provided by image on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 5 | HA | OPNFV_YARDSTICK_TC048 | This test case will verify the high availability of the volume service provided by network on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 6 | HA | OPNFV_YARDSTICK_TC049 | This test case will verify the high availability of the storage service provided by object storage on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Additional Components: Swift | ||
| 7 | HA | OPNFV_YARDSTICK_TC050 | This test case will verify the high availability of control node. When one of the controller failed to connect the network, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 8 | HA | OPNFV_YARDSTICK_TC051 | This test case will verify the high availability of control node. When the CPU usage of a specified controller node is stressed to 100%, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 9 | HA | OPNFV_YARDSTICK_TC052 | This test case will verify the high availability of control node. When the disk I/O of a specified disk is blocked, which breaks down the Openstack services on this node. Read and write services should still be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 10 | HA | OPNFV_YARDSTICK_TC053 | This test case will verify the high availability of the load balance service(current is HAProxy) that supports OpenStack on controller node. When the load balance service of a specified controller node is killed, whether other load balancers on other controller nodes will work, and whether the controller node will restart the load balancer are checked | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 11 | HA | OPNFV_YARDSTICK_TC054 | This test case will verify the high availability for virtual ip in the environment. When master node of virtual ip is abnormally shutdown, connection to virtual ip and the services binded to the virtual IP it should be OK | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA |
IPv6
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27509
Link to the introduction of IPv6 test cases status: Ipv6 Test Cases Status
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | IPv6 | Test Case 1: Create and Delete an IPv6 Network, Port and Subnet | To test if Create and Delete an IPv6 Network, Port and Subnet is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 2 | IPv6 | Test Case 2: Create, Update and Delete an IPv6 Network and Subnet | To test if Create, Update and Delete an IPv6 Network and Subnet is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 3 | IPv6 | Test Case 3: Check External Network Visibility | To test if Check External Network Visibility is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 4 | IPv6 | Test Case 4: List IPv6 Networks and Subnets of a Tenant | To test if List IPv6 Networks and Subnets of a Tenant is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 5 | IPv6 | Test Case 5: Show Information of an IPv6 Network and Subnet | To test if Show Information of an IPv6 Network and Subnet is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 6 | IPv6 | Test Case 6: Create an IPv6 Port in Allowed Allocation Pools | To test if Create an IPv6 Port in Allowed Allocation Pools is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 7 | IPv6 | Test Case 7: Create an IPv6 Port without Security Groups | To test if Create an IPv6 Port without Security Groups is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 8 | IPv6 | Test Case 8: Create, Update and Delete an IPv6 Port | To test if Create, Update and Delete an IPv6 Port is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 9 | IPv6 | Test Case 9: List IPv6 Ports of a Tenant | To test if List IPv6 Ports of a Tenant is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 10 | IPv6 | Test Case 10: Show Information of an IPv6 Port | To test if Show Information of an IPv6 Port is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 11 | IPv6 | Test Case 11: Add Multiple Interfaces for an IPv6 Router | To test if Add Multiple Interfaces for an IPv6 Router is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 12 | IPv6 | Test Case 12: Add and Remove an IPv6 Router Interface with port_id | To test if Add and Remove an IPv6 Router Interface with port_id is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 13 | IPv6 | Test Case 13: Add and Remove an IPv6 Router Interface with subnet_id | To test if Add and Remove an IPv6 Router Interface with subnet_id is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 14 | IPv6 | Test Case 14: Create, Update, Delete, List and Show an IPv6 Router | To test if Create, Update, Delete, List and Show an IPv6 Router is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 15 | IPv6 | Test Case 15: Create, Update, Delete, List and Show an IPv6 Security Group | To test if Create, Update, Delete, List and Show an IPv6 Security Group is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 16 | IPv6 | Test Case 16: Create, Delete and Show Security Group Rules | To test if Create, Delete and Show Security Group Rules is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 17 | IPv6 | Test Case 17: List All Security Groups | To test if List All Security Groups is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 18 | IPv6 | Test Case 18: IPv6 Address Assignment - DHCPv6 Stateless | To test if IPv6 Address Assignment - DHCPv6 Stateless is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 19 | IPv6 | Test Case 19: IPv6 Address Assignment - Dual Stack, DHCPv6 Stateless | To test if IPv6 Address Assignment - Dual Stack, DHCPv6 Stateless is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 20 | IPv6 | Test Case 20: IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless | To test if IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 21 | IPv6 | Test Case 21: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless | To test if IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 22 | IPv6 | Test Case 22: IPv6 Address Assignment - SLAAC | To test if IPv6 Address Assignment - SLAAC is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 23 | IPv6 | Test Case 23: IPv6 Address Assignment - Dual Stack, SLAAC | To test if IPv6 Address Assignment - Dual Stack, SLAAC is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 24 | IPv6 | Test Case 24: IPv6 Address Assignment - Multiple Prefixes, SLAAC | To test if IPv6 Address Assignment - Multiple Prefixes, SLAAC is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 25 | IPv6 | Test Case 25: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, SLAAC | To test if IPv6 Address Assignment - Dual Stack, Multiple Prefixes, SLAAC is OK. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirements: ipv6 network configuration | ||
| 26 | IPv6 | Test Case 26: Service VM as an IPv6 vRouter | OPNFV_YARDSTICK_TC027 To do a basic verification that IPv6 connectivity is within acceptable boundaries when ipv6 packets travel between hosts located on same or different compute blades. | Colorado Scenario: nosdn, nofeature VIM: Openstack Components: Keystone, Nova, Neutron, Glance SDN: not support SDN |
VPN
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27511
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | VPN | VPN CRUD | 1. Create VPN instance and check command return code 2. Check if VPN instance is present 3. Create IETF VM interface and check return code 4. Verify IETF VM interface 5. Create VPN interface for IETF interface 6. Verify VPN interface 7. Verify FIB entry after create 8. Delete VM VPN interface 9. Verify after deleting VM VPN interface 10. Delete VPN instance 11. Verify after deleting VPN instance 12. Delete VM IETF interface 13. Verify after deleting VM IETF interface 14. Verify FIB entry after delete | |||
| 2 | VPN | BGP VPN CRUD | 1. Create BGPVPN 2. Create BGPVPN with malformatted route target (e.g. ASN:NN) should fail. 3. Create BGPVPN with invalid route target (e.g. 65536:0) should fail. 4. Getting the VPN list works without producing an error. 5. Updating an existing BGPVPN works. 6. Displaying parameters of an existing BGPVPN works. 7. Deleting a BGPVPN works. 8. Associating an existing BGPVPN with a Neutron network works 9. Getting the associated Neutron network works. 10. Deleting the network association works. 11. Associating an existing BGPVPN with a Neutron Router works 12. Getting the associated Neutron router works. 13. Deleting the router association works | |||
| 3 | VPN | Data Center Gateway integration | Data Center Gateway integration, check if routes are correctly exchanged between SDN controller and datacenter gateway | |||
| 4 | VPN | Inter-DC communication | Inter-DC communication, Check if communication between two data centers connected through an MPLS backbone works. | |||
| 5 | VPN | Subnet connectivity | VPN provides connectivity between subnets. different subnets cannot reach each other unless they are connected to the same Router. | Colorado Scenario: odl_l2, bgpvpn VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone SDN: ODL Feature: no feature needed | ||
| 6 | VPN | Tenant separation | VPNs enable using the same IP address ranges in different VPNs, which is an important feature for tenant separation in the DC. This test verifies if the correct VM is reached under a given IP address when IP addresses are used multiple times in the same DC. | Colorado Scenario: odl_l2, bgpvpn VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone SDN: ODL Feature: no feature needed | ||
| 7 | VPN | VPN provides connectivity between subnets using router association | VPN provides connectivity between subnets using router association | Colorado Scenario: odl_l2, bgpvpn VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone SDN: ODL Feature: no feature needed | ||
| 8 | VPN | A subnet with a router attached can be associated to a VPN with network association and given a floating IP | A subnet with a router attached can be associated to a VPN with network association and given a floating IP | Colorado Scenario: odl_l2, bgpvpn VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone SDN: ODL Feature: no feature needed |
Fault Management(Doctor)
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27513
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Fault management | Relaunch instance ? | When an instance fails, can the platform restart the instance automatically? | |||
| 2 | Fault management | Scale-up ? | Can a new instance of a given type be launched when a certain load is hit? | |||
| 3 | Fault management | Subscribe to a fault event | Listen for an instance fault , and verify that when the instance is killed we hear about it | |||
| 4 | Fault management | failure notification | To test VM failure notification time. Boot a VM and create an alarm event in Ceilometer using the consumer ip. Once Doctor components are started, a failure is injected to the system, then the consumer receives a failure notification. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Additional component: Cinder, Ceilometer, Aodh, Congress, Vitrage Feature: Doctor |
KVM
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27515
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Kvm | Timer Test without workload | Measure system high resolution by using Cyclictest, getting latencies as results. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance Special Requirments: 1. BIOS setup 2. kernel configuration 3. run time environment setup refer to https://wiki.opnfv.org/display/kvm/KVM4NFV+Test++Environment |
Parser
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27517
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Parser | verify the function of Yang-to-Tosca in Parser | Offers the capability to translate yang based scriptors to tosca formate templates | VIM: Openstack not necessary, run in jumpserver Feature: no feature needed | ||
| 2 | Parser | verify the function of Policy-to-Tosca in Parser | Enables policy related fields in tosca could be translated into heat orchestration template correctly | VIM: Openstack not necessary, run in jumpserver Feature: no feature needed | ||
| 3 | Parser | verify the function of Tosca-to-Heat in Parser | Heat-translator call class of ToscaTemplate in tosca-parser firstly to validate and parse input yaml file, then tranlate the file into hot file | VIM: Openstack not necessary, run in jumpserver Feature: no feature needed |
virtual Traffic Classifier
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27519
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | References |
|---|---|---|---|---|---|---|
| 1 | VTC | opnfv_yardstick_tc020 | To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Additional Components: Heat Special: Hardware: 1. physical node hosting the packet generator must have 2 NICs which are DPDK compatible 2. The switch used must support multicast traffic and IGMP snooping 3. The corresponding ports to which the cables are connected need to be configured as VLAN trunks using two of the VLAN IDs available for Neutron. Software: vtc, DPDK_pktgen, python-dev, python-pip, python-mock, tcpreplay, libpcap-dev, smcroute | ||
| 2 | VTC | opnfv_yardstick_tc021 | To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure in the presence of noisy neighbours. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Additional Components: Heat Special: Hardware: 1. physical node hosting the packet generator must have 2 NICs which are DPDK compatible 2. The switch used must support multicast traffic and IGMP snooping 3. The corresponding ports to which the cables are connected need to be configured as VLAN trunks using two of the VLAN IDs available for Neutron. Software: vtc, DPDK_pktgen, python-dev, python-pip, python-mock, tcpreplay, libpcap-dev, smcroute |
Copper
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27521
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Copper | DMZ Placement | Pause VMs running in a DMZ zone, if the image running is not allowed to run in that zone | VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone Additional Components: Ceilometer Feature: no feature needed | ||
| 2 | Copper | SMTP Ingress | Identify VMs that have TCP port 25 open for ingress | VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone Additional Components: Ceilometer Feature: no feature needed | ||
| 3 | Copper | Reserved Subnet | Delete any subnets created in restricted network ranges | VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone Additional Components: Ceilometer Feature: no feature needed |
Promise
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27523
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Promise | Capacity Management | Including add providers, update the capacity, and query the current capacity and utilization of a provider | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Feature: no feature needed | ||
| 2 | Promise | Reservation Management | Allows a Consumer to request reservations for resource capacity (create, update, cancel, query reservations) | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Feature: no feature needed | ||
| 3 | Promise | Allocation Management | Create an instance of specified resource(s) for immediate use utilizing capacity from the pool and destroy an existing server instance and release it back to the pool | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Feature: no feature needed |
Multisite
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27525
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Multisite | Quota Management API | This testcase includes 9 functions: 1. Update global limit for a tenant 2. Get global limit for a tenant 3. A tenant can also get the global limit by himself 4. Get defaults limits 5. Get total usage for a tenant 6. A tenant can also get the total usage by himself 7. On demand quota sync 8. Delete specific global limit for a tenant 9. Delete all kingbird global limit for a tenant | Colorado Scenario: os-nosdn-multisite-ha VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance, ceilometer SDN: not support SDN Feature: no feature needed | ||
| 2 | Multisite | Quota Class API | This testcase includes 3 functions: 1. Update default quota class 2. Get default quota class 3. Delete default quota class | Colorado Scenario:os-nosdn-multisite-ha VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance, ceilometer SDN: not support SDN Feature: no feature needed |
SDN controller(ODL)
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27533
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | ODL | restconf modules | verify Restconf is OK | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 2 | ODL | neutron reachability | Get the complete list of networks Get the complete list of subnets Get the complete list of ports | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 3 | ODL | neutron networks | Checking Network created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 4 | ODL | neutron subnets | Checking Subnets created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 5 | ODL | neutron ports | Checking Port created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 6 | ODL | neutron delete ports | Checking Port deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 7 | ODL | neutron delete subnets | Checking Subnets deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 8 | ODL | neutron delete networks | Checking Network deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed |
SDN controller(ONOS)
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27535
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | ONOS | Virtual Network NBI Test -Network Post | Belonged to tier FUNCvirNetNB, this is create Network northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Data via HTTP, 3) Get Data via HTTP, 4) Compare Send Id and Get Id | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 2 | ONOS | Network Update | Belonged to tier FUNCvirNetNB, this is update Network northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Data via HTTP, 3) Update Data via HTTP, 4) Get Data via HTTP, 5) Compare Update data | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 3 | ONOS | Network Delete | Belonged to tier FUNCvirNetNB, this is delete Network northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Data via HTTP, 3) Delete Data via HTTP, 4) Get Data is NULL | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 4 | ONOS | Subnet Post | Belonged to tier FUNCvirNetNB, this is create Subnet northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP, 4) Get Subnet Data via HTTP, 5) Compare Post Subnet Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 5 | ONOS | Subnet Update | Belonged to tier FUNCvirNetNB, this is update Subnet northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP, 4) Update Subnet Data via HTTP, 5) Get Subnet Data via HTTP, 6) Compare Subnet Data, 7) Delete Subnet via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 6 | ONOS | Subnet Delete | Belonged to tier FUNCvirNetNB, this is delete Subnet northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP, 4) Delete Subnet Data via HTTP, 5) Get Subnet Data is NULL | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 7 | ONOS | Port Post | Belonged to tier FUNCvirNetNB, this is create Virtualport northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP(Post port need post subnet), 4) Post Port Data via HTTP, 5) Get Port Data via HTTP, 6) Compare Post Port Data, 7) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 8 | ONOS | Port Update | Belonged to tier FUNCvirNetNB, this is update Virtualport northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP(Post port need post subnet), 4) Post Port Data via HTTP, 5) Update Port Data via HTTP, 6) Get Port Data via HTTP, 7) Compare Post Port Data, 8) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 9 | ONOS | Port Delete | Belonged to tier FUNCvirNetNB, this is delete Virtualport northbound test, in order to determine if North bound can handle the request, including steps: 1) Generate Post Data, 2) Post Network Data via HTTP(Post Subnet need post network), 3) Post Subnet Data via HTTP(Post port need post subnet), 4) Post Port Data via HTTP, 5) Delete Port Data via HTTP, 6) Get Port Data via HTTP, 7) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 10 | ONOS | Router Post | Belonged to tier FUNCvirNetNBL3, this is create Router northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post Router need post network), 2) Post Router Data via HTTP, 3) Get Router Data via HTTP, 4) Compare Post Router Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 11 | ONOS | Router Update | Belonged to tier FUNCvirNetNBL3, this is update Router northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post Router need post network), 2) Post Router Data via HTTP, 3) Update Router Data via HTTP, 4) Get Router Data via HTTP, 5) Compare Router Data, 6) Delete Router via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 12 | ONOS | Router Delete | Belonged to tier FUNCvirNetNBL3, this is delete Router northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post Router need post network), 2) Post Router Data via HTTP, 3) Delete Router Data via HTTP, 4) Get Router Data is NULL | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 13 | ONOS | RouterInterface Post | Belonged to tier FUNCvirNetNBL3, this is create RouterInterface northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network), 2) Post Subnet Data via HTTP(Post port need post subnet), 3) Post Port Data via HTTP, 4) Post Router Data via HTTP, 5) Put RouterInterface Data via HTTP, 6) Get RouterInterface Data via HTTP, 7) Compare Post Port Data, 8) Del RouterInterface Data via HTTP, 9) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 14 | ONOS | RouterInterface Delete | Belonged to tier FUNCvirNetNBL3, this is delete RouterInterface northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network), 2) Post Subnet Data via HTTP(Post port need post subnet), 3) Post Port Data via HTTP, 4) Post Router Data via HTTP, 5) Post RouterInterface Data via HTTP, 6) Del RouterInterface Data via HTTP, 7) Delete Port Data via HTTP, 8) Get Port Data is NULL, 9) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 15 | ONOS | FloatingIp Post | Belonged to tier FUNCvirNetNBL3, this is create Floatingip northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Get Port Data via HTTP 6) Post FloatingIp Data via HTTP 7) Get Port Data via HTTP 8) Get FloatingIp Data via HTTP 9) Get FloatingIp Data via HTTP 10) Compare Post FloatingIp Data 11) Post FloatingIp Clean Data via HTTP 12) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 16 | ONOS | FloatingIp Update | Belonged to tier FUNCvirNetNBL3, this is updateFloatingip northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Post FloatingIp Data via HTTP 6) Post Delete Data via HTTP 7) Post NewPort Data via HTTP 8) Post NewFloatingIp Data via HTTP 9) Get NewFloatingIp Data via HTTP 10) Compare Post FloatingIp Data 11) Post FloatingIp Clean Data via HTTP 12) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 17 | ONOS | FloatingIp Delete | Belonged to tier FUNCvirNetNBL3, this is delete Floatingip northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Post FloatingIp Data via HTTP 6) Post FloatingIp Clean Data via HTTP 7) Get FloatingIp Data is NULL 8) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 18 | ONOS | Gateway Post | Belonged to tier FUNCvirNetNBL3, this is create Gateway northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Get Gateway Data via HTTP 6) Compare Post Gateway Data 7) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 19 | ONOS | Gateway Update | Belonged to tier FUNCvirNetNBL3, this is update Gateway northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Post New Router Data via HTTP 6) Get Gateway Data via HTTP 7) Compare Post Gateway Data 8) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed | ||
| 20 | ONOS | Gateway Delete | Belonged to tier FUNCvirNetNBL3, this is delete Gateway northbound test, in order to determine if North bound can handle the request, including steps: 1) Post Network Data via HTTP(Post port need post network) 2) Post Subnet Data via HTTP(Post port need post subnet) 3) Post Port Data via HTTP 4) Post Router Data via HTTP 5) Post Del Gateway Data via HTTP 6) Get Gateway Data via HTTP 7) If Gateway Data is NULL 8) Clean Data via HTTP | Colorado Scenario: ONOS VIM: Openstack Components: Neutron, ONOSrest Feature: no feature needed |
Open source VNF running on NFVI
The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27539
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Open Source VNF | cloudify_ims | This test case deploys an OpenSource vIMS solution from Clearwater using the Cloudify orchestrator. It also runs some signaling traffic. | VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance Special Requirements: Cloudify |
4 Comments
Unknown User (rprakash)
Based on today's Dovetail meeting tried summarize the category Basic VIM Operations (vim-ops) summary to carry forward to link to test cases. Have not removed the details as listed by vim-ops details and we will keep revising till we get a relation between test design, testcases and tie them back to gerrit for test-sutie and test cases. So please feel free to continue adding test cases while we work with IPv6 test templates to co-relate the two. Like Mathew to review and correct the summary. Thanks Prakash
Unknown User (matthewli)
hi Prakash, looks good to see progress. Actually, I don't know the test case selection criteria, searched in Obsolete: high level topics (to be removed), however, it's empty, anyway, if it's brain storm here, it's good.
from test case technical area, some experience are given here. 1) for vimops, if the sub-testcases of each test case can be found out in tempest repo(further, the version is very important, different version tag is suitable to different openstack versions, Mitaka, Newton, etc, test case can be changed as the version goes on), maybe a useful reference is defcore, such as http://git.openstack.org/cgit/openstack/defcore/tree/2016.08.json I wound't say that we are the same, it can be good input and reference added by OPNFV requirement. 2) for ipv6, it seems we are at a right way 3) for networking/compute/storage, also HA, functest/yardstick can provide test cases here and we have people who are familiar with them, some gaps are needed, such as ipv6 vPing
Unknown User (r-mibu)
Please, read carefully doctor requirement docs. Test cases in "Fault Management - test cases" seems not aligned with doctor project's objectives/deliverable. Or, may I edit this page?
Unknown User (hongbotian)
sure.