The table below captures the current status of functest integration. Please remove the issue from the table when it is resolved,
| Test suite | Test case | Issue | Resolution |
|---|---|---|---|
| Healthcheck | |||
| airship-opnfv-functest-healthcheck-latest-connection_check-run | |||
| airship-opnfv-functest-healthcheck-latest-tenantnetwork1-run | |||
| airship-opnfv-functest-healthcheck-latest-tenantnetwork2-run | |||
| airship-opnfv-functest-healthcheck-latest-vmready1-run | |||
| airship-opnfv-functest-healthcheck-latest-vmready2-run | |||
| airship-opnfv-functest-healthcheck-latest-singlevm1-run | |||
| airship-opnfv-functest-healthcheck-latest-singlevm2-run | |||
| airship-opnfv-functest-healthcheck-latest-vping_ssh-run | |||
| airship-opnfv-functest-healthcheck-latest-vping_userdata-run | |||
| vping_userdata-run | Upstream bug: AIRSHIP-95. Nova instance console log api returns emptry | ||
| airship-opnfv-functest-healthcheck-latest-cinder_test-run | |||
| airship-opnfv-functest-healthcheck-latest-odl-run | |||
| airship-opnfv-functest-healthcheck-latest-tempest_smoke-run | |||
| airship-opnfv-functest-healthcheck-latest-tempest_horizon-run | |||
| Smoke | |||
| airship-opnfv-functest-smoke-latest-tempest_neutron-run | |||
| neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_dhcp_port_status_active | Requires neutron extensions not supported in Airship DPDK | ||
| neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcp | same | ||
| neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_net | same | ||
| neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_external_update_policy_from_wildcard_to_specific_tenant | same | ||
| neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_allocate_floatingip | same | ||
| neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_attach_ext_gw | same | ||
| neutron_tempest_plugin.api.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrange | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_floatingip_with_timestamp | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_router_with_timestamp | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_floatingip_attribute_with_timestamp | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_router_attribute_with_timestamp | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_floatingip_with_timestamp | same | ||
| neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_router_with_timestamp | same | ||
| QOS test cases | Upstream: QOS support in DPDK | ||
| neutron_tempest_plugin.api.test_security_groups.RbacSharedSecurityGroupTest.test_regular_client_shares_to_another_regular_client | TBD | ||
| airship-opnfv-functest-smoke-latest-tempest_cinder-run | |||
| cinder_tempest_plugin.api.volume.test_volume_revert.VolumeRevertTests.test_volume_revert_to_snapshot | AIRSHIP-96: upstream bug related to kernel 4.15 | ||
| airship-opnfv-functest-smoke-latest-tempest_keystone-run | |||
| airship-opnfv-functest-smoke-latest-refstack_platform-run | skipped | ||
| airship-opnfv-functest-smoke-latest-tempest_full-run | |||
| tempest.api.compute.admin.test_instance_usage_audit_log_negative.InstanceUsageAuditLogNegativeTestJSON.test_instance_usage_audit_logs_with_nonadmin_user | AIRSHIP-104: Airship rbac policy inconsistent with tempest default | https://review.opendev.org/c/airship/treasuremap/+/767395 | |
| empest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration | 1289064: upstream: live migration is broken in numa. Needs Train | ||
| tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration_paused | same | ||
| tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration | same | ||
| tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration_paused | same | ||
| tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_list_migrations_in_flavor_resize_situation | TBD | ||
| tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_resize_server_revert_deleted_flavor | TBD | ||
| tempest.api.compute.admin.test_simple_tenant_usage.TenantUsagesTestJSON.test_get_usage_tenant_with_non_admin_user | AIRSHIP-105: Airship rbac policy inconsistent with tempest default | https://review.opendev.org/c/airship/treasuremap/+/767395 | |
| tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port | fail 36.011 | ||
| tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers | fail 89.588 | ||
| tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_duplicate_network_nics | fail 126.883 | ||
| tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_multiple_nics_order | fail 125.284 | ||
| tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_verify_resize_state | TBD | ||
| tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment | TBD | ||
| tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_auto_to_manual | TBD | ||
| tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_manual_to_auto | TBD | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output | Upstream bug: AIRSHIP-95. Nova instance console log api returns emptry | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_server_id_in_shutoff_status | same | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_with_unlimited_size | same | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm | TBD | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped | TBD | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert | TBD | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert_with_volume_attached | TBD | ||
| tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm | TBD | ||
| tempest.api.identity.admin.v3.test_application_credentials.ApplicationCredentialsV3AdminTest.test_create_application_credential_with_roles | AIRSHIP-106: Airship rbac policy inconsistent with tempest default | ||
| tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_list_groups | AIRSHIP-107: possible rbac? | ||
| tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_usersz | AIRSHIP-108: possible rbac | ||
| tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_users_with_name | same | ||
| tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeStaticTestJSON.test_list_projects_by_unauthorized_user | AIRSHIP-109: airship rbac policy is different than tempest default | ||
| tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential | AIRSHIP-110: airship rbac policy is different than tempest default | https://review.opendev.org/c/airship/treasuremap/+/767395 | |
| tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential_expires | same | ||
| tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_list_application_credentials | same | ||
| tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_query_application_credentials | same | ||
| tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_password_history_check_self_service_api | TBD: possible rbac | ||
| tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_account_lockout | TBD: possible rbac | ||
| tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_update_own_password | TBD: possible rbac | ||
| tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_deactivate_reactivate_image | TBD: possible rbac | ||
| tempest.api.network.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcp | FUNCTEST-1023: L3 test cases need to be skipped based on Neutron extension list or blacklisted | ||
| tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_list_external_networks | same | ||
| tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_show_external_networks_attribute | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_duplicate | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrange | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_router | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_64_subnets | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_invalid_options | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_eui64 | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_ra | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_ra_no_dhcp | same | ||
| tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_two_subnets | same | ||
| tempest.api.network.test_ports.PortsIpV6TestJSON.test_port_list_filter_by_router_id | same | ||
| tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_id | same | ||
| tempest.api.volume.admin.test_volume_manage.VolumeManageAdminTest.test_unmanage_manage_volume | TBD | ||
| tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_from_deactivated_image | AIRSHIP-112: possible rbac? | ||
| airship-opnfv-functest-smoke-latest-tempest_heat-run | |||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_port | FUNCTEST_1022: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list. | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_subnet_fixed_ip_sec_group | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_update_server_with_subnet | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_network | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_same_fixed_ip | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_subnet | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_multi_networks_swaps | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_network_port | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_network_subnet | same | ||
| heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_subnet_network | same | ||
| heat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_raw | same | ||
| heat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_software_config | same | ||
| heat_tempest_plugin.tests.functional.test_os_wait_condition.OSWaitCondition.test_create_stack_with_multi_signal_waitcondition | TBD | ||
| heat_tempest_plugin.tests.scenario.test_base_resources.BasicResourcesTest.test_base_resources_integration | TBD | ||
| heat_tempest_plugin.tests.scenario.test_remote_deeply_nested.RemoteDeeplyNestedStackTest.test_remote_nested | TBD | ||
| airship-opnfv-functest-smoke-latest-tempest_scenario-run | |||
| tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops | Upstream bug: AIRSHIP-103. Nova metaservice url not reachable from guest vm | ||
| airship-opnfv-functest-smoke-latest-tempest_telemetry-run | skipped | ||
| airship-opnfv-functest-smoke-latest-tempest_slow-run | |||
| tempest.scenario.test_volume_backup_restore.TestVolumeBackupRestore.test_volume_backup_restore | FUNCTEST-1019: teste volume backup depends on floating ip | ||
| tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devices | FUNCTEST-1020: test device tagging still uses tenant network | ||
| airship-opnfv-functest-smoke-latest-rally_sanity-run | |||
| NovaServers.boot_and_live_migrate_server | No support from OpenStack Helm. Recommend to blacklist. | ||
| airship-opnfv-functest-smoke-latest-patrole-run | |||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip | FUNCTEST-1021: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list. | ||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip_floatingip_address | same | ||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_delete_floating_ip | same | ||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_list_floating_ips | same | ||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_show_floating_ip | same | ||
| patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_update_floating_ip | same | ||
| patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_is_default | same | ||
| airship-opnfv-functest-smoke-latest-refstack_compute-run | skipped | ||
| airship-opnfv-functest-smoke-latest-refstack_object-run | |||
| airship-opnfv-functest-smoke-latest-networking-sfc-run | skipped | ||
| airship-opnfv-functest-smoke-latest-networking-bgpvpn-run | skipped | ||
| airship-opnfv-functest-smoke-latest-tempest_barbican-run | skipped | ||
| airship-opnfv-functest-smoke-latest-tempest_octavia-run | skipped | ||
| Benchmark | |||
| airship-opnfv-functest-benchmarking-latest-rally_full-run | |||
| NeutronNetworks.create_and_show_ports | needs blacklist. It needs router extension. | ||
| NovaServers.resize_server | SLA. Retest on pod 15. pod17 too small. | ||
| GlanceImages.create_and_deactivate_image | SLA. Retest on pod 15. pod17 too small. | ||
| airship-opnfv-functest-benchmarking-latest-rally_jobs-run | skipped | ||
| airship-opnfv-functest-benchmarking-latest-vmtp-run | skipped | ||
| airship-opnfv-functest-benchmarking-latest-shaker-run | skipped | ||
| VNF | |||
| airship-opnfv-functest-vnf-latest-cloudify-run | FUNCTEST-1017: cloudify vpn deploy uses floating ip. Shoudl change to use "public" network or skip. | ||
| airship-opnfv-functest-vnf-latest-cloudify_ims-run | skipped | ||
| airship-opnfv-functest-vnf-latest-heat_ims-run | skipped | ||
| airship-opnfv-functest-vnf-latest-vyos_vrouter-run | skipped | ||
| airship-opnfv-functest-vnf-latest-juju_epc-run | skipped |