Page tree
Skip to end of metadata
Go to start of metadata


Content

JIRA issues of CI (with label "yardstick_ci")

Key Summary P Assignee Reporter Status
Loading...
Refresh

About

This wiki lists the Yardstick CI test cases candidates for Fraser. It works for yardstick team, installers team, scenario owners as the dashboard of Yardstick CI status. It'll be updated every two days. And CI-related JIRA issues can be tracked from here.

Recent CI status

APEX

os-nosdn-nofeature-ha

+=======================+====================+==========+========+======+
|       TESTCASE        |       DATE         | POD_NAME | RESULT | JIRA |
+=======================+====================+==========+========+======+
| opnfv_yardstick_tc002 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc076 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc079 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc082 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc083 | 2018-3-21 16:20:19 | lf-pod1  |  PASS  |  -   |
+=======================+====================+==========+========+======+

os-nosdn-nofeature-noha

+=======================+====================+==========+========+======+
|       TESTCASE        |       DATE         | POD_NAME | RESULT | JIRA |
+=======================+====================+==========+========+======+
| opnfv_yardstick_tc002 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-21 22:22:07 | lf-pod1  |  PASS  |  -   |
+=======================+====================+==========+========+======+


COMPASS

os-nosdn-nofeature-ha

+=======================+====================+=============+========+======+
|       TESTCASE        |        DATE        |  POD_NAME   | RESULT | JIRA |
+=======================+====================+=============+========+======+
| opnfv_yardstick_tc002 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc076 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc079 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc082 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc083 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc043 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc055 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc063 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc073 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc075 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |  -   |
| opnfv_yardstick_tc045 | 2018-3-27 23:59:58 | huawei-pod1 |  FAIL  |      |
| opnfv_yardstick_tc046 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc047 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc048 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc049 | 2018-3-27 23:59:58 | huawei-pod1 |  FAIL  |      |
| opnfv_yardstick_tc050 | 2018-3-27 23:59:58 | huawei-pod1 |  FAIL  |      |
| opnfv_yardstick_tc051 | 2018-3-27 23:59:58 | huawei-pod1 |  FAIL  |      |
| opnfv_yardstick_tc052 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc053 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc019 | 2018-3-27 23:59:58 | huawei-pod1 |  PASS  |      |
| opnfv_yardstick_tc025 | 2018-3-27 23:59:58 | huawei-pod1 |  FAIL  |      |
+=======================+====================+=============+========+======+

DAISY

os-nosdn-nofeature-ha

+=======================+====================+==========+========+======+
|       TESTCASE        |       DATE         | POD_NAME | RESULT | JIRA |
+=======================+====================+==========+========+======+
| opnfv_yardstick_tc002 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc076 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc079 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc082 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
| opnfv_yardstick_tc083 | 2018-3-21 18:04:53 | zte-pod2 |  PASS  |  -   |
+=======================+====================+==========+========+======+

FUEL(amd64)

os-nosdn-nofeature-ha

+=======================+====================+==========+========+================+
|       TESTCASE        |        DATE        | POD_NAME | RESULT |      JIRA      |
+=======================+====================+==========+========+================+
| opnfv_yardstick_tc002 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc005 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc010 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc011 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc012 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc014 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc037 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc069 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc070 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc071 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc072 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc076 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc079 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc082 | 2018-3-21 22:24:47 | lf-pod2  | FAIL   | YARDSTICK-1094 |
| opnfv_yardstick_tc083 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc046 | 2018-3-21 22:24:47 | lf-pod2  | FAIL   |       -        |
| opnfv_yardstick_tc049 | 2018-3-21 22:24:47 | lf-pod2  | FAIL   |       -        |
| opnfv_yardstick_tc050 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc052 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
| opnfv_yardstick_tc053 | 2018-3-21 22:24:47 | lf-pod2  | PASS   |       -        |
+=======================+====================+==========+========+================+

FUEL(arm64)

os-nosdn-nofeature-ha

+=======================+====================+==========+========+======+
|       TESTCASE        |        DATE        | POD_NAME | RESULT | JIRA |
+=======================+====================+==========+========+======+
| opnfv_yardstick_tc002 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc005 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc010 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc011 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc012 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc014 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc037 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc069 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc070 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc071 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc072 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc076 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc079 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc082 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
| opnfv_yardstick_tc083 | 2018-3-29 5:15:29  | arm-pod6 | PASS   |  -   |
+=======================+====================+==========+========+======+

JOID

os-nosdn-nofeature-ha

+=======================+===================+=============+========+======+
|       TESTCASE        |       DATE        |  POD_NAME   | RESULT | JIRA |
+=======================+===================+=============+========+======+
| opnfv_yardstick_tc002 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc076 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc079 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc082 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc083 | 2018-3-20 5:32:03 | intel-pod18 |  PASS  |  -   |
+=======================+===================+=============+========+======+

os-nosdn-nofeature-noha

+=======================+===================+=============+========+======+
|       TESTCASE        |       DATE        |  POD_NAME   | RESULT | JIRA |
+=======================+===================+=============+========+======+
| opnfv_yardstick_tc002 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc005 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc010 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc011 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc012 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc014 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc037 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc069 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc070 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc071 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
| opnfv_yardstick_tc072 | 2018-3-20 6:50:37 | intel-pod18 |  PASS  |  -   |
+=======================+===================+=============+========+======+

Fraser Release Gating Test Cases Candidates

Generic test cases (cover Compute/Storage/Network) in os-nosdn-nofeature-ha scenario

### without pod.yaml

+=======================+==========+======================================================================+
|       TESTCASE        | CATEGORY |                             DESCRIPTION                              |
+=======================+==========+======================================================================+
| opnfv_yardstick_tc002 | Network  | Evaluation of network latency between two VM instances.              |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc005 | Storage  | Measure Storage IOPS, throughput and latency using fio.              |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc010 | Compute  | Measure memory read latency using lmbench.                           |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc011 | Network  | Measure packet delay variation (jitter) using iperf3.                |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc012 | Compute  | Measure memory read and write bandwidth using lmbench.               |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc014 | Compute  | Measure CPU Processing speed using Unixbench.                        |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc037 | Network, | Evaluate the IaaS compute capacity and network performance with      |
|                       | Compute  | regards to CPU utilization, packet flows and network throughput.     |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc069 | Compute  | Measure memory read and write bandwidth using ramspeed.              |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc070 | Compute  | Measure network throughput and packet loss using pktgen.             |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc071 | Compute  | Measure cache hit/miss ratio and usage, network throughput and late. |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc072 | Network  | Measure network throughput and packet loss using pktgen.             |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc076 | Network  | Monitor error rate of IP datagram, ICMP message, TCP/UDP segment.    |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc079 | Storage  | Measure storage and file system performance using bonnie++.          |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc082 | Compute  | Measure various software perf events by perf.                        |
+-----------------------+----------+----------------------------------------------------------------------+
| opnfv_yardstick_tc083 | Network  | Measure throughput per VM using netperf.                             |
+=======================+==========+======================================================================+

### with pod.yaml

+=======================+==========+=====================================================================================+
|       TESTCASE        | CATEGORY |                                     DESCRIPTION                                     |
+=======================+==========+=====================================================================================+
| opnfv_yardstick_tc043 | Network  | Measure latency between NFVI nodes using ping.                                      |
+-----------------------+----------+-------------------------------------------------------------------------------------+
| opnfv_yardstick_tc055 | Compute  | Collect hardware specification of a pod from /proc/cpuinfo /proc/meminfo.           |
+-----------------------+----------+-------------------------------------------------------------------------------------+
| opnfv_yardstick_tc063 | Storage  | Measure disk size, block size and disk utilization of a pod using fdisk and iostat. |
+-----------------------+----------+-------------------------------------------------------------------------------------+
| opnfv_yardstick_tc075 | Network  | Measure network capacity and scale of a node.                                       |
+=======================+==========+=====================================================================================+

HA test cases already in Dovetail scope as ha.tc001 ~ ha.tc008

### with pod.yaml

+=======================+==========+===========================================================+
|       TESTCASE        | CATEGORY |                       DESCRIPTION                         |
+=======================+==========+===========================================================+
| opnfv_yardstick_tc019 |    HA    | Openstack Controller node nova-api service failure.       |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc045 |    HA    | Openstack Controller node neutron-server service failure. |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc046 |    HA    | Openstack Controller node keystone service failure.       |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc047 |    HA    | Openstack Controller node glance-api service failure.     |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc048 |    HA    | Openstack Controller node cinder-api service failure.     |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc051 |    HA    | Openstack Controller node CPU overload.                   |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc052 |    HA    | Openstack Controller node disk I/O overload.              |
+-----------------------+----------+-----------------------------------------------------------+
| opnfv_yardstick_tc053 |    HA    | Openstack Controller node load balance service failure.   |
+=======================+==========+===========================================================+

K8s test case in CI k8 scenario

+=======================+==========+==============================================================+
|       TESTCASE        | CATEGORY |                       DESCRIPTION                            |
+=======================+==========+==============================================================+
| opnfv_yardstick_tc080 |    k8    | Measure network latency between containers in k8s using ping |
+-----------------------+----------+--------------------------------------------------------------+
  • No labels