Page tree
Skip to end of metadata
Go to start of metadata

Please note that this page is a draft page aiming to prepare the Euphrates release...(image src:

Key Summary T Created Reporter Status
YARDSTICK-895 Enable iommu support in grub New Feature Dec 14, 2017 Janx Malanik Open
YARDSTICK-893 Fix proxy env handling and ansible multinode support Bug Dec 14, 2017 Janx Malanik Open
YARDSTICK-872 Remove tool provisioning in PROX helper Bug Nov 28, 2017 Rodolfo Alonso Hernandez Open
YARDSTICK-877 Add separate builders for Yardstick project jobs Bug Dec 04, 2017 Rodolfo Alonso Hernandez Open
YARDSTICK-882 bottlenecks VNF scale-out scale-up Improvement Dec 05, 2017 Ace Lee Open
YARDSTICK-879 Support NSB image build(ansible) Improvement Dec 04, 2017 Ace Lee Open
YARDSTICK-857 tc006 failed due to volume attached to different location "/dev/vdc" Bug Nov 21, 2017 Rex Lee Open
YARDSTICK-842 load image fails when there's cirros image exist Bug Nov 13, 2017 Rex Lee Open
YARDSTICK-844 yardstick do not use EXTERNAL_NETWORK to create heat stack Bug Nov 14, 2017 Rex Lee Open
YARDSTICK-845 tc005 default use local disk Bug Nov 14, 2017 Rex Lee Open
YARDSTICK-849 kill process do not accurately kill the process like "nova-api" Bug Nov 15, 2017 Rex Lee Open
YARDSTICK-868 Cannot SSH into Heat instances when scaling Bug Nov 27, 2017 Ross Brattain Open
YARDSTICK-875 Check if multiverse repository is available in Ubuntu Bug Nov 30, 2017 Rodolfo Alonso Hernandez Open
YARDSTICK-874 Specify supported architecture for Ubuntu backports repository Bug Nov 30, 2017 Rodolfo Alonso Hernandez Open
YARDSTICK-852 tc078 cpu2006 fails in some situation Bug Nov 16, 2017 Rex Lee Open
YARDSTICK-867 testcase tc078 have no data stored or dashboard to show results Bug Nov 24, 2017 Rex Lee Open
YARDSTICK-854 yardstick docker lack of trex_client Bug Nov 20, 2017 Ace Lee Open
YARDSTICK-850 tc023 miss description and tc050-58 wrong description Bug Nov 15, 2017 Rex Lee Open
YARDSTICK-847 tc006, tc079, tc082 miss grafana dashboard in local deployment Bug Nov 14, 2017 Rex Lee Open
YARDSTICK-681 convert ping test to use SNAPS Sub-task Jun 19, 2017 Rex Lee Open
YARDSTICK-846 influxdb docker ip change when the docker restart Bug Nov 14, 2017 Rex Lee Open
YARDSTICK-836 Remove fetch openrc when using yardstick env prepare Task Nov 06, 2017 Jack Chan In Progress
YARDSTICK-690 Support Ubuntu 16.04 for Yardstick Dockerfile Story Jun 26, 2017 Jing Lu In Progress
YARDSTICK-796 l2fwd test in vsperf Tgen mode(host trafficgen) Improvement Aug 23, 2017 Ace Lee Open
YARDSTICK-789 Add sample test case yaml for common OpenStack operation scenarios Story Aug 21, 2017 Jing Lu Open
YARDSTICK-649 Record log of each task, using task_id as log file name Task May 15, 2017 Jack Chan Open
YARDSTICK-795 l2fwd test in vsperf Tgen mode(VM trafficgen) Improvement Aug 23, 2017 Ace Lee Open
YARDSTICK-648 more RestApi implemented Epic May 15, 2017 Jack Chan Open
YARDSTICK-639 Rewrite prepare env bash to ansible Story May 12, 2017 Maciej Skrocki In Progress
YARDSTICK-646 would change source list which is dependant and should be move to dockerfile Bug May 15, 2017 Rex Lee Open
YARDSTICK-701 heat context should support using block-device Improvement Jul 04, 2017 Rex Lee Open
YARDSTICK-383 support the integration of Qtip Story Nov 01, 2016 Rex Lee In Progress
YARDSTICK-675 Create RestAPI to support update yardstick service to new version Task Jun 06, 2017 Rex Lee Open
YARDSTICK-881 TC081 Documentation erroneously mentions it as TC080 Bug Dec 04, 2017 Md Safiyat Reza Open
YARDSTICK-862 More parameters for test case should be open to config with descriptions Story Nov 22, 2017 Rex Lee Open
YARDSTICK-858 tc023, tc077, tc082 has no rst doc in the repo Bug Nov 21, 2017 Rex Lee Open
YARDSTICK-855 pretty cli format: yardstick runner/scenario/testcase list Task Nov 20, 2017 Rex Lee Open
YARDSTICK-708 SLA 'assert" and "monitor" types inconsistency Story Jul 06, 2017 Jing Lu Open
YARDSTICK-698 add test suite chapter into user-guide Improvement Jul 03, 2017 Rex Lee Open
YARDSTICK-676 Ha testcase start service script fail on centos env Task Jun 07, 2017 Rex Lee Open
YARDSTICK-787 testcase sla verification need to be unified by framework Improvement Aug 17, 2017 Rex Lee Open
YARDSTICK-894 Remove unnecessary function definition in IxLoadTrafficGen Bug Dec 14, 2017 Rodolfo Alonso Hernandez Open


Yardstick Framework

1Pass parameters between scenarios      

Allow parameters to pass between scenarios. We should just use Jinja2 again.  Use an alternative delimiter like [[ ]] or << >> and then load the entire context_cfg, scenario_cfg and any local vars into the Jinaj2 namespace.  This gives us the full power of Jinaj2 to do concat or filtering

2heat context enhancement with more openstack features

flavor support . Create flavor using heat.

3more RestApi implemented Jack 
4report status(pass/fail) for each test case in CI
  • also should take care of the exception in each test case

  • should discuss how to define pass/fail

5record log of each task, using task_id as log file name

for example: (6908cc23-a90e-43f5-a7b7-25227ede60bd.log)

6collect SUT envrioment information
can re-use ansible facts,  ansible -i inventory -m setup > facts.json0
7Support for multi dispatcher

http + file +DB

8support for multi test summary output 

report + dashboard

9Grafana improvement

align with yardstick GUI(embed)

result format and style update?we plan to use highcharts to draw charts)


requirements from kvm project

post-execute script support (candidate solution by node context with ansible script)

11Update ansible integration
  • Ansible for post-HEAT VNF creation
  • Ansible for undercloud config,  control, compute node
  • requires every target has python2,  cirros might not have python2 
  • can specify path to remote python interpreter, e.g.  /usr/bin/python3



NSB Features

1Heat extensions for NSBconnect to TGHighRoss 
2New TGs
  • ixia/IxNet L1-L4 (should have REST API) (medium)
  • ixload L4 -L7  (should have REST API)  (high)
  • Ansible to deploy 4 VMs onto baremetal
  • Spirent   (high)
  • pktgen  
  • prox (see below)  (high)
  • TRex(Integrated)
  • Moongen (low)

for node context

  • collectd plugins  (high)
  • which plugins do we need
  • • Libvirt plugin
  • • DPDK Stats plugin
  • • DPDK-Events plugin
  • • OvS Events plugin
  • • vswitch-stats plugin
  • • Huge pages plugin
  • • RAS Memory plugin
  • Unreleased plugin in testing to get Memory bandwidth, L1, L2, cache utlization etc.  
  • which plugins do the installers install?
  • action: find out specific versions for each plugin
  • We have a quick hack shell script to install collectd as part of NSB code, but this should be removed
update userguide with info on collectd prerequisites 
check with installers:
  • Fuel ---Ruijing Guo
  • Compass --- Kubi
  • Apex   --- Ross 
4 KPIS Use openstack API to collect data
  • Or use SNAPs





1Integrate with Prox
  • Adding new TG tool and VNF simulator,  DPDK application called Prox 
  • tg class
  • VNF class
2Traffic generator abstraction
  • Re-use traffic generators with common API
3Integration with SNAPS-OO
  • SNAPS is python 2, would need python 3 support to directly import
  • use SNAPS for calling Heat
  • use SNAP for any random openstack API call,  e.g. nova live-migrate
  • using existing Heat YAML abstraction, but convert that to equivalent SNAPS calls to create VMs and networks
4Integration with IXloadNew TGs
  • ixload L4 -L7  (should have REST API) 



New Test Cases



IDStoryDescription OwnerComments
1live migration test
  • do live migrate without load
  • do live migrate with memory load
  • report downtime and migration time
2Stress testing  Rex 
3Scale-out tests
  • Stress tests require monitoring SLA during scale-out
  • This requres a dynamic context, where the test case has a refernce to the context
  • can VCPU at runtime?
{% for cpu in 1 2 4 %}
    name: yardstick-new-flavor
    vcpus: {{ cpu }}
    ram: 1024
    disk: 2
 {% endfor %}   
5Ha testing
    • HA Test cases on Management Components such as haproxy and pacemaker
    • HA Test cases on Instance Scenarios
      (cooperate with HA team)
6Container Pingrequirement from Open Retrieve teamMedium  



Yardstick GUI


1upload openrc file and source it. High  
2run a test case High  
3upload a testcase yaml file and then run i Medium  
4run a test suite Medium  
5create a test suite and then run it. Medium  
6get a task list and check it  High  
7get result of certain task High  

include yardstick report to show



userguide improvement(medium)
devguide improvement(focus on )
code clean(low)
  • run pylint
  • refactor
code directory adjustment to make it more clear for developers  (need more info)
  • currently in yardstick there're more features, so it requires to reorganize the code directory to be clear
  • No labels