Page tree
Skip to end of metadata
Go to start of metadata

Please note that this page is a draft page aiming to prepare the Euphrates release...(image src:

Key Summary T Created Reporter Status


Yardstick Framework

1Pass parameters between scenarios      

Allow parameters to pass between scenarios. We should just use Jinja2 again.  Use an alternative delimiter like [[ ]] or << >> and then load the entire context_cfg, scenario_cfg and any local vars into the Jinaj2 namespace.  This gives us the full power of Jinaj2 to do concat or filtering

2heat context enhancement with more openstack features

flavor support . Create flavor using heat.

3more RestApi implemented Jack 
4report status(pass/fail) for each test case in CI
  • also should take care of the exception in each test case

  • should discuss how to define pass/fail

5record log of each task, using task_id as log file name

for example: (6908cc23-a90e-43f5-a7b7-25227ede60bd.log)

6collect SUT envrioment information
can re-use ansible facts,  ansible -i inventory -m setup > facts.json0
7Support for multi dispatcher

http + file +DB

8support for multi test summary output 

report + dashboard

9Grafana improvement

align with yardstick GUI(embed)

result format and style update?we plan to use highcharts to draw charts)


requirements from kvm project

post-execute script support (candidate solution by node context with ansible script)

11Update ansible integration
  • Ansible for post-HEAT VNF creation
  • Ansible for undercloud config,  control, compute node
  • requires every target has python2,  cirros might not have python2 
  • can specify path to remote python interpreter, e.g.  /usr/bin/python3



NSB Features

1Heat extensions for NSBconnect to TGHighRoss 
2New TGs
  • ixia/IxNet L1-L4 (should have REST API) (medium)
  • ixload L4 -L7  (should have REST API)  (high)
  • Ansible to deploy 4 VMs onto baremetal
  • Spirent   (high)
  • pktgen  
  • prox (see below)  (high)
  • TRex(Integrated)
  • Moongen (low)

for node context

  • collectd plugins  (high)
  • which plugins do we need
  • • Libvirt plugin
  • • DPDK Stats plugin
  • • DPDK-Events plugin
  • • OvS Events plugin
  • • vswitch-stats plugin
  • • Huge pages plugin
  • • RAS Memory plugin
  • Unreleased plugin in testing to get Memory bandwidth, L1, L2, cache utlization etc.  
  • which plugins do the installers install?
  • action: find out specific versions for each plugin
  • We have a quick hack shell script to install collectd as part of NSB code, but this should be removed
update userguide with info on collectd prerequisites 
check with installers:
  • Fuel ---Ruijing Guo
  • Compass --- Kubi
  • Apex   --- Ross 
4 KPIS Use openstack API to collect data
  • Or use SNAPs





1Integrate with Prox
  • Adding new TG tool and VNF simulator,  DPDK application called Prox 
  • tg class
  • VNF class
2Traffic generator abstraction
  • Re-use traffic generators with common API
3Integration with SNAPS-OO
  • SNAPS is python 2, would need python 3 support to directly import
  • use SNAPS for calling Heat
  • use SNAP for any random openstack API call,  e.g. nova live-migrate
  • using existing Heat YAML abstraction, but convert that to equivalent SNAPS calls to create VMs and networks
4Integration with IXloadNew TGs
  • ixload L4 -L7  (should have REST API) 



New Test Cases



IDStoryDescription OwnerComments
1live migration test
  • do live migrate without load
  • do live migrate with memory load
  • report downtime and migration time
2Stress testing  Rex 
3Scale-out tests
  • Stress tests require monitoring SLA during scale-out
  • This requres a dynamic context, where the test case has a refernce to the context
  • can VCPU at runtime?
{% for cpu in 1 2 4 %}
    name: yardstick-new-flavor
    vcpus: {{ cpu }}
    ram: 1024
    disk: 2
 {% endfor %}   
5Ha testing
    • HA Test cases on Management Components such as haproxy and pacemaker
    • HA Test cases on Instance Scenarios
      (cooperate with HA team)
6Container Pingrequirement from Open Retrieve teamMedium  



Yardstick GUI


1upload openrc file and source it. High  
2run a test case High  
3upload a testcase yaml file and then run i Medium  
4run a test suite Medium  
5create a test suite and then run it. Medium  
6get a task list and check it  High  
7get result of certain task High  

include yardstick report to show



userguide improvement(medium)
devguide improvement(focus on )
code clean(low)
  • run pylint
  • refactor
code directory adjustment to make it more clear for developers  (need more info)
  • currently in yardstick there're more features, so it requires to reorganize the code directory to be clear
  • No labels