Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page is being started to continue the discussions from http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2018/opnfv-testperf.2018-08-02-15.08.html

Note

Please treat this page like an Etherpad - feel free to make changes and comment.


Problem Statement

  • Testing projects in OPNFV are diverse and each has their own way of being set up and run.
  • Test Results DB can be run locally, but lacks documentation, and is not necessarily used consistently by all projects. 
  • Gathering results from each project varies greatly
  • There is no central place for gathering information about the SUT (ie how many compute, control and storage nodes are there)
  • Integrating test projects is not well understood (ie: can I have Barometer running while I execute Yardstick and how to set that up?)
  • Highlighting benefits of test project integration (ie: what can Barometer show me about systems while Bottlenecks is running?)
  • The Test Results DB is used as part of the OPNFV release process, but there does not seem to be anyone that owns this anymore.  Lack of PTL and method of reporting problems is a risk.

Ideas

  • Creation of a single container or script that facilitates the pulling in and configuration of each test project.  This can also start common services like Grafana, InfluxDB, Mongo, Test Results container, etc.  It can also house a web server with documentation.
  • Creation of test project overview documentation - started at PlugFest: https://etherpad.opnfv.org/p/plugfest_test_tools_overview

Projects Results DB

Bottlenecks

  • Results pushed to test results DB
  • Store results locally as Json files
  • Real time monitoring results are stored in Prometheus and displayed in Grafana

Dovetail

  • Uploads results for processing with OVP.
  • Local results are stored in ____

Functest

  • Results are pushed to test results DB.

QTIP:

  • Stores results in ____

StorPerf

  • Stores raw data in local Carbon/Graphite container
  • Stores final results in local StorPerf container
  • Results pushed to test results DB
  • Has its own reporting module that can read from local StorPerf container or test results DB
  • Has specific JSON format for the details of the test run stored in test results.

VSPERF

  • Stores results in ____

Yardstick

  • Stores results in local InfluxDB container
  • Results Data pushed to community InfluxDB (online) and results pushed to test results DB
  • Reports via both local Grafana container and community Grafana (online)
  • The raw result is json style and can support other way to store the result

Viewtracker