- Evaluation of Benchmark Repeatability, aided by new methods
- Overview of intermittent perf. problems vs. resource exhaustion
- Software techniques wrt. VSPERF to find resource exhaustion and look for patterns for intermittent perf. problems
- First round, new search algorithm was pretty successful. Better repeatability.
- Monitoring System Behaviors under Long Duration Testing
- Overview of long duration testing dashboard
- Evaluation of tests & tests results for tests already run, dashboard and log analysis; discussion on how to look for problems
Al – there are two types of long-duration testing: continuous operation i.e. spin-up/tear-down (what Gabriel presented) and system-left-alone (say a network service is constantly running & being monitored). Both have merit, and perhaps the 2nd one should be considered.
Long duration testing -- Several errors hypervisor, RabbitMQ, Ceph etc. due to long duration testing that appear randomly. Need to figure out whether these are OPNFV bugs vs. upstream bugs. Testing should be part of the release criteria.
- Gambia Release Themes (Etherpad)
- New projects, returning projects, projects not participating listed
- CD-based release process discussed
- Key features discussed
- Discussion around a possible separate "test project release", projects are in good shape – discussion around packaging the db/API
- 5 projects use trex, differences in trexx drivers cause performance deltas
- VSPERF, NFVBench, Yardstick working on an experiment to test the same L2 forwarding test and compare performance results
- Next the idea is to develop a common driver/set of libraries, harmonize requirements
- Release automation
- Laas Update
- LaaS 1.0 (today)
- Roadmap: dynamic allocation, multi-user booking, auto deploy of OPNFV, etc. by end of summer'18
- Intro to OPNFV Testing Projects (Etherpad)
- Yardstick and NSB; integration with Bottlenecks, SampleVNFs; initial k8s tests and discussion on futures
- VSPERF motivation & overview = NFVI datapath benchmarking; IETF draft, detailed test specification. Commercial/open source traffic generator support (5 tools xena, moongen, trex, ixia, spirent). Additions over time e.g. noisy neighbors. Looked at the test matrix.
- NFVbench overview = NFVI dataplane benchmarking; It's a standalone tool not tightly integrated with OPNFV. Production focus for full stack benchmarking with automation.
- Bottlenecks overview. Simple stress test framework. Presented long duration testing part already at the plugfest.
- Functest overview. 5 test cases each with a different docker container. k8s support wip.
- Doteail & OVP overview. OVP is the program and Dovetail is the automated testsuite for 3rd party NFVI/VIM. Mandatory & optional test cases.
- Create YouTube recordings/webinars?
- General feedback in co-location with ETSI
- Opportunity to work with commercial products/platforms
- Advertise collab sessions ahead of the event
- OPNFV orientation (incl. installer sessions, test overview)