Page tree
Skip to end of metadata
Go to start of metadata


Day 1

  • Evaluation of Benchmark Repeat-ability, aided by new (search) methods
    • Collaboration with ETSI TST009
    • Presentation
    • Initial testing by Sridhar Rao indicates that Binary Search with Loss Verification shows potential to improve Throughput Repeat-ability over Time. (TEST)
  • Auto Project Presentation
    • Introduction: Post-installation, installer agnostic ONAP installation, CPU agnostic using OOM
    • Explore NFVO, VNFM, design, policies, closed-loop automation, alarm correlation, DCAE (analytics), SDN-C etc. Also explore design-time onboarding, deployment, ops (A&AI)
    • Other goals: Analyze power consumption per CPU etc. 
    • 3 Use Cases: Edge, resilience, E-vCPE
    • Lot of discussion on infra and CI strategies
  • Edge Cloud Requirements Session
    • China Mobile requirements around edge, motivation around the effort, architecture; why this makes sense for the community
    • Discussion around edge cloud project, what its focus should be and how to get OPNFV onto the edge radar
    • Edge cloud project collaboration/discussion with OpenStack EC WG; room for collaboration on CI/testing. OPNFV can play a unique role by using our strengths, identified low-hanging fruits.
  • Monitoring System Behaviors under Long Duration Testing
    • Mainly focused on monitoring
    • Presentation by Emma on Barometer, how these metrics can be used by Yardstick NSB
    • Presentation by Rutuja on monitoring architecture (Prometheus, Barometer/collectd/CAdvisor, Grafana)
    • Presentation by Gabriel on Long Duration Testing – will last for 1-2 days (TEST)
      • Normal usage
      • Stressful usage
  • Hardware Acceleration
    • Motivation for hardware acceleration and relevance to MEC
    • Which VNFs need to be accelerated
    • Types of hardware acceleration, compare/contrast, look-aside/inline
    • Requirements for the future, thinking around a new potential OPNFV project around common APIs around accelerators
    • Presentation
  • OPNFV Docs
    • worked with the removal of submodules
    • planned another docs hacking day August-22 (2 days before Mliestone 6)
  • TSC task force on CD based release process
  • Dovetail & OVP: 
  • SFC scenario deployed on LaaS with XCI, connected to ETSI Hub. Found 2 VNFs that want to test NSH. Testing with 1 of the VNFs. (TEST)
  • Tungsten Fabric might start with with XCI and other interop/testing work; Juniper person is delayed. (TEST)
  • Compass is testing interoperability. (TEST)

Day 2

  • Yardstick planning session for Gambia
    • Rex showed Yardstick planning in Gambia for a review - Etherpad
      • Installation/Deployment: container splitting, shell→ansible, create tests for Ubuntu, CentOS, Suse and container mode installation
      • Usability features: improve reporting and test suite results, renaming test cases, TGaaS
      • Testing (CI/CD): CI Grafana upgrade and data migration
      • Code quality improvements: unit tests, error handling and shade conversion
      • Features/improvements: Message Queue, HA testing, test cases tiers
      • Kubernetes support: includes context enhancement and more test cases for the k8s scenario
    • Yardstick team talked about the documentation improvement work to do with user guide and dev guide in Gambia
      • complete introduction and examples need to added about the key features, details can be add into the dev guide
      • GUI usage and guide should be added and more demo videos
    • Discussion around minimal requirements for CD process from Yardstick
      • The gap is the functional test to verify Yardstick testing framework itself.
      • for the test cases, Yardstick could support with different confidence levels by test suites or tiers in Gambia.
  • Fraser release retrospective
  • OPNFV hands-on
    • Ryota gave an update on OPNFV hands-on lab conducted in Japan: Apex install, Functest based on Euphrates and updated for Fraser
    • Amar gave an update on LFS164/264 online courses
    • Discussion around making the hands-on lab program more formal; adding other installers such as Fuel
  • MANO API collaboration
    • ETSI GS NFV-TST10: MANO API conformance testing specification
      • Goal is to have automated system that consumers can just use and run 
    • Using OPNFV scenario as a NFVI+VIM platform for this MANO conformance testing, to gain full automation of the tests 
    • 1st use case type: Simulate various HW failures
      • Can this be a part of Barometer?
    • 2nd use case type: monitor OS NB APIs to verify correct operation of MANO under test 
    • Challenge: complete call flow isn't detailed yet by specs. There are ideas on how to move forward with this 
    • Actions: 
      • Pierre to detail use cases to see if there are more, and to extract call flow examples to study further 
      • Pierre and Tim to approach Barometer team when ready 
  • CD based release process
  • Fuel installer overview/update
    • Basic overview of Fuel
    • Details on Fuel architecture & functioning – bare metal POD and virtual POD; x86 and ARM (see user guide)
    • Deployment, troubleshooting discussion
    • Hands on virtual deployment on Enea Lab and Google Cloud
  • Yardstick/NSB
    • Overview of Yardstick and NSB as one of the use cases
    • Demo video
  • Auto discussion continued
    • Continued discussion on infra and CI strategies
    • Put together a rough plan to go further on deploying ONAP on OPNFV
  • Release meeting
  • Continuing to test RedHat and WhiteStack with Dovetail (TEST)
  • SFC testing with some of the VNFs (TEST)

Day 3

  • DP Benchmarking with TRex
    • Presentation
    • Discussion on traffic generators, NFVbench; updates on what's new in NFVbench, how to use traffic generators (to help harmonize its use), real-life considerations
  • Edge & Edge Pharos (Etherpad)
    • Federated Keystone investigation between PODs
      • Glance federation is even less mature so not in scope for now
      • Generate testing requirements for edge wrt. Keystone etc.
      • Next steps: take 2 OPNFV scenarios and federate them (Keystone); now it can be included in CI; run Tempest test for federation
    • Edge Pharos
      • 1-2 node installs can be done already and put into CI already. So don't need to create a new spec right away. Need to learn & iterate.
      • But need to consider refreshing the Pharos spec since it hasn't been changed for a long time
  • Intern report out
    • Presentations posted (SofiaRutujaWang Meng)
    • Feedback from interns: better documentation to help on-boarding
    • Two had prior internship experiences with Mozilla and OpenStack
  • XCI
    • Motivation for XCI, history behind Ansible/Bifrost; motivation for expanding of tooling (Helm, Terraform etc.), broadening to cross-CI between other communities, allowing projects the ability to create scenarios (as opposed to depending on the installer team)
    • Motivation to bring XCI, CD concepts to all current installers
    • Expanding to other OpenStack projects (e.g. Masakari), ONAP etc. and will pave the way for CD
    • Introduction to OpenCI
  • OPNFV Release Process
    • Continued discussion on CD planning from yesterday
    • 3 abstract confidence gates and minimum testing for those gates
    • Test frameworks and how they could provide a path to release when working on master – methodology recommendation from Fatih: test framework & stack leapfrog each other, while successively validating each other.
    • This first release using CD process is highly experimental, set expectations accordingly. 8-10 projects are CD only, its possible there may not be any release artifacts.
  • Compass installer
    • Update of the Compass release in Fraser
    • Introduction on how to use it
  • Yardstick Web GUI Hands-on
    • GUI is available in addition to CLI since Euphrates; the session had complete demo of the GUI
    • Deploy Yardstick Service via kubernetes and expose the service to users presentation
  • POD12 Connectivity Demo
    • Internal discussion: Use VSPerf to set up the datapath through 0, 1, 2 VNFs without using TRexx so that NFVBench etc. can be used instead with that datapath.
  • DP Testing with Bottlenecks
    • Discussing Bottleneck and NFVBench and integration between them
    • Introduction to NFVBench and use cases
    • Introduction to Bottlenecks and how to run through CI
    • Make it easier for users to use these tools
  • Compass/Bottlenecks being tested as a Dovetail optional task. Used to test creation of multiple VMs simultaneously. Tests the capability of NFVI/VIM around this.
  • WindRiver has begun some NFVBench testing with Titanium cloud; they have also started some Dovetail work.
  • Dovetail testing with Nokia; need to fix some Functest issues. Discovered another bug in Functest, fixed (vIMS test case had an issue). Tomorrow WindRiver, Nokia.
  • Yichen – Wind River testing ongoing with NFVBench.
  • Amar – os, nosdn, nofeature, noha with Fuel tried out on GCP with basic Functest/Yardstick testing for future training projects

Day 4

  • Evaluation of Benchmark Repeatability, aided by new methods
    • Overview of intermittent perf. problems vs. resource exhaustion
    • Software techniques wrt. VSPERF to find resource exhaustion and look for patterns for intermittent perf. problems
    • First round, new search algorithm was pretty successful. Better repeatability.
  • Monitoring System Behaviors under Long Duration Testing
    • Overview of long duration testing dashboard
    • Evaluation of tests & tests results for tests already run, dashboard and log analysis; discussion on how to look for problems
    • Al – there are two types of long-duration testing: continuous operation i.e. spin-up/tear-down (what Gabriel presented) and system-left-alone (say a network service is constantly running & being monitored). Both have merit, and perhaps the 2nd one should be considered.

    • Long duration testing -- Several errors hypervisor, RabbitMQ, Ceph etc. due to long duration testing that appear randomly. Need to figure out whether these are OPNFV bugs vs. upstream bugs. Testing should be part of the release criteria.

  • Gambia Release Themes (Etherpad)
    • New projects, returning projects, projects not participating listed
    • CD-based release process discussed
    • Key features discussed
    • Discussion around a possible separate "test project release", projects are in good shape – discussion around packaging the db/API
  • TGaaS
    • 5 projects use trex, differences in trexx drivers cause performance deltas
    • VSPERF, NFVBench, Yardstick working on an experiment to test the same L2 forwarding test and compare performance results
    • Next the idea is to develop a common driver/set of libraries, harmonize requirements
  • Release automation
  • Laas Update
    • LaaS 1.0 (today)
    • Roadmap: dynamic allocation, multi-user booking, auto deploy of OPNFV, etc. by end of summer'18
  • Intro to OPNFV Testing Projects (Etherpad)
    • Yardstick and NSB; integration with Bottlenecks, SampleVNFs; initial k8s tests and discussion on futures
    • VSPERF motivation & overview = NFVI datapath benchmarking; IETF draft, detailed test specification. Commercial/open source traffic generator support (5 tools xena, moongen, trex, ixia, spirent). Additions over time e.g. noisy neighbors. Looked at the test matrix.
    • NFVbench overview = NFVI dataplane benchmarking; It's a standalone tool not tightly integrated with OPNFV. Production focus for full stack benchmarking with automation.
    • Bottlenecks overview. Simple stress test framework. Presented long duration testing part already at the plugfest.
    • Functest overview. 5 test cases each with a different docker container. k8s support wip.
    • Doteail & OVP overview. OVP is the program and Dovetail is the automated testsuite for 3rd party NFVI/VIM. Mandatory & optional test cases.
    • Create YouTube recordings/webinars?
  • General feedback in co-location with ETSI
    • Opportunity to work with commercial products/platforms
    • Advertise collab sessions ahead of the event
    • OPNFV orientation (incl. installer sessions, test overview)

Day 5


  • No labels