Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 42 Next »

Test and Performance Weekly Meeting

This weekly meeting is for topics common across projects focused on testing … this is a good place to start if you want to get high-level status of the individual projects and learn about hot topics related to testing OPNFV

  • Test projects should try to have a rep attend this meeting
  • New test related projects should be discussed in this forum to gain feedback from the OPNFV testing community
  • Agenda varies but typically is structured ...
    • 15 min report out / sharing by projects (high level status / plans)
    • 20 min discussion on test project collaboration topic/s
    • 20 min – share new test projects or other test topics
  • There is a Test and Performance Operational Meeting immediately after (topics include dashboards, tools, KPIs, traffic profiles, etc.) See https://wiki.opnfv.org/meetings
  • When: Every Thursday 7:00-8:00 PDT  / 14:00-15:00 UTC
  • IRC channel: #opnfv-testperf @ Freenode

 


Useful links ...

Testing overview page with links to testing projects

Pharos project page


Meeting Chair and Minutes Schedule

This is a proposed schedule for the weeks ahead.  Please feel free to sign up and change as needed.

WeekDateAgendaMinutes
27 YardstickFunctest
28 StorPerfCPerf
29 VSperfQtip
30 BottleneckYardstick
31 QtipStorperf
32 CPerfVSperf
33 FunctestBottlenecks
34 StorPerfQtip
35  Functest
36   
37 Functest 
38 Qtip 
39  StorPerf
40   

Proposed Agenda Topics


Agenda - August 18th, 2016 (55 min)

chair: Functest / minutes: Bottlenecks

Agenda - August 11th, 2016 (55 min)

chair: CPerf / minutes: VSPerf

Agenda - August 4th, 2016 (55 min)

chair: Qtip / minutes: Storperf 

Agenda - July 28th, 2016 (55 min)

chair: Bottleneck / minutes: Yardstick

  • minutes: 

Agenda - July 21th, 2016 (55 min)

chair: VSPerf / minutes: Qtip

  • Action points follow up (5')
  • Status on testing white paper (A.Serda)
  • Results of APAC friendly timeslot for this meeting email
  • ...
  • AoB (5')
  • minutes: 

Agenda - July 14th, 2016 (55 min)

chair: Storperf / minutes: CPerf

Agenda - July 7th, 2016 (55 min)

chair: Yardstick / minutes: Functest


Agenda - June 30th, 2016 (55 min)

Agenda - June 16th, 2016 (55 min)

Agenda - June 9nd, 2016 (55 min)

Agenda - June 2nd, 2016 (55 min)

  • Action points follow up 5'
  • Colorado status of the test projects 15'
  • Review of test initiatives for the Summit in Berlin 15'
    • Conversation with the testing community
    • Organization
    • Global testing approach (Alan's paper)
  • Discussion on Test domains 15'  
    • how to identify the domains (system/components/ETSI)
    • how to "build" a map and detect uncovered or poorly covered areas
    • how to reduce possible overlap between test projects
  • AoB 5'
  • Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2016/opnfv-meeting.2016-06-02-14.00.html


Agenda - May 19th, 2016 (55 min)

Agenda - may 19th, 2016 (55 min)

Agenda - may 12th, 2016 (55 min)

Agenda - April 6th, 2016 (55 min)

  • It will be good to get a readout from test-project reps on release-C plans (PTLs please attend or send reps where possible)
  • Discuss criteria for including tests are part of CI for release-C
  • Test resources for release-C (Testing) 
  • Plugfest – test cases

Agenda - March 24th, 2016 (55 min)

  • Plugfest
  • Resources for release C

Agenda - March 10th, 2016 (55 min)

  • Benchmarking VNF/VNFIs - 30 min (Al)
  • Storperf - 15 min (Mark)
  • Pharos - 10 min (Trevor)

Agenda - March 3rd, 2016 (55 min)

  • Continue from last week with what Test Projects delivered in release B ...
  • VSPERF - 15 min (Maryam)
  • Bottlenecks - 15 min (Hongbo / Yaoguang)
  • Storperf - 15 min (Mark)
  • Time allowing -> Release C common topics across test projects including test cycles, test strategy, infrastructure, coverage, automation, etc.

Agenda - February 25th, 2016 (55 min)

  • Test projects - what was delivered in release B ...
  • Qtip demo - 15 min
  • Functest - 15 min
  • Yardstick - 15 min
  • VSPERF - 15 min
  • Bottlenecks - 15 min
  • Storperf - 15 min

May run over into Yardstick meeting (or continue next week)

Agenda - February 18th, 2016 (55 min)

  • Plugfest test-cases (30 min)
  • Testing plans for release C (25 min)

Agenda - February 4th, 2016 (55 min)

_Meeting Cancelled
_

Agenda - January 28th, 2016 (55 min)

  1. Plugfest testplan
  2. Brahmaputra test readiness status
  3. Test success criteria (now that scenarios clarified)
  4. Plan for the visualization of the results (alternative to LF dashboard)
  5. Organization Asian/European slot
  6. Troubleshooting/debugging slot

Agenda - January 21st, 2016 (55 min)

  1. Brahmaputra release (35 min)
  2. Test project status (5 min)
    • yardstick, functest, bottleneck, vsperf, qtip
    • Storperf
    • Companion projects
  3. Slots for testing / responsibility to trigger redeployments (5 min)
  4. Migration of test DB (5 min)
  5. Results display / dashboards (10 min)
  6. Troubleshooting organization/planning (10 min)
  7. Test meeting organization (5 min)
  8. Test project participation in Dovetail / plugfest (10 min)

Agenda - January 14th, 2016 (55 min)

  • Test dashboard demo (Marion, Andrew - LF) - 15 min
  • Test artifacts for Rls B - 15 min
    • Test results
    • Test results sample / template
    • Other test documents?

Agenda - January 7th, 2016 (55 min)

  • Test dashboard - 10 min
    • Update on portal prototype status and roadmap (Marion, Andrew - LF)
    • Next steps and anything needed from other projects (Morgan)
  • Documentation for Rls B - expectations and status - 15 min
    • Test result documentation - define and agree what to document
    • Pharos documents / templates

Agenda - December 17th, 2015 (55 min)

  • Status of the test projects (Morgan) 20 min
    • Functest ... CI labs - Fuel, Joid, Compass, Apex
    • Yardstick
    • Other test projects
  • Retro planning attempt for B-Release from now to .... 20min
  • Showcasing OPNFV test results using Kibana (Frank Brockners and Peter Bandzi) - 15 min

Agenda - December 10th, 2015 (55 min)

As part of test meeting, we’ll also continue the discussion on common configuration files – with the plan to evolve things to closure asap.

Agenda items for this topic include:

Agenda - December 3rd, 2015 (55 min)

  • This meeting slot will be used for discussion on release plan for build/deploy/test
  • CI needs to know which configuration to run and which tests to execute
  • Need common configuration files for ...
    • Build config
    • Deploy config
    • POD config
    • Test config
  • Question 0: what are the combinations of installers/controllers/other sw component versions (compatibility matrix)?
  • Question 1: how many deploy/test environments are needed?
  • Question 2: what community labs will volunteer to provide needed resources?

Agenda - November 26th, 2015 (55 min)

  • Release D milestone - status for test projects (10 min)
  • Dashboard strategy and plan (45 min)

Agenda - November 19th, 2015 (55 min)

  • Digital Rebar demo (this will be recorded) (30 min)
  • Summit discussions re. lab resources (allocation and open issues) (10 min)
  • Yardstick update (10 min)
  • Asia meeting slot (5 min)

Agenda - November 5th, 2015 (55 min)

  • Preparing/organizing test related activities at OPNFV Summit next week (30 min)
  • OpenStack Summit last week - what was of interest to the testing group? (10 min)
  • Update on Functest (15 min)

Agenda - October 22nd, 2015 (55 min)

Agenda - October 15th, 2015 (55 min)

Agenda - October 8th, 2015 (55 min)

  • Rally: presentation of the new features (20 min)
  • Pharos: Test project support (20 min)
  • Yardstick Docker discussion (15 min)
  • Individual test projects status (time allowing)

Agenda - October 1st, 2015 (55 min)

  • Test projects status (10 min)
  • Demo & discussion Test Dashboard (15 min)
  • Yardstick demo - virtual Traffic Classifier (30 min)

Agenda - September 24th, 2015 (55 min)

  • Test projects status (10 min)
  • Discussions/decisions on the test deliverable for R2 (20 min)
  • Discussions/decisions on the dependencies towards features and other test projects (20 min)

Agenda - September 17th, 2015 (55 min)

  • Bottlenecks status (5 min) - Hongbo
  • Transformer status (5 min) - Michael
  • ODL related testing activities (45 min, Morgan)

Agenda - September 10th, 2015 (55 min)

  • Test projects status (only projects not represented last week) - 2 min per project
    1. Bottlenecks - Find system bottlenecks (Huawei) - Hongbo
    2. CPERF - Controller Performance Testing (Red hat) – Daniel
    3. Policytest - Network policy testing (CISCO) - Keith
    4. Affirm - validation criteria for use of OPNFV trademarks (Ericsson) – Chris
    Projects Not yet approved by TSC ...
    5. STORPERF - Storage Performance Benchmarking for NFVI (EMC) - Edgar
    6. Transformer - Carrier Grade Requirements (Ericsson) - Michael

 

  • Logistics for Test projects ”operational” meeting (5 min)
  • Milestone C guidelines (15 min)
    • What are the expectations from Test Projects
    • Test needs from projects and how to integrate (test specs were supposed to have been defined end of Sep)
  • Testing dashboard - update and first demo (15 min)
  • Test definitions for rls B (10 min)
    • What are we testing for each installer
    • Definition of failure / severity

Agenda - September 3rd, 2015 (55 min)

  • Test projects status - all test project leads - 20 min
  • Yardstick - Demo of Virtual Traffic Classifier VNF (Ana, Ericsson) - 30 min
  • Qtip update (Vikram, Dell) - 15 min

Test projects

Projects approved by TSC

  1. Functest - Base system functionality testing (Orange) - Morgan
  2. Yardstick - Infrastructure Verification (Ericsson) – Ana
  3. VSPERF -Characterize vSwitch Performance for Telco (Intel) - Maryam
  4. Qtip - Platform Performance Benchmarking (Dell) - Vikram
  5. Bottlenecks - Find system bottlenecks (Huawei) - Hongbo
  6. CPerf - Controller Performance Testing (upstream) – Daniel Farrell
  7. Policytest - Network policy testing (CISCO) - Keith
  8. Affirm - validation criteria for use of OPNFV trademarks (Ericsson) – Chris


Projects Not yet approved by TSC ...

  9. STORPERF - Storage Performance Benchmarking for NFVI (EMC) - Edgar
  10. Transformer - Carrier Grade Requirements (Ericsson) - Michael

 

Agenda - August 27, 2015 (55 min)

  • Yardstick - Discussion on planned deliverables for Mobility Traffic Profile (Ana, Ericsson) - 20 min
  • VSPERF - update and demo (Maryam, Intel) - 30 min
  • Test dashboard (wiki) (Morgan, Orange) - 10 min

Agenda - August 20, 2015 (55 min)


Agenda - August 13, 2015


Agenda for August 6, 2015

Meeting canceled


Agenda for July 30, 2015

Meeting canceled


Agenda for July 23, 2015

Proposed Agenda (60 min):

Qtip 45min

  • Testing strategy, planning for Brahmaputra (Nauman Ahad / Wenjing Chu)

Pharos 5 min

  • Status update

Functest 5 min

  • Status Update

Yardstick 5 min

  • Status update

Agenda for July 16, 2015

Proposed Agenda (60 min):

Pharos 20 min

  • Release B deliverables
  • LF support plan
  • Near term needs
  • Lab status and availability
  • MAAS follow up

Functest 5 min

  • Status and Update - Morgan is out this week

Yardstick 30 min

  • B-release deliverables
  • PODs to use for test case verification
  • Definition of Done for a test case
  • SLAs for OPNFV infrastructure
  • Weekly meeting for follow up on Jira tasks

Other topics 5min

  • Pharos meeting at ODL Summit

Agenda for July 9, 2015

Proposed Agenda (60 min):

Pharos

  • Planning (15 min)
    • Plans presented to TSC
    • Logistics for Pharos dedicated meetings
  • MAAS demo and discussion - Narinder Gupta - narinder.gupta@canonical.com (30 min)

Functest

  • Status and Update (5 min)

Yardstick

  • Status and Update (5 min)

Other topics

  • Standing agenda (5 min)

Agenda for July 2, 2015

Proposed Agenda (60 min):

Functest

  • Demo Functest

Common topics


Agenda for June 25, 2015

Proposed Agenda (60 min):

  • Discuss test strategy for Rls 2

Agenda for June 18, 2015

Proposed Agenda (60 min):

Pharos

  • No agenda topics

Functest

  • No agenda topics

Yardstick

  1. Demo of the tool
  2. Discussion on use of the tool and how to serve wider needs of testing projects

Agenda for June 11, 2015

  • Due to Arno retrospective this week meeting is changed to 14h30 UCT (7:30 am Pacific Daylight Time) and will be 30 min long
  • Yardstick demo is postponed to the following week (Thursday 18th June)

Proposed Agenda (30 min):

Pharos

  • No agenda topics planned

Functest

  • Test strategy for Release 2

Yardstick

  1. Demo postponed to next week

Agenda for June 4, 2015

PLEASE NOTE MEETING HAS BEEN CANCELLED AS A NUMBER OF ACTIVE PARTICIPANTS CANNOT ATTEND (YARDSTICK DEMO WILL BE RESCHEDULED FOR NEXT WEEK)


Agenda for May 28, 2015

Proposed Agenda (1hr):

OVS characterization test framework

  1. Overview of test tools
  2. Demo
  3. Status and plans

Functest

  • Introduction to Functest
    1. Arno release status
    2. Project organization
    3. Git description
    4. Integration with CI
    5. Results analysis
    6. New test case integration

Yardstick

  1. Status
  2. Demo June 4th
  3. Backlog

Agenda for May 14, 2015

Proposed Agenda (1hr):

Pharos (15 min)

  • Revised Pharos spec for Arno
  • Lab definitions and tracking

Functest (15 min)

  • Organization format (meeting, IRC,..)
  • Beyond R1
    • new test cases
    • new tools
    • analytics on bench results
    • synchro with ETSI Test

Yardstick (30 min)

  • Questions / discussion from project proposal
  • Project scope & synergies with other test projects
  • Project members
  • Organization format (meeting, IRC,..)

Agenda for May 7, 2015

Meeting this week will be 30 min to accommodate Arno readiness call at 7:30 PDT. We will likely only have time for Yardstick topics in the call.

Pharos

  • No agenda items this week

Functest

  • No agenda items this week

Yardstick (30 min)

  • Project scope & synergies with other test projects
  • Project members
  • Organization format (meeting, IRC,..)

Agenda for April 30, 2015

Functest

  • Release 1
  • Validation new committers
  • Review organization format (meeting, IRC,..)
  • Beyond R1
    • new test cases
    • new tools
      • synergies with yardstick
      • analytics on bench results
    • synchro with ETSI Test

Pharos

  • Release 1 documents

Agenda for April 23, 2015

Meeting canceled - prioritizing release activities and meetings


Agenda for April 16, 2015

Meeting canceled - prioritize release candidate meeting


Agenda for April 9, 2015

Meeting canceled - prioritize release candidate meeting

Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT
Your local time

  • Pharos (20 min)
    • Test-bed status
    • Pharos specification and work for release one
  • Functest (30 min)
    • Voting for functest leader
    • Working session re. status and activities for release one

Agenda for March 19, 2015

Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT
Your local time

  • Functest (35 min)
    • status and plan for rls one
    • CI integration
    • Documentation
    • Update on vIMS proposal – Metaswitch
  • Test Tool initiative (25 min) - Ericsson

Agenda for March 12, 2015

Meeting time UTC 14:00-15:00 / 7:00am-8:00am PDT
Your local time

Agenda for March 05, 2015

Meeting time UTC 15:00-16:00 / 7:00am-9:00am PST
Your local time

Minutes

2015-12-03

  • No labels