Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Project: StorPerf - Storage Performance Benchmarking for NFVI

Description

The purpose of StorPerf is to provide a tool to measure block and object storage performance in an NFVI. When complemented with a characterization of typical VF storage performance requirements, it can provide pass/fail thresholds for test, staging, and production NFVI environments.

A key challenge to measuring disk performance is to know when the disk (or, for OpenStack, the virtual disk or volume) is performing at a consistent and repeatable level of performance.  Initial writes to a volume can perform poorly due to block allocation, and reads can appear instantaneous when reading empty blocks.  How do we know when the data reported is valid?  The Storage Network Industry Association (SNIA) has developed methods which enable manufacturers to set, and customers to compare, the performance specifications of Solid State Storage devices (Ref).  StorPerf applies this methodology to OpenStack Cinder and Glance services to provide a high level of confidence in the performance metrics in the shortest reasonable time.

Slides and Demos

Storage Performance Indicators - Powered by StorPerf and QTIP

This video gives a little background on how StorPerf provides reliable metrics for QTIP.

Widget Connector
width600
urlhttps://www.youtube.com/watch?v=J--1Wa5xoIE
height338

StorPerf Overview - OPNFV Summit, 2017

View file
nameStorPerf- Using OpenStack to Measure OpenStack Cinder Performance.pptx
height250

StorPerf Metrics Deep Dive

View file
nameGraphite Deep Dive.pptx
height250

StorPerf Demo - Danube Pre-Release

Widget Connector
width600
urlhttps://www.youtube.com/watch?v=9OonpmJVuA8
height338

 

Project References

Meetings

Excerpt Include
meetings:Storperf Team Weekly Meeting
meetings:Storperf Team Weekly Meeting

 

Table of Contents

Build Status

Build Status

Open Bug List

JIRA
serverOPNFV
columnskey,summary,created,updated,assignee,priority,status,resolution,fixversions
maximumIssues20
jqlQueryproject = STORPERF AND type = Bug and Status != CLOSED ORDER BY priority DESC, updated DESC
serverId96acfcf2-db1a-3859-891e-03a53e9315b0

Euphrates Backlog

JIRA
serverOPNFV JIRA
columnskey,summary,assignee,status
maximumIssues1000
jqlQueryproject = STORPERF and type != Sub-task and Status != CLOSED and FixVersion = "5.0.0" ORDER BY priority DESC, updated DESC
serverId96acfcf2-db1a-3859-891e-03a53e9315b0

F Release Backlog

JIRA
serverOPNFV JIRA
columnskey,summary,assignee,status
maximumIssues1000
jqlQueryproject = STORPERF and type != Sub-task and Status != CLOSED and FixVersion = "6.0.0" ORDER BY priority DESC, updated DESC
serverId96acfcf2-db1a-3859-891e-03a53e9315b0

Key Project Facts

View Git file
pathINFO
repository-id55
languagetext
branchmaster

Test Cases

This is an outline of test cases. A specification will be written capturing actual tests and steps. And of course, the input to the test process will be determined by community participation.

Block Storage

Given the SNIA guidelines, testing of Cinder volumes or Glance ephemeral storage, regardless of back end driver.  StorPerf makes no attempt to read the OpenStack configuration to determine what drivers are being used.

  1. Preconditioning of defined Logical Block Address range
  2. Testing across each combination of: Queue Depths (1, 2, 8) and Block sizes (2KB, 8KB, 16KB)
  3. For each of 5 workloads: Four corners (100% Read/Seq, Write/Seq, Read/Random, Write/Random) and mixed (70% Read/Random).

Object Storage

This is planned for a future release.

Assuming an HTTP-based API, such as Swift for accessing object storage.

  1. Determine max concurrency of SUT with smaller data size (GET/PUT) tests by determining performance plateau
  2. Determine max TPS of SUT using variable block size payloads (1KB, 10KB, 100KB, 1MB, 10MB, 100MB, 200MB)
  3. Use 5 different GET/PUT workloads for each: 100/0, 90/10, 50/50, 10/90, 0/100
  4. Perform separate metadata concurrency test for SUT using List and Head operations

Especially looking for workload recommendations for testing in this area.

Metrics

Initially, metrics will be for reporting only and there will not by any pass/fail criteria. In a future iteration, we may add pass/fail criteria for use cases which are testing viability for known workload requirements.

Block Storage Metrics

The mainstays for measuring performance in block storage are fairly well established in the storage community, with the minimum being IOPS and Latency. These will be produced in report/tabular format capturing each test combination for:

  1. Average IOPS for each workload
  2. Throughput bandwidth.  Note that throughput data can also be calculated based on IOPS * block size.
  3. Avg Latency for each workload

Object Storage Metrics

This is planned for a future realase.

Object storage delivers different storage characteristics than block storage, and so the metrics used to charaterize it vary to some degree:

  1. Transactions per second (throughput can also be calculated from TPS * object size)
  2. Error rate
  3. Per-test average latency

Contributors

User List
groupsopnfv-gerrit-storperf-contributors

Emeritus Contributors

Committers

User List
groupsopnfv-gerrit-storperf-submitters

Emeritus Committers

Viewtracker