Page tree
Skip to end of metadata
Go to start of metadata


DescriptionContainerizing VSPERF: Automated Deployment, Control and Results Management
Status

PROJECT COMPLETED 

Difficulty

MEDIUM 

Description:

VSPERF provides an automated test-framework and comprehensive test suite based on industry standards for measuring data-plane performance which includes switching technologies with physical and virtual network interfaces. For more details see the project Wiki site https://wiki.opnfv.org/display/vsperf . 

VSPERF currently lacking in three main aspects:

  1. Automated deployment 
  2. Remote management - Command and Control
  3. Comprehensive results management.

The First feature would help in deploying VSPERF and appropriate Traffic-generators. The second feature  would help in automating the dataplane performance tests - similar to other testing projects. Realizing this, and the first feature, includes creating a container that will run on a Jumphost. This container will be deployed, configured and controlled through Jenkins job. The container will expose gRPC/Rest Interface for user to control and configure. The container in turn will deploy vsperf on the configured host, runs test, collects and publishes result. One can reuse existing containers approach from Yardstick, Functest and Dovetail.

The second feature would support analysis such as - correlation analysis, Trend analysis, anomaly detection, etc. Comprehensive results management includes (a) Database to store time-dependent data of varying granularity - from live-results (in milliseconds) to end-of-test-results in minutes. This database should also store events generated by VSPERF.  (b) Web-application and a dashboard to analyze and showcase the results (c) APIs for other to consume the results. 

The overall architecture and scope of the project can be depicted as below:

Additional Information:

The first version of the containers can be found here:

https://cloud.docker.com/u/vsperf/repository/docker/vsperf/vsperfc 

https://cloud.docker.com/u/vsperf/repository/docker/vsperf/elk 

Desirable Skills:

    • Python
    • Containers
    • Graphing
    • HTML, CSS and JavaScript rendering of content

Expected Outcome:

  • VSPERF command and control container.
  • VSPERF-Client.
  • VSPERF Results container with custom analytics

Difficulty:

 MEDIUM 

Desired project timeline/completion date:

M1: Custom Results Visualization

M2: Custom Results Analytics

M3: Automated Traffic generator Deployment.

M4: Advanced analysis of the results. 

M5: Advanced command control - Status check, test-restart, etc.

Mentor(s) & contact info:

Intern:

Reports:

Currently all my internship work is submitted on this github repository on containerization branch:  https://github.com/opensource-tnbt/vsperf-docker/tree/containerization

Before using VSPERF client and VSPERF containers user must run the prepare.sh script which will prepare their local environment.

locate vsperf-docker/prepare.sh and run: bash prepare.sh

VSPERF Client:

VSPERF client is a simple python application, which can be used to work with interactive deploy and testcontrol containers.

Description:

VSPERF client is used for both set-up of DUT-Host and TGen-Host as well as run the multiple tests. User can perform different operations by selecting the available options and their sub-options.

VSPERF client provides following options for User.

* Establish Connections
This option allow user to initialize the connections.
[0]Connect to DUT Host: It will establish connection with DUT-HOST.
[1]Connect to Tgen Host: This option will establish connection with TGen-Host.

* Installation
After connection has established, user can perform installation for test environment setup.
[0]Install VSPERF : This option will first check either vsperf is installed on DUT or not. If VSPERF is not installed, then it will perform VSPERF installation process on DUT-Host
[1]Install TGen: This is option will check either t-rex is installed on Tgen or not. If t-rex is already installed then it will also check either is working fine or not. If there is not t-rex installed on Tgen then this option will install t-rex on Tgen-HOST
[2]Install Collectd: This is option will install collectd on DUT-Host.

* Upload Configuration Files
Once set-up is done, User can upload configuration files.
[0]Upload TGen Configuration File: It will upload trex_cfg.yaml configuration file on Tgen.[User can either specify path in vsperfclient.conf or externally provide path for the trex_cfg.yaml]
[1]Upload Collectd Configuration File: This is option is use to upload collectd configuration file.

* Manage DUT-System Configuration
[0]DUT-Host hugepages configuration: This option allow User to manage hugepages of DUT-Host. [User need to provide value for HpMax and HpRequested in vsperfclient.conf]
[1]Check VSPERF dependencies: Using this option user can check library dependencies on DUT-Host.

* Run Test
[0]Upload Test Configuration File : This option will upload the vsperf test configuration file.
[1]Perform Sanity Checks before running tests : This option has certain sub-options , user must perform all sanity checks before running test. User does not able to start the Vsperf test until all sanity checks satisfied. This sanity check option contains following sub-options like to check VSPERF installed correctly, check Test Config's VNF path is available on DUT-Host, check NIC PCIs is available on Traffic Generator, check installed Collectd, connection between DUT-Host and Traffic Generator Host, check CPU-allocation on DUT-host.
[2]Check if DUT-HOST is available : User can check if DUT-Host is available for Test or not. If DUT-Host available for performing Vsperf user can go ahead and start performing test.
[3]Start TGen : This option will start t-rex traffic generator for test.
[4]Start Beats : This option will start beats on DUT-Host
[5]Start Test : If all the sanity check have performed and traffic generator is running then and then this option will start the vsperf test. Whatever test is defined in vsperfclient.conf will perform using this option. User can also perform multiple tests.

* Test Status
[0]Test status : Using this option, After running test user can check whether their test completed successfully or failed. If user is running multiple test then they can identify the failed test name using this option.
[1]Get Test Configuration file from DUT-host: User can also able to read the test configuration file content they uploaded.

* Clean-Up
[0]Remove VSPERF: This option will completely remove the vsperfenv on DUT-Host
[1]Terminate VSPERF: This option will keep vsperfenv on DUT-Host. If there is any process still running related with the vsperf then this option will terminate all those processes like ovs-vswitchd,ovsdb-server,vppctl,stress,qemu-system-x86_64.
[2]Remove Results from DUT-Host : This is option will remove all the test results located in /tmp folder.
[3]Remove Uploaded Configuration Files: This option will remove all uploaded test configuration file
[4]Remove Collectd: This option will uninstall collectd from the DUT-Host
[5]Remove Everything: This option will execute all the options listed above.


How To Use:

Prerequisites before running vsperf client

User must need to install grpcio, grpcio-tools and configparser for python3 environment. User can run the Vsperf client script with python3.

User have to provide credentials for DUT-Host and TGen-host, value for the number of hugepages(HpMax and HpRequested) and vsperf test name under Testcase section.

For Uploading the configuration files user have two choices
1.User can provide path in vsperfclient.conf file.
2.When running client ask for path use specific path where configuration file exist.

T-rex and collectd configuration file should be saved with trex_cfg.yaml and collectd.conf respectively. Test configuration file can be saved according to your preferences and reliability.

Firstly, user **must have to start the deployment/interactive container and testcontrol/interactive container which will run the servers on localhost port 50051 and 50052** respectively for vsperf client.

Run vsperf client:
Locate and run the vsperf_client.py with python3.

[By default user does not require to make any changes in proto/vsperf.proto file to run the vsperf_client.py. However, if user want to add more APIs in current client they have to define them in proto/vsperf.proto file and use this command to make them usable for vsperf_client.py **python3 -m grpc_tools.protoc -I./ --python_out=. --grpc_python_out=. vsperf.proto**]


VSPERF Containers:

Architecture diagrams of the VSPERF-Containers.

deployment:

User have two choices for deployment, auto and interactive.

1. auto
^^^^^^^^^^^^^^^^^^^^^
This auto deployment container will do everything related with VSPERF set-up automatically. For example of installation of VSPERF, T-rex and collectd, upload T-rex and collectd configuration file on DUT-Host, start the t-rex traffic generator. If user find any failed or critical message in outputs, user should first correct those configuration and then run the testcontrol auto container to perform test.
Before installing vsperf and t-rex, container verify the installed vsperf and t-rex and if both are already installed then container will not repeat this process.
User can modify the t-rex(trex_cfg.yaml)and collectd(collectd.conf) configuration files which will upload automatically.

Pre-Deployment Configuration
******************
1.User have to provide all the DUT-Host and Traffic generator related credentials and IP address in list.env.
2.Provide value for HUGEPAGE_MAX and HUGEPAGE_REQUESTED in list.env.
3.Provide option for sanity check YES or NO in list.env file which is optional.

Build
******************
Use **docker-compose build** command to build the container.

Run
******************
Run the container's service with **docker-compose run deploy** command.

Post-Deployment Configuration
******************
After running this container, if user find any failed or critical message in outputs then user should first correct those configuration and then run the testcontrol auto container to perform test.

2. interactive
^^^^^^^^^^^^^^^^^^^^^
This interactive container must run before using the vsperf client. It will just run the container and start the server for the vsperf client. Deployment interactive container handle all vsperf set-up related commands from vsperf client and perform the operation. Deployment interactive container running server on localhost port 50051.


Build
******************
Run **docker-compose build** command to build the container.

Run
******************
Run the container's service with **docker-compose up deploy** command.

Once the server is running user have to run testcontrol interactive container and then user can run the vsperf client.


testcontrol:

For testcontrol also user have two choices , auto and interactive.

1. auto
^^^^^^^^^^^^^^^^^^^^^
This auto testcontrol container will perform test automatically on DUT-Host. This container also performing sanity checks automatically. User will also able to get test-status for all tests. If all sanity check doesn't satisfy then test will not run and container gracefully stopped. User can modify the vsperf.conf file which will be upload on DUT-Host automatically by container and used for performing the vsperf test.

Pre-Deployment Configuration
******************
1.User have to provide all the DUT-Host credentials and IP address of TGen-host in list.env.
2.Provide name for VSPERF_TESTS and VSPERF_CONFFILE in list.env.
3.Provide option for VSPERF_TRAFFICGEN_MODE and CLEAN_UP [YES or NO] in list.env file.

Build
******************
Run **docker-compose build** command to build the container.

Run
******************
Run the container's service with **docker-compose run testcontrol** command.

User can observe the results and perform the another test by just changing the VSPERF_TEST environment variable in list.env file.


2. interactive
^^^^^^^^^^^^^^^^^^^^^
This interactive testcontrol container must run before using the vsperf client. It will just run the container and start the server for the vsperf client. This testcontrol interactive container handle all the test related commands from vsperf client and do the operations. Testcontrol interactive container running server on localhost port 50052.

Build
******************
Run **docker-compose build** command to build the container.

Run
******************
Run the container's service with **docker-compose up testcontrol** command.

After running this interactive containers user can use the vsperf client.

Page viewed times