Page tree
Skip to end of metadata
Go to start of metadata

Introduction

Power consumption is a key driver for NFV.

As an end user is interested to know which application is good/bad regarding power consumption and explains why he/she has to plug his/her smartphone every day, we would be interested to know which VNF is power consuming.

Power consumption is hard to evaluate empirically. it is however possible to collect information and leverage Pharos federation to try to detect some profiles/footprints.

In fact thnaks to CI, we know that we are running a known/deterministic list of cases. The idea is to correlate this knowledge with the power consumption to try at the end to find statistical biais.

High level architecture

The energy monitoring system in based on 3 software components (in blue on the previous picture):

  • Power info collector: poll server to collect instantaneous power consumption information 
  • Energy recording API + influxdb: On oone leg receive servers consumption and on the other, scenarios notfication. It then able to establish te correlation between consumption and scenario and stores it into a time-series database (influxdb)
  • Python SDK: A Python SDK using decorator to send notification to Energy recording API from testcases scenarios

 

Components

Power info collector

It collects instantaneous power consumption information and send it to Event API in charge of data storing.
The collector use different connector to read the power consumption on remote servers:

  • IPMI: this is the basic method and is manufacturer dependent.
    Depending on manufacturer, refreshing delay may vary (generally for 10 to 30 sec.)
    Supported manufacturer are:
    • HP
    • IBM
    • Dell
    • Nokia
    • Advantech
    • Lenovo
    • Huawei
  • RedFish: redfish is an industry RESTFUL API for hardware managment.
    Unfortunatly it's not yet supported by many manufucturer.
    Refer to your manufacturer documentation to see if your servers supports redfish API
    For now it have been successfully tested on:
    • HP
    • Dell
    • Huawei (E9000 class servers used in OPNFV Community Labs are  IPMI 2.0 compliant and use Redfish login Interface through Browsers supporting JRE1.7/1.8)
  •  ILO: HP RESTFULL API:
    This connector support as well 2.1 as 2.4 version of HP-ILO 

What's the best?

We've done some tests to estimate RedFish and IPMI accuracy in comparaison with physical Wattmeter on DELL servers.
According to those tests, IPMI is not very accurate but RedFish is.

This collector is installed "nearby" the POD to monitor and should have a public access to the event API to push the collected information.
Collector can monitor multiple PODs and starts a thread for each server to monitor 

 

 

Install server-collector

To run the server power consumption collector agent, you need to deploy a docker container locally on your infrastructure.

This container requires: 

  • Connectivy on the LAN where server administration services (ILO, eDrac, IPMI......) are configured and IP access to the POD's servers
  • Outgoing HTTP access to the Event API (internet)

 

Image creation

First, you want to build your own image:

Docker build
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/server-collector.dockerfile|docker build -t energyrecorder/collector -

You may also use pre-build image availble in dockerhub

Dockerhub image
bherard/energyrecorder-collector



Setup

Create local folder on your host for logs and config files:

 

Prepare your host environment
mkdir -p /etc/energyrecorder
mkdir -p /var/log/energyrecorder
 
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-logging.conf.sample > /etc/energyrecorder/collector-logging.conf
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/server-collector/conf/collector-settings.yaml.sample > /etc/energyrecorder/collector-settings.yaml

 

In /etc/energyrecorder check configuration for logging in a file named collector-logging.conf:

  • level for ```logger_root```
  • log file name and rotation parameters for ```handler_file_rotating```

 

cat /etc/energyrecorder/collector-logging.conf
[loggers]
keys=root

[handlers]
keys=console,file_rotating

[formatters]
keys=simple


[logger_root]
level=INFO
handlers=console,file_rotating
[handler_console]
class=StreamHandler
level=DEBUG
formatter=simple
args=(sys.stdout,)


[handler_file_rotating]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=simple
args=('/var/log/energyrecorder/server-collector.log', 'a', 100 * 1024 *1024 , 5,)

[formatter_simple]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

 

In /etc/energyrecorder create a configuration for the collector in a file named collector-settings.yaml

cat /etc/energyrecorder/collector-settings.yaml
PODS:
- environment: YOUR-SERVER-GROUP-NAME
  servers:
	# HP ILO Server (Gen9)
  - host: server-ip-or-name[:port]
	# At your choice
    id: server-id
    type: ilo
    user: ilo-user
    pass: ilo-user-password
    #polling internval in seconds
    polling_interval: 10
.........

Define the "PODS" section and their "servers" section according to the environement to monitor.

Note: The "environment" key should correspond to the pod name, as defined in the "NODE_NAME" environment variable by CI when running.

IMPORTANT NOTE: To apply a new configuration, you need to kill the running container an start a new one or restart the existing one (see bellow)

Run

To run the container, you have to map folder located on the host to folders in the container (config, logs)
The command given here assume that you are running the dockerhub image.

 

Docker run
docker run -d --name energy-collector \
           --restart=always \
           -v /etc/energyrecorder:/usr/local/energyrecorder/server-collector/conf \
           -v /var/log/energyrecorder:/var/log/energyrecorder \
           bherard/energyrecorder-collector
Restart container
docker restart energy-collector

 

Energy recording API

An event API to insert contextual information when monitoring energy (e.g. start Functest, start Tempest, destroy VM, ..)

It is associated with an influxDB to store the power consumption measures

It is hosted on a shared environment with the folling access points:

ComponentConnectivity
Energy recording API documentationhttp://energy.opnfv.fr/resources/doc/
influxDB (data)http://energy.opnfv.fr:8086

 

In you need, you can also host your own version of the Energy recording API (in such a cas, the Python SDK may requires a settings update)

If you plan to used the default shared API, following steps are not required.

Image creation

First, you want to build your own image:

Docker build
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/docker/recording-api.dockerfile|docker build -t energyrecorder/api -

You may also use pre-build image availble in dockerhub

Dockerhub image
bherard/energyrecorder-api

 

Setup

Create local folder on your host for logs, data and config files:

 

Prepare your host environment
mkdir -p /etc/energyrecorder
mkdir -p /var/log/energyrecorder
mkdir -p /var/lib/influxdb
 
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-logging.conf.sample > /etc/energyrecorder/webapp-logging.conf
curl -s https://raw.githubusercontent.com/bherard/energyrecorder/master/recording-api/conf/webapp-settings.yaml.sample > /etc/energyrecorder/webapp-settings.yaml

 

In /etc/energyrecorder check configuration for logging in a file named webapp-logging.conf

  • level for ```logger_root```
  • log file name and rotation parameters for ```handler_file_rotating```

 

cat /etc/energyrecorder/webapp-logging.conf
[loggers]
keys=root,api
[handlers]
keys=console,file_rotating
[formatters]
keys=simple
[logger_root]
level=DEBUG
handlers=console,file_rotating

[logger_api]
level=DEBUG
handlers=console,file_rotating
qualname=api
propagate=0

[handler_console]
class=StreamHandler
level=DEBUG
formatter=simple
args=(sys.stdout,)

[handler_file_rotating]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=simple
args=('/var/log/energyrecorder/webapp.log', 'a', 100 * 1024 *1024 , 5,)
[formatter_simple]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

 

 

In /etc/energyrecorder create a configuration for the collector in a file named webapp-settings.yaml  

cat /etc/energyrecorder/webapp-settings.yaml
INFLUX:
    host: "http://localhost:8086"
    db: "NRG"
    user: ""
    pass: ""
BIND: "0.0.0.0:8888"

Where:

  • host: URL for the API to connect influxDB (keep default value if you plan to use influxDB embed in container)
  • db: influx database name (keep default value if you plan to use influxDB embed in container)
  • user and pass: creds define at container run (see bellow)

 

 

IMPORTANT NOTE: To apply a new configuration, you need to kill the running container an start a new one or restart existing (see bellow)

Run

To run the container, you have to map folder located on the host to folders in the container (config, logs)

 

Docker run
docker run -d \
           --name energyrecorder-api \
           -p 8086:8086 \
           -p 8888:8888  \
           -v /etc/energyrecorder:/usr/local/energyrecorder/web.py/conf \
           -v /var/log/energyrecorder/:/var/log/energyrecorder \
           -v /var/lib/influxdb:/var/lib/influxdb \
           bherard/energyrecorder-api \
           admin-influx-username admin-password \
           readonly-influx-username user-password

 

with:

 

parameter namedescription
admin-influx-user-nameInflux user with admin grants to create
admin-passwordInflux password to set to admin user
readonly-influx-user-nameInflux user with readonly grants to create
user-passwordInflux password to set to readonly user

 

NOTE: Local folder /var/lib/influxdb is the location web influx data are stored. You may used anything else at your convience. Just remember to define this mapping properly when running the container.

Restart container
docker restart energyrecorder-api

 

Power consumption Python SDK

a Python SDK - almost not intrusive, based on python decorator to trigger call to the even API

the code has been sumitted, it shall be merged and updated (url of the envt API in the config file + python decorators)

it is currently hosted in Functest repo but if other projects adopt it, a dedicated project could be created and/or it could be hosted in Releng.

How to use the SDK

Use SDK
# import the energy library
import functest.energy.energy as energy 


# Notify that you want power recording in your testcase
@energy.enable_recording
def run(self):
    self.do_some_stuff1()
    self.do_some_stuff2()
 

 

 If you want to register additional steps during the scenarios you can to it in 2 diffferent way:

Notify steps
# notifiy step on method defintion
@energy.set_step("step1")
def do_some_stuff1(self):
...
@energy.set_step("step2")
def do_some_stuff2(self):
...
 
# notifiy step directly from code
@energy.enable_recording
def run(self):
    Energy.set_step("step1")
    self.do_some_stuff1()


    Energy.set_step("step2")
    self.do_some_stuff2()

 

 

SDK Setting

Settings delivered in the project git are ready to use and assume that you'll use the sahre energy recording API.

If you want to use an other instance, you have to update the key "energy_recorder.api_url" in <FUNCTEST>/functest/ci/config_functest.yaml" by setting the proper hostname/IP

 

InfluxDB structure

The influxDB datable name is NRG, it contains 2 measurements:

  • PowerMeasurement: Located in default retention policy. This is raw servers consumptions

  • RunningScenario: Located in a retention policy named running_scenarios_rp. This is the history of ran and running scenarios.

The duration of the default retention policy is 3 weeks (data are dropped after the delay)
The duration of  the running_scenarios_rp retention policy is 1 day (data are dropped after the delay)

Those duration may be changed according to project requirements.


PowerMeasurement structure

Filed/TAGComment
timetimestamp of measurement
environmentPOD Name
hardwareserver on with power was collected
powerconsumption in Watts
scenarioRunning scenario at this time
stepScenario's step

 

Ex:

SELECT  * FROM PowerMeasurement WHERE environment='POD2' GROUP BY scenario

RunningScenario structure

Field/TAGComment
timetimestamp of measurement
envrionmentPOD Name
scenarioRunning scenario
stepCurrent step in scenario
startednot significant

 

Ex:

SELECT * FROM running_scenarios_rp.RunningScenario WHERE envrionment='MYPOD'

Integration in CI

blabla

Result display

Here is an example of result comming from E/// POD1. This sequence represents a complete test suite from CI

You may connect http://energy.opnfv.fr:3000 (admin/admin) for more results.

Analytics

blabla

  • No labels

2 Comments

    1. The github repository was re-restructured but the wiki was not updated. It's now fixed.

      Thanks for your comment