Page tree
Skip to end of metadata
Go to start of metadata

Welcome to the VNF Event Stream (VES) Project Home

This project was approved May 31, 2016 based upon the VNF Event Stream project proposal.

Project description:

Objective:

This project will develop OPNFV platform support for VNF event streams, in a common model and format intended for use by NFV Service Providers (SPs), e.g. in managing VNF health and lifecycle. The project’s goal is to enable a significant reduction in the effort to develop and integrate VNF telemetry-related data into automated VNF management systems, by promoting convergence toward a common event stream format and collection system.

The VES doc source, code, and tests are available at:

Powerpoint intro to the project: OPNVF VES.pptx. A demo of the project (vHello_VES Demo) was first presented at OpenStack Barcelona (2016), and is being updated as the VES schema and other project code is developed.

VES Roadmap:

With the launch of ONAP, the upstream home for VES is now that project. Further development of the initial seed code for VES (AT&T's agent library and collector from the ECOMP Vendor/VF Event Listener (EVEL) project on github) will occur in ONAP as part of the DCAE (Data Collection, Analytics, and Events) project. These include the parts integrated into the VES project:

  • the "collector" (monitor.py), a python-based test/demo client for receiving, validating, and displaying events from VES agents
  • the "agent" (evel_demo.c), a C-based test/demo agent for mapping source telemetry to the VES event format
  • the VES data schema (ves_data_model.json)

Also leveraged by the current test (vHello_VES.sh) is a python-based collectd client (ves_plugin.py) developed in the Barometer project, which maps collectd source data to the VES event format. If/when ves_plugin.py will be upstreamed to ONAP DCAE is TBD.

The long-term roadmap for VES in OPNFV is TBD. Further development of the VES tests and other project goals, e.g. integration with OpenStack, SDNCs, etc will continue in OPNFV for the time being. 

Project Planning

Current Work Items

ItemDescription
VES Agent

Architecture/design for VES Agent and other functions integrated with it in the bare metal host or VDU. See VES Agent.

Modeling Framework

Analysis of modeling frameworks in which to establish a base data model for VES. Options currently being analyzed (see sub-pages for details):

Operational FrameworkAnalysis of frameworks enabling functional tests/demos for VES-modeled telemetry. An operational framework may be related to a modeling framework, or independent. It provides APIs to interact with devices via models, and protocol bindings over which event streams are established and controlled. See Operational Framework.
VES tests/demos

Development of VNF management test/demos based upon reference VNFs running on the OPNFV system.

VES talks at events

OpenStack Seattle Days (Seattle, 9/30/16 - Friday after the ODL Summit)

OpenStack Summit Barcelona

 

Background

In a typical Service Assurance (SA) Environment, the VNF Provider defines and provides the format and protocol for sending telemetry data to SPs responsible for managing VNF health and lifecycle. The standards for telemetry data come in many varieties and for each particular standard, the expected actions also differ from one VNF to another. As an example:

  • Fault Events can be conveyed via SNMP, 3GPP Corba, MTOSI, OSSJ etc. For each standard the actions can differ from one VNF to another (e.g. Critical Severity can be marked as 1 in one implementation, 5 for another).
  • For Measurement events, related to Key Performance Indicators (KPI)/Key Capacity Indicator (KCI), they can be in Comma Separated files (CSV) or XML. Within these file formats, events vary across implementations.

The result of a multitude of standards is that every time a new VNF or a new VNF release is introduced, implementer choice can require SPs to make changes to their SA support system that manages the VNFs. This results not only in added costs for SPs, but also in deployment delays.

Recommendations Summary:

We are recommending a Common Event Data Model:

  • A VNF Event Stream (VES) Common Data Model consisting of a Header and an Event Specific Block (Domain is specified in the header).
  • Additional name/value fields for extensibility.

Use of Restful API using JSON with Common Event Format for Telemetry Data (detailed definitions for Heartbeat. Fault, Measurements (KPI/KCI), Syslogs, Mobile Flow and with extensibility to others, i.e. Usage, Security, Signaling etc.).

Overall Goals / Use Cases:

As a service provider, I need a holistic, unified, modeled solution to monitoring physical and virtual resources, control plane systems that manage those resources, and the applications executing on those resources. I want to avoid having to implement such monitoring using a variety of data models and operational frameworks specific to each resource, control plane system, or application. I need such a monitoring system to address as many different types of monitoring goals as possible, e.g. fault, performance, policy, security, application-specific, and generic analytics collection for machine learning applications or other arbitrary purposes. I need such a monitoring system to be highly scalable, high performance, and tunable in order to optimize obtainable value and consumed resources.

Scope:

The project will develop and integrate into the OPNFV platform:

  • A Common Event Data Model (“VNF Event Stream”)
  • OPNFV platform support the VNF Event Stream, including:
    • A VES Agent that can collect the VNF Event Stream data from the VNF and deliver it to the VES Collector
      • This includes telemetry aggregation from bare metal through to application KPIs - including host OS telemetry, and infrastructure
    • A VES Collector that can consume the VNF Event Stream data, and store the data in a database backend
    • VES plugins for integration with OpenStack infrastructure services such as Monasca and Vitrage
  • A plan for upstreaming the project code

As input this project, AT&T will contribute a proposed specification and implementation of the VNF Event Stream framework developed for AT&T’s ECOMP (Enhanced Control, Orchestration, Management and Policy) platform, and described as part of the “Analytic Framework” support in the ECOMP white paper available at http://att.com/ecomp.

The following diagram illustrates the concept for the VES architecture and scope:

In the diagram above, examples of Other Management Functions" (outside the scope of VES) are illustrated in the following use case diagram:

Testability:

The project will support testability through:

  • VES collector support under all OPNFV base system installers, either as a base system deploy component or post-deploy installed components
  • Test cases to be supported through Functest and DoveTail
  • Reference VNF generating events, for lab testing per the event definitions

Documentation:

The project will produce:

  • Detailed step by step documentation for developing to and using the code and event specification.
  • Blueprint proposals for upstreaming the event library, collector, and specification.

Dependencies:

VES is a standardized event framework that can be used by OpenStack and OPNFV projects.

OPNFV projects that potentially benefit from the VES project:

OpenStack projects that potentially benefit from the VES project, or need to be considered in the VES design:

  • Monasca: a monitoring-agent plugin for VES can provide a new/converged source for Monasca events
  • Vitrage: a VES plugin can provide new data for the Graph DB of Vitrage
  • Ceilometer and AODH: a VES plugin can populate the ceilometer DB
  • Congress: Congress data source drivers can integrate additional data from Monasca, Vitrage, or directly
  • Tacker: (From Sridhar, Tacker PTL) I'd like to explore the possibility of introducing a VES "event collector / listener" in the Tacker side to write policies ( { event-trigger-action} ) based on the event-stream coming from the VNFs.
  • Fault_Genes-WG creation of an OpenStack failure mode taxonomy to be used from design to deployment of OpenStack

Other considerations noted during discussion of the proposal:

  • There are efforts underway in OpenStack to do log normalisation, efforts to do telemetry aggregation for Neutron in Ceilometer (e.g. due to scaling issues), and there are also common needs across VMs, containers, and bare metal for host OS telemetry from something like collectd, and the "standard" ELK stack for log streams.

  • This is an opportunity to ensure that whatever happens at the host and infrastructure layer is scalable in a way that fits the needs of VNFs, however if we focus on something as domain specific as a bus on top of the infrastructure with Yang models to define the object model for the messages, we may end up with something that will look strange to infrastructure management/OpenStack developers.
  • A fault (say, a performance indicator going outside of acceptable bounds) could happen anywhere in the stack, in the guest OS or application code (where you would see it), on the host (if there's a noisy neighbour, for example), on the bare metal (some kind of hardware fault), or even on the network (if there is some network component involved), and that you would want to be aggregating and correlating data from all these sources to do accurate root cause analysis on the origin of the fault and the appropriate mitigating action.
  • Re Fault_Genes-WG, this group is focused on "creation of an OpenStack failure mode taxonomy to be used from design to deployment of OpenStack". This would be important to consider in the model, certainly, similar to the other aspects mentioned re alignment with standards, OpenStack projects, etc.

  • Re data model convergence ("multiple efforts to normalise logs using different technologies "), since we are modeling the data to be collected, we have the chance to evolve that model as the standards and open source converge. But for now it may be difficult to pick a single "gold standard", and that really isn't the intent anyway. The intent of this project is to enable all the relevant data, in whatever format, to be delivered from the hosts/VNFs using an easily-integrated set of agent and collector code artifacts.

  • Re concern over creating another "event bus": this can mean various things. If the protocol/framework for eventing, Rabbit for example is a bus. Or the stream of events related to a particular purpose e.g. monitoring, which operates over a bus. VES is related to both, and is intended to address the very issues arising from having multiple ways to get analytics and fault info today: SNMP, CORBA, MTOSI (via JMS or REST), or the various agents/protocols developed/used by OpenStack as a VIM.

  • As we consider dependencies and opportunities for convergence and "standardization" (which typically in an open source world occurs after convergence, or the excessive success of one approach), we need to consider these in their context and re various forums/projects related to them:
    • application-specific data
    • bare metal / virtualization host-related data
    • measurements
    • alerts
    • faults
    • syslogs

Committers and Contributors:

Committers:

Contributors:

  • Prakash Ramchandran, (Futurewei Technologies Inc.)
  • Ifat Afek, Nokia
  •  (Contributor list is in development)

Planned deliverables:

The project will deliver specifications and code as described in the scope section, and supporting information:

  • Reference VNF generating events, for lab testing per the event definitions.
  • Detailed step by step documentation for developing to and using the code and event specification.
  • Blueprint proposals for upstreaming the event library, collector, and specification.

Proposed Release Schedule:

VES is targeted for the 4th OPNFV release (“D” river). For the Colorado release, an initial reference implementation will be developed and made available for testing on the OPNFV Colorado release platform, as well as by VNF implementers.

Key Project Facts:

Project Name: VNF Event Stream (VES)
Repo name: VES
Lifecycle State: Incubation
Primary Contact: Alok Gupta, AT&T
Project Lead: Alok Gupta, AT&T
Jira Project Name: VNF Event Stream
Jira Project Prefix: VES
mailing list tag: VES
Committers:

Note: "inactive" above refers to non-participation in commits/reviews in the last 3 months

Recent space activity

Space contributors

{"mode":"list","scope":"descendants","limit":"5","showLastTime":"true","order":"update","contextEntityId":6824388}

 

  • No labels