This project will develop OPNFV platform support for VNF event streams, in a common model and format intended for use by NFV Service Providers (SPs), e.g. in managing VNF health and lifecycle. The project’s goal is to enable a significant reduction in the effort to develop and integrate VNF telemetry-related data into automated VNF management systems, by promoting convergence toward a common event stream format and collection system.
The VES doc source, code, and tests are available at:
Powerpoint intro to the project: OPNVF VES.pptx. A demo of the project (vHello_VES Demo) was first presented at OpenStack Barcelona (2016), and updated for the OPNFV Summit 2017 (VES ONAP demo - see below for more info).
The latest project overview and roadmap forward is described in the talk from the 2017 OPNFV Summit: VNF Event Streaming Onboarding Telemetry Policies.pptx
With the launch of ONAP, the upstream home for VES is now that project. Further development of the initial seed code for VES (AT&T's agent library and collector from the ECOMP Vendor/VF Event Listener (EVEL) project on github) will occur in ONAP as part of the DCAE (Data Collection, Analytics, and Events) project. These include the parts integrated into the VES project:
Also leveraged by the current test (vHello_VES.sh) is a python-based collectd client (ves_plugin.py) developed in the Barometer project, which maps collectd source data to the VES event format. If/when ves_plugin.py will be upstreamed to ONAP DCAE is TBD.
The long-term roadmap for VES in OPNFV is TBD. Further development of the VES tests and other project goals, e.g. integration with OpenStack, SDNCs, etc will continue in OPNFV for the time being.
Architecture/design for VES Agent and other functions integrated with it in the bare metal host or VDU. See VES Agent.
Analysis of modeling frameworks in which to establish a base data model for VES. Options currently being analyzed (see sub-pages for details):
|Operational Framework||Analysis of frameworks enabling functional tests/demos for VES-modeled telemetry. An operational framework may be related to a modeling framework, or independent. It provides APIs to interact with devices via models, and protocol bindings over which event streams are established and controlled. See Operational Framework.|
Development of VNF management test/demos based upon reference VNFs running on the OPNFV system.
|VES talks at events|
OpenStack Seattle Days (Seattle, 9/30/16 - Friday after the ODL Summit)
OpenStack Summit Barcelona
In a typical Service Assurance (SA) Environment, the VNF Provider defines and provides the format and protocol for sending telemetry data to SPs responsible for managing VNF health and lifecycle. The standards for telemetry data come in many varieties and for each particular standard, the expected actions also differ from one VNF to another. As an example:
The result of a multitude of standards is that every time a new VNF or a new VNF release is introduced, implementer choice can require SPs to make changes to their SA support system that manages the VNFs. This results not only in added costs for SPs, but also in deployment delays.
We are recommending a Common Event Data Model:
Use of Restful API using JSON with Common Event Format for Telemetry Data (detailed definitions for Heartbeat. Fault, Measurements (KPI/KCI), Syslogs, Mobile Flow and with extensibility to others, i.e. Usage, Security, Signaling etc.).
As a service provider, I need a holistic, unified, modeled solution to monitoring physical and virtual resources, control plane systems that manage those resources, and the applications executing on those resources. I want to avoid having to implement such monitoring using a variety of data models and operational frameworks specific to each resource, control plane system, or application. I need such a monitoring system to address as many different types of monitoring goals as possible, e.g. fault, performance, policy, security, application-specific, and generic analytics collection for machine learning applications or other arbitrary purposes. I need such a monitoring system to be highly scalable, high performance, and tunable in order to optimize obtainable value and consumed resources.
The project will develop and integrate into the OPNFV platform:
As input this project, AT&T will contribute a proposed specification and implementation of the VNF Event Stream framework developed for AT&T’s ECOMP (Enhanced Control, Orchestration, Management and Policy) platform, and described as part of the “Analytic Framework” support in the ECOMP white paper available at http://att.com/ecomp.
The following diagram illustrates the concept for the VES architecture and scope:
In the diagram above, examples of Other Management Functions" (outside the scope of VES) are illustrated in the following use case diagram:
The project will support testability through:
The project will produce:
VES is a standardized event framework that can be used by OpenStack and OPNFV projects.
OPNFV projects that potentially benefit from the VES project:
OpenStack projects that potentially benefit from the VES project, or need to be considered in the VES design:
Other considerations noted during discussion of the proposal:
There are efforts underway in OpenStack to do log normalisation, efforts to do telemetry aggregation for Neutron in Ceilometer (e.g. due to scaling issues), and there are also common needs across VMs, containers, and bare metal for host OS telemetry from something like collectd, and the "standard" ELK stack for log streams.
Re Fault_Genes-WG, this group is focused on "creation of an OpenStack failure mode taxonomy to be used from design to deployment of OpenStack". This would be important to consider in the model, certainly, similar to the other aspects mentioned re alignment with standards, OpenStack projects, etc.
Re data model convergence ("multiple efforts to normalise logs using different technologies "), since we are modeling the data to be collected, we have the chance to evolve that model as the standards and open source converge. But for now it may be difficult to pick a single "gold standard", and that really isn't the intent anyway. The intent of this project is to enable all the relevant data, in whatever format, to be delivered from the hosts/VNFs using an easily-integrated set of agent and collector code artifacts.
Re concern over creating another "event bus": this can mean various things. If the protocol/framework for eventing, Rabbit for example is a bus. Or the stream of events related to a particular purpose e.g. monitoring, which operates over a bus. VES is related to both, and is intended to address the very issues arising from having multiple ways to get analytics and fault info today: SNMP, CORBA, MTOSI (via JMS or REST), or the various agents/protocols developed/used by OpenStack as a VIM.
Li Yuanzhen(email@example.com), ZTE
The project will deliver specifications and code as described in the scope section, and supporting information:
VES is targeted for the 4th OPNFV release (“D” river). For the Colorado release, an initial reference implementation will be developed and made available for testing on the OPNFV Colorado release platform, as well as by VNF implementers.
Project Name: VNF Event Stream (VES)
Repo name: VES
Lifecycle State: Incubation
Primary Contact: Alok Gupta, AT&T
Project Lead: Alok Gupta, AT&T
Jira Project Name: VNF Event Stream
Jira Project Prefix: VES
mailing list tag: VES
Li Yuanzhen(firstname.lastname@example.org), ZTE (inactive - email bounces)