Calipso Project Proposal
- Proposed name for the project: Calipso
- Proposed name for the repository: calipso
Project “Calipso” illuminates complex virtual networking with real time operational state visibility in a highly distributed virtual infrastructure (like OpenStack and others).
Calipso provides visible insights using virtual topological representation and graphs, adds monitoring per object in the inventory to reduce error vectors and troubleshooting cycles for VIM operators and administrators.
Calipso interfaces with the virtual infrastructure (starting with OpenStack) through API, DB and CLI adapters, discovers the specific distribution/plugin version and based on that collects detailed data regarding running objects in the underlying workers on the different hosts. Calipso analyzes the inventory for inter-relationships and keeps them in a common data model. Calipso also represents the typologies in real-time (automatic updates per changes in VIM), monitors the related objects and analyzes some data for impact and root-cause analysis. This is done with the objective to lower and potentially eliminate complexity and lack of visibility from the VIM layer as well as to offer a common and coherent representation of all physical and virtual network components used under the VIM. Calipso is to work with different OpenStack flavors, plugins and installers.
Calipso is to save network admins discovery and troubleshooting cycles with *networking aspects* . Calipso helps estimate the impact of any micro failure in the infrastructure to support appropriate resolutions.
i.e. connectivity, topology and related stats – as well as their correlation.
In short - the project will focus on:
- Discovery : detailed inventory and inter-connection analysis, smart/logical and automated learning from the VIM, based on specific environment version/type etc.
- Visualization : represent the result of the discovery in brows able graph topology and tree.
- Monitoring of the discovered objects and inter-connections : use the discovered data to configure monitoring agents and gather monitoring results.
- Analysis : throughput, impact and root-cause analysis for troubleshooting.
for (3.) we are also planning to utilize the work done in Barometer.
The project also develops required enhancements to individual components in OpenStack like Neutron, Telemetry API and the different OpenStack monitoring agents.
Calipso focuses on scenarios, which require OpenStack maintenance and troubleshooting enhancements using operations dashboards.
Calipso is to support both OVS and FD.io/VPP as underlying virtual forwarders as well as several network services plugins like DHCP services, gateways etc.
Calipso builds a fully automated operations app for collecting information about physical and virtual networking components under the VIM. Calipso represents the different components in an easy to operate graphical way and integrates with monitoring and analytics for impact analysis and root cause analysis.
The scope will also include a plugin framework, message bus and API to extend the application to more plugins and use-cases. A high-level representation of Calipso is shown below:
Key work items include:
- Create a scanner/discovery module to grab data from VIM through API, DB and CLI.
- Create common DB with a model to capture common distribution and plugins.
- Create visualization module to interface in real-time with the DB and present the topological and relationships.
- Create monitoring module to grab status and statistics per object in the inventory.
- Create analysis module to run several algorithms against the discovered data.
- Enhancements to VPP and ODL drivers for OpenStack for seamless integration.
- Enhancements to ODL and/or OpenStack API to allow seamless integration.
- Enhancement to monitoring systems providing dedicated and new client-side checks.
- Testing (provide for Functest, Yardstick, etc. tests, consider tests which evaluate specific functionality of the application).
- Integration into installers, using micro-services architecture.
- Correlation with ‘Pinpoint’ at OPNFV for fault isolation and alarm/event correlation, providing discovery inventory, relationships and state/status for running the analysis offered by Pinpoint.
- Integration with ‘Bamboo’: Calipso can either be deployed “stand-alone” or be integrated with project Bamboo, in which case components of Bamboo are leveraged by Calipso (e.g. database, message broker, etc.).
- Correlation with ‘Vitrage’ at OpenStack, to research potential re-use of Calipso’s underlying data.
- OPNFV projects:
- Bamboo (currently in proposal stage)
- Testing projects (FuncTest, YardStick, …)
- Deployments tools / Installers
- Upstream projects (some examples):
- OpenStack : Neutron, Telemetry
- ODL Neutron Northbound
Committers and Contributors
- Project Lead: Koren Lev (firstname.lastname@example.org)
Scenarios using Calipso framework against several common OpenStack distributions and plugins: Composition, automated deployment, automated testing
Proposed Release Schedule
The project initial deliverable is expected to be available in time for the OPNFV “E” release.