Page tree
Skip to end of metadata
Go to start of metadata

Requirement

Common

1.0

OVS DB Multi-instance support

 

2.0

SNMP Support (w/o TRAPs supported)

 

OVS events plugin

1.0

Export OVS interface link status as a metrics and generate a notification in case of the link status change

 

2.0

The link status change event must be detected within 10 ms

 

3.0

Send notification feature should be configurable through configuration file (enable/disable)

 

4.0

A notification should be generated in case of OVS DB connection lost (configurable).

 

5.0

The sending of first notification (generated when a new bridge/interface is added) should be configurable (enable/disable)

 

6.0

Interval to collect OVS interface link status metrics should be configurable

 

7.0

List of monitored interfaces should be configurable. If not set, all interfaces are monitored.

 

8.0

The dispatch value (interface link status) should be configurable via plugin configuration file (enable/disable).

 

OVS stats plugin

1.0

Export OVS statistics from regular Linux interfaces and DPDK interfaces (including all error and discard statistics)

 

2.0

Statistics are available for each interface in bridge (including internal)

 

3.0

List of interfaces to monitor should be configurable. If not set, all interfaces are monitored.

 

4.0

Collection interval should be configurable.

 

 

 

Overview

Open vSwitch (Ovs)

 

Open vSwitch, sometimes abbreviated as OvS, is a production-quality open-source implementation of a distributed virtual multi-layer switch. The main purpose of Open vSwitch is to provide a switching stack for hardware virtualisation environments, while supporting multiple protocols and standards used in computer networks.

OvS Database Management protocol

 

In virtualized server environments, a virtual switch (vSwitch) is typically used to forward traffic between different virtual machines (VMs) on the same physical host and between VMs and the physical network. Open vSwitch is designed to be used as a vSwitch in such environments. Open vSwitch (OVS) is open to programmatic extension and control using OpenFlow and the OVSDB (Open vSwitch Database) management protocol.

The OvS DB management protocol is based on RFC7047. It uses JSON (RFC4627) for its wire format and is based on JSON-RPC version 1.0. The schema of the Open vSwitch database is documented in http://openvswitch.org/ovs-vswitchd.conf.db.5.pdf. Based on RFC7047 specification, the OvS DB management protocol should use JSON-RPC TCP transport. IANA has assigned TCP port 6640 for this protocol. The UNIX transport also can be used as is supported by current OvS implementation (OvS DB server application).

OvS events plugin

The OvS link plugin monitors the link status of OvS connected interfaces and dispatches the state of the interfaces through collectd notification mechanism whenever the link state change occurs. This plugin uses OvS DB to receive the link state change notification.

Design/Development Teams

OvS plugin main components

 


Since more than one OvS plugin is going to be developed, it was decided to split the plugin implementation into two main components:

Figure 1. OvS plugin main components

 

OvS plugin

  • This component implements the specific OvS plugin logic which uses OvS utils component to communicate with an OvS database.

OvS utils

  • This component implements some part of the OvS DB transport communication specified by RFC7047 and provides API for OvS collectd plugins.

 

OvS utils module features:

  • Connect/disconnect to/from OvS DB server;
  • Recovery connection mechanism in case of OvS DB connection lost;
  • Subscription mechanism to OvS DB table events like initial/insert/modify/delete table row;
  • API to send custom JSON request to OvS DB server and receive the result (poll some table data, get table information etc.);
  • Handling of “echo” request from OvS DB server to verify the liveness of the connection;
  • Sending OvS DB connection lost notification.

 

 

OvS plugin interaction

 

The general overview of OvS plugin interaction with OvS database is described on figure 2.

Figure 2. OvS plugin interaction

 

OvS plugin/OvS utils internal architecture

 

Figure 3. OvS plugin/utils internal architecture

 

The Poll worker thread listens for incoming OVS DB raw data, parses and validate the JSON data (using JSON reader component) and takes care of calling corresponding handlers. If the OVS DB connection is lost the poll worker will send an event to event worker and try to establish the connection again (repeating in 1 second intervals). Handled JSON requests (implementation based on RFC7047):

  • ECHO request. OVS DB server sends this request periodically to verify the liveness of a database connection. When such request is received by pool worker, the “ECHO” handler is called and appropriate reply is send to the server.
  • Update request. This request is sent by OVS DB server to the plugin to report changes in tables that are being monitored. Once such request is received by poll worker, the appropriate handler registered by plugin is called.

The Event worker thread listens for incoming events (like establish/loss OVS DB connection) from poll worker thread and invokes an appropriate plugin handler.

 

OvS DB multi-instance support

 

The OvS DB multi-instance support is covered by OvS utils API which allows any OvS plugin to open multiple TCP/UNIX connections to different OvS DB instances. The general overview of OvS multi-instance support is described on figure 4.

Figure 4 Multi-instance support

Please note that OvS utils doesn’t create any addition threads to handle multiple OvS DB connections. It uses same number of threads (two) for all OvS DB instances.

 

Third-party libraries

 

To implement JSON routine (parsing, generating JSON) specified in OvS management protocol the YAJL library (https://lloyd.github.io/yajl/) is used. This library is already integrated into collectd which makes the library easier to use in new OvS plugin.

 

 

Limitations

 

  • No known limitation

 

 

OvS events plugin

 

The collectd OvS events plugin state diagram is described on figure 4. It uses OvS utils API (by default) subscribes to link status updates in OVS DB and to receive the interface status change event from OvS database. When connection to OvS database is lost, the proper collectd notification is generated by the plugin.

 

The  collectd OvS events plugin can also be used to poll the OvS interface link status as a statistic but this is not a default behaviour and not recommended from a performance perspective.

Figure 5 OvS events plugin state diagram

Plugin configuration

 

The following configuration options should be supported by ovs_events collectd plugin:  

Name

Description

Comment

Interval

The interval within which to retrieve statistics on monitored interfaces in seconds.

Interval option is supported by collectd and is defined in <LoadPlugin> block. No additional functionality should be developed in ovs_events plugin to support this option.

SendNotification

If set to true, OVS link notifications (interface status and OVS DB connection terminate) are sent to collectd. Default value is true.

true/false

Instance

This block describes the OVS DB server configuration. The name of the instance must be provided by the user. If no instance is configured, the default instance with default configuration is used.

Block.

E.g.: 

<Instance name>  … </Instance>

 

Instance block configuration options:

Name

Description

Comment

Address

The address of the OVS DB server JSON-RPC interface used by the plugin. To enable the interface, OVS DB daemon should be running with "--remote=ptcp:" option. See ovsdb-server(1) for more details. The option may be either network hostname, IPv4 numbers-and-dots notation or IPv6 hexadecimal string format. Defaults to 'localhost'.

 

Port

TCP-port to connect to. Either a service name or a port number may be given.  Defaults to 6640.

true/false

Socket

The UNIX domain socket path of OVS DB server JSON-RPC interface used by the plugin. To enable the interface, the OVS DB daemon should be running with "--remote=punix:" option. See ovsdb-server(1) for more details. If this option is set, Address and Port options are ignored.

 

Interfaces

List of interface names to be monitored by this plugin. If this option is not specified or is empty then all OVS connected interfaces on all bridges are monitored.  Default: empty (all interfaces on all bridges are monitored)

e.g: “br0”, “et0”

DispatchValues

Dispatch the OVS DB interface link status value with configured plugin interval.  Defaults to false. Please note, if SendNotification and DispatchValues options are false, no OVS information will be provided by the plugin.

true/false

NotifyInterfaceAdd

Currently when a new interface is added a notification will show that the link status has change to down. This option allows the user to suppress this notification by setting NotifyInterfaceAdd to false.

true/false

 

 

Here is an example of the plugin configuration section of collectd.conf file:

LoadPlugin ovs_events     
<Plugin "ovs_events">
          <Instance default>
            Port 6640
            Address "127.0.0.1"
            Socket "/var/run/openvswitch/db.sock"
            Interfaces "br0" "veth0"
            DispatchValues false
          </Instance>
          SendNotification true
</Plugin>

Supported metrics and events

 

The ovs_events  plugin monitors the link status of Open vSwitch (OVS) connected interfaces, dispatches the values to collectd and sends the notification whenever the link state change occurs. This plugin uses OVS database to get a link state change notification. The ovs_events plugin should collect the following metrics and notifications:

 

Plugin

Type

Type Instance

Description

Comment

ovs_events

gauge

link_status

Link status of the OvS interface: UP or DOWN

 

 

 

Notifications:

Name

Type

Type Instance

Severity

Description

ovs_events

gauge

link_status

Warning on Link Status Down/OKAY on link Status Up

Link status of the OvS interface: UP or DOWN
Severity will be configurable by the end user

 

 

 

 

SNMP support

 

The SNMP OVS interface link status is provided by standard IF-MIB (http://www.net-snmp.org/docs/mibs/IF-MIB.txt). Please note, that information about ports that are not supported by standard SNMP (e.g. like DPDK ports) aren’t available there.

 

 

 

OvS stats plugin

 

The plugin collects a statistics of OVS connected interfaces/ports/bridges. It uses OVS Utils API to get the statistics from OVS database. The internal OVS stats plugin state diagram is described on figure.

Figure 6 OvS stats plugin state diagram

 

Plugin configuration

 

The following configuration options should be supported by ovs_stats collectd plugin:  

Name

Description

Comment

Interval

The interval within which to retrieve statistics on monitored interfaces in seconds.

Interval option is supported by collectd and is defined in <LoadPlugin> block. No additional functionality should be developed in ovs_stats plugin to support this option.

Instance

This block describes the OVS DB server configuration. The name of the instance must be provided by the user. If no instance is configured, the default instance with default configuration is used.

Block.

E.g.: 

<Instance name>  … </Instance>

 

Instance block configuration options:

Name

Description

Comment

Address

The address of the OVS DB server JSON-RPC interface used by the plugin. To enable the interface, OVS DB daemon should be running with "--remote=ptcp:" option. See ovsdb-server(1) for more details. The option may be either network hostname, IPv4 numbers-and-dots notation or IPv6 hexadecimal string format. Defaults to 'localhost'.

 

Port

TCP-port to connect to. Either a service name or a port number may be given.  Defaults to 6640.

 

Socket

The UNIX domain socket path of OVS DB server JSON-RPC interface used by the plugin. To enable the interface, the OVS DB daemon should be running with "--remote=punix:" option. See ovsdb-server(1) for more details. If this option is set, Address and Port options are ignored.

 

Bridges

List of OVS bridge names to be monitored by this plugin. If this option is omitted or is empty then all OVS bridges will be monitored.)

e.g: “br0”, “br_ext”

 

 

Here is an example of the plugin configuration section of collectd.conf file:

LoadPlugin ovs_stats     
<Plugin "ovs_stats">
          <Instance default>
            Port 6640
             Address "127.0.0.1"
            Socket "/var/run/openvswitch/db.sock"
            Bridges "br0" "br_ext"
          </Instance>
</Plugin>

List of OVS statistics supported by the plugin

 

rx_packets

Number of received packets

rx_bytes

Number of received bytes

tx_packets

Number of transmitted packets

tx_bytes

Number of transmitted bytes

rx_dropped

Number of packets dropped by RX

rx_frame_err

Number of frame alignment errors

rx_over_err

Number of packets with RX overrun

rx_crc_err

Number of CRC errors

rx_errors

Total number of receive errors, greater than or equal to the sum of the above

tx_dropped

Number of packets dropped by TX

collisions

Number of collisions

tx_errors

Total number of transmit errors, greater than or equal to the sum of the above

 

For dpdk ports in openvswitch v 2.6 and above extended statistics (RFC2819) is available:

 

  rx_1_to_64_packets

The total number of packets (including bad  packets) received that were 64 octets in length (excluding framing bits but including FCS octets).

  rx_65_to_127_packets

The total number of packets (including bad packets) received that were between 65 and 127 octets in length inclusive (excluding framing bits but including FCS octets).

  rx_128_to_255_packets

The total number of packets (including bad packets) received that were between 128 and 255 octets in length inclusive (excluding framing bits but including FCS octets).

  rx_256_to_511_packets

The total number of packets (including bad packets) received that were between 256 and 511 octets in length inclusive (excluding framing bits but including FCS octets).

  rx_512_to_1023_packets

The total number of packets (including badpackets) received that were between 512 and 1023 octets in length inclusive (excluding framing bits but including FCS octets).

  rx_1024_to_1522_packets

The total number of packets (including bad packets) received that were between 1024 and 1518 octets in length inclusive (excluding framing bits but including FCS octets).

  rx_1523_to_max_packets

The total number of packets (including bad packets) received that were between 1523 and max octets in length inclusive (excluding framing bits but including FCS octets).

  tx_1_to_64_packets

The total number of packets transmitted that were 64 octets in length.

  tx_65_to_127_packets

The total number of packets received that were between 65 and 127 octets in length inclusive

  tx_128_to_255_packets

The total number of packets received that were between 128 and 255 octets in length inclusive

  tx_256_to_511_packets

The total number of packets received that were between 256 and 511 octets in length inclusive

  tx_512_to_1023_packets

The total number of packets received that were between 512 and 1023 octets in length inclusive

  tx_1024_to_1522_packets

The total number of packets received that were between 1024 and 1518 octets in length inclusive

  tx_1523_to_max_packets

The total number of packets received that were between 1523 and max octets in length inclusive

  tx_multicast_packets

The number of good packets transmitted that were directed to a multicast.

Note that this number does not include packets directed to the broadcast address

  rx_broadcast_packets

The total number of packets (including bad packets, broadcast packets, and multicast packets) received.

  tx_broadcast_packets

The number of good packets transmitted that were directed to the broadcast address.

  rx_undersized_errors

The total number of packets received that were less than 64 octets long (excluding framing bits, but including FCS octets) and were otherwise well formed.

  rx_oversize_errors

The total number of packets received that were longer than max octets (excluding framing bits, but including FCS octets) and were otherwise    well formed.

  rx_fragmented_errors

The total number of packets received that were less than 64 octets in length (excluding framing bits but including FCS octets) and had either a bad Frame Check Sequence (FCS) with an integral number of octets (FCS Error) or a bad FCS with a non-integral number of octets (Alignment Error).

Note that it is entirely normal for etherStatsFragments to increment.  This is because it counts both runts (which are normal occurrences due to collisions) and noise hits

  rx_jabber_errors

The total number of jabber packets received that had either a bad Frame Check Sequence (FCS) with an integral number of octets (FCS Error) or a bad FCS with a non-integral number of octets (Alignment Error).

 

SNMP Support

OvS stats plugin rely on standard Linux SNMP support, thus information about DPDK ports is not available.OvS multi instance support

 

The collectd OVS multi instance statistics support is achieved using ovs_pmd_stats application and collectd exec plugin. The interaction between those applications is described on Figure 7.

Figure 7 Collectd exec plugin/ovs_pmd_stats interaction

The collectd exec plugin forks off the ovs-pmd-stats application and reads all counters from its STDOUT (in specific format described in collectd exec plugin documentation). In turn, ovs_pmd_stats application connects to ovs-vswitch instance using bidirectional binary UNIX socket. After connection is established, JSON RPC message is sent that requests response to the OVS command “dpif-netdev/pmd-stats-show”. In case response message is not full or corrupted, it will be ignored and corresponding error reported. Otherwise, it will be parsed by ovs_pmd_stats application and each counter/value pair will be put to STDOUT in specific format suitable for collectd exec plugin.

One argument is required for ovs-pmd-stats application to get counters from ovs-vswitch:

  • “--socket-pid-file” - The ovs-vswitchd.pid file location to connect to ovs-vswitch daemon.

ovs_pmd_stats application puts counters in the following formats depending to the OVS configuration:

  • For non-core threads (main thread):
    PUTVAL <host-id>/ovs_pmd_stats-main_thread/counter-<counter-name> N:<counter-value>
  • For pmd threads:
    PUTVAL <host-id>/ovs_pmd_stats-pmd_thread_numa_id_<numa-id>_core_id_<core-id>/counter-<counter-name> N:<counter-value>

 

Considerations

Configuration Considerations

Deployment Considerations

API/GUI/CLI Considerations

Equivalence Considerations

Security Considerations

Alarms, events, statistics considerations

Redundancy Considerations

Performance Considerations

Not part of Telemetry so performance is Not Applicable

Testing Consideration

For ovs_events, the timing interval requirement needs to be taken into consideration when conducting tests.

The Tests should be carried out on a system underload as well as a relatively idle system.

Other Considerations
ovs-vswitchd.pid.ctl unix socket file can be used to monitor vswitch liveliness, killing ovs-vswitchd process removes “ovs-vswitchd.pid.ctl” from “/usr/local/var/run/openvswitch” directory.

If a bridge with an interface is created and traffic run through it, after killing the switch no changes were detected in traffic flow and the bridge is still present.
(Not applicable to OvS with DPDK, when flow times out it will no longer be functional)

Impact

The following table outlines possible impact(s) the deployment of this deliverable may have on the current system.

 

Ref

System Impact Description

Recommendation / Comments

1

 

 

Key Assumptions

The following assumptions apply to the scope specified in this document.

 

Ref

Assumption

Status

1

 

 

Key Exclusions

The following exclusions apply to the scope discussed in this document.

 

Ref

Exclusion

Status

1

 

 

Key Dependencies

The following table outlines the key dependencies associated with this deliverable.

 

Ref

Dependency

Status

1

Open vSwitch

 

2

Lib YAJL (version 2 or later)

 

3

 

 

4

 

 

 

 

 

 

 

 

 

 

 

 

 


 [TM1]You mean for the OVSDB protocol? Or in general, what is the context of this statemtent

 [MV2]Right, I mean OVS DB protocol. Updated the sentence.

 

  • No labels