Anil RaoUpdated October 14, 2021.
It is no surprise that OpenStack has evolved into a widely adopted cloud management framework. As it hurtles on a trajectory of rapid growth, a new breed of demands are making themselves felt – demands that a mature platform of this nature and scope must satisfy. One such requirement is the ability to monitor traffic flowing in the myriad of virtual networks found in an OpenStack datacenter.
Conceptually speaking, the monitoring process involves placing tap devices at appropriate locations within the network infrastructure and attaching traffic analyzers to them. These analyzers can then see the same packets passing through those network segments, as if they were also in-line. A logical tap device can be easily constructed using the port-mirroring function of a network switching element, which makes it possible to have a copy of the packets traversing one or more switch ports delivered to another port on the same switch. This capability is supported by almost all modern physical and virtual switches. So, why is it [still] not possible to monitor the activity in OpenStack virtual networks?
The answer to this question lies in understanding two important architectural characteristics of cloud based virtualization platforms, namely multi-tenancy and location independence. The former allows available resources and services to be shared among different groups of users. Each group – known as a tenant – is provided with an environment that is completely isolated from the others, to the extent where members of a group are oblivious of the fact that other groups may be co-existing with them. Multi-tenancy promotes delegation of control in a safe and secure manner. For example, tenants are permitted to create and administer their own private virtual networks. Location independence, on the other hand, is primarily concerned with hiding the identities of individual infrastructure components from the virtualized workloads. This has made it possible to relocate running virtual machines from one host to another. An equally important but perhaps less appreciated benefit of location independence is improved efficiency in resource allocation. Tenants are therefore unaware of the physical hosts on which their virtual machines are running. Furthermore, virtual machines belonging to different tenants may be placed on the same host. In such a shared ecosystem it makes sense that tenants are not given direct access to the underlying switch fabric, consisting of host-level virtual switches, top-of-rack switches, etc. This restriction avoids the possibility of any cross-tenant data leakage. Unfortunately, this means that the port-mirroring capability of those switches is also not available.
OpenStack is not alone when it comes to the lack of traffic monitoring support for its virtual networks. Other cloud solutions, including Amazon Web Services (AWS), also suffer from this limitation for the very reasons described above. There is one aspect of OpenStack, however, that makes it stand out from the rest. It is open source technology! This provides an opportunity for someone (actually anyone) who is able and willing to introduce new capabilities into the platform. At Gigamon we live and breathe network traffic visibility, so we decided to take it upon ourselves and become ‘that someone’ who will push the needle forward. To our surprise and delight we soon learned of a group at Ericsson who had independently arrived at the same conclusion. Our goals matched perfectly and it seemed natural and fitting that we pool our resources together to solve this problem.
Our project is called Tap-as-a-Service. It is a platform oriented solution, designed to operate as an extension of Neutron, the OpenStack network service. TaaS offers a simple API that will enable a tenant (or the cloud administrator) to monitor ports in Neutron provisioned networks. Since it is vital that tenant boundaries are not compromised, a tenant can only monitor its own ports, i.e. any port on one of its private virtual networks or a port created by it on a shared virtual network. The TaaS workflow begins with the creation of a tap-service instance that has a Neutron port serving as the destination side of a port-mirror session. A monitoring virtual machine is usually attached to this port to consume the mirrored traffic. Later, one or more tap-flows can be added to the tap-service instance. A tap-flow represents the association between a (source) port that is being monitored and a tap-service instance. TaaS allows a mirror session to span across multiple hosts, by virtue of remote port-mirroring, thereby ensuring that location independence is preserved.
Tap-as-a-Service (TaaS) Deployment in an OpenStack Cloud.
A reference implementation of TaaS was completed earlier this year and the source code has been uploaded to Stackforge, the OpenStack incubator, where a dedicated GIT repository now exists for this project [1]. At the last OpenStack Summit in Vancouver (May 2015), we did a technical presentation on this work that included a live demonstration of traffic monitoring using TaaS [2]. The response has been very positive, with support pouring in from both the developer and user communities. We will continue to enhance TaaS; some of the planned features are integration with the OpenStack dashboard, support for virtual machine migration, pre-capture filtering and rate-limiting of mirrored traffic. At the same time, we are also continuing our discussions with the Neutron core team to have TaaS accepted as an integral part of future OpenStack releases.
Port-mirroring used to be a switch layer function. Tap-as-a-Service has effectively virtualized this capability and made it available for the users of Neutron provisioned networks. We see TaaS as the basic building block on top of which more complex traffic visibility solutions can be engineered for a diverse set of use cases, ranging from network administration and trouble-shooting to application/network security, data analytics and more.
References
- Tap-as-a-Service code repository.
https://github.com/openstack/tap-as-a-service
- Tap-as-a-Service (TaaS): Port Monitoring for Neutron Networks.
Alan Kavanagh, Anil Rao, Vinay Yadhav, OpenStack Summit, Vancouver, Canada, May 20, 2015.
