AWS X-Ray and Gigamon Visibility Platform for AWS: Different Strokes for Different Folks

At AWS re:Invent 2016 last week, Amazon unveiled a preview of AWS X-Ray, a service to help developers analyze and debug distributed applications. As AWS CTO Werner Vogels shared details at his keynote, it was received with enthusiasm by the audience. Are there any similarities with our Gigamon Visibility Platform for AWS announced earlier this year? Not at all. Gigamon provides access to network data for any tool that needs network traffic. On the other hand, AWS X-Ray provides developers with debug trace capability for their distributed applications. The two are meant for entirely different purposes as this blog will explain.

AWS X-Ray works as follows:

• It tracks requests made to an application (e.g., API calls).
• It collects data points as the request passes through each of the underlying application services.
• It combines the data points gathered from each service into “traces.”
• It presents the collected trace data as a service map. The service map can be used to uncover latencies, http status, etc., for each service.

AWS X-Ray builds on other troubleshooting/monitoring services that are already available in AWS today. For example, AWS CloudTrail records the history of AWS API calls for an account. Amazon CloudWatch offers a set of monitoring services—an important such service is VPC Flow Logs. Originally introduced in June 2015, VPC Flow Logs is a networking feature to capture information about IP traffic going to and from network interfaces in a VPC and storing the resultant “flow log” data in Amazon CloudWatch Logs.

Last month, we introduced the Gigamon Visibility Platform for AWS to provide visibility into data-in-motion in a VPC. Our solution fills a critical need that our customers have repeatedly asked us for—access to network packet data in a multi-tenant infrastructure-as-a-service (IaaS) public cloud, with dials to select what portion of the network data to send to tools. These selection dials are crucial to ensure that receiving tools (themselves implemented as software running on compute instances) do not get overwhelmed by the data they need to process. In security operations management, full-packet network data and its sibling, metadata are integral to tasks such as intrusion detection, data loss prevention, malware analysis and anything that requires content inspection. IT operations management likewise needs access to network data to ensure a premium application experience. Without this, operations teams are unaware of the data exchanged between workloads.

Since we launched trials in May, we have received very strong interest from ecosystem partners to integrate with our offering—the Gigamon Visibility Platform is already validated with fifteen commercial and open-source tools and more are in the works. AWS users who visited our re:Invent booth expressed delight that they could finally have access to network packet data directly from an EC2 instance! Moreover, as customers embrace hybrid cloud architectures, the need to obtain unified views of data-in-motion across both the on-premise data center and cloud accelerates their deployment of an agnostic layer of visibility across the entire organization.

The public cloud boat is so big that multiple techniques are required to get visibility into cloud infrastructure. There simply isn’t a “one-size-fits-all” approach just as log data, flow data, network data (also called “wire data”) and metadata have all co-existed in the on-premise data center. For example, developers need good API tracing capabilities (provided by AWS X-Ray). Administrators and auditors benefit from AWS CloudTrail records. Operations (and developers) need metrics and logs around memory, disk, compute instance utilization (all provided by Amazon CloudWatch) or troubleshoot network-layer discontinuities (provided by Amazon CloudWatch VPC Flow Logs). Security operations and other IT operations teams also need network data (provided by the Gigamon Visibility Platform for AWS) to perform network security analysis and other related tasks.

In a world of distributed systems and disaggregated architectures, the need for an agnostic visibility platform only increases. Having access to the data-in-motion, extracted from network data, is an important toolkit for operations teams to be empowered with. Indeed, this viewpoint is also echoed by Gartner analysts Will Cappelli and Vivek Bhalla, who wrote in a recent research note that “Wire data, log data and API-based utilization metric data are essential sources.”

Cloud adoption is going to be pivotal in 2017 as barriers to entry and security concerns are eliminated. Advanced cloud tools to diagnose data and gain greater intelligence and visibility are now available and each have their purpose. As the saying goes, different strokes for different folks!