What Is Application Metadata (and Why Does It Matter)?

Updated October 28, 2021.

Application metadata is data that summarizes application traffic at a higher, abstracted level. This information provides much greater visibility into how applications are performing, behaving and being used across the network.

A more well-known form of metadata is NetFlow, which can reduce overall data generation by 85 to 95 percent. However, this operates only at OSI Layers 2–4, and while it can tell you what devices are involved, determine how much traffic is flowing and facilitate basic network management, it lacks Layer 7 insights. Generating this metadata can overwhelm infrastructure elements such as routers; relies on packet sampling that prevents full visibility; and can be inconsistent in format, which limits its use by monitoring and security tools.

Why You Need Application Metadata

Application metadata takes metadata to the next level by delivering rich, contextual data to your SIEM, security and performance monitoring tools. The uses of metadata are many. With application aware metadata, IT teams can acquire critical details pertaining to flows, reduce false positives by separating signals from noise, identify nefarious data extraction and accelerate threat detection through proactive, real-time traffic monitoring, as well as troubleshooting forensics.

SIEM solutions use this information to correlate and analyze log data from servers and security appliances. IT teams can also use advanced metadata to automate detection of anomalies in the computer network, stop cyber risks that overcome perimeter or endpoint protection, identify bottlenecks and understand latency issues.

Layer 7: Where the Action Is

For years now, users have relied on a host of platforms, such as enterprise firewalls and application delivery controllers that provide deep insights and control over the traffic deluging data centers. With next-generation network packet brokers, you can not only offload metadata generation but also acquire crucial application intelligence.

Instead of just knowing what devices are communicating, to whom and how much data they are exchanging, application metadata with Layer 7 intelligence goes orders of magnitude beyond NetFlow to provide information on what and how — such as what devices are doing — by looking at the requested resources, visualizing how they are communicating by observing the commands and directives exchanged, and deriving insights into security, network performance and user experience.

Types of Application Metadata

Application metadata includes any metadata above the transport layer. Such higher order data can be rudimentary, such as DNS or IMAP/SMTP for email communications, or more complex, and provide insights into thousands of applications and protocols, including URLs and HTTP.

Some specific advanced application metadata that you can potentially unlock includes attributes such as user and file names, email addresses, software versions used or whether database files are compressed — just to name a few. The tables below show a snapshot of some of the key app and protocol details you can obtain, with the listed values specifying the number of attributes available from Gigamon.

To show the power of application metadata, check out BitTorrent. This is a communication protocol for peer-to-peer file sharing and is sometimes used to send bootlegged content (such as music or video) to others who do not have access rights. Of the 35 attributes available, IT can search out and find such insights as:

• ID and port used by a peer to share files
• Name and size of the uploaded/downloaded files
• Flow encryption level
• Software version
• Whether the session path can be declassified

Gigamon Provides Needed Application Awareness

Application Metadata Intelligence (AMI) expands upon app layer visibility derived from Gigamon Application Visualization and Application Filtering Intelligence (AFI) and supports a comprehensive approach to derive application metadata and user/app behavior.

This solution takes incoming traffic streams and automatically generates more than 5,000 Layer 3 through 7 metadata attributes. AMI is not based on sampling, so it sees all packets; it supports a range of formats, including NetFlow versions 5 and 9, IPFIX and CEF for seamless integration with a broad range of standards-based collectors, storage devices and SIEMs; and it is processed on the Gigamon visibility node so as to prevent any router or other infrastructure elements from experiencing processing overload or performance degradation.

Gigamon extracts and appends elements to NetFlow and IPFIX records. Some additional examples include:

• Identification: Social media user, file and video names, SQL requests
• HTTP: URL identification, commands response codes levels
• DNS parameters: 39 elements including request/response, queries and device identifiers
• IMAP and SMTP email-based communications with sender and receiver addresses
• Service identification: Audio, video, chat and file transfers for VoIP and messaging
• SIP and RTP: Call ID, registration and invite response codes, codec and media types for VoIP

Gigamon’s patented Flow Mapping^® technology can be used to pick and choose from flows to generate NetFlow and application metadata statistics, while at the same time sending the original packets to security and monitoring tools.

Operators can also export NetFlow records, plus other network metadata, to multiple collectors concurrently, creating a single flow source for business-critical management applications such as security, billing, capacity planning and more. And finally, they can filter exported flows so that collectors only receive the specific records relevant to them.

How Can You Apply Application Metadata?

Advanced L7 metadata can be used in a variety of use cases. The principal deployment for AMI is in providing metadata to SIEM tools for security analysis, but NPMD/APM tools consume these AMI feeds too. Here are just a few examples.

• Identify expired TLS certificates, untrusted sources and self-signed certs. This form of application metadata security provides certificate expiry dates, as well as any revoked or expired certificates, along with the names of the application servers using these certs for compliance. Other relevant AMI attributes include Valid Not Before, Valid Not After, Serial Numbers and Signature Algorithm to help validate certificate use.
• Locate weak ciphers. Ideally, clients and servers should only employ the strongest cipher suites available and negotiate to one of these during the TLS handshake, but this is not always the case. AMI can provide application metadata that reveals all TLS connections with weak ciphers, along with the applications and systems hosting those apps, helping you ensure security compliance.
• Analyze HTTP client errors. With AMI, you can analyze HTTP client errors, including the number of HTTP response-code errors relative to the total number of codes, to spot suspicious activity. The distribution of these errors and the clients seeing these codes may also provide further insight. AMI can provide details on a client IP address and the number of errors it has encountered, which can help you spot a hacker trying a brute force attack and getting 401 errors.
• Troubleshoot and remediate poor performance. AMI can isolate the traffic for, and help tools debug poor video conferencing experience in apps like Zoom, GoToMeeting and WebEx. Usable attributes of video embedded in an application include: Starting frames and bits per second rates, and how they change over time; drops from HD to standard video quality; and other attributes. Application and network performance monitoring tools can use this information to determine the user’s true video viewing experience and potential causes of service degradation.
• Evaluate network and application health. Metadata here involves using application broadcast and multicast “control” packets. Applications send these packets at regular intervals, and by analyzing them over time, IT can determine the average interval between control packets and their timing during this period. A differential in interval time between control packets could be due to device malfunction, network congestion or network traffic storms. AMI attributes involving SNMP, STP, UPNP and any broadcast packets can be useful in pinpointing the root cause.

Learn More

This is but a fraction of what is possible. After all, describing how to use more than 5,000 application metadata components would take something the length of an encyclopedia. Check out this video for an AMI overview. To get more AMI use case insights and details, check out this blog post and download this insightful source. Better still, request a demonstration.

---

Further Reading

• “Lean into Zero Trust to Ensure Security in Times of Agility“
• “Understanding IT Stress Points with Application Intelligence“
• “The 5 C’s of Visibility“