Chris EbleyLet’s be honest, as with most things in life, there’s nothing as good as the original. As a native Marylander, I have a soft spot for a crab cake. At the same time (and as someone who travels a lot), I also know the feeling when you’re stuck in the middle of nowhere in a landlocked location and crab cakes show up on the menu. I know I have no control over where the crabs are from, what their recipe is, how fresh the ingredients are, and finally if eating that sandwich is going to achieve what it’s meant to… a happy me.
Pessimism Engaged!
As off topic as my relationship with crab cakes sounds, it’s exactly how I feel when I see the term metadata. By definition, metadata is a set of data that describes other data. It’s an abstraction of what someone considers to be the important notes or excerpts from a packet or a log that sums up the “gist” of what’s going on.
Being a “log aficionado” (read: passionate Splunk advocate), I am a consummate defender of data in its raw form. Too often, I’ve run into scenarios with my customers where the critical piece of information isn’t present. I’m stuck looking at some summarized version of information that was contrived for a specific task too many moons ago that is now being repurposed to tackle a different initiative.
I know what you’re all thinking: “I use metadata all the time in my day to day and it often delivers.” And that’s probably valid. There’s no question I’m painting a darker world view than need be, but I’m also striving to drive home the fact that no one is going to understand your needs as well as you do. Therefore, anytime you have the option to lay eyes on everything (I’m looking at you security analysts), you should jump at the opportunity. In the case of cyber security, data is no different.
Moreover, I’m also aware that “seeing everything” doesn’t come free. Storing data isn’t cheap, and many solutions are licensed by how much they see or limited by throughput constraints that have you making decisions on what you want to monitor or consume before you ever get to the point of analyzing an event (metadata isn’t sounding so bad now right, is it?). Heck, some data is so heavily buried in the confines of the solutions that deliver it that gaining access to something as simple and valuable as a DNS request requires diving into the dark world of debug-mode configurations and porting over logs that read like a small novel—just so you can take a look at a single line.
A half-page diatribe in and I’m just dipping my toe into the water of the real struggle of an analyst. The good news is, well, that there’s good news!
Have Your Cake and Analyze It Too
During my day to day, I’m often asked to help customers walk the fine line between cost and quality, metadata and raw information. Until recently, this balancing act was a daunting task. However, recent innovation has put me in the pleasant position of getting to say yes across the board.
Remember issues you may have had with metadata? What if you were able to configure what you want on your own? What if instead of the basics of source, destination, method, and port, you could add some real meaty stuff like URLs? And what about DNS issue? What if you could simply include that request as a part of the metadata you want and circumvent the entire problem?
Recent Gigamon product releases have enabled my customers to truly gain all the visibility they want—and from a single location. I can send 100% packet data from all points of my network through to advanced security solutions. I can optimize traffic flows for sustained capture of all data entering and leaving my environment (great for those “what the heck just happened” moments). And I can do this while also generating advanced IPFIX flows and feeding enhanced information over to my advanced correlation solutions. All of this is realized from a single solution . . . potentially a single box . . . potentially a single RU.
Now We’re Cooking
Remember the crab cakes and how I was pretty steamed about metadata? That’s been helped by the work of some solid minds and a new ability to “choose your own adventure” data selection all from a single source. Raw packet data and configurable metadata—all in one place, feeding to critical correlation solutions.
At the end of the day, those troubled analysts can look at an alert from their advanced malware solution that’s targeting an endpoint on the network, see raw logs showing critical changes to that same endpoint that match IOCs and validate compromise, look at valuable metadata showing DNS requests coming from that very same endpoint being spammed to a C2 server—again, all in one place serviced by visibility from one company.
Sound pretty simple? It is. Sound pretty powerful? You bet.
Happy hunting!
