Automation and Orchestration — A Panacea for Digital Transformation?

Updated September 30, 2021.

pan·​a·​cea | /ˌpa-nə-ˈsē-ə/

: a remedy for all ills or difficulties: CURE-ALL

In this three-part blog series I will demonstrate how automation, along with orchestration, can dramatically reduce repetitive, error-prone systems managements tasks, focusing specifically on workloads being migrated to the cloud. While there are other writings on automation and orchestrations, this blog series provides a unique perspective of network visibility’s role in broader automation initiatives.

• I begin this first blog post with an introduction to automation and discuss motivations and methodologies
• In the second post I will show how you can use automation to apply extra scrutiny to traffic deemed suspicious by security tools, typically as part of a SOAR playbook
• And in the final post we will review an actual situation where a service provider uses Ansible to configure Gigamon 5G CUPS configuration

The Big Picture

Digital transformation has been key to any industry’s ability to survive and compete, especially in the last decade. There are multiple reasons why this transformation has occurred and has been pushing companies to adopt the changes that are part of the transformation. From an end-user point of view, it is the ease-of-buying experience (smartphones, faster mobile broadband), and from the seller’s side, the availability of multiple tools and resources to overhaul the business.

According to a Forrester Research report on CMDB, IT Infrastructure Library and service management have moved towards a DevOps framework. With the advent of containers and serverless architectures being deployed in the cloud, traditional asset management has gone for a toss. The same people who had challenges with tracking and managing physical infrastructure and related data are facing a new problem with unmanaged assets. The DevOps methodology or framework relies on automation and orchestration.

• Automation – Making manual repetitive tasks run on their own; e.g., open a web browser with a URL.
• Orchestration – Stitching multiple automated tasks to perform a workflow or process; e.g., open a web browser with a URL, log into the website with a username and password, click on a button that puts an item in a shopping cart, checkout and send an email to a customer.

As you can see, orchestration can be a more complex process involving multiple systems, databases, authentication servers and so on. Configuration management not only involves ease of initializing multiple devices and bringing these devices up and running but also provides the ability to modify and automate repetitive tasks with practically zero errors.

My Team/Organization Has Been Allotted a Budget for Automation — What Next?

Planning and forecasting are essential when it comes to implementing automation practices for continuous integration and delivery. Sandboxing the automation environment is a simple best practice before integrating it with the SDLC. There could be multiple areas around the product lifecycle that will benefit from making the shift to an agile DevOps style from a more traditional waterfall methodology. The decision-maker should consider some of these aspects before kickstarting the project: business use case, resources (people and tools), prerequisites and interdependencies. The planning criteria should also include the tools specifically needed for the project. This includes open-source and proprietary scripting languages, version control mechanisms, dashboards, job scheduler and notifiers. Another important detail could be metrics to track, to evaluate if the process is improving.

APIs — Automation’s BFF

Although API (application programming interface) has been around for longer than many people think, the true scale and benefit was realized with the introduction of web API. In simple terms, an API is a definition of interaction between multiple software or hardware components. It was designed to make nonsimilar systems interact and interoperate with a set of rules and formats so that building complex programs can be made easier. Web APIs became the easiest way to build applications quickly and to make programs talk to multiple providers without the need for extensive customization. Almost any modern application either on mobile or IoT platform utilizes APIs to function at the basic level. The APIs in web development, more commonly known as REST APIs, use standardized protocols and formats such as HTTP, URI, XML or JSON.

The Gigamon-FM fabric manager is a single-pane-of-glass management platform that provides users the ability to manage and automate their deployment and integrate with a wide variety of tools using REST APIs. The GigaVUE-FM REST API User’s Guide is a good starting point to understand the architecture and use cases when using the APIs. For quick reference when using the fabric manager, users can also navigate to Administration > API Reference under Support to go to the Swagger page (https://<FM-IP-Address>.com/apiref/apiref.html).

For those who are new to automation using APIs, Postman has a free version and provides a lightweight client to test and validate REST API calls.

Here is a sample REST API JSON schema from GigaVUE-FM.

{
  "portConfigs": [
    {
      "portId": "string",
      "licensed": true,
      "neighborDiscovery": "none",
      "alarmThresholds": {
        "alarmThreshold": 0,
        "alarmThresholdLow": 0,
        "alarmBufferThresholdRx": 0,
        "alarmBufferThresholdTx": 0,
        "additionalProp1": {}
      },
      
        "additionalProp1": {}
      },
      "lock": {
        "lockingUser": "string",
        "sharedWith": [
          "string"
        ],
        "description": "string",
        "additionalProp1": {}
      },
      "share": {
        "toolShareRoles": [
          "string"
        ],
        "additionalProp1": {}
      },
      "timestamp": {
        "appendIngress": false,
        "stripEgress": false,
        "sourceId": 0,
        "additionalProp1": {}
      },
      "ingressVlanTag": 0,
      "taptx": "active",
      "gdp": false,
      "l2greId": 0,
      "vxlanId": 0,
      "headerStrip": "none",
      "fec": "cl91",
      "ptp": {
        "enable": true,
        "localPriority": 0,
        "role": "standard",
        "announceInterval": 0,
        "delayRequestInterval": 0,
        "syncInterval": 0,
        "timestamp": {
          "egress": {
            "insert": false,
            "sourceId": 0
          },
          "ingress": {
            "insert": false,
            "sourceId": 0
          }
        },
        "vlan": 0
      },
      "additionalProp1": {}
    }
  ],
  "context": {
    "totalItems": 0,
    "pageNo": 0,
    "pageSize": 0,
    "sort": [
      "string"
    ],
    "additionalProp1": {}
  },
  "additionalProp1": {}
}

CI/CD – Continuous Integration/Continuous Development, Deployment

To keep up with the pace of supply and demand in the digital space, building and testing applications or products cannot wait for weeks or months. Traditional software development processes have become unworkable since the advent of DevOps with CI/CD pipelines. Continuous delivery and integration are a way of making sure that code check-ins do not fail at the last moment when all the stakeholders collaborate at the end of the software release cycle. Combining automated testing (both white and black box) with a release pipeline that provides repositories for development or code testing and a methodology to push these changes to production simply makes up a CI/CD framework.

To achieve complete visibility and transparency in a CI/CD framework, network traffic is a key asset and should not be overlooked as an optional parameter while drafting SLAs for monitoring. That said, it is important to separate the signal from the noise. Sending reliable, relevant and consistent network data to the CI/CD tools is vital to maintaining a robust hybrid cloud environment.

Automation for the Hybrid Cloud

Migrating to a public cloud infrastructure has its benefits and challenges. While the business transformation has its benefits, technology transformation poses challenges. This is where one looks for the panacea — automation. If you truly want to achieve better ROI by moving the bulk of IT operations to the public cloud, it has to be done using the most efficient and secure process. Since the cloud is already primed for integrating with multiple tools and applications by using APIs, organizations can make use of this opportunity to automate even their legacy processes. By eliminating large chunks of time cycles spent on on-premises networking, data backup and recovery, identity and inventory management, IT teams can now focus on faster deployment and testing with the CI/CD methodology in place. Both NetOps and InfoSec teams can incorporate the same principles of DevOps for software development to navigate the hard road for migration to hybrid cloud.

Many organizations are reducing risks associated with cloud migrations by eliminating the need for reinventing new monitoring and security infrastructure by reusing cloud versions for tools they already own. In these situations, automation can benefit in numerous ways, ranging from spinning up new cloud instances with visibility built into the infrastructure as code to sending selected traffic to tools in other parts of the hybrid infrastructure. In the next blog post, we can look at how and why automating your visibility and analytics platform is both essential and effective.