Most network engineers have, at some point, had to grind through a seemingly endless backlog of network change requests that force them to dart across multiple network technologies (CLI devices, API-enabled controllers, public cloud, etc.) and then make sure adjacent tools are updated to reflect each change. To make these processes less onerous, some enterprise networking teams may jump (albeit tentatively) into network automation, first by writing a few scripts to handle common tasks. From there, they often go after infrastructure automation, trying to agree on a source of truth and building processes to keep things up to date. Then they realize they could make a few other tasks self-service to improve delivery times. Before they know it, instead of simplifying their processes, these well-intentioned networking teams are suffering from “tool sprawl.”
While “tool sprawl” may sound nasty (and possibly contagious), it’s increasingly common and can be both diagnosed and alleviated if caught and managed early…
Data Transformation Benefits the Automation Journey
Enterprise network teams know automation can help, but how do they do it?
One reason automating “the network” has become so complex is because today’s network infrastructure consists of several different types of networks, such as physical, cloud, virtual, etc., all of which have different methods of management (CLI, API, GUI Dashboards). There are also multiple IT systems, network services, and applications that must be utilized as part of the automation journey, adding more complexity.
The automation process starts with identifying a network automation platform that is robust, flexible, accessible to teams of all skill sets, and doesn’t insist you throw out your existing scripts and playbooks. Instead, the best platforms allow teams to bring all of the disparate parts of the network under a common, aligned automation model. Such a platform should lead an organization to scalable enterprise network automation – being able to start with simple task-based scripts and move toward more scalable network automation, addressing a larger number of activities within the automations on each iteration. This will free networking teams to focus on larger business initiatives.
By starting with simple network changes and evolving to a more complex journey of integrating various systems, messaging, and other sources of truth, the number of automation tasks grows, and the need to automate the data transformation processes becomes more of a necessity.
Data transformation is the process of taking data in from a single source, or multiple sources, transforming it into an appropriate format, and sending the data out to be used by another step in a workflow. Network teams are mainly doing this manually (read: inefficiently) by swivel-chairing between systems and dashboards to format and copy and paste data as necessary.
Don’t Get Lost in Translation
When connecting systems and integrating data, a commonly overlooked aspect is the level of translation that needs to occur. While some transformations can be straightforward, there are some that require a greater level of translation – manipulating the information to the point where it makes sense for the customer – and not just the ability to deliver the payload.
Let’s use the post office as an analogy. The post office can deliver a letter from one person to another. But what if that letter is written in Mandarin and the person receiving it can only read English? The post office was able to deliver the letter – it provided the integration function of connecting one person to the other – but now, some level of translation needs to happen. In network automation, that’s what the data transformation function accomplishes. The payload has been delivered, and now the data transformation function needs to manipulate the data to make it readable and consumable at the other end.
Of course, it isn’t quite that simple. The more sources of data you have, the more complicated the process becomes since you’re translating between technologies, which vary tremendously (i.e., cloud technologies vs. traditional hardware). And while network teams can solve translation challenges by writing a script, a function, or a piece of software that is going to perform text manipulation, other challenges lie in wait, including the number of tools being used.
Because teams have to retrieve and manipulate different types of information available in different places, a different set of tools is required for each system. In other words, tool sprawl: the unnecessary buying of new tools, inefficient spending, and disparate data.
Reducing Tool Sprawl via Automated Data Transformation
DevOps engineers understand the burden of writing and maintaining code when integrating with different IT systems. Automating the most basic of IT services can sometimes require integration with the ITSM system, sources of truth, compute, storage, and multiple networking systems. Writing the code to integrate and manipulate data between these systems is a tremendous amount of work, and because the IT ecosystem continues to evolve and change over time (i.e., new systems are added and removed, APIs change, etc.) the task of managing and updating integration code for every automation becomes monumental.
Automating data transformation, by utilizing modern tools to develop transformation rules as objects that can be referenced by the automations can help reduce tool sprawl. And it can do it in a way that is flexible and portable while also offering time savings, reduction in manual errors, and increased efficiency. Data transformations that can be built fast and operate as reusable assets enable network teams to start faster from the beginning and not start from scratch with every automation. In addition to automating the task itself, the automation solution must include automating data transformations. Otherwise, it will always require human intervention to format data between network tasks, leading to inefficiencies, increased costs, and errors.
When data transformation is performed correctly, combined with using only the necessary tools, NetOps and DevOps teams can access data efficiently, reduce manual errors and time, and extract insights for future automation.
Itential’s Advanced Data Transformation Capabilities
Itential is the only network automation and orchestration platform that offers comprehensive data transformation capabilities across any data format required by all your network systems and domains. To achieve this, Itential leverages JSON to provide users with the ability to build any data transformation logic required in the JSON Schema Transformation Designer. For even faster automation, users can also access a library of Pre-Built Data Transformations that is continuously growing to cover all the most common use cases.
To see in-depth how both NetOps and DevOps teams can leverage Itential’s data transformation capabilities inside of an automated workflow to eliminate tool sprawl, check out the full demo series here. You’ll see different options for building data transformations and learn how to leverage these capabilities to improve your end-to-end automation workflows.
Article originally published on Network Computing.
Morgan Stern
Vice President, Automation Strategy ‐ Itential
Morgan Stern serves as the Vice President of Automation Strategy for Itential, where he is responsible for assisting Itential’s global customers in developing, implementing and deriving value from their automation strategies using the Itential Automation Platform. For the majority of his career, Morgan has focused on assisting large enterprise and service provider organizations in maximizing the business impact of new technologies as an architect, consultant, author, and industry speaker.
