The Benefits of Building Data Transformations into Your Network Automations

The term “data transformation” may sound entirely foreign to you, especially for many network practitioners that do not have a background in programming, but chances are you may already be more of an expert in this without even realizing it. Data transformation, especially in the context of making manual changes to network device configurations, has become a subconscious, nearly instinctive process that you navigate with your mind along with a low-tech set of tools. However, as your automation journey moves from automating simple network changes, to include more robust and comprehensive workflows that will integrate ITSM systems, messaging, IPAM, and other sources of truth, it will become necessary to automate these data transformation processes so that they can execute within the automation, and not rely on people to directly intervene to manipulate and transform data between tasks.

The Old Way of Doing Data Transformations

Changes to a network device don’t just occur because someone felt that the network needed some changing (actually, that’s the stuff of nightmares for anyone who is responsible for the network). There’s always a reason for a network change, and that reason will include some initial information and require more specific related data before a change can be made to a device.

An example could be that someone on the application team is deploying a new server in the data center. The initial information could be they need 10 gig ethernet connections in rack X in data center Y. From that initial information, more data must be gathered — which top of rack switches are on that rack, do they have ports available, what VLANs are needed for those ports, do we need new IP addresses for this server, etc. That data can be gathered from several sources, whether they are IPAMs, databases, ServiceNow, or a custom application. Once that data is gathered, it needs to be put in the correct format, according to the appropriate CLI syntax defined by the vendor of those network devices.

This process of taking data in from a source (or multiple sources), transforming it, and outputting data appropriate for CLI changes is a form of data transformation, accomplished every single day by network engineers all over the world. Their manual process would include opening lots of browser windows to different applications for data gathering from sources of truth, copy and pasting all that data into a notepad application (shout out to Notepad++ and Sublime for the years of service), creating CLI commands in the notepad with the gathered data so you can copy and paste it into the network device to make the changes. While you may believe that automation is just being able to automate the CLI changes to a network device, there’s a mountain of work going on before that can occur — it’s just YOU doing all the work, and that is why automating data transformation is critical to your organization’s automation journey.

The Benefits of Building Data Transformations into Your Network Automations

Automating data transformations is important for several reasons.

Time Savings
The most obvious benefit is that it saves a tremendous amount of time for the network practitioner who is already buried under a growing backlog of work. Automating the network changes alone is not going to dig them out from under the backlog avalanche. In fact, just automating network change tasks only saves them 8%-10% of the overall process. To refund the remaining 90+%, you must also integrate the data gathering steps and automate the data transformation process.

Reduce Manual Errors
Another benefit of automating data transformations is that will reduce errors that inevitably creep into the manual process. I will volunteer to admit this on everyone else’s behalf — it’s really easy to copy and paste the wrong CLI commands into the right router (or the right CLI commands into the wrong router), especially when you’ve done this two dozen times in a row, and more especially with its 3 AM because you can only make changes during the maintenance window. No one wants to make a mistake, but it’s almost unavoidable in today’s environment, and automating the data transformation process can help to eliminate these types of errors.

Increased Efficiency
As I’ve already discussed, reducing time, and increasing accuracy bring along a natural increase in efficiency, but there are other efficiencies that can be had as well. Data transformations that can be built quickly and operate as reusable assets means that you can start faster from the beginning, and these assets don’t have to be recreated from scratch for every automation—anyone that builds automations can leverage the work you’ve already done (and tested) within their own automations. Finally, and for selfish reasons, the most important benefit for automating data transformations is to save the sanity of the networking team. Our IT environments are growing more and more dynamic and complex, requiring more network changes at an increasing rate, so the prospect of day after day of work that requires the same manual process to swivel-chair between GUI dashboards, copy and paste data, transform in a notepad, copy and paste CLI, then check to make sure nothing was broken and repeating these dozens of times will drive anyone into madness.

Getting Started Automating Data Transformations with Itential

Getting started automating data transformation with the Itential Automation Platform starts with identifying a common network use case. What’s that use case that always pops into your head when you think about the work you do day to day, the one that makes you say “if I had the power to build an automation, I’d automate _________ so I don’t have to do this.” (Odds are, we already have that covered in our Top 10 use cases, which you can check out here.)

Once you’ve identified that network task, consider the processes that occur before and after the actual network change. What systems do I access and what data do I use from those systems? Any time you access one of those systems for information, whether it be for IP addresses or inventory or whatever, that data will need to be transformed into a format that is required by another system or task in your automation. It’s useful to think of a data transformation task as taking “Data In”, transforming it into an appropriate format, and sending “Data Out” to be used by another step in a workflow.

The transformation process will correlate to the same manual process you are already engaging in today, and with our drag-and-drop transformation canvas you can get started very quickly by using different methods of defining and modifying data. In fact, if you check out this recent demo, I show you how to build a transformation from scratch, explain each step in the process, and use it in a network change automation.

Data transformation is a normal, but often overlooked, part of the overall process of making network changes, so in addition to automating the change itself, your automation solution must include automating data transformations — otherwise, there will always be a requirement for human intervention to format data between network tasks, and that leads to inefficiencies, errors, and quite possibly an avoidable bout of temporary insanity for my network friends.