“Engineers don’t need a lot of help on pushing configuration files.”
Even after initial training efforts with Python and Ansible, Fiserv’s network teams found that many engineers weren’t putting their new skills into practice. Not out of resistance—just because, at the end of the day, the cost-benefit didn’t make sense. How much time could it really save, when engineers can push config pretty quickly already?
True success and scaling their automation capabilities came from shifting the strategy. Viewing activities like querying systems of record or performing post-checks — activities typically relegated to the realm of ‘tribal knowledge’ — as opportunities for automation allowed engineers to focus on automating things that saved them a lot of time. Then, as the approach matured, they embraced orchestration, building end-to-end workflows that could be delivered as products with a self-serve model. You can listen to this recent Packet Pushers episode to hear Fiserv’s Director of Architecture & Automation, Michael Wynston, talk through this journey and evolution.
Theirs is a story network engineers and architects everywhere can learn from. Automation is about more than just optimization. It’s about unlocking new levels of efficiency for how you deliver network engineering across the business.
Training & Skills: The Foundations for Automation Success
Before you learn to run, you learn to walk. The modern network engineer must be adaptable and ready to learn how to interact with different automation technologies — to build an automation practice where you can deliver orchestrated services at scale, engineers must have some level of familiarity with things like Python, Ansible, APIs, JSON, and the controllers that are used in the organization’s environment.
This isn’t so much about which specific skill(s) you build. It’s more about the mindset and the capability. Today, Python is the name of the game for network automation, but tomorrow it could be something else. The key is being willing to learn and adapt, embracing an automation-first mindset.
In the podcast, Michael talks about one of their early forays into automation, a big investment push into engineers training to use Python and Ansible. The training would prove to be vital to every automation initiative that came after it. However, training alone didn’t do the trick.
“They came in, they fished, everybody ate. They left, we starved,” Michael said of the consultants they hired to run training. “We had some success with automation that was engineers developing Python scripts or Ansible Playbooks for themselves. But that doesn’t scale across the entire organization.” It was critical for network engineers to pick up these skills, but scaling automation would take a wider approach, enabled by a platform that could extend and maximize their impact.
Automating Tribal Knowledge to Scale Engineer Impact
For any network change, there are multiple steps that must be completed. These can span different teams, different network domains, and multiple external systems like IPAM or change management platforms. Some of the processes and activities that must be performed are tribal knowledge held within a single team or even a single engineer—making it difficult to scale those processes as automation efforts evolve.
If you need to call the same experienced network engineer every time a particular type of incident occurs, what happens when they’re sick? On vacation? When they retire? If that same person could instead turn their experience toward building scalable assets for the organization that formalize and standardize their tribal knowledge as repeatable processes, that’s a level of impact that goes far beyond reactively performing manual activities.
Automating the following activities that often involve tribal knowledge was key to Fiserv’s network automation teams accelerating their service delivery capabilities:
- Executing commands across multiple device vendors’ CLI.
- Identifying and interacting with systems of record.
- Pre-checks when making a change.
- Post-checks to ensure a change is successful.
- Producing documentation to record a change.
- Notifying other systems and teams of a change.
When tribal processes are turned into automation assets for the organization, teams are able to collaborate more easily, and workflows are less likely to get ‘stuck’ at different points where individuals don’t know what to do next. Fiserv’s network team was able to use Itential’s API integrations with various external systems to build automations for tribal processes directly into their networking workflows.
Read this blog for a full breakdown of tribal knowledge as it relates to network automation, or watch this demo to see how Itential helps you automate tribal processes.
Tying It All Together with Orchestration
If you and your network team have successfully acquired new automation skills across tools like Python and Ansible, domain-specific controllers like firewall or SD-WAN controllers, and vendor-specific point solutions, and you’ve built out automations for tribal processes, the next step is tying different automated steps of a change process together to produce an end-to-end outcome. That’s orchestration: the ability to coordinate automation across your network’s multiple domains, cloud environments, and integrated external systems to deliver a standardized service from start to finish.
In the Packet Pushers episode, Michael outlines how the team progressed to a point where they can orchestrate across all their different vendor platforms and deliver infrastructure services as self-serve products for end users to consume. That’s the model for modern network and infrastructure teams: supporting business agility by delivering complex services rapidly and at scale.
Successful orchestration requires a few things. The ability to integrate with all the systems and platforms in your network is key. You also need capabilities for manipulating data between different automated jobs — for example, you might need to pull an IP address for a router from an IPAM, and then convert it into a different format for a Juniper device vs a Cisco device. And finally, you need the ability to maximize the impact of the assets you’re building, such as individual scripts for pushing configuration or a set of CLI commands represented as a template.
The only automation and orchestration platform that offers all of the above is Itential — that’s why our platform is central to Michael’s story, and that’s how we’ve been able to help countless other network and infrastructure teams transform and evolve from ad-hoc scripting and manual operations all the way to delivering self-serve products.
Automation isn’t just optimization. It’s about enabling a new way to deliver network engineering and infrastructure, powered by an automation-first mindset. The Itential Automation Platform is focused on making this evolution possible by making it easier.
“By having something that lowered the bar of entry, something that enabled us to reuse what we already built and create a workflow of multiple automation things, [Itential] accelerates our ability to deliver automation.”
– Michael Wynston, Director of Architecture and Automation, Fiserv
Learn more about how Itential can support your team with this series of demos — or check out the Packet Pushers episode I’ve been talking about to hear Michael lay out the full story of Fiserv’s network automation journey.
Kristen H. Rachels
Chief Marketing Officer ‐ Itential
Kristen serves as Chief Marketing Officer for Itential, leading their go-to-market strategy and execution to accelerate the adoption and expansion of the company’s products and services.
