The Three Phased Approach to Evaluating Network Automation Use Cases

As network automation moves from infancy to adolescence, organizations are starting to ask important questions about the impact of initial automation implementations:

• Did we gain the efficiencies we expected?
• How much time/money/effort did we save?
• How much should we expect to save?

At the same time, network automation initiatives have moved from group level automations of common tasks to larger automation use cases that address multiple domains and multiple business groups within the organization. These larger automation scenarios require a substantial investment of time, resources, and expenses for proper planning, designs, development, testing, and implementations. This means creating a financial justification of these investments is necessary for decision makers to understand the impact these larger automation use cases can have on business operations.

Empirical results for network automation are promising, with some use cases exceeding a 90 percent reduction in manual effort. Unfortunately, not all use cases deliver this level of savings, such as processes that require significant physical effort – those that cannot be 100 percent automated (i.e. hardware installation, cabling, and technician dispatch). With that said, a big component of selecting and prioritizing use cases for network automation is evaluating which of those provide the greatest benefits, making the investment of time, funds, and effort worthy to complete.

The Three Phase Approach for Analyzing Use Case Benefits

Based on our experience with multiple automation teams across our global customer base, we recommend an iterative approach for financial analysis for selecting automation use cases. The first phase, Preliminary Analysis is focused on estimating the rough order-of-magnitude savings for a set of use cases. The second phase, Detailed Analysis, looks at the most promising use cases to capture the specific activities, effort, and savings achievable. The third phase, Analysis Validation, creates a feedback loop by analyzing a completed automation use case to validate the savings and refine the model.

Let’s look at each of these phases in more detail.

Phase 1: Preliminary Analysis

There are two objectives in the preliminary analysis phase. The first is to set expectations for the potential magnitude of the level of benefits (savings, both expressed as effort and as time-to-complete for processes) for a group of use cases. The second is to provide sufficient data needed to identify the use cases that have the highest potential for savings and efficiency. The process is described below:

• Identify the candidate use cases to be analyzed.
• Define high-level data points for each use case, such as the frequency that the use case is executed, the duration to perform the process, and the amount of human effort required to execute the process.
• Define the estimated cost to perform the process, in terms of effort hours and hourly headcount costs.
• Estimate the anticipated efficiencies for the automation of the use case.
• Calculate the current metrics (non-automated) and the projected metrics (automated).
• Review the results from all of the selected use cases, with consideration of how each use case aligns with corporate strategic initiatives and objectives, such as a new service launch, a cost cutting initiative, or initiatives to drive agility or faster time to market.
• Prioritize the use cases based on impact and select the target cases for detailed analysis. The target cases will become the “short list” of candidates for the first (or next) wave of automation.

Phase 2: Detailed Analysis

The objective of phase two is to develop a comprehensive examination of the short list of candidates. This includes taking the following actions:

• Detailed process reviews of all activities
• Define each activity as inputs, human effort, data requirements, or outputs
• Establish activity and process durations – the duration of each step and for the entire process, including delays due to handoffs, non-active time, and transitions
• Identify the costs associated with each activity
• Measure the impact on revenue for use cases where increment revenue or acceleration of time-to-revenue is the focus, as opposed to use cases focused on cost/effort reductions
• Verify use case execution quantities – review historical and forecast data to determine how many times the process is executed
• Distinguish environmental quantities -such as the number of devices, number of subscribers, or any other items where activities are driven by quantities
• Project the impact of automation for each activity and the process on the whole
• Evaluate implementation costs (platform, development, testing, operations of the automation solution)
• Conduct a financial analysis – (cashflows, payback, ROI, etc.) as well as a benefits analysis – (projected metrics, efficiencies, accelerators).

A completed detailed examination should provide sufficient information for decision makers and budget owners to make an educated investment decision to move forward with the automation of use cases.

A critical factor for success is access to subject matter experts who can share insights on the real-world effort and complexities of performing the steps within the process. One often under looked component of the business justification is the inclusion of mechanisms and governance to measure and quantify the results of the automation efforts. This is a key requirement for the final phase, Analysis Validation.

Phase 3: Analysis Validation

There are two objectives for phase three:

• Provide business leadership with quantified data that demonstrates the realization of the business benefits delivered by the automation of the use case.
• Supply empirical data to the automation team that can be used to create more accurate models for the Preliminary and Detailed analysis phases.

Analysis Validation is only possible if the implementation of the automated use case includes the capability to capture and analyze data associated with the automation activities. Key data points are contingent on the ability to capture the amount of manual effort still in the process, as well as capturing the duration of tasks and the duration of the overall process, from start to finish. The ability to automatically collect this data is entirely dependent on the capability of the Automation Platform. For example, an automation use case built entirely on scripts will likely not have a mechanism to capture this data, while an automation platform will capture a rich set of execution data points which can be analyzed in multiple ways.

Once this data has been collected and analyzed, the automation team can compare the actual results against the projected results. This enables them to understand which aspects of the automation exceeded expectations, and those that fell short. Both should be analyzed to understand the discrepancies and to look for opportunities for improvement. Additionally, the actual results can be used to refine the models used in the earlier steps to improve accuracy for analysis and prioritization of future use cases.

Taking an iterative approach to financial analysis enables automation teams to more quickly identify the use cases to focus on, then scrutinize the most promising cases to provide decision makers with enough detail to make an investment decision. Incorporating Analysis Validation into their process increases confidence in the model while demonstrating that the automation system has delivered on its promises. Looking forward, we should anticipate that those teams who take a disciplined approach to financial analysis will realize greater business benefits at a faster rate than those who take an ad hoc approach to analysis and implementation.

To learn more about creating a metrics driven network automation strategy, you can watch my on-demand webinar, “Measuring & Maximizing the Business Impact of Network Automation.” Once you’re ready start doing preliminary analysis, check out Itential’s Automation Value Calculator to start calculating your success metrics.

Article originally published on ITProPortal.com.