Overview
The introduction of 5G into the market, along with innovations in edge computing, and disaggregation of the RAN have raised the profile and importance of the mobile edge network. According to a recent CTIA survey, mobile traffic volumes have exploded, with a more than 108x increase of traffic in the past 10 years and more than 42.2 trillion MBs being delivered over mobile networks in 2020.
As a result, mobile operators are struggling to keep up with demands to expand network capacity and modernize the IP network components that are delivering traffic across their network, accelerating the need for automation.
The introduction of 5G into the market, along with innovations in edge computing, and disaggregation of the RAN have raised the profile and importance of the mobile edge network. According to a recent CTIA survey, mobile traffic volumes have exploded, with a more than 108x increase of traffic in the past 10 years and more than 126 exabytes of mobile data traffic in Q1 of 2023.
As a result, mobile operators are struggling to keep up with demands to expand network capacity and modernize the IP network components that are delivering traffic across their network, accelerating the need for automation.
Drivers for Backhaul Automation
There are multiple factors that are driving the need to evolve the IP mobile network, including:
- Throughput and latency demand for 5G
- Expansion of the access network sites (macro and small cell sites) to provide coverage and density for high-speed access methods such as millimeter wave.
- Increasing demand for hybrid-cloud and multi-cloud access at the edge.
- IoT Growth and growth in fixed wireless subscribers. Data-only devices are currently estimated to comprise 41% of all mobile devices, and this number is expected to grow significantly.
- Interest in 5G network slicing, and the need to support TN (Transport Network) slicing functionality.
- RAN disaggregation and introduction of technologies like O-RAN and virtualized RAN elements.
The net result of these factors is an ongoing need for mobile operators to initiate projects to upgrade and overhaul their IP backhaul networks. At the same time, they are looking for opportunities to improve operational efficiency and velocity. Many have used tools like scripts to automate at the task level, but most have failed to realize the reduction in effort and time that they’ll need to meet the aggressive timelines that their leadership have mandated. Others have embarked on more ambitious automation initiatives which have enabled them to not only meet the timelines, but to accelerate their new technology programs.
IP Backhaul Components & Architecture
To understand the benefits of IP backhaul automation, it’s important to identify specifically what the IP backhaul network includes.
Traditionally, the backhaul network included all the network elements that carried mobile traffic from the cell sites into the mobile core network. Backhaul elements could include optical transport, microwave, ethernet, or other components that were in the communication path. Increasingly, as mobile data has grown in volume and importance, IP technology has become the standard. For the purposes of this paper, we will be focusing on the IP components of wireless backhaul.
IP backhaul has traditionally comprised Cell Site Routers (the routers that are deployed at the cell site and connected to the mobile radio node) and the Aggregation Routers (routers that are deployed to hub sites and provide a hub connection to multiple cell site routers). The cell site routers transmit IP traffic to the hub aggregation routers, which in turn route traffic into the mobile packet core.
In the 4G architecture, each LTE cell site has a radio base station called an eNodeB. Each cell site typically has one Cell Site Router, which is connected by a communication link (typically optical or ethernet) to the Aggregation Router at a regional hub site.
Advances in transport technologies, and new capabilities in the 5G architecture have created several new options in the backhaul environment. These changes fall into a few different categories.
- Distributed RAN functionality, such as Open RAN (O-RAN) which focuses on virtualization and disaggregation of the eNodeB functionality into CU/DU/RU functions, so they can be deployed in a distributed model and enable mobile operators to optimize function placement for performance and cost optimization.
- Front-haul technologies, including introduction of options for fronthaul and mid-haul, by utilizing technologies like CPRI (common public radio interface) and fiber to enable centralization of eNodeB assets that had previously been required to be collocated at the cell site due to latency requirements.
- Growth in deployment of small cells to augment macro network coverage.
These technologies create opportunities for carriers to cost optimize their backhaul networks while still providing options for low-latency, high performance communications. Mobile operators can design their networks to take advantage of cloud technologies, virtualization, and merchant silicon hardware. With options, however, comes complexity.
The topology of the backhaul network is changing from a simple hub and spoke model to a potentially much more diverse model, with elements like virtualized RAN being distributed at multiple sites or technologies like CPRI creating additional links in the end-to-end communication path.
This complexity creates an opportunity for CSPs to leverage automation to mitigate the additional engineering complexity that technologies like O-RAN, CPRI, and virtualized routing introduce.
Essential Components of a Backhaul Automation Solution
At a high level, the elements of a backhaul automation solution are not that different than other automation or orchestration solutions, and can be mapped to the Automation & Orchestration Maturity Model
For the IP backhaul use case, the ability to automate and orchestrate a wider range of endpoints will be essential, as operators are quickly moving from physical devices to virtualized or containerized routing functions at the cell site, and are moving toward disaggregated platforms (routing software and whitebox devices) in place of traditional routers.
3 Phases of the Backhaul Automation Journey
PHASE 1
Task Automation with Scripts
Most automation journeys begin with simple scripts. These can be things like shell or python scripts and they typically focus on automating a specific task – for example, if the engineering team needs to change the NTP server on every one of their cell site routers, they may make a python script that will connect to each router, change the NTP server address, and move on to the next one. Scripting is very powerful for task level automating, so it is useful at the individual engineer level for improving productivity. However, the limitations of scripting become more apparent as teams attempt to scale up – by exposing the scripts to a larger audience and by trying to automate more complex activities with scripting. Neither of these scenarios works well. In the first case, the sharing scripts with other members of the organization requires training, version control, and usability factors such as code commenting and documentation. In the second case, where engineers attempt to automate complex activities with scripting, the overhead associated with maintaining the code base becomes overwhelming and scripting typically hits a ceiling with respect to scalability. Nonetheless, in some situations, it can be extremely valuable.
PHASE 2
Process Automation
The second phase of the automation journey focuses on evolving from task-centric automation to process-centric automation. This typically requires more sophisticated tools, but the business benefits are significantly better than task-centric automation for a few reasons. First process-centric automation includes more activities within the automation domain. The more activities that can be automated, the more manual activity and effort can be pulled from the process, and the more idle time can be removed. This results in faster execution of the business process with less effort. Using the example from above – while the task-focused approach focuses on automating the change itself, a process-focused automation approach would include all of the steps prior to and after the change. So in a process-centric model, the automation system would collect all of the variables to be changed from destination systems and network elements, it would create a list of target routers, perform the desired changes, and then perform health checks after the change was made to ensure that the routers are still operating properly, such as a call to an external test system or service assurance platform. The process-focused approach not only automated each step, but it eliminates any idle time between steps – so the changes can be performed more rapidly, with more accuracy and less manual effort.
PHASE 3
Self-Service Networking, with Support for Closed Loop & Machine to Machine (M2M) Automation
The third phase in the evolution of the Automator strategy focuses on exposing the automations to external teams and systems – so instead of a human being the initiator of the automation activity, the Automator is prepared to accept triggers from systems or from the network itself. There are a wide range of scenarios where this can have a transformative impact on engineering and operations activities. For example, the automation system could be configured to listen for DHCP requests from newly connected devices and perform a complex configuration of the device to enable it to participate in network service delivery.
Alternatively, the automation system could be configured to perform closed loop activities as described earlier. In this phase, the business value of the automation efforts goes up significantly, as the volume of activity, and subsequently the value delivered, can go up at a much faster rate due to the activities not requiring any interventions from engineering or operations to execute.
Key Backhaul Automation Use Cases
Several key use cases are emerging for IP Backhaul Automation to eliminate manual effort and enable engineers to focus on high-impact activities, and they fall into three categories: implementation, operations and service enablement.
Implementation
Implementation use cases include new device/function installation/configuration, capacity addition, and circuit/traffic migration. Implementation use cases take advantage of capabilities like Zero Touch Provisioning (ZTP) techniques to enable completely automated configuration of the new cell site routers or aggregation routers. In this scenario, the automation platform is integrated with engineering databases and the DHCP environment to enable autodetection of a new device, along with deployment of a gold-standard configuration modified with device-specific variables provided by the engineering database.
Capacity additions and migration activities are happening with greater frequency as a consequence of 5G roll-outs, with activities that can be automated including the provisioning of the cell site and aggregation routers to support the capacity change, as well as coordination and orchestration between the IP device configuration changes and the changes to the underlying transport, such as the optical links or other physical layer changes.
Service Enablement
5G Service Enablement automation activities are related to configuring and implementing the IP backhaul devices to support, or improve support, for advanced 5G functionality such as 5G network slicing. This topic will be explored in more detail later in this whitepaper.
Operations
Operations use cases include automation of the activities associated with ongoing health and operation of the IP backhaul network, such as IP backhaul device upgrades, configuration compliance, device health checks, maintenance window automation, fault triage, and break-fix configuration activities.
Automation of these activities allow operations teams to perform significantly more activities with significantly less manual effort – enabling them to improve network reliability and performance in multiple ways:
- By replacing manual keyboard inputs with automated changes, accuracy improves dramatically, and eliminates outages caused by incorrect input
- By automating activities like health checks and pre- and post-checks, operations team members can perform sophisticated checks in seconds instead of hours. This enables them not only to automate the checks that they were previously performing but perform more checks than would have been too time consuming in the manual method – improving accuracy and confidence that changes will not inadvertently cause disruption.
- All data associated with the execution of automated activities is saved to a database, which can be analyzed by machine learning systems to further improve quality and performance. Operations teams can begin to use this data to implement closed-loop automation use cases for triage and resolution of network errors.
Business Impact & Benefits of Backhaul Automation
Given the large number of IP backhaul devices currently deployed, and the maturity of the engineering and operations activities related to those devices, mobile operators have a reasonably comprehensive understanding of the costs and resources required to deploy and operate the IP backhaul network. This makes the IP backhaul network an ideal candidate to define and quantify the business impact and benefits for automation.
Based on our work with automating IP backhaul network elements for two of the largest mobile operators in North America, Itential has developed a comprehensive business analysis model that has identified the following areas as major drivers for business value.
Business Value Drivers
Reduce Manual Effort
Whether the use case is IP backhaul implementation or operations, automation drives significant reduction in manual effort. In multiple deployment projects, automation has shown to reduce manual effort for cell site router and aggregation router installation/configuration by more than 50%. For operations use cases, automation has been able to reduce manual effort by more than 90%.
Reduce Process Duration
Another area that has been particularly important for 5G programs is the ability of automation to reduce the end-to-end duration of a business process or project. Operators
who have used automation as part of their upgrade initiatives have been able to eliminate project backlogs and accelerate project timelines, given their project managers the ability to deliver key capabilities in advance of deadlines. The downstream impact of accelerating IP backhaul projects is a significant benefit to the other parts of the business, enabling leadership move forward on activating new features capabilities that have the potential drive new revenues, such as 5G slicing.
Increase Engineer Productivity
There have been several ways that IP backhaul automation has increased engineer productivity. For example, by automating time consuming activities like health checks, engineers can perform more activities in each maintenance window. Additionally, by using automation to execute multiple activities in parallel instead of the normal serial approach, engineers can perform more iterations by a huge factor. In one real world example, engineers that could only manually perform 2 upgrades in a five our maintenance window were able to perform 100 automated upgrades in those same 5 hours.
Free up Engineering Resources
from repetitive, simple tasks
Once the simple but time-consuming tasks (like health checks) are automated, engineers can use their time to focus on activities and initiatives that drive greater business value. Time is freed up to allow the engineers to work strategic projects and deploy new functionality, new vendors, and new capabilities.
Key Metrics for IP Backhaul Automation
On a recent IP backhaul project, the Itential team performed a detailed analysis to calculate the business impact of automation using two key metrics – workload unit cost (WUC) and engineer productivity. During this analysis, Itential engineers worked with the mobile operator to identify and quantify all the activities related to their manual processes for cell site router deployment, upgrades, and configuration management. A number of these activities also utilized scripting to automate specific tasks but fell short of full-process automation. The team also utilized data from previous projects to quantify the efficiencies of performing the same activities with
full process automation. For deployment related activities, the Workload Unit Costs decreased between 50% and 67%, while the WUC for operations tasks decreased by 70% to 90%. For those same sets of activities, engineer productivity improved by 200% for day 1 (deployment configuration for aggregation router) and by 600% for upgrade activities.
Business Analysis Highlights
As part of the business analysis, the team looked at multiple scenarios, ranging from extremely conservative to more aggressive. In all cases, the resulting cashflow from reduced effort was positive, and in most cases the payback period for implementing a comprehensive automation program for IP backhaul was below nine months. In the average case, payback was in 6 months, with the use cases that drove the most significant savings being device upgrades, device compliance, and maintenance window automation.
How Backhaul Automation is an Enabler for 5G Network Slicing Capabilities
While the business case for IP backhaul automation is extremely positive based on the efficiencies gained through automation of deployment and day-to-day operations, one of the major technical benefits that automation provides is its ability to support new 5G capabilities and features, such as network slicing.
The IP backhaul network is a component of the Transport Network (TN), which is one Of the three network slicing domains in the 3GPP model (RAN, Mobile Core, and Transport Network (TN)).
Within the 3GPP network slicing specification, the TNs involvement in network slicing is primarily related to ensuring that the network can meets the technical requirements for each network slice, particularly in the areas of latency, bandwidth, and traffic isolation.
The network slicing architecture identifies controllers which perform the “Network Slice Subnet Management Function” for each of the domains, including the transport network domain. The primary requirements for the TN NSSMF are that it can dynamically configure the transport network components to create the QoS and VPN features required by each network slice, and that it can expose the ability to manage the TN elements to a higher layer, cross-domain controller called the “Network Slice Management Function” NSMF.
An IP backhaul automation system can act as the NSSMF or in coordination with a TN NSSMF if it has the ability to work real time and can configure/manage the IP backhaul elements to create network slice config components that conform to the network slice profiles defined by the operator.
We see this as a significant opportunity for the IP transport teams to participate in providing more value to the operator and to its customers – when an automation platform can expose more functionality to other orchestrators and controllers, the network can deliver more responsively and can offer higher value capabilities.
Summary
The explosive growth of mobile data, and the forecasts for continued growth, are driving mobile operators to invest heavily in their IP backhaul networks. The intensity and volume of changes to the IP backhaul network is at the highest point in the history of the mobility industry, driven largely by 5G deployments and the accelerated adoption of technologies such as CPRI, virtualized RAN and small cells. Mobile operators have a significant opportunity to de-risk and accelerate their technology introductions, while dramatically reducing their cost to deploy and operate their IP backhaul networks through a process-centric approach to automation.
