September 19, 2017

Capacity Planning for Next Generation Utility Networks (Part 2)

Capacity planning guidelines for utility transport networks

In PART 1 of this blog, we provided an analysis of utility applications and expected capacity demands. Applications, especially security and surveillance, are rapidly increasing capacity requirements for utility networks. In this blog, we will outline an approach for performing capacity planning for a sample utility network in light of the applications traffic requirements.

Figure 2 shows the sample network and per-site capacity requirements – which includes nine substation sites and one main control site – each running all services outlined in PART 1. While rings are the preferred utility topology due to fault tolerance, they also create more complex capacity planning scenarios. To keep the analysis simple and conservative, we will use the lower-end capacity range for each application.

Figure 2 – Sample network diagram and capacity requirements.

Site 1 represents the control site, where most services originate and terminate. Traffic loading for each site is calculated by aggregating traffic for all the applications running at that site (per the table above). Traffic loading for each microwave link is calculated by aggregating traffic flows from each site to the control site that traverses the link. Under normal conditions, traffic from each site will follow the shortest path to the control site, assuming a basic IP network transport scheme. Based on that, traffic loading for each link is shown in Figure 3. The green bars represent the density of traffic as you move closer to the control site and represent how different links can require a different capacity based on their proximity and route to the control site.

Figure 3 – Traffic loading for each link under normal operating conditions.

It is important to note that the traffic loading should not be confused with network capacity. The traffic loading represents what the per-site and link loading will be, based on the services running through the network. The necessary network capacity is calculated based on traffic loading, failure scenarios and future growth factors. This will be covered a little later.

The proper dimensioning of each microwave link needs to consider additional traffic loading as a result of failure scenarios. In a ring topology, a failure on one of the two links feeding the control site (Link 1 or Link 10) represents the worst-case scenario for traffic loading as it shifts the traffic from all remote sites to the other link feeding the control site. Figure 4 below shows traffic loading for each link after a failure of Link 10. In this case, Link 1 will need to handle traffic from all remote sites.

Figure 4 – Traffic loading for each link under a link failure scenario.

Similar to Figure 3, the red bar represents the density of traffic around the ring and once again, the loading increases drastically as you approach the control site. In comparison to the loading under normal operation conditions, failure condition changes traffic loading significantly. With ring systems, link and network capacities need to be based on worst-case failure scenarios. It is also important to dimension each microwave link so that it operates without congestion, no more than 80% of its capacity, under such failure scenarios. This helps provide optimal performance for delay and jitter sensitive applications under failure cases. For example, in the sample network, the worst-case traffic loading on Link 1 is ~452Mb/s. Therefore, it should be designed with a minimum capacity of 570Mb/s (excluding growth); this ensures that even with a utilization factor of 80%, the link can handle the necessary loading of the link.

So far we’ve considered a failure over link 10, which adds the highest loading burden on Link 1. Another scenario to consider is a failure over Link 1, in which case the traffic would traverse in the alternate direction and the major traffic-loading burden falls under Link 10. In fact, in this second failover scenario, the traffic loading per link becomes a mirror reflection of the loading we just calculated. This is because the sample ring network provides symmetry in terms of sites and routing to the control site. In this latter calculation, you could rotate the diagram shown in Figure 4 diagram by the y-axis.

Another factor to consider when dimensioning the microwave links is the anticipated growth in network size, by adding spur links or sub-rings for example, as well increasing traffic demands of the existing applications or new applications. Growth factors can be included in the initial design of the links by reducing the load factor under worse case failure from 80% to 60% or less. Alternatively, the link design can be modular allowing for incorporating additional RF channels based on future demands. Considering utility networks are deployed to last at least 15 to 20 years, growth factors are important aspect of any capacity planning effort. Typical growth factor variables range from 15-20%.

Given the above analysis and considerations on how to properly dimension traffic, we can summarize the equation for link capacity as follows:

This equation demonstrates the relationships between the traffic loading, utilization and future growth. If we apply this equation to the links (assuming a 20% growth factor) in the above sample network, then the per link capacities required for this design are:

Similarly:
Link 2 Capacity: 603Mb/s
Link 3 Capacity: 527Mb/s
Link 4 Capacity: 452Mb/s
Link 5 Capacity: 377Mb/s
Link 6 Capacity: 377Mb/s
Link 7 Capacity: 452Mb/s
Link 8 Capacity: 527Mb/s
Link 9 Capacity: 603Mb/s
Link 10 Capacity: 675Mb/s

You can see the symmetry in the above capacity requirements, which follows the requirements of failover in case the closest links to the Control site fail. The above capacities represent the minimum per link capacities needed to ensure adequate bandwidth under normal operating conditions, worst-case failover scenarios and to account for growth.

Summary

Utilities require ultra-high capacity networks to keep up with growing volumes of mission critical data. While “backbone” links in older networks were designed with 50Mb/s, newer networks can easily require a minimum of 300Mb/s, and upwards of 1Gb/s. Factors such as the size of the network, network topology, failure scenarios, the volume of services and anticipated growth will affect these estimates. Making assumptions about network configuration and traffic loading, this blog provides a baseline for calculating the network link capacities. This analysis can and should be extended to other configurations and can easily be adjusted to fit either different network topologies, different traffic loading of individual services, or a combination of the two.