Complex Conversions Simplified

Large-scale conversions on time and under budget

The increasing nationwide drive to create ‘gigabit cities’ through high-speed data networks is inspiring many communities, municipal utilities, and telephone companies to jump into the broadband business. However, a complex underbelly of service management challenges can suck profits right out of an endeavor without careful planning and proper execution.

Even local telcos and municipal utilities that have marketed and supported services to subscribers for years may not be prepared for the unique challenges of broadband, especially when coupled with voice and video. Minimizing these hurdles requires effectively integrating telecom business operations with various vendor technologies.

Frontier Communications is a good case study for providers of all sizes who need to improve the integration and management of complex broadband business operations and technology. Frontier’s acquisition and conversion of former Verizon fiber customers reveals lessons for managing these challenges and mitigating the risk associated with them.

In July of 2010, Frontier acquired from Verizon Telecom nearly 200,000 customers, legions of new staff (about 8,000), and a complex array of legacy Verizon billing and back office systems that they (Frontier) agreed to lease only for a short time in order to save on expenses. The challenge was to figure out a way, in the time allotted, to successfully take full control of these subscribers in all respects: billing, provisioning, monitoring, etc. and to do so with as little disruption as possible to the customers involved, their services, or to Frontier’s cash flow.

Frontier had to figure out a way to quickly, economically and successfully take billing, provisioning and monitoring control of nearly 200,000 new subscribers – and to do so without disrupting their services or Frontier’s cash flow.

This was a serious mountain to climb and an incredibly short time in which to climb it. The good news is that with help from ETI Software Solutions, Frontier accomplished this task within the allotted timeframe and under budget.  How they got this done is where things get interesting.

Meeting the Frontier Challenge

Before looking at how Frontier was able to successfully convert these customers, it’s important to understand the specifics of what they were facing. The subscribers purchased from Verizon were spread across a myriad of states and supported by multiple FTTH technologies. Factor in PSTN gateways, core service routers, VOD systems and digital addressable control systems – all purchased from different vendors – and the challenge becomes even more difficult. Alcatel, Motorola, Tellabs, GenBand, Juniper and SeaChange …in order to take full control of the new subscribers, all of these systems and their requirements had to be addressed.

The first priority for Frontier was how to solve the provisioning challenges of the Motorola Digital Addressable Controller (DAC), the brains behind the STBs, since this accounted for the largest percentage of daily provisioning events. The journey from customer-initiated service request (e.g., ‘add HBO to my account’) to delivery can be a long and winding road. For many providers, this is a manual process involving a number of hand-offs starting with Customer Service and ending with network operations / fulfillment. Known throughout the industry as a ‘swivel chair’ process, manual fulfillment processes flow can be labor intensive, slow and error prone. For this reason, automation is not a luxury but a necessity.

“The amount of work to build a system in-house to implement and manage this integration [with the DAC] along with all the other technologies, would have taken years, but we only had seven months,” said Jesse Ross, Frontier Manager, Information Technology, OSS/BSS Applications. “We also knew that maintaining integrations to all these systems was going to be an ongoing challenge as we continue to roll out new services such as VoIP and cloud computing. It made sense to have a third party that can handle complex integrations and maintain them over the long haul.”

“It was easy to make a business case for Frontier in terms of a buy vs. build decision,” says Frank Gine, President of ETI Software Solutions. “When we started talking with Frontier, they explained that the customers they were buying from Verizon were spread across several states and supported by myriad different technologies – Motorola, Alcatel Tellabs, Juniper – the list just went on and on,” he continued. “The huge selling point of our Triad software is that it was already integrated and deployed with most of these [technologies] out of the box.  The icing on the cake was that we were able to provide all the provisioning capabilities that Frontier needed through one application, which greatly simplified the architecture from what existed with Verizon’s in-house application suite.”

Resolving the Switchover Headaches

Managing integrations between the front office applications, back office applications and the network is often one of the most challenging aspects for service providers. The infrastructure that providers build to offer triple play services, or even just video services, incorporates a variety of hardware and software products from a multitude of different vendors. Each product, be it hardware or software, will typically have its own proprietary API or interface specification that defines how external systems need to communicate with it.  While these interfaces enable an eclectic mix of products to work together, there is usually a high cost associated with building and maintaining them.

“The night of the conversion went smoothly and the ability to manage customers’ video services transferred to Triad without issue,” said Jesse Ross, Frontier Manager, Information Technology, OSS/BSS Applications.

Ross commented, “Integration projects can be very difficult, time consuming and expensive. You can either do this work in-house, go with large companies with framework-based solutions, or go with a smaller company with an off-the-shelf product that while maybe not as customizable still gets the job done. It’s difficult for many providers to do the necessary development work in-house since they don’t often have either the necessary skill sets on staff, or the bandwidth to tackle these kinds of projects,” Ross continued. “Big software firms are usually expensive. Licensing issues aside, the real, and often hidden cost is in the professional services required to write custom integrations within the product framework. These costs can easily skyrocket depending on the number of integrations involved or by how complicated they are to write.

“On the flip side, the risk in working with a smaller company’s product is its ability to scale, and how flexible it is,” Ross said. “It has to be able to adapt to your network configuration. The other concern is that the vendors you select, large or small, are going to be around for the duration.”

It required fewer than five meetings to get ETI staff up to date with the details of Frontier’s network configuration, to understand the transition issues for converting customers from Verizon’s systems, and to put a plan in place to make it happen. Ross quickly realized that with a single investment in ETI, he could solve all of Frontier’s video and access network provisioning needs. Moreover, ETI also offered well defined, business-level APIs that made it easy for Frontier to integrate their other systems to Triad. Coupling these inbound APIs with Triad’s out-of-the-box provisioning capabilities, Ross and Frontier had found the way off of the Verizon systems. To help ensure that the project went as smoothly as possible, ETI also taught Frontier staff how to use Triad to load, scrub and verify much of the data themselves.

“Working with a customer-focused company such as ETI allows us to reduce upfront costs. Plus, the quality of their custom software development is better,” observed Ross. “A bigger vendor’s software might have more features, but it definitely costs more to buy, plus it requires greater customization – and thus a bigger budget – than people expect. Large companies have less incentive to make customized changes, whereas ETI will evaluate a change request on its marketability and price its development accordingly.”

Ross estimates that without ETI and Triad, Frontier would have had additional labor costs for 33 people working for six months to convert the Verizon subscribers and integrate the systems, including 20 people just for software coding tasks alone. With a loaded labor rate of $80 per hour, this represents about $3.8 million in cost savings. Just as important was that Frontier staff alone could not have pulled off the conversion within the timeline they had to meet.

Bringing Data Services Online

ETI’s Triad continued to be a profitable investment for Frontier as they completed the turn up of flow-through provisioning of Frontier’s video interfaces. The next step was to do the same thing for Frontier’s newly acquired fiber optic infrastructure, spread across three technology platforms: Alcatel, Motorola, and Tellabs.
“Having three flavors of FTTH all under one roof isn’t a problem for Triad,” said Chris Beisner, ETI Vice President of Product Management. “When a Frontier customer calls in, no one has to touch anything. Triad talks to whatever EMS [element management system] is necessary, plus any related devices like the [GenBand] G6s and Juniper [service routers].”

Short and Long-Term Savings Boost Bottom Line

In addition to speeding up the switchover process and saving on human resource costs on the front end, Frontier has also found that Triad creates long-term savings by reducing ongoing labor costs. The application reigns in about 54,080 person hours per year – about 18 full-time employees – that otherwise would be spent managing end-user services. In addition, field technicians have access to Triad via proxy applications developed by Frontier. Made possible by Triad’s inbound APIs, this enables technicians to be more self sufficient allowing them to remotely manage device assignments, change services, etc.

Triad contributes to significant cost avoidance that further boosts the bottom line. Frontier’s Ross explains, “We have a sustainable model now that we’ve all but eliminated manual provisioning from our service operations. Before, if we added 100,000 subscribers, our cost to service them would go up because we’d need to hire more people to support the additional customers. Today if the same scenario occurred, we would have to hire only five people instead of 15 thanks to our automated processes.”

Frontier also discovered that Triad helps close the “drift” that that typically occurs between billing data and the ‘as-provisioned’ state of devices and services on the network when manual processes are in place. The longer these manual processes are left in place and the customer base grows, the greater this drift becomes, increasing the number of customer billing and service mistakes.  The automation brought about by installing Triad minimizes these mistakes and also speeds up service delivery – essentially recapturing lost revenue that results from delays in order completion. The result? Operating costs go down and customer satisfaction goes up.

Conclusion / Summary

For Frontier, the decision to work with ETI boiled down to time-to-market and low risk. Because ETI had the majority of interfaces required by Frontier already built and available, it was an easy decision to use them. The added benefit to Frontier for using an outside solution was that it freed up their IT staff to focus on integrating Triad into Frontier’s architecture and left them with enough bandwidth to also build an enterprise-wide frontend that tied all of these systems together.

While not every service provider may be faced with the same challenge of converting a large number of subscribers onto their existing platforms, almost all will wrestle with the complexities of system integrations. As the Frontier case study illustrates, attempting to build and maintain these integrations in-house may not be the most efficient or advantageous option. In order to make the best decision, providers must identify the priorities, opportunity costs, and risks associated with different approaches.