The following transcript has been edited for length and readability. Listen to the entire discussion here on The Broadband Bunch.
When you’re a service provider, being up-to-date means you’re falling behind. At Gosfield Communications, Rob and his team take the approach to concentrate on the here and now but also keep an eye on the future. Join us as we chat with Rob about the wins and challenges of being a service provider in rural Canada.
Craig Corbin:
Hello everyone, and welcome to another edition of The Broadband Bunch. I’m Craig Corbin. Thanks so much for joining us. In the world of communications, change happens quickly, and the ability for providers to adapt with the speed of light can determine their long-term viability in the competitive landscape. Our guest today planned and executed a forklift upgrade for a traditional telephony provider so that their core carrier network could support triple-play services and fiber to the home. He is a member of the board of directors for ITPA also on the NTCA technical committee, the Chief Executive and Technology Officer for Gosfield North Communications, Rob Petruk. Rob, welcome to The Broadband Bunch.
Rob Petruk:
Thank you for having me. It’s a pleasure to be here and happy to chat.
Craig Corbin:
Absolutely. There is so much going on that we can talk about, but before we get started, for those that are not familiar with you or a recent appearance that you made on the webinar series, our zero-touch learning series sponsored by The Broadband Bunch, which was a fantastic hit by the way. Give us a 30,000-foot overview of your background in the industry.
Rob Petruk:
Yeah, of course. I’ve got 24 years of experience in IT and telecommunications. 20 years being in particular in telecommunications, I started off in 1999 as a technician with one of the larger carriers here and basically was super fortunate and was able to hold a myriad of positions, roles, and responsibilities throughout my 11-year tenure there. Everything from, again, being a residential technician, all the way up to doing research and development on projects that were outside of the box of what a traditional telecom carrier would offer, I guess is the best way to put it. Working with some pretty crazy technology there, wearable computers.
Rob Petruk:
And this was back in the early 2000s before it was even relevant or prevalent today. Wearable computers, telematics, wireless hotspots, the whole ball of wax. My job was to test and vet and make sure that all of that equipment would work properly and talk to all of the other equipment that’s required to provide services like that. Fast forward 11 years at Bell Canada, and I was hired to Gosfield. They were looking for a technical and operations manager and alluding to what you mentioned earlier. They needed someone to come in basically and rebuild the carrier system from scratch.
Craig Corbin:
No simple process.
Rob Petruk:
It was really easy, a wink and a pinch and pulled a plug here, added a new one there and everything was up and running. No, actually it was pretty involved. It was basically a room filled with beige and brown equipment. I’m pretty sure there were some vacuum tubes involved in some of the equipment that we were still using. Fortunately, I’m experienced with DC voltage and vacuum tubes, so I wasn’t afraid of that kind of electrocution. But nonetheless, we decommissioned basically an entire room full of equipment and started replacing it with its modern equivalents, I guess you could say. We went from traditional analog switch telephone service to state-of-the-art IP-enabled voice services. Access hardware capable now of 10 gig residential services. Built an IPTV head-end from scratch so that we weren’t leasing service from another provider and just reselling another provider service.
Rob Petruk:
It was really important for me and for my board of directors to make sure that we owned the responsibility of making sure that the services would work appropriately. In a carrier industry, it’s one thing, it’s very easy to resell someone else’s product, but then when there’s a problem, you’re consistently telling your customers, hey, sorry, we have this issue, you’re making excuses, but you don’t have the ability to fix, repair, upgrade or maintain it. From my experience at Bell, I realized that in my opinion, that is the only way that as a service provider, you should operate.
Rob Petruk:
You need to be able to make sure that you are managing, maintaining, operating, and provisioning your own services on your own system. That way, if God forbid something does go wrong, you’re able to get it up and running, repair it, and maintain it appropriately in an appropriate timeline.
Craig Corbin:
Just a really quick, not to interrupt, but I’m really intrigued with what you mentioned about building your own IPTV solution. That’s another phenomenal effort on your part. Talk about that.
Rob Petruk:
Yeah, for sure. It was a lot of work and a lot of fun and a lot of headaches all at the same time. Again, dwelling on my experience as a residential repair technician, I always feel like a service provider, when we’re providing service, you have to take into account how the consumer or your subscriber is going to experience what you’re providing first. With the technical background I have, it’s really easy to get lost in a sea of routers and switches, and coding equipment and encryption equipment to make a system operate. But at the end of the day, your subscribers aren’t going to see that. They don’t care about the blinky lights in the building in the middle of nowhere. They’re going to see what pops up on their screen. They’re going to be concerned with the experience they have when they’re browsing through stations, how they interface with the service you’re providing, in this case, IPTV.
Rob Petruk:
What we actually did, or what I actually did was I went around to a bunch of the other independent phone companies that were providing television to see what systems they had in place and what they were using. And I actually did a road trip. I spent probably the greater part of a year visiting my peers and understanding what’s required, what their pain points are, what they like about the systems they were operating and running. And lo and behold, we settled on a system, a middleware system, which is actually what presents the information to a subscriber on the screen. We settled on a middleware system that I felt met all of our requirements from a company that was very willing to work with us and make sure that we launched appropriately. The company was its Innovative Systems. They were great to work with. They still are. Very supportive. They wanted to make sure that our launch was as successful as it possibly could be.
Rob Petruk:
At that point in time, after we had the middleware selected, we had to start to look at how we would get the content, which is the actual channels that you’re going to provide. There are a few different ways in Canada that you’re able to acquire television content, to provide to your subscribers. But geographically where we were, we were rather isolated in terms of connectivity to Canada’s fiber backbone and infrastructure, at least at the time we were. We had to go with an actual satellite dish solution. We had a really awesome gentlemen fly in on a helicopter, erect a bunch of satellite dish towers attached to one of our buildings and put up some massive satellite dishes that would help us deal with issues with rain fade and environmental conditions that we have in our neck of the woods. Basically to start ingesting all of our content over satellite. From there, I thought, hey, smooth sailing. We’re getting our content over satellite, it’s going to be great. Absolutely not, the worst case of my thoughts that I could ever have at that point in time.
Rob Petruk:
Once you get everything from a satellite dish, you still have to encode it and transcode it to be able to give it out to your subscribers over IP. It’s a little bit different with IP television than a traditional like RF or quantum system where you actually have to convert MPEG-2 content to MPEG-4 to deliver it over your network.
Craig Corbin:
Interesting.
Rob Petruk:
Yeah. The subscriber at the end of the day has no clue, care, or concern about any of this stuff, but-
Craig Corbin:
They just see the content.
Rob Petruk:
Yeah, exactly. But it makes a difference to us as a service provider. The reason being, the reason that we had to do all this work was, a MPEG-2 signal can come in anywhere from 10 to 22 Megs in terms of the size of the signal that you’re getting from the satellite. The issue with that is that if you’re providing service to someone who’s on a copper service from you and only has a 20 Meg connection to the host. Automatically, they can’t get that station. There’s just not enough bandwidth to the host.
Rob Petruk:
Worked with a company at the time who was Inca and is now VC to create an encoding and transcoding system to take those larger signals, compress them, tweak them, refine them, adjust them, and make them into a bandwidth size that was digestible for our end-use customers. I actually, at the time, employed a brilliant gentleman who helped basically stitch everything together and help us put all the pieces together. And lo and behold, from there, after about six months of testing, within a year of getting all of the equipment and stock, connecting it all together, testing the system, we launched our own IPTV service for our subscribers.
Craig Corbin:
That is so exciting. And when you talk about any kind of endeavor like that, you’re also dealing with the infrastructure that you inherited, the legacy copper, being able to find something that would perform well over that, as well as your fiber to the home, which we’ll get to it a bit later in the conversation. But I’m curious when you talk about content, of course, today that now includes streaming and over the top. And I think there’s an interesting story in the Gosfield history of the time when Netflix began their streaming services and a challenge that was presented almost immediately. Share that if you would.
Rob Petruk:
Absolutely. It was a terrible year. I had just been hired to Gosfield. The first major project they gave me was fork lifting our customer management billing system from an old AS/400 system to a modern Telco Customer Utility Management billing and integrated system, which I view as super awesome. And I can’t imagine existing with AS/400 in this day and age. But I think I could still program in it. But yeah, it was an interesting year. I was handed this massive project with a team that was not ready for this kind of transition of a major software platform. And it’s not their fault. There was just nothing like this that had ever gone on at Gosfield. We started with what I would like to call this forest fire of migration from one billing system to another. And then we to get into 2010, holiday season, end of the year, Christmas holidays and everything like that. 2010 in Canada was the year that Netflix offered its first-month free service and Netflix actually officially launched. It was Christmas Eve.
Craig Corbin:
Lovely.
Rob Petruk:
Actually, everyone thought the best Christmas gift would be to give all of their loved ones, Netflix. And at the time Gosfield only had two 45 Meg connections for all of our 1500 subscribers. If you do the math even quickly, there’s not enough bandwidth at all to support 1500 subscribers, all using Netflix at the same time. As I mentioned to you in our earlier chats, I basically drove to the Gosfield office, sat down in my chair, looked around and realized there was absolutely nothing I could do, started to well up and thought that I had made the worst decision of my life leaving Bell Canada.
Rob Petruk:
Yeah, the phone was ringing off the hook. Everyone was mad, my Internet’s not working. It basically brought not Netflix in particular, but our lack of capacity brought our system to a screaming halt. During probably one of the busiest times of the year, when everybody wants to make phone calls and chat, but at that time it would have been like things like MSN messenger and stuff like that, and IRC. But basically, everyone’s chatting and sending pictures and using the earlier precepts of FaceTime chat with loved ones, and we just didn’t have the internet capacity to let anything happen. Very fortunately though, just prior to this, I had chatted with my board and started the wheels in motion to do the forklift upgrade of our entire carrier system.
Craig Corbin:
Excellent. Excellent. So there was light at the end of the tunnel.
Rob Petruk:
Yes, it was a very dim, possibly a candle maybe behind an opaque black wall, but it was there and we could feel that it was there. We just couldn’t see it exactly. It took a little bit of time.
Craig Corbin:
I understand that. And I know that that was not a pleasant experience, but you live through it and the result has been phenomenal. And quite honestly that example, I think also ties to one of the concepts that I believe you hold very close and it’s with regard to the concept of being future proof. Well, you can’t really be future-proof, but you can be future-ready. Talk about that.
Rob Petruk:
Oh, absolutely. So again, 2010, I would say was a lesson learned. A hard lesson, but a very good lesson. Basically, that became the catalyst for me as the executive technology officer to never become complacent or to never be searching for what we need to be doing to marginalize any sort of software service piece of hardware that could ever come out or be launched again in the consumer market. It started a footprint of constant change, constant upgrades, constant evaluation and always looking, always managing, always working towards where do we need to be five years from now, where do we need 10 years from now? What equipment do we need to be putting in the houses now that will carry us for the next four or five years so that we’re not constantly upgrading, replacing, adjusting, fork lifting systems and hardware on a regular basis?
Rob Petruk:
That’s where, as a service provider, that’s where you lose a lot of money. That really set the foundation for us as a service provider to always look and see, take our historical data, apply some algorithms to it and look to see where we need to be 12 months, 96 months, 10 years down the road, and everything like that. Right now the benefit of all of the adjustments that we’ve made is now we don’t have to forklift anything. Now it’s just a matter of adding a component here or adding a component there. A really great example is we’re preparing to upgrade to 100 Gig transport links from 10 Gigs.
Craig Corbin:
Exciting.
Rob Petruk:
It’s really, really cool. We’re actually going to have access to 600 Gigs of capacity for Gosfield. And there are those who would say, well, maybe that’s unnecessary and I could see exactly, why don’t you just get multiple 10 Gig links and keep layering them and everything like that? Well, the issue with something like that is that, let’s say you have 10, 10 gig links. Well, what happens when eight of them fail? Or what happens when you’re constantly monitoring 10 different aspects or 10 unique aspects of your network, where if you upgrade to a hundred Gig links or you get a few hundred Gig links, you’re monitoring one big link.
Rob Petruk:
Yes. If it goes down, it’s terrible, but that’s also why you would have redundancy and backups and everything like that. That was another important lesson learned from the whole of Netflix. And I hope they’re not going to listen to this and try and sue me or hold me liable, but it wasn’t their fault. It was Gosfield’s fault. But the upgrade from 10 Gigs to 100 Gigs and everything like that is not just to meet our needs now, it’s to make sure that five years from now, we still have what is going to be required to provide service. There could be another service that launches this year. There could be another piece of hardware that comes out.
Rob Petruk:
I know we’re on the verge of 8K television and streaming. We as service providers need to make sure that we’re ahead of that curve so that we’re not in a situation where an app or service or something launches. And we don’t have the horsepower behind us to provide it. I do not want to spend another holiday or any other day of the year sitting in my office crying because we didn’t plan for something that could have been accommodated for. And again, going back to the comment you made about future proof, I hate using that term personally. You can never be future-proof, but you can be future-ready.
Craig Corbin:
That is so important for everybody in the business. And it’s obvious that there are considerations that have to be made in focusing on that. By the way, you’re listening to The Broadband Bunch and our sponsors UTOPIA Fiber, building a more connected nation, DxTEL, creators of the Harper broadband marketing library, and sponsored by your zero-touch automation experts, ETI Software Solutions. Our guest today, Rob Petruk, the Chief Executive and Technology Officer for Gosfield North Communications.
Craig Corbin:
And Rob, you talked so much about the change and the speed of change and being able to try to prepare Gosfield for what’s down the line years, not just months. But part of that equation is the reality that you and all providers deal with, what I guess could be best described as an onslaught of in-home technology. And that’s a huge challenge when you have to approach the way that you provide services to your customers. Share your thoughts in that regard.
Rob Petruk:
Oh, absolutely. And happy to. I was actually just speaking about this a little bit earlier today as well. We take a two-prong approach to it. The first thing that we have to realize as service providers are you have to decide where you’re going to deliver the internet to in a home or business. We utilize a two-box solution. Off of our fiber network, we go to an ONT. The ONT holds the phone, internet, and television services from there. Once we hit the ONT, we then have our customer premises equipment, which is talking about IOT and smart homes and everything like that. This is where having that flexibility between the connection to the house and what’s going on inside the house becomes paramount. Much like we do with our transport network and television services and internet backbone and everything like that. It’s important for us as service providers to make sure that we’re not putting in the technology that’s going to meet today’s needs.
Rob Petruk:
We need to make sure that we’re looking to a year down the road, two years down the road, three years down the road, four years down the road, and everything like that as well. I found in my time at Gosfield you kind of get stuck in this rut between, we’re going to put this piece of residential wireless mesh equipment in the house, and it’s going to be capable of X and Y. And everyone leaves Z out of the equation or Z, sorry. Depending on from the US or Canada who’s listening. But as service providers, we weren’t focusing enough, in my opinion, on where our trouble tickets were coming from and how this co-relates is you can deliver a 10 Gig service to a household. But the problem is that if a device in the house is having a bad experience, so someone’s tablet isn’t connecting appropriately, or it’s not getting the speeds that it needs to. The customer is ultimately going to be unhappy in their experience rating with you is going to be lower zero.
Rob Petruk:
When it comes to the customer premise equipment as service providers, I think we need to be on the cutting edge and make sure that we are providing not like A/B/G/N wireless. And in my opinion, I don’t even think service providers should be looking at AC anymore. We need to be looking at wireless 6 and AX. We need to be looking at mesh. We need to be looking at devices that we can put in the house that has a Zigbee and Z-Wave connectivity and things like that because ultimately our subscribers are going to go and buy things from Amazon other online marketplaces, Best Buy, wherever it might be, or they’re going to buy them from individual unique websites. It might be from Google, who knows. But they’re going to be adding this technology like ad nauseum to their houses as time goes on.
Rob Petruk:
And the issue is, is that if we don’t have, as service providers, a rock-solid foundation for connecting the wireless devices in the house, if our customer premise equipment and wireless mesh networks are not optimizing the devices that are connected appropriately, or in most cases, auto optimizing. If they’re not utilizing wireless transport between the access points and mesh nodes appropriately, all of the trouble report calls are going to come back to us as the service providers. One of the last lines of just about every piece of electronics that I’ve ever seen, that’s connected to the internet says, after you’ve gone through like these five to 20 steps, call your internet service provider.
Craig Corbin:
Exactly.
Rob Petruk:
Right. But I mean, if we’re servicing your house or your business with a Gigabit of internet, it takes a lot to consume a Gig in terms of bandwidth, like speed-wise, not consumption, not the goes into, goes out of data being transferred, but to actually utilize a full Gig. You’ve got to have a lot of stuff running and connected all at once in your house. And so, it becomes not a point of just the speed that you’re delivering, because a hundred Meg service might be enough. You need to make sure that what you’re implementing in the subscribers’, residents, business, home, whatever is appropriately managing the hardware and equipment that they’re purchasing that they’re bringing into their house at the same time.
Craig Corbin:
To that point, one of the realities is that all meshes are not created equal.
Rob Petruk:
Agreed.
Craig Corbin:
Because that’s something that plays a big role in the performance of multiple devices around the home.
Rob Petruk:
Yep. 100%. As service providers, just myself, we buy tons of equipment and test it. As a team, as the Gosfield team, I pretty much throw this out to my board of directors and my staff like, hey, I need to test this equipment, who’s in? And usually, everyone’s pretty eager to put up their hand because they know whatever one gets selected is the one that we’re going to have to support anyway. And we’re small enough that that works, but I’ll do some prior vetting to understand how the technology is going to work, how they interact, how are the access points or nodes or mesh nodes talking to each other? Can they be wired? Are they wireless? Are they capable of both? You basically have to go through a pretty advanced checklist before you even bring a product into your fold to test in real-time, let’s say. A lot of the research on a mesh node or on a wireless mesh system can be done by readings through the white papers and understanding how they’re going to connect to each other.
Rob Petruk:
It’s one of the benefits that we have right now is that there are lots of providers that are coming out. In my opinion, some have more appropriate methods of the nodes communicating than others. Some are a little bit more service provider friendly. Some are a little bit more consumer-friendly. Generally, obviously, we’re going to stick with the ones that are service provider friendly, but we also have to be aware that if we’re providing service to house X, house Y may have a system that’s going to interfere with the wireless service that we’re providing to the adjacent house. There’s this other variable that we also have to take into consideration at the same point in time. We go through a pretty serious, and I go through a pretty serious vetting process when we’re determining what we’re going to deploy and why.
Rob Petruk:
One really important feature though, that I think should never be overlooked by a service provider and something that we’re implementing is you need to make sure that whatever system you’re putting in gives you and your technicians and your support staff the ability and the visibility to understand what devices are connected in the customer premise and see what devices could be having issues, and why. Because ultimately they’re going to call us when something’s not working as I’d mentioned earlier.
Craig Corbin:
And that’s huge?
Rob Petruk:
100%. And the other thing too, as service providers, you need to be able to push that ability to the subscribers themselves. It’s almost like after they’re done their troubleshooting checklist and it says, calls your ISP. You need to educate your subscribers and you need to make sure that you have a system in place that tells them no, no, no, go to your My Gosfield login. And this is something we’re implementing right now. And go look in our support section. Read through what’s in there first. Look at your wireless mesh network that shows a list of the devices that you have connected and look and see what’s connecting poorly and let’s try and figure out why. My goal is within a year or two, if not sooner, to have customers doing this sort of self-serving support and authentication of what could be wrong on their network before they call us.
Craig Corbin:
I love that.
Rob Petruk:
Yeah. And push the ability. Let’s say they do need another mesh node in the house or something like that. We want to facilitate them being able to order that through a web-enabled interface. We still want people to call in and are happy to have people call in, and we’re happy to chat on the phone. But the reality is that that’s not conducive for everyone. And some people don’t want to call in and talk to someone. They want it fixed immediately. They want to read about it on the internet. They want to click a radio button, hit submit, and then have the wireless node show up at their house.
Craig Corbin:
Yeah, Rob real quickly to that point, the topic of digital literacy. You talk about educating the consumer. Digital literacy is such a huge challenge for some, but if you are making an effort as a company to place a premium on that, I would assume that you’re going to see a very positive long-term response by raising the level of knowledge in your consumer base.
Rob Petruk:
I agree 100% as well. Yeah, the goal for us and for Gosfield as we’re rebuilding our website is that there is an interactive index support section. You can go and type in your question. If the question’s already been answered, an answer will appear. If the question hasn’t been answered, or if it’s a new or unique question, you submit it. And we post the answer there for all to see in a searchable indexed, browsable manner, basically. And I think the biggest hurdle for that is going to be getting the subscribers used to going there to look, in some cases. And as odd as this sounds it’s really not an age demographic thing. I have subscribers who are in their 60s and 70s who have gone back and taken courses on networking at our local college or taking courses iOS and everything like that because they have an iPad and they want to know how to use it better. And we politely say, we’re happy to show you everything we can. But the reality is we’re not specialists in everything.
Rob Petruk:
So, go take a course. And lo and behold, I have some subscribers who are in their 60s and 70s who can run circles around just about anybody on iOS or Android now. And that sort of set that mentality for me of the more we allow our subscribers or the more information we can give to our subscribers, the more information we can present and make accessible. The more likely they are to go there first because ultimately you want to fix your own stuff. Right?
Craig Corbin:
Absolutely.
Rob Petruk:
You don’t want to be calling someone.
Craig Corbin:
Exactly. And you may have mentioned earlier making information available to the consumer, be it telemetry or whatever metrics are available, but just putting the power in their hands also then decreases the load on your staff. So that much of what can be handled easily is already handled before they have to call if they have to call.
Rob Petruk:
Yep. That is it exactly.
Craig Corbin:
That’s awesome. Yep. We have, unfortunately just about run out of time, but before we wrap things up, I do want to ask one of our traditional questions, Rob. And we refer to it as the back to the future question. And assuming that you could hop in the DeLorean and take yourself back X number of years and whisper something in your ear that would change the trajectory or the timeline of certain developments within the organization. What would that be?
Rob Petruk:
That’s a really good question. I have two parts to the answer. The first thing is I don’t think I should have been as afraid to fail earlier on in my career.
Craig Corbin:
That’s a great answer.
Rob Petruk:
I have learned so much by making mistakes that now the mistakes become fewer and further between because you’re able to take those past experiences and apply them to what you’re doing going forward. You can start to understand the hurdles and roadblocks, and staggers, and jags ahead of time. And again, it goes back to like the future-ready comment. You can almost avoid a series of horrible things going wrong by having these previous experiences of failure. Now, obviously, you wouldn’t do it in a production setting, but don’t be afraid to experiment. Don’t be afraid to fail. That’s basically what I should’ve told myself 20 years ago. Fall flat on your face and just pick yourself up, dust yourself off and keep going.
Craig Corbin:
And learn from it and succeed to even greater, greater heights. And you have done a phenomenal job of that. By the way, before we wrap up, I do want to make mention you are, I believe if memory serves, on the list of speakers for a very notable event in the fall, I believe the Broadband World Forum, you’re on tap for. That’s exciting.
Rob Petruk:
It is. I’m unbelievably excited and happy to even be invited and unbelievably excited to participate. I absolutely love doing stuff like this. I love chatting with people and I love sharing our stories. And I love hearing others, other stories of others as well. I figure in our community, especially in telecom, the more we can learn from each other, the faster we progress. The faster we can get to this Star Trekian sort of, where there’s a single touch screen in a room and you just talk into the air, and everything kind of happens.
Craig Corbin:
I like it.
Rob Petruk:
I’m really about advancement. So, I’m happy to share and happy to be a part of this community. Thank you also for having me.
Craig Corbin:
Oh, absolutely. And Rob, thank you for the time. I do look forward to the opportunity to circle back and visit again and learn more about what’s going on at Gosfield North Communications. Phenomenal success story. We appreciate you, what you’ve done in the industry, and for certainly being part of the podcast today. Our thanks again to our sponsors, UTOPIA Fiber, DxTEL, and ETI Software Solutions. On behalf of everyone here, thanks for letting us be a part of your day. We’ll see you next time right here on The Broadband Bunch. So long, everyone.
Join us here on the web at Broadband Bunch, to see the latest episodes, news, and information. The Broadband Bunch is sponsored by Utopia Fiber, DxTel and ETI Software.
