A Data Center View of Fiber, AI

Fiber is one of the two key elements enabling the AI revolution, on equal footing with power. Over the past decade, fiber’s role within the data center has continued to grow as the need to build more compute within the same physical footprint is displacing the use of copper wiring for faster connectivity and lower power consumption. At the same time, fiber continues to respond to the call for faster connectivity linking data centers, enterprises, and homes as businesses discover ways for AI to improve their operations, call centers, and interactions with customers.

A few data center owners are focusing their expansion and operational efforts by integrating fiber operations into the core business, providing them with an intriguing model. Based in Michigan and primarily focused on providing services within the state, 123NET has been in operation since 1995, first starting out as a dial-up provider, then becoming a CLEC and building its first data center while starting to lay fiber.

“You don’t see too many data centers operating their own commercial fiber networks,” said Chuck Irvin, Chief Revenue Officer, 123NET. “It’s our mission to provide world class connectivity to Michiganders, for businesses and residents and communities. And so we haven’t shied away from anything that it takes to get that done. We’re very vertically integrated, and we actually do the majority of our construction in-house. We have our own directional drill crews, fiber splicing crews, design permitting, fiber placement; we do that all in-house. We believe that allows us to control costs, control those resources, and really control the customer experience of delivering service.”

Today, 123NET operates four large data centers and 4,000 miles of commercial fiber in Michigan, along with fiber to the home services that pass close to 40,000 residential addresses. Irvin said the company’s early decision to operate carrier neutral facilities, despite a short-term economic impact, proved to be the right one in growing its data center business.

“It’s been a really good decision for 123NET up to be a true carrier neutral facility in Michigan. In the early 2000s, Cogent was looking to establish a presence in our data center,” said Irvin. “We thought long and hard about whether we should be carrier neutral. We were selling bandwidth at the time for $50 a Mbps and Cogent was selling it for $3 a Mbps. By letting them in, they were going to be a tenth of the price we could afford to charge customers.

“But we had this theory that if you bring carriers in, you’re going to attract more data center customers more easily. Over the last two decades, we’ve become the carrier hotel in our state. More and more providers are bringing fiber cables into the data center. The data center’s value increases and becomes more attractive for folks to put their cabinets and their gear and workloads relative to how much fiber and connectivity there is.”

Irvin said that having 10 or more connectivity providers coming into their facilities made them the go-to place to move server workloads, since more enterprises and cloud services could quickly move data around to their respective customers, with the Detroit Internet Exchange turbocharging the data center’s popularity by enabling its members to directly transfer traffic among themselves.

Based in Michigan, 123NET has a complex portfolio of assets, including four large data centers in the state,
two Internet Exchanges, and 4,000 miles of fiber. Plus, it offers residential FTTH. Source: 123NET.

The more fiber, the more valuable the data center, as it provides resilience, enables customers to choose connectivity, and to directly connect to other
customers. Source: Source: 123NET.

“Today, there are 85 members in the Detroit Internet Exchange,” said Irvin. “Amazon, Disney, Meta, Apple, Google, T Mobile, Akamai, General Motors, Microsoft, Baker College are all participants. All of those providers have had to either bring connectivity into the data center with their own fiber cables or purchase connectivity across somebody else’s fiber cables to appear on the Detroit Internet Exchange.”

AI’s need for fiber within and outside of the data center depends on the workloads being serviced, Irvin said. For training AI models, the need is for fiber within the data center to connect GPU cores, servers, and racks, with the process being very power-intensive but not moving data outside of the building. Once the model is built, fiber connectivity across multiple paths is necessary to make it accessible and usable across the network. More connectivity means more accessibility.

“Southfield, Michigan, and our facility there is an interesting example of how cities and data centers come to be located,” said Irvin. “You often see cities near rivers, since they were important thoroughfares throughout history. The same with railroads. What we’re seeing now is data centers popping up where there are large concentrations of fiber. AT&T and Level 3 built their facilities in Southfield, you see so many different towers in the area. We built our data center next to Lumen, formerly known as Level 3’s data center, at the intersection of 10 Mile and Evergreen, one of the most densely populated fiber intersections in the entire state.”

By placing its data center in Southfield, 123NET was able to leverage the existing available connectivity in the area. Today, 40 different service providers come into the Southfield facility through eight physically different entry points. Connectivity into and within the building is available through any number of modalities.

“Customers might lease a fiber or a pair of fibers from one of our cables,” said Irvin. “Some might bring in their own cable, some might lease capacity on someone else’s cable. Some rent conduits from us and bring in multiple cables. We’ll sell single fibers, pairs of fiber, in batches of buffer tubes of 12.”

Depending on the circuit and distance, customers may be connected through a simple patch cable or want fiber spliced all the way to their racks on long haul routes to reduce losses.

In addition to the Detroit Internet Exchange, 123NET is in the process of growing its Grand Rapids Internet Exchange (GRR-IX) located on the western side of state, which will have its own unique fiber connectivity.

“It is going to have some connectivity into Chicago,” said Irvin. “With our partner PFN, we are working on an $87 million grant project to build two fiber cables across Lake Michigan, with one route connecting Chicago and Benton Harbor, Michigan. Chicago is a major connectivity hub, the internet exchange hub of the Midwest. Having a satellite exchange in Grand Rapids that connects to that will be beneficial to everyone in the region.”

123NET’s involvement with fiber to the home was an accident of circumstances. The company had built out a commercial fixed wireless network to support enterprise customers that included fiber built from its towers to the data centers, then reaching out further to deploy fiber to support its business customers.

“During COVID, we had a massive amount of inbound inquiries for residential services,” said Irvin. “We’re fortunate enough to work with all three of the major automotive manufacturers in Michigan. The head of purchasing from one of those companies called us up and asked us to build fiber in his neighborhood. There were 25 homes and he got half of them to sign up, so we built it as a pilot. It was a success. We found out the capabilities we had built in commercial fiber were highly translated to residential fiber. We started building larger neighborhoods and shortly thereafter won some grants and that turbocharged our residential growth. We expect to get close to 90,000 to 100,000 passings in the next three years or so.”