A major outage that stopped T-Mobile customers from being able to use voice and some data services for much of Monday has been blamed on a fiber circuit failure, and redundancy issues.

The issue impacted other carriers when customers tried to contact T-Mobile subscribers.

T-Nobile

Server crash failure error outage
– Thinkstock

"Our engineers worked through the night to understand the root cause of yesterday’s issues, address it and prevent it from happening again," Neville Ray, T-Mobile president of technology, said.

"The trigger event is known to be a leased fiber circuit failure from a third-party provider in the Southeast. This is something that happens on every mobile network, so we’ve worked with our vendors to build redundancy and resiliency to make sure that these types of circuit failures don’t affect customers. This redundancy failed us and resulted in an overload situation that was then compounded by other factors. This overload resulted in an IP traffic storm that spread from the Southeast to create significant capacity issues across the IMS (IP multimedia Subsystem) core network that supports VoLTE calls."

Ray said that the company has "worked with our IMS (IP Multimedia Subsystem) and IP vendors to add permanent additional safeguards to prevent this from happening again," and was still working to determine "the cause of the initial overload failure."

It is not known which third-party provider’s fiber circuit failed. Reports on Monday suggested a Level 3 issue, but the company's owner CenturyLink disputed the notion.

Further reading