Sat-3 sneezes, SA online catches a cold

That'll teach us to use cheap offshore hosting. A major undersea cable link between SA and the EU is having trouble, leaving internet service providers scrambling and customers fuming. Including yours truly.

For many years it was the saviour of South African internet users. The Sat-3 undersea fibre optic cable, properly known as SAT-3/WASC/SAFE, runs all the way up the coast of Africa to Spain. Before the arrival of johnny-come-latelies like Seacom, the vast bulk of internet traffic was moved either that way or via expensive, high-latency satellite connections. Even today Sat-3 still carries the majority of traffic between South Africa and Western Europe. And as of Wednesday it started having serious trouble – leaving much of that traffic homeless. Then, as traffic started being diverted to alternative links, those routes became saturated, and the knock-on rapidly reduced international access speeds for much of the country.

Worst affected were those with essential servers in data centres in, say, Germany. Companies like, say, The Daily Maverick (though it's just our primary mail servers; luckily our web servers are staunchly American). And the many, many internet service providers that rely on the European link as a backup for more direct routes to the United States, through which still flows the vast majority of internet traffic.

Let's put it this way. If one of the other undersea cables were broken by a ship dragging its anchor at the same time, or sudden sunspot activity causes trouble with satellite uplinks, then everybody would have been in deep trouble.

Telkom, the local operator of Sat-3 and the lead in the consortium that owns it, says this is a case of planned maintenance that went awry. Technicians were fiddling with power units in preparation for some work due to be done on the part of the cable between Accra and Abidjan – over the weekend, when the impact would normally be minimal. Something went badly wrong, leading to the outage. By late Thursday afternoon Telkom was estimating a return to full service within hours rather than days, but that doesn't come with a guarantee.

Don't even ask what would have happened if we were in the middle of the World Cup, with hungry Europeans soaking up high-definition (and bandwith-intensive) live video. True patriots don't ask those kind of questions. Instead they just pray really, really hard.

UPDATE: We hear that several major financial institutions had complete international connectivity outages and that both Sat-3 and Safe, the portion of the cable network that runs past Madagascar to India and points east, went entirely dark for around one hour on Thursday afternoon. Which isn't supposed to happen, because Safe is the redundant path for Sat-3, and some Telkom customers pay good money for that redundancy and the assurance that they'll never lose all internet access.

However, we also have confirmation that links on Sat-3 are starting to reach normal capacity again, as of 19:00 on Thursday, so it looks like Friday may be a better day. At least for everybody who doesn't have to explain to very important, very irate customers what, exactly, went wrong.

By Phillip de Wet

Thursday 21 January, 2010
Print | Email | Facebook | Tweet this | More  | Follow us on Twitter  | RSS
Top Stories

You must be logged in to leave a comment. Please login or sign up.
Well I am feeling the brunt of this technological mishap here in Lagos. I have been waiting the whole day for large files to be sent to me from SA, and the clients are fuming! I now really understand how our Nigerian friends feel when they say 'sorry, the internet it down' and we are not patient with them. I will try not to scowl when they start scoffing at we South Africans and our crappy internet.
Is that what happened? And no explanation from Telkom or courtesy message to its SA users? (Note to Telkom: SMS me this important bit of information if you can't email me. You have all my details already...) Bunch of bastards! I spent 40 minutes trying to get online today and when I did, eventually after accessing my email messages at a limping pace, I abandoned every document download I attempted. Growl.
Telkom don't give a damn, they're nothing more than crony, rent-seeking charlatans.
The question in my mind is why local companies take the short term view of using offshere hosting instead of developing the local IT capacity. Price is always a convenient excuse but I believe that if the will was there ways could be found to build local capacity which would lower the cost of hosting locally
Those companies that offer local hosting usually offer overseas hosting as well, and the difference in price and features makes it pretty clear where the problem lies.

The problem with local hosting is simply the cost of bandwidth in South Africa. While a 512mbps leased line might be affordable, it is hardly sufficient for hosting in an environment where many users have faster connections than that at home.

Getting a decent speed connection, on the other hand, will bankrupt most small companies. At the same time they have to compete with foreign companies that can offer higher speeds and almost unlimited bandwidth for a fraction of the price. Hardly a business proposition.

Hopefully things will improve this year. But we have been saying that for a decade, so I am not optimistic.