hi Stephen
Hope you got some shut-eye after last night activity…
Have you managed to get down to what the issue was? I want to write a summary email to my clients and would like to let them know some details, to help them feel more secure - so any details you can give me would be good.
(Such a summary would be good for you to give to your own customers too ;))
thanks.
me too
It is already planned and I’ve been gathering details today, I plan to have it out in 24-48 hours.
Quick summary now is, earlier in the day there were some fiber cuts and manually rerouted around them during this time some of the BGP announcements were sent via a different router. For some reason (this we are still trying to determine) this triggered some filters at other carriers 2 of which replied and got things on last night for us, finally. However we do not know why they did not add our route and indeed our AS number to their allowed filters.
The AS number is in the radb routing database, but some locations don’t use these, they do them all manually. So from there to get things up Zayo and RCN (a regional company in the north east) aided us by allowing our AS and IP to announce, which brought all the others up.
We are still trying to determine why exactly these filters that were set did not have us in allowed listing when they reconfigured, because we’d been there since 2010, and not an unknown/new IP range.
Good detective work to you and your techs Stephen, I was worried it was going to go on a lot longer than it did. Was the middle of a business day here in Australia when this all started.
thanks