Yesterday disaster decided to strike and lucky me, it chose me to be struck.

Let me take you through the events.

I finished my work day drove home and had some dinner, during which I decided to read a book on my tablet. Whilst enjoying my food and book, an e-Mail notification popped up. My ISP informed me that they would do some maintenance work this day between 08:00 and 17:00. I got this mail at around 19:30.

I finished my dinner and decided to measure my internet speeds in case something went wrong during this maintenance and was rather surprised. I had almost 200 Mbit down and 30 up, I pay for 80/15.

Grand I figured, free upgrade! (I should have become nervous right there.)

But little did I know, something else changed too.

I first started to notice something was wrong when I tried to get into my jellyfin server.

Funny I thought, the container must be down for some reason. I logged into my server and checked, nope, up and running. Connected directly on the local network, worked perfectly.

So I tried another service, my webmail, also down.

The next logical step was, that my firewall was messing something up, I logged in, everything seemed fine, so I grabbed a laptop went down to the server and connected to the other side of the firewall and tried gaining access trough the open ports.

Worked like a charm, so it wasn’t the firewall either.

Had to be the modem now, so I rebooted it. No change, no services available.

I then dawned on me, something must be wrong with my dns-provider. I went back up to the office and

logged in. Nothing changed, all just as it has to be.

Maybe something with my dns-server then? I switched it on one of the devices, but no change.

This is where I got stuck, what could possibly have gone wrong?

I have a couple of wireguard-vpn connections to customers of mine, so I figured I’ll just open a tunnel and see if I can get access through that, I picked one where I knew they had a very, let me emphasize this VERY diligent IT department and figured, if it didn’t work via them something very bad must have happened.

To my astonishment, the vpn connection was refused.

Imagine me, sitting there in pajamas scratching my head in utter disbelieve, the internet was up, I could browse the web, only the dns pointing at me was failing and now an outbound vpn wasn’t working???

What the hell?

At this point it hit me in the face, they, as many others, have limited vpn access to my IP address. My static IP address that is not supposed to change, that I have had since 2014.

“When you have eliminated the impossible, whatever remains, however improbable, must be the truth.” As Sherlock Homes said. What remained was this, my IP must have changed. I checked and it had.

At that hour I could not reach anyone at my ISP so I redirected my dns entries to the new one and went to bed.

Today I called in, they could not explain what went wrong and promised a quick fix. At the time of writing this, my ISP could not tell me, if at least the new IP was static, if I would get back my old one or what went wrong.

I now, for 22 hours, have no access to many sites and can only hope, nothing goes wrong over there.

I also have no idea as to when this will be resolved.

For all I know, it is very possible, that in two hours I will have a new IP address and all services will fail again, including my own vpn I use to connect home, right now, I am, for the lack of another expression, utterly fucked.

In defense of my ISP, in 9 years of being with them I haven’t had as much of a short internet outage, this is the first error they made and I got callbacks during the day informing me of progress and everything, I really can’t be mad at them, they messed up, but everyone does once in a while. Still sucks…

But it made one thing very clear to me, everything needs a backup, even ones IP address.

Right now I am contemplating on how to solve this for the future, the two ideas I am coming down to is either getting a separate connection, something not to expensive with a second static IP, or getting a VPS in some datacenter where I can setup a wireguard server and have that outgoing IP registered with my clients as well.

One thing I am certain of is, this can’t happen again.

Any suggestions welcome!