Server Upgrade and Downtime

Just letting everyone know that if you were having trouble reaching TheBeachcats.com over the weekend it wasn't your fault. I know there are a lot of fellow nerds here so I'll give some details on what happened.

I upgraded to a new dedicated web server. The old one was going on seven years old, it was functioning but it's OS software (Cloudlinux) could not be upgraded to the latest version. So I spec'd out a new server at the same datacenter where the old one was located. the servers both use the WHM/cPanel control panel which includes a ttransfer tool to clone the content and settings to the new server.

That went well and I was able to test the site functions using the IP address of the new server. All looked well so I changed the name servers to point to the new server IP.

DNS propagation is a dicey thing since DNS servers all over the world have there own policies. Some update quickly and others use extended cache times and can take a couple of days.

I wasn't very worried about down-time since the old server was still online and figured that anybody who's dns server didn't point them at the new server, would still see the old one.

After I started getting texts from members who couldn't get to the site I realized there was a problem (I could see the site, but others could not.).

After a bunch of investigation and trouble tickets with the datacenter it turned out that DNSSEC was the problem. I had forgotten that when using DNSSEC hashes to verify the DNS, that the hashes had to be cleared and updated to reflect the new DNS server IP address.

After that was fixed the site started propagating better, but slowly, around the world. Even now there are many that haven't updated.

So spread the word that TheBeachcats.com lives!

--
Damon Linkous
1992 Hobie 18
Memphis, TN

How To Create Your Signature

How To Create Your Own Cool Avatar

How To Display Pictures In The Forums.
--
Thanks Damon for the update.
Strong work!
Bob

--
Sheet In!
Bob
_/)_____/)_/)____/)____/)_____/)/)__________/)__
Prindle 18-2 #244 "Wakizashi"
Prindle 16 #3690 "Pegasus" Sold (sigh)
AZ Multihull Fleet 42 member
(Way) Past Commodore of Prindle Fleet 14
Arizona, USA
--
I tried to access the site, but did not get a redirect until today. Not a very active time of year anyway, but good to see things up and running well.

--
Tom
NACRA 5.7 (1984 Sail 181)
Pennsylvania
--
I want a refund !!!! ;)
What language is that post in? I understood...nada...
Computers are still magic boxes to me.
Tried to place a for-sale ad today and received a database error :(
samc99usTried to place a for-sale ad today and received a database error :(


It's a coincidence that you found this old topic right when the site had issues again.

The database errors should be cleared up, try again and let me know please.

This recent issue was a doozy. This past Friday, the physical drive on the web server that contains the backup partition was failing, to many disk errors to complete backups. So the server was taken offline in the night and the drive replaced and backups started working. (there was a LOT more involved to get this done without too much danger of complete data loss).

Then just four days later the primary NVME hard drive that contains the OS, boot, and file partitions started failing, AND it turned out the backup install had been botched so that the only backups that existed for all the websites (including TheBeachcats.com) on the servers were located on the main (failing) drive.

The server had to be taken offline and shut down, physically removed from the rack and a new empty NVME drive installed, the failing drive was added as an additional drive to restore backups.

That worked until it didn't, TheBeachcats.com and six other sites got restored and then the drive failed completely. That would have been it for my web business EXCEPT that, back on Friday when the backup drive failed I had taken and downloaded complete account backups, so was able to upload them to the new backup drive and restore from there. A number of the websites are over 10GB (TheBeachcats.com backup files are 22GB!) so I was really glad I have ATT 1GB Fiber now and way faster uploads than I did last year.

Anyway it's been a fun sleepless week getting all my clients websites working perfectly again since the OS was installed with all default settings and many of them have customizations. Luckily most of them had very little disruption overall.

So please let me know if you note problems with this site, error messages or blank screens, raw code appearing, anything weird just shoot me a message and screenshot at webmaster @ thebeacchats.com

Thanks,

Damon Linkous

--
Damon Linkous
1992 Hobie 18
Memphis, TN

How To Create Your Signature

How To Create Your Own Cool Avatar

How To Display Pictures In The Forums.
--
Thank you Damon for all you do!!!
Fair Winds and Calm Seas!

--
Sheet In!
Bob
_/)_____/)_/)____/)____/)_____/)/)__________/)__
Prindle 18-2 #244 "Wakizashi"
Prindle 16 #3690 "Pegasus" Sold (sigh)
AZ Multihull Fleet 42 member
(Way) Past Commodore of Prindle Fleet 14
Arizona, USA
--