VGFacts

Hey everyone! I'm sure most of you are curious as to what's been happening these past 11 days. I'm sorry it took so long to get everything back online and then to post this thread but I am here to finally explain what happened.

11 days ago, I was doing some routine maintenance on the server and when I was done, rebooted it to make sure everything started up correctly. I'm fairly certain that nothing I was doing at the time could have caused what happened next but it's unfortunately rather impossible to say for sure. Anyway, the server was taking a ridiculous amount of time to restart so I decided to check into it and found out that it was hanging on boot.

After spending a few days attempting to recover, I resigned myself to the fact that the server was lost. I was able to boot into recovery and grab all of the files, including the database, so we didn't lose anything but it still meant I'd have to rebuild the server from scratch. This is where the fun begins.

This was my first time actually doing a Linux server build on my own and I wound up doing it three times in this process so I'm fairly competent at it now. Getting the server up and running was actually fairly simple. Most of the software installed right away, I figured out the best way to secure it, and even managed to successfully sync all of the backed up files from my local machine to the server. With some hiccups along the way and guidance from Rob when I hit a wall here and there, everything seemed to be going smoothly. So smoothly, in fact, that I brought the server back up the other night and many of you made it on to the VG Resource forum for a bit.

Unfortunately, nothing except the VG Resource forum was working. All of my code, which relied on a specific way of interacting with the database, appeared to be broken in this new environment and hours (and days) of troubleshooting proved fruitless. I was at my wit's end and honestly didn't know what I was going to do until today.

One of the first things I did, even before I started copying everything back to the server, was to go and change the saved passwords I had in my code for connecting to the database. I was on Skype with Dazz at the time and he can vouch for the fact that I actually did think of this step so when the site wasn't working, the fact that a password might be wrong didn't even cross my mind. The error log pointed to there being a problem with the database connection itself so that's what I spent my time researching.

Well... it turned out to be a password. I have two database connection functions written for the Resource sites - one to connect to the site database and the other to connect to the forum. I updated the password for the site database and the config file for the forum itself but forgot to change the password in my own forum function, which completely broke the site (as I'm sure many of you saw). Days of work (literally) and two server rebuilds later, it turns out all I had to do was update one line of code...

With that out of the way, a little housekeeping commenced, I fixed folder permissions which allowed uploads again, and everything seems good now. Of course, please report any bugs you run into and I'll do my best to address them as quickly as I can.

Welcome back, sorry for the extended down time, thank you for your support and encouragement, and let's all hope this never happens again!

TL;DR Server crashed, lots of work went into fixing it, even more work went into figuring out why it still didn't work, turned out to be a wrong password.

Thanks for trying really hard for us. I really like being around here we you guys. I mean it. Nice to be back with friends.

Come here everyone! Big Hug! Big Group Hug!

Just make sure you get some well deserved rest, Petie. It sounded like you and Dazz were pulling all nighters over this.

Good lord man, how have you not been driven insane? Well done and bravo.

I was really starting to worry is it was this heartbleed thing I keep hearing about. Its good to know you got things going again regardless of the fact.

Thank you for trying, boss. :)

Its good to see this back. :)

Awesome. Glad to see the forum back and do appreciate the effort that yourself, Dazz and the others do.

It's great to see everything back. Thanks for all your hard work!

Thank God! I had all this juicy trivia lined up to be submitted and was afraid I'd forget it by the time the site came back up. Thank you for all of your hard work!

(04-15-2014, 03:03 AM)Psychospacecow Wrote: [ -> ]I was really starting to worry is it was this heartbleed thing I keep hearing about. Its good to know you got things going again regardless of the fact.

Speaking of Heartbleed... Is VGFacts affected/fixed/going-down-again-to-fix-it? ;)

I mean, I appreciated the vacation. I had life stuff to catch up on. But I'm glad to have to site back up in time to take care of some things on my RSS feed before they expire.

Thank you for busting your ass to get us back up!!

(04-15-2014, 11:36 PM)Spherix Wrote: [ -> ]

(04-15-2014, 03:03 AM)Psychospacecow Wrote: [ -> ]I was really starting to worry is it was this heartbleed thing I keep hearing about. Its good to know you got things going again regardless of the fact.

Speaking of Heartbleed... Is VGFacts affected/fixed/going-down-again-to-fix-it? ;)

I mean, I appreciated the vacation. I had life stuff to catch up on. But I'm glad to have to site back up in time to take care of some things on my RSS feed before they expire.

I thought I replied to this already... Oops!

No, this downtime was not related to Heartbleed. While we are currently running a vulnerable version of OpenSSL, we're not actively using SSL on any of our sites at the moment so there is nothing to exploit. Of course, this does mean you should be careful with your passwords. Since you're logging in on an unencrypted connection, the risk of password capture is always there. While it may not be the end of the world to have your account here stolen, it would be a very different story if you used the same username and password for your bank account as well and they got in there too.

Best practice is to use a unique password for each site (I do) but at the very least, make sure passwords for more sensitive accounts (bank, PayPal, etc.) are unique.

As to the second half, once it's available through the normal channels, I will of course patch OpenSSL but that should only take a minute or two so there won't really be any downtime for it.