04-14-2014, 09:09 PM
Hey everyone! I'm sure most of you are curious as to what's been happening these past 11 days. I'm sorry it took so long to get everything back online and then to post this thread but I am here to finally explain what happened.
11 days ago, I was doing some routine maintenance on the server and when I was done, rebooted it to make sure everything started up correctly. I'm fairly certain that nothing I was doing at the time could have caused what happened next but it's unfortunately rather impossible to say for sure. Anyway, the server was taking a ridiculous amount of time to restart so I decided to check into it and found out that it was hanging on boot.
After spending a few days attempting to recover, I resigned myself to the fact that the server was lost. I was able to boot into recovery and grab all of the files, including the database, so we didn't lose anything but it still meant I'd have to rebuild the server from scratch. This is where the fun begins.
This was my first time actually doing a Linux server build on my own and I wound up doing it three times in this process so I'm fairly competent at it now. Getting the server up and running was actually fairly simple. Most of the software installed right away, I figured out the best way to secure it, and even managed to successfully sync all of the backed up files from my local machine to the server. With some hiccups along the way and guidance from Rob when I hit a wall here and there, everything seemed to be going smoothly. So smoothly, in fact, that I brought the server back up the other night and many of you made it on to the VG Resource forum for a bit.
Unfortunately, nothing except the VG Resource forum was working. All of my code, which relied on a specific way of interacting with the database, appeared to be broken in this new environment and hours (and days) of troubleshooting proved fruitless. I was at my wit's end and honestly didn't know what I was going to do until today.
One of the first things I did, even before I started copying everything back to the server, was to go and change the saved passwords I had in my code for connecting to the database. I was on Skype with Dazz at the time and he can vouch for the fact that I actually did think of this step so when the site wasn't working, the fact that a password might be wrong didn't even cross my mind. The error log pointed to there being a problem with the database connection itself so that's what I spent my time researching.
Well... it turned out to be a password. I have two database connection functions written for the Resource sites - one to connect to the site database and the other to connect to the forum. I updated the password for the site database and the config file for the forum itself but forgot to change the password in my own forum function, which completely broke the site (as I'm sure many of you saw). Days of work (literally) and two server rebuilds later, it turns out all I had to do was update one line of code...
With that out of the way, a little housekeeping commenced, I fixed folder permissions which allowed uploads again, and everything seems good now. Of course, please report any bugs you run into and I'll do my best to address them as quickly as I can.
Welcome back, sorry for the extended down time, thank you for your support and encouragement, and let's all hope this never happens again!
TL;DR Server crashed, lots of work went into fixing it, even more work went into figuring out why it still didn't work, turned out to be a wrong password.
11 days ago, I was doing some routine maintenance on the server and when I was done, rebooted it to make sure everything started up correctly. I'm fairly certain that nothing I was doing at the time could have caused what happened next but it's unfortunately rather impossible to say for sure. Anyway, the server was taking a ridiculous amount of time to restart so I decided to check into it and found out that it was hanging on boot.
After spending a few days attempting to recover, I resigned myself to the fact that the server was lost. I was able to boot into recovery and grab all of the files, including the database, so we didn't lose anything but it still meant I'd have to rebuild the server from scratch. This is where the fun begins.
This was my first time actually doing a Linux server build on my own and I wound up doing it three times in this process so I'm fairly competent at it now. Getting the server up and running was actually fairly simple. Most of the software installed right away, I figured out the best way to secure it, and even managed to successfully sync all of the backed up files from my local machine to the server. With some hiccups along the way and guidance from Rob when I hit a wall here and there, everything seemed to be going smoothly. So smoothly, in fact, that I brought the server back up the other night and many of you made it on to the VG Resource forum for a bit.
Unfortunately, nothing except the VG Resource forum was working. All of my code, which relied on a specific way of interacting with the database, appeared to be broken in this new environment and hours (and days) of troubleshooting proved fruitless. I was at my wit's end and honestly didn't know what I was going to do until today.
One of the first things I did, even before I started copying everything back to the server, was to go and change the saved passwords I had in my code for connecting to the database. I was on Skype with Dazz at the time and he can vouch for the fact that I actually did think of this step so when the site wasn't working, the fact that a password might be wrong didn't even cross my mind. The error log pointed to there being a problem with the database connection itself so that's what I spent my time researching.
Well... it turned out to be a password. I have two database connection functions written for the Resource sites - one to connect to the site database and the other to connect to the forum. I updated the password for the site database and the config file for the forum itself but forgot to change the password in my own forum function, which completely broke the site (as I'm sure many of you saw). Days of work (literally) and two server rebuilds later, it turns out all I had to do was update one line of code...
With that out of the way, a little housekeeping commenced, I fixed folder permissions which allowed uploads again, and everything seems good now. Of course, please report any bugs you run into and I'll do my best to address them as quickly as I can.
Welcome back, sorry for the extended down time, thank you for your support and encouragement, and let's all hope this never happens again!
TL;DR Server crashed, lots of work went into fixing it, even more work went into figuring out why it still didn't work, turned out to be a wrong password.