Server / Network Updates

[Tweak]

Nothing to be worried about but I wanted to pass the message on from our technicians.

We will be performing essential network maintenance between midnight and 4am BST on Friday 9th October 2009.

During this time there will be three brief interruptions to network connectivity in our data centre. Each service interruption should last approximately one minute.

Most likely it'll not even be noticed but you've now been notified.

[Dessum]

Thanks for the notice Tweak

[KittKat chunky~]

yep, thankies   

[Tweak]

Apologies for the downtime lately, we're currently investigating the issue fully and trying to solve the problem.

[Tweak]

Ok, we now have a reason for the system downtime but no reason for that reason. It's a serious technical issue which should not be ocurring, the server is stable again and we'll be investigating the issue all night.

An update to this problem should be available sometime tomorrow. Apologies for the inconvenience.

[Asia Kali Yusufzai]

As always, we are forever grateful for your endless toil.

If I had enough money, i would express it in wealth.

[Xleena]

Gaaah, you guys are always working too hard. Thanks for all the work you've done for the forums anyway

[Tweak]

And the plot thickens >_>

Server was stable all weekend and then became unresponsive an hour or so ago. We've got a sysadmin booked to take a good look at the server tomorrow.

Apologies to everyone, once we know the cause, we can work on fixing it.

[Tweak]

Curiouser and curiouser...

So a data centre sysadmin took a look at the server today and it's strange. The memory and swap dump is happening so quickly that the server has no time to log any issues, so at present we have no clue as to the problem. Some new logs have been set up to try catch the issue when it next occurs but we're currently suspecting there is a memory leak in one of the scripts hosted.

I will be taking some time to look through the newer hosted sites to see if anything is obvious, although I do have an idea which it may be (whether or not I'm right though). Rest assured though, this problem will be found and fixed and that all data is perfectly safe. Also rest assured it's not the forums or anything associated with them.

[Dessum]

Thanks for the update Tweak, glad to hear that the forums isn't in any danger!

[Tweak]

Thanks for the update Tweak, glad to hear that the forums isn't in any danger!

Now now, I didn't say that. I just said this issue wasn't caused by them!

[Dessum]

Still glad! x3 Any idea yet as to how long till the issue's fixed?

[Tweak]

Can't say for sure. Until the server burns up again we can't start to check what it is. I have done a few code changes on one of the sites on the server, we'll see if that makes a difference.

[Tweak]

As some may have notice we lost it again yesterday evening, unfortunately the logs are not showing anything so it'll be some trial and error changes to try to prevent it from occurring.

I have installed a new proxy server over the current webserver to try reduce some load, this forwards all requests through to the normal server but due to its improve handling of requests the main webserver can do less work overall, this is still being adjusted so please report any slow loading or other problems through out bugs board.

Apologies for all these problems, the easiest way would be to put another server in and load balance them, unfortunately at this time I cannot afford to do that. But the server is more than powerful enough, I just need to get it perfect.

[Asia Kali Yusufzai]

how very interesting. Let's just hope this can have some kind of amicable resolution. No need to do it fast, just that it's done at all is more than enough.