Well it finally happened, i managed to ready a different web server. For anyone interested i switched to lighttpd aka lighty from Apache2.
Hopefully this will resolve the slowness the site was experiencing from time to time, however if that isn't the case and you experience any kind of slowness please let me now, either as a reply here, in a pm, the char or mail.
Enjoy the new server.