fsrtwrace.net
- Thread starter epwatson
- Start date
epwatson
Charter Member
So here is the deal on what happened.
I run the site on a shared hosting provider in the mid-western United States. This services does allow "unlimited" bandwidth but under a fair use policy. Normally my service gets maybe a couple hundred MB of bandwidth per day, so big deal. During the 4 days of the race, it accumulated almost 4GB of bandwidth, which normally isn't bad but based on normal usage this was out of the ordinary. As such, the provider enacted a policy to "SLOW DOWN" the traffic. It would block various IP's for a period of time in a hope to slow down the traffic. To top it off, the error page it tried to output was actually on the server so since the server was denying requests it couldn't "find" the error page and hence threw out an Error 404. This was due to a misconfiguration.
Now alot of this was my own fault unfortunately. The main page had a map on it and team AVSIM uses a tracker for the race (includes weather charts and our various route options). Both of these maps used a 60 second update frequency. It not only refreshed the current positions of the aircraft but also the route polylines. This racing polyline accounted for nearly half of the bandwidth usage, this one request. The problem was instead of just asking for the most RECENT part of the route, i asked for the entire route every refresh for all 3 teams, so as the race progressed, this data pull got longer and longer. Not only does it pull the ICAO code of the airport but the co-ordinates and the information put into the info windows that pop up when you click on them. By Monday evening this was turning out to be 300kb each pull. So if you left this map up all day, it pulled this file 1440 times times the number of people pulling the site. While checking the logs, I found out during peak times the server was being hit with up to 30,000 requests per hour. Every map refresh was 5 requests (1 for path, 1 for current positions, and 1 for each airplane icon created dynamically). This was on top of the normal forum refreshes people were doing and such. To be honest I NEVER in my wildest dreams imagined such data usage all at once. I did manage to throw in a few optimizations Monday night to help the server recover on Tuesday.
So NEXT year, i am working out a new strategy. I will either drastically optimize my code to slow down the bandwidth demands or I will purchase one month a virtual server (dont need dedicated) to run the site for that one month to hopefully eliminate this issue.
My sincere apologies for the outage. I had no way to predict it.
I run the site on a shared hosting provider in the mid-western United States. This services does allow "unlimited" bandwidth but under a fair use policy. Normally my service gets maybe a couple hundred MB of bandwidth per day, so big deal. During the 4 days of the race, it accumulated almost 4GB of bandwidth, which normally isn't bad but based on normal usage this was out of the ordinary. As such, the provider enacted a policy to "SLOW DOWN" the traffic. It would block various IP's for a period of time in a hope to slow down the traffic. To top it off, the error page it tried to output was actually on the server so since the server was denying requests it couldn't "find" the error page and hence threw out an Error 404. This was due to a misconfiguration.
Now alot of this was my own fault unfortunately. The main page had a map on it and team AVSIM uses a tracker for the race (includes weather charts and our various route options). Both of these maps used a 60 second update frequency. It not only refreshed the current positions of the aircraft but also the route polylines. This racing polyline accounted for nearly half of the bandwidth usage, this one request. The problem was instead of just asking for the most RECENT part of the route, i asked for the entire route every refresh for all 3 teams, so as the race progressed, this data pull got longer and longer. Not only does it pull the ICAO code of the airport but the co-ordinates and the information put into the info windows that pop up when you click on them. By Monday evening this was turning out to be 300kb each pull. So if you left this map up all day, it pulled this file 1440 times times the number of people pulling the site. While checking the logs, I found out during peak times the server was being hit with up to 30,000 requests per hour. Every map refresh was 5 requests (1 for path, 1 for current positions, and 1 for each airplane icon created dynamically). This was on top of the normal forum refreshes people were doing and such. To be honest I NEVER in my wildest dreams imagined such data usage all at once. I did manage to throw in a few optimizations Monday night to help the server recover on Tuesday.
So NEXT year, i am working out a new strategy. I will either drastically optimize my code to slow down the bandwidth demands or I will purchase one month a virtual server (dont need dedicated) to run the site for that one month to hopefully eliminate this issue.
My sincere apologies for the outage. I had no way to predict it.