Victorian firefighters extinguish online congestion during bushfires
- 27 February, 2013 14:51
Firefighters at CFA Victoria deployed content delivery management (CDM) software during the recent Victorian bushfires as an influx of residents using a mobile app that provides fire warnings caused its website to fail.
CFA is a first responder and emergency management organisation funded by the state government in Victoria. It has 61,000 people, including 59,000 volunteers, operating out of 1253 fire stations around Victoria. Bushfires are just one area in CFA’s jurisdiction; it is also a first responder for road rescue, hazardous materials and other emergencies.
IT’s role at CFA is to communicate information to both internal operations people and the general public who the organisation protects, CFA executive manager, ICT services, Michael Foreshew told Computerworld Australia.
This includes informing fire fighters how best to extinguish a bushfire and sending warnings to the public about potential danger areas.
CFA used to rely on mainstream media to communicate warnings to the general public, but about three years ago began using Internet technologies to directly warn citizens abut emergencies, Foreshew said.
CFA has a system called “One Source, One Message” that allows CFA officials in the field to produce alerts that are automatically and simultaneously sent to the CFA website, Twitter, Facebook, mainstream media and the federal SMS alert system.
CFA also produces RSS and XML data feeds that can be read by mobile apps and other readers. Apps have been developed for Android, Apple, Windows Phone and BlackBerry mobile operating systems.
On 4 January with bushfires raging across Victoria, one mobile app called FireReady “bombarded our website ... with so much traffic that the website would time out waiting for the data to come back,” he said.
“We initially thought it was a denial of service attack,” Foreshew said. “However, none of the perimeter security went off.”
Eventually, CFA discovered that a poorly designed FireReady app was the source of the problems.
The app drew “a lot of data every 60 seconds,” he said. “It doesn’t just take the changes; it reloads everything.”
In addition, “rather than query our proxy servers, which had been set up to handle this ... it forced our proxies to go back and request the data” from the database.”
The inefficiencies combined with 417,000 users using that app at once crippled the CFA website.
“The website was still producing the pages,” but anybody who tried to access the website received server timeout errors, he said. “What was then happening is people grew frustrated with the website, so they downloaded the app. It was a compounding problem.”
After identifying the app was the problem, CFA engaged Internet content delivery management software vendor Akamai, a company it had already been talking to about another initiative, Foreshew said. On 9 January, Akamai presented a possible solution on and deployed it in about four hours, “free of charge,” he said.
The software quickly resolved the network congestion, and CFA this week signed a one-year contract with Akamai after navigating a government review, said Foreshew.
At the peak of the emergency, CFA served 1.9TB of data in one day, Foreshew said. CFA’s 1Gbps link was running at about 650Mbps to 670Mbps trying to serve the large volume of data, he said.
However, after about 15 minutes with the Akamai service, “we were down below 13Mbps and we haven’t gone above that ever since,” he said.
CFA had to find money and satisfy procurement guidelines before signing the contract, Foreshew said. “CFA and a number of government organisations are facing reduced budgets, so we needed to reprioritise.”
The Akamai CDM was attractively priced and “provided a large number of servers across a variety of locations,” Foreshew said. Alternative CDM providers presented a smaller number of larger capacity servers in a centralised location, he said. CFA decided that the Akamai CDM provided better redundancy.
Akamai “properly” addressed cloud provisioning, he added. “What we saw in some of the other providers were outsourcers trying to buy their way through an inflection point of technology.”
Meanwhile, CFA is redesigning the mobile app that caused the traffic congestion in the first place. It is also exploring alternative ways to deliver information to mobile devices, Foreshew said.
“We’ve seen advancements in the use of HTML5, for example,” which could allow CFA to consolidate its mobile delivery rather than have a separate app for each mobile OS, he said.
“There are no sacred cows,” he said. “We [will] have a look at what is the most appropriate, most efficient, most effective way of getting the information out to people.”
Follow Adam Bender on Twitter: @WatchAdam
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.
Ruggedized scientific calculator perfect for extreme math
How to Switch From iPhone 5S to BlackBerry Z30 (and Why)
How to Switch From iPhone 5S to BlackBerry Z30 (and Why)
CIOs to Become In-House Brokers -- and That's a Good Thing
The future of computing
The F5 DDoS Protection Reference Architecture part 1 of 3
Distributed denial of service attacks (DDoS) attempt to make a machine or network resource unavailable to its intended users, with a wave of crippling attacks on enterprises since 2012. This whitepaper offers guidance to security and network architects in designing, deploying, and managing architecture to protect against increasingly sophisticated, application-layer DDoS attacks.
Valuable data can be a needle in a haystack, but by leveraging the value in existing information assets, organisations can generate real and achievable gains in revenue generation, IT investments and productivity gains. This whitepaper discusses how Information Management (IM) is a multi-faceted discipline that can be employed to meet or exceed your business objectives.
Migrating from BlackBerry? See Our Trusted Method.
Are your business leaders demanding a migration plan from BlackBerry? Let the mobile experts at Good help you migrate without migraines. Our Professional Services team has deep experience supporting Fortune 500 organizations through the transition; read this sample planning chart and see our trusted method.