Victorian firefighters extinguish online congestion during bushfires
- 27 February, 2013 14:51
Firefighters at CFA Victoria deployed content delivery management (CDM) software during the recent Victorian bushfires as an influx of residents using a mobile app that provides fire warnings caused its website to fail.
CFA is a first responder and emergency management organisation funded by the state government in Victoria. It has 61,000 people, including 59,000 volunteers, operating out of 1253 fire stations around Victoria. Bushfires are just one area in CFA’s jurisdiction; it is also a first responder for road rescue, hazardous materials and other emergencies.
IT’s role at CFA is to communicate information to both internal operations people and the general public who the organisation protects, CFA executive manager, ICT services, Michael Foreshew told Computerworld Australia.
This includes informing fire fighters how best to extinguish a bushfire and sending warnings to the public about potential danger areas.
CFA used to rely on mainstream media to communicate warnings to the general public, but about three years ago began using Internet technologies to directly warn citizens abut emergencies, Foreshew said.
CFA has a system called “One Source, One Message” that allows CFA officials in the field to produce alerts that are automatically and simultaneously sent to the CFA website, Twitter, Facebook, mainstream media and the federal SMS alert system.
CFA also produces RSS and XML data feeds that can be read by mobile apps and other readers. Apps have been developed for Android, Apple, Windows Phone and BlackBerry mobile operating systems.
On 4 January with bushfires raging across Victoria, one mobile app called FireReady “bombarded our website ... with so much traffic that the website would time out waiting for the data to come back,” he said.
“We initially thought it was a denial of service attack,” Foreshew said. “However, none of the perimeter security went off.”
Eventually, CFA discovered that a poorly designed FireReady app was the source of the problems.
The app drew “a lot of data every 60 seconds,” he said. “It doesn’t just take the changes; it reloads everything.”
In addition, “rather than query our proxy servers, which had been set up to handle this ... it forced our proxies to go back and request the data” from the database.”
The inefficiencies combined with 417,000 users using that app at once crippled the CFA website.
“The website was still producing the pages,” but anybody who tried to access the website received server timeout errors, he said. “What was then happening is people grew frustrated with the website, so they downloaded the app. It was a compounding problem.”
After identifying the app was the problem, CFA engaged Internet content delivery management software vendor Akamai, a company it had already been talking to about another initiative, Foreshew said. On 9 January, Akamai presented a possible solution on and deployed it in about four hours, “free of charge,” he said.
The software quickly resolved the network congestion, and CFA this week signed a one-year contract with Akamai after navigating a government review, said Foreshew.
At the peak of the emergency, CFA served 1.9TB of data in one day, Foreshew said. CFA’s 1Gbps link was running at about 650Mbps to 670Mbps trying to serve the large volume of data, he said.
However, after about 15 minutes with the Akamai service, “we were down below 13Mbps and we haven’t gone above that ever since,” he said.
CFA had to find money and satisfy procurement guidelines before signing the contract, Foreshew said. “CFA and a number of government organisations are facing reduced budgets, so we needed to reprioritise.”
The Akamai CDM was attractively priced and “provided a large number of servers across a variety of locations,” Foreshew said. Alternative CDM providers presented a smaller number of larger capacity servers in a centralised location, he said. CFA decided that the Akamai CDM provided better redundancy.
Akamai “properly” addressed cloud provisioning, he added. “What we saw in some of the other providers were outsourcers trying to buy their way through an inflection point of technology.”
Meanwhile, CFA is redesigning the mobile app that caused the traffic congestion in the first place. It is also exploring alternative ways to deliver information to mobile devices, Foreshew said.
“We’ve seen advancements in the use of HTML5, for example,” which could allow CFA to consolidate its mobile delivery rather than have a separate app for each mobile OS, he said.
“There are no sacred cows,” he said. “We [will] have a look at what is the most appropriate, most efficient, most effective way of getting the information out to people.”
Follow Adam Bender on Twitter: @WatchAdam
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.
Yahoo Mail still down for some users, after an attempted fix
Queensland government to provide 200 services online by 2015
CIOs need to get their house in order, CFO panel says
Is Data Complexity Blinding Your IT Decision-Making?
Why IT projects really fail
Top 10 tips for Migration
As users bring multiple devices to the workplace, IT departments need to have a single view of all their mobile devices. Find out how to build a secure and reliable management platform for next generation mobile computing across multiple platforms. Click for more!
Efficient Data Management in Three Simple Steps
Gartner reports that Business Intelligence, Mobile Technologies and Cloud Computing rank 1-2-3 as the 2013 Global CIO Technology Priorities. These three trends, labelled the “Perfect Storm” of new technologies, are transforming every link in the IT value chain, promising to deliver more efficient, responsive and dynamic IT operations. But this also means massive shifts in the way IT applications and services are created, deployed and maintained. This whitepaper aims to help you begin the journey to efficient modern data management
‘A Little Extra Service’ Raises Customer Satisfaction and Lowers Costs
Companies are responding to the digital generation’s preference for online support, with new channels like Live Chat and Email Management. These mobile-friendly solutions give customers the right answers at the right time, when self-service just isn’t enough, and phone calls are undesirable. Read about these new touch points and the importance of a personalized web self-service.