4. Keep asking, keep changing -- and keep the data.
New practices in BI are echoing the "agile" methodologies programmers are finding effective. Complex techniques, statistical analyses, and new analytical models emerge and disappear. For example, Ryan Hawk, T-Mobile Director of Information Management, and his analytics team needed to build models of telecom usage-propensity to churn, revenue generation, and more- but were challenged because "data is a business case-we have to decide what we can afford to store on our MPP systems," Hawk says. "The hardest thing is having to purge data every 60 days-you can't do much trending."
By shifting their data warehouse into an agile, virtualized infrastructure, T-Mobile now has flexible access to more data and can analyze and rethink at will. Like Fox, they're able to build analytical "sandboxes" on-demand to discover new questions. Grab data to explore those questions. Tear it down, and do it again. Data is the other element of the "cloud;" keep it where you need it, and use as appropriate.
5. Run programs "close to the data."
Dolan's team at Fox might work with two weeks' worth of data: 100 billion lines, 10s of terabytes. Exporting, transforming, moving and distributing that data in chunks (extract, transform and load: ETL style), constrained by bandwidth and system load factors, used to take 3 to 4 days. Rebuilding all the joins, indexes and other structures within the data would consume another day or two. But with new in-database analytics technology, Fox can run programs directly in the database, eliminating the bottlenecks standing between his team and business insights. According to Dolan, "Inside our Greenplum database, setting up two weeks' worth of data takes us about 20 minutes."
Exploiting information has become an imperative for all businesses, and this continues to become even more important as data growth accelerates and new streams of precious information emerge. Supporting the teams who will provide agile response is a competitive necessity.
Merv Adrian is principal at consulting firm IT Market Strategy.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.