After geologists assess the information, it's sliced and diced against financial realities. "The amount of money we spend is very high-$100 million for a well alone," de Souza says. "We want to get it right."

To reach its goal of becoming one of the five biggest oil companies in the world by 2020, Petrobras has to take some calculated risks. Recovering oil from this find will be expensive partly because it's so far down in the earth. "No company has tried to explore under it," he says. But promising data has triggered major staffing decisions: Petrobras has created a new group of senior managers to oversee exploration of this area and plans to hire 14,000 drillers, geologists and engineers. It takes years to go from initial exploration to crude oil production and sales of finished gasoline, so companies have to model markets five, 10, 15 years out. They use a mix of their own intelligence and public data, such as from the Energy Information Administration (EIA), says researcher Knapp.

For example, automakers continue to improve the fuel efficiency of their cars and light trucks, as well as to build electric-gas hybrids. By 2030, the average light-duty vehicle will get 27.9 miles per gallon, 40 per cent more than in 2006, according to the EIA. A highly simplified analysis suggests that if people use less gasoline, gas prices should drop, which makes expensive drilling less profitable, Knapp explains.

Although demand for gas is growing in China and India, so far it's not enough to offset the expected drop in US demand. New well and rig technologies could take some of the cost out of drilling, but no one knows exactly when or by how much. There is no shortage of data points; the value is in interpretation. "It's about filtering rather than finding a piece of information," he says. "Understanding what this whole pile of stuff can do for you is the key."

Adjusting to Change in Real Time

Every Wednesday morning, the shouts and hand gestures that make the Nymex trading floor in New York frantic begin to calm. Petroleum traders are waiting for the release of data from the US Energy Information Administration (EIA) on countries' inventories of crude oil and gasoline, as well as world crude prices.

At 10:30 a.m., the EIA's website sees a storm of activity: 1,000 page views per second for 15 seconds, says Charlie Riner, a lead analyst for the site. Oil companies, commodities traders, analyst firms, and government agencies in the United States and other countries have written bots to collect the data. Then traffic ebbs.

Inventories are the most closely watched data in the industry, says Joanne Shore, a senior petroleum analyst at the EIA, the statistics keeper for the US Department of Energy. "This is what moves markets when it comes out," Shore says. If, for example, US supplies fall sharply from the week before, that can mean demand is rising and prices likely will, too. It's not only traders who want this data. Corporations such as Valero fold it into its analysis of inventories and market activity so that they always know their standing compared to rivals.

In the oil and gas business, you are what you own. The amount of crude waiting to be refined, or the already-processed liquid in storage tanks ready to be sold and delivered, represents much of a company's value at a given moment. As a refiner, Valero buys barrels of oil to heat and pressurize into other products, such as diesel fuel, asphalt and lubricants. The US$95 billion downstream company owns 17 refineries that together can produce 3.1 million barrels of product per day.