Critical.
Authoritative.
Strategic.
Subscribe to CIO Magazine »

NLA web services traffic rises above 2.5 billion in 2009

The National Library of Australia (NLA) has more than doubled its web services traffic to reach more than 2.5 billion hits in 2009.
National Library of Australia assistant director-general of information technology, Mark Corbould

National Library of Australia assistant director-general of information technology, Mark Corbould

The National Library of Australia (NLA) has more than doubled its web services traffic to reach more than 2.5 billion hits in 2009.

Speaking at the CIO Summit 2010 in Sydney, NLA assistant director of general information technology, Mark Corbould, attributed a lot of the organisation’s success to the use of open source solutions.

In 2006 across all of its online assets including the public catalogue and the Trove search engine, the NLA had half a billion web server requests. This jumped in 2008 to over one billion and last year hit more than 2.5 billion.

“We are now probably the highest ranked cultural institution in Australia in terms of our web presence,” Corbould told the summit attendees.

In April the NLA unveiled its Trove search engine that was built on an open source platform.

The search engine provides access to more than 90 million items about Australians and Australia, sourced from more than 1000 libraries and cultural institutions across the country.

Courbold said for an organisation that costs roughly $70 million a year to run with 450 staff and 10 per cent of its budget going towards IT, there was little chance of securing a commercial software licence to do what the NLA does.

The Trove project’s team of five developers used SOLR 1.4, which internally uses Lucene 2.9, for the main bibliographic search database and the web page archive, and MySQL 5 for managing all data relationships.

“The overpowering characteristic was that it was the only one we could afford,” Courbold said.

The project team also opted for Jetty as a web server, Nginx as the HTTP front-end/reverse proxy, Java Server Pages (JSP) for the newspapers part of the site, and Restlet and FreeMarker for the remaining portions of the service.

Additionally, one of the main steps taken was to use Solid State Disks (SSDs) – four Intel X-M25 160GB drives in each machine – for the Lucene indices to achieve the necessary performance. Trove issues more than 8000 IOPS (input/output operations per second) to the SSDs, which the team says would be expensive to achieve with even the fastest SAN setup.

“A lot of what we do is not well supported by the mainstream market,” Courbold told the CIO summit.

“When you want to harvest the web or when you want to integrate access to varieties of collections we have the market place doesn’t service us well.”

(See the CIO Summit 2010 in pictures)

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

More about: Intel, MySQL, National Library of Australia
References show all

Comments

Post new comment

The content of this field is kept private and will not be shown publicly.
Users posting comments agree to the CIO comments policy.
Login or register to link comments to your user profile, or you may also post a comment without being logged in.
Related Coverage
Related Whitepapers
Latest Stories
Community Comments
Tags: Trove, open source, National Library of Australia, CIO Summit 2010
Latest Blog Posts
Whitepapers
  • The eGuide to Data Movement and Governance: Helping Business Professionals Stay Up to Speed
    You fail an audit. Or customer information is compromised. Or you are called on the carpet for failing to meet a critical customer SLA. At that point you realise just how important it is to your organisation and to your career. How do you prepare for that moment? More importantly, how do you prevent it from happening in the first place? It is absolutely critical that you understand the possible consequences of a failure to properly monitor, control, and protect the movement of data. Missed opportunities and lost revenue might be the least of your worries. In some cases, poor practices can lead to lawsuits, fines, and even the failure of the business itself. The purpose of this eGuide is to help you grasp the measures that can keep your organisation on track to meet objectives and in line with regulations.
    Learn more »
  • 8 reasons why Citrix NetScaler beats the competition
    Application delivery controllers (ADC) are one of the most critical elements of cloud infrastructures and enterprise data centre architectures. ADCs strongly impact performance, scale and security of the entire application environment, so it is extremely important for IT leaders to choose the right one.
    Learn more »
  • BI Optimisation: Building a Better Business Case for Business Intelligence
    "Business Intelligence helps you make better decisions." This is how the business value of Business Intelligence (BI) is often described. How do you measure the effect of better decisions? How do you determine the quality of a decision to start with? And how can you predict how other decisions might have played out? Read on.
    Learn more »
All whitepapers
rhs_login_lockGet exclusive access to Invitation only events CIO, reports & analysis.

HP and IDG news, product videos and resources