Whether you're thinking of building a brand-new data warehouse infrastructure, or worrying about how you can get the most out of an existing one, these massive beasts of the data management world present organisations with significant challenges to keep them up and running at peak performance.

Speak to an expert today - contact us now - speak to Rob Bailey from Logicalis

The major challenge with new data warehouses is one of planning, says Rob Bailey, a business intelligence specialist at Logicalis. "It's difficult to work out exactly what you're going to need to build. You can, for example, calculate how many transactions your systems handle in a year and then extrapolate that out over a three-year period," he says. "But additional requirements - such as staging areas for data cleansing - are going to add significant overheads to that rough estimate."

And it's not just a matter of knowing what hardware and networking capabilities will be required to guarantee peak data warehouse performance. There's also the matter of the number of users the system will support, the frequency with which they'll run queries against the warehouse and the complexity and volume of those queries, he says.

As a result, it's all too common to end up with a data warehousing infrastructure that isn't completely balanced in terms of CPUs (central processing units), memory and data warehouse throughput.

The challenge with existing data warehouses, meanwhile, is one of fine-tuning. "You may have a lot more information about the amount of data you need to store, the workload the infrastructure handles and where bottlenecks in data warehouse throughput currently exist - but what you might not be taking into account is the times that end-users give up on running a query, or don't bother to do so at all, because they know that response times are so poor."

So how do you configure a data warehouse? Logicalis'experience is that clients need to take a balanced approach to data warehouse infrastructure, one example being where Oracle software and IBM hardware have been streamlined in an Oracle Optimised Warehouse.

The Oracle Optimised Warehouse is effectively a prepackaged configuration in which CPUs, memory and I/O (input/output) have been combined in quantities designed to support a data warehouse of around 5 terabytes (TB) in size. And if 5TB isn't enough, two or more of these ‘building blocks' can be combined to create data warehouses of up to around 20TB, using Oracle's Real Application Clusters technology.

"This takes much of the guesswork out of data warehousing and gives room for a growing data warehouse to expand," Bailey explains. And that might well be necessary, he adds. "After all, users are more likely to take advantage of data warehousing if they know that the queries they want to run will execute smoothly and efficiently."

Speak to an expert today - contact us now - speak to Rob Bailey from Logicalis

Your Comments and Questions

Rob Bailey, about 1 month ago

Yes, indeed they have, and you're right that there are probably not as many dashboards out there as one might expect. I can imagine at least three possible reasons for this. Firstly, as is often the case in all branches of IT, it is more than likely that BI vendor marketing has been running ahead of the ability of their products to deliver. I see evidence now though, from Business Objects, Oracle and others, that the products have now caught up. Secondly, dashboard products have possibly been mis-sold, in that the impression can be given that all you need to do is pull together existing data from your existing systems (ERP, CRM or whatever) and present it via a dashboard and you will magically achieve sensational business insight. This of course is not the case; for one thing it is often not easy to extract and 'federate' data in this way efficiently. And it is even harder to know that when looking at data from different sources you're comparing like with like. Are they looking at the same date ranges for example, or the same product set? People normally resolve this sort of issue by developing a data warehouse or data marts, so that such problems of inconsistency and performance can be addressed before presenting data to the dashboard, or any other BI tool for that matter. My opinion is that an interactive dashboard providing truly useful business insight must be supported by a well-implemented data warehouse. The third possible problem is reaching the right people to sell the solution to - i.e. the executives who would use it. They aren't necessarily easily convinced that BI is useful or relevant to them, and neither are their IT departments or indeed more junior management, who may even have concerns that their senior management will find out more than they would like about what's going on! Vendors know this, of course, and are eagerly trying to target those senior executives via 'Executive Briefings' and the like. Lets see how they get on!

Steven Stanford, about 1 month ago

I agree with your response Rob, but haven't the vendors been talking about dashboards that provide senior execs with KPIs for many years now? Why hasn't this taken off yet?

Rob Bailey, about 1 month ago

That certainly can be a problem, and companies adopting BI tools and building data warehouses should certainly not overlook the need for user training, no matter how easy to use the vendor says the solution will be. Its interesting also that BI tool vendors still think of users of their 'analytic' or ad-hoc query tools as 'power users'. That means people who need a good understanding of the data and how its structured, and how to use the advanced features of their tools, before they can convert the huge amounts of data stored in the data warehouse into useful information. One key user group that this very often excludes is senior executives, who have neither the time nor the inclination to learn how to use the tools, and often aren't even given access. Senior executives don't want or need 'power user' tools though - what they need is a dashboard, a single web page with the relevant KPI's, plus the ability to intuitively and dynamically drill down below the surface. This is an area that BI vendors such as Business Objects are taking increasingly seriously, and an important side-effect of a good dashboard deployment is that senior executives start to appreciate the value of BI, and become more inclinded to fund its adoption throughout the enterprise. A win-win for the vendor and the customer!

Victoria, about 1 month ago

I think business intelligence would be more valuable and relevant if it were used more widely throughout organisations. However, rather than being a problem of user experience – with it taking too long to run a query, for example – I think the issue is one of training: business users aren't trained in using the data warehouse, it's left to the IT or business analysts. As a result, I believe organisations are missing out on some significant opportunities to gain insight into their customers and business operations.

What's Your Question or Comment?

Name

Email Address

Your Website URL (please include http://)

Your Question or Comment

type the text from the image