Relational OLAP query optimization

In the era of big data, companies are collecting and analyzing massive amount of data to help making business decisions. The focus of Business intelligence has been moved from reporting and performance monitoring to ad-hoc analysis, data exploration and knowledge self-discovery, where the user's train of thought is important. Business intelligence system must provide real time multidimensional analytic ability over big data volumes in order to meet the demands. Relational OLAP technologies provide better support for user-driven analysis over big volumes of dynamic data. In-memory OLAP technologies enable real time analytics experience. Their combination is the new trend to provide real time multidimensional analytics over big data volumes. However, Relational OLAP and in-memory OLAP have their own shortcomings and challenges. Relational OLAP could cause non-optimal relational database access. It often has intensive I/O and CPU demands. The biggest challenge of In-memory OLAP is combinatorial explosion. Transferring huge amount of data into multi-dimensional cache (cube) not only very time consuming but also takes considerable amount of resources. Increasing hardware resources, employing distributed in-memory data store, or re-designing MDX engine used by ROLAP to adopt hard to implement algorisms, e.g. parallel computation, are typically ways to overcome the above challenges. Instead of those costly approaches, this paper discusses several techniques that provide a way to improve performance and scalability without piling up hardware resources or going through major re-architecture. These techniques have been implemented in IBM Cognos Business Analytics (BA) solution and have been bringing success to customers since then.