Real Time BI used to be hot topic since 10 years ago. As a result of big data phenomenon, data volumes and the diversity of new data sources are exploding. Now it is the emerging Operational Intelligence supports the real time BI on big data.Although visibility into streaming big data by itself is extremely valuable to the business, its value can be further enhanced by combining it with data that already exists in structured databases and data warehouses. For example, HR churn trending data in conversional DWH can be combined with advanced analytics data might be hosted in big data store. That can be used to identify red flags on hidden risks …
The question is if you need to move structured data to close to the big data and perform the join or leave them what they are and join them across the network.
In certain cases, we do not want to move large chunk of data over the wires, so OI vendor starts to provide add-on such SQL or DB connect. But we need to keep an eye on performance and check if it has caching capability. If not, there is possibility that your queries may impact database performance.
Data Virtualization servers have started to offer access to Hadoop, and with that they have entered the market of SQL-on-Hadoop engines. Current DV servers has runtime query engines that offer data federation capability for non SQL data sources, data lineage and impact analysis features, caching capabilities to minimize access of the data source, distributed join optimization techniques and data level security. I think they will raise the bar more in this market …