Tuesday, July 16, 2013

Near Real time Data/Analytics...

In the recent years with the advent of newer technologies and  lot of application integration efforts, there has been steady increase in demand for near real time data for performing reporting and analytics. In the past years there was lot of operational reporting related efforts. Now a trend is emerging where the Business want to capitalize on the availability of close to real time information to perform analytics. One of the areas where lot of improvements are made is in the area of CDC, change data capture. The feature introduced in SQL Server is being taken advantage of in lot of the projects I am working on. Initially there were other interface systems to provided change data information, now we are able to streamline all the requirements via SQL Server CDC and get change data information. I see the use of sophisticated replication and mirroring technologies being used to reduce the time gap for data availability. This has bought in the need for performing very good architecture to enable near real time analytics. I see a lot of competition between SQL Server and other Database systems to come up with tools/ technology to address these needs. The concept of datawarehouse would probably be modified to handle the near real time requirements along with Unstructured data information. Microsoft is working with Hadoop based vendors to provide integration with NoSQL Data. Please refer to the below link for Microsoft Hadoop Distribution:

To quote Microsoft:
HDInsight is Microsoft’s 100% Apache compatible Hadoop distribution, supported by Microsoft. HDInsight, available both on Windows Server or as an Windows Azure service, empowers organizations with new insights on previously untouched unstructured data, while connecting to the most widely used Business Intelligence (BI) tools on the planet.

HDInsights enables you to realize new value in the data you have and can combine these new insights with 3rd party datasets simply and elegantly. The time spent by your data analysts trying to surface the right data and source for your precise needs is costly. By connecting to external data sources you can begin to answer new types of questions and deliver new value in ways that previously were not possible.

No comments:

Post a Comment