As Charles McLellan writes, the data collection is all pervasive and pervades enterprises of all sizes and definitions and the most important question today is about the how little control an individual as on his personal information, meanwhile astute enterprises with their Big data driven initiatives using large amounts of user data is targeting very specific groups with their services or in case of non-commercial enterprises seeking support to a particular goal or cause.

As he points out further in this article, enterprises to gain business insights from this deluge of information should have in place right planning, budget, tools, and expertise. This will enable them to analyze and get actionable data and help enterprises to implement revenue –generating innovations in a pertinent segment of the business to gain competitive advantage.

Charles McLellan quotes reports from IDC and EMC to project the total data generated globally by 2020 to around 44ZB (zettabytes), and another by Seagate’s Data Age 2025 to around 163ZB, and for enterprises with vast amounts of data coming in it will be easy to get overwhelmed with the possibilities. And with data coming in different types like structured, unstructured and real time, it should be understood that all data is not conducive to analysis. And according to IDC by 2025, 20% of the data will be critical (Data necessary for user daily life) and around 15 % will be hyper critical (Data with direct and immediate impact on user’s life).

With Artificial Intelligence and Machine Learning being the foremost technologies deployed to understand Big Data, the data available for use will be further restricted. In fact, according to IDC by 2025, only 15% will be tagged and so useful for AI /ML analysis and further in this only 3% will be suitable to be analyzed by cognitive systems.

The coming trends for the Big Data industry according to this ZDNet special report by Charles McLellan is  “ AI, Machine Learning, Automation and Cognitive systems”, along with “ data driven business applications “

For detailed picture on Big Data visit: http://www.zdnet.com/article/turning-big-data-into-business-insights-the-state-of-play/

Joe McKendrick begins by telling us about the rising expectations of users either employee or customers when visiting a website. With fast galloping technology coupled with increasing bandwidth speeds, the users of connected networks and devices expect information in nanoseconds.

With intense competition driving the enterprises to focus on user experience to increase business revenues, and for this enterprises encourage data and development teams to delivers applications with recommendation engines based on customer insights, which in turn requires not only data but fast and intelligent data in real time or near real time.

With the emergence of in–memory databases, machine learning … and other Gen Next technologies, a typical data eco-system which previously was tasked to deliver static reports based on historical data is today designed to be a  real-time or near time application to help deliver operational intelligence to decision makers on an instantaneous  basis.

Enterprises accustomed to batch mode of data up-gradation usually on 24-hour cycle recognize the benefits accruing to their organizations by keeping the data refreshed on a constant basis. And hence task their data managers to build, maintain and support a fast or streaming data eco-system to support highly interactive and intelligent applications.

Lastly, Joe McKendrick suggested few key elements necessary for fast and intelligent data eco- system.

•  Mind data storage 
• Alternate databases
• Employ analytics close to data
• In–Memory Options
• Machine Learning and another real time approach
• Cloud
• Enhance skill base
• Look at Life–cycle data management.

For more, please read: http://www.dbta.com/BigDataQuarterly/Articles/8-Rules-of-the-Road-for-Fast-Data-Management-and-Analytics-119939.aspx

And according to Stephen DiFranco, this acquisition is all about disrupting the retail with Amazon bringing the on-line analytics to off-line shopping.

With Amazon getting a patent for “Physical store on-line traffic control “ just days before acquiring Whole Foods, Stephen DiFranco is sure that Big Data will be a big presence in the off-line retailing from now on. With store visitors encouraged to get connected to the store Wi- Fi, they will then by tracked to know about their typical route thru’ the store, with pertinent suggestions sent to customers to prompt them to buy complementary products. Another scenario may be reminding the customers about frequently brought items and the directing them to that location.

With other biggies in the retail sectors moving for consolidation, and retail margins being very thin, Amazon buying Whole Foods is a great advantage since food retail is best for collecting brick and mortar customer data thereby allowing them to leverage data to push volumes.

With food retailer being the most frequently visited stores, as shopping for food is done twice a week. And with customers looking for convenience, they will be purchasing the same items again and again, week after week.  And this behavior gives out preferences across product lines and lends itself to behavior analysis giving out clear buying patterns across products and categories.

Amazon using a big data application and massive amounts of data pertaining to a number of visits, product category, minimum purchased, can lead the way in enhancing the customer experience by helping in decision making by delivering notifications, discount offers, and other information at an appropriate time.

To read more, visit: https://www.entrepreneur.com/article/296075

As Brett MacLaren puts it, this journey from analog processes and workflows to digital domain starts quite simply with enterprises using the data created in all its attributes to derive business advantage leading to data- driven decision making and this adaption is successful in a few limited and much published cases where the companies are digital from the start helped by massive investments in enabling technologies and human resources adept in leveraging them.

The challenge for most other enterprises is not only to invest in tools and hardware which vendors claim to be proficient in  ROI but to get people who can leverage these tools and deliver value, and most critically this should happen when the enterprise is focused  the meeting the existing business demands. The compulsion to juggle these two crucial activities becomes too demanding for many of the enterprises. And to realize the promise of Big Data enterprises should work on solutions involving several critical components some technological along with one critical aspect – corporate culture.

When it comes to addressing the role of corporate culture in harnessing the benefits of Big Data, Brett MacLaren suggests creation of a high profile leadership position to lead the enterprises thrust to data reorganization. The mandate for this leadership which many enterprises designate as Chief data Officer is to drive home the business value of data and be the pivotal point between business and IT and helping both to recognize the strategic role of data.

Enterprises hoping to realize the benefits of Big Data must spread the culture of data awareness and data competency across enterprise. Big Data and other related tools and technologies are clearly cutting edge and are kind of starting points for adaption of machine learning, deep learning and other parallel processes that can deliver tremendous value to enterprises by bringing the data to the point of decision making.

For more, please visit: http://bigdata.cioreview.com/cxoinsight/big-data-separating-the-hype-from-reality-in-corporate-culture-nid-24174-cid-15.html

AI advancement – Big Data is advancing the speed and capacity of Artificial Intelligence and taking it to the next level, for example  Google DeepMind AI beating humans in the game of Go, and becomes unbeatable as the game progresses due to AI and the Big Data applied to its functioning.

Tax Shelters Unveiled – Investigative journalists collaborating across continents and using cloud based data analytics and Big Data were able to pursue effectively and unveil tax shelters now famously known as “Panama papers”.  This is one of the first known instance of the real-world good, Big Data can contribute to bring about.

Human Trafficking – Big Data is lending its helping hand to “Polaris Project” in its fight against human trafficking, even though Polaris project has made tremendous progress over the years  in their fight, Big Data became their strongest tool in the year 2016 to decipher the complex numbers and patterns to give useful insights and help victims of this  horrific crime.

Cancer Research – Intel’s Bryce Olson , himself a cancer survivor lead, Trusted Analytics platform is “ a collection of Big Data tools and data analytics, to help in breaking down of DNA – the complex code of human genetics to give insights into where cancer begins and how it can be controlled”.

HIV Outbreaks – When Centre of Disease Control were struggling to contain the HIV outbreak in 2016, which was taking extensive death toll and seriously damaging the health of survivors, they turned to Big Data for insights to fight the outbreak.

In 2016 we were witness to the fact that Big data in addition to its applications in optimizing business efficiency and effectiveness can also be deployed to harness real- world good, and we are sure to see more such deployments in the coming years.

For more on this, please read: http://insidebigdata.com/2017/02/26/2016-big-year-big-data/

According to Eric Mizell the steady advances in CPU, storage and memory, networking technologies laid foundation for economical cognitive computing. This was pushed further in terms of price and performance in data analytics by the appearance of solid state storage and Random Access Memory (RAM). These advances have managed to shift the performance bottleneck from input/output devices to processing and laid stress on greater processing rate. But this makes cognitive computing and other mature analytical applications the preserve of a few very large organizations with multi-core CPUs deployed in several clusters of servers.

And as an answer to this need for affordable processing power came Graphical Processing Units (GPU)s with its capability for parallel processing bestowing ability to process data 100 times faster than configurations containing cores. GPUs were initially designed for graphics and installed on a separate card with its own memory (Video RAM) and this configuration was popular with gamers who were looking for real-time graphics. And as the processing power and programmability of the GPUs increased over time, it came to be used in additional applications.

We do know the real benefit from cognitive computing can be derived when it is real-time, and this can be achieved economically when we use GPU acceleration.  With variety of analytical processes like artificial intelligence, business intelligence, machine learning, natural language processing, Cognitive computing is best suited for acceleration using GPUs. The cognitive computing workloads with its repeated and similar instructions are well suited for parallel processing by the thousands of small efficient cores of GPUs.

Presently GPU acceleration is being deployed by Amazon and Nimbix, Google is readying to equip its cloud platform with GPUs for Google Compute Engine and Google Cloud Machine Learning Services.

For more, please visit https://www.rtinsights.com/gpus-the-key-to-cognitive-computing/

The authors quotes a report from Accenture which compares the revolution wrought by Big Data to business process to the way internet revolutionized lives in the 1990s. They quote from this same report to say 83% of the enterprises envisage using big data to gain business advantage.

Writing next on how insurance industry especially the claims department is in the forefront of development and deployment of data analytics to increase process efficiency, fraud detection and in claim assortment process. The major challenge for deploying this data analytical process is the prohibitive costs since only about 15% of the data may be converted into actionable information. And the other 85% data which is available in unstructured form is difficult to decipher and extract any useful business information. And this percentage jumps even higher in the claims department with witness reports, claimant statements and other notes being verbose and unstructured in nature.

The insurance industry and the data engineers are looking to harness Text mining or Text analytics to extract valuable data from this 85% of unstructured data. As we know text mining in its basic level is scanning for keywords or phrases and looking for relationships within data but with advances in natural language processing (NLP) and decision logic, it now has the capability for sentiment analysis thru’ which enterprises can discover customer’s opinion about a particular product. In its application for the insurance industry, text mining algorithm can scan the claimant’s social media content in real time to validate information provided in the claims.

Text mining also helps in streamlining the claims processing, thereby increasing process efficiency, and it can also be useful in the development of new products and overcome lacunae in the present offerings.

Another major application of text mining in the insurance industry is in the identification of fraud potential, as majority of frauds are known to happen during claims, Text mining   for example may red flag an instance where several claimants are using the exact same words or sentences in their claims, a sure sign of suspicious activity.

In this way and in many other ways insurers can harness text mining and other data analysis tools to increase process efficiency and bring innovation in terms of product offering or market reach.

For more on this, visit: http://www.claimsjournal.com/news/national/2016/12/05/275316.htm

I’ve tried to highlight some of the findings here,

• Industries that are in the forefront of creating BigData apps are Software & computing (18%), Finance (11.6%), Manufacturing (10.9%) and Retail (9.8%). One industry which is notable for not being there, Heatlhcare (4.6%).
• Areas which are attracting most attention in Big Data, capturing higher volume of data than traditional databases (22.6%), analyzing unstructured data (21%), visualizing data (20.7%)
• Top three reasons for movement to Big Data from traditional database, size in data (40.8%), complex, unstructured data (38.1%) and need for real-time data analysis (17.7%).
• A surprising fact of the survey, 30.1% of developers involved in Big Data development are from companies with 100 or less employees.
• Industries being targeted for Big Data development, Software & Computing industry (17.5%), Manufacturing (15.8%) and financial industry (14%).
• Data sets Big Data developers are most commonly working on, sales and customer data (9.6%), IT-based data analysis (9.4%), informatics (8.7%) and financial transactions (8.4%).
• Departments that most commonly use Big Data solutions in a company, Marketing (14.4%), IT (13.3%), R&D (13%), followed by Sales (12.6%).
• Biggest problem related to Big Data developers are working on, Quality of data (19.2%), relevance of data acquired (13.5%), volume of data processed (12.6%).

The above are just a few I found interesting, for complete article click here.