Yet, no matter what the industry, creating the type of intelligent discovery environment required to generate competitive advantage from big data is often less than straightforward.
While in most instances, traditional analytics and methodologies are fairly robust, in big data projects, the introduction of new analytics technologies will inevitably create a learning curve. In many cases, this is compounded by the use of previously unfamiliar open source data storage tools such as Hadoop that historically have never existed in the organisation.
For IT, creating a discovery-led environment also tends to require a shift in mind set. Traditionally, IT culture has always been very requirements-led, making traditional BI, analytics and data warehousing a tough juggling act. Indeed, simply retrieving the data and working out what to do with it required a high degree of flexibility.
Big data discovery projects take this tough equation even further - and therefore away from the IT department's comfort zone. No longer is there a pre-defined "need". Rather, the challenge now lies in identifying "the question". And there are many more new and in-depth questions being asked than ever before.
Despite the potential challenges, the continuing hype around big data has been a major catalyst for new big data projects, securing buy-in from the organisation as well as prompting IT workers to begin their own individual or "skunkwork" experiments that are outside the usual rules of IT. Often, because these technologies are seen as being time-consuming or disruptive, they tend not to get immediately passed to the BI department, who in turn cannot inform IT of existing challenges in the traditional environment that could be solved using the new capabilities.
For this reason, even if the discovery capabilities are kept separate, from an execution perspective, it's still important to find ways to join them together. Where it exists, the most logical approach is for the BI competency centre to drive the big data exploration and execution process.
In industries where individuals and departments work to produce their own analysis of data sets, it can also be harder to "stitch together" pools of information and explore trends or changes over time. This lack of collaboration also means it takes much longer to reach a common solution and understanding of the potential opportunities. Crucially, the process of pooling together data sets can help to make it possible to realise this value.
Indeed, some of the best opportunities are extensions of existing opportunities that were previously prevented by cost and processing limitations. In retail, for instance, basket analysis is a well-recognised tool for cross-promotions and marketing. Yet very little is known about external events "outside" of the basket.
For example, there are many cases when a customer will return to a site to get an item they did not purchase originally, such as matching shoes to accompany a dress or handbag, or an HDMI cable needed for an electrical item. Using sequential affinity analysis, the retailer can capitalise on this increased intelligence to send specific and timely email marketing that's more likely to drive traffic and increase revenue.
A successful big data discovery environment can also enable retailers to understand customer behaviour better because it allows them to look for changes in an individual's basket over a period of years, rather than weeks or months. This then enables the retailer to assess the impact, for instance, that becoming a family can have on the purchasing patterns of a previously single customer.
Increasing the scope for discovery also presents countless opportunities for other industries to identify previously unrecognised insight and process this information to improve quality and efficiency, as well as drive sales.
For example, the use of path analysis can enable cable or satellite broadcasters to use set-top box data to look for common sequences of events that have led to different users having to restart their machines. When combined with other information, such as the specific model of machine, the type and version of the software running on it, as well as other variables such as existing complaints about other boxes and the region it is located, it is possible to proactively identify that a customer will either currently be encountering problems or that they may do so in the near future.
This "cross-checking" capability also helps to rule in or rule out certain variables that don't have an impact. One of the many practical benefits is that the exercise can enable the cable or satellite broadcaster to "push" software upgrades to affected customers' hardware before they have to deal with mass complaints.
Ultimately, the underlying business challenges for IT projects tend to be around increasing profitability, reducing costs, improving process efficiencies or finding previously untapped opportunities. Likewise, the objectives of discovery-led big data projects also have parallels with traditional analytics: namely, the chance to increase granularity of information and uncover important new patterns and trends.
---
Autor(en)/Author(s): Kevin Long
Quelle/Source: Kevin Long, 10.04.2013
Bitte besuchen Sie/Please visit:
