As an enthusiastic statistician, I have regularly looked forward to the adventure of exploring a new data set and looking for innovative insight. I am convinced that the experience is completely analogous to the creative process elsewhere — the sense of angst and the moments of illumination and clarity of purpose. So, I am left perplexed at the sense others clearly have — they react as if data was somewhat tyrannical and suppressive, restricting innovation and acting like a brake. I want to propose how things might change to make my own experience more prevalent.

First, I want to clarify what I mean by data in this context. What I am talking about are those fragments of digital information that arise from something happening, be it a purchase, a click, an email open, a customer service request, an unsubscribe, a failed order. This is in contrast to data specifically generated for insight alone, typically in a research project, a field where there are already lots of creative approaches. Hopefully, with that clarified, we can look at the implications for people, process and practice within the marketing data field. For each of these areas, I want to describe some of the current expectations of the role of data and how this can act to suppress creative approaches and then indicate how the role could be broadened in the future.

It is enlightening to consider in which order people, process and practice should be addressed. In the long run, the people aspects have to be resolved and normally I would begin here. However, currently the barriers to a change of approach are a result of the personality of those people who have been recruited into the current data roles. It should also be noted that these issues are not purely a symptom of the marketing sector — it is apparent that even the science community is concerned about the current availability of competent analysts.

Typically, once the right people are in place, this allows new processes to flourish that can then deliver new practices. Thereby hangs the problem. Current expectations of practice generate an expectation of particular approaches and, as a consequence, this acts as a disincentive to the recruitment of particularly creative and innovative people within the marketing data sector. As a result, I want to start by inspiring people with the possibilities of what could be achieved, so that sufficient current incumbents allow novelty to flourish, processes to begin, that attract new entrants into the sector.

First, the practices need to be extended. Currently, analytics get pigeon-holed into benefits like optimization and efficiency, focused on incremental changes and reporting of campaigns. These approaches, for example, restrict analytics into campaign blinkers, examining responses within a 3-month window, for example. I am not suggesting that these practices are not worthwhile, but merely showing that they are only part of the story. There is little time to allow analysts to become involved in searching for structural changes and new product development, identifying changes in paradigms or underlying behaviour. It is a real shame, for example, that the highly respected analytics that have been built upon the Tesco Clubcard may well still be optimizing responses from offers, but have failed to identify the structural changes within the FMCG/supermarket marketplace. Was nobody looking at these longer-term trends?

When I was beginning a transfer from physics to statistics back in the 1990s, I was inspired by George Box, an innovator of statistical methodologies, who called for more scientific thinking that seeks to understand structures and abstractions that exist behind the immediately presenting data. In contrast, mathematical and technical approaches tend to look to implement rules and processes, rather than generate meaning. Recently, he has been quoted continuing to bemoan the dearth of scientific approaches, including the opportunity to trial success and failures and provide the opportunity for analysts to challenge the status quo, disrupt and subvert current perspectives. This type of approach might truly warrant the title ‘Data Scientist’.

These practices lead to the privileging of certain processes. For example, it is common prejudice that a single customer view will solve all sorts of problems. Operationally, I am sure this is true, but from an analytical perspective this will remove from view changes in behaviour and might well remove dirty data that holds clues to behaviour change. A single version of the truth will contain within it bias and blind spots that hide behavioural changes. Following Alan Mitchell, it is important not to confuse the different uses of data, overall uses can be grouped into two distinct groups. Mitchell distinguishes between logistics and insight. Logistics covers data used to ensure that a communication is used to reliably transmit a message between the organisation and an individual in a timely manner. Insight covers data used to identify novel relationships and behaviours useful for the development of business strategy. Insight shouldn’t be built on pre-processed data — for example, it is no use analysing a database after it has been filtered through a goneaway suppression file as some of the most valuable and volatile population will have inadvertently been excluded from the analysis.

Again, while A/B testing could be seen as appropriate for developing a campaign, the fact that so much is delivered easily within, for example, Google Adwords restricts the scope for change. In many ways, it has more the characteristics of an assumptive close by Google — do you want a red or blue one? — rather than testing other channels. We focus on response analysis that, as a consequence, privileges short-term behaviour, and thus potentially fills the database with volatile people and is vulnerable to the ‘last click’ winning. We run for volume and instinctively suffer from confirmation bias, measuring the behaviours we want to see, rather than testing for the negatives and alternatives. Our preference appears to be for precise answers that might be in the wrong place, rather than accepting an approximate assessment of trends and longer-term behaviour.

We can step back and increasingly use pseudonymized data, which provides protection from data security and privacy issues, but also allows the option to see the wood rather than just the trees. At early stages in analysis, it helps to move away from the communication process. Initially, we should seek to understand patterns of behaviour reflected within the data and subsequently identify where this can relate to the delivery of communication. This allows wider scope for exploration and also focuses attention on hermeneutics — what does a piece of data and analytics mean, how has the process of data creation distorted the outcomes, where do the typical assumptions of analysis fail and what does this tell us? This can take us away from the temptation to just count a few numbers and see the largest volume as the best.

So, if we begin to see the extended expectations of what data can deliver and, as a result, allow a broader range of processes to be applied, this should begin to nurture a culture where additional characters and personalities can thrive, rather than just seeking to extend recruitment to those with mathematical degrees, IT, programming or accounting, assuming that these people are there to test hypotheses and act as gate keepers. Admittedly, we need plenty of people who will run standard processes, act in accordance with good governance and provide individual-level verification, but also data should be a place where renegades are encouraged to challenge the status quo, propose new approaches and methods, a number of which will fail. Analysts who sit alongside the creatives and copywriters, rather than the suits and accountants.

In conclusion, to answer the question posed in the title, when it comes to creativity, data is currently often a curse. It is typically the tool used to crush the latest innovation before it has had the change to find its natural territory. However, with a change in expectations, the recognition of a broader repertoire of approaches and scale, data can take its place at the heart of innovation, novelty and structural change. But this will only happen once this marketing data sector becomes a place where innovative, curious and subversive people feel naturally at home.