The Journal of Database Marketing is not what it was. Quite literally.

Over the years it has changed and evolved. It has shifted focus in an effort to ensure that it both remains true to its original scope and also takes account of new developments in the field of database marketing. Not an easy job: from issue to issue, year to year, the changes are slight, almost imperceptible. Yet over the years they will add up, so that the journal you read today is, most likely, a very different beast from the one you read a decade ago.

I was reminded of that quite forcibly by two events in the last month. First was a response from one of our reviewers after they had looked over one of our academic papers. They had comments to make: areas where they felt it could be tightened; but overall, it was a good paper and worthy of publication. Their issue, however, was that they felt the paper ‘fell outside the scope of the journal as they understood it’.

I was puzzled. The paper dealt with the tailoring of product for customers. This was, it felt, pretty much wholly within the scope of the journal, which, as well as dealing with the fairly focused issues associated with Database Marketing, was also about the management of customer strategy. The latter is a wide area, deliberately so – and not quite the same as customer relationship management, which was one option looked at around the time it was decided to expand the journal's scope.

Why? More of that later. However, I still had to deal with my reviewer, as clearly they had a quite different take on the journal from what I now had. Then it dawned: it was several years since the individual in question had reviewed for us. Some reviewers are involved on a regular basis. Others are contacted only occasionally, perhaps when a specific topic calls for more specialist input than usual.

However, apart from a few generic papers, this reviewer had not been widely exposed to the journal input for a while. They still, as our conversation quickly revealed, saw the Journal of Database Marketing as almost exclusively about that topic – and very little about wider customer issues.

Meanwhile, I was being asked to referee some papers for another journal, one that, I realise now, I had not been closely associated with for almost a decade. I found myself reacting in very similar ways. I had, in my own mind, a very clear idea of what the journal was about and what it was meant to be about, and the papers I was reading, while perfectly competent and of interest, were nonetheless at odds with what I remembered that journal to be about.

What caught my attention, however, was that in both cases the shift in emphasis had been in a similar direction, away from hard data and analysis and back towards a softer construct of the customer universe. As someone brought up in years when direct marketing – and later, database marketing – was making its impact on the marketing community, I cannot say that I am wholly comfortable with this result.

For those who have come to marketing late (by which I mean since the 1980s), the difficulty for those of us who were there then is the shift away from an empirical view of the world that has taken place since. In the late 1970s and early 1980s, a proliferation of data and computing power meant that marketing started to be directed and targeted as never before.

That in turn gave rise to an increasing weight being placed on analysis and analytical techniques – specifically, the entire canon of mathematical statistics – starting with simple a/b split tests and, by the end of the 1980s, giving rise to the creation of some quite complex customer models based on multivariate analytical techniques. Cluster analysis and CHAID were much talked about and widely regarded as the source of new truths about the customer base.

There seemed to be a much greater call for statistical analysis: behind that, however, lay a much bigger claim altogether. That was a claim to empiricism: the idea that instead of using research and analysis to probe various aspects of customer attitudes and behaviour – and then fitting one's marketing approximately to the results of the same – one could start to move forward to a world in which the very behaviour of the customer base became the sole (or main) determinant of what one did next.

Market. Collect response. Analyse response data. Re-market. With each turn round this virtuous circle, so it was presumed, the targeting would improve and, more to the point, decisions would not be informed by the analysis: they would be dictated directly by it. This was a significant departure from the old ways, and for a while it did look as though it was going to win out in many traditional marketing departments.

My own first inkling that this was not to be so came when I had created – and was running – a major customer database for one of the UK's largest financial institutions. The data were not perfect: but it was a serious advance on what went before. More importantly, it was starting to give us a picture of who was responding when, how and for how much. As the analysis function began to gain in confidence, so we developed an analytical pack designed to provide senior managers with real insight into what the customers were doing.

We also started to know what promotions would work, what would not – and to be able to evaluate the customer base in intimate detail. We assumed that senior managers bought into what we were doing: and sadly, we were wrong.

First off, the regular reporting gradually mutated into ritual. Those receiving our reports were not doing anything useful with them – even when we pointed out as best we could the opportunities that lay within the database. Increasingly, discussion and argument focused not on insights, but on the format and layout of the reports.

Second, it became clear that far from buying into our empirical view of the world, senior managers saw database analysis as little more than an excuse to do research on the cheap. The crunch came with one particular TV campaign. The data we were seeing on our database spoke eloquently to us. It told us that the campaign was a failure: an expensive flop.

That, however, was not how the Marketing Director saw it. Look at the TV campaign, we were instructed: produce a report to show how it had worked. Our first attempt, combining tact with honesty, was not a success, as however we dressed it up it could not hide the fundamental truth that the campaign in question was providing leads (and sales) at a large multiple of other media. We were told we had ‘got it wrong’: we were sent back to try again; over a succession of trials and errors, we eventually came up with a report that did not exactly prove the efficacy of the campaign – but reported prettily on it – and which did not highlight the more inconvenient outcomes (like the major loss it had made).

Since then, I have encountered analysis after analysis going the same way: in some corners of the marketing world, those places such as mail order operations and direct sell financial services that die or live according to their cost per response, the old discipline still reigns. Elsewhere, database and data and the analysis thereof has mostly fallen back into its old ways.

The entire exercise is there to illustrate and to prettify the subjective conclusions of those who prefer to live according to their creative impulse. None of the work is entirely bad; even if customer response modelling has taken a back seat of late, financial modelling remains strong.

Therefore, to the extent that the marketing world is interested nowadays not in pure database marketing, but in a bowdlerised version of the same: in creating and managing customer groups, but doing so in a way that is empathic and instinct driven, rather than pulled forward by the data; to that extent it is right that the journal should adapt. Database marketers of old have become the customer relationship and customer strategy managers of today. It is entirely a softer discipline; however, it is not without its merits, and it feels appropriate to help marketing managers do what they are inclined to do anyway, well, as opposed to evangelising them about methods they are not interested in.

So much for why the journal has shifted its grounds: a lot of that change took place almost a decade ago now, when a meeting of various members of the Executive Board – industry practitioners and leading academics sat down and took a long view as to the direction that marketing was likely to take. I remain impressed by the fact that collectively they appear to have got it about right.

The second question – the one that still, at times, frustrates – is why the marketing world cannot follow the original direction set by direct marketing. In part, it seems that in a world split between creative and analytical tendencies, it is the lot of the analyst always to come in second. Without returning to the very serious questions raised by Stone and Starkey in the last issue about academic propriety and rigour in response to the modern market, this does still echo thoughts provoked by that piece.

Analysis is difficult. Allowing oneself to be led by analysis is even harder, whether one truly understands what the analysis is telling you or even worse if one does not. On the one hand, it means abdicating responsibility for some parts of the marketing strategy conversation and for many that feels like too much of a loss. Faced with a choice between improved results and retaining control over their domain, the average marketing manager, it seems, prefers to opt for control, however unscientific.

On the other hand, it means buying into a world view that is mechanistic and deterministic: a view in which customers are little more than cogs in a machine and all you need to do is pull the right lever. For many, that is unacceptable as a view.

Somewhere in there is the simple fact that many business people simply cannot cope with anything other than simple presentation of figures. In respect of a marketing competition that I am regularly invited to mark, I am constantly astonished by the low level of numeracy displayed by individuals who otherwise appear to be highly gifted verbally and in business terms.

Is that universally true? I am well aware that I write from the perspective of the UK, which has a reputation for a lack of mathematical skill within its education system. In the past, going no further than across the channel into France I was able to recruit statistical analysts of a calibre far in advance of any candidates I had chanced upon in the UK. Equally – though I am nervous of playing to national stereotypes – I have been seriously impressed by the work of analysts who have come to the UK after being educated in the Far East: in India, Singapore and China.

Clearly, this flight from analysis and insistence on treating customer strategy as a soft science is not global: perhaps, therefore, as the journal increasingly reaches out across the world, it may in time find itself starting to return more closely to its empirical roots.

Of course, there are other difficulties, other reasons why the straightforward promotion–response view of the world cannot hold sway as it used to. These have to do with the fact that both media and customer bases have been rapidly fragmenting. First it was products, proliferating from a few dozen per market sector in the sixties to hundreds in the 1980s and 1990s.

Then it was consumers, who more and more are recognised as acting individually – as ‘segments of one’ – making their categorisation difficult and trend analysis next to impossible. Last but by no means least has been the fragmentation of media: the new electronic channels have added new routes to contact; however, within each channel, individuation has been the story.

On television and radio, the simple UK view of two terrestrial TV channels and four or five radio stations in the late 1960s has exploded to hundreds of both, all available pretty much anywhere in the UK. This is even before taking account of the Internet and the fact that in theory that makes every channel in the world that chooses to put its product onto the web is also now accessible. No longer hundreds of channels, but potentially thousands.

Then, within the new media, there is the ‘blogosphere’ and social networking: a world in which individuals appear to be consuming information as never before; yet almost every single individual's consumption profile is different from every other. In such a world, some degree of empirical marketing is possible – but nowhere near to the extent that once was thought.

Recently, such change has given rise to papers looking at the wider social network, asking what effect that has on purchasing decisions and questioning how business can measure the impact of those who are non-customers, yet still have a major influence on others’ purchasing decisions.

So no: the journal is not what it was. It has adapted to changing conditions and to changing demands by its customers – the academics and business leaders who read it regularly.

For their part, change is the result of two quite different drivers: first a lack of ability when it comes to dealing with figures, which in turn leads to a rejection of marketing that is too mechanistic; and second, the most enormous amount of change in the world at large.

Putting that together, it is clear that the rigorous view, formed when this journal was first established, of marketing as capable of coming close to science is no longer tenable. In its place, we are back to the view of marketing as insightful, strategic, creative, relying on database and the information coming from database to illuminate the darker recesses of the customer psyche.

In one sense, that is a falling back from the high aspirations of the 1980s; in another, perhaps, it was always going to happen. Because in the end, that was the practical, pragmatic thing to do, which is also, it might be argued, very much in the spirit of the empirical approach.

Not quite as scientific, but much more realistic.