Personalized subject lines in email marketing

In the academic literature, there is a growing stream of publications that address how companies can use insights from behavioral economics in marketing. However, often the question remains if these results can be reproduced in another experimental setting. In this article, we use the work from (Sahni et al., Marketing Science 37:236–258, 2018) to test this thesis. In their paper, (Sahni et al., Marketing Science 37:236–258, 2018) show that adding the first name of the recipient to the subject line of an email increases the performance of email campaigns. But in the last years, this marketing trick has become a common practice in email marketing, and therefore, it might be that the behavior of recipients changed as they got used to it. In our empirical research, we find no indication that using the first name of the recipient in the subject line has a positive effect on opening rates or click-through rates; as such we are not able to reproduce the findings from (Sahni et al., Marketing Science 37:236–258, 2018). However, we do find that using the surname of the recipient has a positive effect on opening rates and click-through rates; an indication that similar marketing techniques still have potential, but that they need to be more creative and surprising.


Introduction
In recent years, it has become a common practice of marketing professionals to use insights from behavioral economics to improve the performance of marketing campaigns.Especially in digital environments, consumers have to make choices all the time, and marketers, building on insights from behavioral science, are trying to understand how they can influence them.In contrast to other fields of business research, marketing has always been closely intertwined with psychology, as marketing scholars understood early that most decision-making processes are not rational and social components play an important role in the outcome.However, in the last 20 years, these ideas have become more mainstream in economics, and also other fields of business research started to develop a profound interest in human psychology.This scientific boom is closely aligned with the publication of two books: Thinking, Fast and Slow by Daniel Kahneman and Nudge: Improving Decisions about Health, Wealth and Happiness by Richard Thaler and Cass Sunstein.
In essence, these books brought behavioral economics to a wider public and sparked the interest from academics, governments, and businesses around the world.In the academic literature, this led to a growing stream of publications on the topic, but together with the boom also came the first critical voices.As Hummel and Maedche (2019) write in their excellent overview article on the effectiveness of nudging, publication bias, or the tendency from scientists and academic journals not to publish insignificant findings, lets us most probably overestimate how effective nudging really is (see also Mertens et al., 2022).In addition, and equally important, as Caraban et al., (2019, p.10) note, "we have a very limited understanding of the long-term effects of nudging in a technological context." For marketing professionals, these observations are essentials because many use insights from behavioral economics to improve the success of their marketing campaigns, but it could be that these effects fade-out over time and/or do not function as expected with the target audience.An often-referred paper in the field of behavioral economic and marketing is the work from Sahni et al. (2018).In their article, Sahni et al. (2018) show that simply adding the name of the recipient to the subject line of an email significantly increases the performance of email campaigns.In the last years, this "trick" has become a common practice from marketers to catch the attention of their audience, hoping to increase the opening rates and click-through rates of their email campaigns.However, interestingly, the data Sahni et al. (2018) use in their article date back to 2014-almost a decade ago-and a time in which email marketing, and broader digital marketing, underwent important technological and social changes.
Although the work from Sahni et al. (2018) has been cited over 200 times according to Google Scholar, we are-to the best of our knowledge-the first academic study that reproduce their experimental setting, by including the first name of the recipient in the subject line of an email.In essence, the article from Sahni et al. (2018) fits in a tradition of papers addressing the role of personalized content in digital marketing-as such their work is often cited by other scholars addressing this topic.The article which comes the closest to the research design of Sahni et al. (2018) is Hartemo (2022), but in this article, the subject lines of email campaigns are personalized based on the interest of the recipients in specific degree programs.Therefore, in this paper, it is our objective to see if the findings from Sahni et al. (2018) still hold in today's world-or if the behavior of the audience changed as they got used to this email marketing trick.

Literature
In their research, Sahni et al. (2018) found that simply adding the first name from the recipient to the subject line of an email increases opening rates with 20%.In essence, their findings show the strength of personalization in digital marketing, adding a name to the subject line does not create any additional costs for companies, but they could benefit strongly from higher opening rates, especially when these translate into higher sales numbers.Interestingly, in their experiment, the number of sales leads even grow with 31% in the group that received a personalized email.The findings from Munz et al. (2020) show even more promising applications of personalized emails.In their research, they worked together with a charity platform and used name similarities to increase donations.More specifically, they matched the surname of a potential donor, in this case, the recipient of the email, with the surname of the beneficiary.The email campaign was designed to raise money for classroom projects in the USA, and the beneficiaries were teachers.Put into practice: when you are called "Smith" you would receive an email about a teacher called "Smith."Here, it is important to highlight that Munz et al. (2020) only used real names, and no fake identities were created to match with the donors.
Like Sahni et al. (2018), they find that just adding a matching name to the subject line strongly increases opening rates-with around 27%.But the impact on the donations is more staggering.In the group that received a personalized email, with matching surnames, the average donation was double as high, as in the group that received a standard email.In general, both Sahni et al. (2018) and Munz et al. (2020) show the power of personalized emails-with a minimum effort you can strongly increase the performance of an email campaign.As such, it is not surprising that marketing professionals around the world started to apply similar methods in their email marketing campaigns.Sahni et al., (2018, p.237) even wrote that "the findings motivated our partner to alter its default email strategy to include the recipient's name in the subject line of the email."

Research design
In this project, it is our objective to evaluate if this marketing trick-simply adding the name of the recipient to the subject line of an email-still works in today's world.To answer this question, we did two randomized field experiments with different target audiences.In the first experiment, we worked together with an alumni club and invited university students to join an exclusive career platform.The alumni club offers career advice to university students, and email marketing is an important tool for them to keep students up-to-date about upcoming events and the services they offer.In the experiment, the university students were randomly assigned to two groups: the control group received an email without a personalized subject line, and the treatment group received an email where the first name of the recipient was included in the subject line.The subject line in the treatment group would read, for John Smith: "John, a successful start for your career".Following Sahni et al. (2018), the first name of the recipient was added at the beginning of the subject line.The content of the email and the other design elements were exactly the same.
In the second experiment, we worked together with a start-up company and invited local and regional politicians to participate in a survey about social media.The start-up company contacted these politicians, as they develop digital tools to manage social media accounts, and they wanted to understand better how different professional groups use social media.In the experiment, the politicians were randomly assigned to two groups: the control group received an email without a personalized subject line, and the treatment group received an email where the title and surname of the recipient were included in the subject line.Given that these politicians were contacted in a more formal context, we decided to use the title and surname of the respondents and not the first name.The subject line in the treatment group would read, for John Smith: "Mr.Smith, how do you view digitalization in politics?"The content of the email and the other design elements were exactly the same.
The emails in both experiments were sent in German, but for the convenience of the readers, the subject lines were translated into English.The original subject line was "John, dein erfolgreicher Karrierestart" in the first experiment and "Herr Smith, wie beurteilen Sie Digitalisierung in der Politik?" in the second experiment.In addition, in the second experiment, the German titles "Herr" and "Frau" were used.These titles correspond to the English equivalent Mr. and Ms.
In both experiments, the following performance metrics are evaluated for the control group and the treatment group.
• Open/send: the opening rate is the percentage of send emails that are effectively opened by the recipients.• Clicked/send: the click-through rate is the percentage of send emails where the recipients followed the call-to-action in the email and clicked on the embed link.• Unsubscribed/send: the unsubscribe rate is the percentage of send emails where the recipients opted to no longer receive emails and unsubscribed from the email list.
To evaluate if the differences between these metrics in the control group and the treatment group are statistically significant, we use an independent sample T-test.
Finally, we want to highlight that the definitions of the performance metrics, and in a broader sense, the methodological framework we use in this paper is similar to the work of Defau et al. (2023).However, it is important to underline that the data we use in this paper is different from Defau et al. (2023), which researched the impact of authority figures on the performance of email marketing campaigns.The data we use in this paper was collected in two independent experiments.

Experiment 1
The results from the first experiment are presented in Table 1.In contrast to Sahni et al. (2018), our findings offer no support for the thesis that using the first name of the recipient in the subject line of an email raises the performance of email campaigns.Interestingly, in our experiment, the opening rates are even 3.2% lower in the treatment group, suggesting a potential negative effect from using the first name in the subject line.However, it is important to highlight that this difference is not significant (p > 0.05).The click-through rates in the treatment group and the control group are similar, offering an additional indication that using the first name of the recipient in the subject line of an email might no longer work in the current market environment.Finally, unsubscribe rates are 14.5% higher in the treatment group, but this observation should not be overstated, given the relatively low number of recipients that unsubscribed from the mailing list: 15 respondents in the treatment group versus 13 respondents in the control group.
In general, the results from this experiment are surprising, as they differentiate strongly from the findings of Sahni et al. (2018) and Munz et al. (2020).However, as Caraban et al., (2019, p.10) wrote, "we have a very limited understanding of the long-term effects of nudging in a technological context."Therefore, it might be that the effect from this marketing trick has faded-out, as more companies introduced it in their communication strategy, and it became a common technique in email marketing.In addition, also the composition of our sample might play a role, with university student who grew up in a digital world and therefore potentially more aware of online marketing techniques.Finally, as Lorente-Páramo et al. (2020) write, specific cultural dimensions can also play an important role in opening rates and click-through rates of email campaigns, and using this email marketing technique might just not work in every country or region in a similar way.

Experiment 2
The results from the second experiment are presented in Table 2.In contrast to the previous experiment, we find that using the title and the surname of the recipient in the subject line has a small-but positive-effect on opening rates and click-through rates.The opening rates are 11.8% higher in the treatment group, an observation which is significant (p < 0.05).In addition, the click-through rates are 14% higher in the treatment group, but this finding is not significant (p > 0.05).Finally, the unsubscribe rates in the experiment are very low: only one respondent in the treatment group unsubscribed from the mailing list.In general, these results could be related to the fact that using the surname of the recipient in the subject line of an email is not so common as using the first name.Therefore, this marketing technique might still have more potential, as it is not so well known with the broader public.In addition, the composition of the sample might also play a role in this outcome, with local and regional politicians, who are-in general-older than the university students included in the first experiment and maybe less experienced in the digital world.

Conclusion
In the academic literature, there is a growing stream of publications that address how companies can use insights from behavioral economics in their daily marketing activities.However, often the question remains if these findings can be reproduced in another experimental setting.In this article, we use the well-known work from Sahni et al. (2018) as the starting point to test this thesis.The original data from their research project date back to 2014, almost a decade ago, and a time in which important changes took place in digital marketing and email marketing.In general, the results from our experiment show no indication that using the first name of the recipient in the subject line of an email has a positive effect on opening rates or clickthrough rates-as such we cannot replicate the findings from Sahni et al. (2018).However, we do find that using the title and surname of a recipient in the subject line has a positive effect on opening rates and click-through rates; an indication that similar marketing techniques might still have potential, but that they need to be more creative and surprising to catch the attention of the target audience.Especially since technology changes fast and consumers adapt their behavior-a reminder for marketers that the effect from specific marketing tools can fade-out over time.
Funding Open access funding provided by Johannes Kepler University Linz.

Table 1
Result experiment 1-first nameThis table presents the results from the first experiment.The control group and the treatment group received the same email, only the subject line was different.The p-value was obtained using an independent sample T-test

Table 2
Result experiment 2-surnameThis table presents the results from the second experiment.The control group and the treatment group received the same email, only the subject line was different.The p-value was obtained using an independent sample T-test