Aims and Objectives
In early discussions between the EMIS Associated Partners, the prevention-planning objectives of EMIS were to identify prevention needs commonly unmet across diverse groups of MSM (priority aims), and to identify subgroups of MSM who have multiple prevention needs poorly met (priority target groups). Other objectives included: capacity building and knowledge transfer for European online research among MSM; the generation of MSM datasets in countries with fewer research resources; to facilitate dialogue between community, academic and public health sectors; and to maximise the educational impact on respondents of taking part in the survey—EMIS was a major opportunity to increase community knowledge about HIV.
The drive for a pan-European MSM survey came from a meeting in February 2007 organised by two of the authors (UM and AJS) working at the Robert Koch Institute (RKI, Berlin), which brought together researchers engaged in community surveying among MSM across Europe. Following this meeting, the RKI invited four organisations (in Germany, Italy, The Netherlands, Spain and UK) and a technical co-ordinator to be Associate Partners (APs) of EMIS.
The APs designed and sought funding for the survey from the Executive Agency for Health and Consumers (EAHC) of the European Commission. The first application having been declined, the APs redesigned and re-costed the project, and the second application was supported by the EAHC and was co-funded by the six APs. The Scientific Coordinator (AJS) took the lead in recruiting a further 77 collaborating partners (CPs) from community, academic and public health organisations across 35 European countries, aiming for one community and one either academic or public health partner in each country. In addition, three large and three smaller commercial gay dating websites were contracted with requests to support recruitment and dissemination and to advise on the acceptability of methods. Collectively, all partners are referred to as the EMIS Network.
EMIS was funded from March 2009 to September 2011. Funding facilitated the establishment of the EMIS Network, development of the questionnaire, a 3-month period of data collection, preparation and distribution of national and international data sets, and write-up in a variety of reports. Representatives of the EMIS Network met twice during the funding period, and the APs met a further four times. All other business was conducted electronically. The project was directed by a Steering Group of senior representatives from the six APs and supported by an Advisory Board consisting of two elected representatives from the CPs, and representatives of the ECDC and WHO/Europe. The project activity was divided into nine Work Packages each of which was designated a lead AP. An overall scientific co-ordinator was based at the Robert Koch Institute.
The population of concern to the EMIS Network is men living in Europe who have sex with men and/or feel attracted to men. This was operationalised in four inclusion criteria: self-identified as male; living in Europe; at or over the age of homosexual consent in country lived in; sexually attracted to men and/or had sex with a man in the last 12 months.
A description of the nature and purpose of the study was provided, but no assessment of whether participants understood it was undertaken. Participants were required to indicate that they understood these and that they consented to take part.
The AP leading questionnaire development (Maastricht University) requested all EMIS partners to supply pre-existing national or regional questionnaires for MSM, in English. In total, 23 questionnaires from 20 countries were received. Individual questions were organised according to frequency, diversity, terms used, subjects and response formats. Literature reviews were conducted to ensure the scientific basis of the questionnaire was supported by psychological and behavioural theories and previous studies.
Previous questionnaires, core indicators, scientific literature, consultation with experts and feedback from the APs resulted in the first draft of a questionnaire presented to the first general meeting of the EMIS Network in December 2009. All topic areas and items in the first draft were considered by the Network using a floating roundtable system, with discussion focussing on acceptability (to collaborators) and prioritisation. In addition, the meeting reached agreements on four design areas: recall periods, informed consent, the lower bound age and range of languages.
Following the first general meeting, APs re-examined the draft and made extensive amendments. The content of draft 2 was organised in four key conceptual areas:
The levels and distributions of sexual HIV/STI exposure and transmission facilitators (‘behaviours’),
The levels and distributions of unmet sexual health needs (‘needs’),
The population coverage and acceptability of prevention interventions (‘intervention performance’),
The information needed to compare samples and target interventions (‘demographics’).
With a limited length and a large collaboration, many interesting questions were suggested that could not be asked. Pertinence to the above areas was a key criterion for consideration in the survey. The design sought a balance of questions across the four areas.
The funded proposal did not specify an intended completion time for the survey. Initial discussions centred on the desirability of 20 to 30 min as maximising data collection within a tolerable length based on previous experience. The first pretesting (see below) showed a wide variation in completion times with a central tendency toward the top of our desirable range. Although the variation diminished in subsequent testing and in the field, the median completion time dropped only slightly.
The survey required questions that were relevant for all MSM across differences in biological and social gender, sexual identity, and social and political living environments. Designing questions to collect demographic data was often challenging because they require different but comparable constructions in different countries. For example, information on migration history, ethnicity and religion were difficult to query across Europe, due to large differences in immigration history, immigration laws and minority concepts.
Draft 2 suggested an appropriate order for the questions. APs agreed to start and finish the survey with relatively neutral questions to reduce respondent discomfort and under-reporting.
Following broad agreement among APs, draft 2 was posted on the EMIS website in mid-January 2010. The Network was asked to provide feedback within 4 weeks on: the length; the balance of question topics; the acceptability of questions for specific countries and clarity of the (English) wording. Involving potential respondents in survey design can simplify data collection and analysis (Daley et al. 2003). CPs were asked to pilot the English questionnaire for comprehension and length by asking five MSM to complete it using paper-and-pencil and to record their feedback. Detailed comments were received from 21 EMIS partners. Completion times were obtained from 51 men and ranged from 10 to 49 min with a median of 30 min. In addition to highlighting numerous minor issues, this process identified survey length as a key area of concern. Modifications based on this feedback resulted in draft 3, a paper version of which was circulated to APs at the end of February 2010 and approval sought within 1 week. Only few (minor) changes were needed to reach AP approval of draft 3 for transfer online.
Online Transfer and Piloting
The benefits of self-administration to maximising valid responses to sensitive questions apply to online surveys: Specifically, respondents are less likely to over-report desirable behaviours and less likely to under-report socially undesirable behaviours because of the sense of anonymity and/or confidentiality afforded by online surveys (Bradburn et al. 2004).
The transfer of a paper survey to an online survey requires numerous small (and occasionally larger) modifications. The questionnaire was constructed within the chosen Internet survey software (www.demographix.com) in English. Eleven men in London were observed completing the survey online and responded to questions about how they answered the survey, to ensure correct interpretation of questions and to identify difficulties in completion. The resulting online version was sufficiently different from draft 3 to warrant being called draft 4.
Draft 4 online was shared with both APs and CPs at the end of March 2010, and all were invited to pretest it in English with MSM who had not yet seen the survey, focussing on: routing (serving or skipping questions based on previous answers); response sets that varied by country (region and education qualifications); the HIV-test setting response subset (since settings varied greatly); completion time and acceptability. Comments and completion times were collected from 76 online pilots with a median time of 26 min (range, 10–45 min). Comments on draft 4 were also received from 26 partners. In response, several changes were made online to create draft 5 which was discussed at the third Steering Group meeting in London (April 2010).
Survey length remained an area of disagreement between APs. Longer surveys can collect more data per case but have greater attrition, making findings less generalisable. This was the only design issue that required a vote by APs to resolve. The decision of the group was towards a longer survey with greater attrition. Some minor changes to draft 5 arising from Steering Group discussion were made, and all routing associated with the English-language questionnaire was checked by three researchers working independently. The final English language online version was signed-off by the APs at the end of April 2010. It sought 278 data items (although not all respondents were asked all questions), covered the six core ECDC indicators and nine of ten MSM-specific ECDC indicators (European Centre for Disease Prevention and Control. Mapping of HIV/STI Behavioural Surveillance in Europe. Stockholm 2009) and is available at http://emis-project.eu/research-questions-and-covered-items.
The survey was presented on 25 core pages and, depending on answers to some questions (e.g. country of birth, HIV testing history, sexual partners and substance use), another 18 sub-group pages. To minimise completion time, the survey was tailored using intra-questionnaire filters (routing) wherever possible. For example, questions regarding non-steady sexual partners were not shown if the respondent had already stated they had not had any non-steady partners.
Many EMIS partners were concerned about the sensitive nature of the data (including sexual behaviours and partial postal codes). This was expected to negatively affect overall response rates, item non-response rates and response accuracy (Tourangeau and Yan 2007), perhaps because men would be afraid that their data would be misused by third parties (Singer et al. 1995). To allay these fears, the opening page of the survey described the study aims and informed potential respondents that their data would be anonymous, that their privacy would be maintained in line with the European Data Protection Directive, that no IP-addresses or other data that could be used to identify computers was saved, and the survey software installed no cookies or any other trace files on computers. We told respondents this because anonymity in Internet-based surveys has been shown to reduce social desirability effects (Joinson 1999). As a consequence, however, respondents could not pause the questionnaire and sign-in later to finish it. This was considered an acceptable loss to ensure anonymity. Not collecting IPs also meant that it was possible for one person to submit two or more questionnaires (while the absence of material incentives and the length of the questionnaire act against this).
The age of consent being different in different countries posed a challenge for a unified approach. The solution was to ask respondents to declare that they understood the aims of the study and that they were old enough to legally have sex with men in the country they were living in. That men were at the legal age of consent was not tested for each country individually. Men were asked to check a box that said “In the country I live in, I am old enough to legally have sex with men” and were left to judge this for themselves. We wanted to ensure respondents were aware of the aims of the study and participated on a voluntary basis.
Finally, discussions about sexual vocabulary had reached no firm consensus across countries. Familiar wording is widely believed to increase reporting of socially ‘undesirable’ behaviour (Daley et al. 2003) and was common in the submitted national questionnaires used for draft 1. On the other hand, some collaborators felt the survey would not be taken seriously (have authority, or appear scientific) if it used slang vocabulary. It was decided to use both formal and informal terms wherever possible, with one following the other in parenthesis. The order of the terms and the colloquialisms used were determined by the country leads with experience in MSM surveying in their own countries. The English language version of the survey uses formal language first, followed by a colloquial paraphrase in parenthesis, for example: Did you ejaculate (cum) into his rectum (arse)?
Translation and Online Preparation
Translations were carried out online, using the survey hosting software to display the English version on the left half of a screen and a duplicate on the right half to be over-written with the translation. This process minimised routing errors and copy-and-paste errors.
The survey was translated into 24 further languages, including 20 of the 23 official languages of the EU. It was not available in Maltese and Slovak (no EMIS partner had been established in either of these countries at the time of translation) or Gaelic Irish (as it is a geographically concentrated minority first language in the Republic of Ireland). In addition to these 20 official EU languages, EMIS was available in Norwegian and Ukrainian (additional funding was available for these non-EU languages), Russian (a minority language in Poland, Lithuania, Latvia and Estonia, and the most frequent immigrant language in the EU), Turkish (a minority language in Bulgaria and second most frequent immigrant language in the EU) and Serbian (a minority language in Hungary and intelligible to many EU immigrants from the former Yugoslav states). Regrettably, the partner from the Former Yugoslavian Republic of Macedonia withdrew from the EMIS Network when we were unable to identify resources for a Macedonian language version.
Translation was an interactive process involving native-speaking stakeholders from the EMIS Network and two native-speaker translators for each language. We also involved several multi-language proof-readers to compare the translations with the English original and with each other. The proof-readers ensured a harmonised multi-language questionnaire while deliberately maintaining certain differences, identified as culturally appropriate, such as explicitness of language or addressing respondents formally or informally. In all languages, simple rather than specialised terms were aimed for (Dillman 2000).
APs checked the translated versions for visual integrity and layout online and ensured that all language versions were structurally identical and had the same routing between questions, and all saved their data in an identical format. CPs confirmed the terminology fitted with their perceptions of the norm for the target group in their country, reviewed the final survey and signed-off their main language version by the end of May 2010.
A 3-month field work period in June–August 2010 was chosen early in the planning process other planning milestones slotted into place before it. The length and specific months were chosen to fit with the largest number of pre-existing behavioural surveillance tools in operation across Europe.
EMIS was available for completion online for 12 weeks, between 17:00 h (British Summer Time) on Friday 4th June and midnight on Tuesday 31st August 2010. The survey was functionally identical in each of the 25 language versions, but each version had its own Uniform Resource Locator (URL, the global address of documents and other resources on the World Wide Web). While each URL could be accessed directly, all advertisements and invites directed users to a dedicated landing page, hosted alongside the survey software. This landing page presented the 25 language names (‘English’, ‘Deutsch’, ‘Čeština’, etc.) in a five-by-five rectangle and carried a counter for the total number of surveys submitted to date across the 25 languages. By clicking directly on a language name, respondents were taken to the respective version of the questionnaire.
Our online survey provider stored all incoming data on multiple secure and encrypted data servers. These were backed-up daily and hourly at times of the highest volume of incoming data. Page view data were captured by our survey software, allowing estimates of attrition across the survey. Respondent-derived data (answers) were only transferred to our servers when the respondent clicked ‘submit answers’ at the end. Data from respondents who broke-off or who did not submit at the end of the survey were not captured. This would have been feasible (if technically complex) by using cookies to store data as it was input. However, doing so would compromise our ensuring the anonymity of respondents and was decided against.
Following submission, all respondents were sent to an HIV prevention website appropriate to the language of survey completion and country of residence. Exit websites were selected by EMIS partners.
Our survey software provider produced a consolidated version of the 25 surveys so responses could be monitored in real-time, both within any of the 25 language versions and in a single consolidated database. National lead partners had real-time online access to data collected that enabled them to see how many respondents had accrued from each promotional activity and conduct some descriptive analyses as fieldwork progressed. The AP with overall responsibility for survey promotion also monitored national response rates daily during the first 6 weeks and then weekly to the end of the survey. This centralised and decentralised monitoring of responses was used to plan specific promotions to stimulate recruitment in countries where lower responses than projected were observed.
Planning for the promotion of EMIS to potential respondents began 6 months prior to its launch. APs agreed that promotion would target national and transnational commercial and NGO websites, social networking websites and blogs, with printed posters for display in community venues, and business cards for hand-to-hand distribution. The APs committed to providing promotional materials in all 25 survey languages if requested.
During the latter stages of questionnaire development, all partners were asked to identify appropriate national and supranational websites for promoting EMIS. Representatives from one of the APs (Sigma) supported national CPs to establish contact with webmasters and reach agreements about the type of advertisement and promotion that was feasible, and fees payable, aiming to guarantee maximum visibility for minimum investment. If a fee was requested, the promotion lead AP (Sigma) liaised directly with the webmasters to ensure payment was made, and a precise contract was written and signed.
All websites promoting EMIS were allocated a specific URL for use on all their online advertisements. This URL took people to the EMIS landing page, and when they made their language selection, the source URL was captured as the first item of data from that respondent. Consequently, we know through which websites respondents were recruited. Overall, at least 237 unique websites recruited to EMIS of which 22 (9 %) required payment.
As a final method of recruitment, on submission, respondents were asked to nominate up to three friends to invite via E-mail to complete the survey.
Visual Identity, Buttons and Banners
After consultation and debate at the first general meeting, the Steering Group agreed on the broad principles for the visual identity of EMIS. Notions of community, inclusion and participation were central to these requirements. Thereafter, all partners were included in two online polls to establish preferences for a visual identity. The agreed visual identity was then used to develop the printed (offline) promotional materials and the online buttons and banners for websites.
All partners were polled for the choice of a core promotional slogan. The English version of the winning slogan was “Be part of something huge!” and an image of people putting their hands up, conveying the large-scale communitarian nature of the survey. In English rotating banners, the slogan alternated with the words “EMIS Men’s Sex Survey” and “Do me now!” in order to be both intriguing and mildly sexually suggestive. Some National EMIS partners chose to modify the slogan to best match vernacular characteristics.
A large number of clickable promotional banners and buttons were developed for use on websites. The promotion lead AP asked partners and prospective advertisers to confirm the number, size and specification of buttons and banners required and co-ordinated their production by Sparkloop®. Ultimately, 191 different buttons and banners were produced in 24 languages. These are available at www.sparkloop.com/visuals/emis. Clickable banner advertisements were used on national and transnational websites, some through paid advertising, but the majority carried at no charge.
Trans-national Online Promotion
In most countries, the largest proportion of respondents were recruited from five pan-European gay commercial and community websites sending instant messages (IMs) to their members and another four websites placing prominent banner advertisements for EMIS. Among the eight trans-national websites paid to promote EMIS charges varied considerably, with no obvious relationship between agreed costs and the number of recruits arising from any specific advertiser.
IMs consist of a short text and/or picture to the users’ personal message box describing and endorsing the survey. Five international websites were paid to send IMs: PlanetRomeo®, Manhunt/Manhunt Cares® and Gaydar®, each of which has membership across Europe; Qguys® for countries within the Commonwealth of Independent States and the Baltic countries and Qruiser® for Scandinavia. The invitation to take part was drafted by the APs, modified by the website representatives and agreed by both parties. The invitations came from and were signed by the websites. A typical invitation opened with the words “Because we know you are committed to getting the best sex with the least harm, we want to introduce you to questionnaire about sex, health and relationships”. The invitation then stressed the size and safety of the survey, suggested it would take about 25 min before offering potential respondents an exchange:
“Taking part might mean you find out something new and the information you provide will help programmes better meet our needs. A high response rate will also help us send a signal to governments that HIV and sexual health are still very important to the gay community. It could also mean that services for gay and bisexual men get funded.”
The invitation was crafted to maximise impressions of inclusion and impact by associating participation with positive and feasible community outcomes.
PlanetRomeo® was the single largest EMIS recruiter, advocating for the survey through multiple channels and recruiting 103,000 men. It was the first website to be contracted to send IMs, which were temporally staggered to 1,060,772 members across all target countries. IMs were sent in 25 different languages, matched to the member’s profile. Response rates to the PlanetRomeo® IMs varied by country from 5 % to 15 %, with a median response rate of 10 %. Later, during the fieldwork, the site carried a homepage article about the survey and a survey advertisement banner. PlanetRomeo® recruited more than 75 % of the sample in ten countries and 50–75 % in a further eight countries (see Table 1).
Manhunt/Manhunt Cares® recruited over 12,000 respondents from the second tranche of IMs. These were sent in six languages (English, French, German, Italian, Spanish and Portuguese) to 181,000 members during weeks 4 and 5 of data collection. A sub-tranche of IMs was sent in the last week of recruitment to men who had joined Manhunt since the first messages were sent out. Manhunt/Manhunt Cares® accounted for over 50 % of respondents in Portugal and substantial proportions in Spain, Republic of Ireland and the UK (see Table 1).
Gaydar® recruited 11,000 men, having undertaken the third large-scale international paid advertising, sending IMs and carrying banner advertisements during the last 5 weeks of the recruitment period. Gaydar® was asked to target those countries which had not yet reached three respondents per 10,000 general population. Five weeks before the end of fieldwork, these were Bulgaria, Czech Republic, France, Hungary, Ireland, Latvia, Lithuania, Moldova, The Netherlands, Poland, Portugal, Romania, Russia, Serbia, Sweden, Turkey, UK and Ukraine. This flexible strategy had variable success. Gaydar® accounted for almost half of the UK respondents and significant proportions in Ireland and Portugal (see Table 1).
In the latter half of the recruitment period, Qguys® was contracted to send IMs to all its members in the Russian Federation, Belarus, Estonia, Latvia, Lithuania, Moldova and Ukraine. IMs were sent in either Russian and English or Latvian and English, depending on the language of registration of members. Almost two thirds (63 %) of the 2,800 men recruited by Qguys® lived in the Russian Federation, a quarter (27 %) in Ukraine and a small proportion (5 %) in Belarus.
Similarly, Qruiser® was contracted to place banner advertising on its website and later to deliver pop-up messages to its members in Sweden, Finland, Norway and Denmark, matching the language in the message with the preferred language of the member. The vast majority (90 %) of the 2,377 men recruited through Qruiser® lived in Sweden with the bulk of the remainder in Finland.
Three other international websites recruited to EMIS using banner advertisements only, and none recruited more than 1,000 men. Banner advertising was purchased on Recon® and Barebackcity® in response to requests from a number of EMIS partners; however, the majority of respondents from these two sites lived in the UK and Germany, respectively. Finally, the main website of the International Lesbian, Gay, Bisexual and Intersex Association provided free advertising and recruited more than 100 men resident in a range of European countries.
During the last month, a final method of targeted recruitment using Google Adwords®, presented targeted (paid) advertisements to people who used specific phrases in the Google search engine. Advertising was targeted at men in countries where respondents per 10,000 total population was still below 1 (Poland, Romania, Russia, Slovakia and Turkey) but with limited success, recruiting 315 completed qualifying surveys across these five countries.
In addition to advertising, promotion was attempted in most participating countries using Facebook®, the world’s largest online social network site. Facebook® promotion included the establishment of EMIS Event Pages in all countries and targeted approaches to popular opinion leaders, HIV and LGB organisations and gay commercial organisations. All these people and organisations were asked to promote EMIS though their Facebook® networks. This free but very time-intensive approach did not result in a large number of respondents overall (approximately 1,500) but contributed a reasonable number of men in some countries, especially Sweden and Italy, but also Denmark, UK, Slovakia and Belgium.
National Online and Offline Promotion
At least another 227 national websites carried buttons or banners and recruited to the survey (some agencies did not seek a unique URL for their site but instead copied one from another website, so the actual number of sites recruiting to the survey will be above this number). Most national HIV and LGBT NGOs supported the project, usually at no cost. Fifteen national websites were paid for their promotion, including sites in Austria, Bulgaria, Denmark, Greece, Ireland, Italy, Latvia, Lithuania, The Netherlands, Romania, Slovenia, Spain and Switzerland. In some countries (e.g. Germany, Sweden, UK) partners financed additional national advertising. While paid-for advertisements were usually more productive than free advertisements, there was no direct relationship between the cost of advertising and the volume of banners served, or the number of men recruited. All requests from Collaborating Partners for funds for national advertising were fulfilled. National websites recruited more than 50 % of respondents in 13 countries: Bulgaria, Belarus, Denmark, Czech Republic, Estonia, Finland, Lithuania, Latvia, Moldova, Norway, Russia, Slovakia and Slovenia. Some national websites recruited men beyond their own borders, for example, a Czech website recruiting men living in Slovakia.
Partners complemented online promotion by distributing paper materials in a range of gay community settings. Materials, including business cards and posters, were developed by a lead AP in collaboration with CPs, who specified dimensions and quantities. The AP and CPs formulated precise wordings for cards and posters. Sparkloop® was commissioned to produce the materials. Many partners included referral information to national services on their promotional materials. Overall, 27 different versions of the business card were produced, and 139,350 were printed and delivered to 37 EMIS partners. In addition, 23 versions of the poster were produced, and 6,635 were printed and delivered to 24 EMIS partners. EMIS partners then often shared them with other agencies in their countries increasing dispersal of offline survey promotion.
No country-specific recruitment was undertaken in five countries (Cyprus, Malta, Bosnia, Luxembourg and Croatia) as there was no active CP there at the time of data collection. CPs in a further four countries (Greece, Hungary, Serbia and Slovakia) identified no suitable country-specific websites. In Turkey, only local and regional, but no national, LGBT organisations exist. Due to the terms of the grant, non-EU countries were ineligible to draw on the EMIS budget for promotion costs. Thus, for many of these 11 countries, response rates (based on total population size) were lower than elsewhere.
Attrition Across the Language Versions
Figure 1 shows the proportion of men who continued with the survey at each of the 25 core pages. The first page of the questionnaire constituted the introduction to EMIS and represents 100 % in the graph. The proportion of respondents served each of the subsequent 24 core pages is plotted on the graph. The proportion of respondents who proceeded from page 1 to page 2 (by confirming that they had read the introductory text, consented to participate and were old enough to legally have sex with men in their country of residence), varied from 36 % (Slovenian) to 76 % (English) and accounts for the majority of the total attrition across the survey. Slovenian stood out from all other languages in the size of this drop, most likely because the most productive Slovenian promotional site was not gay-specific but a generic dating website with MSM sections, therefore many people who accessed the introductory page were not MSM.
Almost all respondents in all languages moved from page 2 (the first page of questions) to page 3. Of those presented with page 2, the proportion who reached the 25th page (‘Submit’) ranged from 62 % (in Turkish) to 76 % (in Norwegian), with a mean of 68.5 % across the 25 languages.
There were 184,469 cases in the consolidated EMIS data file at the close of data collection. When downloaded, three cases were found to have been created by incorrect kerning in the survey software (the creation of another case by misreading of a comma in an open-ended response), resulting in 184,466 cases. Of these, 464 cases stated no country of residence, and another 1,963 stated a country of residence outside of Europe and were removed from the datasets leaving 182,039 cases (98.7 % of submitted) known to be living in Europe.
Non-qualifiers are respondents who did not meet the criteria for inclusion in the study. The number of cases submitted and the number of non-qualifiers in each country are presented in Table 2. Non-qualifying cases included:
Two cases did not check that they read and understood the introduction;
279 cases indicated they were women (but not transgender women);
196 cases provided no evidence for homosexual desire, gay/bisexual identity or sexual behaviour with men
303 cases gave no numeric value for age;
24 cases gave an age between 1 and 12;
33 cases gave an age over 89.
Some cases were disqualified on more than one criterion (for example, being both a woman and missing age); therefore, the total number of non-qualifying cases is less than the sum of the exclusions. A total of 544 cases living in Europe were excluded, leaving 181,495 cases who met the qualifying criteria.
EMIS data can be divided into national datasets based on current country of residence, regardless of the language used to complete the survey or country of birth, and combined into a pan-European dataset. National datasets have been made available for all 38 countries with 100 or more qualifying cases (those countries’ names not in brackets in Table 2). Thirteen European countries and states did not reach 100 qualifying cases: Albania; Andorra; Armenia; Azerbaijan; Georgia; Iceland; Kazakhstan; Kosovo; Liechtenstein; Monaco; Montenegro; San Marino and Vatican City. Data from the 291 MSM living in these countries or states are not included in any dataset.
The pan-European dataset includes respondents from all of the 38 countries in Europe with 100 or more qualifying cases.
Discrepant Data Flags
In several places, the questionnaire allowed logically inconsistent data to be supplied, where answers to two questions cannot both be valid. Inconsistent data could be submitted by moving backwards and forwards in the survey and changing previously given answers and simply by supplying inconsistent answers in different parts of the survey. To increase the quality of the data, we constructed six ‘discrepancy flags’ which indicated whether the respondent had supplied inconsistent data in six areas: age (six possible inconsistencies), HIV testing history (four possible inconsistencies), STI testing (three possible inconsistencies), sexual practices (seven possible inconsistencies), steady partners (seven possible inconsistencies) and non-steady partners (13 possible inconsistencies). Overall, 14.2 % of qualifying cases had one or more discrepancies (the maximum number observed was 11 out of a possible 30). National databases contained all cases so national leads can make exclusions according to their own needs. To strike a balance with case retention, the APs agreed that pan-European data analyses will exclude cases with two or more discrepancies, excluding 3.7 % of qualifying cases in EMIS countries. Hence, if the sample includes only residents of the 38 countries with more than 100 eligible respondents, and excludes all men where two or more data discrepancies were observed, data from 174,209 men are eligible for analysis.