Abstract
Background
The spread of substandard and falsified (SF) medical products constitutes a growing global public health concern. Some countries use portable, handheld screening technologies (STs) in the field to accelerate detection of SF medicines and reduce the number of medicine samples that undergo costly and time-consuming confirmatory analysis.
Methods
A multi-country, multi-stakeholder landscape assessment utilizing qualitative methodology was used to examine practices and perceptions related to the use of STs. Qualitative interview guides were designed using the results of a literature review and comprised of open-ended questions with the study participants, who were from national medicine regulatory authorities, pharmaceutical manufacturers, pharmacies, and distributors. Ten geographically and economically diverse countries were selected: Argentina, China, Egypt, India, Jordan, Mexico, Nigeria, Philippines, the United States, and Zimbabwe. Of the completed 53 interviews, 32 were in-person, 16 were phone interviews, and 5 were via written questionnaires.
Results
Data analysis shows a wide variation in understanding and usage of STs in different sectors. Virtually all of the study participants indicated a lack of objective, accessible information on STs to advise them on what technologies would be beneficial for their needs. Study participants also described their ideal capabilities of the next generation of STs, including shareable spectral libraries, lower acquisition costs, lesser training requirements, and in-country maintenance and technical support.
Conclusion
The results and recommendations presented in this article can be used to help regulators communicate and justify their needs to acquire and invest in new STs. There is a need for additional standardized, trustworthy and scientifically sound evaluations of STs, and to support regulators to effectively deploy the most promising technologies. ST manufacturers can take into account some of the limitations of the technologies the interviewees identified in this article, such as a dearth of technologies, which provide quantitative information about the active ingredient, and take steps to address them to better serve their customers. These results and recommendations, can catalyze research and actionable interventions into the development, review, application, and use of STs.
Similar content being viewed by others
Background
The spread of substandard and falsified (SF) medical products continues to be a growing global concern [1,2,3]. Their prevalence in the public, private, and informal market sectors threatens global public health by jeopardizing patient safety, diminishing confidence in health systems, increasing treatment failure, wasting valuable resources, and contributing to the development of drug resistance [1, 4,5,6]. Although SF medicines negatively impact public health in both developed and developing nations, available data unequivocally demonstrate that developing countries have greater numbers of poor quality medicines circulating in their markets and, as a consequence, suffer greater health burdens [1, 2, 7, 8].
Accurate global estimates of the prevalence of poor quality medicines do not exist [1, 4, 9, 10]. However, a reasonable prevalence estimate for falsified medicines in developing countries ranges from 10% to 30% [11, 12]. The literature suggests that the global market for falsification of medicines, at $431 billion USD per year, may be on par with that of illicit drugs, at $435 billion USD per year [13, 14]. The World Health Organization (WHO) is in the process of completing a study on the public health and socioeconomic impact of SF medical products based on published, reliable surveys conducted over the past 10 years. This study will provide a benchmark against which to calibrate future responses to SF medicines.
Over the last 5 years, research in screening technologies (STs) has expanded to more than 20 unique, portable technologies available to address poor quality medicines. Some STs are commercially available while others are still in the process of development or field testing [6, 7, 15,16,17]. The screening analysis methods currently used in countries include physical, visual, chemical, and microbiological analyses. By confirming ink color, language, spelling, shape, size, and other variable data, visual and physical inspection of the finished pharmaceutical product and its packaging can determine a suspected poor quality product due to wrong or tampered packaging [18]. Tablets, capsules, and liquids can also be examined for imprint, color, and odor irregularities, as well as visible contamination. Alternate light sources can also be employed. For example, the U.S. Food and Drug Administration Counterfeit Detection Device Version 3 uses ultraviolet and infrared light to identify falsified products [19]. Chemical analysis can identify medicines with little or no active ingredient or incorrect active pharmaceutical ingredient by using colorimetric tests, thin-layer chromatography, or various forms of spectroscopy. Finally, microbiological analysis can demonstrate the potency of antibiotics and sterility of injectable drug products.
A comprehensive article in 2014 by Kovacs et al., details the available field and laboratory STs for identifying poor quality medicines, including each technology’s need for electricity, sample preparation, reagents, portability, level of training required, and speed of analysis [15]. In addition, all STs were categorized by cost: $10,000 USD; $10,000 to $100,000 USD; and greater than $100,000 USD. The paper identified two key issues when examining STs: 1) performance data was not always available for each technology in use or STs under development; and 2) a gold standard was lacking as a comparator for all technologies available to detect SF medicines. Lastly, a WHO Collaborating Center survey in 39 countries outlined the quality control laboratory techniques (e.g., chromatography, spectrophotometry) used to test for SF medicines, but it did not differentiate between traditional, laboratory-based techniques, and STs used in the field [20].
Developing countries urgently need inexpensive, easy-to-use, portable, and rapid methods to detect poor quality medicines and diagnostics throughout the supply chain [21,22,23,24,25]. STs are not meant to obviate the need for a functional Official Medicines Control Laboratory (OMCL), but they should be an integral part of a country’s quality assurance toolkit, particularly in low and low middle-income countries where poor quality medicine prevalence is higher and the number of highly trained lab technicians is lower.
The aim of screening is to reduce the number of samples an OMCL must test, which subsequently reduces the burden on the laboratory and its limited resources [21]. Screening methods can identify suspect products, but they cannot replace confirmatory quality control testing required by each country’s legal framework [20]. Making these technologies more accessible will help control the proliferation of SF medicines, protect consumers, and generate accurate estimates for the prevalence of poor quality medicine [6].
In summary, there is a technology gap in the development of affordable, easy to use, and precise STs. There is also an information gap of standards, data, and objective evaluations needed to compare and contrast STs so that countries can decide which are most appropriate for their needs. The purpose of this study is to carry out a global landscape assessment of the benefits and limitations of STs and to more accurately ascertain current practices and country needs.
Methods
This research study was composed primarily of a literature review and a qualitative research component. This article focuses on STs that are available for finished pharmaceutical products. It does not cover authentication technologies (e.g., holograms, fluorescent inks, barcodes, chemical taggants) that enable authentication of a medical product but often do not assess its key quality attributes (e.g., active pharmaceutical ingredient identity). In the context of this article, the term “medicines” includes branded and generic finished pharmaceutical products for all types of therapeutic indications and dosage forms. Substandard and Falsified medicines, as defined by the WHO Member State Mechanism, includes those that are substandard or out of specification (i.e., authorized medical products that fail to meet either their quality standard or their specifications, or both) and falsified (i.e., medical products that deliberately or fraudulently misrepresent their identity, composition, or source). The term, “poor quality medicines” encompasses all SF medicines. “Track and trace systems” are not considered medicine-quality ST; however, they are useful and efficient tools for fighting the distribution of poor quality products. Examples of track and trace methods include radio frequency identification, traditional and two-dimensional barcodes, microtags, nanoencryption, and mobile verification.
The literature review focused on the problem and scope of poor quality medicines globally and the current practices relating to the application of STs for assuring the quality of medicines and their relative advantages and limitations. The search terms used were “poor quality medicines,” “substandard medicines,” “falsified medicines,” “counterfeit medicines,” “screening technologies,” “detection technologies,” “medicine screening,” “medicine testing,” “counterfeit medicine detection,” “substandard medicine detection,” “falsified drug detection,” “poor quality medicine detection,” and substituting the term “drugs” for medicines for all search terms. PubMed, Web of Science, Google Scholar, Essential-drug list serve, and other available databases, including the WHO’s global digital library, Worldwide Antimalarial Resistance Network’s Antimalarial Quality Literature Surveyor, and the U.S. Agency for International Development’s Development Experience Clearinghouse, were used to search for literature. The literature review focused on articles from 2012 to October 2016. The justification for this time period being that the report, Countering the Problem of Falsified and Substandard Drugs in 2013 captured voluminous citations on the scope and problem of poor quality medicines globally and the article Technologies for Detecting Falsified and Substandard Drugs in Low- and Middle-Income Countries in 2014 provided a comprehensive list of STs that exist [6, 15].
The goal of the qualitative research component was to elicit a wide range of responses from the survey participants to better understand the current landscape. A multi-country, multi-stakeholder landscape assessment utilizing qualitative methodology was used to examine practices and perceptions related to the use of STs. Qualitative interview guides were designed using the results of a literature review and comprised of open-ended questions with the study participants, who were from national medicine regulatory authorities (MRAs), pharmaceutical manufacturers, pharmacies, and distributors. Ten geographically and economically diverse countries were selected: Argentina, China, Egypt, India, Jordan, Mexico, Nigeria, Philippines, the United States, and Zimbabwe. Of the completed 53 interviews, 32 were in-person, 16 were phone interviews, and 5 were via written questionnaires.
Country selection procedure
Country selection criteria are listed in Fig. 1 and Table 1. Local manufacturing capacity, while not explicitly used as criteria for country selection, was considered. For example, Argentina, China, India, Nigeria, and the United States are major pharmaceutical manufacturing countries globally and in their respective regions.
Key Country Selection Criteria Considered — World Bank development classification, number of MRA employees, pharmaceutical sales, and population. 1Estimates based on information obtained from interviews and available literature and reflect predominantly numbers associated with Federal level staff. Apart from those provided for the Goa FDA (a state level institution), these numbers do not include provincial, state, city level staff, or contractors. 2Business Monitor International (BMI). 3United States Census Bureau: International Database [database on the Internet]. Available from: https://www.census.gov/popclock/world. 4BMI and WHO Global Health Expenditure Database. 5 Babu, G. (2017). Personnel shortage, lack of funding hold back India’s drug regulator. [online] Business-standard.com. Available at: http://www.business-standard.com/article/companies/personnel-shortage-lack-of-funding-hold-back-india-s-drug-regulator-115111100633_1.html [Accessed 13 Oct. 2017]
The following factors were used for country selection:
-
Geography: Countries were divided into regions using the WHO classification (i.e., Africa, Americas, Eastern Mediterranean, South-East Asia, Europe, and Western Pacific). Europe was the only region not included in this study (the Limitations section provides details).
-
World Bank Development Classification: countries were selected from all classification areas ranging from low-income to high-income Organization for Economic Co-operation and Development countries [26]
-
SF data: for each potential country, SF data, reports, and articles were examined. After preliminary selection, countries were included if literature indicated local problems with SF medical products.
-
MRA functionality: pharmaceutical sector analyses were available for many countries and if not, available information on country MRAs was examined. The number of MRA employees was noted in the selection process, as this can be an indicator of an MRA’s budget and capacity to carry out its core functions.
-
Availability and willingness of countries to participate in the study: countries needed to be available and willing to host the qualitative researcher and arrange interviews with the country MRA, OMCL, manufacturers, and pharmacies or to arrange for the interviews to be done via phone.
-
Pharmaceutical sales as a percentage of health expenditure: countries where pharmaceutical sales as a percentage of total health expenditure was varied were selected.
-
Country population estimates for 2016: the number of country inhabitants was considered. The study wanted to examine countries with large populations (over 1 billion) and countries with smaller populations (less than 15 million) [27].
Interview guide development
Qualitative research methods, such as in-depth interviews, were determined to be the best approach to soliciting current information about STs in use in the selected countries. The in-depth interview guides, including the types of questions to be asked, additional probes, and which personnel should be interviewed, were constructed based on the results of the literature review. Interviews were conducted and participants were asked open-ended questions on the following: the use of STs (i.e., when, where, how, and why) to detect poor quality medicines along the supply chain by their respective organizations; the types of STs in use and reasons for using them; the benefits and limitations of using these STs; and what is needed to better utilize STs and what would be characteristics of the ideal ST.
Based on the literature review, interview guides were developed (see Additional file 1) for the different groups participating in the study (MRA and OMCL staff, pharmaceutical manufacturers, and distributors and pharmacies. All interview guides covered issues around medicine quality issues, the status of STs, and post-marketing surveillance (PMS) activities being carried out in their respective countries.
Sampling plan
In-depth, semi-structured interviews of key-informants representing government regulators (R), manufacturers (M), and distributors and pharmacies (DP), were carried out in 10 countries from April–September 2016. For each country, interviewees included key staff from the national MRA and quality control laboratory, pharmaceutical manufacturers (private and public, where possible), and distributors and pharmacies (private and public, where possible).
Data collection
The participants represented various sectors involved in assuring the quality of medicines in their respective countries; participants self-identified as regulators, manufacturers, distributors or pharmacists in each of the 10 countries.
Recruitment of participants was done in multiple ways. With the help of U.S. Pharmacopeia contacts on the ground in each country, the study team contacted persons who agreed to be interviewed. Prior to the interview, participants had been informed of the interviewer’s arrival in country (or in the case of telephone interviews, a specific date and time was agreed upon) and had been briefed on the goals and objectives of the study. In most countries, interviews were carried out until saturation was reached for the issues around the use of STs. Interviews were done in English, with the exception of Mexico and Argentina where the interviews were carried out in Spanish. To maintain reliability, only two researchers carried out the interviews; the lead qualitative researcher carried out interviews for eight countries and a second researcher, who was trained to use the interview guides by the qualitative expert, conducted the Spanish-speaking interviews.
Where in-person interviews were not available, phone interviews were carried out. In a few instances with regulators and manufacturers, interviews were not an option; therefore, written questionnaires in English were sent out to these participants. All in-person interviews, phone interviews, and written questionnaires used the same interview guides specific for each organization.
Interview guides for R, M, and DPs, with open-ended questions to guide the interviews, were prepared prior to data collection. However, some spontaneous generation of questions also took place as the interview progressed and necessitated follow-up or probing questions to solicit details on topics that emerged during the interview.
Data analysis
Data included verbatim transcripts from in-person interviews and phone calls as well as from written responses to questionnaires sent to some of the study participants. Textual data were analyzed after de-identifying the data. Coding and data analysis were carried out using qualitative data analysis software, MAXQDA, Version XII [28]. Codes were used for each of the key themes and subtopics that emerged as the data was analyzed. Codes were revised and refined as new themes and topics emerged during the data analysis. A single researcher carried out the coding and analysis, ensuring no discrepancies in the use of codes. Inter-coding agreement of a part of the data and comparison of the use of codes was done to ensure reliability, and to resolve any discrepancies in coding. An independent translation company translated and transcribed the Spanish interviews and recordings from Argentina and Mexico into English.
The primary purpose of the qualitative data analysis was to identify themes or patterns in the responses to the research questions that would be included in a review of STs. A theme represented a pattern in the responses to each question and emerged from the coding of the data. All emergent themes and sub-themes were considered regardless of the frequency in the data set. A thematic analysis was conducted by developing detailed descriptions of the important issues around the use of STs for confirming medicines quality and detecting SFs. To compare common themes and patterns across interviews, data was grouped according to the country in which the interview took place.
Results
Of the 53 interviews carried out for this study, 32 were in-person (Argentina, India, Mexico, Nigeria, Zimbabwe), 16 were phone interviews (China, Egypt, India, Jordan, Philippines, USA), and 5 were written questionnaires (China, USA). Results of the analysis showed a wide variation in understanding and usage of STs in different sectors. Overarching themes included a need for current information and data on the capabilities of available STs, the importance of transferable spectral libraries, lower acquisition and maintenance costs, simpler training requirements, access to in-country maintenance and technical support for technologies, as well as expectations and technical suggestions for the next generation of STs.
While most information related to the use of STs is summarized in Tables 2–4, additional information was collected throughout the course of the interviews. This information is included in Additional file 2 and Additional file 3. The literature review results were used to develop the background, formulate the interview guides, identify themes related to the use of ST’s and determine the types of personnel to be interviewed. Therefore, the results section focuses on the interviewees’ thoughts and perceptions related to the use of ST and poor quality medicines in their respective countries.
Table 2 summarizes the information provided by regulators, manufacturers, pharmacies and distributors on their current screening and quality control practices.
All of the regulators and manufacturers interviewed have quality control laboratories to assure product quality either prior to release, in the case of manufacturers, or post market, in the case of regulators. Understandably, none of the distributors or pharmacies have quality control laboratories. Qualification or vendors and documentation checks are therefore used by these organizations to assure product quality of incoming samples. While most regulators conduct some form of post market surveillance, in countries where screening technologies are not being used all of these samples must be tested at the quality control laboratory.
Table 3 provides information on current use practices related to screening technologies as well as the benefits and limitations interviewees are facing in the deployment of these technologies.
Handheld Raman and near infrared spectrometers are commonly used by manufacturers primarily for raw material screening, while some regulators are using Minilab™ in addition to handheld Raman spectrometers as part of their post market surveillance programs. Those regulators not currently using STs cited cost as one of the major prohibitive factors while others highlighted that screening results cannot be used for regulatory action.
Table 4 summarizes the feedback provided by interviewees on the ideal qualities of a screening technology. Table 5 provides acronyms used throughout this article.
These results indicated a trend toward the following characteristics deemed ideal for a screening technology; compact size and easy to use, calibrate and maintain, fast and reliable with greater sensitivity and specificity and an ability to provide quantitative information, transferable data libraries, simple training requirements and finally, in-country technical support.
Discussion
The data obtained from the 10 countries selected in this study have shown wide variation in the awareness, understanding, and usage of medicine quality screening technologies in different sectors around the world.
Regulatory authorities
While regulators in China, India, Nigeria, the Philippines, and the US have been using one or more STs for several years, regulators in Argentina, Egypt, Mexico, and Zimbabwe are not using these devices. Jordan was using one ST for years but stopped. Interviewees provided a variety of reasons for using or not using these technologies. While Nigeria and the USA import many of their medicines and deploy screening technologies at their borders, other countries like India and China focus their efforts more on identifying poor quality medicines already in circulation in their local formal and informal markets. According to a regulator in China, “we collect samples nationwide from the market and they [inspectors] bring the collection back to the lab and test them.” All of the countries surveyed have informal markets where SF versions of essential, common, fast-selling, as well as expensive, low-volume medicines are found at cheaper prices. If inspectors had access to affordable, easy-to-use, handheld STs with analytical techniques recognized by the local MRAs, rapid, evidence-based decisions could be made about suspect products in these settings. The deployment of mobile labs in China and India is an excellent success story and has provided valuable expertise and flexibility to regulators keen on facilitating quick quality analyses and taking appropriate regulatory actions. In the USA, screening using handheld detection technologies at import locations and international mail facilities, immediate SF alerts, and sharing of information between domestic and international regulators has been successful in rapidly detecting poor quality medicines. These two examples provide a possible framework for large and small countries alike that are eager to incorporate STs into their quality control and assurance systems.
Regulatory authorities in nine out of 10 countries believe that screening technologies can and do enable more efficient and risk-based PMS by increasing the volume of samples that can be rapidly screened in field settings and reducing the volume of samples that must be transported to and tested at quality control laboratories; thereby increasing coverage and decreasing the overall cost of surveillance. However, the effective deployment of STs is predicated on an understanding of their capabilities. As such, most regulators also expressed an interest in having information about the capabilities of existing STs to understand what is out there and inform their future procurement decisions.
Manufacturers
Contrary, but complementary to regulators, manufacturers deploying STs were using them almost exclusively to confirm the identity of raw materials. It is impossible for manufacturers that produce a large variety of products to test every container of active ingredient and excipients that they receive using compendial methods. Preferred technologies such as handheld Raman and near infrared spectrometers therefore provide quick and reliable qualitative identification information about these materials. A U.S. manufacturer indicated that these “peer-reviewed technologies are acknowledged as accurate, effective, and suitable as predicates for expert testimony in court.” Deployment of STs is also in response to most regulators requiring manufacturers to inspect their raw materials used in production. As pharmaceutical manufacturing regulations in developing countries begin and continue to incorporate Quality by Design principles, which refer to manufacturers understanding and subsequently designing a manufacturing process that consistently delivers the desired product quality (FDA 2006 – Guidance for Industry, Q8 Pharmaceutical Development), we would imagine that manufacturers in these settings might start to incorporate STs into their systems even more [29].
The absence of routine PMS by manufacturers in most of the countries surveyed means that only products that receive consumer complaints get checked for quality. Although a Philippine manufacturer is interested in deploying screening technologies “to check incoming raw materials and to use it in the finished product line”, several manufacturers in lower and lower-middle income countries included in this study felt that it is the responsibility of the MRA to monitor post market medicine quality. The authors of this article challenge this perception because the reality of MRAs in lower income countries is one of being overburdened and underfunded. It would therefore behoove these manufacturers, particularly those from China and India that are exporting great quantities to these lower income countries, to consider implementing their own PMS programs. Firstly, by demonstrating a commitment to PMS they would be projecting transparency and a reputation for accountability to their customers, which may indirectly drive sales through an increase of ‘brand trust’. Concomitantly, data acquired through PMS could be shared with local MRAs to foster collaboration, detect problems with vendors, and identify systemic issues within local supply chains.
Pharmacies and distributors
Pharmacies and distributors in all 10 countries stressed that the STs they are aware of are cost prohibitive and some distributors mentioned the need for extra space to carry out screening activities as a drawback. Therefore, they rely heavily on track and trace technologies and complement these approaches with good procurement practices and documentation checks to maintain customer confidence. As one pharmacist in Argentina said, “Our main tool is the evaluation of suppliers and clients…whose products we will be selling.” Track and trace systems can verify the authenticity of a product but cannot determine whether possible improper storage and transport have compromised the potency or even the identity of the active ingredient. We believe STs can elegantly complement the track and trace technologies currently in use; particularly in countries where the supply chain is fragmented, and high heat and humidity contribute to medicine degradation. It would be interesting to understand further how pharmacies and distributors define and quantify ‘cost prohibitive’, whether their procurement practices and existing checks incorporate elements of risk management or cost-benefit analysis, and how the role of STs could mitigate risk and reduce cost of their operations in the long term.
Users need to be able to determine the benefits and identify the limitations of using specific STs. Interviewees identified constraints to using STs, which included cost, development of spectral libraries, limited human resources and trained personnel, and availability of technical support and customer service.
Screening technologies—Benefits, limitations, and the ideal instrument
Virtually all of the regulators, manufacturers, pharmacies, and distributors indicated a lack of trustworthy, accessible information on screening technologies to inform them on what technologies would be beneficial for their needs. In fact, through their responses, a number of participants demonstrated a lack of understanding about the capabilities of existing STs. This reinforces the need to have information on types of STs as well as clear explanations of how they function. The information should be at a level that is understandable not only by highly technical quality control lab staff but also regulators, MOH officials who make high-level budgetary decisions, customs agents, distributors, pharmacists and even patients. Therefore, a key recommendation from this study is the need for a standardized, trustworthy, scientifically sound, and comprehensive technical review of the capabilities of current STs that is accessible to any organization interested in using these STs to reduce the prevalence of poor quality medicines globally, regionally, and locally. These reviews should also be objective, in that they are performed by an independent organization with technical expertise on the techniques that underpin the technologies of interest. This would provide valuable information to current and potential users to identify technologies that are most suitable for their particular requirements.
Several interviewees indicated that objective reviews of STs would be valuable to inform existing and potential technology users and guide the direction of future quality control testing activities. A regulator from China said reviews would be useful to know what is being used outside of the country while a Zimbabwean regulator expressed that this information would allow them to assess and compare technologies and select those most appropriate for their needs. However, comparative or ranked reviews of technologies should be avoided when evaluating technologies that employ different techniques. In the same way that Fourier Transform Infrared spectroscopy should not be compared to High Performance Liquid Chromatography, a handheld Raman instrument should not be compared to an alternate light source instrument. The technologies provide complementary information, and while one may be more suitable in a particular context or setting, it would be inaccurate and misleading to categorically state that one is ‘better’ than another.
The context for implementing STs varies across user groups and, as such, may require additional tailored work, context-specific research, and targeted evaluations. For example, a manufacturer seeking to identify a few raw materials in the relatively controlled setting of a receiving warehouse may require a different technology than an inspector sampling and screening hundreds of products in rural pharmacies where temperatures are high and power sources are not available. In another setting, patients in urban areas would benefit from an internet-connected, smart phone-compatible device that provides a simple yes-no result. Conversely, a global procurement agency conducting quality control analyses after receiving a shipment might require multiple, more complex technologies that can quantify the active ingredient of interest, detect impurities, and measure dissolution. In fact, global health and donor organizations are integral in assisting lower income and lower-middle income countries to obtain, pilot, maintain, and train staff for these STs. Significant international aid is given to procure medicines and it is often a requirement and always in the donor’s best interest to ensure the quality of these donated medicines is maintained throughout the supply chain as exemplified by the Global Fund’s Quality Assurance Policy, which requires that the “source and quality of the raw materials entering into the finished product meet accepted quality standards” and that “quality control measures are in place and adequate” for all pharmaceutical products procured with Global Fund money [30]. A more specific example of the value of post-marketing surveillance can be seen in Liberia, where with support from the U.S. Agency for International Development-funded, U.S. Pharmacopeia implemented Promoting the Quality of Medicines program, the Liberian Medicines and Health Products Regulatory Authority used Minilab™ and confirmatory testing to show that half of antimalarial medicines sampled in a 2010 and 2011 study were of poor quality [31]. Therefore, the international donor community should be further engaged in funding and supporting countries to select STs and implement their use.
Although interviewees stressed a lack of objective information about the available STs and the fact that “there is no one perfect technology”, most regulators were able to identify the features of an ideal technology for their specific settings as highlighted in Table 4. In some instances, Minilab™ has been replaced by Raman and near infrared spectrometers because these do not require chemical reagents for testing. Users of one technology were reluctant to acquire a different technology after investing significant time in its calibration and maintenance and the development of customized spectral libraries. Low-maintenance instruments are therefore preferred because service providers and instrument vendors are often not located in the country and it is costly to schedule travel for servicing and repair of inoperable instruments. In the Philippines, where Minilab™ is being used for surveillance and screening, procuring and importing reagents for the kits requires special permits from law enforcement. Therefore, many of the products in informal markets in the remote areas of the Philippines do not undergo adequate and sufficient screening. This is a challenge faced in many remote areas of the developing world so the ability of a technology to operate effectively without the need for large amounts of consumables and reagents is critical in these settings. Expanding upon this, many interviewees stressed the importance of a screening technology that is easy to transport (i.e., can be carried by one person, as small as a briefcase), rugged, and simple to train staff on and use. According to one of the pharmacists interviewed in Zimbabwe “[a device] should be simple and practical such that it can be integrated into our current receiving systems. For example, the dispatch clerk or a warehouse manager can actually use it, where it won’t need another specialized individual to come on board. This technology should help us understand the extent of the problem.”
A technology that provides quantitative information about the active ingredient was also mentioned several times as a need and instrument manufacturers should heed this recommendation. Minilab™ provides semi-quantitative information but is very operator dependent, meaning the accuracy of the results depend on the user’s skill level and familiarity with thin layer chromatography. Contrary to this, most handheld spectrometers are generally operator independent when performing identification tests, but at the very least, require complex offline chemometric analysis to quantify the active ingredient. The example included in the results section can be used by instrument manufacturers to continue to refine their technologies and to provide the support that goes along with them.
Regulators, manufacturers, distributors, and pharmacies all highlighted the limited information available on the financial resource requirements of procuring and deploying STs. These requirements encompass not only the upfront price of a technology but also cost per test, cost of consumables and accessories, cost of calibration and maintenance, and the human resource needs. Regulators especially, but also manufacturers, distributors, and pharmacies, should budget for all these items in the use of STs for PMS and/or quality assurance and quality control activities. Very few of the stakeholders interviewed had financial resources to purchase and ‘pilot’ instruments. Because of limited financial resources, as well as the time spent in the initial stages of procuring and deploying any given technology, access to practical budgetary information in addition to performance capabilities is invaluable in assisting users to identify how effective and sustainable a given technology is prior to procurement.
One idea that emerged from this study is the potential for ST equipment manufacturers to establish a tiered pricing mechanism similar to what some pharmaceutical manufacturers do with medicines. For example, the cost for a low-income country to purchase a specific handheld spectrometer would be A, while in a high-income country it would be C, and in middle-income countries it would be B, somewhere between A and C. In many countries, procuring equipment from outside the country and obtaining permission from the government to import these instruments requires tremendous amounts of paperwork and time. A manufacturer in Argentina stressed that “you have to meet hundreds of legal requirements and criteria just to get them [customs] to consider allowing equipment to enter the country.” Attempts to obtain the necessary permissions are often met with failure to acquire the technology needed. In Egypt, Jordan, and Mexico, regulators and manufacturers wanted to see detailed guidelines for the use of specific STs and acceptance of these technologies by the MRAs as a permissible form of testing, from which results could be used to prompt recalls and enforcement action. Currently, the absence of such recognition by either governments or pharmacopeias is one of the limiting factors for regulators to deploy these technologies at ports of entry. More specifically, the analytical techniques used by many STs are generally not the techniques accepted by the recognized local or international standard. Although certain MRAs have established standards that enable enforcement action on the basis of screening technology results, further legal and regulatory recognition would facilitate the incorporation of technologies such as handheld Raman or near infrared spectrometers into their existing PMS programs. Therefore, users must take into consideration not only costs and import restrictions, but also regulatory recognition of technologies as part of the process of medicine quality testing and subsequent enforcement action against poor quality medicines.
These sentiments reflect divergent thoughts of interviewees about the role of STs. With the advent and increasing uptake of continuous manufacturing, there has been fervent discussion about process analytical technology—which often includes STs such as handheld spectrometers—and its role in redefining, or controversially, replacing traditional quality control testing [32]. Certainly, as this new paradigm continues to evolve, the ubiquitous validation and recognition of screening technology test results to prompt legal action may become a reality. However, for the time being, quality control testing and the laboratory-based techniques it encompasses will remain the primary method for MRAs to obtain the data needed for enforcement action. Presently, screening technologies are just that—screening technologies. There is no ‘silver bullet’ technology so multiple STs are often, if not always, needed to paint a complete initial picture of the quality of a medicine. They should be used to conserve the resources of, reduce the burden placed on, and drive the sustainability of OMCLs and help manufacturers, distributors, and pharmacies confirm the quality of medicines they are producing, procuring, and selling. They are the integral second part of Pribluda et al.’s Three-Level Approach [33, 34]. An excellent example of this is in Nigeria, where the use of multiple handheld STs has reduced the prevalence of SFs in circulation. Nigeria’s National Agency for Food and Drug Administration and Control has incorporated the use of handheld STs in their government quality surveillance activities at the borders and at sentinel sites throughout the country and the use of results for enforcement after further confirmatory testing.
Although only 10 countries were included in this study, they represented five of the six WHO geographic regions and all World Bank economic classifications. This was done deliberately to ensure that the perspectives, successes, and challenges from a variety of settings were identified. This will also allow countries that were not included to glean information about settings similar to theirs and to identify opportunities from settings dissimilar to theirs. In-depth interviews of over 50 participants across three categories of organizations enabled the qualitative researcher to reach saturation. Therefore, we are able to provide a deeper and more robust understanding of SF perceptions and ST use rather than focusing solely on one type of organization, given that regulators have different needs than manufacturers, pharmacies, and distributors. However, it would be useful to re-visit some of the countries surveyed to see if after acquiring new technologies their experiences mirror the experiences of the countries that were already using such technologies. For instance, Zimbabwe was in the process of procuring handheld Raman spectrometers to use at their ports of entry while Nigeria has used these technologies at their ports of entry for almost a decade. Will Zimbabwe’s MRA experience the same issues that were faced by Nigeria or will Nigeria’s challenges and successes inform and enhance Zimbabwe’s activities? In this vein, this article can drive collaboration between organizations to share best practices and experiences.
Limitations
While this article has generated novel data and information on the topic of ST use, there are limitations to the paper. Firstly, Europe—the sixth WHO region, was not included in this study, which we consider a limitation. The focus for country selection initially was on low, middle, and upper-middle income countries with available SF data; the included Organization for Economic Cooperation and Development and high-income countries (USA, Mexico, and Argentina) were added based on U.S. Pharmacopeia contacts and feasibility to facilitate interviews. Secondly, difficulties in contacting interviewees and scheduling time for visits and phone interviews meant that the data from a few countries was not as expansive as data from other countries. Thirdly, discussing SFs in all countries can cover sensitive topics in the public and private sectors, irrespective of whether one is a regulator, manufacturer, pharmacy, or distributor. Although interviewers took precautions to avoid the use of leading questions, the nature of the topics of discussion may have resulted in the introduction of some bias that influenced the response of participants from giving divulging sensitive information. We strongly believe we were able to engage participants who were willing to share their perceptions based on the confidentiality of the research methods. However, some countries were more reticent to openly discuss the issue of SFs.
Conclusions
This is the first multi-country, multi-stakeholder landscape assessment that utilizes qualitative methodology to examine practices and perceptions related to the use of medicines quality screening technologies. Existing and potential technology users can benefit from this landscape assessment and use the information to better understand these available technologies and the real benefit they can provide. It is well documented that compendial testing of all sampled medicines by OMCLs in lower to middle-income countries is virtually impossible due to budgetary limitations and human resource constraints.
This article can be used to help regulators communicate and justify their needs to acquire and invest in new STs. It identified the need for objective, technical reviews of the capabilities of current STs that are accessible to organizations interested in using these technologies, while confirming that there is no ‘silver bullet’ technology. Interviewees also highlighted the need for information on the financial resource requirements of procuring and deploying technologies. To support this technical information, strategies to enable regulatory recognition of ST results, where appropriate, could engender the broader update of these instruments by regulators. ST manufacturers and other organizations supporting these activities can take into account these recommendations and some of the limitations of these technologies identified by the interviewees and take steps to address them to better support their stakeholders.
As more countries and users realize the value of these technologies and begin to invest in them, an ongoing dialogue between regulators, pharmaceutical manufacturers, ST device manufacturers, pharmacies and distributors, academia, and donors who may purchase these instruments will be paramount to ensuring their effectiveness and continued value.
Medicines quality screening technologies play a pivotal role along the medicines supply chain by assuring the quality of medicines that are manufactured, distributed, marketed, and prescribed. They are and will continue to be, part of the future in the fight to control the proliferation and public health impacts of SFs globally, particularly in developing countries.
References
Substandard, spurious, falsely labelled, falsified and counterfeit (SSFFC) medical products: Fact Sheet. World Health Organization; 2016 [cited 2016 May 13]; Available from: http://www.who.int/mediacentre/factsheets/fs275/en/.
Buckley GJ, Riviere JE, Gostin LO. What to do about unsafe medicines? BMJ. 2013;347:f5064.
Kelesidis T, Falagas ME. Substandard/counterfeit antimicrobial drugs. Clin Microbiol Rev. 2015;28:443–64.
Johnston A, Holt DW. Substandard drugs: a potential crisis for public health. Br J Clin Pharmacol. 2014;78:218–43.
Kaur H, Clarke S, Lalani M, Phanouvong S, Guérin P, McLoughlin A, et al. Fake anti-malarials: start with the facts. Malar J. 2016;15:1.
Gostin LO, Buckley GJ. Countering the problem of falsified and substandard drugs. Washington D.C.: National Academies Press; 2013.
Nayyar GM, Attaran A, Clark JP, Culzoni MJ, Fernandez FM, Herrington JE, et al. Responding to the pandemic of falsified medicines. Am J Trop Med Hyg. 2015;92:113–8.
Blackstone EA, Fuhr JP Jr, Pociask S. The health and economic effects of counterfeit drugs. Am Health Drug Benefits. 2014;7:2017.
Dégardin K, Roggo Y, Understanding MP. Fighting the medicine counterfeit market. J Pharm Biomed Anal. 2014;87:167–75.
Christian L, Collins L, Kiatgrajai M, Merle A, Mukherji N, Quade A. The problem of substandard medicines in developing countries. Madison: Workshop in International Public Affairs, La Follette School of Public Affairs at the University of Wisconsin–Madison; 2012.
Pisani E. Antimicrobial resistance: what does medicine quality have to do with it? London: Review on Antimicrobial Resistance; 2015.
Counterfeit Medicines: an update on estimates: International Medical Products Anti-Counterfeiting Taskforce (IMPACT)2006.
Garrett L. Ensuring the Safety and Integrity of the World’s Drug, Vaccine, and Medicines Supply. Council on Foreign Relations Press; 2012 [cited 2016 May 31, 2016]; Available from: http://www.cfr.org/pharmaceuticals-and-vaccines/ensuring-safety-integrity-worlds-drug-vaccine-medicines-supply/p28256.
United Nations Office on Drugs and Crime. Technology, emerging drug markets and the future of trafficking: silk road 2.0. Princeton model United Nations conference. Princeton: Princeton University, Princeton, New Jersey; 2014.
Kovacs S, Hawes SE, Maley SN, Mosites E, Wong L, Stergachis A. Technologies for detecting falsified and substandard drugs in low and middle-income countries. PLoS One. 2014;9:e90601.
Barras J, Althoefer K, Rowe M, Poplett I, Smith J. The emerging field of medicines authentication by nuclear quadrupole resonance spectroscopy. Appl Magn Reson. 2012;43:511–29.
Chen C, Zhang F, Barras J, Althoefer K, Bhunia S, Mandal S. Authentication of medicines using nuclear quadrupole resonance spectroscopy. IEEE/ACM Trans Comput Biol Bioinform. 2015; https://doi.org/10.1109/TCBB.2015.2511763.
World Health Professionals Alliance. Be Aware: Tool for Visual Inspection of Medicines. World Health Professions Alliance; [cited 2016 Feb 15]; Available from: (http://www.whpa.org/Toolkit_BeAware_Introduction.pdf).
Batson JS, Bempong DK, Lukulay PH, Ranieri N, Satzger RD, Verbois L. Assessment of the effectiveness of the CD3+ tool to detect counterfeit and substandard anti-malarials. Malar J. 2016;15:1.
Brits M. Combating unsafe medical products: outcomes of a survey on testing of suspect medicines. WHO Drug Inf. 2014;28:317.
Hajjou M, Qin Y, Bradby S, Bempong D, Lukulay P. Assessment of the performance of a handheld Raman device for potential use as a screening tool in evaluating medicines quality. J Pharm Biomed Anal. 2013;74:47–55.
Fernandez FM, Hostetler D, Powell K, Kaur H, Green MD, Mildenhall DC, et al. Poor quality drugs: grand challenges in high throughput detection, countrywide sampling, and forensics in developing countries. Analyst. 2011;136:3073–82.
Ranieri N, Tabernero P, Green MD, Verbois L, Herrington J, Sampson E, et al. Evaluation of a new handheld instrument for the detection of counterfeit artesunate by visual fluorescence comparison. Am J Trop Med Hyg. 2014;91:920–4.
Mori M, Ravinetto R, Jacobs J. Quality of medical devices and in vitro diagnostics in resource-limited settings. Tropical Med Int Health. 2011;16:1439–49.
Glass BD. Counterfeit drugs and medical devices in developing countries. Research and Reports in Tropical Medicine. 2014;2014:11–22.
World Bank. Country and Lending Groups. [cited 2016 January 15]; Available from: http://data.worldbank.org/about/country-and-lending-groups.
United States Census Bureau: International Database [database on the Internet]. Available from: https://www.census.gov/popclock/world.
MAXQDA. Berlin, Germany: VERBI Software GmbH; [cited 2016 June 19]; Available from: http://www.maxqda.com.
Shah R. Quality by Design in Pharmaceutical Manufacturing. Chicago: AAPS; 2009.
The Global Fund to Fight AIDS, Tuberculosis and Malaria: Quality Assurance Policy for Pharmaceutical Products. Geneva, Switzerland [cited 2017 December 15]; Available from: https://www.theglobalfund.org/en/sourcing-management/quality-assurance/.
Promoting the Quality of Medicines (PQM) Annual Report on Activities for October 1, 2012 –September 30. Washington. In: DC: USAID; [cited 2018 January 15]; available from; 2013. http://pdf.usaid.gov/pdf_docs/PA00JMFV.pdf.
Chen JG, Gregory; Ng, Patrick; Rydzak, James; Kord, Alireza. Using process analytical Technologies for Real Time Monitoring and Quality Assurance in chemical development. American pharmaceutical review: GlaxoSmithKline Pharmaceuticals; 2010.
Pribluda VS, Barojas A, Coignez V, Bradby S, Dijiba Y, El-Hadri L, et al. The three-level approach: a framework for ensuring medicines quality in limited-resource countries: Pharmaceutical Regulatory Affairs: Open Access; 2014. p. 2014.
Pribluda V. RL. A Multi-Sectorial Approach to Ensuring Medical Product Quality & Supply Chain Integrity. United States Pharmacopeial Convention; 2016 [cited 2016 April 15].
Gostin LO, Buckley GJ, Kelley PW. Stemming the global trade in falsified and substandard medicines. JAMA. 2013;309:1693–4.
McGinnis M. Media Reports on Medicine Quality: Focusing on USAID-assisted Countries: United States Agency for International Development and the Promoting the Quality of Medicines Program, Implented by the United States Pharmacopeia 2015.
Cutting Edge Technologies. Lagos, Nigeria: NAFDAC; 2016 [cited 2016 July 21]; Available from: http://www.nafdac.gov.ng/index.php/component/content/article/187-nafdac-innovations/240-cutting-edge-technologies.
Ifijeh M. Nigeria: drug counterfeiting has reduced drastically - NAFDAC. AllAfrica; 2015; Available from: http://allafrica.com/stories/201505281489.html.
Akiny O. Counterfeit drugs in Nigeria: a threat to public health. Afr J Pharm Pharmacol. 2013;7:2571–6.
Maponga C, Ondari C. The quality of antimalarials. A study in selected African countries. Geneva: World Health Organization; 2003. [cited 2016 March 15]; Available from: http://apps.who.int/medicinedocs/pdf/s4901e/s4901e.pdf
Mandizha R. Fake medicine vendors mushroom as poverty bites. The Zimbabwean; 2015 [cited 2016 March 15]; Available from: http://www.thezimbabwean.co/2015/08/fake-medicine-vendors-mushroom-as-poverty-bites/.
World Health Organization: Prequalification Programme. Available from: http://apps.who.int/prequal/.
Argentina Federal Agents Investigate Fake Drug Crime Ring. The Partnership for Safe Medicines; 2010 [cited 2016 July 21]; Available from: http://www.safemedicines.org/2010/12/argentina-federales-investigate-fake-drug-crime-ring-60.html.
At least 42 police raids on the case of counterfeit drugs. Buenos Aires, Argentina2010 [cited 2010 December 9]; Available from: ww.momento24.com.
ANMAT: First National Drug Regulatory Agency designated as Reference Authority by PAHO. Buenos Aires, Argentina [cited 2016 July 21]; Available from: http://www.anmat.gov.ar/webanmat/institucional/anmat_referencia_ops_en.asp.
Derecho M. El Perfil de la Falsificacion de medicamentos en Argentina. Buenos Aires, Argentina: Revista SAFYBI Asociacion Argentina de Farmacia y Bioquimica Industrial; 2011.
American Enterprise Institute. The deadly world of fake Drugs: 2011.
Karlage KL, Franklin SJ, Mufich WC, Goetz KJ, Sabelka JV, Hoye WL, et al. Comparative evaluation of pharmaceutical products obtained in Mexico: augmenting existing scientific data. Drug Dev Ind Pharm. 2012;38:808–14.
Pan American Health Organization. PAHO recognizes COFEPRIS as a National Regulatory Authority of Regional Reference. Washington, DC, USA: 2012 [cited 2016 July 21]; Available from: http://www.paho.org/hq/index.php?option=com_content&view=article&id=6967%3A2012-paho-recognizes-cofepris-as-national-regulatory-authority-regional-reference&catid=1443%3Aweb-bulletins&Itemid=135&lang=en.
Bate R, Jin GZ, Mathur A, Attaran A. Poor quality drugs and global trade: a pilot study: National Bureau of economic research 2014.
Moran B. Cracking Down on Counterfeit Drugs. Public Broadcasting Service; 2013 [cited 2016 May 20]; Available from: http://www.pbs.org/wgbh/nova/next/body/uncovering-counterfeit-medicines/.
Galal R. Knockoff drugs endanger Egyptians. Al-Monitor; 2015.
Egypt seizes large amount of suspected fake drugs. CNN Money. 2009.
Faucon B. No cure for fake drugs. 2010.
Hundreds of pharmacies in Jordan violated the law in 2008. 2009.
JD 218,147 worth of counterfeit medicines seized over past 6 months. 2013.
Bagcchi S. Indian generic drugs debate heats up. Lancet. 2014;384:1334.
Chaudhiri S. Ranbaxy to pay $500 million in adulterated drugs case. . The Wall Street Journal; 2013 [cited 2016 January 15]; Available from: http://www.wsj.com/articles/SB10001424127887323716304578481182961557130.
Harris G. Medicines Made in India Set Off Safety Worries. New York Times; 2014 [cited 2016 May 31, 2016]; Available from: http://www.nytimes.com/2014/02/15/world/asia/medicines-made-in-india-set-off-safety-worries.html?_r=1.
Personnel shortage, lack of funding hold back India’s drug regulator: Business Standard 2015 November 11.
Uy J. FDA warns vs 20 unregistered drugs. 2015.
Taylor P. Philippines officials sacked over fake drugs shipment.2015.
Wong JQ, Baclay JRM, Duque RG, Roque PMS, Serrano GKT, Tumlos JOA, et al. Prevalence of Philippine prescribing, dispensing, and use behavior in relation to generic drugs and their risk factors. Philippine Journal of Development. 2013;40:125.
Torstensson D, Pugatch M. Keeping medicines safe: a study of the regulations guiding the approval of medicines in emerging markets. Stockholm. Network. 2010:26–32.
M. Lee, L. Hirschler. Special Report: China’s “wild east” drug store. Reuters; 2012 [cited 2016 May 31]; Available from: http://www.reuters.com/article/us-china-pharmaceuticals-idUSBRE87R0OD20120828.
Philipp J. Pharma Supply Scrutiny in India and China Could Tip Scales. Epoch Times; 2014 [cited 2016 May 31, 2016]; available from: http://www.theepochtimes.com/n3/546850-pharma-supply-scrutiny-in-india-and-china-could-tip-scales/.
China's Healthcare Sector, Drug Safety, And the U.S.-China trade in medical products, (2014).
Acknowledgements
We would like to thank U.S. Pharmacopeia’s Review of Screening Technologies for the Quality Assurance of Medicines expert panel for their expertise, feedback and time in developing this article.
We would like to thank all of the interviewees and other partners who kindly agreed to participate in this project, without whose openness and insight this article could not have been published.
We would like to thank Emily Kaine and Kelly Willis for their input and leadership.
We would like to thank Chimezie Anyakora, Damian Cairatti, Kevin Cao, Nonyelum Okpagu, Chaitanya Kumar Koduri, Ruther Mortel, Richard Rukwata, Satheesh Shetty, and Koduru Surendra Nath, for their assistance, patience and support in-country.
Funding
All steps that made up the development and publication of this article were funded by the United States Pharmacopeial Convention. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Author information
Authors and Affiliations
Contributions
Conceived and designed the experiments: AN BT LK LMR. Collected the data: AN LMR.
Analyzed the data: AN LK LMR. Wrote the paper: AN LK LMR. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
In 2016, the Global Public Health Technology Review program, of the U.S. Pharmacopeia, performed original research examining global practices related to the use of medicine quality screening technologies. As part of the article, local stakeholders in the ten countries selected for the study (Argentina, China, Egypt, India, Jordan, Mexico, Nigeria, Philippines, USA and Zimbabwe) were interviewed, either in person via telephone or through a written questionnaire. After consultation with the U.S. Pharmacopeia’s Legal division it was determined that no formal external approval (such as Institutional Review Board approval) was needed. However, the following steps were taken as part of the data collection and publication process:
-
1.
The study protocol meets the criteria for exemption per the regulations found at 45 Code of Federal Regulations 46.101(b)(2).
-
2.
Data was de-identified. Identifying details were kept in a location separate from the information provided by the interviewees. Information in the article was attributed to the country and type of organization (regulatory authority, manufacturer, pharmacy or distributor).
-
3.
The interviews did not collect private information.
-
4.
Verbal informed consent (or in some cases when a written questionnaire was utilized, email informed consent) was obtained from all adults who participated in the qualitative interviews. Prior to commencing the interview, the qualitative researcher read aloud the purposes of the study and requested each individual to ask any questions, give their consent or to refuse to participate. In cases where a questionnaire was filled out, the participants received a description of the study’s purpose. Since this research was determined to be exempt from IRB review (based on number 1. above) written informed consent was not required. No minors were included in the study.
-
5.
All components of the article (research, write up and publication) were funded by the U.S. Pharmacopeia, not by the federal government or any other external stakeholder.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1:
In-depth interview guide (approx. 60–90 min) for regulators. Interview guide template used for the interviews conducted with regulators (DOCX 21 kb)
Additional file 2:
Screening technologies by country, government Regulator [R], Manufacturer [M], and Distributor/Pharmacy [DP]. Identifies the screening technologies currently being used by the organizations interviewed, grouped by region and country. (DOCX 42 kb)
Additional file 3:
Additional interview information on use of screening technologies in countries surveyed. Highlights additional information gathered from interviews, which was not directly used in the results of discussion sections. (DOCX 27 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Roth, L., Nalim, A., Turesson, B. et al. Global landscape assessment of screening technologies for medicine quality assurance: stakeholder perceptions and practices from ten countries. Global Health 14, 43 (2018). https://doi.org/10.1186/s12992-018-0360-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12992-018-0360-y