Health Care Analysis

, Volume 16, Issue 2, pp 161–175 | Cite as

Participatory Workshops are Not Enough to Prevent Policy Implementation Failures: An Example of a Policy Development Process Concerning the Drug Interferon-beta for Multiple Sclerosis

  • Margriet Moret-Hartman
  • Rob Reuzel
  • John Grin
  • Gert Jan van der Wilt
Open Access
Original Article


A possible explanation for policy implementation failure is that the views of the policy’s target groups are insufficiently taken into account during policy development. It has been argued that involving these groups in an interactive process of policy development could improve this. We analysed a project in which several target populations participated in workshops aimed to optimise the utilisation of an expensive novel drug (interferon beta) for patients with Multiple Sclerosis. All participants seemed to agree on the appropriateness of establishing a central registry of Multiple Sclerosis patients and developing guidelines. Nevertheless, these policy measures were not implemented. Possible explanations include (1) the subject no longer had high priority when the costs appeared lower than expected, (2) the organisers had paid insufficient attention to the perceived problems of parties involved, and (3) changes within the socio-political context. The workshops in which representatives of the policy’s target populations participated did not provide enough interactivity to prevent policy implementation failure.


Argumentative policy analysis Interferon-beta Multiple sclerosis Participatory workshops Interactive policy development 


There are numerous examples of health policy measures that have failed in their implementation. A review from the Netherlands Court of Audit indicated that no evidence of a full policy implementation can be found in previous audits [20]. For example, the applied measures for reducing greenhouse gas emissions have not met pre-determined targets, the legal rules for maintaining food safety have not been able to prevent targeted risks, and policy measures for cost control of medicines have not resulted in more structural cost control [19].

The theory of argumentative policy analysis offers a possible explanation for such policy implementation failures [8, 15]. The basic idea of this theory is that actors’ behaviours can be explained by different views on a problem and the argumentation behind these views. According to the argumentative approach to policy analysis [7, 8], action is driven by processes of problem setting [27] in which actors define coherent sets of problems and solutions that correspond to these actors’ normative and empirical background theories. The way in which problems are defined depends on the assumptions the actors make about the situation and their beliefs regarding what is good practice (normative values).

Combining these findings with those of classical implementation theory, Grin and Van de Graaf [10] have argued that a policy will only be effective if both implementers and target populations consider the proposed policy measure meaningful. (Target populations consist of persons who will experience the consequences of a policy when implemented.) This means that the proposed policy should (a) make sense in the light of problems perceived by the target populations and (b) be consistent with their normative and empirical background theories.

However, the fact that policy problems and associated solutions tend to shift over time, different actors with different background theories being involved successively, renders this rather complicated. In other words, a policy measure is not invented at a specific moment in time, but develops over time. Nevertheless, the challenge is to identify policy measures that cohere with the views of all actors involved.

If the argumentative policy theory is correct, then it is advisable to identify the policy’s target populations and involve them in the process of policy development. An interactive process of policy making could thus ensure that policy coheres with and prevent that it diverges from the views of target groups.

The objective of this paper is to analyse a specific instance where target populations were involved in the process of policy development. The policy institution in this example was the Health Care Insurance Board in the Netherlands (HCIB). It is an advisory board to the Ministry of Health, particularly with respect to coverage and reimbursement issues. The Department of Policy Analysis of Medicines (PAM) is responsible for identifying developments that may jeopardise optimal medical care, analysing the nature and size of such threats and conducting further research that may provide a basis for policy decisions. The annual work programme in which topics are prioritised is submitted to the Ministry of Health for approval.

The policy objective discussed in this paper was to promote the appropriate use of a recently introduced drug for patients with multiple sclerosis. Multiple Sclerosis (MS) is a neurodegenerative disease characterised by neurological dysfunction. The drug interferon-beta (IFNβ) appeared to be a promising treatment. However, its costs were high (€ 12,000 per patient annually) and the evidence of its long-term effectiveness was limited. Therefore, policy measures to guide the prescription of this medicine were considered necessary. Representatives of prescribing physicians (neurologists), health insurance companies, and patients were invited to participate in two workshops. During these workshops, participants discussed policy measures that could promote the appropriate use of the new drug. Although two concrete policy measures had been proposed, these have never been implemented.

In this paper we evaluated whether the theory of the argumentative policy analysis could explain the proceedings in this case study. It offers a description of the process of the policy development that included the two workshops, as well as an analysis of the views of policy makers and target populations in order to assess whether the proposed policy measures fitted the perceived problems and underlying background theories.


Relevant documents were analysed and semi-structured interviews were held with the various stakeholders. These documents included correspondence, reports from the Board, internal memos on this subject, a report from Health Council on interferon-beta, reports from meetings, research proposals, and research reports. Interviews were held with three HCIB employees, two employees from the Ministry of Health, the organisers of the workshops, two neurologists, a patient, and a medical advisor from a health insurance company. Participants of the workshops were contacted and interviewed to reconstruct the proceedings of the workshops. MS patients were contacted via a Dutch MS patient organisation. Representatives of target populations were interviewed to assess their problem definitions and underlying argumentation.

Interviews were transcribed verbatim. A summary from the interview and a concept report were sent to respondents for verification and literature was used to check the findings from interviews or documents. In line with the theory of argumentative policy analysis, we used the method of reconstructing interpretative frames to analyse target populations’ argumentation [10, 11, 18]. The idea is to reconstruct (1) how problems are defined, (2) how solutions or policy measures are judged, (3) what theoretical and normative assumptions shaped them, and (4) what normative preferences underlie this all. Together, these four ‘layers’ of evaluation entail an individual’s interpretative frame.


Proceedings of Project

The content of the workshops was, to a large extent, determined by two reports concerning IFNβ. These had been issued by the HCIB and by the Health Council, respectively. In July, 1995, the Ministry of Health asked both institutions to provide advice regarding the introduction of IFNβ on the Dutch market. The Ministry asked for recommendations with respect to whether or not restrictions could be imposed on the reimbursement of the drug.

In May 1996, a committee from the Health Council reported to the Ministry of Health [9]. This committee was comprised of three neurologists, a chemical technologist, and a HCIB staff member. It concluded that IFNβ could be a promising new drug, but also cautioned against unrealistic expectations. Clinical trials had shown that IFNβ could decrease the rate and severity of exacerbations, but there was no evidence of IFNβ preventing the onset of disability. The Health Council emphasised that the drug should be prescribed only to patients who meet eligibility criteria for the trials. These criteria were: (a) clinically definite Relapsing Remitting MS; (b) at least two exacerbations in the two previous years; (c) mild to moderate disability; and (d) age of 18 years or older. The Council also proposed the following policy measures: (1) develop a guideline for treatment of MS patients with IFNβ; (2) ensure that the drug is prescribed by neurologists who have sufficient experience with the diagnosis and treatment of MS patients; (3) properly instruct and guide MS patients; (4) conduct a systematic follow up of patients using IFNβ in order to evaluate side effects; (5) provide clear indications on which patients should be treated with IFNβ; (6) establish a national registry of MS patients in order to conduct further research on the effectiveness of IFNβ; and (7) conduct a re-appraisal after a number of years.

June 1996, the HCIB issued recommendations that corresponded partially with the Health Council’s report [30]. The HCIB recommended that: (a) patients fulfil the criteria described in the Health Council’s report; (b) a treatment protocol be developed; (c) health insurers approve reimbursement requests before starting treatment; (d) a prospective registration be established; and (e) the use of IFNβ be re-assessed after 3 years. Restricting prescription to a limited number of experienced neurologists was considered impossible, because IFNβ had already been included in health care packages.

In 1999, the department of Policy Analysis of Medicines (PAM) started a project on IFNβ. Initially it they considered evaluating other, less expensive interventions that could be effective, but are of no interest to the industry. Indications had been received from the field that a much cheaper drug, namely methotrexate (used in oncology and rheumatoid arthritis), could be as effective as interferon. Furthermore, PAM considered developing a national database of MS patients. PAM later decided that conducting a clinical trial was not its responsibility. PAM staff, in collaboration with neurologists, decided to initiate the development of a treatment protocol and a national registry of MS patients. They proposed the organisation of two workshops.


In 2000, these workshops were organised by an external institute for policy research. The first workshop aimed to provide an overview of the criteria that are used for prescribing IFNβ. This workshop was preceded by an inventory study amongst health insurance companies into current reimbursement practices with respect to IFNβ. This inventory study was performed by a HCIB department. The results from the inventory study on current reimbursement practices indicated that health insurance companies did not assess reimbursement requests against medical content. Assessments were limited to an administrative review of the completeness of data [4]. Nevertheless, health insurance companies considered the pre-utilisation approval effective, because it functioned as an administrative barrier. The authors of this report recommended the following: (a) assess whether a pre-utilisation approval by a central committee could improve the assessment of reimbursement requests; (b) develop a clear protocol and design application forms.

Neurologists, medical advisors from health insurance companies, policy makers, and a representative from a patient organisation participated in the workshop. During the workshop, vignettes with patient descriptions were used to discuss which patients should be treated with IFNβ. The organisers concluded that only a few problems were perceived with respect to criteria for IFNβ use. Health insurers questioned whether the situation was indeed problematic given the small size of the patient population and IFNβ’s status as an essential medicine. Neurologists argued that problems arose incidentally. Most often, these problems were related to reimbursement requests for continued use of IFNβ. All participants agreed that subjectivity in decisions on prescription should be minimised. The participants claimed that clear distinctions should be made between the criteria for initialising treatment and the criteria for continuing treatment. The development of a guideline by neurologists was considered relevant. According to the participants, small adjustments of the current criteria would be sufficient [2].

The second workshop aimed to obtain advice on how existing databases could be improved so that the effects of IFNβ or other new drugs for MS patients could be evaluated. In preparation for this workshop, a neurologist was asked to make an inventory of available databases that contained data on IFNβ users. This inventory showed that data on MS patients were no longer being collected systematically in the Netherlands [2]. Former local databases were no longer up-to-date. The participants in the workshops agreed that a national database could be relevant and could serve the following goals: (a) policy making (such as financial surveys, planning health care capacity); (b) research (effects of treatment; monitoring for side effects); and (c) clinical practice support for neurologists (reflection on clinical practice, improvement of expertise). Participants agreed on the inclusion of general static data in the database, such as demographic details, diagnosis (type MS), the criteria used to make the diagnosis, and the initial treatment. Participants disagreed on the kinds of dynamic data, such as treatment details and information on physical functioning. A database including a broad range of data could be useful for research, but this was considered expensive. A small database that includes static data only could be used as a sample frame for selecting potential participants in further studies. In their report, the organisers concluded that additional research was needed to ascertain which variables should be included in the database.

Proposed Policy Measures

Based on the organisers’ report, the HCIB proposed the following policy measures to the Ministry of Health [5]:
  • Have neurologists develop an evidence-based guideline for the treatment of patients with multiple sclerosis.

  • Have reimbursement requests for IFNβ appraised on medical grounds. Possibly, implement a central pre-utilisation approval by experts (neurologists).

  • Define clear criteria for (dis)continuing IFNβ reimbursement.

  • Establish a national registry of MS patients. The database should enable scientific research, the improvement of treatments, monitoring, and the evaluation of efficiency and therapeutic value of new interventions.

In 2003, a medical advisor from a health insurance company mentioned that some small changes had been made to the procedures established for judging reimbursement requests (requests for continued use). A neurologist mentioned working on a guideline for diagnosis and treatment of MS patients. According to a PAM staff member, neither a central pre-utilisation approval nor a national database to prospectively register patient data had been established.

Reconstructed Interpretative Frames

In June 2003, we conducted interviews to reconstruct interpretative frames of policy makers and policy’s target populations. The actors’ views are summarised in Table 1 (IFNβ guideline) and Table 2 (national database).
Table 1

Views towards guideline concerning IFNβ use


Judgement of solutiona

Problem definition

Background theories

Normative values

Ministry of health (1996)


Clear criteria for reimbursement IFNβ needed

IFNβ is an expensive medicine; uncertainty on meaning of effects

Possibly inappropriate use because high expectations

Only if evidence of effectiveness is available, a medicine can be at the expense of the community

Cost control


PAM staff


A central pre-utilisation approval for IFNβ use

A clear treatment protocol on IFNβ should be available

Health insurance companies judged reimbursement requests differently

Individual health insurer lack expertise to judge reimbursement requests

Differences in judgement result in differences in care

Situation IFNβ resembles situation growth hormones, for which a central pre-utilisation approval in combination with central registration was set up

Central pre-utilisation approval prevents inappropriate use

Cost control; efficiency

Equal accessibility

PAM staff (2003)

Central pre-utilisation approval no longer needed

Situation concerning IFNβ was relatively stable; no longer problems concerning reimbursement requests

Costs committee higher than possible savings IFNβ use

Criteria for central pre-utilisation approval:

– medicine is expensive

– inappropriate use expected

– decisions on treatment are complex

Cost control


Medical advisor health insurance company


Guidelines on IFNβ needed for decisions reimbursement

Unclear when IFNβ is (still) meaningful

Pharmaceutical companies benefit from extending criteria

Expensive drugs are financed from public funds, on a solidarity base

Cost control



Not guideline on IFNβ solely but guideline on diagnosis & treatment MS

Increase possibilities for diagnosis and therapy; variance in MS care

As in other fields increasing attention to evidence based medicine

Provide clarity for neurologists on how to care patients



National guideline might be useful

Differences between hospitals care (IFNβ not offered in every hospital; contradictory advises on lifestyle)

No common knowledge on MS

Prognosis differs between individuals

Improve or maintain quality of life

a‘+’ actor defines the intervention meaningful; ‘−’ intervention is not meaningful

Table 2

Views towards a national registry of MS patients or IFNβ users


Judgement of solutiona

Problem definition

Background theories

Normative values

Ministry of health (1996)


For evaluation long-term effects

Uncertainty on (meaning of) effects of IFNβ

Only medicines of which evidence of effectiveness is available at the expense of the community

If no evidence on effectiveness, use is inappropriate


PAM Staff


Database with only limited patient data to take samples for research


Database with all relevant data for research

For each new medicine a study on effectiveness is inefficient

Health insurance companies judged reimbursement requests differently, leading to difference in health care

Aim of national database to optimise IFNβ prescription: to monitor use of IFNβ in practice (which patients (indications), volume, costs) and to evaluate effectiveness IFNβ


PAM Staff (2003)

Registration without central judgement committee difficult

Central committee no longer relevant

No problems perceived

Central pre-utilisation judgement provides a minimal registration of patient data


Medical advisor of health insurance company


Prospective database only way to assess long-term effectiveness

Judgement of relative value as compared to alternative interventions not systematically but only in the neurologists head

Expensive drugs are financed from public funds

Pharmaceutical industry’s power is underestimated

Clear vision from the ministry of health on pharmacy

Quality of care



Meaningful for evaluating effectiveness of medicines and knowledge on MS

National coverage might be difficult

Clinical trials have limited follow up period

Distrust amongst neurologists

MS is a chronic disease, invalidating on long term

MS neurologists have their own “kingdoms”

Quality of care

Increase knowledge on disease and treatment

a‘+’ actor defines intervention as meaningful; ‘−’ intervention is not meaningful

Ministry of Health

Initially, the Ministry contended that the long-term effectiveness of the drug was not established sufficiently, while the acquisition costs were high. Trials had shown that IFNβ decreased the number of exacerbations, but it was unknown whether IFNβ would prevent disability. The potential target population, namely patients with MS, is large and the expectations of both patients and physicians may also have been high. As a result, the risk that the drug could be used inaccurately was considered to be high. Consequently, additional policy measures to control IFNβ use in clinical practice were requested.

According to employees from the Ministry of Health, the HCIB report [5] had revealed that only health insurers perceived few problems with respect to judging reimbursement requests. The costs of a committee for a central pre-utilisation approval were considered relatively high, while the costs of the interferon-beta prescription were less than expected, namely €18 million (in 1999) versus the €90-180 million estimated by the Ministry. Therefore, both interventions were considered to be no longer relevant.

PAM Staff

According to the PAM staff, the main problem was that the use of IFNβ was expected to extend to other subgroups of patients. A central pre-utilisation approval in combination with a national registry of MS patients could provide a solution. This contention was made because the situation concerning IFNβ resembled the situation concerning other drugs for which a central pre-utilisation approval had already been established. In that case, requests for reimbursement of drug use were judged by a central committee of medical experts. For this purpose, a clear protocol including criteria for IFNβ use is needed. Simultaneously, patient data were recorded in a national registry.

At the time that PAM staff started their project, it was undesirable to evaluate why the proposed policy measures had not been implemented until then. Reason was a change in the relationship between the HCIB and the Ministry of Health. In 1999, the HCIB, an independent advisory board, was established as the successor to the Sickness Funds, which was a politically involved advisory board comprised of actors from the field.

After completion of the project, the PAM staff considered the option of having reimbursement requests judged by a central committee no longer meaningful. They considered the implementation of policy measures difficult because the drug had already been introduced several years earlier. Limiting or discontinuing the reimbursement of IFNβ would have been practically impossible. Furthermore, the scale of problems relating to the prescription and/or reimbursement of IFNβ and the costs related to IFNβ use turned out to be much smaller than expected. PAM considered the initiation of a national MS patient registry without the central pre-utilisation judgment to be unfeasible.

Medical Advisor Health Insurance Company

According to a medical advisor from a health insurance company, the problem was attributable to a lack of clarity on exactly what kind of patients benefit from IFNβ. IFNβ is an expensive drug and the costs are either carried by the community (public health insurance) or reimbursed on an individual basis (private health insurance). Preferably, physicians should develop a guideline on the relative position of IFNβ in relation to alternative interventions. At the time of IFNβ’s introduction, a central pre-utilisation approval could have been relevant. However, time had passed and the prescription of IFNβ had become common practice. Unfortunately, new drugs are often introduced and included in the health care package long before all medical specialists agree on criteria for treatment. Professionals cannot develop guidelines quickly. In contrast, the time that passes between a drug being introduced into the market and its inclusion in the health care package is regulated by law and, in most cases, quite short. As a temporary solution, the drug could be included into the health care package under certain conditions, such as the registration of patient data.


A neurologist mentioned working on a guideline for diagnosing and treating MS patients as part of an initiative from the Dutch Society for Neurology. A guideline for diagnosis and treatment was considered relevant as it could support evidence-based practice. As in other medical fields, knowledge and treatment options have increased. The development of a guideline is a time-consuming endeavour as, often, it has to be done alongside the professional’s usual activities. A central pre-utilisation approval was no longer relevant. Prescribing IFNβ was considered common practice, also in small general hospitals. Most neurologists were already familiar with the indications for treatment. IFNβ was proven to be effective in one type of MS that is characterised by invalidating exacerbations (relapsing remitting MS) through clinical trials that demonstrated IFNβ’s ability to decrease the severity and frequency of these exacerbations. A national database could still be useful. However, its purpose must be clear. An appropriate goal could be to evaluate the long-term effectiveness of IFNβ. The neurologist questioned whether participants would have come to agreement on specific recommendations, such as which data should be collected and who should become the owner of the database. Most neurologists do have “their own kingdoms” and do not want to share these with others.


The patient claimed that agreement amongst neurologists about MS treatment could be useful. For him, the main problem was that MS care differs between hospitals. He conveyed his experiences with receiving contradictory answers to questions from numerous health professionals and the option of using IFNβ had not been discussed in the hospital where he was initially treated. For this patient, the exacerbations are highly invalidating. Obviously, his preference is to lead a normal life, in so far as that is possible. MS has a high impact on his life.


Although results from the workshops showed that all respondents agreed that developing a guideline and a national database could be meaningful, these policy measures have never been implemented. A number of factors can provide an explanation for the proceedings in this project.

Firstly, the costs of IFNβ appeared to be lower than expected. Perhaps the information campaign on IFNβ had had this effect [3]. It may also have been that the calculation by the Ministry of Health, which differed from calculations made by the Health Council and rested on the assumption that all MS patients are treated with IFNβ, were unrealistic. In any case, the subject of IFNβ no longer had high priority. From the point of view of policy makers, the expected high costs of IFNβ in combination with its uncertain effectiveness were the main problems behind the proposed policy measures. Apparently, however, the problem of the high costs had still higher priority than the limited knowledge on IFNβ’s effectiveness. As a result, policy makers left it to the medical profession to implement policy measures. Neurologists, however, considered cost-containment not their problem and not their responsibility.

This also explains why the initial questions about the long-term effectiveness remained unanswered, without this being perceived as a problem. The effectiveness of IFNβ and the possible role of the industry was an issue of international debate. Clinical trials have shown that IFNβ decreased the number and severity of exacerbations in patients with relapsing-remitting MS [16, 23]. Probably, this is enough reason for neurologists to claim the right to prescribe IFNβ. That evidence of long-term effectiveness, in terms of preventing disability, and safety was still limited [26] is not of great concern to them. Neither is the possible influence of the industry on prescription practices through the financing of clinical trials that had been discussed [13, 21, 22, 28]. Some authors have even challenged the hypothesis of MS as an inflammatory auto-immune disease [1]. A national registry could have been relevant from this point of view. In the UK, the Department of Health initially refused to reimburse IFNβ use for reasons of costs [17]. In 2002, however, agreements were made with the industry concerning the funding of IFNβ. The Department of Health announced that it would reimburse IFNβ for MS patients who agreed to participate in a monitoring program on the effectiveness of IFNβ.

Secondly, the policy development process from the beginning centred around two specific policy measures, whereas it could have started with a broader scope, actors involved first eliciting the problems they perceive and adequately structuring the problem. Now, the proposed measures appeared not to be the most optimal solutions for the problems as perceived by neurologists, and therefore represented right solutions to the wrong problems. This typically concerns what Hischemuller and Dunn have coined an ‘error of the third kind’ [6, 14]. We have found that on a first order level, all actors agreed that a national guideline and a registration could be meaningful. But as it was, on a second order level they disagreed as to what goals such policy measures should serve. For example, neurologists considered a national guideline relevant, as long it was a broad guideline on the diagnosis and treatment of MS in general. All actors agreed that a national registry could be relevant, but defined different goals. Each goal came with different variables to be included in the registry.

Thirdly, during the workshops, the proposed policy measures remained rather vague and were not elaborated in detail. As a result, participants could easily consent without violating their background theories and preferences. They had not felt the urge to (re)consider these theories and preferences. This, however, was necessary, as is clearly illustrated by the respondents who questioned whether participants would have come to agreement on specific recommendations, such as which data should be collected and who should become the owner of the registry. Such an agreement requires that policy measures are made sufficiently substantive for every actor to understand what are the consequences. Moreover, it requires that background theories and normative preferences are elicited and scrutinised.

Fourthly, changes in the socio-political context affected policy development. Opinions with respect to which kind of policy measures are the responsibility of the HCIB had changed over time. The PAM staff considered conducting a clinical trial to obtain relevant information that would not be provided by the industry. Simultaneously, their position towards another Dutch institute involved with subsidising clinical research changed. Putting out clinical trials was then no longer their responsibility. Furthermore, the relationship between the HCIB and the Ministry of Health changed over the course of this project. These changes strongly affected ideas with respect to the Board’s responsibility on initiating a national registry of patient data. On this basis we conclude that the workshops failed to meet the objectives set for interactive processes in policy development, namely to prevent policy implementation problems as a result of diverging views amongst target groups. Moreover, we concede that they could not have met those objectives. From the case of IFNβ, we infer that an interactive process should meet the following criteria:
  1. 1.

    The interaction should cover the whole process from problem structuring to policy implementation, in order to be able to deal with problem shifts and changes in the socio-political context. It is not enough to reduce interactivity to workshops at one or two moments in time.

  2. 2.

    Actors involved should resist the temptation to think that policy problems can be understood at a first order level. That is, background theories and normative preferences should be explicated in the problem definition phase. Only if problem structuring is taken seriously in this sense can one think of developing solutions that could meet with the approval of all target groups.

  3. 3.

    Developing widely endorsed solutions requires that actors involved are willing to learn from one another and adapt their views if necessary. A process of interactive policy development should include room for such learning processes [12, 24, 25, 29]. Grin and Van de Graaf [10] have argued that learning is likely to occur only if external events urge a revision of background theories and preferences, or if repeated failures show actors that their background theories are not functional.

  4. 4.

    Proposed policy measures should be sufficiently elaborated, as to enable target groups to assess their consequences and constructively engage in the interactive process. As the case of IFNβ shows, actors involved are not willing to reconsider their background theories and preferences, should proposed policy measures not be sufficiently elaborated and actors involved not understand what is at stake.


In sum, this study has shown that the organisation of workshops in which target populations participate does not qualify as an appropriate process of interactive policy development. From the beginning, emphasis had been put on a limited number of interventions aimed to control the expected increase in treatment costs. Although target populations participated in policy development, perceived problems and which interventions could provide a solution had been discussed insufficiently. For policy development to be successful, interactive methods are needed, in which problem definitions and assumptions are explicated and discussed, providing an opportunity for mutual learning between actors involved.



This project was initiated and funded by the department of Policy Analysis on Medicines at the Dutch Health Care Insurance Board.


  1. 1.
    Behan, P. O., Chaudhuri, A., & Roep, B. O. (2002). Pathogenesis of multiple sclerosis revisited. Journal of the Royal College of Physicians of Edinburgh, 32, 244-265.Google Scholar
  2. 2.
    B&A. (2000). Gebruik van interferon-beta bij multipele sclerose. Den Haag: B&A groep.Google Scholar
  3. 3.
    Coolen, R. (1997). Patientenvoorlichting interferon beta 1b beinvloedt keuzeproces. Pharmaceutisch Weekblad, 132, 1394-1396.Google Scholar
  4. 4.
    CVZ. (2000a). Onderzoek handelswijze ziekenfondsen bij aanvragen interferon beta bij multipele sclerose. Amstelveen: College voor zorgverzekeringen.Google Scholar
  5. 5.
    CVZ. (2000b). Evaluatie van het gebruik van interferon-beta bij multipele sclerose. Amstelveen: College voor zorgverzekeringen.Google Scholar
  6. 6.
    Dunn, W. (2004). Public policy analysis. An introduction (3rd ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.Google Scholar
  7. 7.
    Fischer, F. (1999). Evaluating public policy. Chicago, USA: Nelson-Hall Publishers.Google Scholar
  8. 8.
    Fischer, F., & Forester, J. (1993). The argumentative turn in policy analysis and planning. Durham, NL: Duke University Press.Google Scholar
  9. 9.
    Gezondheidsraad. (1996). Toepassing van interferon-beta-1b bij patiënten met multiple sclerose. Den Haag: Gezondheidsraad. Publicatie 1996/05.Google Scholar
  10. 10.
    Grin, J., & van de Graaf, H. (1996). Implementation as communicative action. Policy Sciences, 29, 291-319.CrossRefGoogle Scholar
  11. 11.
    Grin, J., van de Graaf, H., & Hoppe, R. (1997). Interactieve technology assessment. Een eerste gids voor wie het wagen wil. Den Haag: Rathenau Instituut; W57.Google Scholar
  12. 12.
    Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park: Sage Publications.Google Scholar
  13. 13.
    de Haan, R. J., & Vermeulen, M. (1999). Interferon beta-Ib nu ook voor de secundair progressieve vorm van multiple sclerose? Nederlands Tijdschrift voor Geneeskunde, 143(14), 709-711.PubMedGoogle Scholar
  14. 14.
    Hisschemöller, M. (1993). De democratie van problemen. De relatie tussen de inhoud van beleidsproblemen en methoden van politieke besluitvorming. Amsterdam: VU Uitgeverij.Google Scholar
  15. 15.
    Hoppe, R., & Peterse, A. (1998). Bouwstenen voor een argumentatieve beleidsanalyse. Den Haag: VUGA.Google Scholar
  16. 16.
    IFN MS Group. (1993). Interferon beta-1b is effective in relapsing remitting multiple sclerosis. Clinical results of a multicenter, randomized double-blind, placebo-controlled trial. Neurology, 43, 655-661.Google Scholar
  17. 17.
    Lie, R. K. (2004). Research ethics and evidence based medicine. Journal of Medical Ethics, 30, 122-125.PubMedCrossRefGoogle Scholar
  18. 18.
    Moret, M., Reuzel, R. P. B., van der Wilt, G. J., & Grin, J. (2007). Validity and reliability of qualitative data-analysis: Reconstructing interpretative frames. Field Methods, 19, 24–39.CrossRefGoogle Scholar
  19. 19.
    Netherlands Court of Audit. (1992). Cost containing medicines. 22920, no 2 (in Dutch).Google Scholar
  20. 20.
    Netherlands Court of Audit. (2003). Between policy and implementation. 28831, no 1-2 (in Dutch).Google Scholar
  21. 21.
    Pieters, T. (1998). Marketing medicines through randomised controlled trials: The case of interferon. BMJ, 317(7167), 1231-1233.PubMedGoogle Scholar
  22. 22.
    Polman, C. H. (1999). Interferon beta-1b nu ook voor de secundair progressieve vorm van multiple sclerosis? Nederlands Tijdschrift voor Geneeskunde, 143, 815-817.PubMedGoogle Scholar
  23. 23.
    PRIMS. (1998). Randomised double-blind placebo-controlled study of interferon beta-1a in relapsing/remitting multiple sclerosis. Lancet, 352, 1498-1504.CrossRefGoogle Scholar
  24. 24.
    Reuzel, R. P. B. (2001). Health technology assessment and interactive evaluation: Different perspectives. Enschede: PrintPartners Ipskamp.Google Scholar
  25. 25.
    Reuzel, R. (2004). Interactive technology assessment of paediatric cochlear implantation. Poiesis & Praxis, 2, 119-137.Google Scholar
  26. 26.
    Rice, G. PA., Incorvaia, B., Munari, L., Ebers, G., Polman, C., D'Amico, R., & Filippini, G. (2002). Interferon-beta in relapsing-remitting multiple sclerosis (Cochrane review). In: The cochrane library, issue 1. Oxford: update software.Google Scholar
  27. 27.
    Schön, D. A. (1983). The reflective practitioner. How professionals think in action. New York, USA: Basic Books.Google Scholar
  28. 28.
    Vermeulen, M., & de Haan, R. J. (1999). Nieuwe therapieen in de neurologie, maar wie wordt er beter van? Nederlands Tijdschrift voor Geneeskunde, 143(35), 1764-1766.PubMedGoogle Scholar
  29. 29.
    Yanow, D. (2000). Conducting interpretative policy analysis. Thousand Oaks: Sage.Google Scholar
  30. 30.
    Ziekenfondsraad. (1996). Vervolg-rapport aanspraak interferon-beta-1b. nr.717. Amstelveen: Ziekenfondsraad.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Margriet Moret-Hartman
    • 1
    • 2
  • Rob Reuzel
    • 1
  • John Grin
    • 3
  • Gert Jan van der Wilt
    • 1
  1. 1.Department of Medical Technology AssessmentNijmegen University Medical CentreNijmegenThe Netherlands
  2. 2.HeelsumThe Netherlands
  3. 3.Department of PoliticsUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations