Introduction
A common criticism of the pharmaceutical industry is that it fails to achieve transparency
[1, 2]. To allay concerns, the industry needs to collaborate openly with stakeholders
to make medicines widely available and affordable—this will mean companies need to
address some of their practices [1, 3, 4]. We suggest that an important practice to
address is the presentation of health economic models.
Formal health economic models are commonly used to support decision makers when deciding
whether funding particular pharmaceuticals will improve the health of a population
[5, 6]. The value of a formal model lies not only in its results but also in the revelation
of the assumptions about both the data and the logical relationships it embodies [6].
Presented properly, the model provides clarity on key issues.
The value of adopting the correct modelling methodology has been recognised by national
and international bodies who have issued guidance for developing models; for example,
the reports generated from the recent US Panel on Cost-Effectiveness in Health and
Medicine [7] and the collaboration between the International Society for Pharmacoeconomics
and Outcomes Research and the Society of Medical-Decision Making (ISPOR-SMDM) [8,
9]. Similarly, the European Network for Health Technology Assessment (EUnetHTA) is
also working towards international level collaboration, providing a core health technology
assessment (HTA) model framework and a guide to best practices [10]. Multiple guidelines
can produce some inconsistencies [5, 10–14] but they have helped produce some clarity
in dealing with uncertainty and validating and reporting models transparently [9].
Despite these welcome developments, individual models developed in the same therapeutic
area may be significantly different from one another. Occasionally, efforts have been
made to standardise them [15]. However, standardisation is not the norm and, in our
experience, a model code is rarely shared.
It is not just pharmaceutical companies that have been challenged regarding the level
of information shared. For example, the UK National Institute of Health and Care Excellence
and the US Institute for Clinical and Economic Review are involved in debates around
the extent their health economic models should be made publicly available [16, 17].
To facilitate this transparency, there is suggestion in the literature that open source
modelling is desirable [8, 9]. The definition of open source can vary by discipline
[18–21]. An open source health economic model is defined here as one that is available
to those who wish to access it. This means that the model, its underlying code and
a written report describing its aim, methods, structure and results would all be accessible
[8]. This could be in a fully public space (e.g. freely downloadable on the Internet)
or only readily available to those who request it with conditions attached to access.
An open source culture allows existing models to be updated to answer new research
questions and decision problems and creates a transparent public arena for model validation,
education and collaboration across research, industry and healthcare communities [4,
8, 9]. There are few examples of published fully open source models; Sullivan et al.
is a recent example in pain therapy where the code was made available in both Excel
and R [22].
The objective of this letter is to highlight the paucity of examples of open source
models in the literature and to report on a short piece of research undertaken to
gain insight into the appetite of researchers to provide and use open source models.
Exploratory Expert Survey
An exploratory double-blind survey of health economic stakeholders was undertaken
between 25 April, 2016 and 6 May, 2016 to assess whether there is any desire for open
source health economic models. To gain access to a wide variety of people with experience
and an active interest in health economics, the LinkedIn platform was used to access
the relevant groups: ISPOR; HTA in Europe (maintained by EUnetHTA); and Institute
for Medical Technology Assessment.
Eight out of the ten survey questions were quantitative, while two were qualitative,
with the opportunity for the respondents to populate open text boxes. Within the qualitative
responses, a framework analysis was carried out to identify themes based on the respondent’s
answers.
Despite the short time the survey was open, 46 participants responded, and 35 completed
the whole survey. Respondents worked predominantly in the pharmaceutical industry
(35%), academia (33%) and government (20%), and the remaining worked in other fields
such as independent research, non-government organisations, community pharmacy and
hospitals. Among the respondents, 33/41 who answered (81%) had experience with one
or more health economic models in their careers. Among the participants, 18/41 (44%)
had both requested information and received a request for information about a model
that was not publicly available. There were a number of reasons that the participants
needed the information; Fig. 1 shows the most frequent (24/41; 59%) was learning about
the technical aspects of the model for use in a different disease or decision problem.
Across all stakeholders and countries, making health economic models available in
an open format was considered beneficial; 34/35 (97%) of participants said that it
would be occasionally or very beneficial to have access to the code when reviewing
existing models.
Fig. 1
Reasons for needing access to health economic models (41 responded, who could choose
up to six answers that applied to them)
One of the survey questions focused on if respondents ever experienced challenges
in accessing health economic models. Among the respondents, 26/41 (63%) had experienced
challenges in accessing full details of a health economic model. Challenges included
ownership and confidentiality issues, such as a lack of willingness to share and not
knowing who to contact or being unable to contact someone to request information on
a health economic model. Two qualitative questions were asked: what was their perceived
value of improving access to health economic models and what were the best strategies
to enable sharing. Figure 2 highlights the most common key themes mentioned by respondents
relating to the perceived benefits.
Fig. 2
Frequently mentioned benefits of open source health economic models
The best strategies to encourage the sharing of health economic models as suggested
by the respondents included:
Collaboration: different stakeholders working together to develop open platforms or
libraries to encourage sharing and to maximise the social value of their research;
examples were journals, HTA agencies or consultants actively involved in publishing
and sharing their health economic models and promoting a sense of sharing and value
of open source.
Transparency: such as authors providing supplementary materials and highlighting relevant
sources where information was gathered.
Confidentiality: finding ways to share models while keeping the necessary proprietary
aspects confidential.
Changes to regulatory processes: potentially integrated open access processes across
organisations and formal request processes by health authorities.
Consistent use of language: making models easier to understand across stakeholders
and countries.
Discussion
The limited results of this exploratory analysis supported viewpoints found within
the available guidelines and limited literature. Across all stakeholders and countries
in the survey, making health economic models available in an open format is expected
to be beneficial within industry, academia, government, non-government organisations
or hospitals. This is consistent with arguments made by Fazio et al. that transparency
with models are desired by various stakeholders [23]. The ISPOR-SMDM guidelines (specifically
the ISPOR guidelines on model transparency and validation [8]) and the EUnetHTA joint
action projects support the views of respondents on collaboration, transparency, confidentiality,
processes and consistency, while the literature explores how best to improve this
[4, 8–10]. It is also consistent with a proposal by Afzali et al. that there should
be “reference models” that are built by an expert group for a particular therapy area
and then made available to pharmaceutical companies for drug submissions [24]. However,
we caution that any reference model approach would need to allow stakeholders the
ability to debate, challenge and change the model as appropriate.
There were inconsistent qualitative responses from some participants who wanted tighter
confidentiality yet more open access. This could be partially owing to the often high-profile
and emotional nature of healthcare allocation decisions. This means policy makers
and other stakeholders may not want to share all information in fear of it complicating
the debate or hindering current or future decisions, as well as an organisation’s
specific needs for not wanting to be fully transparent. The ISPOR-SMDM guidelines
address some of these issues. They outline that there is potentially a conflict between
the scientific desirability of making all methodological and technical details of
a model available to peer reviewers and to other researchers and the need to protect
intellectual property generated by substantial investments in the development of a
model [8]. Rejecting the latter would significantly reduce the incentive to create
models, and intellectual property rights cannot be ignored. One possible solution
is to make a full technical documentation available within whatever agreements right
holders feel are necessary to grant them adequate protection [8]. This should allow
for detailed review of any model by other researchers, provided they accept the confidentiality
restrictions [8]. We suggest that this is often the most practical solution in the
short term. Despite challenges, there are already examples of open source models that
are fully available in the public space [22].
Conclusion
The study reported here was small and lacks generalisability. However, it did demonstrate
that among the respondents there is an appetite for greater use of open source models
and the topic is deserving of further exploration. Future investigation could be particularly
beneficial if it also focuses on the obstacles to transparency and how they may be
overcome.