The general issue of balancing the value of evidence about the performance of a technology and the value of access to a technology can be seen as central to a number of policy questions. Establishing the key principles of what assessments are needed, as well as how they should be made, will enable them to be addressed in an explicit and transparent manner. The aims of this research are to (1) establish the key principles of what assessments are needed to inform an 'only in research' (OIR) or 'approval with research' (AWR) recommendation, (2) evaluate previous National Institute for Health and Clinical Evidence (NICE) guidance in which OIR or AWR recommendations were made or considered and (3) evaluate a range of alternative options to establish criteria, additional information and/or analysis that could be made available to inform the assessments needed. All NICE draft and final guidance up to January 2010 was considered in the review of NICE technology appraisal guidance. Four case studies were used to evaluate the range of options of what information and analysis could be made available to inform the assessment required. These were based on a reanalysis of existing health technology appraisals for NICE or the Health Technology Assessment programme. A critical review of policies, practice and literature was undertaken using traditional systematic searching based on initial search terms informed by key publications. An iterative approach was adopted using 'pearl growing' evaluated through capture-recapture methods. In addition, grey literature, policy documents and other sources, such as special interest groups and the expertise of the Advisory Group for the project, were used to contribute to this process. A series of recommendations, or options, for NICE to consider were developed with the involvement of key stakeholders. These establish the key principles and associated criteria that might guide OIR and AWR recommendations and identify what, if any, additional information or analysis might be included in the technology appraisal process, including how such recommendations might be more likely to be implemented through publically funded and sponsored research. To meet these aims the research is broadly structured as follows. A critical review of policy, practice and literature in this area informs the development of a coherent conceptual framework to establish the key principles and the sequence of assessment and judgements required. This sequence of assessment and judgement is represented as an algorithm, which can also be summarised as a simple set of explicit criteria or a 7-point checklist of assessments. A review of previous NICE guidance in which OIR or AWR recommendations were either made or considered was undertaken to examine the extent to which the key principles are evident. The application of the checklist of assessment to a series of four case studies informs considerations of whether or not such assessments can be made based on existing information and analysis in current NICE appraisal and in what circumstances could additional information and/or analysis be useful. Finally, some of the implications that this more explicit assessment of OIR and AWR might have for policy (e.g. NICE guidance and drug pricing), the process of appraisal (e.g. greater involvement of research commissioners) and methods of appraisal (e.g. should additional information, evidence and analysis be required) are drawn together. At each stage this research has been informed by a diverse and international Advisory Group and the feedback from participants at two workshops involving a wide range of key stakeholders, which included members of NICE and its Advisory Committees (including lay members and other NICE programmes), patient advocates, manufacturers, and research and NHS commissioners, as well as relevant academics. Further research is required to establish how these considerations could be integrated within a practical value-based pricing scheme. In addition, irrecoverable opportunity costs are commonly associated with many health technologies that offer future benefits following treatment. The significance of these types of irrecoverable costs is not widely recognised and further research to demonstrate their potential impact more generally is needed. The categories of guidance available to NICE have a wider application than is reflected in the review of previous guidance. Importantly, determining which category of guidance will be appropriate depends only partly on an assessment of expected cost-effectiveness. As well as AWR for technologies expected to be cost-effective and OIR for those not expected to be cost-effective, there are other important circumstances when OIR should be considered. In particular, for technologies expected to be cost-effective, OIR rather than approve may be appropriate when research is not possible with approval and OIR or even reject, rather than AWR or approve, may be appropriate even if research is possible with approval when there are significant irrecoverable costs. The National Institute for Health Research Health Technology Assessment programme.
