Does the type of funding influence research results – and do researchers influence funders?

Does a shift from hard to soft funding have an impact on research outcomes? Existing literature suggests that moving from hard money, such as lump-sum government-funded research, to commissioned research entails a greater risk of the researchers being influenced by the principal (the funding body). Based on literature and an empirical study, we identify two types of researcher roles: the influential consultant and the technical realist. The first type studies more advanced, important, and diffuse topics on behalf of principals high up in the hierarchy. They have a much greater experience of the issues discussed in this paper than the technical consultants. During the course of this study, we also discovered that the balance of power is not necessarily as one-sided as theory suggests: researchers can wield significant influence over the principals as well.


Introduction
The philosophy of science is about as useful to scientists as ornithology is to birds.
Attributed to Richard Feynman (see Trubody, 2016) In this paper, we ask whether a transition in research funding from hard money to soft money influences research results. By hard money, or 'blue sky' money, we mean a steady stream of government funds, provided more or less as a lump sum via the government budget. By soft money, we mean other types of research grants, from long-term project funding to commissioned research, the latter being softer than the former. In Norway, this debate has gained a head of steam in recent years in both the media (see e.g. Hultgren and Moe, 2011;Rolness, 2016;Sellaeg, 2016;Aftenposten, 2016;Tellmann, 2016) and in the academic literature (see e.g. Ottesen et al., 2002;Andvig, 2008;Kjaernet, 2010;Fossheim and Ingierd, 2012;Tostensen, 2012;Tostensen and Kaiser, 2015).
The background for this paper was an initial curiosity about the relationship between funding and research results, based on personal experiences. Because research results influence public policy and hence people's lives, we thought it important to gain ethical, philosophical, and scientific perspectives on how research funding influences results. Until 1997, there was very little public debate related to commissioned research and research funding in Norway (Kaiser et al., 2003). Lately, several researchers have been critical of the increased share of commissioned research in Norway, both at universities (Østerud, 2016) and in institutes (Andvig, 2008;Kjaernet, 2010). Andvig (2008, p.17) writes about foreign aid, oil, and foreign policy as particularly sensitive areas. On foreign policy, he writes, 'research commissioned [by the Ministry of Foreign Affairs] sometimes becomes consultancy looking with partially controlled conclusions' (our translation).
Scientific methods aim to bring forward tenable statements regarding reality (Kaiser et al., 2003). However, such methods use abstractions, idealizations, and assumptions that work only in theory. When science becomes an ever more important adviser for external interests, often characterized by a great deal of complexity (e.g. within politics), problems become evident. When assumptions behind results are violated, science can only provide partial answers with a high degree of uncertainty. The fact that many hold a misguided belief that science can solve every problem only serves to amplify the difficulties (Sarewitz, 1996). Researchers are increasingly used as experts in consultancy work (see e.g. Feighery, 2011), so that research is to an increasing extent influencing high-stake issues. The combination of high uncertainty and high-stake issues is called 'post-normal science' by Funtowicz and Ravetz (1990, 1993 and by Ravetz and Funtowicz (1999). 1 The rest of this paper is organized as follows: first, we introduce our hypothesis. Second, we discuss some views of scientific ideology, scientific ethics, academic freedom and the roles of researchers and their stakeholders. The view a person holds on these topics can help explain their views on research funding. Third, we concentrate on possible challenges resulting from the changes in funding -the principal (the funding body) applying directions to research, hidden agenda, and self-censorship. Then, we discuss the method used for the empirical section before presenting the results from the study. Finally, we construct two typologies and summarize our findings.

The relationship between funding and research results
A state of things in which a large portion of the most active and inquiring intellects find it advisable to keep the genuine principles and grounds of their convictions within their own breasts, and attempt, in what they address to the public, to fit as much as they can of their own conclusions to premises which they have internally renounced, cannot send forth the open, fearless characters, and logical, consistent intellects who once adorned the thinking world. (Mill, 1901, p.60)

Hypothesis
The main question in this paper is whether the principal influences the results of research through the choice of funding. The hypothesis can be summarized as below and as in Figure 1. For the remainder of this section, we will elaborate on each of the components in Figure 1. The idea is that the principal, through the choice of funding type, has a direct influence on the results of a research project through: • applying directions to what is being researched, how research is done and how results are presented; • commissioning research in order to promote their opinions (hidden agenda); • enforcing self-censorship upon on the researcher, for example by not awarding new projects if the researcher arrives at unwanted conclusions. • In addition, the view of scientific ideology, ethics, academic freedom and roles are factors that can significantly affect ones assessment of these issues.

Scientific ideology
If one holds a particular scientific view, this can help explain their answers on whether a link between funding and results is problematic. While there are undoubtedly various definitions of scientific paradigms, our brief summary relies on Tranøy (1991) and on Kaiser, Rønning, Ruyter et al. (2003). In the tradition of the ancient Greeks, the ultimate goal of science is to acquire knowledge to enhance one's capabilities and realize oneself; i.e., science is a goal in itself (Tranøy, 1991, pp.74-6). In the 17th century, understanding nature's laws so that humans could control nature became a dominant view. Later, material welfare became more important and the scientist became a benefactor through inventions. Objectivity in science is perhaps best known through the ideas of Weber (1904Weber ( , 1922. In a Weberian context, science is value free and the scientist is politically, morally and religiously neutral. A view in strict opposition to the Weberian is that of John Bernal (1939), who proposes a carefully-planned and socially-beneficial view of research (Skoie andSåtvedt, 1998, cited in Kaiser, Rønning, Ruyter et al., 2003). Science is legitimized by the benefits it provides, autonomy is limited by governmental research policy, neutrality is replaced by a comprehensive ethical obligation concerning production and application, and science forges close alliances with various user groups (Kaiser, Rønning, Ruyter et al., 2003). The views are summarized in Table 1. Which views one subscribes to can help understand one's views on topics discussed below.

Scientific ethics
It is often claimed that the divide between basic and applied research is gradually being eroded (Gibbons, Limoges, Nowotny et al., 1994;Kaiser, Rønning, Ruyter et al., 2003). The increasing commercialization of universities has been discussed by many over the last decades (for example, Slaughter and Leslie, 1997;Etzkowitz et al., 1998;Marginson and Considine, 2000;Greenberg, 2001). Ziman (1996, cited in Kaiser, Rønning, Ruyter et al., 2003 discusses what he calls 'postacademic science'. While he acknowledges the positive sides of applied research, he is also concerned about the loss of scientific integrity. With conflicting interests between technological benefits   Weber (1904Weber ( , 1922, Baker (1939, cited in Rull, 2016 and Michael Polanyi (1940, 1951, 1958 Bernalism Usefulness to society Bernal (1939) and objectivity, the benefit aspects often win, while other (often negative) aspects remain unresolved. This one-sidedness, he argues, must be countered with reinforced research ethics. When scientific research is dependent on soft money from an external source, knowledge gaps can occur. Some areas of science can be left unfunded or incomplete because the principal is not interested in knowing the results, even though society might be interested -similar to the concept of 'undone science' (see Frickel, Gibbon, Howard et al., 2010;Hess, 2016). Krimsky (2013) summarizes research on whether a 'funding effect' exists in some research areas. In drug efficacy and safety studies, he finds correlations between the funding source and whether results are positive. While this is not definitive evidence of bias, Krimsky considers the correlations so large that it constitutes prima facie evidence of bias. For tobacco studies, he finds clear differences between research funded by the cigarette industry and research funded by non-profit and governmental organizations.

Academic freedom
A related discussion is that of 'academic freedom' (see, for example, Fuchs, 1963;Kirk, 1963). Though the term does not have a coherent definition, it has been discussed since at least the 1930s (McGucken, 1978) and has roots dating back at least to 1158 (Metzger, 1973, cited in Karran, 2009d, when the Holy Roman Emperor issued a proclamation protecting travelling scholars. Karran (2007Karran ( , 2009a compares several countries and gives Sweden, Denmark and Great Britain a low academic freedom score, while Finland, Slovenia, the Czech Republic, Hungary and Spain score well (Norway is not included in the studies). In a suggested ethics overhaul, Bryden and Mittenzwei (2013) suggest that researchers who cannot have their academic freedom guaranteed should not be allowed to publish in peer reviewed journals. One's view of academic freedom may very much depend upon one's view of scientific ideology. A Bernalist would emphasize a close link between society and the researcher. Academic freedom might therefore not be so important to the Bernalist, while a Weberian would take the opposite view.

Roles: researcher, principal and politician
The division of roles between the principal and the researcher can often seem unclear. For example, the principal can have reservations regarding the choice of theory or methods (Kjaernet, 2010;Tostensen, 2012;Richter and Hostettler, 2015). In the social sciences, the difference between what is 'reality' and what are the subjective judgements of the researcher makes it easier to discredit or downplay research conclusions (Tostensen, 2012). We can make a distinction between the role of the politician, choosing the goal, and the role of the researcher, choosing the means. In practice, this is complicated by differing views of roles. For example, in the positivist tradition of Ayer (1936), the analyst is independent and uses scientific methods to look for verifiable causal relationships. In the action research tradition of Lewin (1944Lewin ( , 1946 and Skjervheim (1957Skjervheim ( , 1976, the researcher is more of a social actor who cannot act independently of his or her research. Furthermore, a researcher can work within various paradigms. Gummesson (1991) discusses a scientific paradigm and a consultant paradigm. The two paradigms have a lot in common, albeit with vital differences. A researcher in the scientific paradigm aims to contribute to new knowledge, to publish in peer reviewed journals and to achieve promotion. On the other hand, satisfying the client and securing funding are central to the consultant paradigm.
The relationship between politicians and researchers is also significant, and of increasing importance as politicians attain more control of research funding. This problem was, for example, discussed in relation to the Vietnam War and US foreign policy and whether researchers made themselves available to the government (Chomsky, 1969(Chomsky, , 2002. A Weberian might emphasize this distinction strongly, whereas a Bernalist might not worry too much about government interference if this is believed to make science more useful to society.

Applying directions: the principal's means of influence
The principal can influence what is being researched (through direction of funding), how research is designed and conducted, and how the results are published, published and utilized (Kjaernet, 2010;Richter and Hostettler, 2015).

What is being researched
What is being researched is naturally dependent on where funding is available. Even though the interests of the principal and the researcher might be aligned from the outset, there is a danger that the principal over time has an interest in developing blind spots where no good research exists; for example, in political relations in countries with dubious political systems (Kjaernet, 2010). A good relationship with all stakeholders is suggested as a remedy for such blind spots developing (Richter and Hostettler, 2015). Securing sufficient flexibility in the research contract, while still answering the principal's questions, is obviously a difficult trade-off.

How research is conducted
Choice of methods should ideally be left to the researcher (Kjaernet, 2010). If it is not, the principal could, for example, specify a level of analysis which hides important results. Such an experience is reported by Richter and Hostettler (2015). They are told that their research reports are too long, that the empirical results contradict themselves and that, even if the researchers found that what was relevant changed during the course of the research project, they were not allowed to modify the project (for example, by switching from interviews to focus groups).

How results are presented and published
Principals would obviously like to utilize the results in a way that benefits themselves. They also often wish a speedy publishing process and are not interested in peer review or discussing the research with other researchers (Richter and Hostettler, 2015). If researchers lose touch with other researchers internationally, long-term research quality may suffer as a result. Again, a Weberian will likely voice the strongest opposition to such influence.

Hidden agenda
Principals can have a hidden agenda when they commission research. From their own experiences, Richter and Hostettler (2015, pp.496-8) write: [Behind commissioned research contracts] often lies a hidden agenda. [. . .] Project leaders picked up formulations in the report that showed weak points in their own work or that of collaborators. They insisted upon changing the wording of the report to obtain a more favourable version. [. . .] The project leaders [wanted to use the results] for legitimization only. In their view, an 'independent' voice to endorse the project to support its reputation and ensure funding in the future seemed politically correct.
We can distinguish among several causes of hidden agendas (Bjartveit and Roos, 2005, chapter 8): the principal can be in need of an alliance partner, saying 'this should be done'; the principal can be in need of a guardian, because principals understand their incompetence and need someone to cover for them; or the principal can need help to suppress feelings (for example, concerning dismissals). There does not always have to be vicious intent behind hidden agendas.

Self-censorship
To some researchers, it can seem risky to propose conclusions that oppose what the principal, more or less openly, wishes. This is a recognized phenomenon, for example, in pharmaceutical research (Rampton and Stauber, 2002). Self-censorship is also discussed by Ham (1999), Kayrooz and Preston (2002) and Kjaernet (2010). Self-censorship can result in restraint in publishing results; for example, because the researcher wishes to sustain good relations with important sources of funding (Kayrooz and Preston, 2002).

Method
Everything must be taken into account. If the fact will not fit the theory -let the theory go. (Agatha Christie, 1920) The empirical part of this paper is based on a series of ten interviews conducted with researchers and research managers at the Norwegian Defence Research Establishment (FFI). They represent various parts of a project organization with 700 employees, of which 500 are researchers. FFI is owned by the Ministry of Defence so that its director general answers to the board and the board answers to the Ministry of Defence. Figure 2 provides a graphic illustration of how FFI hard money grants in constant 2015 Norwegian kroner were reduced by one third in the period from 2000 until 2015, and how hard money as a share of total income was halved. 2 Hard money is provided as a lump sum annual grant which the institute can spend as they see fit. Individual researchers therefore cannot investigate any topic they like, but there is still a considerable amount of autonomy within each project. For hard money, the principal is the FFI director general.
A qualitative, interview-based method has been used. We adopt such an explorative design since we are doing an in-depth study of a topic on which knowledge is scattered. The interviews were conducted as semi-structured interviews. The interview guide starts with general questions and moves towards core questions as we progress. The guide was adjusted several times before the first interview, based on feedback from various sources. The interview guide is reprinted in the Appendix.
In order to be allowed to conduct a study, it can be important to anchor interviews with gatekeepers (Hammersley and Atkinson, 1996). Our gatekeepers were research directors and the rest of the top management team. All informants were therefore briefed in advance that the study was being conducted with permission from the top management team, as well as the department of human resources. They were also informed that they would remain anonymous. The informants were selected by the strategic unit at FFI, based on a set of criteria in order to obtain a variety of perspectives and make the study more representative. The aim was to achieve a 50-50 split between researchers and research managers, a range of work experience, and a fair reflection of the proportion of male and female researchers.
The goals during the interview process were to gain an understanding of how the informants experienced the situation and, as far as possible, to isolate their own views. Based on the criteria of Thompson (1978, cited in Dalen, 2004, we emphasized interest in, and respect for, the informant, showing empathy and tolerance, and listening carefully. All informants were informed of the criteria selection process, that interviews would be transcribed and deleted after use, and that nobody should be identified during the interview. At the start of the interviews, the informants were encouraged to take centre stage and reflect on the questions we asked. In cases where we had cooperated with the informant in the past, we emphasized that they should try to treat the interviewer as a researcher, not an acquaintance. All interviews ended with an open 'do you have anything to add' question. The interviews lasted for an average of 40 minutes. Organizing and analysing the data were achieved through an iterative coding process. Coding is defined by Strauss and Corbin (1990, p.57) as 'the operation by which data are broken down, conceptualized, and put back together in new ways. It is the central process by which theories are built from data.' The interviews were compared for similarities and differences in order to locate variations. In the empirical part of this paper, results are expressed as a narrative 'state of affairs'. Results are presented by theme, and extremities and contrasting views being used to emphasize variation. Turning points are particularly valuable as the same informant can reflect on what happened before and after. Typologies are used in the concluding sections of the paper in order to emphasize the main points. Quotes are used heavily in the discussion and we add only short comments as we go to allow the reader to make up her mind before we present more detailed comments at the end of each sequence. Square brackets are used in cases where they increase anonymity without altering the contents of the quotes. All interviews were conducted in Norwegian, so quotes are our translation.
Finally, we emphasize the issue of pre-understanding, which all researchers have, and which it is important to describe before the discussion starts. We conducted this study because of personal experiences and pieces in the media and in the literature. The author's personal experience is that principals often have a clear opinion of how results should be presented and research conducted. Mainly, this has been constructive, though it is important to be allowed some free research time to build and develop a knowledge base. This author's initial thought was that there would be a great deal of scepticism about soft money and a wish to return to hard money because hard money leaves more room for classic research as well as allowing the researcher to shape his or her own work to a greater extent. Conducting this research as an internal study has the advantage that it is easier to know what to look for, while the most obvious drawback could be the bias this may have introduced.

Results and analysis
A man might say, with enough truth to justify a joke: 'Science is what we know, and philosophy is what we don't know'. But it should be added that philosophical speculation as to what we do not yet know has shown itself a valuable preliminary to exact scientific knowledge. (Russell, 2009, p.24) In this section, results from the empirical study are presented as a narrative based on the patterns identified during the coding process, with brief comments in between. Two typologies based on the findings are presented in the final section of the paper.

Introductory comments
The informants were first asked to say a few words about themselves and their role in the research programme. The average research programme contained somewhere between ten and twenty researchers. The number of principals varied from one to four, while two researchers had "many" principals, typically within the defence industry. Though not asked to say anything specific on funding, one quote surfaced which was relevant to many of the opinions later expressed about the relationship between funding and long-term relevance: What I see as time goes -we get more and more assignments which are of shorter duration. [There is] more research-based analysis support driven by political needs, a type of reform or something one wishes. Perhaps a few of the projects should surface in advance, so that one could reach a certain level of knowledge before the project begins. Now, there is an expectation of results from day one. I think the principal has something to learn regarding planning. That is my mantra every time I meet with them. 'If you are going to think like that, you have to start thinking early' -that is, investing in advance . . . Then you can deliver results faster.
In the remainder of this section, we discuss scientific ideology, ethics, and roles (the three main parts of our hypothesis) and what the informants see as necessary to have freedom in research.

Scientific ideology
Concerning scientific ideology, most of the informants seemed to believe that there is no single right answer, but that society needs both basic research and research directed towards solving given needs: In general, it is important to have both types because basic research is the origin of unknown and potentially great insights. In applied research, the task is formulated within a box. You can do a lot of useful stuff within that box, but you don't necessarily find solutions that lie outside.
Every research group has a need for some unrestricted research which is not directly governed by the needs of the principal. All groups need that, partly to build competence, but also for the maintenance of competence. We can establish ourselves as a good research group -establish networks and spend the money to form a foundation to be used next time around.
Here the reasoning behind claiming that scientific freedom is important seems to be grounded in Weberian ideology itself, but in fact serves as a foundation for short-term research. Two of the informants disagreed with the others and claimed that research should be useful to society. The rest of the informants favoured varying degrees of combinations between Weberianism, Baconianism, and Bernalism. One of the two more Bernalist informants said: I want to see greater benefits, not a hundred years in the future, but at least a few years into the future. I believe research could have been utilized better in order to create new jobs. There is too much being researched for research's own sake, I think.
On the question of whether the ideological view is different depending whether the subject of the question is FFI or research in general, the answer was mainly that FFI's research should serve society to a greater extent than research in general. However, several informants did not think that whether research is conducted at FFI or research in general makes a difference. In an FFI context, several informants are worried that the share of hard money is becoming so small that the base competence needed to perform rapidly commissioned research is vanishing: I think it was [a former director] who claimed that FFI should be a small institute for large projects and not a large institute for small projects, but I feel that is the direction it has taken. We are a large institute for small projects, at least within my field of research.

What is being researched: demands
On the relationship between funding and what is being researched and how, several informants claimed it is more important who is responsible for the project at the principal than which form of funding is used: Projects which are directly funded are as a rule active, but that is very dependent upon the person. The two projects [I work on] are funded using the same type of funding, but the principal behaves very differently. Some are very laid back and listen to results summaries twice a year, while there is a lot closer contact in other projects.
For the hard money, we can write project proposals exactly as we like.
Some informants experienced a difference in how demands are set, depending on the type of funding: In commissioned research, we face much higher demands. The further you move in the other direction, the less good customers I find them to be.

The same informant wishes to have a principal with demands:
It is comfortable to have a customer who does not set demands, but I am not sure that is how we obtain the best results.
A somewhat surprising result was that several informants influence the principal, not exclusively the other (and expected) way round. For example, one informant describes how the research programme attempts to implement some long-term, hard money funding in every project to sustain the competence some feared would disappear.
Some of what we try to do when we do commissioned research, to avoid falling into the trap of no more research (which we have always managed to do) is to create room in the commission for not only short-term funding, but also a long-term component. We have worked to create an understanding on this and it has worked relatively well.
We try to get fresh funding into already existing projects, and can offer to alter focus somewhat. If we get no money, we can reduce research on areas of interest to that principal. We see that the balance of power is in no way as one-sided as in Kjaernet (2010) and Richter and Hostettler (2015). The informants seem to think that, to a large extent, there are good opportunities to influence what is being researched within a given frame, and that the researchers have a great degree of autonomy in their job. This finding questions the view that the researcher has lost all control over what science is for. As a final note, no one claimed that the involvement of the principal in what was being researched was unethical.

How research is conducted: control
Questions concerning how research was conducted saw several informants say that control has increased over the years. One of the informants described how the deadlines were gradually becoming shorter and shorter: An important difference is the aspect of time. The research we have done for [. . .] is very short term. They demand answers in very short time.
Other informants, while noting that project specifications had become more detailed, stated that they were not supplied with enough information about the project itself: There has been a change over the last ten years. The project specifications are more and more specific. Demands for specifications have increased. There is less freedom in that sense with regards to what type of output we shall produce.
If the principal wants to exert control, it is in a sense good, but we sometimes feel they do not realize what is in their own best interest. Sometimes we feel that they do not provide us with all of the information we need. Sometimes we are not well enough integrated.
Asked to elaborate, the informant gave the example of a project where there was disagreement with the principal regarding method: The reason is disagreements concerning method and that they have a range of arguments we do not find very realistic. Then there are the more research-based viewpoints from our side versus more practical 'thinking' from their side -'it cannot be that expensive'. They handle this by shutting us out of certain processes.
This was one of few examples in the data where we could identify a clear attempt by the principal to apply direction. Research is not necessarily controlled directly, but to shut out the research group from some processes can be a way of avoiding unwanted results. The example above is from a project of great interest to the principal, where politics is of great importance, where there is a great deal of uncertainty, and where retesting results is difficult. In general, research with these characteristics brought more of the challenges described in this paper than research on less abstract topics, of more technical character -and of less importance to the principal's senior management.

How results are being presented: carefulness
Informants were also asked whether they feared reactions when presenting results. Most claimed they have not been influenced, while some reflected on whether there is such a relationship: An important aspect with hard money is that it gives you the freedom to voice your own opinions. That is, to give unpopular messages. It sits in the back of your mind, a kind of precautionary thing, when you have a direct relationship with the principal and a bag of money and [he] is about to refinance the project as the current project expires.
Another informant had a specific example of an episode with negative consequences: We had an example a couple of years ago when we delivered a few unpopular reports. In the following years, funding became more difficult to come by. We thought that some wanted less of us than more of us.
Asked whether more caution was exercised after this experience, the informant replied: No, I wouldn't say that. But you can say that one of the reports led to changes for the establishment, as timeliness was emphasized more: 'do not come with results that do not fit in at this very time'. It has led to us adapting, but also to seeing whether our work fits with the internal processes of the principal. Not just to avoid being unpopular, but to gain influence within the principal. For various reasons, reports didn't fit within the principal's time lines, but it has led us to plan with longer lead times now.
The episode was a turning point for this informant. There are very few other examples of such turning points in the data, especially among informants who work with technical tasks (for example, developing hardware and testing ammunition). Among researchers working with more abstract problems, or within areas where conclusions cannot be drawn with certainty, sensitivity was higher.

How results are presented: form of publication
While some informants had not reflected upon the relationship between funding and publication, some had. A typical reflection among these was: The more hard money, the more reports and international peer-reviewed journals is the direct goal. The closer you come to direct funding, the more we sense that the principal is not interested in large, voluminous reports. They want something rather short, concise . . . preferably a powerpoint, some conclusions, something they can use going forward. They want straight talk. The degree of detail depends a whole lot upon the type of funding, I find.
None of the informants found any of this ethically objectionable. On the other hand, several found it academically unacceptable: Sometimes you would wish to follow a specific track. It is not because you are not allowed to [that you cannot do this]; it is because there is no time.
The same informant reflects on the establishment's goal of more peer-review publishing and found that this did not match up with an increasing amount of soft money: I feel we have been directed towards [more commissioned research] and I don't think that adds up with regards to the goal of more academic publishing. I don't understand how they aim to unite those goals. Funds are collected both from commissioned research and from research closely tied to a political project where one must deliver a type of consultancy work, but at the same time you are evaluated on whether you deliver academically. The data do not allow a conclusion to be drawn on whether the informants find the link between form of publication and funding ethically dubious, as not many informants had strong views.
To summarize, when it comes to influence, a project seems to be more dependent upon the person responsible at the principal organization than upon the form of funding. The balance of power is not one-sided: many researchers say that they themselves influence the principal, either through arguing for a part of the project being funded long term or through choosing what is presented, thinking of future funding or the direction of the project. On the other hand, some informants expressed a cautious attitude towards presenting very controversial results. Many of these conducted research within strategically important topics, often with no definite answer. Finally, since there was so little variation on the view of scientific ideology, we were not able to establish any clear links between ideology and the topics described in this section.

Self-censorship: conclusions and future work
To the question of whether the conclusions of a project affect the probability of future research funding, most informants said that the important thing is to reach a conclusion, not what the conclusion is. If the conclusion is sound, the principal is generally fine with that: No, I have not experienced that, as long as it is well grounded and based on facts. But this could also have to do with the fact that much of what we do involves projects that are embarked upon because no one knows what to expect.
On the question of whether the issue in the previous question influences how results are presented, this is also denied: We are very clear that we stand for what we mean. We shall not be directed. We have worked on that a great deal, since we have principals so close. The ethical side must be sufficient. We shall not be influenced in one direction or the other. Few informants had given subconscious processes much thought, but one had: You can always wonder which subconscious processes are going on. That is my biggest question. Like we do now, with analyses for [. . .] -we know many will react adversely to those. It will jeopardize some jobs and have large consequences. We will publish anyway. We don't tie our hands, at least not consciously. We conduct our research and publish, and then they just have to handle it. I have often thought there might be something unconscious going on.
Several informants discussed the relationship between self-censorship and the funding situation. Most informants said it is easier to be free and independent when there is no lack of demand for new projects: We have been in a fortunate situation for years. We have enough projects to choose from. We have not had to sell out, prostitute ourselves, to get funding.
A variant of adapting results is the researchers themselves adapting the conclusions so that they can twist the project in a way they find interesting: No, I don't think we do anything to please the principal. But I do believe that we who are so interested in technology, might portray the results as a little more positive than they are in the hope that they will continue to fund exactly what it is we are doing.
Again, we see that the balance of power is not as one-sided as assumed in theory, which is that power mainly lies with the principal. In this case, the researchers view their expert power as so great that they aim to influence rather than to be influenced: To a great extent, they lean on us, at least on the technological side. They know the practical side; we are more of a technological base and know what is possible to do. When you procure equipment, we are almost used as bullshit detectors.
Again, this informant underlines a central conclusion of this paper: technologically oriented researchers, often working with specific problems within smaller systems, have fewer experiences of the type of influence described earlier. Researchers working on large analysis projects had to a greater extent experienced these problems: What you talk about there is perhaps mostly related to large studies at an overall level, where you consider large structures and suggest major changes, propose to sell off the frigates, for example. We are not there.
When it comes to self-censorship, informants are to a great extent in agreement that they do say what they mean. The main worry is that a tougher funding situation in the future will increase the probability of self-censorship.

Hidden agenda
Very few of the informants experienced a hidden agenda. On the other hand, many people had indeed experienced an agenda, but this is almost exclusively seen as transparent and unproblematic, for example: I believed they had an agenda of supporting a reform with fact-based knowledge. They felt something worked in such and such a way in the organization, but they had no numbers [to back it up]. But it was never a case that if we found something else, that was problematic. Not very hidden -rather very open.
Of course, there can be hidden agendas we can't see.
A couple of informants had experienced hidden agendas: One did not see this as a problem: Yes, well, it is very easy to see when the principal wants something or other without saying so out loud -they want system x instead of system y. If  Another informant said that one can say no to these tasks, at least as long as the financing situation is as good as it currently was: There can be a sense that it has been controlled by [the principal . . .]. It can almost be interpreted as a commission for a specific result. You do not want to get into that. We've had sufficiently robust funding that we've not had to run after that type of task.
It can be anything from small things, where they want to buy that system, so we begin there and move backwards, instead of starting at a conceptual level and see what you really need. We should be neutral enough that we are not tied, but can consider the conceptual.
On the specific question of whether research had been ordered for the purpose of being able to use the FFI logo, many said yes, while at the same time claiming this was unproblematic: No, but we have some people at the principal who know that it is beneficial to have an FFI report in their hands when pushing something through. They say it is a great advantage having FFI on their side, but I don't feel it's a hidden agenda.
Most researchers did not currently see hidden agendas a problem, but if the funding situation were to worsen, it would be more difficult to say no to projects where the principal has an agenda, hidden or not.

Conclusions
[No] matter how many instances of white swans we may have observed, this does not justify the conclusion that all swans are white. (Popper, 2002, p.4)

Summary
We have looked at whether a principal, through their choice of funding, can influence the results of a research project through applying directions to what is researched, through commissioning research to strengthen a case (hidden agenda) or by enforcing self-censorship on the researcher; for example, by not awarding any future projects if the researcher comes to unpopular conclusions. To study these problems, we con ducted qualitative interviews with ten researchers and research managers at the Norwegian Defence Research Establishment.
The canvas for this discussion is the timeless debate on scientific ideology and ethics. While one ideological view is that research should provide some form of value, others argue that science should be value free and look for answers for the sake of the answers themselves. In our empirical study, nobody expressed extreme views in either direction regarding this scientific ideology, as most preferred a mix. Because of the small variation in ideology answers, it did not help explain the informant's reflections on the current debate on funding, and on the questions of applying directions, hidden agendas, and self-censorship. During the course of our study, however, we discovered that the balance of power between researchers and principals is not as one-sided as suggested by some of the literature. Researchers also have a fair amount of power. This is an area in need of more research.

Typologies
Based on the discussion above, a range of researcher roles emerges. At either end of the scale, there are two main types of researcher role: • The influential consultant: works with projects where the results potentially have a major influence on the development or direction of the principal. Projects are often done for the most senior management level at the principal. The consequences of not following the researcher's advice often do not become clear until years later. In the terminology of Funtowicz and Ravetz (2003), this resembles post-normal science. • The technical realist: often works with scientific or technical tasks directly for a principal located below the top level. Results and conclusions are often clear and immediately visible. Not following the researcher's advice can result in visible and immediate consequences.
In the terminology of Funtowicz and Ravetz (2003), this resembles applied research.
A researcher is seldom at one extreme of the scale, but will rather be somewhere along the axis from influential consultant to technical realist. A researcher can even simultaneously have roles near both ends of the scale. He or she can, for example, be involved as an analyst in long-term planning, as an influential consultant, while at the same time working as a technical realist on a project doing research on a specific type of technology. The tasks and the setting more than the personality of the researcher, determine whether a researcher comes in touch with the issues discussed in this paper. Whether the respondents were male or female, researcher or research manager, or had little or much experience, did not matter. More experienced research managers obviously have more experience than a new recruit, but the type of work they do is more important than sex, title, or experience.
The research conducted by the influential consultant is more in danger of being subject to control simply because the results are uncertain (and so are easier to influence or to question) while at the same time they have a major impact. This also influences the possibilities of future work for the researcher: the work of the influential consultant is by nature more controversial, so the question regarding self-censorship also becomes more important to this group. The same goes for the hidden agenda issue: the research and analysis done by the influential consultant is more politicized and therefore more exposed to this sort of influence.

The direction of influence and further research
Much of the theory upon which this paper is built takes the implicit view that power lies with the principal. However, as in any market, what is important is the relative distribution of power between the demander, i.e. the principal, and the supplier, i.e. the researcher. As long as there are multiple potential principals, the researcher should possess at least some power. Our hypothesis suggested that the direction of influence should go from the principal to the researcher. However, the interviews showed that influence goes both ways and that while some researchers find the shift in funding type challenging, researchers have discovered ways to adapt to maintain a certain influence over what is being researched.
Given that this study contradicts some previous studies in its finding that researchers themselves exert a considerable degree of influence, further studies might be in order to identify the causes of this and other differences. To discuss the balance of strength in the research markets is an obvious direction for future research to take. The seminal works within the power-base literature (French andRaven, 1959 andRaven, 1965) and influence theory (e.g. Cialdini, 2001Cialdini, , 2006Cialdini, , 2009 could be suitable points of departure. The importance of the expert and informational powers from Raven (1965), and the authority and scarcity principles of Cialdini could perhaps be of use in such studies. Combining analysis of relative strength between principals and researchers, as well as relative power and influence, would fill a gap in the literature.
One hypothesis might be that the way FFI is financed allows the researchers to exert more influence since there are several possible principals, not just one principal, such as a research council. Furthermore, if demand for research falls below current supply, researchers might have to take on projects they find more ethically questionable. A key to understand who defines the role of science is determining who has the greatest power -politicians, the general public, the principal, or the researchers themselves. To do this, a larger sample than that used in this exploratory study is needed. Researchers and research managers from a variety of institutions should be interviewed, as well as representatives of various principals. Further research will help answer the important question of who defines science in our time.
• What do you think about the relationship between the type of funding and how the results of a project are presented? • Can you give me concrete examples illustrating the difference?

Conclusions and future work
• What are your experiences regarding whether the conclusions of a project influence whether the principal orders new projects?
{ Does this influence how you present the results?
• Do you put more emphasis on presenting the parts of the result you think the principal wants than into results you believe are in conflict with what the principal wants?

Hidden agenda
There has been a great deal of debate concerning hidden agendas in research; that is, that the principal wants something other than is publicly stated.
• Have you experienced that the principal sometimes has a hidden agenda in ordering research and wishes a particular result?
{ Do these wishes depend on type of funding?

{
Have you experienced the principal at times wanting only to use the FFI name and logo?
• Can you give me examples of experiences of hidden agendas?

Final comments
• Is there anything you would like to add in order to fill in any blanks?