An Empirical Study Of Innovation in Manufacturing Teams : A Preliminary Report

We are currently engaged in a collaborative research project, the aim of which is to determine how new ideas are introduced and managed during group discussion. To this end, we have collected and transcribed a corpus of team meetings within UK manufacturing firms, and are beginning to code these meetings in order to characterise the turn-taking and decision making which occur within them. This paper situates this work by explaining why the general approach is necessary for understanding what happens during group decision making and describes the research methods which are being used and the areas which we intend to investigate.


Introduction
There is currently a move towards team work in industry; changes come too quickly for companies to rely on maintaining the status quo and multidisciplinary teams, representing the diverse knowledge and skills of the workforce, are believed to be best equipped to cope with the problems which result.This reliance on team work raises as a matter of some urgency the question of how teams may be organised to best address the tasks with which they are charged.Although group decision making has been studied in the laboratory, decision making in a working environment is actually quite different.In addition, in order for decisions to be made in a group, the participants need to communicate their ideas and thoughts to each other effectively.The transfer of information and ideas between people is highly constrained by both the physical facts of how people communicate and by the need for the communicators to come to a mutual understanding of what the communication means.Although these constraints have been studied primarily in the context of two person dialogue, they have an even larger impact on group meetings.
Our work attempts to remedy some of the shortcomings of past work on group decision making by studying natural teams making decisions in their normal working environments and by considering how the act of communication constrains the team process.Our primary aim, in concert with the emphasis on adapting to change, is to determine how new ideas arise during group discussion and what happens to them once they have been expressed.Our approach is to characterise what happens during meetings using subjective annotations on a corpus of meeting transcripts, and then to investigate the relationship between these annotations and innovation, using the past literature as a guide.

Past studies of innovation in groups
Dunbar [1] argues that although most of the psychology literature on group decision making seems to show that groups do not improve upon the decision making of individuals, this is an artefact of the style of their experiments; individuals do as well as groups in the situations tested because the tasks which are set to the groups are things that each individual can easily do alone and which do not take much time or thought.Dunbar claims that in domains where knowledge plays an important part in decision making, such as scientific laboratories, and where groups are constituted to represent a range of skills, groups foster new ideas and aid discoveries which could not easily be made by individuals.Since we are considering how to foster innovation in manufacturing teams, this suggests that we should particularly note the makeup of teams and the types of problems being solved.Dunbar also suggests that analogies are an important part of how groups of scientists introduce new ideas and develop their research plans, especially when similarities between two experiments or when systems of relationships between two domains were explored.
West [2] is a study of group decision making for complex tasks in the workplace which broadly accords with Dunbar's results for scientists.Building on Schön [3], West argues that groups best deal with uncertain situations by reflecting on their actions.Reflecting involves asking questions about the groups' goals, making plans, and explicitly considering the process of coming to a decision, rather than allowing the decision to come about in the most obvious way.West suggests that groups are best equipped to reflect if they are heterogeneous, so that they contribute different views of the problems to be solved, and that they are most likely to reflect when they are interrupted in their normal working patterns by conflicts, institutional crises, organisational problems, and changes in the membership of the group.West also links social factors to reflection, with groups more likely to reflect if they are supported by their institutions and if the individual members are happy in their jobs, although he points out that it is unclear which is cause and which is effect.

Past Studies of Communication in Groups
Understanding communication is important to the study of group decision making because the physical facts of faceto-face communication affect the decisions which are made.In a meeting, only one person can take the floor at any one time because it is only possible to attend to one person's voice at a time and because it is not possible to speak and listen at the same time [4].However, people interpret contributions in terms of the immediately preceding context; regularities in the sequencing of contributions (such as answers following questions and greetings following greetings) have long been noted [5] and explained in terms of the interlocutor's goals [6].Meanwhile, people have poor memory for what they have been told previously and by whom [7].These facts of communication make it difficult to have discussions which do not wander from one point to the next and to agree on decisions, since a decision can not be ratified unless all of the participants understand that it has been made.They work against novel contributions by establishing patterns and discouraging reflective behaviour [8].They also exert more pressure on large groups than small ones, since the more people who compete to take the floor, the harder it is for participants to make contributions at the correct time for them to be fully understood, especially where the contribution involves a new idea which needs input from several group members.
There have been several empirical studies of group studies relating to group size or relative talkativeness.People contribute unequally to group discussions; as the size of a group increases, the inequality of contributions becomes greater [9], but people generally report being happiest in groups where the participants contribute equally [10].The talkativeness of an individual is relatively constant in the same group over different meetings [11].but influenced by emotional state, status and value to the group, motivation to perform a task, prominence of the seating position, and importance of the person to communication outside meetings [10].People who speak least tend to make comments to specific individuals, while the people who speak most tend to direct their comments to the group as a whole and also have the most comments directed towards them [12].In addition, people are unexpectedly likely to speak after people sitting across from them instead of next to them [13], probably because people use eye contact cues in taking over the floor.All of these results suggest that the circumstances in which a group meets can affect the final outcome of the meeting.
In addition, there has been some work on predicting who will speak next in a small group setting, usually done without having the content of the utterances available.The average time between two utterances by the same speaker is shorter than one would expect if speakers took turns randomly, and in some kinds of groups, there are patterns of pairs of individuals who tend to speak before and after each other [10,14].In family groups, turn-taking is related to role, with communication between a father and a child going through the mother [15].We would expect on the basis of these results to find turn-taking patterns in our meetings.Where the patterns arise from constraints on communication and from social factors such as status, it is likely that they inhibit the expression of new ideas.

Research Methods
Because we are concerned with where new suggestions arise in meetings, the content of the contributions is important to our study.Bales [12] devised a set of categories, further used by Klein [16], describing the function of meeting contributions and expressed hypotheses about the relative timings of the categories, but was hampered by the lack of recording and language corpus technology.It was not feasible at that time to collect large amounts of data for transcription, and recording could not be done outside the laboratory.Even if the recordings could have been made, techniques for handling large language corpora were not far enough advanced to be able to handle a reasonable study.Yet, as Dunbar [1] and West [2] have both argued, laboratory studies of group decision making are inappropriate if we wish to draw conclusions about how decisions are made in real world situations, simply because it is not possible to construct a laboratory experiment in which the participants are as heterogeneous and as committed to fulfilling their goals as they are in their daily lives.In order to understand decision making in the workplace, it is necessary to study it in the workplace.Our research methods follow the tradition set by Bales [12], updated for modern computers, and are similar to the empirical techniques which are currently popular for the study of spoken dialogue [17][18][19][20][21][22][23][24][25].

Data Collection
We have audio taped ordinary business meetings from five teams within UK manufacturing SME's ("small to medium-sized enterprises").One further team has yet to be recorded.All meetings were held on the manufacturers' sites and were completely ordinary from the participants' points of view, except for the fact that they were being recorded.Meetings were audio taped using two PZM tabletop microphones placed at opposite ends of the table and either recording personal stereos or recording DAT players; we found that this arrangement interfered minimally with the meetings whilst providing good enough recordings for orthographic transcription.Some meetings were also videotaped using a single stationary camcorder; this does not provide enough information to study gesture or eye contact, but can be helpful for determining who was speaking at any one time.In all cases, the participants involved gave their permission for audio recording on the condition that the recordings be used only by our immediate research group for transcription and coding, and that no excerpts of the transcripts or audio tapes be distributed more widely without being anonymised first.Permission for video recording was only given to aid original transcription.
The teams involved in the collection were chosen to provide a range of informal meeting types.Meetings lasted anywhere from fifteen minutes to all day but were usually just under an hour long.For each team a series of five or six meetings was recorded, except one team where two long meetings were recorded instead.The number of participants in a meeting varied between four and a dozen; we rejected larger teams not because the meetings which they hold tend to have formal turn-taking arrangements and because large meetings are much more difficult to transcribe.Some teams were reasonably homogenous and met as a matter of routine; other teams were attempting to solve particular problems which their companies were facing, and were composed of people representing a wide range of skills and expertise.The collection was designed to give us a manageable set of meetings which represents the range of meeting types which occur, as far as is possible with a natural collection.

Transcription and Coding
Once we have the meetings on audio tape, we have temporary secretaries transcribe them in a way which records not just what each of the participants said but also, in the case of overlapping speech, roughly when each of the participants started speaking.This is impossible to do under our recording circumstances when many people are speaking at once, but in these cases usually that section of tape was impossible to transcribe altogether.Most teams were orderly enough that not much material was omitted in this way, although within a single contribution there are often unintelligible words.The rough location of backchannels is also noted; backchannel utterances are ones which do not attempt to take the floor, such as "uh-huh".The transcripts are then checked for accuracy and transformed into SGML, a standardised mark-up language [26], so that they can be processed by machine and can be coded further using graphical user interfaces.
In addition to the base level transcription, we are categorising what was said according to how it contributes to the meetings, using the following categories: • Agreement: Agrees with something which has been said or suggested, either actively or passively (by doing something to comply with the suggestion).

• Suggestion:
Makes a new suggestion about some action to take or reiterates a suggestion which has already been made.

•
Information: Gives some information, opinion, evaluation, or analysis, or provides clarification for some information, opinion, evaluation, analysis, or suggestion which has already been made.

•
Request for information: Asks someone else to give information, opinion, evaluation, analysis, or clarification.

•
Request for suggestion: Asks someone else to suggest a possible action to take.• Disagreement: Disagrees with something which has been said or suggested, either actively or passively by not behaving compliantly.

•
Positive Affect: Makes a comment or joke which is supportive of someone in the group and not to anyone else's detriment but which does not fit any of the previous categories.

•
Negative Affect: Makes a comment or a joke which is openly hostile, negative, or withdrawn but which does not fit any of the previous categories.

•
Acknowledgement: Acknowledges that someone has said something and seems to mean "I'm listening" or occurs to keep communication open, but does not clearly express agreement or disagreement with another speaker and does not attempt to take the floor.
Details of the coding scheme are given in [27].This is a refinement of Bales' original categories [12], with a few minor changes.We rejected Bales' expressive distinctions because they did not seem useful for the current purpose, and we abandoned the distinction between information and opinion because we did not feel it would be reliable; it is difficult to tell the difference in these meetings without having specialist knowledge of the manufacturing industries involved.In addition, we thought it was useful to add a category, acknowledgement, which is primarily for non-committal backchannel utterances; under Bales' scheme these utterances would largely have been ignored because they are short and typically concurrent with other speech, and his coders were working without the benefit of audio recordings.Note that many of these categories resemble Searle's speech acts [28], although Bales' original formulation predates Searle.The primary difference is one of emphasis; although speech acts can provide a formal underpinning to work, empirical work has its own set of concerns.Empirical categories must be reliably identifiable given the conditions under which the corpus was collected, make useful distinctions, and occur frequently enough for statistical results.
In addition to changing some of Bales' category distinctions, we have changed the scope of the codings.Bales segmented contributions by simple sentence, roughly marking one code for each verb and its associated constituents.Without audio recordings, it was necessary for his coders to describe very small and regular chunks if they were to be accurate.Since we are working from transcripts and audio tapes, we find it more convenient to use larger segments.For most categories, it is important to know how many instances of the category occurred, where they occurred, and who was speaking; for instance, it should be possible to identify cliques within a group by looking at who expresses agreement explicitly and when.
Because our work focuses on innovation, it will be especially important to be able to count the number of suggestions which are made.However, it is difficult to see where one bit of information or opinion ends and the next one begins, especially since participants tend to convey all of the information they think is relevant to a meeting in long monologues, interspersed with periods of discussion.Our approach is not to be concerned about segmentation for most categories.For informative utterances, it is less important to know how many ideas were expressed than to know roughly how much information was given; we can measure this roughly by the number of words used to express it (although of course that will not describe whether or not the contributions were truly informative for their hearers!).Most of the other categories tend to occur in single instances; for instances, when participants express affect, agree, or make a request, they tend to only do it once during their turn.Our coders are instructed to make all segments as large as is convenient except where the contribution being coded is a suggestion, in which case they are expected to indicate where each suggestion begins and ends.
In addition to these investigations, we are considering codings which connect directly with work by Dunbar [1] on analogy and by West [2] on reflexivity, since both of these authors claim discuss the emergence of new ideas within a group.For analogies, we are considering how to adapt Dunbar's classification scheme of analogies in scientific discovery (local, median, and distant) to the manufacturing domain.Much of West's reflexivity concerns turning the discussion in a meeting towards the process of meeting itself.Under this definition, suggestions, requests for suggestions, informative utterances, and requests for information, can all be reflexive; we hope to be able to subcode these utterance types reliably so that we can consider whether or not reflexive teams really are more innovative.We also hope to develop a way of annotating how the different suggestions in a series of meetings relate to each other; for instance, a suggestion might be made for the first time, repeated, modified, or merged in a compromise with another suggestion.Such an annotation scheme would be useful for tracking the progress of decision making in the meetings and could be based on pragma-dialectic argumentation theory [29].
Reliability of data is a major issue in any work which uses subjective coding schemes; if different people can not agree on how to apply the codes, then it will be unclear how to interpret any statistical results based on the coding [30].All parts of our transcription and coding are subjective, including simple judgements such as which person was speaking.We are currently testing the reliability of our transcription and coding where it impinges on the hypotheses which we wish to test, and reliability issues will be paramount in the further development of our coding schemes.In addition, it is important to keep in mind that meeting participants will naturally view the proceedings differently than someone working after the fact from a transcript.In particular, events which are not reflected in the transcript may have a major impact on the course of a meeting.Results from this style of empirical work require careful interpretation, but this approach does have its benefits in firming up otherwise impressionistic views of the data.

Areas of Investigation
Most of the results in this field to date have either been speculative because they have been based on very limited observations of groups or have observed the turn-taking structure of the group but not had recourse to the content of the contributions made.Our data collection techniques leave us in a position to address many of the open issues.We plan studies in the following areas:

•
Group size: Past studies of group size suggest that group discussions are more successful if they are kept small.We suspect that the more people competing for the floor, the harder it is to keep the meeting on track.Group size should affect how easily participants find it to express novel suggestions.

•
Turn-taking: Dabbs and Ruback [10] and Parker [14] note that in their groups, the order of turns taken is not random, but that recent speakers are unexpectedly likely to speak next.The facts of communication support this result if the meeting is coherent; contributions should be relevant to the preceding context, and previous speakers are most likely to have something relevant to say, besides having the meeting's attention focused on them so that they are more likely to obtain the floor.We have informally noted that previous speaker is a better predictor of who will speak next in one of our small, well-run groups than in another large, chaotic group.We intend to further explore such predictors and differences among the groups.Having the decision making coding available and categorising contributions as backchannels, interruptions, or sequenced (latching) utterances will help us to characterise the order of turns more closely than has previously been done.

•
Well-formed sequences of contributions: Many authors suggest that successful decision making should exhibit clear phases discussions which show clear phases in the type of contribution made (e.g., [12,31]) although it is often unclear how to interpret evidence and extrapolate to other group types in such studies [32].
It should be possible to use our coding to look for phases in the recorded meetings.Carlson's [33] argumentation-oriented "dialogue games" suggests a formal approach to working out what patterns to expect; it involves characterising logically the reasons why a contribution is made in terms of what information is conveyed, and then looking for what behaviours that contribution requests or enables.Although our coding scheme is currently developed simply as mark-up for empirical work, we intend to characterise our set of moves formally so that we can develop hypotheses about the ordering of categories in successful and unsuccessful meetings.

•
Facilitation and Chairing: Some of our groups had excellent chairmen, and one had a professional facilitator; these key people were not always present for all meetings in the recorded series.Other groups lacked obvious leadership.Good chairmen and facilitators seemed to summarise points frequently, ask people who had not had a chance to speak for their observations, and make suggestions and group goals more concrete.We hope to use the decision making coding and turn-taking metrics to describe the actions of these participants and to discuss what effect their actions had on the course of the meeting.Although there are many informal recommendations about how to run a meeting, and some recommendations for highly structured decision making (e.g., [34]), we believe that observing what happens in practice in meetings will help to instruct such recommendations.

•
Use of backchannels and agreement: In order to affect the outcome of a group, a decision must be mutually acknowledged and agreed by all of the group's participants.Silence in a meeting may be seen as tacit agreement, but it may also show a reluctance to accept what has been decided which will affect the future course of group.Meanwhile, we have informally observed that backchannel utterances occur when a participant seems to wish to take the floor and during the part of a meeting when a decision is being ratified.Backchannel utterances are notoriously difficult to record, but a preliminary reliability study suggests that in our working practice, although some transcribers pick up more backchannels than others, the difference is one of sensitivity rather than being random.We hope to study the location of backchannels in order to explain how they are used.

•
Since the ultimate goal of our work is to make useful suggestions about how to run meetings, in addition to observing what actually happens in the meetings collected, we also need to have some sense of which groups were successful at implementing innovations and which were not.The work described here is taking place as part of a larger collaboration with the Institute of Work Psychology at the University of Sheffield and the Centre for Research in Education, Development, Instruction, and Training at the University of Nottingham.
In particular, Sheffield is investigating a link between team climate and innovation [35], using standardised questionnaires [36], interviews with management, and expert judgements of how innovative teams are.Although we are only using a few of these firms for our own study, we intend to use the information which they are collecting to place our firms in a wider context and thus to give us speculative results relating meetings to innovation and team climate.

Conclusions
Because the research project which we describe is looking in detail at natural meetings in a business setting, our results should be of interest not only to the academic community, but also to practitioners in industry.For instance, understanding the effects of group size will give weight to advice to keep groups small and will help practitioners know what to do when a large group is unavoidable.Our results should help to explain good chairing behaviour and which elements of structured decision making provide benefits for the participants.In addition, understanding the role of backchannel utterances will be of interest to designers of videoconferencing facilities, since current videoconferencing does not allow for backchannel utterances, even though they might be important signals for turntaking and for reaching a consensus.Although the project is still in the early stages, with data collection and transcription nearly completed and coding underway, we feel that this style of work is an important development for the field of group decision making because it will allow us to investigate many phenomena more clearly than with other methods.