Brainstorming Under Constraints: Why Software Developers Brainstorm in Groups

Group brainstorming is widely adopted as a design method in the domain of software development. However, existing brainstorming literature has consistently proven group brainstorming to be ineffective under the controlled laboratory settings. Yet, electronic brainstorming systems informed by the results of these prior laboratory studies have failed to gain adoption in the field because of the lack of support for group well-being and member support. Therefore, there is a need to better understand brainstorming in the field. In this work, we seek to understand why and how brainstorming is actually practiced, rather than how brainstorming practices deviate from formal brainstorming rules, by observing brainstorming meetings at Microsoft. The results of this work show that, contrary to the conventional brainstorming practices, software teams at Microsoft engage heavily in the constraint discovery process in their brainstorming meetings. We identified two types of constraints that occur in brainstorming meetings. Functional constraints are requirements and criteria that define the idea space, whereas practical constraints are limitations that prioritize the proposed solutions.


INTRODUCTION
Idea Generation has been a topic of creativity research for centuries.Brainstorming, in particular, has grown to become synonymous with idea generation.Brainstorming was developed in the 1930s by Alex Osborn, an advertising executive who co-founded the world's largest advertising agency network.Osborn (1953) first coined this process "brainstorming" and published it in his book, Applied Imagination.Contrary to decisionmaking techniques that aim to systematically eliminate unsuitable ideas to reach a final consensus, the brainstorming process focuses on gathering as many ideas as possible.The four brainstorming rules are: (i) Focus on quantity (ii) Withhold criticism (iii) Welcome unusual ideas (iv) Combine and improve ideas In his initial description of brainstorming, Osborn claimed that by reducing the amount of criticism from self and others during this creative process, a group of individuals could produce better results in terms of quantity and quality.
In the past half-century, a plethora of studies focused on enhancing the brainstorming process have been published.Surprisingly, research has shown that group brainstorming is not as effective as brainstorming separately as individuals.Diehl and Stroebe (1987) reviewed brainstorming productivity losses and laid out five of the most common hindrances of the group brainstorming process: Evaluation apprehension: "working in a group makes one's contributions visible to others, and despite the usual brainstorming instructions not to evaluate others' ideas, the members of a group can still be reticent to contribute their ideas."Free riding: "individual members of a group might not expend the effort since other members of the group are contributing ideas."Limited air time: "when only one person can speak at a time, there is limited time for each individual to contribute."Production blocking: "at each moment only one line of ideas is being generated, since they are reported serially; groups will therefore tend to pursue fewer different kinds of ideas." Brainstorming Under Constraints -Why Software Developers Brainstorm in Groups Shih Venolia Olson Cognitive inertia: "because of limited air time, individuals often have to hold on to their contributions until they get a chance to report them, and as a result they might forget them, or they might decide not to offer them; in either case, the act of holding on to them will prevent them from thinking of other ideas." Alternative methods have been developed to overcome these hindrances.The nominal group technique (NGT) is one effective method in which each member is given time to brainstorm alone without communicating with other members.The individually generated ideas are later pooled together in a merged list.This process essentially eliminates idea evaluation and criticism during the idea generation process; some researchers choose to call this "deferred judgment" because idea selection is postponed until a later stage of the process (Grossman, 1984).Dennis and Valacich (1993) reported that in more than 50 laboratory studies, groups employing NGT generated a far greater number of creative ideas than groups brainstorming while communicating with each other (also known as interactive groups).Other reviews of brainstorming research also found brainstorming using NGT to consistently produce better ideas in terms of quality and quantity (e.g., Barki & Pinsonneault, 2001;Mullen, Johnson, & Salas, 1991).However, NGT can lack the social and collaborative aspects of brainstorming as a group exercise because it prevents participants from actively engaging and building upon group ideas.Many studies reported that participants have more positive reactions in interactive group sessions than in nominal groups, including overall satisfaction and perceptions of group effectiveness (e.g., Mullen, Johnson, & Salas, 1991;Stroebe, Diehl, & Abakoumkin, 1992).For this reason, despite the obvious performance gains under the nominal group settings, practitioners continue to brainstorm in interactive groups.Some researchers choose to call this common notion, the "illusion of group productivity" (Pinsonneault et al., 1999;Stroebe, Diehl, & Abakoumkin, 1992).
Subsequently, researchers have considered how collaborative technologies such as electronic meeting systems and group decision support systems might influence the brainstorming process.These electronic brainstorming systems (EBSs) often incorporate benefits of NGT by supporting anonymous and parallel input (e.g., Connolly, Jessup, & Valacich, 1990), which eliminate both evaluation apprehension and production blocking (Diehl & Stroebe, 1987).Usage of an EBS typically involves multiple users entering ideas anonymously into a central repository in parallel.The ideas stored in the central repository are selected at random and presented to the participants at specific intervals or upon request in order to trigger new ideas.A review of prior studies by Barki and Pinsonneault (2001) reported that EBS groups produced more, as well as better ideas, than conventional interactive brainstorming groups.However, existing empirical evidence that compares performance of EBS and nominal brainstorming groups remains inconclusive.Despite the increased idea performance experienced in laboratory settings, most of these solutions have not been widely adopted because they have failed at providing group well-being, such as reinforcement of group culture and socializing activities, and member support, which involves establishing personal status by demonstrating expertise and building knowledge networks (Dennis & Reinicke, 2004).For this reason, brainstorming in interactive groups without the help of EBSs continues to dominate brainstorming practices in both corporate and academic environments (Dennis & Reinicke, 2004).
In software engineering, brainstorming is often taught and used in the context of a design methodology (Robinson, 2004).Studies have shown that brainstorming is often used in the informal and early-stage idea generation phase of software design (e.g., Wu, Graham, & Smith, 2003) and has been incorporated into requirement engineering and agile software development methodology as a standard idea generation technique (Nuseibeh & Easterbrook, 2000;Paetsch, Eberlein, & Maurer, 2003).In fact, some researchers claimed that brainstorming and its variations have become the "definitive basic method for finding ideas" in software design (Koberg & Bagnall, 1974).Given the prevalence of brainstorming practices in software design methodology, the proposed work aims to characterize brainstorming practices in the domain of software development.The resulting findings can then be used to provide design recommendations that are both feasible and adoptable for brainstorming groups in real world settings.First, we discuss the motivation and research questions for this study.Then, we present work that is relevant to brainstorming in context.We then describe the research site, Microsoft Corporation, and the employees that use brainstorming to complete their tasks.After, we discuss the data collection and analysis methods.We then discuss preliminary findings from the data analysis.We highlight the key contributions of this research.Finally, we discuss future work that would further enhance the research.

Most of the past attempts on improving brainstorming productivity by inducing process and
Brainstorming Under Constraints -Why Software Developers Brainstorm in Groups Shih Venolia Olson technology modifications have been shown to be successful in laboratory studies but have failed to gain acceptance in practice.To date, interactive group brainstorming in face-to-face settings continues to be the preferred method.Researchers attributed the failures to the lack of support for group well-being and member support (Dennis & Reinicke, 2004).The lack of support for social needs resulted in lowered overall satisfaction and perceptions of group effectiveness when using the EBS systems (Mullen, Johnson, & Salas, 1991;Stroebe, Diehl, & Abakoumkin, 1992).However, besides the social benefits, little is known about how brainstorming is appropriated and adapted to fit a variety of different contexts.There have not been many studies that focus on detailing why and how brainstorming is actually practiced, rather than how brainstorming practices deviate from the prescribed rules.Furthermore, research has shown that people who produced the fewest ideas produced about the same number of outstanding and high quality ideas as the people who produced the most ideas (Briggs, Reinig, & Shepherd, 1997).
Therefore, there appears to be more than just the sheer performance measurements such as quantity and quality at stake when practitioners view brainstorming efficiency.Assessing the underlying intention and understanding brainstorming practices in real settings is essential for developing brainstorming productivity metrics that matter to practitioners.
Since brainstorming is a technique that is designed to support the inherent needs of generating better ideas, we want to understand idea generation from the perspective of its practitioners.This work focuses on a naturalistic field study of brainstorming practice in the domain of software engineering.The observational study attempts to address the following research questions: Why do software teams continue to brainstorm in groups?Is it because they strive for idea quantity, quality, or a combination of the two?Are there other external factors that are not typically measured by the traditional brainstorming productivity metrics?If so, what contextual requirements do software teams fulfill in the group brainstorming sessions?How do software teams brainstorm?Do software teams follow the brainstorming rules in their brainstorming sessions, or do they practice brainstorming differently?Given that the software industry is one that focuses on technological innovation and that the employees tend to be early adopters of technology, why is there not a collaborative brainstorming tool that has been successfully developed and adopted?Are there technology or process changes to the brainstorming process that would be helpful for this population?
These questions can lead to understanding the factors that influence brainstorming behaviors and best practices in organizational settings.By taking the different goals, expectations, and norms of users and those in the organizational ecosystem into account, we are hoping to discover the underlying values of organizational brainstorming practices that are essential for making idea generation more effective in different parts of the software development cycle.

RELATED WORK
As the "illusion of group productivity" continues to puzzle brainstorming researchers, many have attempted to resolve the mystery in laboratory experiments.
Researchers have uncovered important social benefits such as active social interaction, criteria negotiation, and social comparison-the act of being exposed to a high number of ideas and to common ideas, and found that they enhanced the generation of additional ideas (e.g., Dugosh & Paulus, 2005;Shepherd et al., 1995).Rietzschel et al. (2006) questioned the effectiveness of reducing evaluation apprehension by postponing idea evaluation and compared the differences between quality and quantity of ideas generated in interactive and nominal groups.Their results indicated that without actively engaging in group discussions-though nominal groups are capable of generating higher quantity ideas that are more original-the ideas were generally less feasible than those generated by interactive groups.More importantly, the final decisions made from ideas generated in both nominal and interactive groups were of equal quality.Therefore, higher idea quantity resulting from following strict brainstorming rules is not sufficient to lead to better solutions, and groups may choose to continue brainstorming in interactive groups because the inconvenience brought by the imposed process changes does not outweigh the perceived performance gain by the group.While these prior studies can be used to explain why brainstorming practitioners choose to brainstorm in groups than alone, the performance metrics still focus strictly on idea quantity and quality.Other hidden yet dominant factors that continue to drive groups to brainstorm interactively still remain to be uncovered.
In the organizational context, the word "brainstorming" has been appropriated to fit a variety of different idea generation needs (Isaksen, 1998).Sutton and Hargadon (1996) published an influential study on the brainstorming practices at IDEO, an internationally renowned product design firm.Deviating from the quantity-and quality-centric

Brainstorming Occurrences
In order to understand the status of brainstorming usage at Microsoft's Redmond campus, we sent out a short survey to 100 randomly selected software developers asking for their brainstorming practices.We received 42 responses, a 42% response rate.The results indicated that a majority of the software teams have participated in brainstorming meetings (95.2%).Our interviews also revealed that most of the software developers have received formal training on the brainstorming process as described by Osborn (1953).The software developers also noted that brainstorming meetings served a very particular purpose and were unique from other early-stage activities such as design and storyboarding meetings.
As Table 1 shows, although brainstorming is considered to be an early-stage activity that typically occurs in the beginning of the software development cycle, the activity actually takes place across all software development stages.In terms of frequency, over half of the developers brainstorm on a weekly basis, and over three quarters of the developers participate in brainstorming meetings at least once a month (Table 2).

Data Collection
Initial informal interviews and email exchanges were used to build relationships with the software teams and to understand the basic brainstorming practices at Microsoft.Knowledge learned from the informal discussions was used to identify groups that are representative of the software development process.We conducted an in-depth case study by selecting one team from each of the software development phases and observing each team complete a brainstorming topic in its entirety over time: Planning.PMs meet daily over a span of a week to brainstorm about feature specs Implementation.SDEs meet biweekly over a span of 8 weeks to brainstorm about an architecture plan for a security update platform.
Stabilization.SDETs meet once per milestone to brainstorm about possible causes and solutions to software bugs and strategies for supporting products on different operating systems.
Overall, we observed about 20 hours of brainstorming sessions.All meetings were video and audio recorded.Informal follow-up interviews were conducted to clarify intentions, job specifications, and other technical details.

Data Analysis
Both quantitative and qualitative methods were used in analyzing the collected data.All recorded meetings were transcribed.The transcripts were first coded using the grounded theory approach to see if any general themes emerged from the brainstorming conversations (Strauss & Corbin, 1998).We then compared the emerged categories with existing literature in order to correlate and cross-validate our findings.We found that the Issues-Alternatives-Criteria coding scheme developed by Olson et al. (1992) for analyzing coordination in design meetings of software development teams had many overlaps with our meeting coordination categories, and that the creative process stages described by Amabile (1996) best described our task categories.Therefore, we constructed a coding scheme based on our analysis while borrowing the language used in prior literature.The coded results were used to compare brainstorming activities across different software development stages.The coding scheme is described below (Table 3).

Theme
Coding Category

Examples of Activity
Task Gathering information or resources.This includes looking for requirements and background information that are essential in understanding and resolving raised issues.
Generating ideas.This includes proposing solutions and alternatives.
Evaluating, modifying, and communicating ideas.This includes evaluating whether the proposed idea is suitable for the issue at hand.
The group coordination focused activities contain the following categories: Logistics.This includes scheduling, delegating tasks, keeping time, and other activities not directly related to the brainstorming topic.Recap and Scenarios.This includes restating and summarizing the current ideas, as well as scenario walkthroughs in order to make sure all meeting participants are on the same page Miscellaneous.This includes digressions, jokes, and other activities that are not relevant to the categories listed above.
In order to check for coding reliability, a 30-minute session was coded by two researchers.We achieved high degrees of reliability in our coding (Cohen's Kappa = 0.75).The agreed coding scheme was then applied to the rest of the brainstorming sessions.We describe the findings in the sections below.

Time Spent on Activities
The coded conversations are broken down into the following categories listed in Table 4. Overall, software teams spent the least amount of time gathering information about the brainstorming topic.This is due to the fact that most members of the team are domain experts and specialists who already have prior knowledge and experience in dealing with the issues under investigation.In their brainstorming meetings, time spent on gathering information typically involved searching for existing emails, agenda list, and feature specs.Those were then projected onto the projector screen in the conference room.However, the projected information was only meant to be used as a reference to guide the discussion.If a piece of information was deemed to be important, it was usually forwarded to other team members following the meeting.Therefore, the conversation in the brainstorming meeting tended to be fluid, and software team members often responded to raised issues without the need to look up additional information.
The category that had the most disparity across the software development phases was problem identification.This was because of the nature of the tasks that PMs, SDEs, and SDETs face at different phases of the development process.In the planning phase, PMs arrived at the meeting knowing the topic of focus, thus they spent more time on building scenarios and use cases when they generated ideas.In the implementation phase, SDEs broke down the features into multiple problem spaces and attempted to tackle each problem one after another by walking through the necessary components in order to make sure that the proposed architecture accommodated all project requirements.In the stabilization phase, SDETs went down a long list of bugs and support issues and discussed potential solutions for them.
Accounting for over a third of the brainstorming meetings, evaluation stood as the dominant category of activities across all three phases of software development.These preliminary findings suggest that on top of the two primary decisionmaking phases-idea generation and idea selection-the brainstorming participants spent over a third of their time on "constraint discovery".This was in clear violation of the brainstorming rules that demand participants to postpone evaluation until after the idea generation phase.In fact, teams only spent about 12% to 20% of their time generating ideas.Since the participants all received formal training on the brainstorming process, two questions remain to be addressed: (1) what drove them to conduct brainstorming in this fashion, and (2) why do they continue to call the deviated idea generation process "brainstorming"?We will address the first question in the remaining sections of this proposal and delegate the second question to future work.

Constraint Discovery
In our observations, software teams spent the majority of their time seeking clarifications that would more clearly define the problem space, identify resource limitations, and discover constraints that had not previously been discussed.These constraints came in a variety of different flavors.In order to understand the activities coded under the evaluation category, we separated the constraints into "functional constraints" and "practical constraints" and discuss them in more detail below.

Functional Constraints
The functional constraints included domain expertise, past experience, and tacit knowledge about the topic.When a question was raised in the brainstorming meeting: A domain expert could shed light on the capabilities or limitations of a certain application platform.
A developer who had prior experience dealing with a specific API could provide insight on the applicability of the API on the current feature set.A program manager who had been informed on a requirement modification could update the group on last minute changes.
Functional constraints are the bread-and-butter of a successful brainstorming meeting.Without them, there would be no way to shape the brainstorming discussions.

Practical Constraints
The practical constraints are limiting constraints that are associated with the lack of human resources, funding, time, or stakeholder interests.These factors are typically used to discount a series of potential ideas that may be useful but is not of primary concern to the group due to more pressing issues.

Functional Constraint Example (Implementation)
In this episode, a team of SDEs were trying to decide whether a top-down or bottom-up way of retrieving relevant system data objects was more suitable for the problem.The two software developers discussed about why the existing process of combining both approaches was applicable instead of going with solely a single approach.
P1: Well, we know that there are two ways of filling data up and usually we start with bottomup and then go top-down and that's how it works P7 acknowledged that while a visual prototype was a superior choice in generating more concrete feedback, but an impending deadline, a practical constraint, was keeping them at bay and was forcing them to settle with a quicker and less time consuming approach that would hopefully generate enough feedback for their deliverables.

Practical Constraint Example (Stabilization)
A group of SDETs engaged in a discussion about whether to remove test support to the feature of a product.Accompanying the discussion, P11 mentioned the need to remove even more test support on other features because there were not enough resources to fulfill the existing assignments.The practical constraint of not having enough manpower to support all the proposed features was the driving force for prioritization.
P8: So, we can remove our custom proxy now?P9: Um-hum.P10: It has been removed.With the [product name] conversion.We tossed that out.
P11: So, one thing that -sorry, we're jumping to a different topic, but one thing I feel that we need to think is, if we always do additive -things that add, right?Like we were saying, we need to do this, we need to provide that.We need to add one more component.I think that one thing we need to look into is, how do we trim down what we have and support?Like, what -it's clear that -the resources that we have are bounded.

SUMMARY OF EMPIRICAL RESULTS
As illustrated from the field data, brainstorming in the context of software development is about discovering constraints-drawing the proper boundaries to ground the ideas.However, the high dependency on using constraints to shape the idea generation process contradicts with the fundamental notion of brainstorming, which is to reduce evaluation apprehension by postponing idea evaluation until a later stage.The contradiction between the prescribed brainstorming rules and actual deployment in practice is particularly challenging for brainstorming practitioners.One developer commented that: The most difficult part most of the time seems to be keeping the discussion within the realm of true brainstorming.As far as I understand, heavy discussions of implementation, ramifications, etc. of an idea fall outside the bounds.On our team, at least, we tend to think of an idea, then discuss its feasibility for a bit, then maybe come up with another idea, then discuss for a bit.It seems difficult to keep the idea generation the focus.
This divide can be explained by more succinctly examining the role of constraints in brainstorming meetings.Due to the complex nature of software development, most large-scale software projects are faced with overly constrained problems.When members of a software team meet in a brainstorming meeting, each member brings a view of non-overlapping constraints that the project must satisfy.The primary purpose of the brainstorming meeting in the context of software development at Microsoft is essentially to identify all possible constraints and to find the right balance to satisfy the most important ones.When asked about his brainstorming experience, one program manager made the following remark: Coming up with ideas is not hard, but being able to discover the proper constraints that help shape those ideas and implementing the most beneficial ideas is the difficult part.
Although both are important from the business operation perspective, functional and practical constraints play different roles in shaping ideas.Functional constraints are value-neutral constraints that are necessary for bounding the idea within the desirable context.On the other hand, practical constraints such as time and resources that must be met in order to ensure the survival of a business hinder the creativity of the problem space and room for trial and exploration.Therefore, the brainstorming process could use a helpful amendment that would require brainstorming groups to state their functional constraints upfront without having to worry about the practical constraints.Furthermore, providing a communication channel to negotiate criteria and incentives in evaluating idea quality as a group in the idea generation process could benefit the group decision-making process as a whole.
Researchers on other creativity processes have also started to analyze the role of constraints in early-stage design meetings (Onarheim & Brainstorming Under Constraints -Why Software Developers Brainstorm in Groups Shih Venolia Olson Wiltschnig, 2010;Stacey & Eckert, 2009), and our findings contribute to the existing literature.Overall, our findings provide the following contributions to a relatively scarce body of brainstorming research in the field: From the theory perspective, the study generates empirical results that could enhance the current idea generation model by accounting for constraint discovery, which is technically forbidden by formal brainstorming rules.
From the system perspective, the design recommendations generated from the findings could result in innovative collaborative brainstorming systems.Rather than simply allowing the participants to input their ideas, systems should provide a platform to scaffold the constraint discovery process.
From the experimental design perspective, the findings could generate more naturalistic tasks that incorporate constraint components to better emulate real world scenarios.The constructed tasks would allow experimental research in the laboratory setting to more accurately measure and predict brainstorming's impact.

LIMITATIONS AND FUTURE WORKS
The results of this work identified two types of constraints that occur in brainstorming meetings.Functional constraints are requirements and criteria that define the idea space, whereas practical constraints are limitations that prioritize the proposed solutions.However, there still remains a significant amount of work to be done before these preliminary findings can fully represent the general brainstorming practices.The field study was conducted at Microsoft, which is known to be peculiar in many of its software practices (Cusumano & Selby, 1998), and may not share the same design methodologies, development practices, and culture with other large software companies, smaller start-ups, or open-source communities.
Two eminent questions immediately follow the results of our study.First, why do software teams continue to call these meetings "brainstorming" even if they choose not to follow the conventional brainstorming rules?Is it simply a matter of convenience?This could potentially be a deeper issue, especially because the software teams that we have observed were all familiar with the conventional brainstorming rules.It is important to unpack the meaning so we can more effectively design for the purpose of brainstorming meetings and better incorporate the findings into future brainstorming models and systems.We plan to develop a survey that could be used to validate the empirical findings based on observations of the software teams at Microsoft.Second, can the brainstorming model generated based on the software development practices at Microsoft be generalized to other software communities?We will survey other software organizations in order to verify these findings on a more general level.

Table 1 :
Brainstorming Under Constraints -Why Software Developers Brainstorm in Groups Shih Venolia Olson Brainstorming Occurrences across Different Stages of Software Development

Table 2 :
Frequency of the Brainstorming Meetings

Table 4 :
Time Spent in Meetings right now.So, we can stick to that.I don't see a problem with that process.P2: Right.Another thing is that certain things, you don't have any choice.For certain things you need to do bottom-up.