Collaboration Constraints for Designers and Developers in an Agile Environment

The relationship between developers and designers is an important factor for successful UserCentred Design (UCD) work in an Agile Software Development (ASD) process. However, the determinants and long-term barriers for successful ASD in this relationship are still unclear. We asked designers and developers in an online survey about their roles, perceived level of ASD implementation, satisfaction with ASD, teamwork satisfaction, and perceived quality of communication and collaboration with the other role. Developers rated the main factors for successful ASD work as having access to designers and an optimal environmental setup. Designers’ main concern was related to improving wider teamwork, in particular sharing of information. A qualitative analysis of respondents’ comments confirmed these results broadly, and indicated that close cooperation and informal communication between designers and developers was desired. Respondents also perceived senior stakeholders’ hierarchical decision-making process as a barrier to successful ASD. Co-location and pairing of designers and developers was therefore also seen as an opportunity to enhance more localised and autonomous decisionmaking.


INTRODUCTION
User Centred Design (UCD) and Agile Software Development (ASD) are two relatively recent advances that have significantly changed the way we design and build software. 'Agile' is a summary term for a process where the requirements (e.g., the users' or functional requirements) are addressed by applying solutions iteratively through collaboration between self-organizing, crossfunctional teams (Dybå & Dingsøyr, 2008).
ASD promotes adaptive planning to counter the perceived shortcomings of traditional plan-driven methodologies, and encourages delivery of early versions of solutions that get continuously improved. This way the team is thought to be able to respond to changing user requirements or business needs. Agile teams strive to deliver an early and fast production of working code and make frequent and incremental changes. This is often achieved through paired programming in short iterative cycles, with contingent user feedback. Importantly, it requires a high degree of collaboration and shared decision-making (Drury-Grogan, & O'Dwyer).

Agile and User-Centred Design
Agile methodologies and its variants (Dybå & Dingsøyr, 2008; see Inayat et al., 2015) are often perceived to be at odds with User-Centred Design (UCD) techniques (Salah, Paige and Cairns, 2014). UCD places the user and their requirements at the centre of the design process, aiming to achieve a positive subjective experience and high objective performance with the interface.
Researchers and practitioners in UCD have developed methodologies, techniques and processes that enable design teams to create, prototypes and test solutions before developers (programmers) are involved. UCD has been found to positively impact the results of the design and development efforts by reducing customer complaints, training needs, and increasing uptake of resulting products (Bias & Mayhew, 2005).
In a recent systematic literature review (Salah, et al., 2014) on Agile and UCD integration a number of main issues are identified that impact on design and development work. First, ASD promotes the elimination of much of the up-front planning work to remain responsive to changing requirements. This means that there is little time for the usual research, analysis of requirements or any elaborate prototyping characteristic in UCD. Another issue is that the UCD work is not easily divided up in 'chunks' to fit the agile work practice. The lack of pre-planning of defined design goals makes determining the size of design chunks difficult. Furthermore, designers usually take a more holistic view on the interaction design and information architecture of a website or product, so therefore modularization and iteratively adding features may be averse to their way of thinking and working.

Designers and Developers
The focus of the current investigation was therefore on designers' and developers' working relationship in integrating UCD and ASD. Salah et al. (2014) found in their review that the work-dynamic and relationship between designers and developer changes in an ASD setting. This is true in particular for designers, as their job requires them to be "on call" and supply ad-hoc solutions, reviews, and feedback in a team-oriented design process such as ASD. The importance on "working software" as the main yardstick for the design and development progress is a challenge for the designer-developer relationship.
To date many studies (Dybå & Dingsøyr, 2008;Salah et al., 2014) address possible barriers to a successful ASD implementation, but there are only few studies on how these two crucial roles interact and collaborate. Brown, Lindgaard and Biddle (2011) observed that much of the interaction time between these roles was used to "re-align" individual work progress to ensure a common understanding of the project aims and ensure product development plans were on track. Ferreira, Sharp and Robinson (2012) found in ethnographic studies that successful integration of Agile and UX work relies on attitudes and work practices such as mutual awareness, expectations about acceptable behaviour, negotiating progress and general engagement with each other. But there is a lack of rigorous insight or evaluation (see a review by Jurca, Hellman, & Maurer, 2014) whether and how designers and developers differ in their attitudes and practices, and how their co-operation is determined by organizational structures and decision processes. The current study therefore investigated designers (user experience designers) and developers (front-end programmers) who are already supposed to be working in agile. The aim was to learn how their roles differ in the ASD process and how they perceive its outcomes.
In particular, the study aimed to explore whether designers and developers had different perspectives in terms of how ASD works in their organisation, how well the wider team (including other roles, e.g. business analyst) worked together, and whether there were specific issues in designerdeveloper collaboration and communication that would impact on the current processes, their goals or objectives. To do this we created an online survey following some initial qualitative interviews with stakeholders from the various teams (see methods).

Participants
The study took place within a large organisation based in the United Kingdom. It includes several very large online products with an output of editorially steered content alongside traditional TV and radio broadcasting. It is structured into audience facing product teams, each with their own set of designers and developers. These disciplines work alongside business analysts, testing specialists and project managers, all with varying experience and responsibilities. The disciplines within the teams report to a set of product stakeholders such as product managers, technical leads and creative directors.
The participants were designers and developers from within the organisation. Their work experience varied within their role: For example, some were Junior Designers and some were Senior Designers, yet all were classed as 'Designers'. There were 109 participants, (24 women) in the sample. We asked for a broad job role description to classify them as designers (n=54) or developers (55). We did not ask for their age.
To select participants for the questionnaire we chose a sampling method known as 'purposive sampling'. This enabled us to focus our study on two roles in the organisation, designers and developers. To gain suitable participation and the appropriate amount of quantitative data for this study, designers and developers from across the organisation were asked by email to complete the online questionnaire.
All of the product teams which took part in the study have adopted the agile process to varying degrees of flexibility, some involving mixed approaches to Scrum and Kanban methods.

Measures
We employed both quantitative and qualitative methods with the goal to gain a broad insight from across the organisation and its teams and disciplines. Before we started data collection with designers and developers we interviewed nine stakeholders from the organisation (and one external expert on ASD). Together with the background research from the literature, we used the results of the interviews to derive a set of questions for designers and developers that would tap into commonly observed issues in agile collaboration. The data from the senior stakeholders are not reported here.
Data was gathered using an online questionnaire tool (Typeform -http://www.typeform.com) that provided user-friendly participation across any device and within a single URL. Most questions were answered using the Likert scale. We used a 7-point scale ranging from "Strongly Disagree" to "Strongly Agree" with the middle option of "Neither agree or Disagree".
First, there were basic biographic questions, including on the length of employment, including a question on their perceived level of ASD knowledge. There were then 5 items to rate on how well ASD was implemented across the organisation and teams: how well ASD was working in their team, in the whole organisation, in their own product area, how well it served purposes such as achieving successful Responsive Web Design solutions, and to what degree ASD could be improved.
Then followed 4 validated questions on teamwork which were taken from Lurie, Schultz, and Lamanna (2011). Participants were asked to what degree they were encouraged to share ideas, whether they had enough information to do their job well, if the team members make a real effort to understand work-related issues and problems, and to what degree they felt able to act on a team vision (a 5 th question item on leadership from the original scale was not used). This was followed by 5 questions specifically on the working relationship and collaboration between designers and developers. The questions -derived from interviews with senior stakeholders and literature review -covered to what degree the two roles worked closely enough together, how productive the working relationships were, if the two disciplines contributed equally and finally if designers and developers had similar skills. Each of these survey sections could be commented on in free form as well.

Quantitative analysis
Of the 109 responses, 2 were not analysed as the respondents identified themselves as business analysts or senior management respectively. The remaining respondents were 52 who described themselves as working as designers and 55 working as developers. Responses were transferred into Excel and SPSS for further analysis. Raw data were screened for missing values and possible outlier influences. First, length of service (in days) and familiarity with ASD concepts were analysed for group differences. There was no difference in length of service, t (105) <1, with means of 814 days (SD = 852) for developers and 577 days (SD = 844) for designers. However, there was a significant difference in the amount of perceived knowledge about ASD concepts. Developers scored significantly higher, t (105) = 4.02, p < .001 (M = 6.5, SD = 0.57) than designers (M = 5.8, SD = 1.23).
The main quantitative analysis was applied to the rating scores (raw scores ranging from 1 to 7). This covered the 3 main sections of the questionnairesatisfaction, teamwork, and collaboration. These ratings were analysed using a multivariate approach to control for inflated Type-1 errors of significance testing. There were two approaches to the quantitative analysis. First, because we had many variables that may differ between the two groups of participants, we used a multivariate analysis of variance (MANOVA). In a MANOVA one can compare whether groups of participants significantly differ in two or more quantitative variables of interest without having to perform multiple simple group mean tests (which would inflate the risk of appearing to detect a statistical difference where there was none).
The second was a multiple regression approach to test which of a number of quantitative variables correlate significantly with an outcome variable. Here, we were interested to which degree variables such as length of employment, agile knowledge, team work satisfaction, quality of work environment, and quality of teamwork would correlate with (and therefore predict) overall satisfaction with the agile development process.

Analysis ASD satisfaction
To explore the effect of the factor participant group (designer vs developer) on the five dependent variables of the category "ASD-satisfaction" (ASD in team, ASD satisfaction overall in the organization, ASD satisfaction within respondent's product area, ASD fit with design goals, and to what degree the process could be improved) a MANOVA was performed. Dependent variables were the five scores on satisfaction ratings as dependent variable (ranging from 1 to 7) with group (of participants) as fixed factor (Box's assumption of equality of covariance matrices was supported, p = .17). There was no main effect for the factor participant group, Wilks' λ = .984, F(4, 102) < 1. Including Sex as an additional fixed factor (2-way MANOVA) did not change this result, Wilks' λ = .972, F(4, 100) < 1.
Thus, the two groups did not differ in their perception to what degree ASD is already applied in the development process, both on team level and in the organization as a whole, and with what success (see Figure 1). Both roles thought that there was significant scope for improvements of the ASD process.

Analysis ASD teamwork
A further MANOVA was run with the items measuring teamwork (sharing ideas, having enough information, effort in problem-solving, team vision, Lurie et al., 2011) as independent variables. There was a main effect for the factor participant group, Wilks' λ = .902, F(4, 102) = 2.77, p = .031, partial eta2 = .098 (Box's assumption of equality of covariance matrices was supported, p = .12).
Follow-up ANOVAs (analysis of variance) for the factor participant group revealed only a trend for an effect of the dependent variable shared teaminformation, F (1,106) = 3.91, p = .064, with developers perceiving to have more relevant information (M = 5.36, SD = .98) than designers (M = 4.98, SD= 1.12).
In a separate analysis of one item that did not fit directly into the teamwork aspect there was a significant difference between roles in the perception of how well the physical environment is set up to support collaboration in ASD, t (105) = 1.95, p = .019; developers rate this higher (M = 4.27, SD = 1.56) than designers (M= 3.6, SD = 1.27). Figure 2 shows the pattern of results.
Overall then, there was some difference on perceived quality of teamwork between designers and developers, with designers rating this slightly lower (in particular regarding perceived information sharing and to a lesser degree shared effort).

Figure 2:
Percentage of participants in both groups responding "agree" or "strongly agree" to questions of familiarity and level of perceived implementation of ASD.

Analysis cooperation and collaboration
Finally, a two-way MANOVA was planned with the variable group (of participants) on the set of questions asking about the perceived quality of cooperation and collaboration between designers and developers. However, Box's assumption of equality of covariance matrices was not supported, p = .005). Therefore, the MANOVA could not be run. Figure 3: Percentage of participants in both groups responding "agree" or "strongly agree" to questions of collaboration between Designers and Developers in ASD Nevertheless, multiple t-tests with the betweenfactor group revealed a significant difference between ratings for the items perceived equal contributions, t (105) = 1.95, p = .05; developers rate this lower (M= 4.32, SD = 1.70) than designers (M= 4.90, SD = 1.33). Also significant was the perception that designers use developer's skills, t (105) = 2.79, p = .008; developers rate this lower (M= 2.96, SD = 1.36) compared to designers (M= 3.69, SD = 1.42). However, these results are based on multiple comparisons, and only the difference in sharing skills would survive a Bonferroni correction and should be considered reliable. Figure 3 shows the pattern of results.

Modelling the ASD satisfaction scores
In a final set of analyses we employed a multiple linear regression analysis to investigate possible relationships between the tested variables (questionnaire items) in the two groups. This correlational approach was used to uncover possible associations between our variables that were not evident in the comparisons of means between groups.
First, we summarised the item groups into subscales by averaging scores (ratings). The averaged score on the questions on ASD (satisfaction and perceived level of implementation, Figure 1) was the dependent variable. The average score from the 4 questions on 'teamwork' (Lurie, 2011; Figure  2) was the first independent (predictor) variable, and the second variable was 'collaboration', which was the averaged score derived from the 5 questions on the specific working relationship between designers and developers ( Figure 3).
All these groups of questions were submitted to a reliability analysis and revealed Cronbach's alphas of > .70, which is sufficient to be used as a subscale. Finally, we had scores from single items as further independent (predictor) variables: length of experience in organisation, knowledge about ASD, and perceived quality of the environmental setup at the work place (in regards to facilitating ASD, e.g. scrum boards).
Using the Enter method, these five variables were entered into separate regression analyses for designers and developers. In both groups the regression model predicted a significant amount of variance for the agile-process score: For developers, the adjusted R square = .295; F(5, 53) = 53, p < 0.001; for designers, the adjusted R square = .369; F(5, 50) = 53, p < 0.001.
Inspection of the multi-collinearity indicator (VIF) revealed that all values were acceptable (between 1.07 and 1.4). Significant predictor variables for the satisfaction with ASD in developers were the the variables environment, beta = .266, p = .30, and collaboration (combined questions), beta = .492, p < .001. For designers the only significant predictor variable was teamwork, beta = .551, p < .001.
Thus, whereas for developers the main factors for a successful ASD process implementation were environmental setup and collaboration with designers, for designers the main predictor was the quality of the (wider) teamwork. These findings are striking because of the lack of overall statistically relevant group differences between designers and developers in their assessment and perception of ASD processes in the organisation (see 3.1.1 and 3.1.2). These points will be discussed in context of elicited comments from both respondent groups.

Qualitative analysis
The free-form comments participants gave in Typeform were also analyzed using a qualitative approach. The comments were imported into a software tool for analyzing qualitative data called NVivo (www.qsrinternational.com).
We coded expressions of opinions, problems, events, reactions and interactions in the text by assigning the piece of text to a category ("node" in NVivo). A category represents a phenomenon, that is, a problem, an issue or an event that is defined as being significant. When categories were found to be conceptually similar in nature they were grouped under more abstract, higher-order categories. Finally, we then used NVivo to create connections between categories and their subcategories.
Following the analysis of the Qualitative data from the questionnaire, we grouped the 'phenomena' into the following categories. These categories are described below with a selection of comments that supports each:

Theme category 1: Collaboration issues that impede ASD
(i) Design work was perceived by both roles (designers and developers) to be happening too much in a plan-driven way (i.e. "upfront" before sprints) with the consequence of developers feeling they cannot engage or contribute early enough. "There seems to be a significant disjoint between the functions required for a leaner approach to UX required in the agile environment." (iv) Developers asked to be more involved in the ideation phase and involved earlier in the process.
"Would be helpful to see designs (even just wireframes) earlier in the process." "Frequently devs are not able to take part in the ideation process because they are assigned to other work -the Product team require them to continuously write code and build stuff rather than getting involved in the early design phase." "I have worked on one project when the dev was involved in the design phase and this was a great success as he contributed some really useful ideas and was able to create prototypes using real data." "Nothing quite beats sitting with them and talking face-to-face and sketching ideas and thoughts together." "Historically developers haven't had a chance to participate in the "design" process." Theme category 3: Perceived need for localised decision-making to enhance ASD process.
(vii) There was a strong desire for more joined-up thinking and discussions between rolesenhanced through regular pairing between designers and developers. Whereas developers were usually co-located in one team, designers often were required to move between different teams and projects.
"For me, the only way to successfully build great responsive products in an iterative way is for the developer and designer to collaborate on the product itself (i.e. the production web site). This means sitting next to each other, and focusing most of their time on the product itself and less of their time on static mockups/prototypes." "I think the more designers and developers work closely together, the better. If designers were colocated with project teams, that would be incredibly useful." "Development and Design would work better if they sit in the same space and also if there's 7 and effort from both teams to initiate a conversation." "The relationship between design and development could be improved by involving UX into the agile process and making the agile process more designer friendly." "Whilst I pair with developers and go to their stand-ups on a daily basis, it still feels like we (designers) work in more of a waterfall manner. How can we integrate more fully with their process and vice versa? -It currently feels very much like a 'them and us' situation." "Working in a Design & Dev pair on every new feature from the beginning works great." (viii) Finally, there was the theme of frustration about the "sign-off" culture and decisionmaking in the organisation. Due to the structure of the teams, there is a clear hierarchy of product owners who make decisions, preventing the teams from being autonomous.
"There is also a major delay in decision makingas in, this holds up projects because people seem reluctant to make decisions on things." "I feel there are too many stakeholders and managers involved to work in an agile way. Getting sign off when there are so many people involved and juggling so many conflicting interests and ideas can be time consuming." Overall, the qualitative analysis of the comments reflects and explains the particular differences we observed between designers' and developers' responses to the questionnaire. For example, one observation was that developers thought the environment is better suited to ASD than designers did, and the same holds for perceived team information. This seems to be because developers spend more time in their designated team, whereas designers move between teams and products.
Similarly, designers thought more than developers that they shared skills with each other. Developers may not be aware of designers' efforts to code (e.g., for prototyping). Developers seem to notice more than designers that productivity was suboptimal when work was not done in physical collaboration, and would prefer early involvement in early design stages and testing. Designers felt more out of the loop than developers in terms of wider team information, possibly because they change teams more often. Both roles thought that for effective ASD processes decision-making was not localized enough to the level of the teams.

DISCUSSION
The current study focused on the collaborative relationship between designers and developers in a large media organisation working within an agile development environment. The results of the survey and the qualitative analysis confirm some of the previously reported factors of successful ASD implementation, but additionally show which of these moderating variables vary across the two roles.
There were significant differences in how designers and developers perceived the factors for successful teamwork and collaboration. This is despite the fact that designers and developers worked in the same organisation (and overall location) and although they appeared to be generally aligned on many questions around agile processes and its current implementation.
Developers' satisfaction with ASD correlated with access to and collaboration with designers, as well as the environmental (physical) setup at work. Designers' satisfaction with ASD, however, was associated with the perceived quality of teamwork. Qualitative analysis suggests that these factors for agile satisfaction were for both groups associated with the level of pairing of roles (and increased physical co-location), adoption of a more iterative workflow (including design iterations) and importantly a more localised decision-making process. These requirements are in contrast to the experienced delayed and removed 'sign-off' process inherited by senior stakeholders. This aspect of the current process is of course almost a direct contradiction to purist conceptions of ASD, which demands self-organising teams that can take decisions and drive development largely autonomously.
Interestingly, the widely reported moderator of differences in experience with ASD (e.g., Drury-Grogan & O'Dwyer, 2013;Serrador & Pinto, 2015; see also Vijayasarathy & Turk 2012), does not seem to have any impact on our measures here: both the length of work as well the explicitly elicited (self-assessed) knowledge of ASD had no significant effect on moderating satisfaction with the development process.
The observation that we did not find an effect of length of employment or ASD experience -at least on the quantitative data -is important. It may be due to the fact that in our sample designers and developers were often already working in agile teams (and possibly due to training in the organisation), therefore on average their exposure or knowledge was already close to ceiling. In other words, our targeted sample of already rather dedicated agile operators may have highlighted more persistent issues that are to do with often intractable barriers to ASD inherent in organizational structures and culture.
Other research finds -similar to our observationsthat barriers to successful ASD reside in a crucial component of the agile philosophy: autonomy and localised decision-making. Drury-Grogan and O'Dwyer observed in their qualitative study (focussing on team meetings) that some team members influenced the decision-making due to their seniority or experience. Serrador and Pinto found that team experience (together with moderators such as quality of vision and complexity of projects) affected outcomes and stakeholder satisfaction. Unlike other research (see a review by Jurca, Hellman, & Maurer, 2014) our teams did not perceive UX work as optional, which may be due to organisational emphasis on UX work and general changes of attitudes in the industry over time.
A particular difference between designers and developers was their perception of having enough information for successful teamwork. Designers rated this lower, possibly because they were often not constantly embedded in a team, or not colocated with their team members. This is probably also the reason why in the regression analysis teamwork was an important predictor of ASD satisfaction for designers, but not for developers. As designers usually spread their time across more projects than developers, there may simply be more opportunities to feel having not enough information in any of them. From the respondents' comments it seems clear that a solution resides in communication and collaboration, ideally requiring designers and developers working physically very closely together, or finding alternative means (e.g., remote communication technology). This also fits with previous observations for a need to frequently 're-align' work processes and product development plans (Brown et al., 2011) and that integration of ASD and UX relies on frequent negotiation between these roles (Ferreira et al., 2012).
Another barrier in the working relationship between designers and developers -which we have not addressed here -may also lie in their different personalities. There are reports of differences in personality and style within software development teams (Capretz & Ahmed, 2010). Acuna, Gomez, & Juristo (2009) found that when student teams adopted Extreme Programming (XP) they decided on their own type of cooperation and they experienced the least conflicts and showed higher levels of job satisfaction. However, it is not clear whether software engineers are different from other groups. Beecham (Beecham et al., 2008) found in a review of 92 papers that just half of studies report that engineers are distinguishable from other occupational roles in terms of motivation. Vijayasarathy and Turk (2012) emphasise the importance of 'enabling factors' such as training and setting norms in the agile environment are important for its success.
One limitation of the current research is that the results appear at first not to be readily generalizable to other environments, which is often the nature of applied research. However, we obtained data from designers and developers over a wide variety of online content, and across a number of teams employing a variety of agile styles and typologies. Furthermore, our results reflect and add to findings from other studies on factors for successful ASD: the crucial role of decision-making (Drury-Grogan & O'Dwyer, 2013); providing opportunities for teamwork and collaboration (Chan & Thong, 2009); the importance of adequate environmental setup (Mishra, Mishra, & Ostrovska, 2012); and the role of organisational culture and management support (Chan & Thong, 2009;Jurca, Hellman, & Maurer, 2014). Furthermore, many other companies and organisations will probably find themselves in a similar situation as the observed environment in a crucial aspect: developers may benefit from closer collaboration with (usually outnumbered) designers (Ferreira et al., 2012). Similarly, risk-averse attittudes in large organisations is common, often entailing top-down control of project work that can derail successful ASD processes. Future research may therefore aim to address how senior stakeholder's adoption of agile philosophy and the associated need to relinquish decision-making powers influences collaboration and cooperation in ASD teams. In particular, it would be interesting to track where, how and when decisions are taken in the development process, and measure their quality and outcome.

CONCLUSION
In conclusion, investigating the relationship between designers and developers in an agile development environment, we find that both groups perceive that collaboration could be significantly improved through a higher degree of co-location and greater co-operation early on in projects. In particular, the current results suggest that agile satisfaction increases with paired collaboration of designers and developers, and encouraging teams to work in iterative cycles. While some details of the reported observations and results may be specific to the current sample, we feel the diversity of teams -and high number of participants -involved in this study will likely extend our main findings to other organisations.
A novel finding was that these factors were not moderated by ASD knowledge or experience. Rather, satisfaction with ASD seems to be associated with the organisation of teams, collaboration and related decision processes. This