Playing the game: HCI careers in the competency

This paper summarises the increasing formalization of HCI-related job-roles in competency frameworks (such as the UK’s Skills Framework for the Information Age – SFIA), and the adoption of these by industry. At present these are mainly in mature areas of HCI such as usability. This trend addresses a common complaint (and one voiced at a workshop at HCI2007) amongst usability professionals that usability does not offer adequate career progression opportunities. Factors that might account for the laggardly response by the usability industry are discussed. The author’s recent experience as a reviewer of the updated version of SFIAplus v4 is reported here, with examples from this and previous versions to illustrate the process by which the HCI community can influence these role definitions. The conclusion is that, to give opportunities to our learners, to achieve the changes our community wishes to see put into practice, and to make our own work relevant, we all need to play our part in this "game".


INTRODUCTION
The rules of the game are changingwe now have rules.In many walks of life, competency frameworks are emerging that allow roles to be benchmarked against each other (to meet legal requirements such as gender pay equality), and individuals are to be benchmarked against roles.Much of this is driven by the trend to outsource human resource (HR) activities and standardisation efforts such as Reusable Definition of Competency or Learning Objective (RDCEO) [2], HR-XML [3], and Sharable Content Object Reference Model (SCORM®) [4] and movements such as the Bologna Process [5], combine to define roles, career opportunities and learning programmes, and to fill learning gaps.All of which may sound a bit mechanistic and reductionist to some in the HCI community, but these are arguably simply the rules of the game, for anyone wanting to make a living and find career fulfilment in a globalised world.
The HCI community has to start playing this game a bit better or our ideas will not gain traction.One of the more mature areas of our "body of knowledge" (BOK) is usability, one in which we are finally beginning to see the world pay heed to the longstanding rants (well-Researched Analyses of Non Task-oriented Stupidities?) from the leaders of our community over many years.
As Jared Spool [1] repeated in his keynote at HCI2007 (citing the CUE studies by Molich et al), we still need to "get our act together to define usability" (and accessibility) to ensure that we can measure it, prevent its lack, optimise methods to minimise problems.But we shouldn't be surprised that, 20 years after this community thought it had defined usability and how to achieve it, the academic knowledge remains partially implemented in practice.As Gaines & Shaw's BRETAM model [6] would have it, we are still 4-12 years away from "mass-produced" usability.Spool [1] however identifies that industry will demand many more user experience professionals over the next few years than exist at present or will graduate with relevant degrees.Now would be an excellent time to ensure that all that the HCI community have preached about usability over the decades, can finally be put into practice.
If, in theory, research and practice are the same, in practice some theorists are so abstracted from practice that they need intermediariestechnology translators.The HCI community tends to seek to understand people's needs, and so is better placed than some computing communities to handle both theory and practice, but breakthroughs in either are achieved by different means.We need to step back from HCI research for a while and think a lot more about what is HCI practice.This in turn requires us to "play the game" as work itself becomes increasingly structured and formalised, something that is a culture shock to many in academiaperhaps even the reason why some choose not to be in practice.To focus on practice is not to denigrate theory and research: every undergraduate needs to be able to engage with relevant theories and to gain basic competency in research skills.There are many inspirational reasons to provide learners with an insight into the latest HCI research.The challenge to HCI Educators is ensuring that fundamental HCI skills are learned effectively and deeply.(This then requires us to define what these fundamentals are, a debate that has bedevilled every HCI Educators' workshop I've had the privilege to attend!).
Defining competency in HCI-related roles helps us to articulate and delineate such a set of fundamental HCI skills, and this process will have a symbiotic relationship with collaborations to produce HCI learning resources, such as HCI commons, which can be seen as defining the BOK for HCI itself.

COMPETENCY IN COMPUTING
Formality and professionalism in work tends to be a sign of maturity of that work domain.It's arguable whether computing has reached this level of maturity.Defining competency means recognising and preventing incompetency.The computing industry generally has had a reputation for questionable competency (the media report a steady stream of failed large IT projects), maverick and often futile behaviour (caffeine and pizzafuelled all-night coding sessions), and a certain callousness towards anyone outside the IT department (as seen in the popular, if disheartening, UK TV show "The IT Crowd").The majority of IT workers in the UK do not have a degree in a computing-related subject.Despite more than doubling its membership, the British Computing Society's membership is vastly outnumbered in the UK profession by non-members, and professional qualifications (C.Eng., C.I.T.P., M.B.C.S.) are rarely a pre-requisite for a job or as a supplier.Thus BCS's current campaign for "professionalism in IT" may seem risible.
Professionalism may arrive sooner than many expect, riding the wave of equality legislation, which demands equal pay for work of equal worth/complexity.Academics will be familiar with the competency definitions for our roles in the UK, HERA(Educational Competencies Consortium Limited)against which we have all been ranked and graded.A similar driver exists in most fields, and the two bodies most relevant to HCI are 2 of the 25 Sector Skills Councils (SSCs) set up by UK government and employers: e-Skills [7] and Skillset [8].The former covers the information & communications technology sector, while the latter covers the audio-visual industries.E-Skills works with partners in the Skills Framework for the Information Age (SFIA) Foundation which defines a reference model for skills (SFIA) onto which the partners build detailed definitionsfor example the British Computing Society's SFIAplus, and e-Skills' procom.By comparison Skillset is a single body that integrates the views of various stakeholders.Some HCI-related competences, for example Web design/development, User Evaluation, etc have fallen uncomfortably between the two.To further complicate matters there is a lesser-known, and arguably less-developed, skills council, CCSkills [9], (formerly CCISkills) which serves the creative and cultural industries, and plans to include Design in its scope).CCSkills have announced a Skills Academy [http://www.ccskills.org.uk/Training/NationalSkillsAcademy/tabid/84/Default.aspx]for the Creative and Cultural Industries but this appears unrelated to Skillset's Academy Network (Screen Academies, Media Academies).
Inevitably each sector skills council reflects the industry as defined by the organisations that make it up and the work practices of these organisations.This is promoted as a strength by the UK government and the Alliance of Sector Skills Councils describe their membership as "employer-driven organisations that together articulate the voice of the employers of more than 85% of the UK's workforce on skills issues."[10].
What follows are brief observations of having read the publications of SFIA and Skillset over the last five years and attempted to influence them and to find opportunities within both trade bodies and academia to put the ideas into practice, but the publications of each body runs to hundreds of pages, and this is not intended to be a thorough analysis.
Skillset reflects the make-up of the audio-visual industries.These are perhaps better known to some in HCI as the "people behind the scenes in show-business", but this spreads into all forms of creative media including websites and computer games.Their competency definitions are less than a page each and reflect a more hierarchical command structure than many in IT or academia might be used to, coupled with much short term contract work and apparently rigid demarcation and specialisationa federation of specialisms.In each one, one may win an Oscarbest visual effects, best sound editing, best make-up etcbut there is a very long tail of roles like deputy wardrobe assistant to Ms so-and-so.A veteran of the industry once explained it to me that for a few there are the awards and for everyone else there is "lousy pay, long hours, insecurity".Set against this there is a "sexiness in being around even the most minor star", and there's always the belief or hope that raw talent and dedication will win just rewards.Skillset [26] reinforce this, stating "Freelancers account for 11% of the workforce in computer games, and 16-17% in web and internet and offline multimedia and are often employed on a per-project basis" and "working practices (...) tend to be characterised by: Blurring of roles; Vague job descriptions; Unpredictable working hours and long days; Frequent changes of employment."SFIA began life as the BCS's ISM (Industry Structure Model) in 1986.SFIA, and derivatives such as SFIAplus [28] and e-Skills procom [29], lays out its competency definitions in a database, and from this generates pages of description for each role at each level.SFIA v4 consists of 86 Skills, and a total of 290 Levelsusually 3-4 per skill within a scale of 7 levels (Level 1 is the entrant level, level 4 the professional "enable" level, and level 7 the strategic leadership level).From this reference model, fuller definitions are created, for different markets, in SFIAplus and e-Skills procom.In SFIAplus there is an acceptance that each job may involve more than one role each potentially at different levels.Each task (a role at a level) has prerequisite experience and qualifications, on-the-job training and personal development activities, as well as specific work activities.For example one of four work activities of a level 3 Systems Ergonomics (HCEV) is "Contributes to detailed designs including for example: user interface (including colour / language / presentation / input methods), user documentation, program specifications, and backup, recovery and restart procedures."[7,ref HCEV3].
Compared to Skillset, the overall impression is of something designed by 1980s IT practitioners!Rather than a top-down hierarchy, there is a hierarchy of experience and a multiplicity of roles.For example, there is a noticeable skew towards higher levels and a dearth of level 1 tasks (see Figure 1).The majority of tasks are at level 5 and above.

Figure 1 Distribution of SFIA v4 Roles at each Level
To contrast further, Skillset's definition of "Evaluate User Testing Of Interactive Media Products" (IM14 in [24]), which leads to jobs such as Usability Tester or Quality Assurance Manager, is less than 200 words.The SFIAplus definition of the comparable HCEV3 is roughly seven times longer.The less detailed nature of Skillset's definitions can be seen in a complementary skill IM6: "Use Authoring Tools To Create Interactive Media Products".This requires practitioners to know/understand "The principles of interaction design and especially issues of usability and accessibility" and to be aware "Why it is important to specify navigation that is clear and intuitive to use".Other parts of the specification are no more detailed than this Skillset and SFIA take different approaches and some in HCI will prefer one or the other, while others may yearn for another SSC (such as CCSkills) to create an Interaction Design competency model that fully fits our community's expectations.

HCI IN SFIA
At a recent presentation to usability professionals I asked the room "When was usability so mature that it was deemed fit for industry by UK government?".The answers varied from 2003 back to 1999.In fact the DTI's "Usability Now!" initiative [12] happened in 1990.Jonathan Earthy credits Charles Brennan [private communication 2005] as starting the effort c.1990 to get HCI roles into ISM/SFIA.Indeed HCEV was accepted at this time though it can be seen more as workplace ergonomics, than usability evaluation, as our community would have preferred.
Textbooks such as Dix et al [13] ensured that usability was at least seen as an aspect of good practice by a software engineer or multimedia developer, but repeated attempts to formalise usercentred approaches into role definitions were rebuffed.By the time of Interact'99 there was a sense in workshops (eg [25]), panels and coffee-time discussions that usability itself wasn't very usable!The Usability Professionals Association was set up in the US in 1991 and over the next ten years built up worldwide membership.From several different directions an effort started to accredit competency in usability.At this time SFIA v2 was adopted in over 20 countries and offered clear routes through organisations (progression) within a variety (~50) of roles (specialisation).However often the entry criteria to roles were kept pragmatically vagueat the time many senior people in IT did not have degrees.But SFIA had virtually no usability roles -Ergonomics and Content Creation (ie technical author) made up the Human Factors subcategory.In pockets there were token HCIrelated roles, but often with a worrying lack of precisionfor example one new role "Website Specialism" included the requirement to make screen designs "tasteful".
At this time I was interested to see if degree courses could be mapped to specific roles.There appeared to be much resistance to this, to avoid confusing experience with qualifications, although a follow-up to SFIA v 2 [14] did attempt such a mapping, and for HCEV the following mappings were made L3 Systems ergonomics evaluation -323 L4 Systems ergonomics evaluation -409, 414 L5 Systems ergonomics evaluation -502, 503 L6 Systems ergonomics evaluation -502, 503 Overall, there remains reluctance by SFIA Foundation representatives, when asked in public meetings, on whether we can map degree exit levels to SFIA entrance levels.Perhaps this is changing slowly as non-graduate managers reach the age of retirement -it is now commonplace to see "Probably educated to degree level" in the Educational Background.In larger companies I would expect an above-average graduate to enter (perhaps after an extended induction period) at level 2 and a similar masters graduate at level 3. Yet the curriculum content is generally two levels higher than this, and certainly assessments test capabilities at least one level higher.Conversely, not every aspect of a role fits into an academic course, and practical experience plays its part.

The Road to SFIA v3
Throughout 2002 there was a range of usability accreditation initiatives from both BCS and the Usability Professionals Association [15,16,17], based on SFIA.Notable in [17] is the desire to get beyond Practice (level 3 of SFIA) and into Professionalism (level 4 and above).However the "license to practice" type of professional accreditation ultimately got nowhereas soon as a body can strip an individual of the right to practice, it needs need to have robust enough definitions to resist and deflect lawsuits.
In 2003-2004 Earthy defined additional roles for SFIA 3 and BCS Interaction SG (then British HCI Group) and others lobbied successfully for their inclusion in SFIA v3.Five roles and a total of 20 tasks at the various levels were defined.In particular, an HCI-related role was one of the 16 skills defined at the top level, 7, SFIA v3: HFIN (Human Factors Integration).While we had cause to celebrate this success, the five categories embedded in SFIA v3 represented a less than ideal compromise for several reasons


The term "non-functional needs" adopted by SFIA to cover usability needs (as well as other areas such as performance and response times), is problematicit appears to marginalise human factorsto be of secondary importance to "functional needs".A more holistic definition of functionality would include issues from Fitts Law to fitness for purpose.Despite these disappointments, the solution is not to ignore the work of SSCsnot least because the global outsourcing of human resources (HR) issues is leading to organisations having to exception-handle roles that are not in the frameworks, and that only further increases the marginalisation of our community.We need to comment and lobbyas we did for SFIA v4 and need to plan for SFIA v5.

SFIA in use
It's worth a reality check on the extent to which SFIA has been adopted by industry.A web-search reveals employment agencies using it in a handful of vacancies [19].Public sector, RBS, BT were all revealed as adopters at a panel session at HCI2008 [20].
Other adopters easily found with a web-search include Norwich Union and Unilever.No-one at the HCI2008 workshop [21], and few in a subsequent panel were familiar with SFIA.From subsequent discussions it appears that those from smaller usability companies can work at only one or two levels of usability roles Interestingly a key theme of the 2007 SFIA conference was the user experience of using the framework.Additionally the consultative website sfia.textmatters.comand the resulting SFIA v4 launch documentation were carried out by "an information design consultancy with a user-centred approach".So it would seem that at least the SFIA Foundation itself uses the ideas by the HCI community.

HCI IN SFIA v4
The SFIA foundation invited a critique of SFIA v3 definitions at sfia.textmatters.org, in early 2008.A similar exercise occurred in 2005 prior to SFIA v3.Members of the BCS Interaction SG took the opportunities to post comments.Some of these were acted on (eg "non-functional needs" improved a little to "non-functional requirements for usability", while non-HCI roles like Business Process Testing now involve greater user input "from statement of user needs and user interface specification").Others were not (eg there was no response to the request to add information architecture as a skill or as part of a role).Such consultations are inevitably frustrating to participantsnot every piece of input is acted on, even if there is no contrary voice.
Once the consultation is over, the SFIA Foundation defines the basic specifications and structure of roles in SFIA.There appears to be limited opportunity for stakeholders to comment on the proposed changes.Where there is more opportunity is when BCS start the process of reviewing SFIAplus, which adds detail to the role definitions.An expert in each field defines these, and then other experts blind peer-review this work.With SFIA v3 this took place only after SFIA itself was finalised.With v4 there was more of an overlap which allowed for some recursion.
In reviewing for v3, I found myself somewhat boxed in by the structurethe changes I proposed tended to be relatively minor, perhaps inevitable with new role definitions.For SFIA v4 I used a card-sorting approach to analyse the career structures in more depth, and took the opportunity to canvas opinion from practitioners and to participate in felicitously-timed workshops such as [21].
My responsibility was for HFIN and HCEV skills and I struggled to find a sufficiently structured approach.I wanted to engender a greater sense of flow through the levels, of creating parity between tasks at the same level, finding delineations between roles, and eliminating repeated, redundant and dead end skills and training activities.On reflection this is overly ambitious for the time expected of a reviewer (roughly 1-2 hours per task level).
The SFIAplus definitions consist of a 4-page generic role spec, each of which consists of SFIA skill title, description, key words, related skills, technical overview(tools, techniques), overview of training, development and qualifications, careers and jobs, professional bodies, standards and codes of practice, communities and events, publications and resources.
For each task (ie the role at different levels), there is a 7-8 page document describing Though highly structured it was hard to see, in a separate 7-page document for each level of HCEV and HCIN, exactly how an individual might tick all the boxes to progress to the next level.Equally it was not hard to imagine that in practice, unless each of the boxes was ticked, then progress would not be possible.In particular, unlike an academic course, where low marks in one area can be compensated for high marks in another, in a competency framework, each characteristic must be achieved above the required threshold.
As other reviewers will have done, I marked up each of the specifications and returned them to the BCS.Within the BCS Members Area of the website there is the opportunity to peruse SFIAplus v4, and it would appear that about half of my resulting recommendations have been adopted, a similar response to the reviewing I did for SFIA v3.The remaining aspects will be the subject of future work, in which BCS Interaction membership will be canvassed for their views on these issues, and encouraged to contribute their thoughts to the next consultation exercise.

Reflections on the process
At a more fundamental level I have raised some issues about factors that did not seem to be easily influenced by the reviewer.A brief summary of any response is included 1) I recommended that a training activity should not be repeated at a subsequent level of a roleor if it is it should be called "advanced xyz".The response was that since training activities are intended to be those that increase effectiveness in the current task, it remains appropriate to have this at multiple adjacent levels.This is regrettable.While a refresher course in many topics is useful, it would be preferable to build on the knowledge already gained at previous levels.An additional dilemma is caused by the purpose of training being not so much as to prepare for the next level up but rather to improve performance in the current level.Given this, it may be better to define separate training for the PDA section that includes specific training for advancement.
2) Behavioural skills are not qualified by depth, and I felt they should appear only, once at the lowest applicable level.. Similarly when the other knowledge/skills is required to the same depth in a higher level of the role, this too is redundant.The response was that skills can be at the same depth in up to two adjacent levels, but this does seem to reduce the capability to easily distinguish between different levels of competency.
3) At levels 6 and 7 almost all Skill depths should be Expert or Proficient.This was certainly not the case in some of the task descriptions I looked at.The response was that this was probably true but not a formal requirement.However the generic statements for levels 6 and 7 include sentiments such as "Has a broad understanding of all aspects of information systems and deep understanding of area(s) of specialisation" and "Has deep understanding of information systems industry and emerging technologies and implications for the wider business environment" respectively.

4)
As a reviewer for both, HCEV6 seemed increasingly redundant in SFIA, as almost all of what is intended appears to be suitable and desirable for HFIN6.I've since been informed that there is still a cadre of long-serving level 6 ergonomists within organisations whose competences and ambitions do not extend to HFIN6.Plainly this is something that should be put on the agenda for SFIA5.

5)
Many of the Skills definitions (at least as supplied in the reviewers guide [22]

TA184 Change Management
Issues and strategies for the successful implementation of change within a business environment.

Depth of Learning
In the course of trying to align individual academic modules with both the roles and the levels in SFIA, the following analysis was prepared, based on review of relevant recent job adverts coupled with an understanding of the roles performed by personal contacts in the industry.Plainly this exercise lacks academic rigour, but can hopefully serve as a useful starting point for a future workshop to refine tables for each of the human factors roles within SFIA.
Fig. 3 considers the training most relevant to HFIN and USEV and then expresses the depth of knowledge (using the same 4 point scale) of these areas anticipated in each level of role, providing a ladder to proficiency and in many cases expert status but the definitions of both proficient and expert are not clear.For example expert could mean "to a professional standard" or "UK Expert", or "Leading international authority" Code Title Description pre 5 6 7 TA002 Standards, Procedures and Tools Standards, procedures, software tools and operating system facilities used in everyday work.

F P E TA004 Service Delivery
The service delivery processes: the systems, products, services, hardware and software environment which are supported by operations staff.

F TA006 Program Design Methods and Tools
Programming or system development methods (eg structured program design).P

TA007 Commercial and Business Practice
Commercial and industrial business practice and terminology, particularly in respect of the activities of the employing organisation.

F P E TA012 Application Area
The structure, business and methods of the employing organisation, particularly in respect of any application area of specialisation.

F P E TA017 Systems Analysis and Design Tools and Methods
Tools and methods used in systems analysis and design.F P E

FUTURE WORK
In the course of this paper a number of topics have emerged as being useful ambitions for events such as HCI Educators, at which the relevant experts can be assembled.
The list of low-level skills that are used to define the roles and tasks of SFIA needs to be evaluated and updated, to ensure that we are ready to feed requirements into the consultative window for SFIA v5, which I would expect to happen in 2010-2011.
Similarly the lists of training activities (perhaps renamed to learning situations?) and personal development activities needs updating and realigning with contemporary approaches to pedagogy and expertise.
HCI-related roles defined in SFIA and Skillset (and any other sector skills council working in our area) need to be evaluated and a plan put in place to remove duplication, ensure the most appropriate SSC is home for the given role, to define qualifications and CPD products that lead to the required level of competency.

CONCLUSION 6.1 Constructivist Approaches to Learning
Despite HCI's common lineage with constructivism, it's surprising how often we revert to instructivist mode when seeking to increase HCI understanding in students.This paper runs the same risks!The HCI2008 workshop organisers [21] asked "Ultimately, is academia responsible for teaching the general theoretical grounding and ensuring a well rounded understanding of the field or is it also responsible for providing real life craft skills?"If we are to help learners build structured careers we need to beyond "teaching" and "providing skills", to facilitate the gaining of competences.We would do better to make learners aware of external expectations and inspire them to desire to meet and exceed these.Students do respond (gleefully) to the criticisms provided by books such as [23].We need to create the climate for intrinsic, rather than extrinsic, motivation, and encourage deep rather than strategic approaches to learning.We need therefore to shift our thinking from defining a BOK to be taught, to instead revealing the causes and consequences of incompetence in the professional practice of human-centred information systems design, which in turn would motivate students to develop the required competence.
The desire to achieve and demonstrate competence is a fundamental one in humans, and, above level 1, in the SFIA framework the emphasis shifts away from providing behavioural instructions to motivating role-players to take responsibilities for coping with imperfect situations and creating the climate for reducing these imperfections.In order to give the next generation of HCI professionals career paths that extend as high as they can achieve, we all need to play this gameto engage with competency frameworks and shape them to reflect adequately our ever changing knowledge and expertise.This will ensure that high quality people remain in the profession, and that HCI ideas will gain traction at every level of industry and the public sector.
It should also be noted that the various SSCs can, and do, cede territory to each other.SFIA has backed away from defining Website Specialism and anything to do with computer games, while Skillset refers to e-Skills/SFIA for "Data analysis and data structure design; Software developmentcomponent creation; [11]ware development -design" etcMore pragmatically, BCS Interaction SG discussed in 2000-2002 how best to get HCI ideas into use, and settled on the then more advanced SFIA as our target.Jonathan Earthy led the attempt to get human-centred roles into SFIA, ultimately succeeding in 2005 with v3.This built on Brian Shackel's last great contribution to HCI -with Earthy, Brian Sherwood-Jones and Catriona Campbell getting the UK e-Envoy to incorporate ISO13407 into the government guidelines for public sector web-sites[11].
[18]s such as Information Architecture remained undefined (nb Skillset now have a definition[18], though is arguably better suited to SFIA).

Table 2 -Candidate Training Activities for HFIN with related depths (from [22] with columns added for
A, F, P, E)