Foot pain drawings (manikins) are commonly used to describe foot pain location in self-report health surveys. Respondents shade the manikin where they experience pain. The manikin is then scored via a transparent overlay that divides the drawings into areas. In large population based studies they are often scored by multiple raters. A difference in how different raters score manikins (inter-rater repeatability), or in how an individual rater scores manikins over time (intra-rater repeatability) can therefore affect data quality. This study aimed to assess inter- and intra-rater repeatability of scoring of the foot manikin.
A random sample was generated of 50 respondents to a large population based survey of adults aged 50 years and older who experienced foot pain and completed a foot manikin. Manikins were initially scored by any one of six administrative staff (Rating 1). These manikins were re-scored by a second rater (Rating 2). The second rater then re-scored the manikins one week later (Rating 3). The following scores were compared: Rating 1 versus Rating 2 (inter-rater repeatability), and Rating 2 versus Rating 3 (intra-rater repeatability). A novel set of clinically relevant foot pain regions made up of one or more individual areas on the foot manikin were developed, and assessed for inter- and intra-rater repeatability.
Scoring agreement of 100% (all 50 manikins) was seen in 69% (40 out of 58) of individual areas for inter-rater scoring (range 94 to 100%), and 81% (47 out of 58) of areas for intra-rater scoring (range 96 to 100%). All areas had a kappa value of ≥0.70 for inter- and intra-rater scoring. Scoring agreement of 100% was seen in 50% (10 out of 20) of pain regions for inter-rater scoring (range 96 to 100%), and 95% (19 out of 20) of regions for intra-rater scoring (range 98 to 100%). All regions had a kappa value of >0.70 for inter- and intra-rater scoring .