895
views
1
recommends
+1 Recommend
2 collections
    1
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Rise of the machines? The evolving role of AI technologies in high-stakes assessment

      ,

      London Review of Education

      UCL Press

      artificial intelligence, assessment, high-stakes testing, language tests

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Our world has been transformed by technologies incorporating artificial intelligence (AI) within mass communication, employment, entertainment and many other aspects of our daily lives. However, within the domain of education, it seems that our ways of working and, particularly, assessing have hardly changed at all. We continue to prize examinations and summative testing as the most reliable way to assess educational achievements, and we continue to rely on paper-based test delivery as our modus operandi. Inertia, tradition and aversion to perceived risk have resulted in a lack of innovation ( James, 2006), particularly so in the area of high-stakes assessment. The summer of 2020 brought this deficit into very sharp focus with the A-level debacle in England, where grades were awarded, challenged, rescinded and reset. These events are potentially catastrophic in terms of how we trust national examinations, and the problems arise from using just one way to define academic success and one way to operationalize that approach to assessment. While sophisticated digital learning platforms, multimedia technologies and wireless communication are transforming what, when and how learning can take place, transformation in national and international assessment thinking and practice trails behind. In this article, we present some of the current research and advances in AI and how these can be applied to the context of high-stakes assessment. Our discussion focuses not on the question of whether we should be using technologies, but on how we can use them effectively to better support practice. An example from one testing agency in England using a globally popular test of English that assesses oral, aural, reading and written skills is described to explain and propose just how well new technologies can augment assessment theory and practice.

          Related collections

          Most cited references 49

          • Record: found
          • Abstract: found
          • Article: not found

          Understanding interobserver agreement: the kappa statistic.

          Items such as physical exam findings, radiographic interpretations, or other diagnostic tests often rely on some degree of subjective interpretation by observers. Studies that measure the agreement between two or more observers should include a statistic that takes into account the fact that observers will sometimes agree or disagree simply by chance. The kappa statistic (or kappa coefficient) is the most commonly used statistic for this purpose. A kappa of 1 indicates perfect agreement, whereas a kappa of 0 indicates agreement equivalent to chance. A limitation of kappa is that it is affected by the prevalence of the finding under observation. Methods to overcome this limitation have been described.
            Bookmark
            • Record: found
            • Abstract: not found
            • Book: not found

            Testing Times

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Assessment Literacy for Teachers: Faddish or Fundamental?

                Bookmark

                Author and article information

                Journal
                lre
                London Review of Education
                UCL Press (UK )
                1474-8479
                10 March 2021
                : 19
                : 1
                Affiliations
                UCL Institute of Education, UK
                Pearson Education Ltd, UK
                Author notes
                Corresponding author: Email: Mary.richardson@ 123456ucl.ac.uk
                Article
                10.14324/LRE.19.1.09
                Copyright © 2021 Richardson and Clesham.

                This is an open-access article distributed under the terms of the Creative Commons Attribution Licence (CC BY) 4.0 https://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.

                Page count
                References: 63, Pages: 14
                Product
                Categories
                Special feature article

                Comments

                Comment on this article

                London Review of Education
                Volume 19, Issue 1

                Similar content 59

                Most referenced authors 173