Education assessment in the 21st century: New skillsets for a new millennium

Education researchers have a unique opportunity. There is a global shift in education toward complex skill sets like collaborative problem solving. For such skills, the assessment challenge has preceded development of how to teach the skills. Researchers can conceptualize the nature of these “21st century skills” and how teaching methods, unbound by curricula, might progress from basic to sophisticated forms. In the same way that in the 1900s, work in measurement methods helped define intelligence and personality, current approaches to assessment are helping define some of these 21st century skills.

But this does not come without challenges. The first is to define the skill, or “construct.” How we explore alternative conceptualizations of a skill determines not only the assessment approach but also can have massive implications for pedagogy and systemwide learning outcomes.

The second challenge is that we are interested in behaviors rather than outputs. Specifically, what can insightful manipulation of behaviors achieve? However, measuring behavior rather than the products of behavior is quite different from measuring well-understood skills like literacy. How do we define a behavior in such a way that it can be measured objectively at large scale?

One school of thought holds that we can only improve what we can measure. The teaching parallel is that we can only teach what the student is ready to learn, and to know the latter we need to be able to describe that readiness in terms of the construct of interest. So exploring methods to measure complex skills establishes pathways to improving the skills.

The third challenge is that the nature of complex skills defies automation; we need them precisely because of this characteristic. Machines and computers have automated many working tasks and jobs, weakening the demand for traditional skills. The recommendation is that we need increasingly to engage in complex, non-routine activities in order to face this global phenomenon. The skills that we need to develop are themselves therefore complex and non-routine. But if skills are resistant to automation, imagine how difficult it is to capture them for measurement purposes!

Complex skills such as creativity, problem solving, collaborative skills, and critical thinking are among those that challenge us. Attempts to use existing approaches to measure these complex skills are ineffective. We have argued in an earlier blog that using conventional assessments to measure conventional academic skills (such as literacy) is analogous to using tibia length to measure height. Extending this argument, using conventional assessments (even when augmented by technology) to measure complex skills is analogous to using waist measurement to estimate body weight to measure cardiovascular health. The more indirect the links to the skills, the greater the information loss.

Hard-to-measure complex skills

Behaviors, in our belief, can be measurements of skills. The assessment challenge is to identify what behaviors indicate what skills. The potential for misinterpreting behaviors is enormous, as most people know from daily communications! However, use of online tasks specifically designed to elicit these skills provides a relatively pristine environment to collect assessment data.

The Assessment and Teaching of 21st Century Skills project, underwritten by Cisco, Intel and Microsoft, and six participating countries, explored exactly this approach through building interactive tasks that required actions hypothesized to demonstrate the subskills underlying the complex collaborative problem-solving skillset. Collaborative problem solving requires both cognitive and social skills: drawing on identifying a problem, what resources are needed, how to manage the resources, how to hypothesize, etc., while at same time using skills like participation, communication, and negotiation. It is clearly a complex matter building an effective interactive environment. Although many existing games and game-like tasks provide an environment in which some of the same skills are exhibited, it is more efficient to make the links between measurement and skills with a made-to-measure approach. If measurements can be refined, online tasks could provide us with an opportunity to understand the paths taken by students through collaborative tasks, moving us past individual student assessment and toward exploring paths taken by groups of students.