TLDR: Technologies need more focus on learning and less focus on measurement and assessment. To achieve this, technologies need to use models of human learning.
…
The focus of much of educational technology concerns assessment: the identification of an individual’s proficiency in a simple description of their performance. This most commonly comes in the form of a single number representing the student’s knowledge (see related post on multi-dimensional measures of learner knowledge for some additional discussion about this). Sometimes, you might get a couple numbers – usually it is just one number though, and usually is in the form of a percentile ranking. It might be something fancier like Rasch index, which is useful if you have some special knowledge (but not useful for most of us!).
The focus on assessment is frustrating for those of us who really just want our students to learn. This is because assessments don’t tend to focus on learning, they tend to focus on reporting. Any parent who has stared at a list of percentile rankings has experienced the frustration of not knowing what to do with this type of data about their kid. Knowing where your child is relative to others may be satisfying to some parents, but isn’t useful. The focus on assessment and reporting isn’t a mystery; it is due to the conflicting constraints on educational entities to report their students’ outcomes to stakeholders. Very often we sacrifice the learning part for the reporting part because reporting tends to be a well-funded enterprise, where learning often plays 2nd fiddle.
Unfortunately also for those of us who consume assessment data, all of these measurement concepts come from statistics (rather than some other discipline focused on learning). This is obvious when you consider that assessments most often don’t really help us educate, they just help us measure. Teachers will very often tell you this, especially during the points of the school year where they have to take their limited teaching time to assess their students rather than teach them. All of this is related to the fact that the sciences of assessment and measurement have historically been mostly separate from the sciences of education and learning.
Technologies, especially those focused on reading, would benefit from a focus on learning rather than measurement. This shift is difficult because the relevant science is much more broad and harder to consume – and there is often times lots of debate about what to teach.
How do we focus on learning?
Focusing on learning in educational technology requires the identification of what aspects of the knowledge domain to teach throughout development. This is hard because not many of us have a coherent understanding of learning to support such a process, let alone the technology to implement it. This is why we need learning systems that embody models of human learning in our educational technologies.
Most assessment products rely on various “latent factors” (aspects of the student that are uncovered from the statistics but not understood by the statistician) that come from statistical models to drive their technology. Alternatively, many technologies (including simpler technologies like print-based resources) are built off of very specific ideas about what to teach and when (i.e., the child is at X point in the curriculum on this day so they should be learning Y). Both approaches make sense, but there is another alternative that comes from cognitive science: make decisions about what to teach based on the deep structure of the learner’s knowledge and established principles of how it is learned. In the specific case of reading, the focus should be on the language and how our cognitive system acquires the structure of language through learning.
Importantly, this type of approach is embodied in mathematical models of learning processes (i.e., what most people call “machine learning”), and offers an alternative solution to the old assessment-first paradigm. A learning-first approach is promising given the incredible capacity of machine learning systems to take in data about what the learner knows and what the learner doesn’t know, and deliver back predictions about what will help fill the gaps in their knowledge based on the structure of the language.