Data and learning: Does more = better?
Over 15 years ago the British mathematician Clive Humby declared that “Data is the new oil”, and there is now no doubt that data is a major commodity. It is mined and gathered through the numerous interactions between humans and digital technologies throughout the course of our daily lives.
This is also the case in education which is increasingly being mediated through digital technologies and in the case of online education is largely dependent on them.
Has data or learning analytics fulfilled its potential?
The potential of this grand new commodity has been talked up for years as a key benefit of the use of digital technologies in education. The idea being that this raw material once refined can improve education, teaching and study in a number of ways.
An area of research and study associated with the use of data in teaching and study is learning analytics, which was defined by George Siemens as:
'the measurement, collection, analysis and reporting of data about learners and their context, for the purposes of understanding and optimising learning and the environments in which it occurs'
There has been a significant amount of research and work in this area, but one of the common, recurring themes is the relatively low adoption of learning analytics amongst teachers and institutions across education.
In fact, wherever you stand on data-informed teaching or learning analytics you have to admit that it has yet to make really significant inroads into education.
There are no doubt numerous reasons for this because education is a complex area. One of those is undoubtedly the challenge of incorporating the use of data into existing practices and ultimately changing those practices in some way.
Data can add to the complexity of teaching
The addition of digitally derived and communicated data adds another layer of complexity to the role of an educator. A role that has arguably got more complex with the addition of digital technologies.
Pierre Dillenbourg’s description of orchestration conveys the complexity of teaching with technologies well:
“Orchestration refers to how a teacher manages, in real time, multi-layered activities in a multi-constraints context. Many pedagogical scenarios integrate individual activities (e.g. reading), teamwork (e.g. problem solving) and class-wide activities (e.g. lectures). Some of these activities are computer-based, some not; some are face-to-face while others are online. This pedagogical integration is mirrored by the technical integration of different tools (simulations, quizzes, wikis, etc.) distributed over multiple artifacts (laptops, sensors, tablets). These integrated scenarios require forms of management referred to as orchestration”
In this context, interpreting data and then being able to act on it in a way that might aid a student's learning process is easier said than done.
The challenges of acting on evidence to adapt teaching are longstanding
In fact, we can be way too myopic in this area. Digital data is just another type of evidence or information that teachers can use to modify their teaching to aid student learning.
We’ve known for a long time through the work of people like Dylan William and Paul Black of the potential of using information gathered through formative assessment, which they defined as
‘…all those activities undertaken by teachers — and by their students in assessing themselves — that provide information to be used as feedback to modify teaching and learning activities. Such assessment becomes formative assessment when the evidence is actually used to adapt the teaching to meet student needs’
However, this doesn’t mean that knowing this has led to a great transformation in teaching and study. In fact you can read post mortems on how the huge potential and promise of formative assessment at that time ultimately wasn’t realised in any major way.
There’s a valuable lesson here for anyone seeking to leverage data in digital and online learning, in that, the promise of data will never be truly realised if there isn’t skill to design the means of gathering relevant and useful information related to the learning process, and the conditions and capabilities of taking that information and doing something with it.
It is entirely reasonable to consider how data might improve the quality of teaching and better help learners towards achieving their learning goals. But in doing so there needs to be consideration of the level of professional skill and ability or readiness to use information to adapt teaching as well as other things.
If formative assessment or what William wished he’d had called it “responsive teaching” isn’t happening in any serious or effective way, then the availability of a digitised form of evidence or information is unlikely to make very much difference.
So whilst you might want to utilize data to improve the educational experience, there is so much more to that than developing the technical capability to collect, analyse and present it.
There are no easy shortcuts to improving educational experiences and too often discourse about data in education is based on the simple equation that more information = better results.
Whilst in some instances the availability of data might help the judgements and decisions that need to be made to support individuals to meet learning goals, it won’t accelerate or bypass the need for developing people’s practice and competence in the art of teaching.
Do we always need the data component of digital technologies?
Thinking more broadly, we also need to seriously question the need for any and every educational technology to provide us with data. Just because it can doesn’t mean it’s of inherent worth and value to the learning and teaching process. There’s a lot of vain, virtual hoarding going on under jejune premises of educational enhancement.
At times we would be well to focus on the learning process and whether the affordances of some digital technologies to stimulate thinking, provoke, nudge, remind etc is actually sufficient and leave it at that.
The gold rush fever around data makes it almost inconceivable that a digital technology company providing a product for education - wouldn’t also consider how data might be leveraged. But this isn’t the only path and it should be entirely plausible that a technology might be provided to support the learning process that simply discards and discounts the data it generates.
Digital transformation often leads to asking the wrong questions
Both the human and professional challenges of acting on data and the perceived necessity and value of it, exemplify how digital transformation efforts can sometimes lead to asking completely the wrong questions.
For instance, how many, if they were asked the question - how can we use data to improve teaching and study? would reply - we need to further develop our educators skills, practice and competence so that they’re better able to design ways to elicit evidence of student learning and use that to adapt their teaching accordingly? Who, in seeking to answer that question might say that they’re not at a level of readiness or maturity to take advantage of any of the potentials of data yet? Is there any scope there to ask if you really need data?
I would argue that framing a question that way is much more likely to result in going down a road of exploring how to collect, analyse and present data than it is in addressing core educational and developmental needs.
Instead, starting from a broader premise of how you might improve both teaching and study is a more solid foundation, and should lead you to exploring the evidence and research around the many ways and practices that have the strongest potential to improve teaching and help learners to meet their goals.
From this platform you are able to explore the ways in which digital technologies may play some sort of role in supporting those approaches, but with consideration of the multiple factors that would contribute towards embedding them and capitalising upon them.
Sometimes, as hard as it may be, institutions will need to accept that their level or readiness or maturity might mean that their immediate aspirations around digital technologies need to be pushed back as there are bigger more fundamental matters to address on the road to improvement.
Teaching and learning is a complex act and process. We’d all love to believe that there are simple shortcuts to mitigate that complexity and the hard yards and thought that needs to go into it. If we deceive ourselves and think that data is a simple shortcut then the commodity that it is most likely to resemble is fools gold.
