Is bias undermining the integrity of digital education research?
Photo by Mikhail Nilov from Pexels
The role of digital technologies in our lives, society and world is a topic of huge scrutiny, debate, controversy and opinion. Sometimes this noise can be as loud in education as anywhere else, and over the years much has been written and spoken about, in addition to much research in this area.
If at the beginning of the journey towards the near ubiquity of digital technologies in certain parts of the globe the narrative was purely about the good that would come of it (which is of course both doubtful and naive) then this is certainly not the case anymore.
In a recent keynote I gave at a digital education conference I quoted Rob Reich, Professor of Political Science at Stanford University who in 2018 encapsulated this well:
“If the baseline for making a projection about the next today is the current level of benefit/harm of digital life, then I am willing to express a confident judgment that the next decade will bring a net harm to people’s well-being. The massive and undeniable benefits of digital life – access to knowledge and culture – have been mostly realized. The harms have begun to come into view just over the past few years, and the trend line is moving consistently in a negative direction.”
Now people will take different positions on a statement and perspective like that, but I think it represents a view held by a good number of people, particularly in education.
In fact one might argue that it has become quite vogue to position yourself as anti-tech, tech sceptical or critical when it comes to digital technology in education. It has become the veneer to have and the standpoint to personally signal if you want to align yourself and be publicly accepted into certain crowds and tribes.
This is worrying, not least because a sober, rational and rigorous evaluation of the role of digital technology in education and beyond is extremely important.
If we’re going to move forward in digital education then we need a professional community that isn’t going to blindly and obsequiously follow the views of the loudest and most influential voices, and seek to align themselves with in-crowds.
We also need a research field that seeks truth, is strongly evidence-informed, trustworthy and has integrity, to compliment the work done by practitioners.
Unfortunately though, in the UK I see a growing move away from the sincere pursuit of truth, and instead see researchers and others moving further and further away from the aforementioned fundamentals of a research field.
Why is that the case? Well I think there are a number of reasons for this.
Jonathan Haidt, a social psychologist and Professor at New York University posed an interesting question a number of years ago. Do we want a university founded on finding the truth through debate, or one focussed on changing the world? He argued that in the context of a university both are incompatible.
One of the reasons he gave for that is that any other motive than truth is bad for scholarship because reasoning is very heavily motivated. As human beings we are much better and inclined towards motivated reasoning than objective and balanced reasoning.
He has gone on to say that based on his own research, scholarship that supports a political agenda almost always succeeds, the scholar rarely thinks they’re biased, and this motivated scholarship can spread pleasing falsehoods that cannot be withdrawn and are perpetuated in their field and others.
Such falsehoods can only be dealt with if you have a robust system of accountability. In academia this is peer review, but what happens when you have homogeneity of political views and perspectives? What happens if you have small and narrow cliques that are of the same mind in a particular field? Do you get closer to objectivity and truth? Do you get a diversity of views that through debate and review ultimately sharpen the end product?
What happens if you couple those conditions with the sense that you are pursuing a just cause? That you’re battling against an unrighteous enemy manifested in the form of a particular company, the private sector as a whole, or in fact anyone wanting to influence education that isn’t you or your peers.
Do you have the conditions for a research field that seeks truth, is strongly evidence-informed, trustworthy and has integrity?
Unfortunately, what I think is becoming increasingly evident here in the UK and further afield is a digital education research field that earnestly pursues a cause but not truth. It believes itself to be righteous and this blinds it, and can make it insular and exclusionary.
I sometimes see a narrow fraternity that seems to be adding more and more jargon-laden language to its lexicon, such as platformisation, techno-solutionism, digital deschooling, datafication, Amazonification, learnification and my personal favourite indexlicalization.
When you develop a less and less inclusive vocabulary to communicate your work in a field then it’s valid to ask, who does that really serve?
There are a number of tell tale signs of a degradation of research in digital education, but I want to focus on four.
1. Inaccuracies and false claims
One of the characteristics of the digital education field I observe is a disposition to position themselves as gatekeepers of universities - seeking to repel the influence and infiltration of mainly private actors into this sphere.
When they come to report on the role of the private sphere in education this disposition is impossible to shake and can lead to factual inaccuracies and false claims.
This overriding sense of a cause and the politics wrapped up within it leads to motivated scholarship which means that what’s happening in education in relation to private companies, their influence and the effect can be inaccurately portrayed and falsely reported.
To give an example, in this report jointly-written by a researcher at the University of Edinburgh, the authors claim that there’s an expansion of platforms that “automate the intelligent discovery and recommendation of relevant learning content” (think of a Netflix or YouTube style technology) into higher education. These platforms can sometimes be referred to as learning experience platforms.
In the report, the authors highlight a product called Aula (which calls itself a learning experience platform) as evidence of this. However, this platform is no way a platform that “automates the intelligent discovery and recommendation of relevant learning content”.
They make a false claim that this type of technology has been expanded into universities during the pandemic and seek to justify it by pointing to a technology that doesn’t matchup.
This highlights a lack of rigour and professionalism, but what it also highlights is the predetermined narrative is more important than the observable reality or truth. Presenting the pandemic as a time that fostered a creep towards automation and AI within universities through the adoption of these platforms is more important than whether that is the reality.
For what it’s worth, in the UK I have seen little evidence to validate that universities are adopting this category of platforms in any significant way yet.
To give another example, I have also heard inaccurate claims of a similar nature made about Online Programme Management companies (OPMs). One that sticks in the memory is a claim made at a talk I heard about pandemic privatisation given at a UK university. The claim was that the pandemic led to OPMs expanding the range of things they previously used to offer to include things like marketing, recruitment and learning design.
The commentary here is that these are things that universities typically do themselves, which is of course true. This alleged expansion is therefore presented as a threat to universities as these things might be taken away from them and it is pandemic profiteering that has driven it.
The problem with this though is that any digital education professional who knows anything about OPMs will be able to tell you that before the pandemic OPMs had these functions and more. So it simply isn’t true to say the pandemic led to OPMs diversifying their services in these areas.
Again this is a case, where the narrative, agenda or cause trumps reality.
In both examples, you can say that either there is incredibly sloppy scholarship going on or there is something much more wilful driving it.
2. Conscious omission
The other tell tale sign is not what is said but what isn’t said or included. The result of what is excluded can cement a narrative and perspective that the researcher wants to get across.
One example is an initiative called “Learning and Teaching Reimagined” that culminated in a report. This initiative involved multiple players, such as HE sector organisations Jisc, AdvanceHE and Universities UK as well as private companies Emerge Education and Salesforce. You might’ve noticed that although some of these organisations are close to them none of them are universities.
This critique from the Centre for Digital Education Research at the University of Edinburgh posits this initiative as “private-equity backed technology companies steering the ‘digital transformation’ of higher education and pedagogic practice” and as an example of organisations seeking to seize an opportunity during the pandemic for HE reform.
So where are universities in this initiative? From the critique one might think they are not involved at all. Which when you take a look is actually a big and notable omission. This initiative is led by an advisory board made up of senior leaders working within UK universities, ironically including the Principle of the University of Edinburgh. There was also engagement from HE staff and students in what appears to be decent numbers.
Omitting and airbrushing universities and their staff and students out of the involvement of this initiative cements a narrative of private sector infiltration, influence and pandemic driven opportunism.
This is a common occurrence and symptomatic of a gaze that centres only on specific actors outside of universities. There’s a strange myopia, that on the one hand proclaims the need to view things more holistically but that airbrushes out universities themselves.
I suspect there are two main reasons for this - there is an unwillingness to criticise the thing you’ve positioned yourself as a gatekeeper of and there’s limited knowledge of what happens internally in universities - due the distance that’s been put between themselves and the reality on the ground.
This is another example of the pursuit of a narrative and cause overtaking the clear, factually accurate reporting of something. It’s an example of how omitting something, presumably consciously given the clarity in which university involvement was presented, enables the presentation of narrative but doesn’t clearly present the reality.
3. The lack of evidence and the lack of effort to gather it
I’ve touched upon this earlier and highlighted examples in which false claims with lack of evidence to support them are presented. But I think it’s worth making a broader point here with respect to a pattern that I see - which is very little ethnographic and empirical research.
If you sincerely want to discern and understand the pros and cons of digital technology in education and all that comes with that then this will involve spending time in these environments and earnestly engaging with people in these environments.
Unfortunately, I see little evidence of this and at times what I have actually heard is disrespectful and aloof behaviour from some prominent researchers towards practitioners within digital education working in universities rather than sincere engagement with them.
Quite apart from this being negative for a research field, it plays into the ivory tower characterisation and perception of academia. Most often than not this is not fair to large swathes of academia but it is worrying that UK digital education research can sometimes seem to be the research equivalent of armchair punditry or a gossip columnist.
In engaging with a diversity of people within higher education one can piece together a true picture, but if you stand aloof from that and operate within a narrow community of conforming views and persuasions that pursues a cause then what’s likely to result is distortion and bias.
4. Playing on people’s fears and frustrations
The final point here is an important one - in a sector as fraught as UK higher education. There is much turmoil and distrust between staff and institutions.
There is also much power and traction that can be gained in the pursuit of a cause if you conjoin the issues that underlie the turmoil we see in UK universities and a narrative of private sector infiltration through digital technology and pandemic profiteering.
I’ve observed this used as a tactic that plays on peoples fears, plays into their frustrations and anger and ultimately has the effect of drawing more people to your cause. More often than not such linkages are unsubstantiated, lacking in any nuance or somewhat detached from reality.
This links into my earlier point - in that if you don’t put the hard yards into gathering actual evidence on the ground of the impacts, both positive and negative, and unintended consequences then you run the risk of disinformation that can amplify anger and frustration, which ultimately sows more discord.
A UK research field in need of a hard reset?
What are we to conclude in terms of the state of digital education research in the UK? Can we trust the outputs of digital education research centres like those at the University of Edinburgh and others? How are we to navigate this space and make sense of it?
Well, I think as things stand we almost have to treat these centres and some of the research as we might do a political think tank or a newspaper or media outlet.
We can all point to particular publications whose editorial, view of the world and reporting is heavily influenced by their political stance and viewpoint - when we read their output we have to filter it through that lens.
I think we can’t take at face value that what we’re getting is an accurate picture of what is happening in higher education, but that this is a particular treatment based on the researchers pre-dispositions, particularly if there’s not any substantial or verifiable evidence that accompanies it.
This is a far from ideal state, but perhaps more positively, amongst digital education practitioners and educators I have consistently experienced a collegiality and a willingness to share insights & experiences, discuss and positively engage with each other on the realities.
These professionals are a great source of truth for those who want to engage with the realities of the field. I’d encourage anyone wanting to find things out to reach out to professionals across the public and private sphere and engage in conversation.
I should also say that I’ve also read some valuable digital education research from those earnestly trying to engage with people on the ground to gain a true picture, particularly during pandemic times.
All is not lost, but those engaging with some of the outputs have to be conscious of a need to fact check and not assume the validity because of where it’s coming from or reputation.
This is a slightly depressing reality when you consider that for most observers of higher education it’s likely that digital technologies will continue to play a growing role.
So the importance of a research field with integrity is absolutely paramount. I for one hope the years to come raise up more diversity of thought, greater integrity and a move towards more inclusive communication of findings. This will greatly benefit the field and education as a whole.
