Exploring the challenges and opportunities of a new online learning ranking
Times Higher Education (THE) recently announced the launch of a global online learning ranking to add to their growing suite. This new ranking is an acknowledgment of the increasing number of universities that either have online degree portfolios or are beginning to develop them.
On balance, this seems like a positive step for online education, potentially elevating the visibility of this important way of accessing higher education. A ranking could help students in evaluating the credentials of higher education institutions (HEIs) offering online degrees. It might also encourage higher standards and increased accountability.
Conversely though, rankings are imperfect instruments and since this is THE's first foray into this particular area, it may take some time to refine. This could lead to short-term challenges and possibly deter some institutions from participating initially.
There are inevitable pros and cons, along with a lot of nuance when assessing the online education capabilities and offerings of HEIs. It’s worth exploring these aspects more deeply. But first, it’s worth considering the existing landscape of online learning rankings.
What other rankings are out there?
The existing rankings predominantly focus on courses, particularly online MBAs, rather than on the institutions themselves. The Financial Times Online MBA ranking and QS Online MBA ranking are especially notable for UK institutions. The 2023 rankings feature online MBAs from Warwick, Imperial, Durham, Liverpool, Manchester, Oxford Brookes and Birmingham among their top 10.
Also, there are some US centric rankings with US Today and Poet and Quants (more on them later) each offering an online MBA ranking.
While not specifically an online degree ranking platform, Class Central has been doing a great job of listing online courses from various universities on platforms like Coursera, edX, and FutureLearn for several years. Their site also features rankings of online courses, such as "The Best Free Online Courses of All Time."
Besides these, there are less rigorous attempts at rankings. One questionable example is the Online Learning Index, created in 2022 by the UK-based price comparison site USwitch. This index scores institutions rather than degrees and I would question a lot of the data published here. This wouldn’t usually be something of particular note, but I encountered it being used on the website of a UK university, perhaps injudiciously, to help market one of their online degrees.
Overall, there certainly appears to be a lack of a credible global ranking at an institution level for online education that it makes sense to fill.
Business schools and degrees a notable omission
One notable feature of this new ranking is its exclusion of business schools and degrees. This is significant because online degrees in business and management constitute a large segment of the online degree market.
Research I conducted earlier this year highlighted that over 15% of online postgraduate (PGT) degrees offered by UK universities were in business and management. These degrees also represented about 35% of online postgraduate courses delivered in partnership with an Online Programme Management company (OPM).
This exclusion seems like a glaring and puzzling omission, and THE has not publicly explained their rationale. However, it likely stems from their recent acquisition of Poet and Quants, the company I mentioned earlier. Including business in the online learning ranking could create competition with a company they now own.
This will inevitably have an impact on how comprehensive the ranking will be. UK universities will be currently considering participation, some may have portfolios heavily focused on, or exclusively comprised of business and management subjects. These institutions either won’t be eligible for inclusion despite offering online degrees, or if included, the rankings may offer a limited view of their online education activities.
How do THE plan to rank?
THE has not provided extensive details on their online learning ranking methodology, but it will focus on four main areas:
Resource
Engagement
Outcomes
Environment
Institutions offering online undergraduate, postgraduate degrees, or both are eligible for inclusion.
Resource
Resource is a critical metric. THE's inclusion of factors like people, finance, training, and will be key to indicating the importance institutions place on online education. However, the variance in location of online degrees and approaches among UK universities complicates this. Online degrees are not uniformly situated within institutions, and the way they are run and managed can vary. The most common dichotomy is between centralised and distributed models.
In the centralised model, online degrees are managed by an online arm or unit, often in partnership with an OPM or similar company. The distributed approach is one in which online degrees emerge more organically within specific faculties, managed predominantly at that level.
Some institutions use one approach exclusively, while others employ both, often resulting in significant differences in resources, especially when an OPM is involved. Because, one of the main reasons universities choose to partner with OPMs is the investment they are able to provide.
Given that an OPM partnership is highly likely to include a decent proportion of business and management focused degrees this might be less of an issue, but institutions where both approaches exist might experience either favourable or unfavourable representations in the rankings.
Partnerships that represent a franchised approach to online education, like those with companies like Learna, Robert Kennedy College, and Unicaf, may also present additional complexities. As will comparing universities specifically designed for online distance learning, such as the Open University and the University of London, with those offering online degrees as part of a broader portfolio.
The timing of resource allocation is another aspect to consider. Some online programmes receive substantial upfront resourcing, from learning designers, media specialists, project managers et al, but these are often time-limited and sometimes bought-in externally. This type of investment is a potential hallmark of quality but may be challenging to quantify in rankings.
In terms of training, it’s a no-brainer that institutions that invest in training their staff in online learning and teaching should be rewarded. However there is some nuance here. If itinerant associate lecturers or equivalent are used with significant online teaching experience, the level of training needed may be minimal. Considering the level of teaching experience is important here, especially given the tight margins in online education and the aim to avoid unnecessary training costs.
Engagement
Engagement is one of the most contested terms in education. THE’s measurement of this makes me think of the term “Regular and Substantive Interaction”, used by the U.S. Department of Education. For those that don’t know, regular and substantive interaction is a requirement imposed on online courses in order for them to be eligible for government financial support. It’s used as a means of distinguishing between online distance learning and correspondence courses. Initially, the lack of clear explanation for this term led to varied interpretations, although more detailed guidelines have emerged in recent years. It became a contentious metric, most notably when Western Governors University (WGU) was labeled a correspondence education provider in 2017.
THE plans to assess factors like “levels of student engagement, interaction with teachers and other students, and aspects specific to online learning such as usability and convenience."
Measuring levels of interaction is an interesting phrasing. It might point to a risk of prioritising quantity over quality. Research exists that indicates that quality responses from online educators are more beneficial than frequent, high volume interactions.
Regarding student-to-student interactions, it might be better to evaluate the opportunities for interaction rather than the actual levels. Some students prefer to get their head down and study in a more individual manner, supported by asynchronous self-paced approaches. There needs to be some consideration of this and not an over-privileging of interaction for interaction’s sake.
Engagement is a challenging area and one wonders how these rankings might tackle something like Georgia Tech’s AI teaching assistant Jill Watson. Online students rated the teaching assistant so highly it was nominated for a teaching award. That seems like pretty good engagement in my book, albeit not from a human. With AI assistants likely to permeate online education, this presents a significant and evolving aspect of student interaction to consider.
Outcomes
Outcome metrics are becoming an increasingly important aspect of higher education. THE has indicated that they will consider factors like graduation rate, progress rate, and student recommendation in their ranking. One hope would be that the characteristics of online education, such as how its particular format influences the rate and rhythm of study as well as the demographics it attracts, are accurately understood in evaluating these outcomes.
Earlier this year, THE reported, following a freedom of information request, that the Open University "only successfully graduates just over 20 percent of its overall intake on average." However, this figure is influenced by significant factors (that were mentioned in the article), including that half of the Open University's students belong to the lowest economic groups, and over 60% are first-generation students.
Online learning powerfully reaches different demographics of learners and that should be recognised and valued rather than penalised. It’s a form of study that’s inherently flexible and flexible options are only likely to increase, with more step-on and step-off points, and differing study durations. This flexibility accommodates the competing demands of life, work, and family that students face. This issue is not just for THE to consider in their rankings but the sector as a whole, with the Office for Students (OfS) B3 condition being an example of an aging square peg being painfully rammed into the round hole of online education.
Environment
This is a bit more of an opaque one, the THE explains it like this:
Environment: measures students’ learning environment assessing both the physical aspects such as tech support and offline resources, as well as the human aspects such as the mix of students and staff.
One hope on this one would be that it gauges the extent to which online students have access to student and academic support services comparable to their on-campus counterparts.
Online students are sometimes overlooked when it comes to these services, and those services are not always tailored to the needs and access requirements of online learning. It feels right that providers who excel in providing student and academic support services to online students should be richly rewarded here.
Challenges and opportunities
There’s no doubt that this new ranking will provide challenges to THE and it may take some time to fine-tune. This is to be expected and one would hope that THE engage extensively with institutions moving forward to that end.
This type of engagement could also aid those institutions that might opt out of the first round or have concerns about how the rankings could impact them.
Inevitably, rankings in higher education are divisive. Instances like Columbia University using misleading information to bolster its US News ranking don’t help their cause. There’s also no shortage of people lining up to criticise them and I don’t expect this to stop anytime soon.
On an episode of Online Education across the Atlantic, I half-jokingly said that I thought this ranking was doomed to succeed. What I meant here is that rankings are a type of shortcut we use in decision-making, and a high-profile ranking like this is going to be used by people who are unlikely to scrutinise the methodology but take it on trust. We would be lying if we each said we don’t also rely on shortcuts that we don’t scrutinise thoroughly when coming to decisions in various aspects of our lives.
However, a new data point that could enlighten and elevate online education is a positive thing, we all know it’s not going to be perfect, but in spite of that, this new ranking broadly seems like a step in the right direction.
