Exploring codelists in OpenSAFELY: the ehrQL Codelist Explorer

As part of some broader work looking at how researchers create and reuse elements of their ehrQL¹ code, we wanted to better understand how codelists (which are often the foundation of a variable - see here for more info) are actually used in OpenSAFELY studies. To help us explore this, we built a small web tool called the ehrQL Codelist Explorer, which allowed us to explore every codelist that has ever been used² in studies across the whole of OpenSAFELY. You can take a look yourself here: https://prototypes.bennettoxford.org/codelist-explorer - but bear in mind that this is a snapshot in time, and not something that we regularly update.

What this doesn’t show

The Explorer doesn’t show every codelist that exists in OpenSAFELY studies. It only shows the codelists that have actually been used to query the backend patient data at least once. Codelists in studies that haven’t been run, or codelists that exist in OpenCodelists but not in any study code, are not included.

The codelist view

Initially the display shows a table with one row per codelist.

When we select a codelist, the row expands to show all the different versions of that codelist that have been used.

A concrete example: dementia

There is a codelist called “Codes for dementia”

From the collapsed row we can see:

• it was created by the NHSD primary care domain refsets
• it uses the SNOMED coding system
• it has four different versions
• those versions are used across 23 variables
• spread over four repositories / studies in OpenSAFELY

Expanding the row shows each of those four versions in detail.

For each version, we could see things like:

• when that version was created
• its current status (for example, whether it’s under review or published)
• which release of the coding system it was built against (for example, SNOMED version 39.2.0)
• how widely it’s been used:
  • how many variables
  • how many studies (repos)
  • when it was first used
  • when it was last used

I know I said this was all codelists that have ever been executed. The slight caveat is that if a codelist has multiple versions, only some of which have been executed, then we show all the versions - this is why the version with the tag 20241205 appears, even though it has never actually been executed.

This gave us a quick sense not just of what exists, but what’s actually in use and how widely.

Drilling down to individual usage

Selecting an individual codelist version allowed us to go one level deeper again.

Here we can see the codelist version with tag 21b792ed has been used:

• in 16 variables, and each of their names
• in 2 studies, disease_incidence and inflammatory_rheum

This shows us exactly how each codelist was used in each study.

Searching across codelists

There’s also a simple search box.

This allows us to easily see, for example, any type 1 diabetes codelists using ICD, if we just enter type 1 diabetes icd.

The results immediately show us:

• how many matching codelists there are (in this case just 1)
• how many variables they’re used in (3)
• how many studies they appear in (2)

The repository view

The second main perspective is the repository view. Instead of starting from a single codelist, this allows us to see which codelists are being used in each repository.

In this view:

• each row represents a single version of a codelist
• used in a single variable
• in a single repository

For each row, we can see:

• the repository name
• the codelist
• the specific version
• the variable it was used in
• the status of that version
• the coding system release
• when it was created
• when it was first used
• and when it was last used

Summary

We built the codelist explorer as part of our team’s broader work exploring how researchers create and reuse elements of their ehrQL code. As part of that, we looked at patterns in how codelists are used across OpenSAFELY, including how often the same codelists appear in different variables and studies.This tool allowed us to bring that usage data into a form that was easier to inspect.

We found that codelists were rarely reused across repos/studies, which was much less often than we expected. Codelists are often used across multiple variables, but this seems to be largely concentrated within the same studies. Only a small number of codelists have been used across different studies, and these tend to be codelists created by the OpenSAFELY organisation rather than individual researchers.

The work gave us a better understanding of what codelist use and reuse looks like in practice, while raising further questions about what reuse means and when it is valuable. Our parallel work has identified some barriers to reuse, but has also shown that researchers mean different things by ‘reuse’ in different contexts. Together, these findings are helping us build a more nuanced picture of where reuse might be useful, which will inform the direction of our wider work.

As always, if you have any questions, please get in touch: team@opensafely.org