Radiology reading times

In one of my previous data science roles, I worked on the radiology scheduling problem. We were trying to reduce SLA breach of urgent orders - as this was a performance metric that immediately resulted in penalties and lost revenue.

The engineers had already recorded reading times for orders as a separate column.

The number that could not be right

One of the things I did as part of understanding how radiologists behave - was to look at cumulative sum of reading times for a radiologist over a day, over a week, over a month.

Something immediately struck out as odd. Almost all radiologists had cumulative reading times per day that averaged around 25 hours. This is of courses, physically impossible.

One radiologist, one session, as the metric saw it

Reading time this session

3 h 45 min

Time the radiologist actually worked

1 h 21 min

Which is

2.78×

Overlapping studies each get credited the same minutes, so a 1 h 21 min stretch of work reads as 3 h 45 min. Across a nine hour shift that is about 25 hours of reading time in a day.

Study open Time credited to it Workstation idle

The numbers behind this chart

Study	Open	Closed minus opened	Credited
A	09:00–10:12	1 h 12 min	1 h 12 min
B	09:08–09:30	22 min	22 min
C	09:12–09:51	39 min	39 min
D	09:35–10:18	43 min	43 min
E	09:50–10:10	20 min	20 min
F	10:06–10:35	29 min	29 min
Total		3 h 45 min	3 h 45 min

Two suspects

My suspicion was there was something wrong about the recorded reading times - specifically - they are over recording the actual reading time. Or it is also possible there was something wrong in the cumulative calculation - that some or all orders were being over represented and counted multiple times.

I talked with the engineer who had added this field - and he explained the query that was returning this field. There was no over counting happening of orders.

This meant the likely issue was how we were determining reading times. We pulled the PM in for discussion and it became clear that our existing way to record reading times was simply to look at when the radiologist opened the study and when they closed it.

How radiologists actually work

Now, this is not an inherently bad guess at reading times. The problem is - this is not how radiologists actually work. And the PM was of great help in explaining the radiologist workflow. It was common for radiologists to have discussions about complex orders, to have multiple orders open at the same time and often be able to dictate through all open orders right at the end.

The same shift under the workflow the metric assumed, and the real one

Reading time this session

1 h 35 min

Time the radiologist actually worked

1 h 21 min

Which is

1.17×

With one study open at a time, closed minus opened is almost right - it only over-counts the 14 min the workstation sat idle.

Study open Time credited to it Workstation idle

The numbers behind this chart

Study	Open	Closed minus opened	Credited
A	09:00–09:16	16 min	16 min
B	09:16–09:30	14 min	14 min
C	09:30–09:45	15 min	15 min
D	09:45–10:02	17 min	17 min
E	10:02–10:18	16 min	16 min
F	10:18–10:35	17 min	17 min
Total		1 h 35 min	1 h 35 min

Asking a different question

I then dove deeper into other timestamps and events our software was recording for the radiologist.

This helped us sort all the orders being read by a radiologist in a session by their opening time and dictation end time.

We also removed idle times by looking at timestamps for computer use.

The same session under each rule

Reading time this session

1 h 21 min

Time the radiologist actually worked

1 h 21 min

Which is

1.00×

Splitting each slice by how recently each study was opened, and dropping the idle stretches, the credited time adds up to exactly the 1 h 21 min that were there to give out.

Study open Time credited to it Workstation idle

The numbers behind this chart

Study	Open	Closed minus opened	Credited
A	09:00–10:12	1 h 12 min	18 min
B	09:08–09:30	22 min	9 min
C	09:12–09:51	39 min	13 min
D	09:35–10:18	43 min	15 min
E	09:50–10:10	20 min	9 min
F	10:06–10:35	29 min	18 min
Total		3 h 45 min	1 h 21 min

Why it mattered downstream

The same department, planned with each set of reading times

Allocated times - 13.5 min a study, capacity 4.4/h Recorded times - 37.5 min a study, capacity 1.6/h

The numbers behind this chart

Orders an hour	Allocated times	Recorded times
1.0	17 min	100 min
2.0	25 min	over capacity
3.0	42 min	over capacity
3.6	71 min	over capacity
4.0	135 min	over capacity

A single server M/M/1 queue, which is a toy, but the shape near capacity is the part that matters.

What it changed

This brought down the cumulative reading times per radiologist per day down to about 9 hours - which represented a typical shift.

Cumulative reading time, per radiologist per day	Median	Range	Over 24 h
As recorded, closed minus opened	25.1 h	20.4 – 31.5 h	33 of 45
After allocating the shift	9.0 h	8.0 – 10.8 h	0 of 45

From here, our modelling could proceed on a solid ground.