An academic paper has suggested that a deal between DeepMind, google’s artificial intelligence firm and the UK’s NHS had serious “inadequacies”. More than a million patient records were shared with DeepMind to build an application to alert doctors about patients at risk of acute kidney injury (AKI). The authors said that it was “inexcusable” patients were not told how their data would be used. Google’s DeepMind believed that there were ‘major errors’ in the report.
When the deal between London’s Royal Free Hospital and DeepMind became public in February 2016, some three months after the data started to be collected, it caused an argument over the amount of patient information being shared and the lack of public consultation. There are still big questions to be answered about the tie-up, said Hal Hodson, a former New Scientist journalist, and co-author Julia Powles, a Cambridge University academic. The application contains no artificial intelligence although DeepMind has said that it was hoping to incorporate AI techniques to create smarter alerts in future.
It was difficult for DeepMind to make clear that none of the data collected for the app has been shared with parent company Google. AKI is a serious condition, caused 40,000 deaths a year in the UK and leading to a range of other health issues from minor kidney dysfunction to the need for transplant and dialysis. In February, DeepMind published details about how the app was improving patient care. It revealed that more than 26 doctors and nurses at the Royal Free are now making use of Streams and that each day it alerts them to 11 patients at risk of AKI.
DeepMind has acknowledged that it could have done better in the way it involved with patients whose data was being used and, on the back of the criticism, agreed to set up patient forums. It published a strategy on future patient engagement which opens by saying: “Outcomes are better when clinicians and patients together make the decisions.”
The NHS does have information-sharing agreements with a range of third-party firms, but this is the first such deal with a leading US tech firm. Initially, DeepMind asserted that the NHS had 1,500 other agreements with third-party organizations that process patient data has since been characterized by the NHS as “inaccurate”. The app is currently the subject of an investigation by the Information Commissioner’s Office while the National Data Guardian, tasked with safeguarding health data, is also looking at it.