Managing data as an asset is our first principle for a reason. If we manage our data as an asset across the data lifecycle this will ensure a strong and stable foundation to support everything we do with data. Managing our data well will support us in:
- ensuring data quality and accuracy to support and maintain trust in our data and decisions we make with it
- enhancing decision making by ensuring that decision-makers have access to the data they need when they need it
- complying with legal and regulatory requirements related to data protection and security
- providing efficiencies in delivery and storage thus minimising costs and environmental impact
Managing data as an asset is not the responsibility of a single team, it’s a collaborative effort that requires input and involvement from across the organisation. In recent years we have invested in improving data management beginning with introducing a data governance framework including roles and responsibilities for data governance across the organisation.
This was followed by the implementation of a data asset register where critical data assets are identified, and associated improvement plans are put in place. More recently we have established a data profiling capability and a data architecture function and published our first data standards. The benefits of investing in good data management across the data lifecycle can be seen in the high quality, curated data sets which underpin our external reporting.
We also adhere to the Caldicott Principles which provide a framework for responsible data management in healthcare, emphasising the importance of privacy, confidentiality, security, and compliance with legal and ethical standards.
Whilst we celebrate our successes, we also recognise the challenges we face and remaining gaps which we are dedicated to addressing.
While the introduction of our data architecture function and publication of our data standards are a step in the right direction, we now need to ensure that data architecture becomes embedded within our delivery and operational practice and that the data standards are implemented across the data lifecycle. This includes the use of unique identifiers, the standards for which are set out in our data standards but are not yet fully adopted across our services. Inconsistent capturing of unique identifiers across our services has a direct impact on our Master Data Management (MDM) practices. We have areas of good practice of MDM such as in Primary Care Services but inconsistencies in others including challenges in our citizen customer data where the use of different identifiers across services and systems present difficulties for interoperability and MDM.
To date, our data management practice has tended to focus on structured data. However, like many organisations, we also hold a large volume of unstructured data, some of which is critical for the delivery of our services. We therefore need to ensure our data management approach is adopted across all data, adapting where necessary.
We have also identified limitations in our current data storage and retention practices with particular challenges relating to unstructured data. To support us in addressing this, we have developed a specific data retention and storage strategy which we commit to delivering.
Ensuring we have a comprehensive approach to data management which accounts for all data and services and is embedded into our delivery practices will support consistency across our systems, contribute to improving data quality, and facilitate interoperability and integration.
Data access is a crucial component of data management as it enables the use and reuse of data without which we cannot hope to derive value from it. However, our data is captured in disparate systems which presents data access challenges. We introduced our Data Ingestion Service (DIS) with the goal of providing a timely solution for moving data from operational systems to data storage and access solutions. Although the service has brought some improvements, we recognise that it can be enhanced.
Historically, efforts have focused on enabling access for external users of our Primary Care Services data. As a result, our current data platform and operating model are designed primarily to support external facing data products. They lack the agility to meet more diverse user needs and struggle to meet the growing demand for data access across the organisation in a cost effective, efficient manner.
To address both these issues, we have started planning for the introduction of a Modern Data Platform (MDP) and Data Operating Model (DOM), with project completion expected by the end of 2025. This will enhance how we move data around the organisation, provide improved and efficient data access for both internal and external users, greater flexibility to meet varied user needs and reduce the time taken to gain insights.
By shifting to a more flexible, modular, data platform, with a focus on interoperability, we will be able to adapt more easily, capitalise on the best tools available for different purposes and contribute to the reduction of our technical debt. Overall, it will give us a stronger foundation for creating our data products.
The technical solution will be complemented by enhancements to our ways of working including further development of our data policies, standards and playbooks supporting the standardisation of approaches allowing colleagues to efficiently produce outputs with consistent quality, look and feel which enable users to move swiftly between our data products.
We aspire to provide enhanced data and reporting for both internal and external users of our data. This includes greater focus on the use of anonymised and pseudonymised data, enabling us to do more while reducing the risk to the data subjects. We also believe we could deliver greater value and enhanced insight to the wider health sector by being able to link data sets across different services and with external data. However, our existing governance documents are largely silent with regards to data, meaning we are limited in what we can legally do with the data we hold and how we can share it.
Finally, we do not have a comprehensive view of our data maturity (including data literacy) across our organisation. Understanding our data maturity is key in directing and prioritising our data management activities moving forwards.
Commitment
We will enhance our data management by:
- working with DHSC to attain appropriate legal powers for further added value use of our data
- ensuring compliance, security and efficient storage utilisation for documents and unstructured data
- embedding data related policies and standards across Digital, Data and Technology (DDaT) and the broader organisation and determine the approach for legacy systems
- undertaking and publishing a data maturity assessment to identify further improvements
- improving data access and discoverability
- reviewing and updating our approach to Master Data Management (MDM) across our estate