Transforming healthcare organizations and unlocking new sources of value, Salient CRGT provides end-to-end health analytics solutions, applying leading edge analytics techniques and large-scale health domain expertise to solve your organization’s challenges.
Case Study: Innovation for Storage and Analysis of Veteran Health Records
With an estimated 9 million Veteran patients, and legacy data that spans decades, the Department of Veterans Affairs (VA) maintains one of the largest Electronic Health Record (EHR) systems in the world. This system, known as the Veterans Information Systems and Technology Architecture (VistA), stores the EHR of every visit, diagnosis, lab, treatment, and other type of medical event or procedure that takes place under VA care.
Historically, each VA Medical Center (VAMC) maintained its own EHR using a system designed in the early 1970s for simple look-up and editing of Veteran records, and the vast majority of the data were in unstructured clinical notes that could not be searched or analyzed. In the 1990s, the VA consolidated VAMC-level databases, leaving behind a rich set of historical data in legacy systems that were inconsistently maintained. VA stakeholders wondered: Could the VA reduce storage costs and complexity while experimenting with modern search and analytics capabilities?
The Office for Information (OHI), in cooperation with several other VA offices, established a VistA Data Analytics (VDA) pilot project to consolidate the legacy systems on a modern platform in such a way that they could evaluate new search and analytics technologies for use in VistA, the Business Intelligence Service Line (BISL) and VA Informatics and Computing Infrastructure (VINCI). Our team, consisting of clinical healthcare, VistA, big data, and product experts would perform the implementation.
Creating a modern data integration and analytics platform was not going to be easy. The initial data resided in dozens of different legacy systems around the United States that were implemented using an early database technology known as MUMPS (Massachusetts General Hospital Utility Multi-Programming System) and VA was required to maintain full data lineage of every record for seventy-five years.
The systems were at different version and software patch levels, making it difficult to extract data. Veteran records and the way concepts like diagnoses and procedures were expressed varied by VAMC, so producing an integrated and uniform record would take work.
Using an agile development process consisting of short, iterative development cycles ending with product review demonstrations for stakeholders, our team created an advanced data integration platform that allowed users to perform near real-time analytics and full text, faceted, proximity, and semantic search across hundreds of thousands of integrated, lineage-maintained, and semantically normalized Veteran health records using a modern web user interface.
Our team immediately established an agile process and product backlog while standing up a virtual technical environment and working with VA stakeholders to survey the legacy systems. Engineers worked closely with VA legacy system owners to profile and ship the data, which we then loaded, inspected and extracted using a combination of unique manual and automated methods that ensured no single item of data was lost or altered. We loaded and integrated the data into a NoSQL database capable of supporting search and analytics across the hierarchical structured and unstructured EHR data, where we used automated techniques to enforce a schema and ensure no data were lost or corrupted.
In addition to integrating the data, we enriched it with standard medical ontologies and Natural Language Processing (NLP) annotators, enabling researchers and clinicians to search and analyze the data in unstructured clinical notes by medical concept. This process surfaced numerous references to procedures and diagnoses beyond those captured in the structured data. Our engineers also geocoded locations within the data and exposed a rich interface where users could select concepts like diagnoses or exposures by map regions. The modern web interface, with its integrated patient record views and clinical notes summaries for clinicians; and its charts, graphics and search capabilities for researchers; made performing previously impossible or difficult tasks easy.
“These advances enabled VA users to analyze data across large numbers of records and discover previously hidden concepts in critical clinical notes, all in sub-second time.”
– Chief Information Officer, Salient CRGT
VistA Data Analytics exposed VA stakeholders to a new approach for integrating veteran EHR data for consolidation, search and analytics in research and clinical settings. As the number of veterans grows, VA’s need to provide researchers and clinicians with more effective and efficient technological capabilities will also increase.
As a partner to the Federal government, Salient CRGT is continuing to leverage its best practices around data collection, analysis, and visualization, to help customers throughout the government push forward in their missions.