May 18, 2009, 5:16 PM — Hospitals, medical centers and other health systems are increasingly using digital dashboard technology to provide relevant, up-to-the-minute information to clinicians in a visually rich format to improve the quality of patient care.
Designing and using clinical dashboards requires substantial physician involvement and a well-defined process. The University of Pennsylvania Health System, a.k.a. Penn Medicine, is currently in the midst of developing the Penn Data Store, a data warehouse and series of individual dashboards, to serve our clinicians and researchers. The Penn story may be of use to you and your organization as you move forward in the world of dashboards.
Our goal was to create a single data storehouse with all patient, administrative, financial and supply chain data mapped to a standard data model and vocabulary. We created a number of objectives in support of this goal in areas such as patient-care quality, safety, compliance, research and finance.
After evaluating a number of choices, we decided to develop our dashboard using the enterprise edition of Oracle Corp.'s Business Intelligence Suite. While not all users' needs will be met with predefined dashboards, a significant portion will. We assembled a wide-ranging team and divided each dashboard project into the following phases:
1. Meet with users to determine major data needs
Many users and potential users of dashboards aren't familiar with the systems' power and capabilities. They typically use noninteractive spreadsheets and graphs that present data in fixed rows and columns and lack the flexibility of Web-based tools.
The first step in this phase was to inform clinicians of the many additional possibilities that a more powerful tool offers. We did this by first building sample dashboards to demonstrate the tool's capabilities. We then asked the clinicians to identify several key measures, dimensions and filtering criteria for the dashboards they were interested in. For example, a key measure on our sample Medication Administration dashboard was whether the patient's medication list was reviewed with the patient during a visit. One user said he would be interested in seeing that dashboard display monthly and annual statistics, such as percentage of patient visits during which medication lists were reviewed, both for individual physicians and group practices as a whole. After gathering such input from users, we identified the source data and actions necessary for responding to each request and configured the physical layer accordingly.
Next we demonstrated the graphical capabilities of the tool to our clinicians in order to help expand their thinking about how they could use the dashboards. One user wanted to make it possible for physicians and group practices to compare their performances on certain measures against those of their peers. The ability to summarize and access actual patient visit details was important for doing that. From a security perspective, however, patient names couldn't be displayed and physician names could be shown only to members of the same group practice. The user decided to use bar charts and organized the dashboard into pages that showed annual and monthly statistics by group practice and other metrics. Users could learn more by clicking on the bars in the charts or by clicking on a physician's name if they were in the same practice.
2. Design the presentation layer
For dashboards to deliver the most benefit, users must agree on presentation standards before the design phase begins. It's crucial to achieve agreement on items such as color schemes, graphical objects and navigation standards so future dashboards will look, feel and behave consistently. This will improve user satisfaction and reduce training demands for each new dashboard.
Designers must think carefully about the hierarchy of the data they want depicted in the dashboards and what levels of the hierarchy will be visible. In our example, the hierarchy is Practice Physician Patient. Depending on a user's security status, he may or may not be able to see the patient data.
Once the hierarchy and associated measures such as year, month and practice name were grouped into a dashboard page, we selected the graphical elements. We chose histograms, line graphs, pie charts and simple tables to present the data in an intuitive fashion that met users' needs and simplified navigation through the dashboard pages.
We also developed online documentation for each dashboard and for each page of each dashboard. The overview information for each dashboard is presented as the first page. Additional comments about a specific page of the dashboard are included on the appropriate dashboard page.
That makes it easy for viewers to understand the aggregate data behind the dashboard pages as well as any data limitations and assumptions before seeing the particular data they're interested in. This helps avoid confusion or misinterpretation of the results.
3. Design the semantic layer
The semantic layer maps the presentation layer to the physical layer. Developers prove their worth in the design of this layer. Defining patient populations by disease categories, grouping drugs hierarchically by therapeutic area and organizing physical locations are examples of the challenges that the semantic layer's designer faces. Users play a key role in formulating those definitions. In-depth knowledge of both the presentation layer design and the physical data models is essential.
4. Design the physical layer
As mentioned, try not to change the design of the physical layer. Whenever possible, avoid creating an entire new physical data structure, because doing so generates the need for additional extract, transform and load (ETL) steps each time the clinical data warehouse is updated. Redundantly storing data produces additional storage, backup and maintenance costs and opens up the risk that duplicate copies of data won't be updated with the same frequency as the original.
If the designer and data architect determine that redundant structures are required, however, the designer should clearly document that need and ensure that the online documentation in the presentation layer reflects the redundant sources. The ETL developer will also need specifications for creating and maintaining these redundant sources.
Finally, using realistic data volumes, simulate dashboard performance during this phase by executing typical data queries for each page, drill-down and filter selection.
5. Develop and test all three layers
Users who are new to the dashboard development process will likely need to see how the systems operate with real data. We have found it useful to introduce clinicians to a working prototype to gain early feedback on the design. Regular weekly demo and review sessions then help developers refine and test the design. When you're engaged in this phase of the project, take care to manage scope creep, since participants might be tempted to request new capabilities or data that weren't in the original design. Put off responding to such requests until a subsequent release of the dashboard.
6. Perform quality assurance
Here are several quality-assurance requirements we developed at Penn Medicine:
- Use data from the actual data warehouse, rather than simulated or test data.
- Include monthly and quarterly data warehouse updates during the QA process, especially when the data being used in dashboards will cross calendar months, quarters and years. A year-end data update would be ideal, but most projects can't wait that long.
- Compare dashboard data to the original source systems, to ensure that no translation or presentation errors were introduced during development.
- Test system performance and response times with large amounts of data to ensure that the dashboard responds effectively to users' needs.
7. Conduct pilot tests
Before making a dashboard available, we ask a small group of users to pilot-test it for a few weeks, as a short extension of the QA phase. This provides additional information on usability, performance and quality. If the pilot group recommends moving forward, we proceed to the general rollout. If the pilot group identifies problems, the development team resolves them.
8. Begin general rollout
This phase involves these major components:
- User orientation: New users may need a brief training session. Those who have used dashboards in the past should be able to use the new dashboard with no assistance -- if the presentation layer was designed well.
- Support documentation: Gather design and operational specifications and post them on support sites.
- Help-desk hand-off: Write scripts and give them to the help desk for guidance in responding to support calls.
To date, we've created clinical dashboards on these six subjects at Penn Medicine:
This approach has met our needs. Our next focus will be on evaluating the contribution that dashboards make to our patient-care quality, safety and compliance goals.
Wells is chief technology officer for information services at the University of Pennsylvania Health System. Contact him at Brian.Wells@uphs.upenn.edu.