Cancer staging is essential for prognostication, treatment planning, information exchange, research, and public health.1 The American Society of Clinical Oncology (ASCO) Quality Oncology Practice Initiative (QOPI) has designated timely staging documentation, defined as documenting stage within one month of the first office visit, as a key performance measure.2 However, staging is often documented within the electronic health record (EHR) using free text, which is difficult to abstract without manual chart review or machine-learning.3,4
New patient visit (NPV) staging is educational for oncology fellows given the complexity of the American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) staging criteria.5 Our Comprehensive Cancer Center (CCC) adopted an EHR staging tool (EST) to facilitate structured staging using AJCC TNM criteria. However, at baseline, only 6.1% of new patient visits (NPV) at our Comprehensive Cancer Center (CCC) were staged using the EST. To ingrain the important habit of staging in our trainees and thus promote the associated clinical and public health benefits for patients and clinicians, we applied quality improvement (QI) methods to improve EST usage for fellow-seen NPVs during the 2018-19 academic year.
The University of California, San Francisco (UCSF) hematology/oncology fellowship has seven first-year fellows, and several advanced fellows (16 during our study period), who rotate through ambulatory clinics. In our CCC, fellows see patients in the faculty practice supervised by attendings. The attendings edit and attest fellow notes, but the attending attestation does not typically include additional clinical information; thus, the fellow’s documentation of staging is crucial. Given our focus on TNM staging, we limited our initiative to solid oncology clinics. Given ASCO QOPI’s emphasis on staging within one month of the initial visit, we limited our project to NPVs.
Our EST (Epic, Verona, WI) consists of an optional module that can be completed before, during, or after a visit. The module generates structured AJCC-based staging prompts for any cancer diagnosis in the patient’s problem list and contains additional optional fields for histology, tumor markers, and other disease-specific data. Root cause analysis performed via one fellow focus group, one discussion with fellows and faculty at our division’s conference, and discussion among our QI project team which included fellows, faculty, and cancer registry administrators, identified unfamiliarity with staging and with the EST as leading contributing factors to low baseline EST usage (Figure 1). We targeted our interventions to address this unfamiliarity and the need to integrate staging into the visit workflow.
The primary outcome measure was the monthly rate of timely staging documentation using the EST among fellow-seen NPVs. We aimed to increase monthly staging documentation rate for solid oncology NPVs seen by fellows using the EST to 70% by June 2019.
We pursued two Plan-Do-Study-Act (PDSA) cycles. PDSA 1 lasted from September 2018 through December 2018. We identified lack of access to the staging module in standard work flow and lack of familiarity with the staging module among the fellows as key drivers in the root cause analysis. In PDSA 1, to address lack of access to the module, we embedded the module into the visit navigator to manage variation and improve workflow. PDSA 2 began in January 2019. Early in PDSA 2, to address lack of familiarity with the staging module among fellows, we reemphasized the importance of the EST using instruction sheets, flyers, and monthly email reminders to fellows on ambulatory rotations to exploit and manage variation. The emails included aggregate EST usage statistics for fellows and for each clinic. We sent individualized reminder emails to fellows with individual staging rates persistently below 50% early in PDSA 2. The process measure of individual fellow staging rates allowed us to see the impact of these emails. PDSA 2 lasted until June 2019.
Data collection and analysis
We assessed EST usage monthly with a lag time of 30 days to adhere to the ASCO QOPI recommendation that staging be completed within one month of a NPV. NPVs where staging using the EST was not possible (e.g., carcinoma of unknown primary) were excluded by manual review.
We tracked fellow-seen NPV staging rates, and, using p-type statistical process control (SPC) charting, CCC-wide NPV staging rates. In May 2019, we conducted an ad hoc qualitative stakeholder analysis by soliciting input from fellows about barriers to staging.
Each clinic’s performance was displayed on their visual management boards and highlighted during monthly meetings with CCC leadership. We reviewed the list of fellow-seen NPVs monthly and extracted individual fellow staging rates as a process measure to identify low-performing fellows for individualized intervention.
Following disruption by the 2019 coronavirus pandemic, we explored the extent of the EHR module’s long-term integration into clinic workflow by interviewing a convenience sample of nine faculty in two oncology clinics in January 2021.
We adhered to the SQUIRE 2.0 guidelines for reporting QI work throughout this report.6
Institutional Review Board review was not required for this QI project at our institution.
Fellow-seen NPV staging rates improved from 47% in July 2018 to a mean of 75% during PDSA 1 and 70% during PDSA 2, and fell thereafter with the start of new fellows (Figure 2). We saw particularly high fellow performance in December 2018, likely due to our work publicizing the project and new fellows becoming accustomed to the EHR and clinic workflow. As an unintended consequence of the project, CCC-wide NPV staging rates mirrored fellow rates, improving from 6.1% at baseline to 16% during PDSA 1, 13% during PDSA 2, and fell thereafter (Figure 2). Fellows saw 11% of the 6992 NPVs seen at our CCC in 2018-2019; thus, some but not all of the increase in CCC-wide staging was due to fellows. To our knowledge, there were no other outreach initiatives regarding the EST apart from our project specifically directed toward fellows.
Three fellows (13% of 23 fellows involved in NPVs) were identified as low-performing during PDSA 2 and received individualized emails. Their NPV staging rates subsequently rose above 50%. Our stakeholder analysis of fellows confirmed our QI initiative as a worthwhile educational endeavor but revealed the following barriers: inconclusive pathology reports, inadequate discussions about staging with attending preceptors, time constraints during NPVs, failure to remember to use the EST, and, most prominently, incomplete workup or missing records at the initial visit.
By January 2021, some faculty were using the module (“Makes my life tons easier…a useful patient education tool,”) and others were not (“I am already overwhelmed with documentation requirements.”). One faculty member did not know that the module existed. Another supported the module’s educational value: “When there are learners in my clinic, I push them to do this.”
We developed and implemented a QI project to increase oncology fellow usage of an EST to ensure comprehensive NPV staging. Our fellow-seen NPV staging rate rose substantially during a 12-month period including 2 PDSA cycles. Our experience suggests that, although fellows generally only see a minority of patients at any CCC, fellow-led QI initiatives can change other clinicians’ practice and can be reflected in CCC-wide metrics. Of note, attendings and other team members may have seen our project’s flyers, which may have influenced their behavior.
Sustainability was challenging. After a substantial decrease when new fellows started, we subsequently saw a sharp rise in fellow-seen NPV staging in the second half of the academic year without resumption of our QI efforts. We could not investigate this rise due to the pandemic. Faculty interviews in 2021 suggested variable adoption of the module.
Our initiative’s strengths include our usage of serial PDSA cycles, real-time feedback from publicly posted staging rates on clinic boards, the inclusion of CCC-wide staging rates, and the fellow-led nature of the project which allowed for an educational benefit in addition to systems improvement. Given the ubiquitous use of EHRs, our initiative is transferable to other practices. We had planned a third PDSA cycle including (1) discussing with fellows the importance of completing staging even if workup is initially incomplete and (2) expanding to malignant hematology where AJCC-based staging is not used. Unfortunately, these plans were disrupted by the coronavirus pandemic. We hope to initiate a third PDSA cycle in the coming year.
Our analysis was limited by the inability to obtain a lengthier baseline for fellow NPV staging retroactively; we used CCC-wide baseline data instead. Our first recorded fellow datapoint of 47% was likely inflated due to fellows’ knowledge of the project from fellowship orientation. Fellow inexperience likely contributed to lower staging rates in July, and some of the rise may have been attributable to increasing experience rather than our QI efforts. Although fellow-led projects are educational, fellows lack the experience and authority of faculty, who have been embedded in the clinical system for a longer time and have greater impact on their colleagues’ behavior.
In conclusion, our QI initiative led to an increase in structured staging documentation, which may have clinical and educational benefits. Due to upcoming coding changes, our CCC is developing standardized note templates with the EST embedded. This redesigned workflow will facilitate EST sustainability without the need for reminders.
Conflict of Interest
i. All authors: conception and design
ii. All authors: data collection and assembly
iii. All authors: data analysis, manuscript writing
All authors have approved the manuscript