91 - A one year review of enhancements to our Continuing Review Submission Process.

Background: In late 2022, our IRB enhanced the CRR form by further automating study data from our review system, PIMS. Prior to our enhancements, the CRR generated as a mostly blank form. This freedom resulted in discordance with readily available information in PIMS, which accounted for 45.6% of our pre-review returns.

Program

Description:  The overhaul of our CRR form, which is submitted within our Protocol Information Management System (PIMS), was three-fold. First, questions were revised to be explicit with information bubbles to guide study staff. Second, questions involving data or auditing and monitoring reports already in PIMS were automated. Finally, two new modules were developed to log OHRP Incident Reports and Participant Complaints. Previously, both were tracked locally by study staff.

To assess the success of this program one year post implementation, we employed both quantitative and qualitative measures. To evaluate the quantitative impact, we reviewed submission metrics from one-year pre and post implementation and compared the proportion of Continuing Review submissions that were returned for edits during pre-review by IRB staff.

To understand the impact from a qualitative perspective, we reviewed 50 submissions that were returned during pre-review before and 50 after the implementation, and examined the reasons submissions were returned to staff for updates.

Program

Assessment:  We noted a 9.1% relative change in submission returns during pre-review following implementation (15.33% pre implementation vs 16.72% post implementation; 1.39% absolute change). There has also been an 88% decrease in returns related to newly automated questions. Previously those questions were completed manually by the study team.

Prior to the enhancement, participant status data involved manually transcribing information into a free text section which was supplemented by an automatically generated report from our CTMS. At times status data was discordant. We allowed clarification in the free text section of the form instead of real time data reconciliation within the CTMS. Study teams were alerted to reconcile data prior to the next submission. Presently, submissions are returned for real time participant status reconciliation which we attribute to more stringent requirements in pre-review workflows.

The addition of automated questions/responses has led to study teams having more time to reconcile errant data and ensure it is accurate. This improved the overall quality of CRR submissions across our institution.

Limitations: Due to resources, our qualitative sample size included 100 CRRs with return comments; 50 prior and 50 post enhancement.

This sample size conveyed an errant conclusion that there were novel pre-review return reasons stemming from the enhancement. A common comment was no update on study progress since the previous CRR, resulting in a returned submission. The lack in sample size implied this was a new return comment post enhancement, which is not the case.

Discussion:  In conclusion, the CRR form Enhancement resulted in a more convenient user experience for all parties. These innovations have granted the IRB office more time to focus on pre-review and problem reconciliation. Additionally, IRB reviewers have a more comprehensive snapshot of study status during their review. Finally study teams can focus on thoughtful responses to form questions and data reconciliation.

As an IRB’s portfolio grows, it is essential that the CRR submission process is accurate, scalable, and comprehensive for IRB reviewers. We recommend all institutions adopt automation and leverage technology to improve review workflows and increase accuracy in continuing review submissions.