95 - Effectiveness of a Virtual Training Program, 2023

Background:  Since 2021, the National Center for Emerging and Zoonotic Infectious Diseases (NCEZID) Information Collections and Human Studies (NICHS) Team has provided NCEZID staff with voluntary trainings to improve investigators’ knowledge of information collection and human subjects-related laws, regulations, and institutional policies. The aim is to promote regulatory and institutional compliance and scientific integrity among NCEZID staff.

Program

Description:  Seven trainings in total were presented from April through October 2023, via Zoom. Training sessions were evaluated using a qualitative survey to assess attendees’ self-reported comprehension, and opinions about the training content. Surveys were administered anonymously through the Research Electronic Data Capture® (REDCap) system and analyzed in Microsoft Excel (2023).
The evaluation survey included 11 questions covering three domains: overall course effectiveness, instruction, and delivery. Questions were presented in a 5-point Likert scale format, ranging from Strongly Agree (5) to Strongly Disagree (1). The final question of the survey was an open-ended text field for respondents to provide additional comments.
An End-of-Year (EOY) survey was administered to NCEZID attendees and non-attendees -in November to assess reasons for attending or not attending, and to solicit additional training topics for the 2024 training season. The EOY survey was composed of seven questions, including three open-comment fields.

Program

Assessment:  Attendance ranged from 23 to 64 attendees per session, with a median attendance of 43 across the seven trainings. Survey response rates ranged from 4% to 70%, with a median response rate of 24%. Across all domains of effectiveness, scores ranged from 3.8 to 4.5 (average 4.1), indicating positive results, overall. By comparison, attendance averaged 77 attendees per session with a 21% average response rate across the 2021-2022 training seasons. Domain effectiveness averaged 4.3 for the same time period.
Of 40 EOY survey respondents, 57% reported not having attended a training session in 2023. Reasons for not attending included not knowing about the trainings, being too busy, not being available, and not being interested. Of those reporting attendance, 12% reported attending only one session. The remaining 88% reported attending more than one training, with nearly 25% reporting attendance at 6 trainings. No attendees reported attending all seven. The top five topics of interest to respondents were PRA, project submission processes, Certificates and Assurances of Confidentiality, health equity, and FDA regulations.

Limitations: The primary limitation in 2023 was low participation. We saw a 44% reduction in average attendance per session, compared to 2021-2022. This may be due to waning interest. Most of the trainings in 2023 covered the same topics as in 2021-2022. We will address this issue in 2024 by incorporating topics recommended in the EOY survey, and by providing more detailed information in training announcements.

Discussion:  Despite low attendance in 2023, this program continues to provide an important service to NCEZID staff and is well-received by attendees. Cross-domain ratings remained above 4, reflecting positive scores overall. The team will continue to work on presentation skills, and the duration and pace of trainings to address areas receiving lower scores. In addition, several of the EOY-suggested topic areas have been included in the 2024 training program.
Overall, our findings demonstrate that information collection and human subjects training can be presented effectively in the virtual workspace, but that extra effort may be needed to improve staff attendance.