Skip to main content
Download PDF
Download ePub

A pragmatic randomized trial to compare strategies for implementing primary HPV testing for routine cervical cancer screening in a large healthcare system

Implementation Science volume 20, Article number: 21 (2025) Cite this article

Abstract

Background

Recent updates to national guidelines recommend primary human papillomavirus (HPV) screening for routine cervical cancer screening alongside previously recommended screening options. However, limited guidance exists for implementation approaches that best facilitate cancer screening practice substitution and achieve optimal stakeholder-centered outcomes. We compared “centrally-administered + locally-tailored” (here after referred to as locally-tailored) vs. “centrally-administered + usual care” (here after referred to as centrally-administered) approaches for achieving substitution of HPV and cytology co-testing with primary HPV screening for routine cervical cancer screening to examine the effect of local tailoring on implementation and stakeholder-centered outcomes.

Methods

We conducted a pragmatic, cluster randomized trial embedded in the Kaiser Permanente Southern California (KPSC) health system, randomly assigning site groups to study arms at the level of the geographic service area (12 service area randomized). The study took place between 2020–2022. Centrally-administered implementation strategy bundles included physician and staff educational activities. Sites in the locally-tailored arm underwent local needs assessment followed by local selection, tailoring and deployment of implementation strategy bundles. The primary outcome was the proportion of primary HPV screenings among all screenings performed. Secondary stakeholder-centered outcomes included patient (knowledge, emotional reaction, satisfaction, volume of patient inquiries) and provider outcomes (perception, knowledge, acceptance, and satisfaction) measured via repeated surveys or electronic health records. The generalized estimating equation framework and the difference-in-differences approach were used to compare outcomes across study arms.

Results

The proportion of appropriate screenings (i.e., use of primary HPV screening) during the post-intervention period was high, with no observed difference between study arms: 98.4% (95% confidence interval [CI] 96.3%-100%) for the locally-tailored arm and 99.1% (95% CI: 97.8%-100%) for the centrally-administered arm (p = 0.34). There were no statistically or clinically significant differences in patient- and provider- outcomes between study arms.

Conclusions

Primary HPV screening was feasible and demonstrated high fidelity in all KPSC service areas. The locally-tailored practice substitution approach and centrally-administered practice substitution approach both achieved near complete uptake of primary HPV screening. Further, similar effects on stakeholder-centered outcomes were observed for both approaches. However, generalizability of our findings may be limited due to unique features of our integrated health system.

Trial registration

NCT04371887. Registered 30 April 2020, URL: https://clinicaltrials.gov/study/NCT04371887?cond=primary%20HPV%20screening&rank=5.

Peer Review reports

Background

Widespreadadoption of cervical cancer screening practices in the United States (US) and other developed countries has been an important public health triumph. While screening with the Papanicolaou (Pap) test (a cytology test) is effective in reducing the burden of cervical cancer, recent evidence documents greater effectiveness of screening strategies that focus on persistent infection with high-risk human papillomavirus (hr-HPV) [1, 2], which causes virtually all cervical cancers [3]. h-HPV testing is more sensitive for detection of severe precancerous lesions than cytology [4,5,6,7,8,9,10,11,12], and is as effective as the cytology and HPV co-testing strategy [13,14,15]. Based on this evidence, effective cervical cancer screening programs can primarily rely on hr-HPV testing, with reflex cytology testing only for certain HPV test results. This strategy, labeled Primary HPV screening, also requires fewer tests and costs less per screened woman than co-testing [16]. In 2018, the U.S. Preventive Services Task Force updated its recommendation for routine cervical cancer screening to include primary HPV screening alongside previously-recommended approaches for routine cervical cancer screening for women age 30–65 years. Professional societies such as the American Society for Colposcopy and Cervical Pathology, the Society of Gynecology Oncology, the American College of Obstetrician and Gynecology, and American Cancer Society have released similar recommendations [17,18,19].

Although interest in substituting primary HPV testing for stand-alone Pap testing and co-testing is growing, recent publications suggest that the transition to primary HPV screening might encounter barriers at multiple levels. These include incomplete physician knowledge and confusion regarding the new guideline recommendations [20, 21], physician concern about missing cancer [22], patient attachment to the widely-known Pap test, low acceptance of primary HPV testing, and infrastructure requirements [20, 23,24,25]. To date, insufficient guidance exists regarding how to design and deploy practice substitution strategy bundles to successfully de-implement older practices and replace them with newer practices [26, 27]. This is particularly relevant in cervical cancer screening, where an ever-evolving evidence base, development of new technologies and resulting changes in clinical practice guidelines have necessitated frequent changes in clinical practices [28].

Efforts to substitute newer evidence-based practices in patient care settings have historically relied on a centralized, “one-size fits all” approach involving one or more discrete implementation strategies deployed in a bundle across all participating sites. The one-size fits all approach can be effective in achieving desired changes [29] but often fails to produce reliable or consistent change across sites [30]. Such variability is thought to be caused by heterogeneity in the underlying implementation barriers and in local contextual features such as settings and staff [31]. Local tailoring of implementation strategies and bundles has been recommended to respond to variation in local barriers and context [32, 33]. Evidence from head-to-head comparisons of locally-tailored approaches vs. conventional “one-size fits all” approaches is needed to confirm the value of local tailoring for various types of implementation and de-implementation, including practice substitution [34].

In response to the new recommendations for cervical cancer screening, Kaiser Permanente Southern California (KPSC) decided to implement primary HPV screening to replace its existing co-testing method for routine cervical cancer screening for women aged 30–65. We leveraged and supported this health system initiative by conducting a head-to-head cluster randomized pragmatic trial comparing two practice substitution approaches: a “centrally-administered + locally-tailored” (hereafter referred as “locally-tailored”) versus a “centrally-administered + usual care” (hereafter referred as “centrally-administered”) approach to facilitate adoption of primary HPV screening for routine cervical cancer screening and optimize stakeholder-centered outcomes. As this research was embedded within a large health care delivery system, our results could inform the decision making of clinical and health system leaders interested in selecting, tailoring and deploying optimal implementation strategy bundles to achieve successful de-implementation and substitution.

Methods

Study setting

This study was conducted in KPSC, a large integrated health care delivery system providing care to over 4.8 million members who are broadly representative of the racially, ethnically and socioeconomically diverse residents of Southern California [35]. KPSC members are insured through a variety of commercial, private paid, Medicare, and Medicaid plans, and other low income programs, and represent a wide range of income levels. KPSC delivers care through 15 hospitals and approximately 209 medical offices grouped into 13 medical service areas. The populations served by these medical service areas differed in size and distribution of demographic characteristics and urban/suburban/rural residence. For example, the proportion of Hispanic members in a medical service area ranged from approximately one-third to two-thirds; and the average census block proportion with adults with a college degree ranged from approximately one-fifth to two-fifths. A multidisciplinary HPV Task Force composed of clinical (primary care and ob/gyn physician and nurse leads), administrative, and laboratory leaders as well as health system implementation experts oversaw the health system-led primary HPV screening practice change effort. The HPV Task Force designed and delivered the centrally-administered implementation strategy bundles; the Task Force collaborated with members of the research team to design and carry out the locally-tailored activities in the second study arm (detailed below).

Study design and study participants

This was a prospective, cluster randomized programmatic trial conducted through a research-practice partnership to compare the effects of a locally-tailored versus centrally-administered practice substitution approach to facilitate adoption of primary HPV screening. We measured several system-level implementation outcomes and additional stakeholder-centered outcomes. The study randomization unit was the medical service area. All service areas received the centrally-administered implementation activities comprised of clinician-targeted educational webinars and socialization of the practice change through existing communication networks of clinicians and administrators in preparation for the roll out of primary HPV screening, as well as provision of patient-facing materials to facilitate patient education and explanation. After the region-wide launch of primary HPV screening, sites randomized to the locally-tailored arm participated in a structured process to select, tailor and deliver implementation strategy bundles responding to local needs and circumstances. Sites randomized to the centrally-administered arm did not receive any additional implementation strategies and followed established practice in leveraging the centrally-administered activities to achieve practice change (see Fig. 1 for study activities and study timeline). This manuscript follows the CONSORT guideline for reporting randomized trials.

Fig. 1
figure 1

Timeline of intervention and data collection by implementation strategy arms

Full size image

One service area served as the pilot site for the locally-tailored approach and the remaining 12 service areas were randomized to the two practice substitution approaches. We used matched-pair cluster randomization: the 12 service areas were first matched in pairs on the following factors using a SAS algorithm (SAS Institute Inc) and then randomized: [36] number of clinics within a medical center (a proxy for the difficulty of disseminating information and achieving buy-in and support throughout the entire service area); current cervical cancer screening rate (a proxy for the level of attention and culture related to cervical cancer screening); and average waiting time (in days) for an obstetrics/gynecology appointment (a proxy for provider capacity).

The identification of relevant implementation approaches and the design of the evaluation were guided by the Consolidated Framework for Implementation Research (CFIR) [37] and the Theoretical Domains Framework (TDF) [38,39,40]. We used CFIR to identify relevant organizational and additional contextual factors to guide design of the practice substitution bundles as well as evaluation design and methods, including variable selection, collection of qualitative and quantitative data, and analyses. We used TDF to guide selection of practice substitution strategies targeting clinician knowledge, attitudes, beliefs and practice change, and methods for qualitative data collection and analysis. The KPSC IRB reviewed and approved the study.

All primary care and obstetrics/gynecology departments from KPSC’s 13 service areas participated in the study. Implementation activities were delivered at the service area level, involving all physicians, licensed vocational nurses (LVN), medical assistants, administrators, and all women aged 30–65 years who received cervical cancer screening during the study period.

Practice substitution strategies and bundles

The substitution of primary HPV screening for Pap testing and co-testing in a health system required system-level, non-tailorable infrastructure changes (e.g., to laboratory and electronic health record [EHR] and information technology policies and design), as well as tailorable implementation strategies such as physician and staff training/support and patient education. The implementation strategies and their inclusion in both comparators are described below and additional details are available in the study protocol [41].

Comparator 1

The locally-tailored implementation approach involved a structured process encompassing (1) site-specific assessment of local needs followed by convening of a local site team to (2) review the needs assessment data and (3) select from a pre-developed menu of implementation strategies (forms) grouped by core function and matched to specific needs, followed by (4) deployment of the resulting tailored bundle of implementation strategies at each site [31, 42]. Two levels of local tailoring were used in this study: (1) selection of core functions based on the highest-ranked local needs/barriers identified; and (2) selection of one or more forms (e.g., email communication or hardcopy handout) to carry out each core function to fit the site context.

Comparator 2

Sites randomized to the centrally-administered arm did not receive support from the research team nor use a systematic needs assessment process to guide local tailoring, but some sites may have augmented the centrally-administered implementation activities with minimal local activities.

Study outcomes and data collection

The primary outcome of interest was the proportion of screening visits by women aged 30–65 in which primary HPV screening was conducted. This outcome was measured at the service area level in a three-month window (Oct 2021-Jan 2022) using data from the KPSC EHR. A greater than 5% difference was pre-specified as clinically important by clinical experts on the project. Of note, following the initial roll out of primary HPV screening and after the trial design had been finalized, HPV Task Force leaders decided to facilitate adherence to the new clinical policy through redesign of the health system’s EHR-based cervical cancer screening ordering screens: the health system removed the option to order co-testing for routine screening and listed HPV screening as the only option for routine screening, retaining co-testing as an available option only for patients with a prior abnormal history.

In addition to uptake of a new clinical practice, stakeholder buy-in and satisfaction are critical features of successful implementation: clinician or patient reluctance or dissatisfaction with a planned change can contribute to low rates of initial and/or sustained practice change, and to other adverse outcomes even if practice change is mandated. Acceptability by key stakeholders is thus a critical implementation outcome [43]. To this end, we measured patient- and provider-centered outcomes identified or defined by members of our stakeholder advisory committee as secondary outcomes (Fig. 2). Patient-centered outcomes included knowledge about HPV and HPV testing, acceptance of primary HPV screening, emotional reactions (stigma and embarrassment) to a positive HPV test result, satisfaction with the screening experience, all measured through patient survey (see Appendix Tables 2 and 3 for survey questions). Patient email inquiries about cervical cancer screening, a proxy for patient confusion or unaddressed questions, were also measured from the EHR. Provider-centered outcomes included beliefs about the efficacy of the new test, resistance or acceptance of primary HPV screening (physician only), familiarity with screening guidelines (physician only), knowledge of the follow-up guidelines (physician only), as well as acceptability, appropriateness, and feasibility of primary HPV screening, and satisfaction with the practice substitution process, all measured by provider survey (see Appendix Table 5a/5b for survey questions).

Fig. 2
figure 2

Summary of study outcomes and methods of data collection

Full size image

Tertiary outcomes included extent of provider delivery of patient education about cervical cancer screening, and within the locally-tailored arm only, fidelity, sustainment and scalability of the locally-tailored approach. Tertiary outcomes were measured using provider survey and are discussed in the publicly accessible final report from the Patient-Centered Outcomes Research Institute (PCORI) website.

Patient and provider surveys were administered at three time points: 1) pre-implementation (before centralized educational activities and implementation of primary HPV screening), (2) interim (post-rollout, three months after the system-wide roll out of primary HPV screening); and (3) post-intervention (two months after the research-led implementation strategy bundles were delivered). Women who tested HPV-negative were included in the survey. Survey questions were adopted from validated instruments (e.g., patient HPV knowledge score and stigma and shame about HPV [44,45,46]; provider implementation outcome measures [47]) or published measures when possible [48], while others were developed based on stakeholder input. Provider surveys were administered on-line, whereas on-line and mailed survey methods were employed for patient surveys.

For patient email inquiries about cervical cancer screening, measured via EHR data, the baseline data collection window was 10/19/2019–01/18/2020 (before centralized educational activities started in May 2020 and roll-out of primary HPV screening in July 2020) and the post-intervention data collection window was 10/19/2021–01/18/2022. An electronic algorithm was developed to capture email messages from patients inquiring about the cervical cancer screening test or test results. Chart review was performed to help develop this algorithm and validate its accuracy.

Qualitative interviews were also conducted as part of this study in two phases: baseline interviews and post-implementation process evaluation interviews (Fig. 1). Interviews were conducted with physicians, nurses, and administrators from the primary care and ob/gyn departments, and for baseline interviews, also with women between ages 30 to 65. The interview data collection methods and findings have been reported elsewhere [22].

Covariates

KPSC’s EHR and administrative databases were used to measure patient-, provider- and system-level characteristics including patient age, race/ethnicity, neighborhood income and education level, preferred language, primary home service area, and length of membership; provider age, sex, race/ethnicity, specialty, and years of practice at KPSC, and the number of clinics in each service area.

Analytical approaches

Analysis for the primary outcome compared the average between-arm difference over the three-month observation period. We computed the proportion of primary HPV screening among all screening visits at the service area level. The proportion was estimated as (# of primary HPV screenings in the observation period)/(total # of screening tests performed in the observation period, i.e., [# of primary HPV screenings in the observation period + # of co-testing performed in the observation period for screening purpose]). The number and proportion of co-tests performed for screening purpose was estimated based on site-specific chart review (50 charts randomly selected for manual review from all co-testing orders occurring in each service area—300 total charts reviewed) with sampling weights applied to estimate the weighted proportions of co-tests performed for screening and subsequently the number of co-tests for screening [49]. We also used the sampling weights to calculate the variance of the total number of co-tests performed for screening and obtained 95% confidence limits. Statistical testing comparing the two arms was performed using Z statistics accounting for the design effect.

For all secondary outcomes, we first calculated the distributions of the demographic characteristics of study subjects included for each secondary outcome, overall and by study arm. For each outcome, we estimated changes in each outcome from baseline to post-intervention within each arm, and the difference-in-differences (DID, the main parameter of interest) between the two arms. We used the Generalized Estimating Equations (GEE) framework for marginal models, with different link functions to address the different types of outcomes allowing for clustering. An additional random effect was included to account for the matched design. We adjusted for patient age, race/ethnicity, and years of prior KPSC membership for patient-centered outcomes, and provider age, sex, race/ethnicity, and years of practice at KPSC for the provider-centered outcomes. We accounted for the survey sampling design and survey response by calculating design and response weights as the inverse of the selection or response probabilities.

For patient email inquiries, in order to account for the effect of individual physicians and to facilitate the DID approach, we included only physicians who performed cervical cancer screening in both the baseline and the post-intervention periods (85% of all physicians) and performed a minimum of five cervical cancer screenings in the study windows (74% of all physicians who performed screening in both baseline and post-intervention). We used the same analytical principles as described above, with log-Poisson models with robust standard error. We adjusted for provider department, gender, race/ethnicity, and years of experience in KPSC, and provider-level averages of patient age, race/ethnicity, years of KP membership, and neighborhood deprivation index (NDI) quartile. We encountered very few instances of missing covariate data (< 5% for all covariates). All analyses were conducted in SAS statistical software Version 9.4; Cary, North Carolina, USA.

Results

Primary outcome

Appendix Table 1 shows the number of providers and women screened by primary HPV screening in the study intervention and data collection window. All service areas were found to have a very high proportion of appropriate screenings at post-intervention [the proportion of primary HPV screening (as opposed to co-testing) among all screening visits], ranging from 97.6% to 99.6% (Table 1). The overall proportion was estimated to be 98.4% (95% confidence interval [CI] 96.3%− 100%) for the locally-tailored arm and 99.1% (97.8%− 100%) for the centrally-administered arm (p-value = 0.34). The absolute difference in the proportions between arms did not reach the pre-specified clinically meaningful level (i.e., 5%).

Table 1 Estimated proportion of primary HPV screening among all screening visits by service area and study arms in the post-implementation data collection period
Full size table

Secondary patient-centered outcomes

For the patient surveys, a total of 1,266 (36% response rate, mean age 45) and 949 (31% response rate, mean age 48) survey responses were received from the baseline and the post-intervention survey invitations, respectively. Weighted patient characteristics of the survey respondents are shown in Table 2.

Table 2 Characteristics of patient survey respondents included in the analysis (HPV-negative women only)
Full size table

No significant DID was found between study arms for any patient-centered outcome evaluated (Fig. 3 and Appendix Table 2). Further, no significant change between baseline and post-intervention time periods was noted for patient-centered outcomes measured at both time periods within each arm. For the HPV knowledge score that ranges from 0–6 (6 being the highest knowledge), the overall mean score was 2.7 (standard deviation [SD] 1.5) at baseline and 2.5 (SD 1.5) at post-intervention. For women’s acceptance of primary HPV screening, overall, 6.1% of the women viewed primary HPV screening as an acceptable screening modality at baseline, compared with 5.1% in the post-intervention period. For the stigma score ranging from 1–5 (5 being highest stigma), the overall mean score was 1.5 (0.8) and 1.9 (1.1) for taking the HPV test and testing HPV positive at baseline, respectively, compared with 1.5 (0.9) and 1.8 (1.2) post-intervention, respectively. For embarrassment about taking the HPV test, 21.3% of respondents overall reported embarrassment at baseline, compared with 19.9% in the post-intervention period. For embarrassment about testing HPV positive, 49.9% reported embarrassment at baseline compared with 46.9% in the post-intervention period. For patient satisfaction of the screening visit (score 1–5, 5 being the highest satisfaction), the mean score was 3.9 (0.8) at baseline and 3.9 (0.9) post-intervention.

Fig. 3
figure 3

Baseline and post-intervention patient-centered outcomes by randomization arm

Full size image

For the patient email inquiries, a total of 1,294 physicians were included in this analysis (694 in the centrally-administered arm and 600 in the locally-tailored arm, 70% family medicine/internal medicine and 30% obstetrics/gynecology). These physicians screened a total of 24,150 women in the baseline period and 25,464 in the post-intervention period. In both arms, there was a significant decrease between the baseline and post-intervention periods in the number of patient email inquiries regarding screening (Appendix Fig. 1). The patient message volume (i.e., # messages per physician per 10 screened women) at post-intervention was 76% (p < 0.01) of that at baseline for the locally-tailored arm, and 71% (p < 0.01) for the centrally-administered arm. The DID (measured as ratio of the ratios), however, was not statistically significant (DID = 1.06-fold, p = 0.59).

For all patient-centered outcomes, findings are similar between study arms within each time point. Findings are also similar from both the crude and adjusted analyses.

Secondary provider-centered outcomes

For the provider surveys, a total of 588 (23% response rate) and 1,032 (20% response rate) survey responses were received from the baseline and the post-intervention data collection, respectively. The mean age of survey respondents was 44 (baseline) and 45 (post-intervention) for physicians and 38 (baseline) and 41 (post-intervention) for LVNs/MAs. About half of the physician respondents and the majority of the LVN/MA respondents were female (~ 90%). About 80% of the respondents were based in family medicine/internal medicine (Appendix Table 4a/4b).

No significant DID was found between study arms for any provider-centered outcome evaluated. Survey responses were also similar between study arms at each wave of survey. For perception about the effectiveness of primary HPV screening, 66.6% of the providers considered primary HPV screening very effective for screening women aged 30–65 at baseline, compared with 74.2% in the post-intervention period. For the physicians, there was a significant increase in this proportion between baseline and post-intervention, yet for nurses, a significant decrease was observed for the locally-tailored arm but not for the centrally-administered arm (Fig. 4 and Appendix Table 5a/5b). For acceptance of primary HPV screening (physician only), 6.6% of the physicians selected primary HPV screening as an appropriate screening method for women aged 30–65 years at baseline, compared with 48.5% in the post-intervention period. For familiarity with the screening and follow-up guidelines (physicians only), overall, 27% and 11% of the physicians reported being very familiar with the screening and follow-up guidelines at baseline, respectively, compared with 19% and 13% in the post-intervention period (Fig. 4 and Appendix Table 5a/5b). For knowledge with the follow-up algorithm (physicians only), overall, 8.5% of the physicians answered all three questions correctly, compared with 10.2% in the post-intervention period.

Fig. 4
figure 4

a. Baseline and post-intervention provider-centered outcomes by randomization arm – physicians. aData and figures for acceptance, appropriateness, and feasibility are similar. Only figure for appropriateness is shown here to avoid redundancy. b. Baseline and post-intervention provider-centered outcomes by randomization arm – LVN/MA

Full size image

Overall, 89% of the providers reported being satisfied with the transition process. The proportion was similar between arms. In the adjusted model, the practice substitution approach (locally-tailored vs. centrally-administered) was not significantly associated with provider satisfaction (OR = 0.91 for “satisfied” for the locally-tailored arm compared with the centrally-administered arm, p = 0.75 for this different). Responses to individual survey items regarding provider experience about the transition process are shown in Appendix Table 6.

For all provider-centered outcomes, findings are similar between study arms within each time point. Findings are also similar from both the crude and adjusted analyses.

Discussion

In this study we compared effects of locally-tailored vs. centrally-administered practice substitution approaches on implementation and stakeholder-centered outcomes. The centrally-administered approach is considered the reference approach since it is the standard approach used by KPSC and we were interested in evaluating the potential benefit of adding local tailoring to facilitate cervical cancer screening practice substitution. We found that the rate of appropriate screening (i.e., the proportion of primary HPV screening among all screenings) for women aged 30–65 years was high across all service areas 15 months after the transition, with no benefit observed for the locally-tailored approach on the rate of appropriate screening or on the secondary patient- or provider-centered outcomes.

A powerful system-level implementation approach was deployed at KPSC soon after the initial roll-out of primary HPV screening: system leaders redesigned the test ordering screens in the EHR to eliminate the option for providers to select non-recommended approaches for screening. This approach was designed to facilitate adherence and reduce confusion triggered by the newly released 2019 ASCCP management guidelines specifying the recommended testing approach for patients with an abnormal screening history. The two test options offered in the redesigned EHR order screen were “test for routine screening (HPV with cytology reflex)” and “test for abnormal follow-up (HPV and cytology).” We believe that this EHR redesign was the primary driver of the very high uptake of primary HPV screening and the lack of difference between study arms in the primary outcome of interest: the very high uptake (> 98%) in the reference arm precluded the detection of statistically significant improvement in the locally-tailored arm (ceiling effect, since the uptake cannot exceed 100%), although any magnitude of improvement from 98% would not be considered clinically important.

The “constrained choice implementation” approach, accomplished through a simple change in the EHR test ordering screens, produced very high rates of practice change in our study. This approach warrants further research, both to identify and evaluate additional forms of constrained choice implementation and to carefully study harms and other unintended consequences. As an implementation strategy, constrained choice implementation is feasible for various health care settings. Additional forms of constrained choice implementation may include policy-level changes (e.g., prohibiting outdated practices [50] or mandating desired practices), elimination of reimbursement for outdated practices or conditioning reimbursement for specific practices on delivery of desired supplementary practices, use of formularies restricting prescribing choices, and withdrawal of resources required to deliver an outdated service. However, adverse effects on clinician, staff, and/or patient satisfaction may result from mandating a practice change (e.g., by removing previously-used practice options) when key stakeholders do not support the change. Measurement of unintended consequences and harms is recommended for all types of clinical research, and therefore harms should be measured as secondary outcomes in all implementation studies [51, 52]. In studies of constrained choice implementation where high adoption of the practice change is expected, harms should be considered as the primary outcome.

We also observed null findings on all stakeholder-centered outcomes; as such, it is important to understand if the null findings might be due to low fidelity to the locally-tailored implementation approach. We measured the fidelity via the provider and patient survey using a set of questions for awareness, use, and perceived value for each approach/function. Survey results suggested a reasonable uptake (21–60%) of the locally-tailored implementation strategies and a high perceived value of these strategies for clinical practice. However, respondents in our provider interview reported lower rate of recall of use and lower perceived value of these implementation strategies.

It should be noted that the COVID- 19 pandemic had a significant impact on multiple aspects of the study, particularly for the locally-tailored arm. The pandemic reduced the availability of providers to attend educational meetings and local-tailoring meetings. The pandemic also led to temporary stoppage of all in-person department or staff meetings, which likely resulted in less penetration of the locally-tailored implementation activities, or making them less effective in influencing stakeholder-centered outcomes.

Of note, we had a considerable time lag (~ 4 months) between local team meetings and delivery of implementation strategies. This is mainly due to the additional tailoring requests that required organizational approval, such as the introduction of smart phrases in the EHR system. It is possible that providers might have moved their attention away from this practice change and/or gradually accepted the new practice during this time even without additional implementation strategies (which would still argue against the value of local-tailoring since its superiority over non-tailored approaches, even if observed, could be temporary). Based on these experiences, planning activities to conduct local-tailoring should account for the variable availability and priorities of local teams and administrators and consider the uncertainties when allowing a larger scope of local tailoring. This could be proactively managed to a certain extend by limiting the scope of implementation strategy tailoring, and/or by engaging a larger number and types of clinical stakeholders upfront.

In addition, it is important to note that primary HPV screening has strong clinical evidence and guideline support, does not change the patient experience of the screening process, and was compatible with existing workflows at KPSC [53, 54]. It did not require staff or physicians to take on new roles and in fact had relative advantage in simplifying the workflow. When barriers to the desired practice substitution were assessed in the locally-tailored arm, most of the needs identified centered around provider education, and there was a large degree of overlap in provider needs and patient-level barriers reported across these sites, as well as the selected intervention core functions. These findings suggested that variability in local needs and context might not be sufficient to warrant local-tailoring, which may also in part explain the lack of difference among the stakeholder-centered outcomes. In addition, the value of tailoring by selecting distinct forms also appeared low for this implementation problem.

In a Cochrane systematic review of the effect of implementation strategies tailored to practice determinants, Baker and colleagues identified 32 cluster-randomized trials comparing tailored interventions to no interventions or interventions not tailored [31]. They found an overall odds ratio of 1.56 for tailored interventions compared with non-tailored intervention or no intervention for the outcome of uptake of recommended practice. However, studies showed variable effectiveness. Surprisingly, the pooled odds ratio was higher for the study that had a reference group of non-tailored interventions than those that used no intervention as the reference (odds ratio = 1.79 vs. 1.48). Overall, the number of studies evaluating tailored interventions remains low, and the certainty of evidence remains moderate, with only 20 studies included in this review incorporating some adjustment to local factors (local-tailoring).

Several recently-published articles contribute to the growing body of knowledge and scientific discussion regarding intervention and implementation strategy tailoring [55,56,57,58,59,60,61,62,63]. Several researchers encourage greater use of theory in tailoring, although balance should be achieved between information obtained from theory, stakeholder input and contextual analysis [55, 63, 64]. To date, methods for conducting effective tailoring, including methods to identify important influences on professional practices and effective approaches for selecting implementation strategies to address these influences, are still in development and require further study [64]. To our knowledge, the literature is also sparse on comparing implementation approaches for practice substitution in general. Considering the limited knowledge available, our study adds to the overall evidence-base for implementation strategy tailoring and local-tailoring for cancer screening as well as practice substitution and practice change in general. Future research should evaluate approaches to optimize the means and process of barrier/need assessment, local-engagement, and implementation strategy selection and tailoring and should continue to evaluate the value of local tailoring in different implementation scenarios as well as practice substitution and de-implementation scenarios.

There are several limitations to consider when interpreting our results. As mentioned earlier, unexpected delay in delivering the tailored implementation strategies was a limitation of the study. Moreover, as with any survey, self-report and participation bias are always possible, especially with response rates ranging between 20–36%. Potential participation bias, such that those who were more supportive or unsupportive of the practice substitution may be more likely to respond, calls for caution when interpreting the provider survey findings on their own. That said, we expect such potential bias to be non-differential between study arms. Our patient survey did not include women who screened HPV-positive and thus we could not assess implementation bundle impact on this minority group of women. In addition, the COVID- 19 pandemic forced replacement of some in-person data collection via site visits with survey and phone interviews. Further, the pandemic might have shifted the priority and attention of both the providers and patients. As a result, it is possible that some of the patient and provider findings may not be applicable to the post-pandemic period.

As discussed earlier, our study findings regarding the effects of constrained choice implementation are likely generalizable to various types of health systems for similar practice substitution problems. Computerized ordering systems are widely available in developed countries and especially those with high levels of EHR implementation. Other approaches to constrained choice implementation may be available for practice settings lacking computerized ordering, including modifications to reimbursement policies. Our finding of no difference between locally-tailored and centrally-administered implementation may be generalizable only to health systems in which routine, centrally-administered implementation approaches are well-designed and strong; and to systems with relatively homogeneous care delivery sites. KPSC has robust infrastructure for implementation of new innovations which has contributed to higher rates of quality improvement compared to the national average [65]. Further, sites within KPSC may be more homogeneous in their needs and thus less responsive to local tailoring. Implementation problems and settings for which barriers vary significantly and/or sites are more heterogeneous (e.g., community health centers or private practices) may show greater response to local tailoring. That said, policy and program incentives at the national level and advancement in technology have led to significant health system consolidation and growth of large integrated delivery systems in the U.S.. For this reason, our practice substitution experience and recommendations regarding desired implementation bundle features are likely applicable to a growing proportion of the U.S. healthcare system.

Conclusion

We did not find evidence to support our hypothesis regarding barrier heterogeneity and the benefit of local-tailoring for the substitution of primary HPV screening for the previous practice of co-testing. Our findings suggest no additional benefit in the rate of uptake of primary HPV screening or in stakeholder-centered outcomes associated with local implementation bundle tailoring, in the context of what proved to be a highly effective centrally-administered approach to implementation incorporating redesign of the EHR test ordering screen. Given the addional costs associated with performing barrier/need assessment and implementation strategy tailoring, the centrally-administered practice substitution approach appears to be the preferred method for this type of practice change in settings similar to KPSC. Further research is needed to determine whether this conclusion is generalizable to other implementation problems where barriers vary significantly across sites and may benefit from more complex local tailoring of the practice change bundle.

Data availability

Anonymized data that support the findings of this study may be made available from the corresponding author on reasonable request from qualified researchers with documented evidence of training for human subject protections.

References

  1. Chatzistamatiou K, Moysiadis T, Moschaki V, Panteleris N, Agorastos T. Comparison of cytology, HPV DNA testing and HPV 16/18 genotyping alone or combined targeting to the more balanced methodology for cervical cancer screening. Gynecol Oncol. 2016;142(1):120–7. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ygyno.2016.04.027.

    Article  PubMed  Google Scholar 

  2. Ronco G, Dillner J, Elfström KM, et al. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. 2014;383(9916):524–32. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/s0140-6736(13)62218-7.

    Article  PubMed  Google Scholar 

  3. Castellsague X. Natural history and epidemiology of HPV infection and cervical cancer. Gynecol Oncol. 2008;110(3 Suppl 2):S4-7. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ygyno.2008.07.045.

    Article  PubMed  Google Scholar 

  4. Cuzick J, Clavel C, Petry KU, et al. Overview of the European and North American studies on HPV testing in primary cervical cancer screening. Int J Cancer. 2006;119(5):1095–101. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/ijc.21955.

    Article  CAS  PubMed  Google Scholar 

  5. Mayrand MH, Duarte-Franco E, Rodrigues I, et al. Human papillomavirus DNA versus Papanicolaou screening tests for cervical cancer. N Engl J Med. 2007;357(16):1579–88. https://doiorg.publicaciones.saludcastillayleon.es/10.1056/NEJMoa071430.

    Article  CAS  PubMed  Google Scholar 

  6. Naucler P, Ryd W, Tornberg S, et al. Human papillomavirus and Papanicolaou tests to screen for cervical cancer. N Engl J Med. 2007;357(16):1589–97. https://doiorg.publicaciones.saludcastillayleon.es/10.1056/NEJMoa073204.

    Article  CAS  PubMed  Google Scholar 

  7. Castle PE, Stoler MH, Wright TC Jr, Sharma A, Wright TL, Behrens CM. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: a subanalysis of the ATHENA study. Lancet Oncol. Sep2011;12(9):880–90. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/S1470-2045(11)70188-7.

    Article  PubMed  Google Scholar 

  8. Rijkaart DC, Berkhof J, Rozendaal L, et al. Human papillomavirus testing for the detection of high-grade cervical intraepithelial neoplasia and cancer: final results of the POBASCAM randomised controlled trial. Lancet Oncol 13(1):78–88. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/S1470-2045(11)70296-0

  9. Ronco G, Giorgi-Rossi P, Carozzi F, et al. Efficacy of human papillomavirus testing for the detection of invasive cervical cancers and cervical intraepithelial neoplasia: a randomised controlled trial. Lancet Oncol. 2010;11(3):249–57. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/S1470-2045(09)70360-2.

    Article  CAS  PubMed  Google Scholar 

  10. Stoler MH, Schiffman M. Atypical squamous cells of undetermined significance-low-grade squamous intraepithelial lesion triage study G. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285(11):1500–5.

    Article  CAS  PubMed  Google Scholar 

  11. Carozzi FM, Del Mistro A, Confortini M, et al. Reproducibility of HPV DNA testing by hybrid capture 2 in a screening setting. Am J Clin Pathol. 2005;124(5):716–21. https://doiorg.publicaciones.saludcastillayleon.es/10.1309/84E5-WHJQ-HK83-BGQD.

    Article  CAS  PubMed  Google Scholar 

  12. Castle PE, Wheeler CM, Solomon D, Schiffman M, Peyton CL, Group A. Interlaboratory reliability of Hybrid Capture 2. Am J Clin Pathol. 2004;122(2):238–45. https://doiorg.publicaciones.saludcastillayleon.es/10.1309/BA43-HMCA-J26V-WQH3.

    Article  Google Scholar 

  13. Moyer VA, Force USPST. Screening for cervical cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;156(12):880–91, W312. https://doiorg.publicaciones.saludcastillayleon.es/10.7326/0003-4819-156-12-201206190-00424

  14. Massad LS, Einstein MH, Huh WK, et al. 2012 updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors. Obstet Gynecol. 2013;121(4):829–46. https://doiorg.publicaciones.saludcastillayleon.es/10.1097/AOG.0b013e3182883a34.

    Article  PubMed  Google Scholar 

  15. Saslow D, Solomon D, Lawson HW, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin. 2012;62(3):147–72. https://doiorg.publicaciones.saludcastillayleon.es/10.3322/caac.21139.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Wright TC, Stoler MH, Behrens CM, Sharma A, Zhang G, Wright TL. Primary cervical cancer screening with human papillomavirus: end of study results from the ATHENA study using HPV as the first-line screening test. Gynecol Oncol. 2015;136(2):189–97. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ygyno.2014.11.076.

    Article  PubMed  Google Scholar 

  17. Force USPST. Draft Recommendation Statement. Cervical Cancer: Screening. Accessed May 14, 2018, 2018. https://www.uspreventiveservicestaskforce.org/Page/Document/draft-recommendation-statement/cervical-cancer-screening2

  18. Practice Bulletin No. 157: Cervical Cancer Screening and Prevention. Obstet Gynecol. 2016;127(1):e1–20. https://doiorg.publicaciones.saludcastillayleon.es/10.1097/AOG.0000000000001263. PMID: 26695583.

  19. Huh WK, Ault KA, Chelmow D, et al. Use of primary high-risk human papillomavirus testing for cervical cancer screening: interim clinical guidance. Gynecol Oncol. 2015;136(2):178–82. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ygyno.2014.12.022.

    Article  PubMed  Google Scholar 

  20. Nayar R, Goulart RA, Davey DD. Primary HPV cervical cancer screening in the United States: Are we ready? J Am Soc Cytopathol. 2018;7(1):50–5.

    Article  PubMed  Google Scholar 

  21. Cooper CP, Saraiya M. Primary HPV testing recommendations of US providers, 2015. Prev Med. 2017;105:372–7. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ypmed.2017.08.006.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Hahn EE, Munoz-Plaza C, Altman DE, et al. De-implementation and substitution of clinical care processes: stakeholder perspectives on the transition to primary human papillomavirus (HPV) testing for cervical cancer screening. Implement Sci Commun. 2021;2(1):108. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-021-00211-z.

    Article  PubMed  PubMed Central  Google Scholar 

  23. McRae J, Martin C, O'Leary J, Sharp L, Irish Cervical Screening Research C. "If you can't treat HPV, why test for it?" Women's attitudes to the changing face of cervical cancer prevention: a focus group study. BMC Womens Health. 2014;14:64. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/1472-6874-14-64

  24. Greene J. The Pap Test: 20th Century Success Story, 21st Century Has-Been Managed Care. Accessed 02/07/2018, 2018. http://www.managedcaremag.com/archives/2018/2/pap-test-20th-century-success-story-21st-century-has-been

  25. Saraiya M, Kwan A, Cooper CP. Primary HPV testing: U.S. women's awareness and acceptance of an emerging screening modality. Prev Med. 2018;108:111–114. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.ypmed.2017.12.007

  26. Scott SD, Albrecht L, O’Leary K, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:70. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/1748-5908-7-70.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Gagliardi AR, Alhabib S, members of Guidelines International Network Implementation Working G. Trends in guideline implementation: a scoping systematic review. Implement Sci. 2015;10:54. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-015-0247-8

  28. Squiers LB, Holden DJ, Dolina SE, Kim AE, Bann CM, Renaud JM. The public's response to the U.S. Preventive Services Task Force's 2009 recommendations on mammography screening. Am J Prev Med. 2011;40(5):497–504. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.amepre.2010.12.027

  29. Lokker C, McKibbon KA, Colquhoun H, Hempel S. A scoping review of classification schemes of interventions to promote and integrate evidence into practice in healthcare. Implement Sci. 2015;10:27. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-015-0220-6.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5(9): e008592. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/bmjopen-2015-008592.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;(3):CD005470. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/14651858.CD005470.pub2

  32. Krause J, Van Lieshout J, Klomp R, et al. Identifying determinants of care for tailoring implementation in chronic diseases: an evaluation of different methods. Implement Sci. 2014;9:102. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-014-0102-3.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Carpenter D, Nieva V, Albaghal T, Sorra J. Development of a planning tool to guide research dissemination. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation (Volume 4: Programs, Tools, and Products). 2005. Advances in Patient Safety.

  34. Brown CH, Curran G, Palinkas LA, et al. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health. 2017;38:1–22. https://doiorg.publicaciones.saludcastillayleon.es/10.1146/annurev-publhealth-031816-044215.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J Summer. 2012;16(3):37–41.

    Google Scholar 

  36. Bergstralh EKJ. Computerized matching of controls. 1995. Section of Biostatistics Technical Report 56.

  37. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/1748-5908-4-50.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/1748-5908-7-37.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008;57(4):660–80.

    Article  Google Scholar 

  40. Duncan EM, Francis JJ, Johnston M, et al. Learning curves, taking instructions, and patient safety: using a theoretical domains framework in an interview study to investigate prescribing errors among trainee doctors. Implement Sci. 2012;7:86. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/1748-5908-7-86.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Chao CR, Cannizzaro NT, Hahn EE, et al. A study protocol for a cluster randomized pragmatic trial for comparing strategies for implementing primary HPV testing for routine cervical cancer screening in a large health care system. Contemp Clin Trials. 2023;124: 106994. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.cct.2022.106994.

    Article  PubMed  Google Scholar 

  42. Balasubramanian BA, Cohen DJ, Davis MM, et al. Learning Evaluation: blending quality improvement and implementation research methods to study healthcare innovations. Implement Sci. 2015;10:31. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-015-0219-z.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  44. Cunningham SD, Tschann J, Gurvey JE, Fortenberry JD, Ellen JM. Attitudes about sexual disclosure and perceptions of stigma and shame. Sex Transm Infect. 2002;78(5):334–8. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/sti.78.5.334.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Waller J, Marlow LA, Wardle J. The association between knowledge of HPV and feelings of stigma, shame and anxiety. Sex Transm Infect. 2007;83(2):155–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/sti.2006.023333.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Waller J, Ostini R, Marlow LA, McCaffery K, Zimet G. Validation of a measure of knowledge about human papillomavirus (HPV) using item response theory and classical test theory. Prevent Med. 2013;56(1):35–40.

    Article  Google Scholar 

  47. Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-017-0635-3.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Haas JS, Sprague BL, Klabunde CN, et al. Provider attitudes and screening practices following changes in breast and cervical cancer screening guidelines. J Gen Intern Med. 2016;31:52–9.

    Article  PubMed  Google Scholar 

  49. Shen E. On the use of sampling weights for retrospective medical record reviews. Perm J. 2020;24. https://doiorg.publicaciones.saludcastillayleon.es/10.7812/TPP/18.308

  50. Drescher J, Schwartz A, Casoy F, et al. The growing regulation of conversion therapy. J Med Regul. 2016;102(2):7–12.

    PubMed  PubMed Central  Google Scholar 

  51. Junqueira DR, Zorzela L, Golder S, et al. CONSORT Harms 2022 statement, explanation, and elaboration: updated guideline for the reporting of harms in randomised trials. BMJ. 2023;381: e073725. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/bmj-2022-073725.

    Article  PubMed  Google Scholar 

  52. Wilson JP, Aarons GA, Sales A, Xu D, Wensing M, Hutchinson A, Beidas RS, Geng E. Revisiting the scope and expectations of implementation science and implementation science communications. Implement Sci. 2024;19(1):69.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Franco EL. The time has come to implement primary human papillomavirus screening for cervical cancer in the United States. Cancer. 2023;129(18):2765–8. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/cncr.34899.

    Article  PubMed  Google Scholar 

  54. Harper DM, Bettcher CM, Young AP. It is time to switch to primary HPV screening for cervical cancer. Am Fam Phys. 2024;109(1):8–9.

    Google Scholar 

  55. Albers B, Rapley T, Nilsen P, Clack L. Editorial: Tailoring in implementation science. Editorial. Front Health Serv. 2023;3. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2023.1233597

  56. Valenta S, Ribaut J, Leppla L, et al. Context-specific adaptation of an eHealth-facilitated, integrated care model and tailoring its implementation strategies—A mixed-methods study as a part of the SMILe implementation science project. Original Research. Front Health Serv. 2023;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.977564

  57. Metz A, Kainz K, Boaz A. Intervening for sustainable change: Tailoring strategies to align with values and principles of communities. Hypothesis and Theory. Front Health Serv. 2023;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.959386

  58. von Lengerke T, Tomsic I, Krosta KME, et al. Tailoring implementation interventions of different order in infection prevention and control: A cascadic logic model (IPC-CASCADE). Perspective. Frontiers in Health Services. 2023;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.960854

  59. Leeman J, Draeger LB, Lyons K, Pham L, Samuel-Hodge C. Tailoring implementation strategies for scale-up: Preparing to take the Med-South Lifestyle program to scale statewide. Original Research. Frontiers in Health Services. 2022;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.934479

  60. Potthoff S, O'Donnell AJ, Karlsen AT, Brendryen H, Lid TG. Pragmatic approaches for addressing alcohol in general practice: Development of a tailored implementation intervention. Original Research. Frontiers in Health Services. 2022;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.940383

  61. McHugh SM, Riordan F, Curran GM, et al. Conceptual tensions and practical trade-offs in tailoring implementation interventions. Perspective. Front Health Serv. 2022 2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.974095

  62. Haverhals LM, Magid KH, Kononowech J. Applying the tailored implementation in chronic diseases framework to inform implementation of the preferences elicited and respected for seriously ill veterans through enhanced decision-making program in the United States Veterans Health Administration. Methods. Front Health Services. 2022-September-02 2022;2. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2022.935341

  63. Nilsen P, Potthoff S, Birken SA. Conceptualising four categories of behaviours: implications for implementation strategies to achieve behaviour change. Hypothesis and theory Front Health Serv. 2022. https://doiorg.publicaciones.saludcastillayleon.es/10.3389/frhs.2021.795144.

    Article  Google Scholar 

  64. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177–94. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s11414-015-9475-6.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Kanter MH, Lindsay G, Bellows J, Chase A. Complete care at Kaiser Permanente: transforming chronic and preventive care. Jt Comm J Qual Patient Saf. 2013;39(11):484–94. https://doiorg.publicaciones.saludcastillayleon.es/10.1016/s1553-7250(13)39064-3.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank all stakeholders for assisting with the study design, data collection, evaluation, and dissemination. We are especially grateful to the patient and clinical stakeholders. Their experience, expertise and insights were critical in ensuring the relevance of our study. We thank the primary care and obstetrics/gynecology leaders, physicians, licensed vocational nurses, medical assistants, department administrators and patients who participated in our study interview, surveys and local team meetings or supported the implementation of locally-tailored interventions. We are also grateful to KPSC leadership and the HPV Task Force members for their support for this study. Finally, we thank members of Kaiser Permanente Southern California for helping us improve care through the use of information collected through our electronic health records.

Funding

This study was funded by Patient-Centered Outcome Research Institute (PCORI), award number: CDR- 2018 C1 - 10987.

Author information

Authors and Affiliations

  1. Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2 Floor, Pasadena, CA, 91101, USA

    Chun R. Chao, Nancy Cannizzaro, Erin E. Hahn, Ernest Shen, Chunyi Hsu, Corrine E. Munoz-Plaza, Michael H. Kanter, Benjamin I. Broder & Brian S. Mittman

  2. Department of Health Systems Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA, USA

    Chun R. Chao, Erin E. Hahn, Quyen Ngo-Metzger, Michael K. Gould, Michael H. Kanter & Benjamin I. Broder

  3. Department of Clinical Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA, USA

    Michael H. Kanter

  4. Department of Obstetrics and Gynecology, Anaheim Medical Center, Kaiser Permanente Southern California, Anaheim, CA, USA

    Patricia Wride, Alicia Castaneda & Devansu Tewari

  5. Department of Family Medicine, Los Angeles Medical Center, Kaiser Permanente Southern California, Los Angeles, CA, USA

    Lena H. Ajamian

  6. Department of Obstetrics and Gynecology, Riverside Medicine Center, Kaiser Permanente Southern California, Riverside, CA, USA

    Melissa Hodeib

  7. Southern California Permanente Medical Group Operations, Kaiser Permanente Southern California, Pasadena, CA, USA

    Ivette T. Curiel

  8. Department of Ambulatory Care, Anaheim Medical Center, Kaiser Permanente Southern California, Anaheim, CA, USA

    Stephanie K. Ong

  9. Department of Gynecologic Oncology, Obstetrics and Gynecology, University of California, Irvine, CA, USA

    Krishnansu Tewari

  10. Department of Gynecologic Oncology, Obstetrics and Gynecology, University of California, San Diego, USA

    Ramez N. Eskander

Authors
  1. Chun R. Chao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Nancy Cannizzaro
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Erin E. Hahn
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ernest Shen
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Chunyi Hsu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Quyen Ngo-Metzger
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Michael K. Gould
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Corrine E. Munoz-Plaza
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Michael H. Kanter
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Patricia Wride
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Lena H. Ajamian
    View author publications

    You can also search for this author inPubMed Google Scholar

  12. Melissa Hodeib
    View author publications

    You can also search for this author inPubMed Google Scholar

  13. Benjamin I. Broder
    View author publications

    You can also search for this author inPubMed Google Scholar

  14. Ivette T. Curiel
    View author publications

    You can also search for this author inPubMed Google Scholar

  15. Alicia Castaneda
    View author publications

    You can also search for this author inPubMed Google Scholar

  16. Stephanie K. Ong
    View author publications

    You can also search for this author inPubMed Google Scholar

  17. Krishnansu Tewari
    View author publications

    You can also search for this author inPubMed Google Scholar

  18. Ramez N. Eskander
    View author publications

    You can also search for this author inPubMed Google Scholar

  19. Devansu Tewari
    View author publications

    You can also search for this author inPubMed Google Scholar

  20. Brian S. Mittman
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

CRC conceptualized the study, obtained funding, supervised the study conduct and data collection, interpreted study findings, and drafted the initial draft and critically revised this manuscript. NC performed project management, including supervising, coordination, and budget management, contributed to the development of data collection tools and stakeholder engagement, and critically revised the manuscript. EEH provided scientific inputs for the study design and conduct, led the qualitative data collection and analysis, and critically revised the manuscript. ES provided scientific input for the study design and led the statistical analyses. CH performed project management, including supervising, coordination, and budget management, contributed to the development of data collection tools and stakeholder engagement, and critically revised the manuscript. QNM provided scientific inputs for the study design and conduct, assisted with result interpretation, and critically revised the manuscript. MKG provided scientific inputs for the study design and conduct, assisted with result interpretation, and critically revised the manuscript. CEM conducted qualitative data recruitment, collection and analysis, including focus groups and qualitative interviews. MHK provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. PW assisted with stakeholder engagement, recruitment and data collection, and critically revised the manuscript. LHA provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. MH provided scientific inputs for the study design and conduct, and critically revised the manuscript. BIB provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. ITC provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. AC provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. SKO provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. KT provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. RNE provided inputs for the study design and conduct as a stakeholder, assisted with result interpretation, and critically revised the manuscript. DT conceptualized the study, obtained funding, facilitated engagement, provided critical inputs for the study design and conduct, interpreted study findings, and critically revised this manuscript. BSM conceptualized the study, obtained funding, provided critical inputs for the study design and conduct, interpreted study findings, and critically revised this manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chun R. Chao.

Ethics declarations

Ethics approval and consent to participate

The KPSC IRB reviewed and approved the study. Individuals provided consent to participate in the qualitative interviews. Completion of the study survey was considered implicit consent to participate in the study by the individuals who completed the study surveys.

Consent for publication

Not Applicable.

Competing interests

None to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chao, C.R., Cannizzaro, N., Hahn, E.E. et al. A pragmatic randomized trial to compare strategies for implementing primary HPV testing for routine cervical cancer screening in a large healthcare system. Implementation Sci 20, 21 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-025-01432-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-025-01432-9

Keywords

Implementation Science

ISSN: 1748-5908