Evaluating SinergiAPS, an intervention based on patient feedback to improve patient safety in primary care: a cluster randomized trial

Abstract

Background

Patient safety, defined by the WHO as the absence of preventable harm, is a critical component of healthcare quality and poses a significant challenge globally. This study aimed to evaluate the effectiveness of SinergiAPS, a patient-centred audit and feedback intervention, in improving patient safety in primary healthcare (PHC) centres.

Methods

We conducted a 12-month cluster randomized controlled, multicentre trial. Fifty-nine PHC centres (1053 PHC professionals) in Spain were recruited and randomly allocated (1:1) to usual care or SinergiAPS intervention. The SinergiAPS intervention comprised: a bespoke feedback report with results from audits of patient safety based on the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) questionnaire, administered to 75 patients/centre; a set of educational materials, and a structured template to record safety improvement plans. The primary outcome (at the PHC professional level) was patient safety culture (Medical Office Survey on Patient Safety Culture (MOSPSC) questionnaire). Secondary outcomes (at the centre level) were patient-reported safety (six PREOS-PC scales), and rate of avoidable hospital admissions. After a 12-month follow-up, we conducted 15 semi-structured interviews with PHC professionals to explore their perceptions of the intervention.

Results

During the 12-month follow-up, 10 of the 30 centres in the intervention group held action plan team meetings and eight registered 57 safety improvement action plans. The plans aimed to improve patient activation, address treatment-related incidents, enhance communication between patients and providers, and strengthen patient safety culture. At 12 months, no significant differences were observed in MOSPSC mean score (intervention: 3.60 [95% CI 3.55 to 3.64] vs. control: 3.64 [3.60 to 3.68]). Similarly, no differences were observed in the secondary outcomes, with both groups experiencing a decline in patient-reported safety and avoidable hospital admissions. The qualitative interviews evidenced that the onset of the COVID-19 pandemic 6–9 weeks after initiating the follow-up period severely limited PHC’s capacity of developing and implementing safety improvement action plans, despite high levels of acceptability and perceived utility of the SinergiAPS intervention.

Conclusions

In the context of the health emergency caused by the COVID-19 pandemic, SinergiAPS did not improve patient safety in Spanish PHC centres.

Trial registration

ClinicalTrials.gov (NCT03837912).

Background

Patient safety, defined by the World Health Organization (WHO) as “the absence of preventable harm to patients and the reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum” [1], is one of the fundamental pillars of healthcare quality and represents one of the main challenges for healthcare systems worldwide.

It is estimated that the direct costs of harm (e.g. tests, treatments, and additional healthcare) account for around 2.5% of total healthcare spending [2]. Harm occurring in primary care and outpatient care each year represents over 6% of hospital bed-days. Across countries in the Organisation for Economic Co-operation and Development (OECD), this translates to over 7 million hospital admissions annually. Furthermore, harm occurring within the hospital setting accounts for 15% of daily intensive care activities [3, 4].

In Spain, which leads Europe in Primary Health Care (PHC) utilization, the PHC system comprises 2700 Primary Care centres where healthcare professionals operate in teams. These teams on average consist of 10 doctors, 2 paediatricians, 12 nurses, a midwife, a social worker, and 2 administrative staff [5]. Over the past decade, there has been a heightened emphasis on patient safety in Spanish PHC centres. The APEAS study [6], conducted in 48 PHC centres across 16 regions, estimated that about 3 million adverse events occur annually in Spanish PHC centres, with approximately two-thirds of these incidents being preventable.

Enhancing safety culture, defined as the collective values, attitudes, perceptions, competencies, and behaviour patterns of individuals and groups that influence an organization’s commitment to, and proficiency in, health and safety management [7], represents the foremost challenge in advancing towards a safer healthcare system, as noted by the Institute of Medicine [8]. Despite growing endeavours to devise strategies for bolstering patient safety in PHC centres, aiming to foster a safety culture and curtail preventable adverse events and harm [1, 9], the existing body of evidence regarding the effectiveness of these proposed strategies remains somewhat restricted [10].

The participation of patients and their families is a key strategy to promote safety in healthcare [11]. The benefits of involving patients are significant, with estimates suggesting that it could reduce the burden of harm by up to 15%, potentially saving countless lives and billions of dollars each year [12]. Under the motto “Let’s give voice to patients!”, the WHO emphasizes the need to create platforms at the international, national, and local levels to give voice to patients and their families, ensuring their participation, empowerment, and the valuation of their opinions as an essential source of learning and improvement [13].

However, most research in this area has focused on hospital settings [14]. Despite the emergence of new tools and resources, there remains a gap in understanding how to effectively utilize patient-reported data to enhance safety in primary care [15]. A recent systematic review [16] of randomized controlled trials evaluating interventions to promote patient and family engagement in primary care safety identified 12 trials, primarily targeting medication safety. Meta-analyses revealed no significant effects on adverse drug events (OR = 0.83, 95% CI [0.70, 1.08]) or medication appropriateness (OR = 0.92, 95% CI [0.76, 1.13]; MD = 0.71, 95% CI [− 0.10, 1.52]). Moreover, most interventions were at low levels of patient and family engagement, primarily providing them with information rather than integrating their views into safety planning and improvement initiatives. Our study addresses these gaps by implementing and evaluating SinergiAPS, a pioneering intervention that actively incorporates patients' perceptions and experiences into safety planning in primary care centres. By doing so, we aim to generate evidence on effective strategies to engage patients and families in enhancing patient safety in this setting.

Methods

Aim and hypothesis

This study aimed to evaluate the effectiveness of SinergiAPS, a patient-centred audit and feedback intervention, in improving patient safety in primary healthcare (PHC) centres. We hypothesized that providing primary care professionals with insights into patients’ safety perceptions, as part of an intervention that promotes teamwork and offers training resources on patient safety, would strengthen the patient safety culture among healthcare professionals. In turn, this could lead to improved patient-reported safety experiences and a reduction in avoidable hospital admissions.

Design

Pragmatic, phase 3, 12-month, two-arm, two-level cluster randomized controlled trial (1248 PHC professionals within 59 PHC centres; with randomization at the centre level in a 1:1 ratio). PHC centres were allocated to the SinergiAPS intervention group or to a waiting list control group [17]. The reporting of this manuscript is based on the Consort 2010 statement: extension to cluster randomized trials [18]. Online Appendix 1 provides a more detailed description of the methods followed in this study.

Description of the SinergiAPS intervention

The SinergiAPS (“Sinergias entre profesionales y pacientes para una Atención Primaria Segura|) intervention was delivered through a bespoke online tool and, was targeted at the cluster level. A full description of the intervention is available elsewhere [19]. The intervention involved three stages (Fig. 1):

• a) Measurement: All patients in waiting rooms were consecutively invited to provide feedback about the safety of the healthcare they received in their centre during the previous 12 months. They did so by completing the Spanish online version of the PREOS-PC Compact questionnaire [20, 21] using tablet computers. Patients were given the choice to complete the Spanish, Catalan or English version of the questionnaire. We administered 75 questionnaires per centre (based on the number of responses needed to achieve 0.7 reliability in scale scores [22]). All patients received plain language Patient Information Sheets and consent forms.

• b) Feedback: From 13 January 2020, for a period of 12 months, all participating centres in the intervention group were given access to a password-protected, centre-specific feedback report, accessed through a secured link to our online platform. The report (example available in Online Appendix 2) was automatically generated using a bespoke online tool and provided real-time information about the performance of each PHC centre in comparison with the rest of the participating centres. A set of educational materials and online resources (a repository of publicly available written materials, infographics, and videos from local, national and international healthcare organizations) were also provided through the online platform. Follow-up calls were made by members of our team to ensure all centres had access to the intervention materials and to resolve technical problems, when needed.

• c) Action planning and implementation: PHC centres were instructed to create an Action Planning Team (APT) responsible for receiving the Feedback Report, considering the area(s) that should be addressed, and designing action plans for safety improvement. The APTs were prompted to register their plans in a structured form available via the online tool. The intervention was meant to be self-contained, and therefore the researchers did not support centres in the analysis of the feedback report nor in the action planning and implementation.

Fig. 1

Consort flowchart

SinergiAPS is a theory-based intervention, based on the Clinical Performance Feedback Intervention Theory, which states that behaviour is regulated through comparisons with standards or goals and that feedback can draw attention to existing gaps [23]. The SinergiAPS intervention was developed following the guidelines from the Medical Research Council [24]. The intervention was co-designed with 43 PHC professionals who participated in a qualitative study that involved four focus groups and three semi-structured individual interviews [25]. Before being trialled, the intervention was piloted in a 3-month, single-arm feasibility study in ten PHC centres [19], which showed that the intervention was feasible to be delivered; and allowed us to gather useful information that we used to refine further and improve the acceptability and potential impact of the intervention (e.g. increasing the sample of patients evaluating each centre, or improving the educational materials accompanying the feedback report).

Control group

No contact was made with the control practices after the collection of baseline data. They continued with usual care which involved promoting a safety culture through implementing incident reporting systems and following safe practices through protocols and best practice guidelines. After trial completion, they received the two feedback reports (corresponding to the baseline and to the post-intervention periods).

Randomization

The allocation schedule for random assignment of intervention or control group to PHC centres was computer generated, including stratification by teaching centre (yes vs. no), management area (Mallorca, Catalunya Central, and Camp de Tarragona) and their baseline MOSPSC [26] scores (terciles according to the synthetic index).

Allocation concealment mechanism

The treatment allocation for each PHC centre was kept in within the coordinator team in Mallorca until all the centres had completed the baseline data collection (both in the intervention and control centres (clusters)). Then a research assistant phoned all centres to inform them about their group allocation and emailed access to the intervention materials to those centres in the intervention group.

Blinding

Due to the nature of the intervention, PHC professionals could not be blinded to the assignment to the interventions. However, outcome assessors (i.e. questionnaire administrators and information specialists extracting avoidable hospital admission data), and statisticians were blinded to group allocation.

Eligibility criteria

• Eligibility of the clusters (PHC centres): we included public PHC centres from two regions (Balearic Islands and Catalonia). We included only those centres under the management area of Mallorca, Catalunya Central, and Camp de Tarragona. We excluded those centres attending specific populations or health problems (e.g. women health care centres), those that had been created recently (< 12 months before the start of the recruitment period), and those that had participated in the feasibility study described above.

• Eligibility of the PHC professionals: we included all healthcare professionals working in the centre, including administrative staff. We excluded those with less than 12 months of experience working in that centre, as well as those planning to leave their work at the centre during the next 12 months.

• Eligibility of patients: we invited patients who had visited their PHC centre at least once in the previous 12 months. Patients aged < 18 were included only if their parents or guardians agreed to complete the questionnaire on their behalf. They had to be able to speak Spanish, Catalan or English and provide informed consent.

Recruitment and training of PHC centres

We invited all the PHC centres from three areas (Mallorca, Catalunya Central, and Camp de Tarragona) meeting the eligibility criteria previously described. Centres were asked to consent as a unit, with all professionals willing to participate. Consent was also taken from patients invited to complete the patient survey. All informed consents were sought before randomization. The intervention was standardized across all sites and regions.

Data collection

Data was collected at baseline and 12 months post-intervention (i.e. 12 months after the feedback reports were sent to the centres). We monitored the progress of the intervention in all the centres. Data from patients included patient-reported experiences and outcomes of patient safety in PHC (measured with PREOS-PC questionnaire) and patient sociodemographic characteristics. During the baseline assessment, interviewers invited patients to complete the questionnaire in person and provided assistance if needed. For the follow-up assessment, the questionnaire was administered by email or by telephone with interviewer support. At baseline and follow-up, different groups of patients were surveyed to capture their safety experiences at each time point. Data from healthcare professionals was collected through online questionnaires and included the perceived safety climate (with the Spanish MOSPSC), and sociodemographic and occupational characteristics. The same group of healthcare professionals was surveyed at both baseline and follow-up, with efforts made to retain as many participants as possible. Another secondary outcome was the rate of avoidable hospitalizations. In Spain, hospitalization data is recorded through the “Minimal Set of Basic Data” (CMBD, by its Spanish acronym), a mandatory dataset for all hospitals within the Spanish National Health System. By linking primary healthcare patient data to the CMBD, we were able to track hospital admissions and calculate the rate of avoidable hospitalizations for patients registered at each of the participating PHC centres. Finally, we also extracted data about the centre characteristics, including rurality, list size, number of healthcare professionals and “Mortalidad en áreas pequeñas Españolas y Desigualdades Socioeconómicas y Ambientales”—MEDEA deprivation index. [27]. The information regarding each centre and its characteristics was sourced from the management's information systems.

Outcomes

The primary outcome was the patient safety culture, measured with the Spanish MOSPSC [26] at the individual (PHC professional) level. The MOSPSC is a recognized instrument in Spanish PHC, and it is supported by the Ministry of Health and the main PHC society (Online Appendix 3). This instrument has been validated in a sample of Spanish PHC professionals. It includes 67 items grouped in 13 dimensions. Patient safety culture was computed both as a global score (synthetic index calculated at the healthcare professional level based on the mean score of the 67 items in the questionnaire [28]) and at the individual domain level.

Secondary outcomes were evaluated at the cluster (PHC centre) level and included the six scales in the PREOS-PC Compact questionnaire (a psychometrically robust measure of patient safety): centre activation; patient activation; experiences of safety problems; harm; and overall rating of patient safety. An additional secondary outcome was the rate of avoidable hospitalisations, based on the definition provided by the Agency for Health Research and Quality, which specifically includes conditions such as asthma, chronic obstructive pulmonary disease, congestive heart failure, angina, diabetes, and chronic kidney disease. It was calculated based on data extracted from the Minimum Basic Data Set using predefined ICD-9 codes [29] (full details available in Online Appendix 4). For each participating centre, we obtained the total number of registered patients, and the total number of avoidable hospitalizations recorded during the previous 12 months.

Sample size calculation

The sample size calculation is based on the trial’s main outcome measure—the Spanish MOSPSC, which produces an overall score ranging from 1 to 5. Assuming an average cluster size of 26 professionals per centre, we calculated that 1248 professionals would take part in the study. Assuming a follow-up rate of 80%, we would have complete data from approximately 998 professionals. Considering the cluster design and using a conservative estimation of intra-class correlation of 0.1, this sample size would allow us to detect at least a 0.3 difference in effect size (with 80% power and a significance level of 5%). This would approximately correspond to a difference of 0.8 points in the index (assuming a standard deviation (SD) of 2.3 from a previous study) [22]. We recruited 75 patients per centre, which is the minimum number to achieve a 0.7 reliability of scale scores at the centre level [22].

Based on the results from a feasibility study [19], concerns that lower-than-expected healthcare professionals would agree to participate would affect the trial’s ability to detect an effect of the intervention resulting in a protocol amendment. Therefore, in addition to the 48 centres initially considered, a further 11 health centres were approached to join the study during the recruitment period.

Statistical analyses

All the analyses were in accordance with our pre-specified analysis plan. Baseline characteristics were examined by group using frequencies (with percentages) for binary and categorical variables and means (and SD) or medians (IQR) for continuous variables. The results from the trial are presented as comparative summary statistics (difference in proportions or means) with 95% CI.

We used cluster-specific methods because centres rather than professionals were randomized, and we expected that variance in how professionals implemented the intervention would be partly explained by the centre. Also, centres were managed in three different management areas (Mallorca, Catalunya Central and Camp de Tarragona—each of them substantially different from the management perspective, and available resources). Therefore, we applied multilevel or mixed-effect regression models to analyse the effect of the intervention on post-intervention values adjusted for baseline or pre-intervention values. In the case of MOSPSC, the multilevel models comprised three levels: a first level defined by the professionals, a second level defined by the centre and a third level defined by the area. In the case of PREOS-PC and the rate of avoidable hospital admissions, the multilevel models comprised two levels: a first level defined by the centre and a second level defined by the area. Goodness-of-fit measures based on AIC (Akaike information criterion) and BIC (Bayesian information criterion) reported that the models with the best fit to the data were those that included random intercepts and an identity covariance structure, but without random slopes. Finally, given that the pre- and post-intervention variables analysed did not follow a normal distribution, the resampling technique was applied with an extraction of 1000 samples in each model designed. All analyses were carried out based on an intention-to-treat approach. We used SPSS V29 and applied an α level 0.05 throughout.

Deviations from the protocol due to the COVID-19 pandemic

The WHO declared COVID-19 a pandemic by March 11th, 2020—9 weeks after the feedback report had been delivered to the PHC centres in the intervention group. This new context forced us to introduce two major modifications in the trial: (1) to reduce efforts to monitor and promote the uptake of the intervention by the PHC professionals (who were overwhelmed with an exceptional challenge as first-line workforce to contain the first wave of the pandemic), (2) to change the method for administering the PREOS-PC questionnaire to patients after the 12-month follow-up: from face to face in the waiting room (as in the baseline period) to telephonic or online administration (drawing random samples of patients from electronic health records).

Qualitative study with PHC providers

After the post-intervention follow-up, we conducted a qualitative study which involved semi-structured interviews with 15 healthcare professionals from the intervention group. This included eight doctors and seven nurses, with 14 females and one male. The mean (SD) age was 48 (6) years. The interviews aimed to understand how the intervention was perceived by the PHC professionals, focusing on acceptability, perceived utility, and implementation barriers, including any unintended consequences (interview guide available in Online Appendix 5). We used purposeful sampling to ensure variation in terms of the type of professionals and of centres. Interviews took place telephonically. All 15 interviews were independently examined by two qualitative researchers. Thematic analysis [30] was used to identify recurrent themes and subthemes common to interviewees working in centres.

Results

At the cluster level, of the 80 PHC centres assessed for eligibility, twelve were ineligible (two provided services to specific populations, and ten had already participated in our feasibility study). Nine centres declined participation, due to excessive workload (n = 6), understaffing (n = 2), and lack of interest (n = 1). Ultimately, 59 centres were recruited and allocated to the intervention (30 centres) and control (29 centres) groups. The 59 centres completed the 12 months’ follow-up and contributed to the analysis (Fig. 2).

Fig. 2

Main components of the SinergiAPS intervention: measurement, feedback, and action planning and implementation

At the professional level, 1,971 PHC professionals from the 59 recruited centres were invited to participate and 1053 responded positively (53.4%) and completed the baseline questionnaires between August 2019 and January 2020. Two hundred sixty-six PHC professionals were lost to follow-up, leaving 794 (75.3%) professionals who participated in the follow-up (Fig. 2). The characteristics of the participating centres and healthcare professionals were well balanced (Table 1).

Table 1 Baseline characteristics of healthcare centres and healthcare professionals

At baseline, a total of 4555 patients participated in the study by completing the PREOS-PC questionnaire, with 2279 (50%) from Mallorca, 793 (17.4%) from Catalunya Central, and 1483 (32.6%) from Camp de Tarragona. By follow-up, the number of participating patients was 5922, distributed as follows: 2262 (38.2%) from Mallorca, 1190 (20.1%) from Catalunya Central, and 2470 (41.7%) from Camp de Tarragona. The sociodemographic and clinical characteristics of these patients are detailed in Online Appendix 6.

Primary outcome

At baseline, similar scores of overall patient safety culture were observed in the intervention (3.601; 95% CI 3.563 to 3.639) and control group (3.609; 95% CI 3.571 to 3.646). The multilevel model indicated no relevant differences between intervention (3.597; 95% CI 3.551 to 3.643) and control groups (3.640; 95% CI 3.600; 3.682) at follow-up for the overall score (β = 0.009; 95% CI − 0.055 to 0.038) and for any of the subscales (Table 2).

Table 2 Impact of the SinergiAPS intervention on safety culture measured with the Medical Office Survey on Patient Safety Culture (MOSPSC) domain-specific and overall scores

Secondary outcomes

The impact of the SinergiAPS intervention on patient-reported patient safety measured with the PREOS-PC questionnaire is described in Table 3. The median (IQR) of patient questionnaires successfully administered per centre was 76 (75–77) at baseline and 78 (75–110) and 12-month follow-up. At baseline, no relevant differences were observed between intervention and control groups in any of the six PREOS-PC scales. At follow-up, scores significantly decreased (indicating lower levels of patient-reported patient safety) in the intervention and control group for all the scales, except for the “patient activation” scale. However, the hierarchical model indicated no significant differences between the intervention and control groups at 12 months in any of the scales.

Table 3 Impact of the SinergiAPS intervention on patient-reported patient safety measured with the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) questionnaire

The impact of the SinergiAPS intervention on avoidable hospital admissions is described in Online Appendix 7. At baseline, the rate of avoidable hospital admissions was similar in the intervention (9.33 avoidable admissions per 1,000 registered patients (95% CI 8.11 to 10.54)) and control group (9.51 (8.25 to 10.77)). At 12-month follow-up the rate decreased both in the control (6.91 (5.65 to 8.17)) and intervention group (7.24 (5.68 to 8.79)), with no significant differences between groups observed (IRR = 1.048; 95% CI 0.865 and 1.270). This translates to an absolute effect of 96 avoidable hospital admissions per 1000 patients per year, ranging from 270 fewer to 536 more admissions per year. The hierarchical model indicated a not statistically significant effect in avoidable hospital admissions rate (β = 0.299; 95% CI = − 0.469 to 1.059).

Reach and uptake of the SinergiAPS intervention

Patient feedback was successfully gathered in all centres by administering the PREOS-PC questionnaire to at least 75 patients per centre. The feedback report and other SinergiAPS co-interventions were delivered to 30 intervention centres, with 26 (87%) accessing the online platform. Nineteen centres (63%) created an APT, and ten (33%) held at least one APT meeting. Eight centres (27%) registered 57 action plans (ranging from 2 to 20 per centre). These action plans, described in Table 4, focused on improving patient activation (12%), addressing treatment-related incidents (16%), enhancing patient-provider communication (2%), and fostering a safety culture (14%). A significant portion (47%) addressed broader organizational issues like continuity of care, staff shortages, accessibility, and better management of specialist referrals.

Table 4 Safety problems identified and safety improvement actions proposed by the participating centres, collected in the template

Embedded qualitative study

Four main themes emerged from the qualitative interviews after the 12 months follow-up with 15 healthcare professionals: impact of COVID-19 pandemic; perceptions about the SinergiAPS tool, implementation of the SinergiAPS tool during the trial, and potential large-scale implementation of SinergiAPS as part of routine clinical practice. Online Appendix 8 shows quotations to illustrate and support the findings.

Impact of COVID-19 Pandemic: The COVID-19 pandemic significantly disrupted health centre functioning, exacerbating workload and introducing organizational chaos. Participants perceived increased incidents of diagnostic errors, medication mistakes, and communication breakdowns, often underreported due to time constraints.

Healthcare professionals’ perceptions regarding the SinergiAPS tool: PHC professionals generally viewed the SinergiAPS tool positively both theoretically and in practical application. Professionals valued the use of patient feedback, though concerns were raised about the availability of resources of collect this data. Feedback reports were appreciated for actionable insights, particularly improvement recommendations and open-ended questions. However, supporting materials (training content and action plan registries) were underutilized and deemed impractical by some respondents.

Implementation challenges during the trial: Implementation of the SinergiAPS tool varied significantly across health centres. While some successfully developed action plans and implemented measures, others struggled to comprehensively review the feedback report. The disruption caused by the pandemic was identified as the main factor affecting the implementation of the intervention. Time constraints emerged as a common barrier despite overall positive acceptance of the tool's utility.

Barriers and facilitators for large-scale implementation: While recognizing its potential benefits for patient safety and feedback collection, key barriers for implementation of SinergiAPS as part of routine clinical practice included insufficient time and resources to address safety issues effectively, as well as varying levels of professional engagement and knowledge in patient safety. Motivational factors among staff also posed challenges. Facilitators identified included the simplicity and accessibility of the tool, comprehensive training in safety protocols, and the need for clear leadership to drive safety improvement initiatives.

Discussion

Principal findings of the study

This large trial, involving 59 PHC centres in Spain, found that SinergiAPS—a patient-centred audit and feedback intervention—did not significantly improve healthcare professionals' perceptions of the patient safety culture within their medical offices, as measured by the MOSPSC. Similarly, the intervention did not improve patient-reported safety or reduce avoidable hospital admissions after 12 months. Both groups (intervention and control) experienced a decline in patient-reported safety and avoidable hospital admissions, likely reflecting broader trends in patient safety during the COVID-19 pandemic, when the prioritization of COVID patients may have affected overall safety and service volume. However, our mixed-methods process evaluation showed that PHC professionals viewed SinergiAPS as a valuable and useful tool, and the use of patient-reported data to guide safety improvements was well received. The lack of significant effects may be partially attributed to the reduced capacity of PHC centres to develop and implement safety improvement action plans, due to the onset of the pandemic early in the 12-month intervention period. Despite this, the study demonstrates that SinergiAPS can deliver actionable patient feedback. Ten centres successfully implemented the intervention and designed 57 safety improvement initiatives based on patient feedback. These plans, which aimed to improve patient activation, address treatment-related incidents, enhance communication between patients and providers, and strengthen patient safety culture, may provide valuable examples for clinicians and healthcare managers in future safety improvement efforts.

Strengths and limitations

This study has several strengths, including its large scale, with 59 centres, approximately 1000 healthcare professionals, and 10,477 patients involved. Robust data collection methods were used, employing validated questionnaires and administrative data. A diverse range of centres participated, enhancing the generalizability of the results. The study's simultaneous delivery across three management areas and a 12-month follow-up period are noteworthy, providing sufficient time for the intervention’s impact. High recruitment and retention rates at the centre level further support the trial’s external validity. The internal validity is likely high due to appropriate sample size calculations, clustering considerations, and minimized bias through timing randomization after baseline data collection. Process evaluations and post-intervention interviews added valuable insights into the intervention’s implementation.

However, there are limitations: objective measures of intervention usage were not collected, hindering analysis of a dose–response effect. Pandemic-related changes in questionnaire administration may affect the comparability of scores. Professional participation was lower (53%), potentially introducing participation bias. Selection bias might have led to an over-representation of safety-conscious professionals, though low intervention uptake suggests minimal impact on results. Blinding was not possible for participants, and professional withdrawal rates post-intervention were suboptimal but similar across groups. An intention-to-treat analysis was applied to mitigate bias. Since the SinergiAPS intervention targeted primary healthcare professionals, it is initial co-design included primary care professionals but not patients (patients were only involved in the development of the PREOS-PC questionnaire). A further limitation is the lack of direct patient involvement in the intervention’s design and evaluation. Since SinergiAPS was developed to support primary healthcare professionals, its initial co-design process included professionals but not patients, who were only involved in developing the PREOS-PC questionnaire. Additionally, our post-trial qualitative evaluation was limited to healthcare professionals, without patient perspectives. Incorporating qualitative data from patients would have provided valuable insights into their views on the intervention.

Comparison with previous studies

Our trial's findings contrast with a multi-setting systematic review that indicated patient and family involvement interventions could reduce adverse events, length of hospital stay, and improve patient safety experiences and satisfaction [12]. Most patient safety interventions have focused on hospital settings [15]. In primary care, a recent systematic review [16] found inconclusive evidence for the effectiveness of patient and family engagement strategies, with no significant impact on adverse drug events or medication appropriateness. However, it is important to note that SinergiAPS is a unique and pioneering intervention that involves a comprehensive, feedback-driven approach in primary care, with a focus on patient-reported data to guide safety improvement. Unlike the interventions covered in the aforementioned reviews, SinergiAPS engaged healthcare professionals in action plan development based on patient feedback, which distinguishes it from more traditional one-time patient reporting or education-focused interventions.

Our trial showed a significant drop in patient-reported safety after the COVID-19 pandemic began [31], which is consistent with known safety gaps revealed during the pandemic across all healthcare levels, including primary care [32]. The decline in hospital admissions for asthma and COPD in both groups post-intervention likely reflects the disruption caused by COVID-19, affecting hospital capacity and patient behaviour [33]. The early adoption of virtual primary care visits may have also reduced the need for emergency visits.

Implications for research and practice

Given that the findings of this trial are likely to have been affected by the contextual situation of the COVID-19 pandemic, new studies are needed to strengthen the evidence base concerning the SinergiAPS intervention. A new trial is currently underway [34], involving 109 centres from nine regions in Spain, to evaluate the efficacy and the implementation of SinergiAPS.

Meanwhile, health decision makers and managers should encourage and promote the use of patient feedback, given the insights it can offer into the quality and safety of healthcare, particularly when it comes to staff blind spots or failings in institutional culture [35]. This is strengthened by the findings from our embedded qualitative study which showed that PHC professionals in Spain highly valued and accepted the use of patient feedback for quality and safety improvement purposes.

Conclusions

In conclusion, in the context of a health emergency, SinergiAPS was not effective in improving patient safety in Spanish PHC centres. However, given that the trial was conducted during the COVID-19 pandemic—a period of extraordinary strain on healthcare systems—its findings may not fully reflect the intervention’s potential impact under normal healthcare conditions. Future studies are needed to evaluate its effectiveness in a context where primary care operates under standard, non-pandemic circumstances.