Primary Healthcare meta-evaluation in Brazil: Lessons learned from the external assessment of the program to improve primary healthcare access and quality.

Abstract Introduction: The Primary Healthcare Access and Quality Improvement Program (PMAQ-AB) implemented in 3 evaluation cycles (2011-2019) represents a turning point in the performance assessment of Primary Healthcare (PHC) in Brazil.Methods: A cumulative meta-evaluation of the PMAQ-AB External Assessment (Avaliação Externa do PMAQ-AB - AE/PMAQ-AB) using the qualitative Case Study method. The matrix of indicators was prepared based on the 4 principles of the Joint Committee on Standards for Educational Evaluations (utility, feasibility, propriety and accuracy) articulated to each of the 6 stages of the evaluation proposed by the Centers for Disease Control and Prevention. Triangulation of collection methods (document analysis, interview and focus group) and participants (managers, professionals and evaluators) and Bardin’s content analysis were implemented.Results: The “Involvement of Stakeholders” presented agreement in the instances of the Unified Health System, expanding adhesion of the municipalities in the cycles, credibility of the evaluator and conciliation by the federal government in the interests of certification of the municipalities and of research by the universities as strengths; while the absence of users in the decision-making processes of formulating and executing the program as a weakness. The “Program Description” was positive in the consistency of the AE/PMAQ-AB with the National Primary Care Policy/2011. The “Assessment Design” portrayed the Brazilian PHC and construction of comparable standards and certification criteria. The use of information technology, logistics and guidelines for training the interviewers gave the collection homogeneity, pointing to the usefulness and feasibility. The “Systematization and Analysis of Evidence” had the online validation systems, offer of micro data, availability of electronic reports to stakeholders as strengths, but there were weaknesses in the methodology transmission to municipal managers, incipient strategies of technical empowerment for interpreting and applying the results, in turn negatively impacting the “Justification of Conclusions” dimension. The utility of AE/PMAQ-AB prevailed in “Sharing Lessons Learned” for an assessment culture in management, services, and academia. Conclusion: The meta-evaluation presents a systemic view of the AE/PMAQ-AB, helping managers and professionals in the PHC assessment process. It raises awareness of user engagement in policy and program assessments, making them more plural, participatory and democratic.