Parsi, P. Process evaluation of ‘Country Profiles’, 2010. Utrecht University, WHO CC for Pharmacoepidemiology & Pharmaceutical Policy Analysis, 2010.
In order to create a healthy pharmaceutical situation in a country, various aspects play a key role in achieving this objective. These include the pharmaceutical policy and legal framework (1) access to medicines (2), the quality and safety of medicines (3) and the rational use of these medicines (4). Over the last decades, specific indicators based on these aspects were developed by WHO and partners to identify what structures, processes and outcomes are necessary to improve the pharmaceutical situation in countries. Monitoring and assessment of the indicators is crucial to know what achievements are made and what challenges remain. Several monitoring methods and studies were created over the years including the WHO Level I and II surveys. For 2011, the WHO decided that there was a need for a monitoring method that offered a capacity for countries to update their information, more transparency of the pharmaceutical situation per country, and ownership of the information by the country. The aim was to collect information about pharmaceutical systems at a country level. The ‘Country Profiles’ pilot project was developed which involved collection of secondary data on pharmaceutical indicators and transformation of this into narrative text to create comprehensive documents on the pharmaceutical situation of 13 pilot countries. This was performed by developing a data collection instrument that was prefilled with data available to WHO and sent to the countries to collect information on various pharmaceutical indicators divided in eight section, namely demographics (1), health services (2), policy issues (3), regulation (4), medicines financing (5), procurement and distribution (6), selection and rational use of medicines (7), and household data (8). The data collection instrument included a glossary and manual. After receiving the completed data collection instrument, the information was transformed into a narrative text to create the pharmaceutical country profile documents. The general aim of this report was to investigate whether the new ‘Country Profiles’ method fulfilled its purpose and expectations by evaluating the instrument, the data collection process, the analysis process and the ‘Country Profile’ working process. To serve the purpose of the country profiles project, it was important to have an instrument and method that could collect correct and complete data with proper 6 referencing in a reasonable time frame with the existing human resources in each country. The most important methodology for this evaluation study was to monitor of the process of project. This was carried out in two ways, namely through observations made by WHO HQ staff working on the pilot (1) and collection of comments and remarks provided by the country coordinators and respondents to the instrument from the countries by completing a feedback form and participating in telephone interviews (2). The most important observational methods of monitoring included analysis of the response to indicators and the rate that prefilled answers were changed. The availability and quality of the data was assessed in this way. Twelve countries completed the data collection instrument; six submitted the feedback form and four interviews were conducted. The response rates for all sections were over 85%, except for the sections on Medicines Financing (68%), Rational Use (63%), and Household Data (35%) meaning these had the least available indicators. The sections on Demographics, Health Services and Rational Use had the highest prefill change rate. The remaining sections had a similar rate varying between 10.6% and 16%. Five out of six respondents to the feedback form agreed that prefilling improved the speed of data collection and four out of six thought that the quality of the data collected was improved by prefilling. The instrument had minor limitations regarding the availability of several indicators and a lack of clarity of some questions. The data collection process was done in a longer time frame than planned, but the overall response (12/13) from the countries was high. The method of creating the country profiles documents by transforming the data collection instrument into a narrative text was successful. In conclusion, it is possible to produce comprehensive pharmaceutical country profiles in a reasonable time frame with the existing human resources using this instrument and method. However further work needs to be done to improve the data collection instrument and to automate the analysis of submitted electronic data.