WARN Evaluations of Educational Strategies and Materials
Market Survey to Assess Recognition of Annie and Moxie and WARN Slogans
Evaluating Educational Teleconferences for Pharmacists and Academic Detailers
Evaluating the Usefulness of the WARN Clinical Practice Fact Sheets
Evaluating the Usefulness of the WARN CD-ROM
Survey of Managed Care Medical Directors Regarding WARN Educational Efforts
Evaluating a Pharmacy Outreach Program Utilizing Pharmacy Wholesalers to Deliver WARN Educational Materials
In summer 2000, WARN worked with pharmacy wholesalers to distribute packets of WARN educational materials to pharmacies in fall and winter 2000-2001. Six of the seven wholesalers that serve Wisconsin pharmacies participated in the intervention. Approximately 1,800 packets of WARN materials were distributed.
In January 2001, a survey was mailed to a random sample of 500 pharmacy managers. The purpose of the survey was to determine whether pharmacies received the educational materials; and whether the educational materials were used for patient education. Results were also used to determine if the intervention should be repeated in fall 2001. Out of the 500 mailed surveys, 185 were returned. Sixty percent of respondents received the packet and 90% of those receiving the packet used the materials provided. Many were interested in receiving materials in the future, and stated that receiving materials by mail directly at the pharmacy would be the most effective means of distribution. WARN staff concluded that pharmacies are willing to be partners in patient education efforts to promote appropriate antibiotic use. Several respondents commented that WARN is a very useful program, and that repeated reinforcement is needed to foster changes in patient and prescriber behavior.
Market Survey to Assess Recognition of Annie and Moxi and WARN Slogans
In spring 2001, a research firm was hired to help measure the extent to which the WARN characters and slogans are recognized by Wisconsin residents. The research firm included recognition questions in their monthly random digit dialing telephone survey called Wisconsin Trends. Calls were made during the latter part of April and the beginning of May 2001. The survey found that 92% of respondents (600) knew that “Get Smart about Antibiotics” meant antibiotics should be used only when they can help you; 46% of respondents had seen the Annie and Moxi characters in doctors’ offices; and of those who had seen or heard of WARN in the past 6 months, 33% had heard about WARN via radio or television, 28% had heard about WARN via their doctor’s office; and only 9% had heard about WARN via the internet.
Evaluating Educational Teleconferences for Pharmacists and Academic Detailers
WARN hosted two one-hour conference calls for pharmacists. These conference calls targeted pharmacists who have regular contact with physicians and provide information on appropriate use of prescription medications.
The objectives of the call were to 1) review the latest research findings on strategies to prevent antibiotic resistance; 2) review the CDC’s principles for management of adult upper respiratory tract infections; and 3) provide an opportunity for participants to discuss their organization’s efforts related to antibiotic education. A packet of materials was sent to the participants prior to the call. During the call, participants were encouraged to share activities in their organizations related to appropriate antibiotic use. Thirteen people participated and were asked to complete and return a feedback survey. R esponse was very positive, with indications that the information learned would be incorporated into current and future interventions. Participants indicated they would share information with colleagues who are also working on appropriate antibiotic use program.
Evaluating the Usefulness of the WARN Clinical Practice Fact Sheets
An evaluation survey was developed to assess the usefulness of the Clinical Practice Fact Sheets and to determine influence on patient care. Utilizing the Department of Regulation and Licensing database, a random sample size of 500 (350 MD/DO and 150 NP/PA) was generated. A cover letter, self-administered survey, and a business-reply envelope were mailed to the sample in late January 2001. After four weeks, 104 surveys were returned for a response rate of 21%. Sixty-four or 58.72% reported receiving the fact sheets. Fifty-seven or 52.29% reported reviewing the fact sheets.
Overall, the fact sheets were well received and useful to those who responded to the survey. Respondents suggested similar formatted and evidence-based recommendations in an updated version in 2002. Respondents noted that the recommendations were not necessarily new information, but did reinforce appropriate diagnosis and treatment of common upper respiratory infections.
Evaluating the Availability of WARN Educational Materials in Pediatric and Family Practice Clinics (An Observational Study)
In Spring 2002, WARN staff visited pediatric and family practice clinics throughout the state to assess the proportion of clinics using WARN materials for public education. A random sample of 75 pediatric and family practice physicians was selected from a Department of Regulation and Licensing database. Some physicians
were excluded based on criteria such as receiving their license after January 1999, practicing at the Marshfield Clinic in Marshfield, no longer practicing or retired, practice information not available, no patients seen at facility, and another physician (in the sample) practicing at the same location. Practice locations were determined using the iMIS database at the Wisconsin Medical Society, telephone directories, Internet sites, and other publicly available information. Once the locations were determined, WARN staff began visiting 50 eligible clinic sites between February 1 and April 15, 2002.
A WARN staff member visited each clinic during normal business hours and visually assessed the patient waiting area and other nearby areas that are accessible to the public. A short questionnaire was used to record the presence or absence of WARN materials including posters, pamphlets, stickers, etc. The WARN staff member also recorded any other sources of patient information on appropriate antibiotic use. The WARN staff member asked to speak with the clinic manager, supervisor, or clinician. After an appropriate introduction, the clinic employee was asked questions to assess familiarity with WARN and prior use of WARN materials. After completing the survey, a clinic detailing folder was offered to the employee. The encounter with the staff person in most circumstances took less than five minutes. The WARN staff member had a standardized script to use for each clinic encounter.
The results of the survey show that nearly half of the target clinics used WARN materials for patient education in the past six months. The English parent brochure, CDC brochure, and poster were the most commonly used materials. Most of the materials were displayed in exam rooms and/or waiting rooms.
Evaluating the Usefulness of the WARN CD-ROM
The CD-ROM was developed as an educational tool specifically for primary care physicians. It contains a narrated slide presentation for clinicians on community-acquired antibiotic resistance and judicious antibiotic use. The CD-ROM also contains slide presentations that could be used by physicians for general public and day care staff audiences. The WARN Clinical Practice Fact Sheets, the Wisconsin Antibiotic Resistance Report-Invasive Streptococcus pneumoniae-1999, and a section for viewing and ordering educational materials is also included. This educational tool offers primary care physicians the latest information regarding appropriate antibiotic use.
A randomized survey was mailed to 500 primary care physicians to evaluate the effectiveness of the CD-ROM. A short survey was mailed with a business-reply envelope in March 2001. 128 surveys were returned for a response rate of only 26%. Of the 128 returned surveys, 6.4% reported viewing the CD-ROM. Based on the results of the survey the WARN staff did not redistribute the CD-ROM in year four. Those who viewed the CD-ROM found it useful and informative. There were very few technical difficulties.
Evaluating the Managed Care Resource Binder Mailing to Medical Directors of Managed Care Organizations
A managed care resource binder was developed that included the Clinical Practice Fact Sheets, evidence-based policy recommendations, nationally recognized treatment guidelines and outcomes measures for three upper respiratory illnesses, and a sampling of WARN materials. The resource binder was mailed to 16 medical directors and two pharmacists in December 2000.
To assess the usefulness of the resource binder and to determine how it was utilized within the organization, an evaluation survey was sent to the 18 medical directors who received the binder. The survey was mailed in February 2001. After eight weeks, seven of the 18 surveys were returned for a response rate of 39%. Overall, the resource binder was considered between moderately and extremely useful, with the nationally recommended treatment guidelines and the WARN educational resources identified as most useful. The most common way the binder was utilized within managed care organizations was to share it with a quality assurance or quality improvement department.
Survey of Managed Care Medical Directors Regarding WARN Educational Efforts
In the fall of 2003, medical directors of all Wisconsin managed care organizations were sent a short survey asking for information about patient and physician education practices, measurement of antibiotic prescribing rates, and current interest in WARN educational efforts. Eleven out of twenty managed care organizations responded to the survey. For the most part, medical directors recognize the importance of both physician and patient education about appropriate antibiotic use. Nine out of eleven plans initiated physician and patient education activities to reduce inappropriate antibiotic use in outpatient care through a variety of venues such as newsletters, conferences, and WARN presentations. Many plans indicated they would use or continue to use WARN materials if free of charge. Only three organizations would consider paying for educational materials. Only four of eleven organizations measured antibiotic prescribing rates among physicians for bronchitis and other respiratory tract diagnoses.
