Assessment of Interprofessional Education (IPE) Initiative: Structure, Process, Outcome Part 2
Assessment of IPE Initiative: Structure, Process, Outcome Part 2
Abby A. Kahaleh, BPharm, MS, PhD, MPH
January 21, 2016
Abby A. Kahaleh, BPharm, MS, PhD, MPH
Curriculum SIG Chair
1. Christine K O'Neil, PharmD, BCPS, CPG, FCCP
Professor, Director of Curricular Development and IPE
Mylan School of Pharmacy
2. Elena Umland, PharmD
Associate Dean for Academic Affairs
Professor of Pharmacy Practice
Jefferson College of Pharmacy
Thomas Jefferson University
3. Michelle Z. Farland, PharmD, BCPS, CDEClinical Associate ProfessorUniversity of Florida College of Pharmacy4. Jennifer Danielson, PharmD, MBA, CDE
Director of Interprofessional and Experiential Education & Assistant Professor University of Washington School of Pharmacy
Describe key structure of IPE programs
Review current IPE assessment tools
Explain critical steps in assessing IPE outcomes
Share "lessons learned" in developing, implementing, and assessing IPE
Christine K O'Neil, PharmD, BCPS, CPG, FCCP
Duquesne University Mylan
School of Pharmacy
Creating Interprofessional Education Experiences
Interprofessional education (IPE):
"When students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes. "
World Health Organization. Learning Together to Work Together for Health. Report of a WHO study group on multiprofessional education for health personnel: the team approach. http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf. Accessed August 25, 2014.
The CAPE 2013 Outcomes are embedded in ACPE Standards 2016 in Standards 1-4. These require programs to prepare students for collaborative roles.
Also Standards 2016 include a separate standard addressing IPE within the curriculum.
Specifically, Standard 11
IPE is well-received and is a conduit for enabling knowledge and skills necessary for collaborative work
IPE is less able to positively influence attitudes and perceptions towards others
Evidence of IPE effectiveness is limited
Evidence does suggest that an interprofessional approach improves quality and decreases cost of care.
Hammick M, Freeth D, Koppel I et al. A best evidence systematic review of interprofessional education. Med Teach 2007;29:735-51.
Reeves S, Zwarenstein M, Goldman J et al. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue1. Art. No.:CD002213. DOI:10.1002/14651858.CD002213.pub2.
Evidence for IPE
While IPE is well-received and is a pedagogical method for enabling knowledge and skills, it may be less able to positively inlfuence attitudes and behaviors.
Evidence of the impact of IPE on patient outcomes is limited, however there are 4 studies with positive impact on patient satisfaction, teamwork, error rates, mental health competencies or care outcomes.
Evidence does suggest that an interprofessional approach to care improves quality and decreases cost of care.
Interprofessional Education Collaborative (IPEC)
Expert panel consisting of representatives from:
Association of Colleges of Nursing, American Association of Colleges of Osteopathic Medicine, American Association of Colleges of Pharmacy, American Dental Education Association, Association of American Medical Colleges, and Association of Schools of Public Health.
Competencies are general and flexible enough to be used by any profession:
Domain 1 Value and Ethics for Interprofessional Practice
Domain 2 Roles and Responsibilities
Domain 3 Interprofessional Communication
Domain 4 Teams and Teamwork
IPE learning experiences should be linked to one or more of these competencies.
Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, D.C: Interprofessional Education Collaborative; 2011.
The development of core competencies approved by IPEC an expert panel will facilitate implementation of IPE across programs.
This is an essential tool for curriculum development of IPE experiences.
Of note, these competencies focus on development of IPE skills. The focus of IPE learning experiences is not primarily content.
Planning and teaching by an interprofessional mix of faculty
Link to a minimum of one learning objective (IPEC Competencies) Content relating to interprofessional competence is included and preferably threaded, throughout the course
At least one assignment that necessitates interprofessional group work Experiential courses may be designated as interprofessional.
Student participants in the course/experience represent at least two health professions.
Match students with equivalent levels of education for IPE experiences
IPE experience should be reflective of real practice connections between disciplines
IPE learning experiences should be optimized to achieve IPE outcomes for each of the programs involved and do not necessarily need to involve every health profession
Includes an assessment of growth in interprofessional competence.
Key Elements of IPE Courses & Experiences
IPEC domains link assures that that students, upon completion of the course/experience, will demonstrate an increase in level of interprofessional competence
Ideally, IPE experiences should be present in didactic, experiential, and co-curricular formats.
Present in each year of the professional curriculum to facilitate development of pharmacist as collaborators.
Incorporation of early IPE exposure in pre-professional programs is also an opportunity.
When planning for IPE, there are several considerations for implementation..see list
Health care ethics
Medication errors; patient safety topics
Care of the patient with chronic illnesses or those in special populations.
Buring SM, Bhushan A, Brazeau G, Conway S, Hansen L, Westberg S. Keys to successful implementation of interprofessional education: learning location, faculty development, and curricular themes. Am J Pharm Educ. 2009;73(4): Article 60.
Meyer SM, Garr DR, Evans C. Advancing Interprofessional Clinical Prevention and Population Health Education. Curriculum Development Guide for Health Professions Faculty. http://c.ymcdn.com/sites/www.aptrweb.org/resource/collection/245F0E9A-CA.... Accessed August 25, 2014.
There are some curricular themes that are very conducive to implementation of IPE experiences for students.
IPE is possible with any model of pharmacy education:
Fully integrated academic health center
Partially co-located program with pharmacy and other professions under a common university ownership
Partially co-located with pharmacy and other professions under different university components
Pharmacy with other health professions but no medical school
Pharmacy with no other health education programs on campus
More readily implemented in programs that have co-existing health professions schools
Partnerships with programs outside the school and creative teaching strategies with technology may facilitate IPE experiences in distant partnerships.
IPE is possible with any model of pharmacy education.
5 models have been identified see list.
While IPE is more readily implemented in programs with co-existing health professions schools, distant partnership may be created with programs outside of the school utilizing creative teaching strategies and technology.
Assessment of IPE is Essential
Reflection of what the student has gained
Other assessment tools:
Readiness for Interprofessional Learning Scale (RIPLS)
Parsell G, Bligh J. Med Educ. 1999;33:95-100
Interdisciplinary Education Perception Scale (IEPS)
Luecht RM et al. J Allied Health. 1990;19:181-91
Attitudes Toward Health Care Teams Scale
Heinemann GD et al. Eval Health Prof. 1999;22:123-42
42-item questionnaire developed using the IPEC competencies
Dow AW, DiazGranados D, Mazmanian PE, Retchin SM. An exploratory study of an assessment tool derived from the competencies of the interprofessional education collaborative. J Interprof Care. 2014 Jul;28(4):299-304. DOI:10.3109/13561820.2014.891573. Epub 2014 Mar 4.
Any IPE learning experiences must include an assessment of what the students has gained with respect to IPE competencies.
At minimum, this should be a reflection.
There are several other assessment tools that may be used.
In terms of programmatic outcomes, Dow and colleagues have published a tool based on the IPEC competencies.
Assessment Tools for IPE
Elena Umland, PharmDJefferson School of Pharmacy
Michelle Farland, PharmD, BCPS CDEUniversity of Florida College of Pharmacy
Kirkpatrick Assessment Model
Level 4b: Benefits to patients/clients
Improvements in health or well being of patients/clients
Level 4a: Change in organizational practice
Wider changes in the organization and delivery of care
Level 3: Behavioral change
Identifies individuals' transfer of IP learning to their practice and setting
Level 2b: Acquisition of knowledge & skills
Including knowledge and skills linked to IP collaboration
Level 2a: Modification of perceptions & attitudes
Changes in reciprocal attitudes or perceptions between participant groups, Changes in perception or attitude towards the value and/or use of team approaches to caring for a client
Level 1: Reaction
Learners' views on the learning experience and its IP nature
Need to be here
We are here
Danielson J, Willgerodt M. University of Washington Health Sciences. 2015
Students and/or faculty?
What are you evaluating
Tools to Assess Reaction
Readiness for interprofessional learning scale (RIPLS)
Evaluates the readiness of health professions students for IPE.
Early; prior to the start of any IPE programming.
Potential to repeat it to see improvement in readiness as initial distribution may serve as a baseline.
Interdisciplinary education perception scale (IEPS)
Assess student perceptions of interprofessional experiences.
Early with inaugural IPE programming. Consider repeating to evaluate changes in perceptions.
Note that 12- , 17- and 18-item scales may serve different purposes.
Tools to Assess Modifications of Perceptions & Attitudes
Student perceptions of physician-pharmacist interprofessional clinical education (SPICE)
Assesses pharmacy and medical student attitudes toward interprofessional clinical education.
As early as possible where interprofessional clinical education occurs. First year students following a required IPE activity or course.
Attitudes toward healthcare teams scale (ATHCT)
Assesses team member perceptions of the quality of care delivered by healthcare team; team member attitudes towards physician authority in teams and their control over information about patients.
As used to determine effect of interprofessional interventions, its placement at beginning and end of program. 20-item scale (versus 14-item scale) has benefit of evaluating domain of roles/responsibilities.
Tools to Assess Behavior Change
Interprofessional collaborator assessment rubric (ICAR)
Observational tool to assess learner achievement of interprofessional competency domains
APPEs. Faculty or preceptors observe students in interprofessional activities over time. Consider use at end of week 1, towards middle and at end of clinical rotation.
Interprofessional collaborative competency attainment survey (ICCAS)
Self-assessment of achievement of interprofessional care competencies
APPEs. At start and end of clinical rotation. Also consider pre- and post- other IPE activities as evaluates all 4 IPEC domains.
Tools to Assess Behavior Change
IPEC competency survey instrument
Self-assessment of the achievement of the competencies defined by the Interprofessional Education Collaborative expert panel
Conclusion of APPE. May be introduced early as formative assessment for pre-/post-assessment.
Collaborative practice assessment tool (CPAT)
Assesses the views of team members in a collaborative care team on elements of collaboration
Following a longitudinal experience (Pre-APPE or APPE).
Performance assessment for communication and teamwork tool set (PACT)
Observational tool to assess teams during a live simulated scenario
Team OSCE (pre-APPE or APPE)
Tools to Assess Change in Organizational Practice
Index for interdisciplinary collaboration (IIC)
Assesses aspects and levels of interprofessional collaboration within an organization
Individual & Team
Designed for a longitudinal work environment may be helpful with students completing numerous APPEs at the same institution or residency programs
Survey of organizational attributes of primary care (SOAPC)
Assesses healthcare providers' perceptions of resources available to make changes in the patient care process in primary care settings
Designed for longitudinal teams, may be helpful with students completing numerous APPEs at the same institution or residency programs
IPE Assessment Tool Application
Identify the outcomes you need/want to measure.
Review tools to identify those that best measure the outcomes and are feasible to implement.
Multiple evaluation methods will be needed
Quantitative & qualitative
Self-assessment & observation of behavior
Identification of outcomes will likely vary based on how far into implementation your IPE program is. Initially, assessing reaction may be the most important component, while more developed programs may not feel a need to assess reaction if IP collaboration is part of the organizational culture already.
Jennifer Danielson, PharmD, MBA, CDE
Director of Interprofessional Education and Experiential Education
University of Washington School of Pharmacy
Kahaleh A, Danielson J, Franson K, et al. AJPE 2015; 79 (1): Article 6.
Patient Care Outcomes
Population Health Outcomes
Danielson J, Willgerodt M. UW Health Sciences. 2015
UW SOP IPE Curriculum Framework
UW SOP Assessment Framework
Graduation = Team-based practice ready
Danielson J. University of Washington School of Pharmacy. 2015
What types of IPE programs have you implemented at your college/school of pharmacy?
What are some of the challenges/opportunities of your IPE program?
Which assessment tools have you used to evaluate students' performance outcomes?
What are your future plans for CQI?