File 1 of 1 from Assessment of Interprofessional Education (IPE) Initiative: Structure, Process, Outcome Part 2

Assessment of Interprofessional Education (IPE) Initiative: Structure, Process, Outcome Part 2


Assessment of IPE Initiative: Structure, Process, Outcome Part 2
Moderated by:
Abby A. Kahaleh, BPharm, MS, PhD, MPH
January 21, 2016

Contact Information
Abby A. Kahaleh, BPharm, MS, PhD, MPH
Curriculum SIG Chair
847-330-4537 (Phone)
[email protected]

1. Christine K O'Neil, PharmD, BCPS, CPG, FCCP
Professor, Director of Curricular Development and IPE
Mylan School of Pharmacy
Duquesne University
2. Elena Umland, PharmD
Associate Dean for Academic Affairs
Professor of Pharmacy Practice
Jefferson College of Pharmacy
Thomas Jefferson University
3. Michelle Z. Farland, PharmD, BCPS, CDEClinical Associate Professor, University of Florida College of Pharmacy4. Jennifer Danielson, PharmD, MBA, CDE
Director of Interprofessional and Experiential Education & Assistant Professor University of Washington School of Pharmacy

Describe key structure of IPE programs

Review current IPE assessment tools

Explain critical steps in assessing IPE outcomes

Share "lessons learned" in developing, implementing, and assessing IPE

Presenter I
Christine K O'Neil, PharmD, BCPS, CPG, FCCP

Duquesne University Mylan
School of Pharmacy

Creating Interprofessional Education Experiences
Interprofessional education (IPE):

"When students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes. "
World Health Organization. Learning Together to Work Together for Health. Report of a WHO study group on multi-professional education for health personnel: the team approach. http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf. Accessed August 25, 2014.

The CAPE 2013 Outcomes are embedded in ACPE Standards 2016 in Standards 1-4. These require programs to prepare students for collaborative roles.

Also Standards 2016 include a separate standard addressing IPE within the curriculum.
Specifically, Standard 11


IPE is well-received and is a conduit for enabling knowledge and skills necessary for collaborative work
IPE is less able to positively influence attitudes and perceptions towards others
Evidence of IPE effectiveness is limited
Evidence does suggest that an interprofessional approach improves quality and decreases cost of care.

Hammick M, Freeth D, Koppel I et al. A best evidence systematic review of interprofessional education. Med Teach 2007;29:735-51.

Reeves S, Zwarenstein M, Goldman J et al. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue1. Art. No.:CD002213. DOI:10.1002/14651858.CD002213.pub2.
Evidence for IPE
While IPE is well-received and is a pedagogical method for enabling knowledge and skills, it may be less able to positively influence attitudes and behaviors.

Evidence of the impact of IPE on patient outcomes is limited, however there are 4 studies with positive impact on patient satisfaction, teamwork, error rates, mental health competencies or care outcomes.

Evidence does suggest that an interprofessional approach to care improves quality and decreases cost of care.

Interprofessional Education Collaborative (IPEC)
Expert panel consisting of representatives from:
Association of Colleges of Nursing, American Association of Colleges of Osteopathic Medicine, American Association of Colleges of Pharmacy, American Dental Education Association, Association of American Medical Colleges, and Association of Schools of Public Health.
Competencies are general and flexible enough to be used by any profession:
Domain 1 Value and Ethics for Inter-professional Practice
Domain 2 Roles and Responsibilities
Domain 3 Inter-professional Communication
Domain 4 Teams and Teamwork
IPE learning experiences should be linked to one or more of these competencies.
Inter-professional Education Collaborative Expert Panel. Core Competencies for Inter-professional Collaborative Practice: Report of an Expert Panel. Washington, D.C: Inter-professional Education Collaborative; 2011.
The development of core competencies approved by IPEC an expert panel will facilitate implementation of IPE across programs.

This is an essential tool for curriculum development of IPE experiences.

Of note, these competencies focus on development of IPE skills. The focus of IPE learning experiences is not primarily content.

Planning and teaching by an inter-professional mix of faculty

Link to a minimum of one learning objective (IPEC Competencies) Content relating to inter-professional competence is included and preferably threaded, throughout the course

At least one assignment that necessitates inter-professional group work Experiential courses may be designated as inter-professional.

Student participants in the course/experience represent at least two health professions.
Match students with equivalent levels of education for IPE experiences
IPE experience should be reflective of real practice connections between disciplines
IPE learning experiences should be optimized to achieve IPE outcomes for each of the programs involved and do not necessarily need to involve every health profession
Includes an assessment of growth in inter-professional competence.
Key Elements of IPE Courses & Experiences
IPEC domains link assures that that students, upon completion of the course/experience, will demonstrate an increase in level of inter-professional competence


Curricular Considerations
Ideally, IPE experiences should be present in didactic, experiential, and co-curricular formats.
Present in each year of the professional curriculum to facilitate development of pharmacist as collaborators.
Incorporation of early IPE exposure in pre-professional programs is also an opportunity.

When planning for IPE, there are several considerations for implementation..see list

Curricular Themes
Communication skills
Health care ethics
Treatment adherence
Public health
Emergency preparedness
Medication errors; patient safety topics
Care of the patient with chronic illnesses or those in special populations.

Buring SM, Bhushan A, Brazeau G, Conway S, Hansen L, Westberg S. Keys to successful implementation of inter-professional education: learning location, faculty development, and curricular themes. Am J Pharm Educ. 2009;73(4): Article 60.

Meyer SM, Garr DR, Evans C. Advancing Inter-professional Clinical Prevention and Population Health Education. Curriculum Development Guide for Health Professions Faculty. http://c.ymcdn.com/sites/www.aptrweb.org/resource/collection/245F0E9A-CA.... Accessed August 25, 2014.

There are some curricular themes that are very conducive to implementation of IPE experiences for students.

IPE is possible with any model of pharmacy education:

Fully integrated academic health center
Partially co-located program with pharmacy and other professions under a common university ownership
Partially co-located with pharmacy and other professions under different university components
Pharmacy with other health professions but no medical school
Pharmacy with no other health education programs on campus

More readily implemented in programs that have co-existing health professions schools

Partnerships with programs outside the school and creative teaching strategies with technology may facilitate IPE experiences in distant partnerships.
Implementing IPE:Models
IPE is possible with any model of pharmacy education.

5 models have been identified see list.

While IPE is more readily implemented in programs with co-existing health professions schools, distant partnership may be created with programs outside of the school utilizing creative teaching strategies and technology.


Assessment of IPE is Essential
Reflection of what the student has gained
Other assessment tools:
Readiness for Inter-professional Learning Scale (RIPLS)
Parsell G, Bligh J. Med Educ. 1999;33:95-100
Interdisciplinary Education Perception Scale (IEPS)
Luecht RM et al. J Allied Health. 1990;19:181-91
Attitudes Toward Health Care Teams Scale
Heinemann GD et al. Eval Health Prof. 1999;22:123-42
Programmatic Outcomes
42-item questionnaire developed using the IPEC competencies

Dow AW, DiazGranados D, Mazmanian PE, Retchin SM. An exploratory study of an assessment tool derived from the competencies of the inter-professional education collaborative. J Inter-prof Care. 2014 Jul;28(4):299-304. DOI:10.3109/13561820.2014.891573. Epub 2014 Mar 4.

Any IPE learning experiences must include an assessment of what the students has gained with respect to IPE competencies.

At minimum, this should be a reflection.

There are several other assessment tools that may be used.

In terms of programmatic outcomes, Dow and colleagues have published a tool based on the IPEC competencies.


Assessment Tools for IPE
Elena Umland, PharmD, Jefferson School of Pharmacy

Michelle Farland, PharmD, BCPS, CDE, University of Florida College of Pharmacy
Reference: https://nexusipe.org/advancing/measurement-instruments

Kirkpatrick Assessment Model
Level 4b: Benefits to patients/clients
Improvements in health or well being of patients/clients
Level 4a: Change in organizational practice
Wider changes in the organization and delivery of care
Level 3: Behavioral change
Identifies individuals' transfer of IP learning to their practice and setting
Level 2b: Acquisition of knowledge & skills
Including knowledge and skills linked to IP collaboration
Level 2a: Modification of perceptions & attitudes
Changes in reciprocal attitudes or perceptions between participant groups, Changes in perception or attitude towards the value and/or use of team approaches to caring for a client
Level 1: Reaction
Learners' views on the learning experience and its IP nature
Culture Change
Need to be here
We are here
Danielson J, Willgerodt M. University of Washington Health Sciences. 2015
Students and/or faculty?
What are you evaluating

Tools to Assess Reaction
Tool Name
Brief Description
Team Assessment
Curricular Location
Readiness for inter-professional learning scale (RIPLS)
Evaluates the readiness of health professions students for IPE.
Early; prior to the start of any IPE programming.
Potential to repeat it to see improvement in readiness as initial distribution may serve as a baseline.
Interdisciplinary education perception scale (IEPS)
Assess student perceptions of inter-professional experiences.
Early with inaugural IPE programming. Consider repeating to evaluate changes in perceptions.
Note that 12- , 17- and 18-item scales may serve different purposes.

Tools to Assess Modifications of Perceptions & Attitudes
Tool Name
Brief Description
Team Assessment
Curricular Location
Student perceptions of physician-pharmacist inter-professional clinical education (SPICE)
Assesses pharmacy and medical student attitudes toward inter-professional clinical education.
As early as possible where inter-professional clinical education occurs. First year students following a required IPE activity or course.
Attitudes toward healthcare teams scale (ATHCT)
Assesses team member perceptions of the quality of care delivered by healthcare team; team member attitudes towards physician authority in teams and their control over information about patients.
As used to determine effect of inter-professional interventions, its placement at beginning and end of program. 20-item scale (versus 14-item scale) has benefit of evaluating domain of roles/responsibilities.

Tools to Assess Behavior Change
Tool Name
Brief Description
Team Assessment
Curricular Location
Inter-professional collaborator assessment rubric (ICAR)
Observational tool to assess learner achievement of inter-professional competency domains
APPEs. Faculty or preceptors observe students in inter-professional activities over time. Consider use at end of week 1, towards middle and at end of clinical rotation.
Inter-professional collaborative competency attainment survey (ICCAS)
Self-assessment of achievement of inter-professional care competencies
APPEs. At start and end of clinical rotation. Also consider pre and post other IPE activities as evaluates all 4 IPEC domains.

Tools to Assess Behavior Change
Tool Name
Brief Description
Individual/Team Assessment
Curricular Location
IPEC competency survey instrument
Self-assessment of the achievement of the competencies defined by the Inter-professional Education Collaborative expert panel
Conclusion of APPE. May be introduced early as formative assessment for pre-/post-assessment.
Collaborative practice assessment tool (CPAT)
Assesses the views of team members in a collaborative care team on elements of collaboration
Following a longitudinal experience (Pre-APPE or APPE).
Performance assessment for communication and teamwork tool set (PACT)
Observational tool to assess teams during a live simulated scenario
Team OSCE (pre-APPE or APPE)

Tools to Assess Change in Organizational Practice
Tool Name
Brief Description
Individual/Team Assessment
Curricular Location
Index for interdisciplinary collaboration (IIC)
Assesses aspects and levels of interprofessional collaboration within an organization
Individual & Team
Designed for a longitudinal work environment may be helpful with students completing numerous APPEs at the same institution or residency programs
Survey of organizational attributes of primary care (SOAPC)
Assesses healthcare providers' perceptions of resources available to make changes in the patient care process in primary care settings
Designed for longitudinal teams, may be helpful with students completing numerous APPEs at the same institution or residency programs

IPE Assessment Tool Application
Identify the outcomes you need/want to measure.
Review tools to identify those that best measure the outcomes and are feasible to implement.
Multiple evaluation methods will be needed
Quantitative & qualitative
Self-assessment & observation of behavior

Identification of outcomes will likely vary based on how far into implementation your IPE program is. Initially, assessing reaction may be the most important component, while more developed programs may not feel a need to assess reaction if IP collaboration is part of the organizational culture already.

Presenter IV
Jennifer Danielson, PharmD, MBA, CDE

Director of Inter-professional Education and Experiential Education
Assistant Professor
University of Washington School of Pharmacy

Kahaleh A, Danielson J, Franson K, et al. AJPE 2015; 79 (1): Article 6.


Patient Care Outcomes
Population Health Outcomes
Service Learning
Project-Based Learning
Student Growth
Case Exercises
Active Learning
Danielson J, Willgerodt M. UW Health Sciences. 2015
UW SOP IPE Curriculum Framework

UW SOP Assessment Framework

Attitudes Knowledge
Skills Behaviors
Practice Change
Graduation = Team-based practice ready
Danielson J. University of Washington School of Pharmacy. 2015



Discussion Questions
What types of IPE programs have you implemented at your college/school of pharmacy?
What are some of the challenges/opportunities of your IPE program?
Which assessment tools have you used to evaluate students' performance outcomes?
What are your future plans for CQI?