ACCME Criteria Review: Analyzing Change

Whether you are an accredited provider such as i3 Health or you are an educational partner working with an accredited provider, it's still important to understand the criteria that underly continuing medical education (CME) activities. This is part of a periodic series of posts looking at what each criterion means in general and what it might mean for you.

Today we finish working through the Educational Planning and Evaluation criteria. We have identified the educational needs underlying the activity, what the activity is designed to change, an appropriate format, and how the activity fits into the larger competencies that physicians should meet. So how did we do? The final criterion in this section focuses on how we assess or evaluate an activity and an entire CME program.

This criterion states the following:

The provider analyzes changes in learners (competence, performance, or patient outcomes) achieved as a result of the overall program's activities/educational interventions.

Before we get into the steps needed to meet this criterion, let's start with a vocabulary review. In CME, we talk about activities and programs. In standard American English, those two words could be used interchangeably, but in CME, they are different things. An activity is a single educational intervention. A program is made up of all of activities for which an accreditor provides credit. If you look at i3 Health's website, you'll see several different CME activities that you could attend. If you put all of those activities together, you can see i3 Health's entire CME program. You'll note that this criterion uses both words.

There are two steps to this criterion. We need to evaluate each activity, and then we need to use those evaluations to analyze the full CME program. If you're not an accredited provider, then you don't need to worry about that second part, at least as far as accreditation is concerned. Your activity becomes part of your accrediting partner's CME program, so your partner is responsible for including your activity in their overall assessment. But if you do have a large portfolio of educational activities, it's still a good practice to perform a regular assessment of the impact of the activities you create.

But let's focus on that first part, the activity-level section. When evaluating an activity, we aren't looking for how much people liked it. Sure, it's useful to know if the food was good so that you know what to order (or maybe not order) next time, but that doesn't tell us anything about the impact the activity had. What you really want to know is what your learners got out of the activity. What are they taking back to their practice?Do they have new skills or strategies to use? Are they going to change how they interact with their patients? Are their patients going to have improved health outcomes? That's how we know if an educational activity was effective.

So what is it you should ask? If you're working with an accredited provider, they probably have some standardized questions to meet this criterion. But if you're writing your own evaluation form, go back to those early stages in your activity planning. What are the educational needs the activity is designed to address? Is this designed to change patient outcomes? If not, then you don't need to ask about patient outcomes. But whether you ask about changes in competence, performance, or patient outcomes, the key is to make sure that you're assessing learner change and not merely learner happiness.

To meet the second part of this criterion, you can take all of your evaluations from all of your activities and compile the results of the question or questions you asked about learner change. That will give you a picture of your overall program. To come full circle, this data can then be used when assessing whether or not you're meeting your mission, or the Program Analysis criterion.

Hopefully this series has helped you see how the accreditation criteria build on each other and how they can guide you through the process of creating a CME activity. While this is the last of the core accreditation criteria, this series will continue with a look at the commendation criteria, those best practices that we should strive to meet with our programs.