University of Vermont

College of Medicine

Vermont Medicine Magazine

Multiple Choice: The World of Multiple Mini Interviews

Multiple Mini-Interviews are the latest evolution in selecting medical students at the College of Medicine

Medical school applicants read their topic question outside one of the eight interview stations they will visit during their day-long visit.
Medical school applicants read their topic question outside one of the eight interview stations they will visit during their day-long visit.

A cowbell clangs. The eager applicants have finished the allotted six minutes to expound on a particular topic. Now, they have two minutes to consider their next topic before they sit down with another interviewer and start talking. The cowbell, Vermont-style, will again tell them when to stop and move to a new topic with a new interviewer.

This is Interview Day at the UVM College of Medicine, and the prospective medical students are tackling the multiple mini-interview, or MMI. By the end of the process, they will complete nine highly focused six-minute interviews, covering topics that range from a controversial political issue to a dilemma with a co-worker.

For the last two years at UVM, the MMI has replaced the traditional medical school interview that gave the applicant 45 minutes with one person, often a current or former faculty member, after which the interviewer would then provide his or her evaluation to the full College admissions committee.

That format, though, involved unintentional but inherent unfairness, says Janice Gallant, M.D.’85, the College’s associate dean for admissions. With the single-interview format, one distracted remark, or a slight failure to “click” with the interviewer could ruin an applicant’s chances. Or the sole interviewer, who typically used to see the application file before the one-on-one meeting, might share a personal detail — an alma mater, hometown or beloved sports team — with the prospective student, making a favorable review more likely.

“It was a system that could be affected by unintentional bias,” says William Jeffries, Ph.D., the College’s senior associate dean for medical education. “The human tendency was that for people you would like, you would go and advocate for them in the committee.”

So, starting in 2014, the College switched to MMI, with the goal of diminishing levels of bias and gaining a better, deeper appraisal of the “core competencies” of applicants — areas of personal and professional aptitude that have been identified by extensive research by the Association of American Medical Colleges (AAMC).

Along with the MMI, Gallant and her staff revamped its admissions committee and procedures, and also instituted an interview day teamwork exercise that is unique among medical schools.

The core competencies encompass “soft” skills such as ethics, empathy and adaptability. Not only are those qualities difficult to measure in general, but research indicates that they are not always detected by a traditional lengthy single interviews.

“The personal interview has not been found to predict performance,” Gallant says. Studies have shown, however, that the MMI does correspond with a medical student’s likelihood of success in personal and professional areas.

“It’s a reliable, validated tool that we are using because it’s very compelling,” Gallant says. “The early assessment is that everyone is quite pleased by the results we’re seeing.”

***

UVM has joined early adopters of the MMI among medical schools. As of the 2014–2015 academic year, 30 AAMC member schools reported using the multiinterview method, or 21 percent of total members, says Geoff Young, Ph.D., the association’s senior director of student affairs and programs.

Preference for the MMI is increasing; just 15 schools, or 11 percent of total AAMC members, were using it in 2012. The trend reflects the goal of selecting candidates with those interpersonal strengths now recognized as important for nurturing modern doctors with a more holistic view, Young says.

“The community, I think, is better informed as we think about diversity and about the changing demographics of this country,” Young says. “The best and the brightest doesn’t mean they have the highest MCAT or the 4.0 GPA.”

The purpose of MMIs is not to determine whether applicants are smart enough for medical school. The grade point averages and Medical College Admission Test scores work fine to show whether prospective students can handle the science, the cognitive part. But they don’t predict success in the personal and professional areas, says Harold Reiter, M.D., a professor of oncology who helped create the MMI at McMaster University in Hamilton, Ontario, when he was admissions chair for what is now the Michael G. DeGroote School of Medicine.

Since McMaster became the first medical school to implement the MMI in 2004, Reiter’s and others’ research has shown the process is highly reliable as a forecast of a physician’s future performance. Medical students who do well with the MMI also tend to excel on national licensing exams, which in turn correspond with fewer complaints to medical regulatory boards, a high patient survival rate, and high regard by peers. The MMI doesn’t remove the built-in subjectivity of the interviewers, Reiter explains. The system accepts this bias but provides a broad enough variety of opinion — more interviews — to dilute the influence of it.

“What you’re gaining is multiple ‘biopsies’ of each person,” Reiter says. The College of Medicine now plans to start collecting data to analyze the effectiveness of its MMI and make sure it’s capturing the core competencies as expected, Gallant says. She and her staff have made presentations about the new process at several national and regional conferences.

“We’ve devised an entry system that assesses where applicants stand with core competencies,” Jeffries says. “We’re really not comparing people against each other. We’re comparing people against the standard that we’ve set.”

***

Interview Day starts at 8 a.m. on a Saturday morning in January swept by freezing rain. The 65 prospective students eat a light breakfast of yogurt and fruit and hear introductions in the Hoehl Gallery, adjacent to the Given Building. Gallant, sporting a white coat and a cheerful demeanor, explains to the assembled interviewees that the MMIs were designed “for you to tell us what you think about, what you care about” and to convey a “warm, welcoming and supportive environment” at UVM.

Four MMI “circuits” are set up in separate areas of Given. Each circuit has nine stations on this Interview Day, each focused on one topic or question and staffed by one interviewer. The topics each cover a core competency or combination of them: ethical responsibility; cultural competence; social skills, teamwork; oral communication; service orientation; reliability and dependability; capacity for improvement; and resilience and adaptability.

The College uses topics designed by ProFitHR, a company that spun off from McMaster to help school admissions offices develop their MMI. The questions are swapped every Interview Day and kept as secret as possible.

“This is highly confidential,” Gallant says. “This is like Wall Street.”

The 36 interviewers include faculty members, medical students and members of the community. In a fourth-floor lecture room, they gather to review the questions for the first time and coordinate scoring techniques.

During the MMI, the interviewers speak little. There’s not much give-andtake. They only ask follow-up questions as needed to prompt more information. “This is not a conversation,” Gallant tells them before the start. “This is not even a dialogue. This is more like a monologue.” In preparing for the MMI this year, interviewers were instructed to stay as neutral as possible in their expressions to avoid unintentional encouragement or disapproval; but the applicants gave negative feedback about these stoic encounters. Admissions staff has since loosened things up, allowing the interviewers more ease and expression. Allie Stickney, a community interviewer and retired CEO of retirement community Wake Robin, in Shelburne, Vt., says she appreciated that flexibility. It was distracting to concentrate on keeping her face blank, she says.

“It does turn the interviewing process upside-down,” Stickney says. “The interviewer is not really asking any questions. You’re really putting the ball in the student applicant’s court.”

The MMI is less relaxed than a long interview but also more revealing, Stickney says. Even in six minutes, the applicants share the “whole gestalt” of who they are, how they see the world.

“You can see their minds really working, working hard to pull on all parts of their brain” to answer the question, she says. “They have to pull on a lot of different parts of their experience.”

Interviewers cannot share their occupations, which might skew a candidate’s response or attitude. On this Interview Day, interviewer Francisco Grinberg, M.D., sees firsthand the reason for this.

For his question, intended to gauge service orientation, one applicant tells him that some doctors don’t need to communicate much with their patients — anesthesiologists, for example. Grinberg is a UVM professor of anesthesiology and a practicing anesthesiologist at UVM Medical Center.

His careful response to that prospective student: “You’d be surprised.”

***

While half the students move through the MMI circuits, the other half heads to the Teamwork Simulation. At small tables, they sit in groups of five or six and receive instructions from Shirley McAdam, coordinator of the standardized patient program at the UVM Clinical Simulation Laboratory.

The premise of one simulation: A freak accident has occurred during the International Potato Head Conference. Many are severely injured — broken arms, missing legs, dislodged lips — and the teams must take care of them. When they get the go-ahead, the applicants hurriedly assemble the Mr. Potato Head toys, attaching big feet, goofy ears and mustaches without a snicker or giggle. Near each table, a “rater” stands with a clipboard, watching the teams work and assessing their interaction.

The College developed the teamwork exercise after learning of a similar program used by the University of Massachusetts Memorial Medical Center for its staff. For more than a year, the College tested the simulation on its students, faculty and staff, says Director of Medical Student Admissions Cary Jewkes. The process helps to identify individuals who might not be ready for working in teams.

“We’re looking at how they communicate with each other,” Jewkes says. “It’s not about the number of toys they make but how they do it together.”

In teams, individual characteristics rise to the surface, particularly types who “can’t tamp down their overzealousness or bossiness” or extreme introverts who can’t engage with others, Jewkes says.

Back in the MMI areas, after the final cowbell rings, Gallant hosts “debriefing” sessions to hear the applicants’ thoughts about the questions and setup.

“As you might have heard about UVM, we’re very big on reflection, because that’s the way we all learn together,” she tells them.

A few share that they miss the personal connection of the single long interview. That was partly the impression of Allison Greene, a member of the College’s Class of 2019, one of the first groups of prospective students to go through the MMI.

But Greene says she now sees that the new format pushes candidates to think fast on their feet and get creative.

“It puts you in a situation that you haven’t been in before,” she says. “In that sense, it’s a measure of readiness for a program like medical school.”

In a similar debriefing later with the interviewers, they explain their strategies for scoring each question. One station had a “standardized patient” — with the interviewer acting as a person who has a problem and wants the applicant’s advice.

“What we were looking for were social skills for our core competency and empathy,” says Gayathri Prabhakar, an interviewer at that station and a second year UVM medical student. “The really exceptional applicants were able to validate his concerns.”

The admissions staff absorbs this information and continues to tweak the details. They’ve added amenities such as fresh flowers, water stations at each circuit and granola bars during the debriefings.

“Everything is very intentional,” Gallant says. “Everything is designed to create an environment for every applicant to be successful.”