“The ability to secure meaning in the course of our experience is a basic human need... But meaning is not simply found; it is constructed.”
A few days ago I participated in a Twitter chat led by a physician-educator from the United Kingdom. This Twitter chat (#ukmeded) is usually held on Thursdays at 9:00 pm UK time. This was a rich discussion on Twitter, and it also allowed me to participate in a discussion with people from other parts of the world. The most recent topic was on assessment (follow the link and learn more about the subtopics discussed). One of the interesting questions discussed was:
“Does assessment drive learning?”
According to the ACGME (2011), assessment is defined as an ongoing process of gathering and interpreting information about a learner’s knowledge, skills, and/or behavior. The ACGME also provides a document on the components of a formative evaluation for residents.
The importance of a formative assessment lies in its usefulness of helping learners and assessors to know where they are and what they need to do to improve. This type of assessment, when done impromptu and appropiately in the form of descriptive-feedback, must be timely, specific, and ideally correlative to the six competency domains. Assessments may also be spaced out at longer intervals, but even at these times it should also be specific as to what the learner’s performance level is and a description of how to improve.
Formative assessment has also been used in other settings to evaluate and modify the teacher’s effectiveness by the adult learner. Improvement is also more effective when the assessor focuses on the task performance and not the learner's ego. Dylan Wiliam, Ph.D. has researched formative assessment extensibly and describes two types of feedbacks:
- Ego-involving: Not useful to the learner
- Task-involving: Helpful to the learner
Summative Assessment
Summative assessments, when valid and reliable, are more appropriate to make high stakes decisions. Be aware that “data from the formative assessment should not be used to make high stakes decisions (promotion, graduation),” according to the ACGME. Benchmarks/standards are used to evaluate the learner, and it usually takes place at the end of the year. It is a good habit to be aware of what type of evaluation your residency program uses and its components so you know up to what standards you are being compared to. Adult learners for the most part are self-directed, self-evaluators, and should take a big part in the development of the curriculum where they can state what their needs are. This is a bit controversial due to the fact that evaluators who might be long removed from training lay out different curriculum needs than the adult learner’s.
The Bottom Line
When using formative and summative assessments, like everything else in medical training, it is important to factor in and ensure patient safety. These tools also offer an opportunity for the learner to improve on his/her practice of medicine. So to answer the original question posed by Dr. Natalie Lafferty (@nlafferty) and Dr. Anne Marie Cunningham (@amcunningham), it is my belief that formative assessment drives learning, while summative assessment does not. Summative assessment is what institutions use to tell the world their assessment about the learner.
- Formative assessment is FOR learning.
- Summative assessment is OF learning.
Image 1 source, Image 2 source
Related links:
Article review: Feedback in the Emergency Department
Paucis Verbis: Feedback Card
Not sure your conclusion is entirely correct.
ReplyDeleteEven though Osler perceived exams to be a stumbling block on the apth to true learning, he also recognised the ability of the final exam to sharpen the student's mind:
“I do not know of any stimulus so healthy as knowledge on the part of the student that he will receive an examination at the end of his course. It gives sharpness to his dissecting knife, heat to his Bunsen burner, a well worn appearance to his stethoscope, and a particular neatness to his bandaging.”
— William Osler, from an address to medical students at the University of Pennsylvania, 1885.
Formative assessments may not be as useful if they are not followed by a summative assessment at the end.
Chris
http://lifeinthefastlane.com/2011/04/lessons-from-osler-005-2/
Thanks for your comment Chris.
DeleteI think that is the process of preparing for that final evaluation that drives (external motivator) the learning and not the exam itself. The adult learner is constantly assessing his own knowledge and skills while training for that exam or procedure. The test itself does not drive learning because it lacks specific and sometimes even timely descriptive-feedback.
If there was an easy answer to the summative-formative conundrum we probably wouldn't need blogs like this :)
ReplyDeleteI think the key is in Chris's challenge and Javier's response "I think that is the process of preparing for that final evaluation that drives (external motivator) the learning and not the exam itself". The problem then becomes why would that final learning occur if it wasn't for the exam.
However painful a summative element maybe, in order for the adult learner to really engage in assessing their own skills is not a final end-game an ultimate necessity for effective formative processes.
This comment has been removed by the author.
ReplyDeleteI might be off, but this analogy illustrates my perspective as an M4:
DeleteSummative feedback : Formative feedback As block/shelf/step/in-service exams : Feedback from direct observation.
That is, summative feedback establishes that a student has reached a standardized level of competency; they know the facts, they understand the vocabulary, and they can describe certain types of pathology. Formative feedback, then, assesses how a student applies their knowledge; how well they discern relevant facts, can communicate findings, and interpret patient presentation.
If this is true, then I would argue
1) Summative feedback motivates the creation of a strong knowledge base through standardized tests, but a strong knowledge base then creates greater opportunities to receive formative feedback. Dr. Joshi applies the RIME assessment, ie a student who has advanced from a Reporter to an Interpreter level of knowledge creates more opportunities for meaningful formative feedback. Perhaps, we would not need the motivation of summative feedback if our motivation to gain knowledge was so that we could receive meaningful formative feedback?
2) I think the better question is how can we create a culture that fosters androgogy* in medical school and beyond. (*the development of adult learners, not the study of male sex hormones… ;)) “Adult learners for the most part are self-directed, self-evaluators, and should take a big part in the development of the curriculum where they can state what their needs are.”
3) I think simulation is a great way to establish controlled opportunities for formative feedback and promotes a culture of adult-learning.
This is a lecture which appears to be for faculty audience. It addresses the same question as the title of this blog post, in its own way. The lecture is by Cees van der Vleuten from University of Maastricht. I really like the diagram he uses to represent assessment activities, training activities, supporting activities, and evaluation in relation to time.
ReplyDelete"Moving assessment beyond the psychometric discourse"
https://vimeo.com/18949413