Multiple choice exams: A) Less rigorous B) A model for future assessment

Will moving to online multiple choice exams be

A) Less rigorous

B) A model for future assessment

This time last year, following the departure of Y11, we sat down as a department to discuss a blog by @Xris32 on using multiple choice or short comprehension questions to assess reading before students complete an extended piece of writing for their end of term assessments.

This year, we have embedded the strategy across both language and literature KS3 assessments, and often in lessons when reading new texts, to assess students’ levels of understanding. 

During the end of year exam season, as English teachers, we would always look on with envy at departments that were able to use multiple choice questions or short mark questions as we sat staring at our piles of assessments that never seemed to end. However, due to online learning, this year we have thought deeply about how we can best carry out our end of year assessments at home. 

Students (and teachers!) have become adept at using Google Forms to set lessons. Over time, we have come to the shared conclusion that this is the easiest way for our students to complete tasks. Often students, who may be using a tablet or phone, struggled to switch between an extended video, extract and Google doc. As a result, completion rates were lower and the amount that students wrote was limited. Having seen the effectiveness of the Oak National Academy lessons, we found that short videos, followed by rigorous multiple choice and short answer questions worked well. Students could move through the various stages of the lesson and then were prepared to write a more extended answer at the end of the Google Form.

Levels of engagement with online learning is high, however, we know that the quality of students’ work can often fall short of the quality of their work produced in class. We knew that we could potentially be marking and grading lots of pieces of work that would not match what we knew to be a student’s ability. Yet, we also wanted to get an accurate snapshot of where students are at following a term of home learning. 

For English teachers, writing good multiple choice questions was not a skill that we had used very often. Usually, for English teachers, it is often the mark scheme that determines the difficulty of the task and the quality of an answer rather than the question itself. At our department meeting we read an extract from Paul Bambrick-Santoyo’s brilliant ‘Leverage Leadership 2.0’ - a book that forms the basis of our ‘Leverage Mentoring’ programme within the school.

In this chapter, Bambrick-Santoyo handily uses a Macbeth example to challenge the assumption that multiple choice questions, when done well, are ‘unrigorous’. 

The example he gives is below:

In this scene, Macbeth is discussing a prophecy he received from the witches. Read the passage, then answer the questions that follow:

Macbeth: 

They hailed him (Banquo) father to a line of kings 

Upon my head they placed a fruitless crown,

And put a barren sceptre in my gripe,

Thence to be wrench'd with an unlineal hand, 

No son of mine succeeding. If 't be so, 

For Banquo's issue have I filed my mind;

For them the gracious Duncan have I murder'd

1. The description of Macbeth's "barren sceptre" contributes to the unity of the passage in which of the following ways?

A. As a parallel between Macbeth's possible children and Banquo's possible children

B. As a satirical comment on challenges Macbeth will face with infertility

C. As a comparison between Macbeth's strong formal authority and his lack of

popular influence.

D. As an ironic contrast between Macbeth's power and his inability to produce future kings. 

The chapter argues that to get to the correct answer (D), students need to be able to combine ‘a knowledge of vocabulary in context with the larger connotative and figurative meaning of the phrases’. The mix of possible answers means that not only do they have to accurately understand the phrase ‘barren sceptre’ (B)  but they also have to identify that this is not a work of satire to arrive at D. Ultimately, Bambrick-Santoyo argues that multiple choice questions can add value. In contrast to open ended questions that require students to create their own thesis, questions, done well, can force students to evaluate a number of viable theses and select the most relevant options. 

Having read this, we felt that a combination of both would be beneficial for the end of year exams. When creating questions Bambrick-Santoyo argues that ‘the options and text difficulty determine the rigour’ and for open ended questions ‘it is not the prompt but the rubric that determines the rigour.’ In year team groups, we crafted 30 questions for the language papers and 10 for the literature paper.

We discussed ways that we could make questions progressively harder to enable us to assess all abilities. We considered how we could:

• Begin by focusing on smaller paragraphs and focus on comprehension and meaning
• Build students’ confidence with simpler questions that focused on identifying facts from the text 
• Begin to focus on tone / use more difficult vocabulary to test students’ understanding
• Design questions where students select the best possible quotation to match the point (and select the most viable one)
•  Use evaluation questions with statements where students have to weigh up the best options
• Include language analysis questions where students determine the most appropriate reading of a quotation
• Test vocabulary by asking students to give the most accurate definition of a word in the context of the sentence or determine which quotation best matches the meaning of the word.
• Finish with questions that make comparisons between sources and assess students understanding of the nuances in the differences

For language , we used sources from old KS3 AQA papers as the reading sources for language exams. We don’t usually use these, but they were good texts, suitable for reading ages and fitted into the topics we were working on (Y7 ‘Dalmatians’ sources linked to Y7 Animal Poetry and Y8 ‘Living Dolls’ linked to Y8 relationship poetry with a specific focus on parent and child relationships). This year, the whole exam is multiple choice. For the KS3 literature exams, we used a combination of both. We know that students will have their notes with them and need to acknowledge this fact.  Students will work through 10 multiple choice questions to assess their understanding and meaning of the poem and the writer’s intentions. They will then respond to a typical exam style question to give their response to the poem.   

It is by no means a perfect solution to assessment, or our favoured choice, but perhaps we may begin to change our minds on the power of multiple choice. I’d love to hear how other schools have experimented with online assessment and how it has worked for you. 

We will await the results next week!

Literature example

Language