Skip to content.

Every Decision Counts

Sections
Personal tools
You are here: Home

Every Decision Counts

Document Actions
Making machine-graded multiple choice tests more powerful by allowing multiple marks and introducing partial credit and "multiple decision" questions.

Most bubble sheet scanners will accept, interpret, and record multiple marks for each question. This, in turn, lets students earn partial credit for multiple-choice questions, even when there is only one correct answer. It also allows instructors to ask questions with the possibility of having more than one mark in the correct answer.

A scoring rubric for two types of questions, along with a description of each type, is found below.

TYPE PC: Partial credit on standard multiple-choice questions.

The standard multiple-choice (MC) question has exactly one correct choice and 4 incorrect choices. Even if students are not sure which one is correct, they might be able to eliminate one or more of the choices. This means they can earn partial credit. The more bubbles they fill in, the less credit they receive, but at least they are able to earn some points without resorting to guessing. The table below shows how points are earned for "partial-credit" MC questions.

Points earned on a
Standard MC question
(exactly 1 correct choice)
  Correct choice
marked?
Number
of marks
   Yes       No   
1 5 0
2 4 -1
3 3 -2
4 2 -3
5 1 [n/a]

Note that if students fill in all 5 bubbles they are guaranteed 1 point (out of 5).

Note also that students earn negative points if they fill in more than one bubble and none of them is the correct choice. The primary reason for doing this is to guarantee that if they are "just guessing", the average score is 1 point. In other words, if students are guessing on 5 questions and fill in one bubble each, they expect to get one of them correct. This statistical average is now true whether they fill in 1, 2, 3, or 4 bubbles, which makes it impossible to use gaming theory to maximize one's score. This feature also discourages guessing.

TYPE MD: Multiple-Decision questions.

Some questions are labeled with the phrase, "Mark ALL that apply", which means that possibly more than 1 mark must be made in order to receive full credit. This type of question is really a series of Yes/No questions. Students fill in the bubble if the given choice fits the desired criteria, and they leave it blank if it does not. In other words, students must make multiple decisions in order to answer the question. For instance:

Question 1.Which of the following
numbers is odd?
(Mark ALL that apply.)
 (A) 2
 (B) 3
 (C) 5
 (D) 20
 (E) 25

The correct answer has three marks (B, C, and E). If students answer correctly, they are actually answering 5 questions correctly, because they have left 2 bubbles blank. (And that's why we call this format "Every Decision Counts".) Each decision is worth 1 point, for a total of 5 points if all are made correctly.

If students fill in bubble (A) for this question, they would earn 1 point, because they (correctly) left bubble (D) blank. In other words, they have made 4 incorrect decisions: Filling in (A), and not filling in (B), (C), and (E). But they still earn 1 point for not filling in (D).

If students fill in all 5 bubbles for any MD question, they receive a score of 2 points.

Some MD questions have only 1 mark in the correct answer. This does not mean it is a standard MC question. Consider the question below.

Question 2.Which of the following
numbers is odd?
(Mark ALL that apply.)
 (A) 2
 (B) 3
 (C) 4
 (D) 20
 (E) 30

The correct answer is (B). If students answer (B), they will earn 5 points. If they answer (AB), they will earn 4 points (i.e., one incorrect decision). If they answer (A), they will earn 3 points, because they have made 3 correct decisions, i.e., they did not mark (C), (D), and (E). In other words, students earn only 1 point for marking (B); they earn the other 4 points for not marking (A), (C), (D), and (E).

Note that the identical question without the phrase "Mark ALL that apply" is a Standard MC question, which means if students answer (A), they will earn 0 points, not 3. The reasoning here is that on a Standard MC question, students know beforehand that there is exactly one mark in the correct answer, which is a huge hint. If students answer incorrectly, they deserve no points. But in an MD question, students do not know how many bubbles should be marked. It might be only one, but they do not know this beforehand. Therefore, they deserve credit for not marking those that should not be marked.

Some questions have only 4 "decisions" followed by "(E) None of the above". It is worth a maximum of 4 points, because there are only 4 decisions being made. If all 4 "decisions" are false, then students simply fill in bubble E. An example is below.

Question 3.Which of the following
numbers is odd?
(Mark ALL that apply.)
 (A) 2
 (B) 3
 (C) 6
 (D) 20
 (E) None of the above

The correct answer is again (B), except now, the maximum score is only 4 points. Filling in (E) earns students 3 points, because they have made 3 correct decisions. Filling in (BE) earns them 3.5 points, i.e., the average of (B) and (E). Filling in (AE) earns them 2.5 points, i.e., the average of (A) and (E).

If students are just guessing, they are encouraged to fill in all 5 bubbles. They earn exactly 2 points for this choice.


So, whether a question is earning partial credit on an old-fashioned, standard multiple-choice question or a "Mark ALL that apply" multiple-decision question, they are all truly Every Decision Counts. In all cases, students must decide exactly which bubbles to fill in and which to leave blank. Their scores depend on it!

 

© 2004–2007 Bill Leonard, University of Massachusetts Physics Education Research Group
Log in
« July 2008 »
Su Mo Tu We Th Fr Sa
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
 
 

Powered by Plone

This site conforms to the following standards: