Title: Question Improvement Suggestions
Course: Dev_TOPKIT_SAMPLE_COURSE_Bauer_S_67

10 Examples of Question Improvements

The first three samples are provided here from different academic disciplines, taken from: http://www.caacentre.ac.uk/dldocs/otghdout.pdf

Multiple Choice Questions

1. Present a single, definite statement to be completed or answered by one of the several given choices.

Biochemistry

- Weak Question: Polysaccharides are made up of thousands of smaller unites called monosaccharides

a. are NOT found in the aloe vera leaf

b. are created during photosynthesis

c. can be described by the chemical formula: CHHOH

d. none of the above

- Improved Question: Polysaccharides of the plant cell wall are synthesized mainly in

a. the endoplasmic reticulum

b. the cytosol

c. the plasma membrane

d. the Golgi complex

e. amyloplasts

In the top example, there is no sense from the stem what the question is asking. The second example more clearly identifies the question and offers the student a set of homogeneous choices.

1. Writing distracters: For single response MCQs, ensure that there is only one correct response.

Literature

A. Weak Question: Which of the following texts is considered to represent the pinnacle of modernist achievement?

a. The Waste Land

b. Middlemarch

c. Ode to Nightingale

d. Ulysses

e. Ethan Frome

B. Improved Question: Which of the following texts represents one of the high points of modernist achievement?

a. The Waste Land

b. Middlemarch

c. Ode to Nightingale

d. My Last Duchess

e. Ethan Frome

In the top example, both options "a" and "d" could be considered to be correct.

True/False Questions

1. In the following question, the student's clinical judgment is assessed.

Medicine

A 28 year old woman with one child has taken anti-thyroid drugs for 6 months for thyrotoxicosis. She has a friend who has been successfully treated with radio-iodine. She finds she frequently forgets to take her drugs and wants to stop them to have radio-iodine treatment.

a. She should be told that because of her age radio-iodine is best avoided.

b. The problems associated with radio-iodine should be discussed with her.

c. Surgery as a possible alternative should be discussed with her.

d. She should be advised that some form of further treatment is required.

e. You should find out more about her friends's treatment.

Note: This approach may be used for testing knowledge and judgement in many subjects. When grouped together, a series of true/false questions on a specific topic or scenario can test a more complex understanding of an issue. They can be structured to lead a student through a logical pathway (Brown 1997) as in the above example which simulates a medical diagnosis. Such questions may also be useful to the lecturer for diagnostic purposes, because they can revel part of the thinking process employed by the student in order to solve the given problem.

Examples from Harden, R.m. & Dunn, W.G. (1981) Assessment a Work Manual Dundee Centre of Medical Education in Brown et al 1997.

The following types of questions work well for eliciting responses for the specific levels of Bloom's Taxonomy.

Synthesis Questions

1. The following question formats can elicit responses from the Synthesis level.

  - Creating a unique, original product that may be in verbal form or may be a physical object
  - Combination of ideas to form a new whole
  - What would you predict/infer from...?
  - What ideas can you add to...?
  - How would you create/design a new...?
  - What might happen if you combined...?
  - What solutions would you suggest for...?

Evaluative Questions

1. These types of questions usually require sophisticated levels of cognitive and/or emotional judgment.

In attempting to answer evaluative questions, students may be combining multiple logical and/or affective thinking process, or comparative frameworks. Often an answer is analyzed at multiple levels and from different perspectives before the student arrives at newly synthesized information or conclusions.

- Examples:
- What are the similarities and differences between the deaths of Ophelia when compared to that of Juliet?
- What are the similarities and differences between Roman gladiatorial games and modern football?
- Why and how might the concept of Piagetian schema be related to the concepts presented in Jungian personality theory, and why might this be important to consider in teaching and learning?

Higher-Order Multiple Choice Questions

1. This is an example of a multiple-choice level for an Application outcome:

What problem exists with toe following multiple-choice stem:

is the most common type of test item.

a. Absolute words should be avoided in the stem.

b. The stem contains more than one idea of concept.

c. Not enough information is presented to answer the question.

d. The fill-in the blank should not come at the beginning.

1. This is an example of a multiple-choice level for an Analysis or Evaluation outcome:

An instructor was asked the following question: "Briefly list and explain how you develop a test."

As an answer, this instructor wrote the following:

I being by going through the chapter and writing questions based on the material in the text and my lectures. Then I decided how many questions I want and select the best questions from the list that I have developed. I format the test and add instructions.

After a few days, I review the questions and make any revisions that need to be done and remove any jargon or wording lifted directly from the text. Then I use the test in class. Based on how the class does, I may make changes for the next time I teach the class.

Based on the process described in Effective Tests, how would you judge this instructor's answer?

a. Excellent (all steps in the right order with correct, clear, and complete descriptions)

b. Good (all stages correct in the right order, but the descriptions are not as complete as they should be)

c. Mediocre (one or two stages are missing, OR the stages are in the wrong order, OR the explanations are not complete, OR the explanations are irrelevant)

d. Unacceptable (one or more stages are missing AND the explanations are not complete AND/OR they are irrelevant)

Application-based Multiple Choice Questions

1. This is an example of writing a multiple choice question to be assessed at the Application level (higher order).

Wilma and Fred were arguing over the furniture while they were settling their divorce agreement. Wilma said to Fred, "I never like that chair anyway." At which of Knapp's relational conversation levels is this statement?

a. Integrating

b. Differentiating

c. Sustaining

d. Initiating

This question requires a student to understand the relationship levels and apply a statement to that typology.

Synthesis Questions

1. The following example is related to the Synthesis level of Bloom's Taxonomy.

Student is provided with a vocabulary listing of important terms and concepts.

- Explain why it is likely that a matriarchal family system would be found in a matrilocal or matrilineal society.
- Which one of these combinations of society descriptions is unlikely to exist within one society:

a. neolocal, egalitarian, nuclear

b. patrilocal, patrilineal, patrilocal

c. bilateral, egalitarian, nuclear

d. extended family, conjugal family, kin

Problem-Solving Questions

1. Provide directions which clearly inform the student of the type of response called for.

    - Undesirable: An American tourist in Paris finds that he weighs 70 kilograms. When he left the United States he weighed 144 pounds. What was his net change in weight?
    - Desirable: An American tourist in Paris find that he weights 70 kilograms. When he left the United States he weighed 144 pounds. What was his net weight change in pounds?

TOPKit Sample Course created by Sue Bauer, John Raible, and Jessica Tojo. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.