University of Kansas
Center for Teaching Excellence
“Advancing the knowledge and practice of our profession”
|
University of KansasCenter for Teaching Excellence“Advancing the knowledge and practice of our profession” |
Return to Teaching Tips
In the discussion of limited-choice items below, the term stem is used to refer to the part of the item that asks the question. The terms responses, choices, and alternatives are used to refer to the parts of the item that will be used to answer the question. For example:
Stem: Who is the author of Jane Eyre?
Responses:
Multiple-choice items are considered to be among the most versatile of all item types. They can be used to test factual recall as well as level of understanding and ability to apply learning. Multiple-choice items can also provide an excellent basis for post test discussion, especially if the discussion addresses why the incorrect responses were wrong as well as why the correct responses were right. Unfortunately, they are difficult and time consuming to construct well. They may also appear too discriminating (picky) to students, especially when the alternatives are well constructed, and are open to misinterpretation by students who read more into questions than is there.
Some effective practices in constructing multiple-choice items include:
1. Using the stem to present the problem or question as clearly as possible.
2. Using direct questions rather than incomplete statements for the stem.
3. Including as much of the item as possible in the stem so that alternatives can be kept brief.
4. In testing for definitions, using the term in the stem rather than as an option.
5. Listing alternatives on separate lines (rather than including the options as part of the stem) so that all options can be clearly distinguished.
6. Keeping all alternatives in a similar format (i.e., all phrases, all sentences, etc.).
7. Making sure that all options are plausible responses to the stem. (Poor alternatives should not be included just for the sake of having more options.)
8. Checking to see that all choices are grammatically consistent with the stem.
9. Trying to make alternatives for an item approximately the same length. (Making the correct response consistently longer is a common error.)
10. Using misconceptions students have indicated in class or errors commonly made by students in the class as the basis for incorrect alternatives.
11. Using "all of the above" and "none of the above" sparingly since these alternatives are often chosen on the basis of incomplete knowledge.
12. Using capital letters (A,B,C,D,E) as response signs rather than lower case letters ("a" gets confused with "d" and "c" with "e" if the type or duplication is poor).
13. Trying to write items with equal numbers of alternatives to avoid asking students to continually adjust to a new pattern caused by different numbers.
14. Putting the incomplete part of the sentence at the end rather than the beginning of the stem when using a statement rather than a direct question.
15. Using negatively stated items sparingly. (When they are used, it helps to underline or otherwise visually emphasize the negative word.)
16. Making sure that there is only one best or correct response to the stem.
17. Keeping the number of alternatives at five or less. (The more alternatives used, the lower the probability of getting the correct answer by guessing. Beyond five alternatives, however, confusion and poor alternatives are likely.)
18. Randomly distributing correct responses among the alternatives positions so that there are no discernible patterns to the answer sequence (ABBABBABB, etc.) and a nearly equal proportion of As, Bs, Cs, etc.
True/false items are relatively easy to prepare since each item comes rather directly from the content. The offer the instructor the opportunity to write questions that cover more content than most other item types since students can respond to so many in the time allowed. They are easy to score accurately and quickly. True/false items, however, may not give a true estimate of the students' knowledge since half can be answered correctly simply by chance. They are very poor for diagnosing student strengths and weaknesses and are generally considered to be "tricky" by students. Since true/false questions tend to be either extremely easy or extremely difficult, they do not discriminate between students of varying ability as well as other types of questions do.
Some effective practices in constructing true/false items include:
1. Keeping language as simple and clear as possible.
2. Using a relatively large number of items (75 or more when the entire test is T/F).
3. Avoiding taking statements verbatim from the text.
4. Being aware that extremely long or complicated statements will test reading skill rather than content knowledge.
5. Requiring students to circle or underline a typed "r" or "F" rather than to fill in a "r" or "F" next to the statement, thus avoiding having to interpret confusing handwriting.
6. Avoiding the use of negatives, especially double negatives.
7. Avoiding ambiguous and trick items.
8. Making sure that the statements used are entirely true or entirely false. (Partially or marginally true or false statements cause unnecessary ambiguity.)
9. Using certain key words sparingly since they tip students off to the correct answers. (The words all, always, never, every, none, and only usually indicate a false statement, where as the words generally, sometimes, usually, maybe and often are frequently used in true statements.)
10. Using precise terms, such as 50 % of the time, rather than less precise terms, such as several, seldom, and frequently.
11. Using more false than true items, but not more than 15% more. (False items tend to discriminate more than true items.)
Matching items are generally quite brief and uninvolved and are especially suitable for who, what, when, and where questions. They can, however, be used to have students discriminate among and apply concepts. They permit efficient use of space when there are a number of similar types of information to be tested. They are easy to score accurately and quickly. Among the drawbacks of matching items are that they are difficult to use to measure learning beyond recognition of basic factual knowledge, they are usually poor for diagnosing student strengths and weaknesses, they are appropriate in only a limited number of situations, and they are difficult to construct since parallel information is required.
Effective practices in constructing matching items include:
1. Using only homogeneous material in a set of matching items (i.e, dates and places should not be in the same set).
2. Using the more involved expressions in the stem and keeping the responses short and simple.
3. Supplying directions that clearly state the basis for the matching, indicating whether or not a response can be used more than once, and stating where the answer should be placed.
4. Making sure that there are never multiple correct responses for one stem (although a response may be used as the correct answer for more than one stem).
5. Avoiding giving inadvertent grammatical clues to the correct response.
6. Arranging items in the response column in some logical order--alphabetical, numerical, chronological--so that students can find them easily.
7. Avoiding breaking a set of items (stems and responses) over two pages.
8. Using no more than 15 items in one set.
9. Providing more responses than stems to make process-of-elimination guessing less effective.
10. Numbering each stem for ease in later discussions.
11. Using capital letters for the response signs rather than lower-case letters.
Completion items are especially useful in assessing mastery of factual information when a specific word or phrase is important to know.They preclude the kind of guessing that is possible on limited-choice items since they require a definite response rather than simple recognition of the correct answer. Because only a short answer is required, their use on a test can enable a wide sampling of content. Completion items, however, tend to test only rote, repetitive responses and may encourage a fragmented study style since memorization of bits and pieces will result in higher scores. They are more difficult to score than forced-choice items and scoring often must be done by the test writer since more than one answer may have to be considered correct. On the whole, they have little advantage over other item types unless the need for specific recall is essential.
Effective practices for writing completion items include:
1. Using original questions rather than taking questions directly from the text.
2. Providing clear and concise cues about the expected response in the statement.
3. Using vocabulary and phrasing that comes from the text or class presentation.
4. When possible, providing explicit directions as to what amount of variation will be accepted in the answers.
5. Giving much more credit for completions than for T/F or matching items.
6. Avoiding using a long quote with multiple blanks to complete.
7. Requiring only one word or phrase in each blank.
8. Facilitating scoring by having the students write their responses on lines arranged in a column to the left of the items.
9. Asking students to fill in only important terms or expressions.
10. Avoiding providing grammatical clues to the correct answer by using a /an, etc., instead of specific modifiers.
The main advantages of essay and short answer items are that they encourage students to strive toward understanding a concept as an integrated whole, permit students to demonstrate achievement of such higher level objectives as analyzing given conditions and critical thinking, allow expression of both breadth and depth of learning, and encourage originality, creativity, and divergent thinking. Written items offer students the opportunity to use their own judgment, writing styles, and vocabularies. They are less time consuming to prepare than any other item type. Unfortunately, tests consisting only of written items permit only a limited sampling of content learning due to the time required for students to respond. Essay items are not efficient for assessing knowledge of basic facts and provide students more opportunity for bluffing, rambling, and "snowing" than limited-choice items. They favor students who possess good writing skills and neatness and are pitfalls for students who tend to go off on tangents or misunderstand the main point of the question. The main disadvantage, however, is that essay items are very difficult and time consuming to score and potentially subject to biased and unreliable scoring.
Effective practices for constructing essay questions include:
1. Using novel problems or material whenever possible, but only if they relate to class learning.
2. Making essay questions comprehensive rather than focused on small units of content.
3. Providing clear directions as to the expectations.
4. Allowing students an appropriate amount of time. (It is helpful to give students some guidelines on how much time to use on each question, as well as the desired length and format of the response, such as full sentences, phrases only, outline, and so on.)
5. Informing students, in advance of answering the questions, of the proportional value of each item in comparison to the total grade.
6. Requiring students to demonstrate command of background information by asking them to provide supporting evidence for claims and assertions.
Permission to reprint the testing section, with modifications, was granted by The Ohio State University: Teaching at The Ohio State University: A Handbook. Center for Teaching Excellence/Faculty and TA Development, The Ohio State University, 1992 revised edition, pp. 64-74.
Return to Teaching Tips
| Home | About CTE |
|
CTE
Programs and Events |
CTE Resources |
Copyright 2000 University of Kansas Center for Teaching
Excellence.
Last modified May 22, 2000