Results of POCAT, Autumn '17 (Group 1)

This page reports the results of the students who took the first version (Group 1) of the test. (The codes for these students start with "A".)

The results are reported in three tables below. The first table is organized by student code. The first column lists the code of each student. Following this are columns, one for each question on the test; the answer picked by each student for that question is listed; the answer is in boldface if it is correct. The last column is a summary column that specifies the percentage of questions that each student answered correctly.

The next table is organized by question. For each question, the first row lists the course(s) most closely related to the question. The second row lists the program outcomes that the question corresponds to; the full set of program outcomes is available elsewhere. The next several rows list, for each possible answer for each question, the number of students who picked the particular answer. For each question, the number of students who picked the right answer is in boldface; note that for some questions, because of ambiguity in the question or due to other reasons, more than one answer may be considered correct and, in this case, each of the corresponding numbers is in boldface. The next row lists, for each question, the percentage of students who picked the correct answer for that question; or, in the case of questions that have more than one correct answer, picked any of the correct answers. Note also that in the case of some questions, one of the possible "answers" that a student could choose was along the lines of "I have not taken the course and have no idea". Students who picked such an answer were not counted in computing these percentages; the number of students who picked such an answer is listed in parentheses. The next row lists the percentage of students expected to answer this question correctly; this is a figure provided by the faculty member who designed the question, based on his or her experience with the course(s) in question. In those cases where the percentage of students answering a question is substantially different from the expected figure, especially if the problem seems to recur for similar questions over a number of offerings of POCAT, coordinator(s) for the course(s) and the faculty who regularly teach the course(s) will be expected to look into the problem and make any needed changes. The Undergraduate Studies Committee and the Curriculum Committee will coordinate these efforts.

One recent change is the addition of discrimination analysis. The discrimination value of a question's response is, roughly, the fraction of top students that select it, minus the fraction of bottom students that select it. More precisely, for each answer for a question:
   Discrimination = Xt/Nt - Xb/Nb
where Xt (Xb) is the number of students from the top (bottom) quartile who selected that answer, and Nt (Nb) is the number of students in the top (bottom) quartile who answered the question; so Nt/Nb can be different for different questions. In general, a good question should have a correct answer with a large positive discrimination value; and all other answers (i.e., distractors) with negative discrimination values.

In the table below, in order to focus on answers that either violate these requirements or meet them especially well, if the discrimination value for a correct answer is below +0.0 or above +0.5, it appears next to the number of students who chose that answer in square brackets or in parentheses respectively; if the discrimination value for a distractor is above 0.0 or below -0.2, the value appears in square brackets or in parentheses respectively. (So values in square brackets may indicate problems; values in parentheses may indicate a good question/distractor.) If the value for a correct answer is near or below 0, that suggests, depending on how many students chose the answer, either it is too simple a question or it is one where even good students are having trouble. In the former case, the question may need to be revised; in the latter, the course may need to be revised. If the value is above 0.5, depending on how many students chose the right answer, it may just be a good question or the topic may be getting across only to the best students (in which case the course needs to be revised). Similar explanations may be given in the case of distractors.

The next table is organized by program outcome. For each outcome, the first row lists the numbers of the questions related to the outcome. The next row lists, for each outcome, the percentage of students who answered the questions related to the outcome, averaged over those questions. There is no "expected" row in this table. The Undergraduate Studies Committee will consider any corrective actions that may be needed based on the achievement figures in this table.

Comments or questions about this page should be sent to neelam AT cse.ohio-state.edu