**See ../onlineCollabLearning.html for intro to CONSIDER consider**

From: Neelam Soundarajan <neelam@cse.ohio-state.edu> 
To: neelam@cse.ohio-state.edu, joshi.127@osu.edu 
Subject: CONSIDER this ... 


Guys,

[I am sending this to all the students in CSE XXX.]

[There is a request at the end of this message. Make sure you respond
to it as soon as possible, definitely before classtime tomorrow.
Here is a summary of the request: 
  **Send me your *Gmail* id in a reply to this message.**  ]

I hope all of you finished the Scheme lab. But we are not quite done;
there is one more homework!

This one is going to be rather different from all the other homeworks
and labs in this course (and maybe in all other courses that you have
taken as well). And I think you will have fun with this one.

Before I get to some of the details of the homework and how it is
going to be different, some background.

For a while now, I have been interested in the question of the best
possible approaches for students to learn something. It turns out that
research with children (by Piaget, whose name you might have heard,
and others) shows that the best way for people to learn something
*deeply* is by discussing it with their peers. You might say, isn't
that obvious?  What is so unusual about that? Two things: First, the
discussion must be among peers and should not involve someone like a
teacher; the reason for this is that if there is a teacher involved,
then most students in the group simply accept whatever the teacher
says and don't think deeply about the topic. But when it is only
peers, you have to listen to what the others in the group are saying
and try to understand them and analyze what they are saying because
you might be wrong and one of them might be right; but you can't
blindly accept anything any of them says because they might all be
wrong and *you* might be the one who has the right idea/answer! So 
you have to evaluate what everyone is saying, then decide which answer
is best; in the process, you develop deep understanding. 

Second, the group of peers must involve students who have different
ideas about the topic/problem being discussed. After all, if they all
had the same idea/answer (independent of whether it is right or
wrong), there would be nothing for them to discuss, right?

One other point: it turns out, rather surprisingly, that even if
*everyone* in a group starts out with a wrong answer (but *different*
wrong answers!), then, by the end of the discussion, several of them
will get to the correct answer! How is that possible? It seems that
when you analyze other people's wrong answers to see why they are
wrong and you listen to their analysis of why *your* answer is wrong,
that forces you to fundamentally rethink the whole thing and, in that
process, you often get to the right answer.

That makes a lot of sense but there are numerous problems in having
such a discussion in class such as: we will have to form these small
groups of 4-5 students each with each group including students with
different ideas about the given topic, make sure that one or two
people in a group don't dominate the discussion in the group, devote
(scarce) classtime to the discussion which means the groups can't
really get into extended discussions, etc. People have tried this in
some college classrooms but, because of all these and other problems,
with limited success.

So, of course, being computer science professionals:-), Swaroop Joshi
(joshi.127@osu.edu), a PhD student in our dept., and I have been
working on developing a web app to solve the problem. We (and some
others) have developed a web app called CONSIDER that allows for such
discussions and has a number of advantages over trying to have these
discussions in class. Here is how it works: The instructor first posts
a "lead-in" question to CONSIDER and tells the class about it. Each
student is required to log in (by some specified time) and submit
his/her answer to the question. The question will be something that is
key to the topic but is a relatively simple question, perhaps a
multiple-choice question but one for which the student must provide a
textual explanation of his/her particular choice of answer. If you
submit your answer at some point and then decide you want to change,
you may do so up to the specified deadline (typically 24 to 48 hours
from the time the question was posted).

We (i.e., Swaroop and I) then divide the class into groups of 4--5
students each, trying to make sure that each group includes students
who picked different answers to the lead-in question (or at least
provided different textual explanations even if their choice from the
possible set of options was the same). This may take us a bit of time.

In the meantime, the next time you (the student) log in to CONSIDER,
you will be presented with a more detailed problem on the topic. Note
that the discussion with peers has not yet started; that will start
shortly; but first, you must answer this detailed problem and submit
your answer to this problem. You will have to complete this by the
deadline for this activity (typically 48 hours after the problem was
posted).

So by this deadline, two things have happened. The instructor has
created "conflicting groups" of students with each group containing
students with different ideas about the topic; and each student has
submitted his/her individual answer to the detailed problem. 

The discussion starts at this point. But the structure of this
discussion is *very* different from what you might have on, say,
Piazza:

a. Each group is separate; that is, you only see the posts of the
students in your group. The instructor can see the posts of all the
groups but does *not* participate. 

b. You do NOT know the identities of the other students in your group.
Instead, the students in a group will be labeled, say, S1, S2, S3, S4.

c. The discussion will be organized as a series of "rounds". Each
round will last 24 hours. In a given round, each student will make
exactly *one* post; but he/she can edit that post as many times as
he/she chooses up to the deadline for that round.

d. The other students in the group will not see the post made by a
student in a given round *until* that round ends. So, e.g., in the
third round of the discussion, student S2 will see the posts made by
S1, S2, S3, and S4 in the second round but not any posts that the
other students might have made so far in the *current* round. 

e. In his/her post for the third round, student S2 is *required* to
state, for the second-round posts of *each* of the students in the
group, whether he/she agrees with that post, disagrees with that post,
or is neutral with respect to (perhaps doesn't understand) that post;
and *why*. Note that S2 is required to respond to his/her *own* post
from the previous round; this is because S2 may have found the
second-round post of one of the other students so compelling that
he/she no longer agrees with the position he/she took in the second
round! In fact, *this* is the key. It is when this happens that we
know that the discussion has helped S2 see the light! Anyway, each
student makes one post in each round, stating agree/ disagree/ neutral
with each of the posts from the previous round, the rationale for
this, and his/her current position.

f. The "agree/disagree/neutral" choice is made by clicking a green/
red/yellow button; if you press "red", you better include an
explanation why you disagree! (By the way, the app is usable on a
smart phone, tablet, laptop, or desktop; most current browsers should
work.)

g. For the first round of the discussion, what S2 will see and respond
to will be the individual answers submitted by each of S1, S2, S3, S4
to the problem; if it is an easy problem and S2's individual answer
was wrong, he/she may already see the light and realize what the right
answer is, pick the red button against his/her original post (i.e.,
her individual answer) and the appropriate buttons for the other
students' posts, and submit the correct answer and why she changed
his/her position ... if it is a more complex problem, it may take
several rounds of discussion ...

h. Each round will last 24 hours and you can edit your post for the
current round as many times as you wish until that time expires. And
then the next round will begin. Once a round ends, you can see the
posts of all the students in your group in all the previous rounds but
not edit any of them.

i. How many rounds? The instructor will decide that in advance. 
Typically, three/four rounds should be enough ... (but given that we
are at the end of the semester, for the problem that you will work on
in the next few days, we will only have two rounds). 

j. Once the last discussion round finishes, each student will submit
his/her individual *final* answer to the detailed problem; and he/she
will also be required to submit a clear summary of the group's
discussion. 

k. The grade that the student receives will depend only on the
correctness of this final answer and the quality of the summary he/she
submits. So even if the student's original answer was incorrect, that
will not affect his/her final grade; only his/her final answer and the
quality of the discussion summary he/she submits.

l. One important point: it is *not* the goal of the discussion for the
entire group to arrive at a consensus answer. It is conceivable that
all group members will come to the same final answer but that is not
at all required. The key goal, instead, is to help *each* student
develop as deep an understanding as possible by engaging in serious
discussion with other students who have conflicting ideas about the
topic and do so in a structured fashion.

m. The reason for keeping the student identities anonymous is that it
allows a freer expression of ideas without students having
preconceived notions about who might be right/wrong. The point is that
if S2 thinks S3 is very smart and S4 is dumb, S2 may not analyze S4's
post carefully and may blindly accept what S3 says and both reduce the
effectiveness of the learning. So by having anonymous groups, we avoid
this problem.

n. Sorry for the long message! I hope all that made sense. If you want
to read even more about it, we wrote a paper recently that I have a
copy of on the course homepage (under "Useful info/tips").

o. CONSIDER is implemented on the Google App Engine. So to use it, you
have to have a Google account. Please reply to this message and send
me your Gmail id and I will add you to the list of users to CONSIDER
and send you a link. You will then be able to use it. (And if you do
not have a Gmail address, please sign up for one; it is free and it is
useful even if Google is turning kind of evil over the last few years!)

p. We will talk about this briefly in class tomorrow and the lead-in
question will then be available on CONSIDER. 

q. You will have to submit your individual answer to the lead-in
question by 11:59 pm on Fri., Dec. 4.
The detailed problem will then be available and you have to submit
your individual answer to it by 11:59 pm on Sun., Dec. 6.
The first discussion round will then start and you have to make your
post for that round by 11:59 pm on Mon., Dec. 7.
The second discussion round will then start and you have to make your
post for that round by 11:59 pm on Tue., Dec. 8.
Then you will have to submit your individual final answer to the
detailed problem as well as a summary of your group's discussion and
how it helped you arrive at your final answer by 11:59 am on Thurs., 
Dec. 10. (Note: That is 11:59 am, not 11:59 pm.) [The final exam for
the course is on *Friday*, Dec. 11 at *4:00-5:45 pm* in the usual
classroom.] 

r. This will be part of a research study that we are conducting. What
that means is that, *after* the semester ends and the final grades
have been assigned, we will anonymize all the discussion data using a
unique code for each student, get data about student grades from the
grade roster and anonymize it using the same unique code, and then
(this is the research part) try to see whether there is any
correlation between how effectively a student participated in the
CONSIDER discussion and the student's performance in the final
exam. To include your data in our research study, we have to get your
permission; if you decide that you do not want to have your data
included, that will have NO impact on your grade or anything like
that. You still have to do the homework since it is part of the course
(and if you don't complete it, you will get a zero in it!) but whether
or not you agree to have your *anonymized* data included in our study
is a separate question and will have no impact on your grade. I very
much hope you will agree to it and if you do I will be glad to send
you the results of the research; but note that the results won't say
anything about *your* participation compared with that of others in
the class because, as I said, before we do any analysis, student
information will be anonymized and there will be no way to map any of
the data to any particular individual student.

r. I hope all that made sense. So send me your Gmail address as a
reply to this message.


Hope you all enjoy this last homework!

--Neelam

p.s.: Why is it called CONSIDER? That stands for "CONflicting Student
Ideas Discussed, Evaluated, and Resolved"! Which sort of captures the
idea of the approach; and, of course, in each discussion round as well
as when you submit your final answer, you are considering the (often
conflicting) ideas of the other students in your group.