CONSIDER ... the idea
1. The problem
It is widely accepted (see e.g., the extensive literature on active
learning, constructionism, etc.) that one of the least
effective ways for students to develop deep conceptual understanding
of most topics is for them to sit in a classroom,
listening to lectures delivered by a teacher.
Especially in STEM courses, it often seems to be the
case that many students do not get a good grasp of essential concepts
related to the topic being presented in the lecture(s) even if, with
the aid of homeworks and other activities, they
seem to master the mechanics of answering specific questions of detail
or solving typical text-book problems and answering exam questions on
the topic (see literature by Hestenes, Mazur and many others on conceptual understanding in STEM courses).
This applies even if the instructor tries to engage the class in brief
discussions by posing appropriate questions during the lecture.
Typically, in such discussions, a handful of the same set of students
tend to respond while other students, at best, listen passively.
At the K-12 level, various innovative active learning
approaches have been implemented but at the college-level, the
standard lecture-based approach is very much the dominant mode.
Although many STEM faculty may agree that a number of students in their
classes do not get as good a grasp of essential concepts as they
should, they are reluctant to make substantial changes in the way the
courses are taught, given serious --and legitimate-- concerns regarding coverage
of course material. Our goal is to address the following questions:
- Is it possible to use the affordances of suitable online
technologies to engage undergraduate students in STEM courses in
activities that will help them develop deep conceptual understanding
of key concepts without affecting coverage of course
material?; and
- If so, is it possible to develop a system/app that is easily
usable by STEM faculty, who may not be IT experts, without expending a
lot of effort?
As we will argue, the answer to the first question is a "definite yes";
and the answer to the second question is a "qualified yes", the
system/app that we are building for the purpose, as part of the
CONSIDER approach, being a work in progress. Further, the CONSIDER
approach has a number of important additional benefits as
described below.
Two points should be noted. First, there have been a few attempts,
especially in introductory undergraduate courses
on such topics as physics and chemistry, to introduce various learning
activities. Indeed, there are some similarities between some of the
ideas underlying the CONSIDER approach and peer instruction,
the classroom-based approach that Mazur pioneered in introductory
physics courses. But these attempts have not
carried over to more advanced courses, possibly because of the concern
related to topic coverage. Second, we should stress that the question
is not about team project activities such as the capstone design
projects that are part of most undergraduate engineering programs.
Indeed, the engineering accreditation criteria require engineering programs
to have an intense team-based capstone design activity/course. But the
purpose of these courses is to help prepare students for professional
engineering practice, including working effectively as part of project
teams, rather than helping them individual students develop deep conceptual
understanding which is our focus.
2. The background
Classic work on how children learn by Piaget and others showed that
one of the best ways for children to develop deep understanding was by
discussing the topic in question with their peers. While that seems
obvious, Piaget's
work identified two key requirements to make the discussion truly
effective. First, the discussion must be among peers and should
not involve someone like a teacher. The reason for this is
that if a teacher were involved, then most learners in the group
will simply accept whatever the teacher says without thinking deeply
about the topic. But when it is only peers, each learner has to listen
to what the others in the group are saying, try to understand them,
and analyze what they are saying because, for all the learner in
question knows, the position held by
any of the members in
the group, including himself or herself, may be right and the others
who disagree with that position may all be wrong! Therefore, it is
essential to understand and evaluate what everyone in the group is saying
in order to identify the --hopefully!-- correct answer; in other
words, carefully analyze the position expressed by each of the
students in the group and then decide which answer is best. In the
process, the learner is likely to develop deep understanding.
Second, the group of peers must include students who have different
ideas about the topic/problem being discussed. After all, if they all
had the same idea/answer (independent of whether it is right or
wrong), there would be nothing for them to discuss. And, perhaps
surprisingly, it turns out that even if everyone in a group starts out
with a wrong answer (but different wrong answers!), then, by
the end of the discussion, several of the students in the group often
get to the correct answer! This seems to happen because when a given
student analyzes other students' wrong answers to see why they
are wrong and, in turn, listens to their analysis of why his/her
answer is wrong, that can force the student in question to fundamentally
rethink the whole topic and, in that process, deepen his/her conceptual
understanding and, incidentally, arrive at the correct answer to the
original question.
While Piaget's original work was with very young children, a number of
researchers, e.g., Doise and Mugny, have explored similar approaches,
mostly in K-12, mostly in-class, and found similar results; i.e., that
groups of 4--5 learners with different conceptions, engaging in discussions
in which they try to understand and analyze each
others' conceptions to identify weaknesses in each, can help all
students in the group develop improved conceptual understanding. It
may also be worth mentioning that a number of researchers have made
the case that the ability to engage in effective scientific
argumentation is, by itself, an important skill that students,
especially STEM students, ought to acquire. These researchers have
investigated various approaches, including online systems, to help
students acquire such skills. While our approach will indeed help
students hone these skills, our primary
motivation is to help students develop deep conceptual understanding in
specific STEM topics;
improving their scientific argumentation skills is a side-effect
rather than a primary focus.
3. The idea
Thus a key part of the thesis underlying our work is that a very
effective way for students in undergraduate (and graduate-level) STEM
courses to develop deep conceptual understanding of key topics is by
engaging in group discussions with each group consisting of 4--5
students who have
different conceptions of the topic in
question, with each student trying to understand, analyze, and poke
holes in each of these conceptions,
including his/her own, to
arrive at a refined/modified conception; we will call such groups
heterogeneous group; it is important to note that the
heterogeneity is on the basis of the students in the group having
different conceptions of the topic, not such factors as gender,
ethnicity, etc. The rationale is that the process of engaging in
such careful analysis and discussion of alternative conceptions will
enable each student in the group to develop deeper understanding of
the topic than would be possible otherwise.
But there are a number of potentially serious difficulties in enabling
effective discussions of this kind in STEM classrooms. First,
unlike students in elementary and middle schools, or perhaps even
college students in social sciences, undergraduate students
in STEM courses tend to be unwilling to participate in extended
discussions with their peers. This may especially be the case with
women students as well as students from other underrepresented groups;
compounding this problem is the stereotypical opinions concerning the
abilities of students from particular groups that, according to the
work of some researchers, are fairly widespread.
Further, while some students, not necessarily those with the most
thorough understanding of the topic, may be quick with their responses,
others, perhaps more thoughtful ones, may prefer to take time to
formulate their responses carefully and this type of discussion is
not conducive to that.
Second, class sizes tend to be large, thirty students or more being
common. So it is difficult to organize students into heterogenous
groups of 4--5 each, and have each group engage in deep discussions
without the groups disrupting each other. Moreover,
class meeting periods are an hour or less and that may be insufficient
for these groups to engage in deep discussions; and, of course, this
relates to faculty's concern about potential negative impact on topic
coverage.
Third, these discussions are ephemeral and a student will have to, perhaps
when studying the textbook chapter on the same topic, rely on
memory to recall what was said and to relate it to his/her original
conception of the topic as well as the textbook material,
thereby reducing the long-term value of the discussion.
This brings us to the second key part of the thesis underlying our
work which is that all of these problems can be effectively addressed by
exploiting the facilities of appropriately designed online systems.
4. CONSIDER
CONSIDER is an approach and an online system that enables
instructors of undergraduate or graduate-level STEM courses to have
students in the course organized into small heterogenous groups which,
for convenience, we will henceforth abbreviate to
HGs,
with each group consisting of students with different conceptions about the
topic under discussion; and have the students in each HG engage in discussions with
other students in the group to understand and analyze each other's conceptions,
listen to the other students' analysis of their own conception, and appropriately
revise/refine their original conception of the topic; and do this not only
without compromising topic coverage but also providing a number of
additional benefits.
The name,
CONSIDER, is an acronym for
Conflicting
Student
Ideas
Discussed,
Evaluated, and
Resolved
(or
Refuted!)
Specifically, the approach works as follows:
- Following class discussion on a given topic, the instructor will post, on
the CONSIDER web-app, a suitable question or problem related to the topic
that will require a two-part answer. The
first part
will be a multiple-choice component and the second will be a more detailed
answer.
- Within a specified time, typically 24-36 hours from the time that the question
was posted on the app, each student will be required to log into the app and post
his/her individual answer to the question. After posting his/her
answer, the student may log in again and freely edit it until the deadline.
- At the end of the 24-36 hour period, students in the class will be organized
into HGs of 4--5 students based on their understanding of the topic, as
indicated by their answers. In some cases,
depending on the nature of the topic/question and the specific set of multiple-choice
options, the formation of the HGs can be done automatically by the app, based on
the option chosen by each student in the class; in other cases, the instructor (or
possibly teaching assitants) will have to go through the more detailed answers
posted by each student to form the HGs; more on this below. For convenience,
in the rest of this page, we will assume that each HG will consist of 4 students.
- Once the groups are formed, typically within 6--8 hours of the deadline for
the individual students to post their answers (or far shorter if the groups are
formed automatically), students will receive an email from
the app informing them that the discussion phase of the topic had begun
and that they should log in and participate in the discussion.
- When a student in one of these groups, say, HG5, next logs in,
he/she will see the answers submitted, individually (in step 2), by
each student in HG5. The student will not know the identities
of the other students in HG5. Instead, the students in the group will
be referred to as, S1, S2, S3, S4. Thus what this particular
student, say, S2, will see will be the answers submitted by each of
S1, S2, S3, S4 of the group HG5. Students in each of the other
groups will also be referred to as S1, S2, S3, S4; but there will be
no confusion since students in a given group will not see the
submissions of students in any of the other groups.
- The discussion in each HG will be organized as a series of
rounds. Each discussion round will (typically) be 24 hours
long. In each round, each student in the group will be required to
make one post; the student may log in as many times as he/she
chooses and freely edit the post until the end of the round; other
students in the group will not see the post until the current
round ends and the next round begins.
- The primary point of each round is for each student in an HG to
respond to the posts made, in the previous round, by each of the
students in that HG. We will label the various
rounds R1, R2, etc.; for convenience, we will also use
R0 to refer to the initial "round" in which students posted
their initial, individual answers, step (2) above.
- During any round, when a student, say, S2 of HG5, logs in, he/she
will see the submissions made by each of the four students S1, S2, S3,
and S4, of HG5 in the previous round. For each of those
submissions, S2 will be required to choose one of
agree, disagree, or is neutral/unclear
about the submission, by clicking on the corresponding
button in the app. Note that S2 is required to do this also
for his/her own original submission; indeed, the whole point of the
discussion is that, after reading the previous round submissions by
the other students in HG5, S2 may no longer agree with he/she posted
in that previous round! In addition, S2 will, as part of his/her
submission, post a brief text explanation of his/her current position
on the question and his/her choices of agree/disagree etc.
(Although the focus of S2's post for the current round will be to
respond to the posts made in the immediately preceding round, S2 will
be able to scroll through the posts made by HG5 students in earlier
rounds (if any) and refer to them in his/her textual explanation. Of
course, if the current round is R1, then the only preceding round is
R0 in which each student posted his/her indvidual initial answers.)
- The number of discussion rounds, N, will be determined by
the instructor and the information will be included when the
instructor posts the question on the CONSIDER app in step (1) at the
start of the activity.
- Following the last discussion round RN, there will be a final
summary round in which each student in each HG will post
his/her final answer/position on the question under discussion; and,
also, post a brief summary of the discussion in his/her HG and how it
influenced (or not, as the case may be!) his/her final answer. The
summary round will typically be 36--48 hours long; the duration will
be determined by the instructor and also posted on the app at the
start of the activity. As in earlier rounds, during the summary round,
the student may log in as many times as he/she wishes, until the
deadline, and edit his/her submission. During this round, the student
will be able to see what each of the students in the group posted in
the now-ended final discussion round (and, if necessary, scroll back
through the posts made by the students in the group in earlier
rounds).
Once the deadline for the summary round has passed, students in the group may
log in to the app and will be able to read the final answers/positions
adopted by each student in the group; and, if they are interested, scroll
through the complete discussion, starting with the initial round posts of
all the students in the group.
A central point to note is that the intent is not, unlike is
the case with many approaches to collaborative or cooperative
learning, for the students in a given HG to arrive at a consensus
answer. That may happen but the goal is for each student in
the group to refine/revise his/her understanding of the topic by
carefully analyzing the diverse positions held by
the different students in the group, considering the other students'
analysis of his/her own position and of each other's positions, and,
on the basis of this, to arrive at a final position with which one or
more of the others in the group may agree or disagree to varying
extent.
5. The benefits
The CONSIDER approach addresses the various difficulties associated with
having in-class discussions in STEM courses that we considered earlier
and offers additional benefits.
- Clearly no class-time is used
for the discussion so there should be no impact on course coverage.
-
There is no possibility of some students in a group dominating
the discussion with the other, possibly shy students, being silent
bystanders. The structure ensures that each student in the group
participates equally (although, of course, the length of textual
explanations submitted in each round by the different students in a
group may vary from student to student). Moreover, students who are
quick to formulate responses to others' positions as well as those who
prefer to take time to construct their responses carefully will be
able to participate effectively.
-
The fact that students can
freely edit their posts for a given round until the end of that round
gives students the assurance that they will have sufficient
opportunity to correct any silly mistakes they might make in their
posts, possibly as a result of misunderstanding the previous round's
post of another student.
- The anonymity of students in each group
ensures that any preconceived notions that individual students may harbor
about the abilities of other students (based on gender, ethnicity, etc.)
in the class does not have any effect on the quality of the
discussion or the benefits that students in the group draw from it.
Interestingly, another benefit of anonymity, one that we had not anticipated,
seems to be that students are willing to explore their ideas more freely without
fear of being seen as stupid by their peers
- Once the discussion is complete, students in the group will have
access to the complete transcript of the discussion and will be able
to refer to it as they move on to other topics or when preparing for
final exams, etc.
Indeed, once the CONSIDER activity on a given topic is complete, the
instructor may find it useful to briefly revisit the topic in class,
focusing on some of the common issues that may have surfaced in the
discussions of several of the HGs. This will be very effective since
many of the students have already spent time analyzing the issues
carefully and will be eager to contribute to the class discussion.
In addition, the instructor may find it
useful to make the discussion transcripts of some specific groups that
were especially illuminating available to the entire class so that
everyone can benefit from them.
6. The potential problems
After using a prototype version of the system in a couple of courses,
we have heard a few complaints from students. First, some students
felt that the 24-hour duration of each round was too long and
prevented effective discussions. This would certainly seem to be an
issue, especially when compared to in-class discussions. Of course,
the advantage of the 24-hour round is that it allows all students,
both those who are quick on their feet as well as ones who are deliberate
in formulating responses to participate effectively and the other advantages
noted above. Moreover, the 24-hour duration allows
students who may have different course, work, and family schedules to
participate without impacting their other activities.
On the flip side, there have also been some complaints that expecting
students to log into the app, read the previous round submissions of
the other students in the group, and post a response every 24 hours is
too demanding! This point can indeed be a legitimate concern,
especially at certain time points in the semester when students might
be working on midterms in other courses or have deadlines for projects
in those courses, etc. This is something that the instructor will need
to try to account for in scheduling the CONSIDER activities. Moreover,
based on our experience, we believe that for most problems/topics that
a STEM instructor might want to construct a CONSIDER activity for
require only two discussion rounds, so this is not as serious an issue
as it might seem. The fact that two discussion rounds is sufficient
came as somewhat of a surprise to us because initially we had thought
that we would need a minimum of four rounds of discussion for
most topics but that turned out not to be the case.
From the instructor's point of view, one serious issue relates to
constructing suitable questions with appropriate multiple-choice
answers such that students with different conceptions of the topic are
likely to choose different answers; if suitable multiple-choice
options can be specified, the system can automatically form the
HGs. In the absence of this, the instructor and/or teaching assistant will
have to go through the initial, individual submissions of the students
to form the HGs. This is not problematic in a class of size 30-40 but
it is clearly not scalable. We are currently exploring possible alternatives
to help with this.
One interesting aspect that we had initially thought might be problematic
was that, given the amount of time for each round (including the initial,
individual submission and the final summary round), that students will
consult on-line resources and the like in formulating their posts. This
did, indeed, happen; but, if anything, this is desirable since
students are taking the time to explore the topic. Indeed, depending on the
topic, in many of the HGs, students often quoted, with attribution, Internet
sources with the group discussing the validity of the claims made in those
quotes. This is a far cry from the often disinterested, last minute answers
that students many students submit for home-work problems.
7. The current status and plans
We implemented the initial version of the CONSIDER system as an app
for use on (Android) phones and tablets. But it became clear very soon
that a system of this kind should be used on a laptop or desktop. The
reason is that the amount and kind of text that students need to enter
into the system in formulating and submitting their posts for each
round is typically such that it would be quite difficult to do it on a hand-held
device. Perhaps even more importantly, the posts that a student sitting
at a laptop or desktop makes are likely to be more carefully and thoughtfully
written. Hence we have moved to a web app intended to be used on a desktop
or a laptop. The current version is accessible at
CONSIDER system.
Some of the concerns that students who have used the prototypes have
expressed (apart from the issues related to the mobile version) have
to do with the naturalness of the user interface (UI). For example, one
concern had to do with the fact that when a user made a submission,
the system did not provide a suitable acknowledgment; and there were
instances where students thought they had made a submission but, given
the lack of suitable acknowledgment, were not sure. There were also
concerns about the ease with which posts from earlier rounds and even the
original question was accessible during later rounds (including, especially,
the final summary round) of the activity. We are evaluating these concerns
and exploring possible changes.
Instructors have also encountered problems with the (instructor portion of
the) UI. Thus, e.g., there is no simple way for the instructor to obtain the
transcripts of the discussions in all of the HGs. As another example, the
instructor has no way to see what a given problem/activity will look like
to a student in one of the HGs. Future iterations of the system
will address these.
A more conceptual concern has to do with the problem already mentioned in
Section 6; i.e., the difficulty that STEM instructors are likely to face in
coming up with suitable multiple-choice options for questions that would be
appropriate as CONSIDER activities in their courses. One possible alternative
that we are exploring may be summarized as follows. The instructor designs a
suitable problem for posting on CONSIDER and, with the problem description,
also includes, say, three (declarative) statements concerning the problem;
and rather than choose one of a set of multiple-choice options, the student is
required,
for each of these statements, to pick one of the following:
- The statement is unrelated/irrelevant to the problem under discussion;
- I disagree with the statement but it is relevant to the problem/topic
under discussion;
- I agree with the statement;
- I am not sure.
Note that the three statements are not and do not claim to be complete
solutions to the problem being discussed but may be (possibly
incorrect) pieces of the solution or aspects directly related or
unrelated to the problem/solution. Since the student is required to
choose one of the four options listed above for
each
statement, it is highly likely that different students will choose
different combinations of answers, enabling the system to
automatically form suitable HGs; moreover, unlike in the case of
multiple-choice answers, even if a couple of the statements were
directly related to the correct solution to the problem, there would
still be pieces of the solution that it would be up to the student to
construct. For now, this is at a conceptual phase; it is only after using
it on a few problems, which we are currently in the process of
planning, that we will be able to judge its effectiveness.
Another conceptual question has to do with assessing the performance
of individual students in a CONSIDER activity. One component of this
is, naturally, the correctness of the solutions to the original
problem that individual students submit, as part of their final
submission. That is no different from the way that instructors would
assess standard homeworks. So our question here has to do with the
assessment of the effectiveness of the student's contribution to the
discussion in his/her HG. Systems based on the classic Toulmin
framework are most commonly used for evaluating the quality of
arguments but such an approach seems inapproporiate or, rather,
inadequate for our purposes. The problem is that the framework
considers such questions as whether a given argument is based on
appropriate data, warrants etc. of the given argument. In our
case, though, a key point of a CONSIDER activity is for a student to
integrate/refine his/her own position based on the positions of the
other students in the group. After all, the submissions of a
student who completely ignores important ideas expressed by the other
students in the HG could conceivably qualify as a high-quality
argument as judged by a system based on the Toulmin framework.
An important goal of the CONSIDER project, therefore, is to come
up with an assessment scheme that accounts not only for the quality of
the student's argument in explaining/defending his/her position but also
in suitably integrating/refining/modifying that position based on the
positions of the other students in the particular HG. This point may also
be phrased in terms of the idea of "arguing to learn" which is the focus
of the CONSIDER approach vs. "learning to argue" which is the focus of much
other work on argumentation in K-12 and in higher education. Perhaps not
surprisingly, the focus in systems based on the Toulmin framework in this
other work is on judging how well a student has learned to argue by
assessing the quality of the student's argument; the Toulmin-framework-based
system that we hope to come up with will instead focus on assessing how well the
student is leveraging the argument to learn.
Comments are welcome:
neelam@cse.ohio-state.edu.