All
too often instruction is developed with little thought as to how evaluation of
learning or the effectiveness of the instruction will take place. When
evaluation is considered on the front end of the instructional design process,
it is often limited to evaluating whether the instructional design is more effective
than traditional methods.
For
this week's reflection activities, I would like for you to concentrate on the
following:
Chapter 10 discusses evaluation in
instructional design and provides you with two evaluation models, the CIPP
and Kirkpatrick models for evaluation.
Evaluation of innovative instructional practices began
using methodology familiar to scientists—experimental groups versus control
groups. In the midst of a revolution in
education that spanned the 1960’s, Michael Scriven coined and defined formative
evaluation (designed to improve a product or work) and summative evaluation
(designed to derive any other information about a product or work, including
its overall value). He believed it
necessary to take the intent of evaluation into consideration before analyzing
its effectiveness.
Chapter 10 highlights the CIPP model of evaluation by
Stufflebeam which is comprised of four individual evaluations—context
evaluation (assessment of the environment in which the intervention would take
place), input evaluation (assesses the resources used), process evaluation
(assesses how the program is implemented) and product evaluation (assesses the
results and outcomes). In this model,
the evaluator is an essential part of the evaluation process and is involved
from the beginning.
Additionally, this chapter discusses the Kirkpatrick
model for evaluating training, first proposed in 1959. As I read the paragraph introducing the model
on page 99 (in the 3rd addition of our text), I realized that the
Kirkpatrick model is a perfect example of a “schema” or “mental structure” as
Wikipedia defines it. There are four
levels of evaluation that can be utilized alone or together, but that build on
each other.
·
Level 1: Reaction
·
Level 2: Learning
·
Level 3: Behavior
·
Level 4: Results
The Kemp model is comprised of nine
different steps represented visually by nested ovals. According to this site from the University of
Alberta,http://www.quasar.ualberta.ca/edit573/modules/module4.htm,
the nine steps can be defined as follows:
1.
Identify instructional
problems, and specify goals for designing an instructional program.
2.
Examine learner
characteristics that should receive attention during planning.
3.
Identify subject
content, and analyze task components related to stated goals and
purposes.
4.
State instructional
objectives for the learner.
5.
Sequence
content within each instructional unit for logical learning.
6.
Design
instructional strategies so that each
learner can master the objectives.
7.
Plan the
instructional message and delivery.
8.
Develop evaluation
instruments to assess objectives.
9.
Select resources
to support instruction and learning activities.
The outer two circles are there to remind the
implementer that planning, revision, formative assessment, summative assessment,
project management and supporting the program occur at each of the nine steps
and repeat throughout.
Like the Kirkpatrick model, the TVS or Training Validation
System model (Fitz-Enz, 1994) was designed to evaluate training utilizing these
four evaluation steps:
1.
Evaluate the situation through pre-assessments
2.
Determine the
needed intervention according to the
data from step one
3.
Assess the impact by comparing pre- and post-test
data.
4.
Derive the value of the training using concrete
methods like Return on Investment calculations
Describe
how you would use them to evaluate your instruction.
Honestly, I most likely would not use
the Kemp model to evaluate my instruction because of its specificity and
because it has about twice as many aspect to consider. However, for some, the detail of the model
would be its strength, especially when beginning to understand the basic
concepts of instructional design. Other
strengths are that it addresses sequencing the content in a logical manner
rather than a blanket prescription for implementation in step five and that it
addresses differentiating learning for the individual student in step six. If I were using this model, I would focus on
steps five and six to help strengthen my instruction. In my opinion, the other steps are already
inherent in the traditional teaching cycle.
Conversely, the TVS model seems a little
too succinct and may or may not be the best model for me to use to evaluate my
instruction. I appreciate its emphasis
on pre-and post-testing. I rarely
formally pre-asses my students, but I believe that they could benefit greatly
from it. However, the TVS model does not
seem to clearly explain what should happen in the classroom during the time
between those two assessments.
No comments:
Post a Comment