« July 2014 »
S M T W T F S
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
You are not logged in. Log in
Instructional Systems Design - ISD
04/09/09
Summative Evaluations
Topic: 92- Summative Evals

Summative evaluations -- which link us back to the needs assessment-- are conducted to document the strengths and weaknesses of instructional materials, to verify their effectiveness with target learners, and to make decisions about whether to maintain currently used materials or adopt materials that have the potential for meeting an organization's defined instructional needs. (pp.340-341)

Summative evaluations answer the questions:

  • "Did the intervention, including the instruction, solve the problem that led to the need for the instruction in the first place?" (p.340)
  • "Did it fill the identified gap?" (Dr. Watkins)

The evaluator is not supposed to be familiar with the materials, the organization requesting the evaluation, or the setting in which the materials are evaluated > external evaluator. 

The 2 main phases of summative evaluation:

  1. Expert Judgment Phase: to determine the potential of candidate instruction for meeting the needs of an organization.
    • Overall decisions: do the materials have the potential for meeting this organization's needs?
    • Specific decisions: (p.341)
      1. Congruence analysis: are the needs and goals of the organization congruent with those in the instruction? > information summary form
      2. Content analysis: are the materials complete, accurate, and current?  > product checklist or rating scale
      3. Design analysis: are the principles of learning, instruction, and motivation clearly evident in the materials?  > product checklist or rating scale
      4. Utility & Feasibility analysis: are the materials convenient, durable, cost-effective, and satisfactory for current users?  > information summary form, product checklist, or rating scale
  2. Field Trial Phase: to document the effectiveness of instruction with target learners in the intended setting.
    • Overall decisions: are the materials effective with target learners in the prescribed setting?
    • Specific decisions: (p.341)
      1. Outcomes analysis: > criterion-reference tests, attitude questionnaires, interviews, observations, company records
        • Impact on learners: are their achievement and motivation satisfactory after instruction?
        • Impact on job: are learners able to transfer information/skills/attitudes to the job setting or to units of related instruction?
        • Impact on organization: are learners' changed behaviors (performance, attitudes) making positive differences in the achievement of the organization's mission and goals?
      2. Management analysis: > questionnaires, interviews, observations, company records
        • Are instructor and manager attitudes satisfactory?
        • Are recommended implementation procedures feasible?
        • Are costs related to time, personnel, equipment, and resources reasonable?

Formative Evaluation Vs. Summative Evaluation:

  • Formative Evaluation: to collect data in order to revise instruction
  • Summative Evaluation: to collect data in order to document the strengths and weaknesses of instruction

Posted by Nada at 12:01 AM EDT
Updated: 05/05/09 12:02 AM EDT

Newer | Latest | Older