REVISING INSTRUCTIONAL MATERIALS

The following are notes from chapter 11 of The Systematic Design Instruction, sixth edition by Dick, Carey & Carey.

When performing revisions to your materials, you should consider changes that are made to the content of the materials to make them more accurate or effective as learning tool, and you should also consider changes to the procedures employed in using your materials.

The designer has five kinds of data he or she can analyze in order to revise instructional materials: learner characteristics and entry behavior, direct responses to the instruction, learning time, posttest performance, and responses to an attitude questionnaire, if used.

Upon analyzing data from the one-to-one evaluations, the designer should first describe the learners and their performance on any entry-behavior measures. Next, the designer should bring together all the comments and suggestions about the instruction, and then summarize data that is associated with the posttest. It is also helpful to develop a table that indicates each student's pre and posttest scores, and total learning time. With all this information in hand, the designer is ready to revise the instruction.

Based on learner performance, the designer should first try to determine if the rubric or test items were faulty. If the items are satisfactory and the learners performed poorly, then the instruction must be changed. The designer should carefully examine the mistakes made by learners in order to identify the kinds of misinterpretations they are making and therefore the kinds of changes that might be made.

The data from small groups of eight to twenty learners are of greater collective interest than individual interest. The available data often include item performance on the pretest, posttest, response to any attitude questionnaire, learning and testing time, and comments made directly in the materials. Performance on each item must be scored as correct or incorrect. If an item has multiple parts, then each part should be scored and reported separately, so information is not lost.

The designer should do an item-by-objective analysis using a table to determine the difficulty of each item for the group, the difficulty of each objective, and to determine the consistency with which the set of items within an objective measures learners' performance on the objective. The item-by-objective table should also provide the data for creating tables to summarize the learners' performance across tests, or even each individual learner's performance. Learners who mastered each objective should increase from pretests to posttests. Data from the tables can also be displayed through various graphing techniques.

By comparing pretest with posttest scores objective by objective, the designer can assess learner's performance on each particular objective and begin to focus on specific objectives and the related instruction that appears to need revision. It may also be necessary for the designer to consider alternative strategies to use, as well as to revise the materials to make them fit within a particular time frame. Attention should also be paid to the instructional procedures, especially if there were questions about how to proceed from one step to the next.

Tips to remember: The designer should avoid responding too quickly to any single piece of data, and instead should corroborate these data with other data. And just remember that when you make changes through the revision process, you cannot assume that the remaining unchanged instruction will necessarily maintain its initial effectiveness. You may hope that your changes are for the better, but you cannot assume that they always are.