The effects of feedback on performance and retention of skill for a natural language interface

This study investigated the effectiveness of different types of on-line feedback following user errors for informing users of the information and functions available in a system with a natural language interface. Twenty-four individuals performed a task based on an industry cost savings program, in which they were given general goals to pursue with regard to the task. Three feedback levels which differed according to the type and amount of feedback provided, along with two levels of system complexity, were examined. In addition, subjects performed the task again after one week to determine the effects of feedback on retention. Results indicated that the subjects in the second level of feedback generally performed better with respect to accessing system functions and information than those in the first level. Although there was some performance improvement from the second to third level, it was not significant. However, the third level of feedback did significantly improve the efficiency with which subjects used information to complete the task during the return condition. Overall, feedback did not affect the errors made, though at certain more limited stages of the tasks this effect was observed.