Evaluation of new multimedia formats for cancer communications

Background: Providing quality, current cancer information to cancer patients and their families is a key function of the National Cancer Institute (NCI) Web site. This information is now provided in predominantly-text format, but could be provided in formats using multimedia, including animation and sound. Since users have many choices about where to get their information, it is important to provide the information in a format that is helpful and that they prefer. Objective: To pilot and evaluate multimedia strategies for future cancer-information program formats for lay users, the National Cancer Institute created new multimedia versions of existing text programs. We sought to evaluate user performance and preference on these 3 new formats and on the 2 existing text formats. Methods: The National Cancer Institute's "What You Need to Know About Lung Cancer" program was the test vehicle. There were 5 testing sessions, 1 dedicated to each format. Each session lasted about 1 hour, with 9 participants per session and 45 users overall. Users were exposed to the assigned cancer program from beginning to end in 1 of 5 formats: text paperback booklet, paperback booklet formatted in HTML on the Web, spoken audio alone, spoken audio synchronized with a text Web page, and Flash multimedia (animation, spoken audio, and text). Immediately thereafter, the features and design of the 4 alternative formats were demonstrated in detail. A multiple-choice pre-test and post-test quiz on the cancer content was used to assess user learning (performance) before and after experiencing the assigned program. The quiz was administered using an Authorware software interface writing to an Access database. Users were asked to rank from 1 to 5 their preference for the 5 program formats, and provide structured and open-ended comments about usability of the 5 formats. Results: Significant improvement in scores from pre-test to post-test was seen for the total study population. Average scores for users in each of the 5 format groups improved significantly. Increments in improvement, however, were not statistically different between any of the format groups. Significant improvements in quiz scores were seen irrespective of age group or education level. Of the users, 71.1% ranked the Flash program first among the 5 formats, and 84.4% rated Flash as their first or second choice. Audio was the least-preferred format, ranking fifth among 46.7% of users and first among none. Flash was ranked first among users regardless of education level, age group, or format group to which the user was assigned. Conclusions: Under the pilot study conditions, users overwhelmingly preferred the Flash format to the other 4 formats. Learning occurred equally in all formats. Use of multimedia should be considered as communication strategies are developed for updating cancer content and attracting new users.