The technique of performing classification using association rule mining (ARM) has been adopted to bridge the multimedia semantic gap between low-level features and high-level concepts of interest, taking advantages of both classification and association rule mining. One of the most important research approaches in ARM is to investigate the interestingness measure which plays a key role in association rule discovery stage and rule selection stage. In this paper, a new correlation-based interestingness measure that is used at both stages is proposed. The association rules are generated by a novel interestingness measure obtained from applying multiple correspondence analysis (MCA) to explore the correlation between two feature-value pairs and concept classes. Then the correlation-based interestingness measure is reused and aggregated with the inter-similarity and intra-similarity values to rank the final rule set for classification. Detecting the concepts from the benchmark data provided by the TRECVID project, we have shown that our proposed framework achieves higher accuracy than the classifiers that are commonly applied to multimedia retrieval.