Most video recommender systems limit the content to the metadata associated with the videos, which could lead to poor results since metadata is not always available or correct. Meanwhile, the visual information of videos is typically not fully explored, which is especially important for recommending new items with limited metadata information. In this paper, a novel content-based video recommendation framework, called Video Topic, that utilizes a topic model is proposed. It decomposes the recommendation process into video representation and recommendation generation. It aims to capture user interests in videos by using a topic model to represent the videos, and then generates recommendations by finding those videos that most fit to the topic distribution of the user interests. Experimental results on the Movie Lens dataset validate the effectiveness of Video Topic by evaluating each of its components and the whole framework.