Recognizing human activities from image sequences is an active area of research in computer vision. Most of the previous work on activity recognition focuses on recognition from a single view and ignores the issue of view invariance. In this paper, we present a view invariant human activity recognition approach that uses both motion and shape information for activity representation. For each frame in the video, a 128 dimensional optical flow vector of the region of interest is used to represent the motion of the human body, and a 90 dimensional eigen-shape vector is used to represent the shape. Each activity is represented by a set of Hidden Markov Models (HMMs), where each model represents the activity from a different viewing direction, to realize view-invariance recognition. Experiments on a database of video clips of different activities show that our method is robust.