This chapter examines the problem of classifying collective human activities from video sequences. A collective activity is defined by the existence of the coherent behavior among individuals in a spatial and temporal neighborhood. Examples of collective activities are queuing in a line or talking. Such types of activities cannot be just defined by considering actions of individuals in isolation but rather by observing the interactions of nearby individuals in time and space. In this chapter we discuss recent methods for analyzing collective activities through the concept of crowd context. We present various solutions for modeling the crowd context and demonstrate the flexibility and scalability of the proposed framework in a number of experiments on publicly available datasets of collective human activities.