Facial expressions are of major importance to understanding the emotions, intentions, and mental states of others. Strikingly, so far most studies on the perception and comprehension of emotions have used isolated facial expressions as stimuli; for example, photographs of actors displaying facial expressions belonging to one of the so called ‘basic emotions’. However, our real experience during social interactions is different: facial expressions of emotion are mostly perceived in a wider context, constituted by body language, the surrounding environment, and our beliefs and expectations. Already in the early twentieth century, the Russian filmmaker Lev Kuleshov argued that such context could significantly change our interpretation of facial expressions. Prior experiments have shown behavioral effects pointing in this direction, but have only used static images portraying some basic emotions. In our present study, participants were shown eighteen film sequences of neutral faces across three emotional conditions (neutral, happiness, and fear). The task was to rate the emotion displayed by a target person’s face in terms of valence, arousal, and category. Results demonstrated the presence of a significant effect in terms of valence for both the fearful and happy conditions, and in terms of arousal for the fear condition only. Accordingly, participants tended to categorize the target person’s facial expressions choosing the appropriate emotion category. Our results highlight the context-sensitivity of emotions and the importance of studying them under ecologically valid conditions.