Purpose: To investigate the association between likelihood of frequent vigorous physical activity (VPA) outside of school hours and aspects of the built environment that support exercise. Methods: Self-reported VPA measured in 6046 boys and girls in 80 schools. Multi-level logistic regression analyses were conducted to examine the relationship between frequency of VPA and objective exercise resources within 2 km from each school. Results: Total walking paths was the strongest built environment correlate of frequent VPA. Boys from low socioeconomic backgrounds or attending schools with little walking paths had lower odds (OR = 0.66 and 0.68, respectively) of frequent VPA. Girls from low socioeconomic backgrounds had lower odds (OR = 0.62) of frequent VPA. Interactions between socioeconomic background and total paths showed a decreased likelihood of frequent VPA for boys and girls from low socioeconomic backgrounds attending schools with little urban green space (OR = 0.47 and 0.16 respectively). The cumulative effect of paths, street intersections, and number of sports facilities showed boys and girls from low socioeconomic backgrounds attending schools with low exposure were least likely to achieve frequent VPA (OR = 0.48 and 0.17, respectively). Conclusions: Findings of this study suggest that a lack of supportive physical activity surroundings may have a greater impact on children of low socioeconomic backgrounds than those from more privileged families. Thus, socioeconomic context needs to be considered as part of the physical activity landscape when exploring individual physical activity.