Exploring treatment impact heterogeneity across sites: Challenges and opportunities for early childhood researchers

•Evaluations of early childhood interventions often include multiple sites (e.g., centers). Yet, early childhood researchers rarely examine how intervention effects vary across sites.•In this study, we chart our journey in exploring heterogeneity in the treatment impacts of an existing multi-site large-scale preschool professional development evaluation to better understand the challenges and opportunities in this area.•We generate three broad lessons for early childhood researchers interested in studying treatment impact variation.•Using the right statistical approach—namely fixed intercepts, random coefficient modeling—is critical for generating accurate estimates of cross-site variation.•Measures that traditionally predict average treatment effects (e.g., dosage) can be used to predict treatment impact.•Studies should be designed with treatment impact variation in mind given the trade-offs in statistical power for detecting average treatment effects versus treatment impact variation.