To clarify aspects of the association between physical activity and breast cancer, such as activity intensity required, and possible effect modification by factors such as menopausal hormone therapy (MHT) use. We prospectively examined physical activity in relation to breast cancer risk among 45,631 women participating in the U.S. Radiologic Technologists cohort. Participants provided information at baseline regarding hours spent per week engaging in strenuous activity, walking/hiking for exercise, and walking at home or work. We estimated multivariable relative risks (RR) and 95% confidence intervals (CI) of breast cancer using Cox regression. We identified 864 incident-invasive breast cancers. Greatest risk reduction was observed among women who reported walking/hiking for exercise 10 or more hours per week (RR, 0.57; 95% CI, 0.34–0.95) compared with those reporting no walking/hiking. The association between walking/hiking for exercise and breast cancer was modified by MHT use (p for interaction = 0.039). Postmenopausal women who never used MHT had reduced risks of breast cancer associated with physical activity whereas no relation was observed among ever users of MHT. Our study suggests moderate intensity physical activity, such as walking, may protect against breast cancer. Further, the relation between physical activity and breast cancer may be modified by MHT use.