The steep rise in hip fracture incidence rates with age is not fully explained by an increase in the frequency of falls or by reduction in bone mineral density, suggesting that circumstances of falls may also affect the risk of hip fracture. Previous studies conducted mainly among women have identified the importance of the orientation of a fall in the etiology of hip fracture. In this case–control study among men of 45 years and older, we evaluated how the circumstances of falls affect the risk of hip fracture. We compared 214 cases with hip fracture due to a fall with 86 controls who had fallen within the past year but did not sustain a hip fracture. As expected, in multivariable age-adjusted analyses men who reported hitting the hip/thigh in a fall had a markedly elevated risk of hip fracture (OR = 97.8; 95% CI = 31.7–302). Hitting the knee in a fall was associated with reduced risk (OR = 0.24; 95% CI = 0.09–0.67). Other factors that were associated with reduced risk of hip fracture among men who fell were more hours of physical activity in the past year (OR = 0.84; 95% CI = 0.73–0.97, for each additional 4 h per week), a greater body mass index (OR = 0.60; 95% CI = 0.40–0.90, for each additional 4 kg/m²), and a history of a fracture when age 45 years or older (OR = 0.26; 95% CI = 0.10–0.69). Reported lower limb dysfunction was associated with increased risk of hip fracture (OR = 6.41; 95% CI = 2.09–19.6) among fallers. The increased risk associated with hitting the hip/thigh in a fall and the reduced risk associated with high body mass index suggest that preventive efforts for older men at high risk might include protective hip pads to reduce the force on the hip in a fall. Exercise and strength training programs may also reduce the risk of hip fracture among men who fall.