In wild populations, defining the spatial scale at which management and conservation practices should focus remains challenging. In Atlantic salmon, compelling evidence suggests that genetic structure within rivers occurs, casting doubt on the underlying premise of the river-based management approach for this species. However, no comparisons of within-river genetic structure across different systems have been performed yet to assess the generality of this pattern. We compared the within-river genetic structure of four important salmon rivers in North America and evaluated the extent of genetic differentiation among their main tributaries. We found a hierarchical genetic structure at the river and tributary levels in most water systems, except in the Miramichi where panmixia could not be rejected. In the other cases, genetic differentiation between most tributaries was significant and could be as high as that found between rivers of the same geographical region. More importantly, the extent of genetic differentiation between tributaries varied greatly among water systems, from well differentiated (θ_ST = 0.035) to undifferentiated (θ_ST = −0.0003), underlying the difficulty in generalizing the ubiquity of within-river genetic structure in Atlantic salmon. Thus, this study underlines the importance of evaluating the genetic structure of Atlantic salmon in large water systems on a case by case basis in order to define the most appropriate spatial scale and focal unit for efficient management and conservation actions. The potential consequences of management at an inappropriate spatial scale are discussed.