Two of the major imaging modalities used to detect and monitor breast cancer are (contrast enhanced) magnetic resonance (MR) imaging and mammography. Image fusion, including accurate registration between MR images and mammograms, or between CC and MLO mammograms, is increasingly key to patient management (for example in the multidisciplinary meeting), but registration is extremely difficult because the breast shape varies massively between the modalities, due both to the different postures of the patient for the two modalities and to the fact that the breast is forcibly compressed during mammography. In this paper, we develop a 3D, patient-specific, anatomically accurate, finite element model of the breast using MR images, which can be deformed in a physically realistic manner using nonlinear elasticity theory to simulate the breast during mammography.