After many years of experimental research software systems to support clinical decision-making are now moving into routine clinical practice. Most of the research to date has been on the efficacy of such systems, addressing the question of whether computer systems can significantly improve the quality of doctors’ decision-making and patient management processes. The evidence that they can make a major improvement is now clear and interest is beginning to turn to the question of how we can make such systems safe. We outline some example applications and discuss what we can learn in developing safety cases for such applications from the work and experience of the software safety community. Some distinctive challenges also arise in medicine, and some novel safety management techniques to address them are being developed.