Although information technology (IT)-based applications in healthcare have existed for more than three decades, methods to evaluate outputs and outcomes of the use of IT-based systems in medical informatics is still a challenge for decision makers, as well as to those who want to measure the effects of ICT in healthcare settings. The aim of this paper is to review published articles in the area evaluations of IT-based systems in order to gain knowledge about methodologies used and findings obtained from the evaluation of IT-based systems applied in healthcare settings. The literature review includes studies of IT-based systems between 2003 and 2005. The findings show that economic and organizational aspects dominate evaluation studies in this area. However, the results focus mostly on positive outputs such as user satisfaction, financial benefits and improved organizational work. This review shows that there is no standard framework for evaluation effects and outputs of implementation and use of IT in the healthcare setting and that until today no studies explore the impact of IT on the healthcare system’ productivity and effectiveness.