Error Correcting Output Coding (ECOC) is an established technique to face a classification problem with many possible classes decomposing it into a set of two class subproblems. In this paper, we propose an ECOC system with a reject option that is performed by taking into account the confidence degree of the dichotomizers. Such a scheme makes use of a coding matrix based on Low Density Parity Check (LDPC) codes that can also be usefully employed to implement an iterative recovery strategy for the binary rejects. The experimental results have confirmed the effectiveness of the proposed approach.