The paper focuses on a difficult problem when formalizing knowledge: What about the possible concepts that didn’t make it into the formalization? We call such concepts the unconsidered context of the formalized knowledge and argue that erroneous and inadequate behavior of systems based on formalized knowledge can be attributed to different states of the unconsidered context; either while formalizing or during application of the formalization. We then propose an automatic strategy to identify different states of unconsidered context inside a given formalization and to classify which parts of the formalization to use in a given application situation. The goal of this work is to uncover unconsidered context by observing sucess and failure of a given system in use. The paper closes with the evaluation of the proposed procedures in an error diagnosis scenario featuring a plan based user interface.