This paper presents a theoretical foundation of reasoning by analogy between logic programs. We give a formal definition of reasoning by analogy in terms of logic programming. We also introduce a notion of analogical union of logic programs by which we characterize the reasoning. Then we can regard the reasoning as a deduction from the analogical union. Based on this characterization, we present a general framework which is useful for discussing various requirements of analogies.