Although diagrams have been widely used as methods for introducing students to elementary logical reasoning, it is still open to debate in cognitive psychology whether logic diagrams can aid untrained people to successfully conduct deductive reasoning. In our previous work, some empirical evidence was provided for the effectiveness of Euler diagrams in the process of solving categorical syllogisms. In this paper, we discuss the question of why Euler diagrams have such inferential efficacy in the light of a logical and proof-theoretical analysis of categorical syllogisms and diagrammatic reasoning. As a step towards an explanatory theory of reasoning with Euler diagrams, we argue that the effectiveness of Euler diagrams in supporting syllogistic reasoning derives from the fact that they are effective ways of representing and reasoning about relational structures that are implicit in categorical sentences. A special attention is paid to how Euler diagrams can facilitate the task of checking the invalidity of an inference, a task that is known to be particularly difficult for untrained reasoners. The distinctive features of our conception of diagrammatic reasoning are made clear by comparing it with the model-theoretic conception of ordinary reasoning developed in the mental model theory.