I argue that a target of the rule-following considerations is the thought that there are mental episodes in which a consciously accessible item guides me in my decision to respond in a certain way when I follow a rule. I contend that Wittgenstein’s position on this issue invokes a distinction between a literal and a symbolic reading of the claim that these processes of guidance take place. In the literal sense he rejects the claim, but in the symbolic sense he sees nothing wrong with it. I consider some arguments that Wittgenstein deploys against the literal sense of the claim.