Roger Penrose Quote

Specifically, the awareness that I claim is demonstrably non-computational is our understanding of the properties of natural numbers 0,1,2,3,4,....(One might even say that our concept of a natural number is, in a sense, a form of non-geometric 'visualization'.) We shall see in 2.5, by a readily accessible form of Godel's theorem (cf. response to query Q16), that this understanding is something that cannot be simulated computationally. From time to time one hears that some computer system has been 'trained' so as to 'understand' the concept of natural numbers. However, this cannot be true, as we shall see. It is our awareness of what a 'number' can actually mean that enables us to latch on to the correct concept. When we have this correct concept, we can-at least in principle-provide the correct answers to families of questions about numbers that are put to us, when no finite set of rules can do this. With only rules and no direct awareness, a computer-controlled robot (like Deep Thought) would be necessarily limited in ways in which we are not limited ourselves-although if we give the robot clever enough rules for its behaviour it may perform prodigious feats, some of which lie far beyond unaided human capabilities in specific narrowly enough defined areas, and it might be able to fool us, for some while, into thinking that it also possesses awareness.