the dungeon has...

Does Searle show that a system cannot "understand" merely in virtue of processing symbols in a particular way?

With the advent of computers came a view of the human mind which paralleled their operation. The hardware of the brain was considered less significant and instead study was made into the ‘program’ that was run on it. The intentionality of mental events was seen as an outcome of the syntactical program operated by the mind. For example ‘The Representational Theory of Mind depicts the mind as a semantic engine, a device that operates on purely formal and syntactic principles, but in a way that honours semantics.’[i] In this case this means that the mind is considered to run a program in the same way as a computer, but in our case also honours the notion of our minds having semantic content.

The study of Artificial Intelligence led to the questioning of what intelligence actually is, and more importantly, at what level we could call a machine ‘intelligent’. One of the answers given was the Turing test. It suggested that if a computer could be mistaken as a human in an anonymous question and answer session, then it could be deemed intelligent. Alan Turing said of this:

We now ask the question, ‘What will happen when a machine takes the part of A in this game?’ Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, ‘Can machines think?’[ii]

There is a two-fold problem then, primarily whether the human mind operates in a similar way to that of a computer program, and also whether a computer program that mimics a human mind could be called intelligent, and more importantly whether it could actually be said to understand. The question is then whether the appropriately programmed computer actually has mental states that are equivalent to our own. After all ‘the machine is programmed in such a way that its syntax reflects its semantics: it operates just as though it understood your command.’[iii] The views of the Artificial Intelligence community could be seen to be split into two camps on this issue:

According to weak AI, the principle value of the computer in the study of the mind is that it gives us a very powerful tool… But according to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states.[iv]

This stronger claim about there being a similarity between the human mind and a computer program was questioned by John Searle in his article ‘Minds, Brains and Programs’. He claimed that ‘the original idea of AI… was that running a program was enough on its own for understanding.’[v] However he did not think that this was an accurate picture as: ‘whatever it is that the brain does to produce intentionality, it cannot consist in instantiating a program since no program, by itself, is sufficient for intentionality.’[vi] He used a thought experiment in order to justify his belief that this comparison was invalid. This thought experiment, commonly referred to as ‘The Chinese Room’, attempts to show that running a program based on syntactic rules alone is not sufficient for intentionality (or having any content related beliefs about the world).

Searle imagines himself in a room. From outside the room he is passed squiggles on a piece of paper. He is equipped with a big rule book which tells him that when he receives a piece of paper with a specific squiggle on it he should pass out of the room a specific squoggle, and so on and so forth. The symbols are in fact Chinese. The people outside are passing in questions in Chinese to which Searle is replying in Chinese. However there is no point at which he actually understands Chinese, he is just following syntactic laws. He claims that this is all that a computer could be. Although the Chinese Room has syntax, it does not have semantics.

The original claim made by strong AI was that such a system does in fact have mental states. This is counter-intuitive to the claims of the Chinese Room. There is some sense in which syntactic language is not all there is to having content related beliefs to the world, or intentionality. The Chinese Room is an example of a formal rule based operation, but the ‘Searle-in-the-room’ could not be said to understand. Searle explains this by saying that although the program is purely formal in structure, ‘the intentional states are not in that way formal. They are defined in terms of their content, not their form.’[vii]

His argument can now be summarised as follows:

Programs are formal and syntactical.
Minds have contents that are semantic contents.
Syntax is neither identical nor sufficient by itself for semantics.
Conc. Programs are neither sufficient for nor identical with minds; i.e. strong AI is false.

He has attempted via this thought experiment to prove the views of the strong AI community to be false and therefore that computers could never be said to have any form of intentionality, no matter how complicated the rules that are used, as rules do not amount to meaning.

He asserts that ‘as long as the program is defined in terms of computational operations on purely formally defined elements, what the example suggests is that these by themselves have no interesting connection with understanding. They are certainly not sufficient conditions, and not the slightest reason has been given to suppose that they are necessary conditions or even that they make a significant contribution to understanding.’[viii] It does not matter what the machine in question is capable of, it would still not have understanding. After all ‘[his] car and [his] adding machine… understand nothing: they are not in that line of business.’[ix]

This is a potentially fatal blow to the Artificial Intelligence community as not only does this intimate that they will never succeed in their aims to create an artificial intelligence, but also that their very method for testing, that of the Turing test, may be flawed. In the end the Turing test only measures behaviour, and if the Chinese Room system could potentially pass such a test then so could other machines which also lack understanding. It also suggests that there is a possibility that the Representational Theory of Mind is not wholly accurate either as it too is reliant on syntax.

Searle considers several responses to his argument. First the Systems reply which says that although the individual inside the room does not understand Chinese, the system as a whole does. This immediately sounds peculiar. How can we say that a room could understand anything? Searle considers that instead the man has completely memorised the rulebook, and that even then he still would not understand.

The second reply is known as the Robot reply. It considers that interaction with the world is necessary for the Chinese to come to mean anything. This view has also been used as an argument against the Turing test, because mental ability is not all, ability to function in the everyday world is also a necessary part of having experience. Jaegwon Kim summarises this view:

…it is difficult to see it as a test for the presence of mental states like sensations and perceptions, although it may be an excellent test of broadly intellectual and cognitive capacities. To see something as a full psychological system we must see it in a real-life context, one might argue; we must see it coping with its environment, receiving sensory information from its surroundings and behaving appropriately in response to it.[x]

This view is equivalent, as I have mentioned, with the robot reply. If we take a machine capable of passing the Turing test but also part of a robot so that it interacts with the world about which it talks, would it not then have understanding of the world? Searle’s first response is to point out the slight change in view. ‘The first thing to notice about the robot reply is that it tacitly concedes that cognition is not solely a matter of formal symbol manipulation, since this reply adds a set of causal relation with the outside world.’[xi] It admits that formal processing is not sufficient for intentionality, but also adds an additional clause of causal relation to the world. Even this is not perhaps enough because a Chinese Room situated inside of and controlling a robot would still not be related to the world in the same way as a human mind.

The third possible response is the Brain Simulator reply, which considers the creation of a machine that in every way simulates the operations of a human mind, the same neurones fire at the same time etc. However, Searle comments that simulations of this type are in fact still simulating the wrong thing, as it is not the hardware at issue. It is possible to imagine a Chinese Room in which Searle activates a series of pipes in exactly the same way as firings in the brain, but even that would not be equivalent or meaningful.

The fourth response considers a combination of the previous three responses, a robot with a brain-like computer, as a unified system would surely understand? Searle admits that in such a case we would attribute such a thing with intentionality until given evidence to the contrary or a different reason for the behaviour. The same problems with having a formal program still apply.

The fifth response (known as the Other Minds reply) is not really so much an answer as an attempt at philosophical debate. It considers that the only way that we know Chinese speakers understand Chinese is through their behaviour, and the same reasoning could be applied to a suitably behaving machine. This however is not really relevant to the discussion, at hand.

The sixth and final reply is branded the Many Mansions reply. It says that the argument of the Chinese Room only applies to the current state of technology, and that it is possible that one day machines might be made which do have the kind of causal connections that Searle desires. To this reply Searle has not particular objection, but he does say that this is an attempt to redefine the position of strong AI to allow for change, which could be seen to weaken the strong AI thesis. He says that:

…it in effect trivialises the project of strong AI by redefining it as whatever artificially produces and explains cognition. The interest of the original claim made on behalf of artificial intelligences that it was a precise, well defined thesis: mental processes are computational processes over formally defined elements.[xii]

So, in effect, you could suggest that if the position of strong AI were reformulated then Searle would have no problem with the potential for intentionality in machines as long as it was not based on ‘formally defined elements’. However, Searle could be considered to be making more than one claim within the paper:

He makes two main claims: that computational theories, being purely formal in nature, cannot possibly help us to understand mental processes; and that computer hardware – unlike neuroprotein – obviously lacks the right causal powers to generate mental processes.[xiii]

In the paper he appears to suggest that the biology of the brain is also a contributing factor. He says that part of the reason he has understanding ‘…is because I am a certain sort of organism with a certain biological structure.’[xiv] It is the biology of the brain that gives us causal powers. His suggestion is that inorganic matter such as metal and silicon could never have such causal powers anyway. This is a claim that is not so well supported within the text. Although it is possible that there is an element which is brought to the mind purely by the biology of the brain, it is as yet uncertain whether this is necessarily the case.

To conclude, although attempts have been made to disprove the conclusions of the Chinese Room, it seems entirely rational to think that merely processing symbols in a particular way is not sufficient for the production of understanding. After all, syntax is not equivalent to semantics, and never will be. However, if allowance is made for changes in science which permit for the production of machines that are not entirely based on linguistic style rules and programming, then the possibility of creating a machine which could be said to have understanding of the tasks it performs seems more likely. Searle may have been right about the fact that following rules alone is not sufficient for understanding, but he has not yet shown that it is not in some way also a contributing factor.

- - - - - - - - - - - - -

[i] Heil, J. Philosophy of Mind, Routledge 1998, p109
[ii] Turing, A. ‘Computing Machinery and Intelligence’, The Philosophy of Artificial Intelligence, Ed. M. Boden, Oxford University Press 1990, p41
[iii] Heil, J. Philosophy of Mind, Routledge 1998, p109
[iv] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p509
[v] Crane, T. The Mechanical Mind, Penguin Books 1995, p127
[vi] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p519
[vii] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p517
[viii] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p511
[ix] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p511
[x] Kim, J. Philosophy of Mind, Westview Press 1998, p98
[xi] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p514
[xii] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p516
[xiii] Boden, M. ‘Escaping From The Chinese Room’, The Philosophy of Artificial Intelligence, Ed. M. Boden, Oxford University Press 1990, p89
[xiv] Searle, J. ‘Minds, Brains and Programs’, The Nature of Mind, Ed. D. Rosenthal, Oxford University Press 1991, p516

TrackBack URL for this entry: http://bluejoh.com/cgi-bin/mt/mt-tb.cgi/408