Follow project on Twitter

2.3.2. Fundamental flaw in the Turing test

The Turing test has a fundamental flaw: The quality of the jury isn't specified. So, any chatbot can pass the Turing test if a jury is selected who is easily impressed, or if the subject (chatbot) is presented as a foreign child who may have problems to understand the given sentences.

Besides that, chatbots are unable to reason logically. So, it is extremely simple to determine whether the subject is a person or chatbot: Let the subject perform an intelligent reasoning task, as described in the challenge to beat my Controlled Natural Language reasoner.

For example, provide the subject with a sentence like “Paul is a son of John” and the following algorithm:
• Swap both proper nouns;
• Replace basic verb “is” by possessive verb “has” (or vice versa);
• Replace preposition “of” by adjective “called” (or vice versa).

Now ask the subject to apply the given algorithm to the given sentence, which should result in a different sentence with the same meaning. The outcome must be: “John has a son, called Paul”, as described in the first block of my challenge. To be sure, ask the subject to apply the given algorithm in the opposite direction, to convert “John has a son, called Paul”. The outcome must be of course: “Paul is a son of John”.

Not a single scientific paper supports the conversion of a sentence like “Paul is a son of John” to “John has a son, called Paul” – nor vice versa – in a generic way (=through an algorithm). So, it would become immediately clear if the subject is a person or a chatbot.

Another way of separating humans from chatbots as a jury, is to only present confusing phrases that are not finished, completely out of context and not related to each other. If the subject initially responds despairingly – and stops responding after a while – then the subject is human. But if the subject keeps responding cheerfully with full sentences, then the subject is a chatbot.

Thinknowlogy 2018r3

will be published in November this year. It will contain part 4 of an internal redesign. See my planning for the other planned publications. Follow this project on Twitter, subscribe to project updates on SourceForge, subscribe to project updates on GitHub or receive an email when a new version is available.

Join my Fair Science Supporters group on LinkedIn to support a fair practice of science, and/or my Fair Science • Artificial Intelligence (AI) and knowledge technology (NLP) group, which is dedicated to a fair practice of Artificial Intelligence and knowledge technology. Or send me a LinkedIn invitation.

God has created laws of nature to make his creation run like clockwork. It includes: Natural Laws of Intelligence embedded in Grammar. Thinknowlogy implements these natural laws of intelligence in software.