Can an Artificial Intelligence Get into the University of Tokyo?

Tainaka　Could you tell us the objectives of the project?

 

Miyao　We are researching the process of thinking by developing a computer program that will be able to pass the University of Tokyo entrance exam. The program will need to integrate multiple artificial intelligence technologies, such as language understanding, in order to develop all of the processes, from reading the question to determining the correct answer. While the process of thinking is first-nature to people, many of the processes involved in mental computation are still mysteries, so the project will be taking on challenges that previous artificial intelligence research has yet to touch.

 

Tainaka　You're not going to making a physical robot?

 

Miyao　No. What we'll be making is a robot brain. It won't be an actual robot that walks through the gate, goes to the testing site, picks up a pencil, and answers the questions.

 

Tainaka　Why was passing the university entrance exam selected as the project's goal?

 

Miyao　The key point is that what's difficult for people is different than what's difficult for computers. Computers excel at calculation, and can beat professional chess and shogi players at their games. IBM's "Watson" question-answering system*1 became a quiz show world champion. For a person, beating a professional shogi player is far harder than passing the University of Tokyo entrance exam, but for a computer, shogi is easier. What makes the University of Tokyo entrance exam harder is that the rules are less clearly defined than they are for shogi or a quiz show. From the perspective of using knowledge and data to answer questions, the university entrance exam requires a more human-like approach to information processing. However, it does not rely as much on common sense as an elementary school exam or everyday life, so it's a reasonable target for the next step in artificial intelligence research.
Tainaka　Elementary school exam questions are more difficult?

 

Miyao　For example, consider the sentence "Assuming there is a factory that can build 3 cars per day, how many days would it take to build 12 cars?" A computer would not be able to create a formula that expresses this in the same way a person could, near-instantaneously. It wouldn't understand the concepts of "car" or "factory", so it wouldn't be able to understand the relationship between them. Compared to that, calculating integrals is far easier.

 

Tainaka　The National Center Test for University Admissions is multiple choice, and the second-stage exam is a short answer exam, right?

 

Miyao　Of course, the center test is easier, and it has clear right and wrong answers, making it easier to grade. For the second-stage exam, examinees must give written answers, so during the latter half of the project, we will be shifting our focus on creating answers which are clear and comprehensible to human readers.

 

Tainaka　Does the difficulty vary by test subject?

 

Miyao　What varies more than the difficulty itself are the issues that have to be tackled by artificial intelligence research. The social studies questions, which test knowledge, rely on memory, so one might assume they would be easy for computers, but it's actually difficult for a computer to determine if the text of a problem corresponds to knowledge the computer possesses. What makes that identification possible is "Textual Entailment Recognition"*2, an area in which we are making progress, but still face many challenges. Ethics questions, on the other hand, frequently cover common sense, and require the reader to understand the Japanese language, so they are especially difficult for computers, which lack this common sense. Personally, I had a hard time with questions requiring memorization, so I picked ethics. (laughs)

 

Tainaka　So ethics and language questions are difficult because they involve common sense.

 

Miyao　Similar challenges are encountered with English, other than the common sense issue. For example, English questions include fill-in-the-blank questions, but it's difficult to pick natural conversational answers without actual life experience. Reading comprehension questions test logical and rational thought, but it's not really clear what this "logical and rational thought" consists of. The question, then, is how to teach "logical and rational thought" to computers. Also, for any subject, questions sometimes include photos, graphs, and comic strips. Humans understand them unconsciously, but it's extremely difficult to have computers understand them.

 

Tainaka　Aren't mathematical formula questions easy to answer?

 

Miyao　If they were presented as pure formulas, computers would excel at them, but the reality is not so simple. The questions themselves are written in natural language, making it difficult to map to the non-linguistic world of formulas. The same difficulty can be found with numerical fields, like physics or chemistry, or in fields which are difficult to convert into computer-interpretable symbols, such as the emotional and situational experience of reading a novel. That's what makes elementary school exams difficult.

 

Tainaka　There are a mountain of problems.

 

Miyao　There are many problems that nobody has yet taken on. That's what makes it challenging, and it's very exciting working with people from different fields. Looking at the practical results of this project, our discoveries and developments will be adapted for use in general purpose systems, such as meaning-based searching and conversation systems, real-world robot interfaces, and the like. The Todai Robot Project covers a diverse range of research fields, and NII plans to build an infrastructure, organizing data and creating platforms, and bring in researchers from both inside and outside Japan to achieve our objectives. In the future we will build an even more open platform, creating opportunities for members of the general public to participate as well, and I hope anyone motivated will take part" (http://21robot.org/introduce/NII-Interview/, accessed 12-30-2013).