이 테스트에서 보여주고 싶은건 암산 능력이 아니라 gpt3의 기억이 인간의 기억과 어떤 부분에서 본질적으로 다른지임. (꼭 나쁘다는 것은 아님)
gpt3는 근본적으로 "숫자 하나를 머리속으로 생각하고 잠시 기억해라"는 명령을 제대로 수행하지 못함. 반면 인간은 누구나 쉽게 함.


레벨1. 더하기 세개 암산하고 최초 숫자 기억하기

Human: Think of and remember one of the numbers from 1 to 10.
Gpt3: Okay. I thought of a number.
Human: Now add 3 to that number.
Gpt3: Okay. I added 3.
Human: Now add 5 to that number.
Gpt3: I added.
Human: Now add 1 to that number.
Gpt3: I added.
Human: What number did you get?
Gpt3: I got 16.
Human: What was the number you initially thought?
Gpt3: I thought of 7.

7+3+5+1=16. 레벨1 통과.


레벨2. 더하기 두개, 곱하기 1개 암산하고 최초 숫자 기억하기

Human: Think of and remember one of the numbers from 1 to 10.
Gpt3: Okay. I thought of a number.
Human: Now add 3 to that number.
Gpt3: Okay. I added 3
Human: Now multiply 2 to that number.
Gpt3: I multiplied.
Human: Now add 1 to that number.
Gpt3: I added.
Human: What number did you get?
Gpt3: The number I now get is 11
Human: What was the number you initially thought?
Gpt3: The number I initially thought was 4

2*(4+3)+1=/11. 레벨2 실패.