For as long as I can remember, people have been fascinated by robots that can think for themselves and be “intelligent.” This has amazing potential but also comes with some concern that robots will think for themselves and no longer be controllable. In extreme cases, we imagine possibilities like you find in Terminator 2.
Part of this is a curiosity about when the line will be crossed from unintelligent to intelligent. People have debated various tests to measure when this line has been crossed. I want to share two of the most famous examples and focus on one of them, as I think it has application in mathematics education as well.
Perhaps the most famous test is the Turing Test, created by Alan Turing in 1950. You may remember Alan Turing as the main character from the movie The Imitation Game. He proposed a test where a human would write a question which would then be given to two recipients. One recipient would be another human and the other a computer. Those recipients would send back a response, and if the person who asked the question could not tell which was from a human and which was from a computer, then the line of intelligence would be crossed.
Another test, from John Searle, is the Chinese room. In this thought experiment, imagine a man who does not speak Chinese sitting in a room. He has boxes of Chinese characters and a book which gives him a list of the characters he might receive and what character he should send back in return. Then, a native Chinese speaking woman comes up to the room, writes a character on a piece of paper and slips it under the door. The man inside picks it up, looks up what he should do in the book, and slides a character back to the woman outside.
Let’s consider each person’s perspective. The woman outside asked “Do you speak Chinese?” and received the response, “Yes, fluently.” So, from her perspective, the man inside the room understands Chinese. However, from the perspective of the man inside the room, he has no idea what he was given or what he sent back. He would likely say that he does not understand Chinese.
This makes me think about how the Chinese room thought experiment might apply to math education. Consider my own experience. I have a Bachelors of Science in Mathematics from UCLA. So, that proves that I understand college level mathematics. Or does it?
Unfortunately for me, the majority of my middle, high school, and college mathematics experience felt like I was in the Chinese room. My professors handed me numbers to use. I plugged them into a formula. I gave back the corresponding value. From their perspective, I understood mathematics (or at least understood enough to pass the class). From my perspective, I often had no idea what I was doing. I was essentially taking in a character, looking it up in a book to see what character I should send back, and sending it back.
I am concerned that we may be creating our own Chinese rooms in education when we mistake students who give us correct answers to our problems with students who are “intelligent” and have deep understandings of mathematics. I have more questions than answers at this point:
- How do we ask questions that cross the line to measure when students are “intelligent”?
- Are problems like the ones on Open Middle and my problem-based lessons Chinese-room-proof?
- If so, will they always be Chinese-room-proof or will technology eventually make the line even fuzzier?
- What other resources are there for preventing these false positives?
I’d love to hear what you think about my questions or whatever else you’re wondering. Please let me know in the comments.
Problem 1
There is a huge divide in the teaching world between those who understand math and those who understand how to teach it. Teacher preparation programs fall short when it comes to narrowing that gap. There are too many teachers, especially elementary, that don’t know what they don’t know. I have long been a proponent of restructuring 4th and 5th grades where math and reading are taught by specialists and not generalists.
Problem 2
STEM activities that incorporate math but don’t teach it. I have searched hard over the years to find Stem or any other real-world activity that I can connect to my everyday lessons. Where is an activity that teaches multiplying and dividing fractions or solving rational 2-step equations? I have a lot of “fun” activities but nothing that requires deep thought. I would greatly appreciate help here.
Great question! I too am concerned.
How do we ask questions that cross the line to measure when students are “intelligent”?
Let’s talk about gifted programs. My week ended with an announcement that next school year, 4th and 5th grades will have one combo class (4/5 and 5/6) of “gifted or highly proficient” as measured by test scores. A current teacher will be the one to teach this class and I’d bet the chosen ones will have very little if any professional learning.
How does a standardized test score measure intelligence? Jo Boaler’s work makes me re-think gifted programs.
What are your thoughts?
Intelligence cannot be accurately measured nor should we be concerned about it. Comprehension should be our focus. The whole point of a GT program should be to challenge students whose comprehension is above grade level. More importantly we should alos be challenging students on or below grade level as well. This is difficult for teachers whose district mandates teaching curtriculum with fidelity and for teachers who do not have good comprehension of the math they teach. This holds particularly true at the elementary level.
Standardized test scores are just that…a measure of the depth of a student’s comprehension of a set of standards. These scores should be one of several factors considered when placing a student in a leveled course of instruction, not the sole factor.
You are both hitting on some complex issues.
Karen, I understand your concerns but I’m not sure I see it as black and white as you described. For example, I graduated from UCLA with a B.S. in mathematics. When I graduated, I would have certainly described myself as someone who “understood” math, but the reality was that I had so little conceptual understanding that I didn’t even realize that I didn’t understand math.
I also don’t think that this is an elementary or secondary issue. I think it’s an everyone issue.
Liz, regarding your concerns about gate/honors/regular classes, you might find this post interesting: https://robertkaplinsky.com/whats-the-difference-between-honors-and-regular-math-classes/
The more I work with pre-service and in-service teachers, the more I think all grades should be taught math by a specialist.
Perhaps it’s worth considering what it even means to be a specialist. Is a university professor a specialist? Is a math coach a specialist? Is a teacher who only teaches math a specialist? Is someone who teaches math and another class a specialist?
In my experiences, everyone brings something to the table, but all of us also need more support to get to a place where we’re meeting every student’s needs.
Agreed! I have been saying this my whole career. I taught HS for five years, and have taught at the two-year college level for 28 years. If I ever retire, I want to teach K-2 mathematics.
Agreed!
I am not so worried about a specialist… as long as they can share their curiosity and joy in discovery.
I like Dr Sugata Mitra’s “Grandama Cloud”. https://www.youtube.com/watch?v=FZLYIe_jmWM&t=100s
Give kids interesting questions in an appropriate order and they can do the rest.
Thank you Robert, love the thought experiment. I use interview assessments as much as possible (Dr. Nicki Newton’s Math Running Records and the GLoSS Assessment), since it is a lot harder to get a false positive when you get to converse with someone. These scenarios require a barrier (the message needing to be sent in the Turing Test or the door in the Chinese room), and that barrier is a good metaphor for bad assessments since they prevent us from seeing what is otherwise obvious.
That sounds great Berkeley. I can certainly see how interview assessments could defeat the Chinese Room. What I wonder though is how this might scale. Is there a way to incorporate something that is Chinese Room proof at scale like on a standardized assessment?
A colleague of mine has a great solution to this: assessment days are game days – the students play a math game they’re already familiar with while the teacher does a short interview assessment with each student.
Robert I loved this blog. You are spot on about using the term “chinesse room proof”. I think that one way to make sure our math classes are Chinese room proof, is to always provide opportunities for students to explain how the thought about it, how they used numbers flexibly to solve. And for students to make connections to other strategies or concepts. That, I think, is humanly mathematical.
Explanations are certainly a part of it. I think we should also focus on the kinds of problems students work on as well. All of it matters.
Really thought provoking post, Robert! This is something that I have also been wondering/considering ever since I began my master’s program in curriculum and instruction with an emphasis in math. That was the first time I was exposed to the “big ideas” of math and realized that math was so much more than just performing calculations and plugging numbers into formulas.
As a teacher, this idea is something that I struggled with my students on. I wanted to spend more time on comprehension, but didn’t have a great deal of support. I strongly believe that this is something that needs to continue to be explored and discussed.
Definitely. I’m concerned that too many of the tests we hold in high esteem (state tests in the US and TIMSS internationally) can’t tell the difference between a robot and a problem solver, yet we use that data to make major decisions.
Mind blown.
The connection to the Chinese Room is completely accurate. It encapsulates how I was taught math and how some still believe a math classroom should look. Watch the teacher perform some algorithm (almost always without context) and then repeat said algorithm with slightly different numbers. I think utilizing more tasks that have some level of ambiguity like those in Open Middle is definitely a way to help change this, but only if we make other alterations to our structure. If we still follow the same procedures that are void of inquiry, exploration, and thought then these rich tasks are not Chinese-Room-proof.
Good points. For me, I wonder how long it is until technology catches up and even Open Middle problems are no longer Chinese Room proof.
1. I like this analogy. Certainly, we all know stories of students passing tests (in all subjects) without understanding the content. The key, in my mind, is the human interaction. Today, I had a student give me a paper with numbers that looked all wrong to me. He was comparing four brands of vanilla extract to determine the least expensive. His rankings were correct, but the numbers leading to them looked as if they were coming out of left field. With no other info, I might have thought he had guessed. Because I had the time to ask him what he did, he explained his method, and I discovered he had approached the problem by making every bottle equivalent to the first bottle a 2-ounce bottle. I assumed he would use unit prices and compare those. I love the insight that those conversations give me into the way a student’s mind works. If the woman spoke to the man instead of slipping a character under the door, she would quickly know what he knew and did not know. If we allow for the time to talk to our students, we can know what they know, and we can ask better questions to deepen their understanding.
2. I have no math training–or teacher training, for that matter. I have been teaching in private schools for 32 years. I am an American Studies and English major with a Master’s in English who teaches history, English, math, and music. I am an excellent teacher because, in part, I am never lazy about thinking I am good enough. I genuinely love teaching, and I love discovering people who are doing it really well and finding out more from them. I don’t think one needs to be a math specialist, but I do think one needs to be devoted to communicating effectively, no matter what the subject is, to be a good teacher.
Thanks Katrien. Good points. Having conversations with students is a great way to defeat robot problems. I’ll also say that you’ve come very far for not having formal math training. Hopefully my online workshop and others you’ve taken on Grassroots Workshops have been helpful.
There is no doubt that many math teachers do not understand “the why” when it comes to the content they are teaching. We still have a long road to travel with pre-service teachers, but that’s not all……parents push back when the math their children learn differs from how they were taught decades earlier. They can be defensive and consciously undernine the efforts of the teachers. And let’s not talk about administrators who don’t understand how to “evaluate” math lessons based in conceptual understanding!
Good points Jen. Honestly, I probably have a long way to go with understanding the why. It’s a never ending process where the more I learn, the more I realize I still don’t know. Definitely something we should all keep working towards.
I have been like many teachers when it comes to teaching mathematics. I used what I call tricks now. These tricks are PEMDAS, FOIL and SoaKahToa for trig and many others. I used these until I found out by mistake mind you, that my students did not know the why. I knew this was a problem and have been looking for a better way to teach my students the why of math. I think I helped teach generations of students to be math robots or Chinese math students. This is a long process and I too am starting to understand some of the why’s myself. I would recommend to any math teacher to look into problem-based lessons and work on finding the answers yourself and they will help you understand the why’s just as much as your students. Robert thanks for always seeming to send out websites that cause me to think about the why’s before I try to teach my students
Thanks Mark. I appreciate it. My hope is to get us to reflect on what we do and why we do it… hopefully to prevent us from repeating the same mistakes that were made when we were students and by us early in our careers.