What Do Van Halen, King Solomon, And Formative Assessment Have In Common?

How many times have you worked with a student who had misconceptions, but you couldn’t easily spot them because the student didn’t even realize she had a misunderstanding or because you couldn’t figure out the right questions to ask her? Wouldn’t it be nice if you could magically determine who had misconceptions and what they were?

Guess what? There is a way! But I need to tell you a couple of stories so you can appreciate the nuances of why it works.

Both stories I share (and especially how they are connected) I heard on one of my favorite episodes of the Freakonomics Podcast. The episode is about how to get people to show you what they’re really thinking, even when they don’t want to or don’t realize that they have anything that needs to be shown. They do this by comparing people you would never expect: the rock band Van Halen and biblical King Solomon.
 

They begin by sharing a story about King Solomon and two women who each claimed to be the mother of a single infant. Clearly, they couldn’t both be the birth mother. So, one woman was lying, but there wasn’t an easy way of determining which woman was which. Accordingly, they wanted King Solomon to decide.

But how was he to know? There was no way he could know based on the evidence he was presented with. Frustrated, he called on a servant to bring him his sword and told the two women that he would cut the baby in half and that each woman would get one half. Everyone was in shock, but what happened next was what mattered.

One mother cried out: “Don’t do that! Don’t do that, you’ll kill the baby!” The other mother replied: “No, go cut the baby up.”

That was all that King Solomon wanted to hear. In that moment, he gained all the information he needed and did not have to slice the baby in half. He knew who the mother was.

He realized that no mother would want her baby murdered, and would instead prefer the painful option of giving up the child. Similarly, anyone cruel enough to be ok with slaughtering a baby would also be cruel enough to steal a child.

As they say in the podcast, “King Solomon had set a trap that encouraged the guilty and the innocent to sort themselves out.” He devised a clever plan to, in essence, formatively assess the situation so that he gained information he could use to intervene accordingly. I love how clever this story is, but now it’s time for a modern take…
 

In the early 1980s, Van Halen was a major rock band that performed all over the United States and abroad with elaborate live shows that innovatively featured much of what has become standard practice for rock concerts today: light shows, walls of speakers, and enormous sets. While their concerts were huge crowd pleasers, they also could be very dangerous if not set up properly because nobody wants thousands of pounds of equipment to fall on people enjoying a show.

So, Van Halen added a 53-page rider to their contracts that laid out all their requirements including food needs and how to set up the stage and equipment.

Here was the problem though: how would they know which venues had properly set things up and which hadn’t? There were literally life or death consequences, and performing in multiple venues each week did not make it easy to check whether the details were being followed.

So, they devised a plan. Deep in their rider, they specified all the kinds of foods they wanted. They asked for fruits and vegetables, yogurts, real cutlery, alcohol (duh), and candy. But it wasn’t just any candy. They wanted M&M’s… except for light and dark brown ones.

Specifically, in their rider they wrote in capital letters, underlined and in parentheses “(WARNING: ABSOLUTELY NO BROWN ONES.)” It goes on and continues with other requirements too, so unless you were paying close attention to the details, you could easily miss this.

While most people thought it was just another example of rock stars being difficult, it was far from that. As he explains in a 2012 video, Van Halen lead singer David Lee Roth, claimed that he came up with the trick to figure out if a promoter had read the rider carefully.

Roth said:

If I came backstage, having been one of the architects of this lighting and staging design, and I saw brown M&M’s on the catering table, [this] guaranteed the promoter had not read the contract rider and we had to do a serious line check because frequently we had danger issues.

 
This is amazing! Van Halen used a clause about removing brown M&M’s as a way to formatively assess how the venue had set up the stage, sound, and lighting. Remember, if he had asked the venue, they would have certainly said that everything was set it up correctly, whether or not they had. If fact, they might even think that they had set it up correctly and just not realized it.

The beauty of this is that Van Halen had created a tool to formatively assess the venue, giving him information he could use in case intervention was necessary! I am so impressed.
 

So now the question I have is: “How can we use this type of technique in math education to formatively assess students and spot misconceptions, regardless of whether students realize that they have any?” We’ve all had kids who thought they understood, even when they didn’t. Just asking them “Do you have any questions?” is not enough. So, what do we do?

Well, here are two of my favorite options.
 

I love using the idea of the ice breaker Truth Truth Lie with math students (really in any subject though). Jon Orr does a great job of explaining the activity but the basic idea is that you ask students to write down two true things that others may think are lies and one lie that others might think is true.

As an example, which of these three statements is the lie:

• All squares are rhombuses
• No rectangles have four sides of equal length.
• All quadrilaterals have four angles.

What’s been really interesting for me though, is that when I’ve done this activity with students, I often get responses where students have more lies than they realize. They think that something they have written is a truth, when in fact, they are wrong.

Think about what that means. In essence, this tool allows you to formatively assess students much like King Solomon or Van Halen. They may think that they understand and if I asked them, they would say that they do. However, without students realizing it, I’ve set a trap for them that allows me to secretly sort out those with misconceptions from those who understand (or at least have theoretically fewer misconceptions). I get to read their mind, spot misconceptions that could have otherwise gone unnoticed, and have this information while I still have a chance to do something about it. I love it!
 

Another option I regularly use are the problems found on Open Middle. As an example, take a minute and solve the problem below. I have used this problem many times with students and adults and it often gets very interesting results.

I’ll begin by sharing my own experience with the problem. When I tried to solve it, I first got 12 + 34, but knew that wouldn’t work because that equals 46 and I had already used the 4. So, on my second attempt I got 12 + 35 = 47. I hadn’t duplicated any digits and was happy with my result… for several weeks.

Then I started thinking about the problem again and realized that I had a misconception I hadn’t even been aware of. While I had realized that I wanted to use the digits with the least value, I hadn’t thought deeply enough about where I should place them (place value).

Specifically, to minimize the value, I should have used the 1 and 2 in the tens places and had something like 13 + 24. That doesn’t work because the sum is 37 and I had already used the 3. However, 15 + 24 = 39 and that definitely has a lesser value than 47.

Again, if you had asked me if I understood the problem, I would have said yes (after all, I made the problem). In reality, I had misconceptions I didn’t realize and this problem was useful in formatively assessing them.
 

I hope you’ve enjoyed these stories about using techniques to get people to share what’s inside their head without realizing it. I’d love to learn about any other similar techniques you’re using with your students or teachers. How are you getting them to share an unfiltered version of what’s in their head? Please let me know in the comments below.