[This is one of a series of posts that explore real world examples of mathematical modeling to help educators better understand its applications. To learn about Spies and Analysts, I recommend either reading this blog post or watching this webinar (with elementary, middle, and high school versions).]

For years I’ve been trying to find a way to talk about mathematical modeling that made sense to me. I’ve finally found something I’m happy with and it has to do with the differences between spies and analysts. I’m sure this sounds strange, so let me explain.

The United States’ Central Intelligence Agency (CIA) is responsible for acquiring and using information to help American causes. There are two main groups in this agency: the people who acquire information (we’ll call them spies) and the people who break down and connect the information (we’ll call them analysts). Both are important because having more intelligence about what others are doing allows the United States to make more informed decisions. Let’s consider how this played out in both World War 2 and September 11th.

World War 2 and September 11th
In World War 2, we had many spies whose information was pivotal to the missions’ successes. It would follow that if we had more spies then, perhaps there would have been even more information leading to more successes.

Now let’s consider September 11th. What if we had more spies gathering information about what was going to happen? Would that have helped? It may seem like it would have, but unfortunately it wouldn’t have been as valuable as having more analysts.

Prior to the attack on 9/11/2001, the US intelligence community had received many tips about suspicious activity. For example, they knew about suspected terrorists who had been receiving flight instruction. So, in this case, the issue wasn’t that the information wasn’t available. The issue was that there weren’t enough analysts to take the information they received and determine what was useful and what was unimportant.

So, the way I see it, there are two primary components in this process:

1. Spies figure out what information is needed and acquire it
2. Analysts take the given information and work with it

Hopefully what results is actionable information that gives the US intelligence community what they want. If it doesn’t, then they need to modify their plans and look at where the process broke down. Was it an issue of the spies not getting the necessary information? Was it instead that the information was there but the analysts didn’t put it together?

The takeaway here is that we need a combination of both spies and analysts to find success. Interestingly, I believe that this structure of spies and analysts is very useful for gaining deeper understanding of mathematical modeling. To explain what I mean, let’s consider how we predict weather and earthquakes.

Weather and Earthquakes
Weather happens all the time and we’ve got great technology to measure it. We have plenty of data to work with and plenty of opportunities to verify if our forecasts are correct. So, why are our weather forecasts often incorrect? The problem here isn’t with the spies but with the analysts. We have not figured out how to use the data to get highly accurate forecasts. We need better analysts to take the available information and create a formula, or mathematical model, that more accurately predicts what will happen.

Now let’s think about how we predict earthquakes. Right now, we’re pretty awful at figuring out when future earthquakes will take place. We know that they will happen, but we can’t tell if they will take place in a week, a month, a year, a decade, or a century. So, how can we improve? Should we be investing in better spies or analysts? In this case, the problem is with our spies, not our analysts. Earthquakes happen so rarely that it’s hard to capture data on them. We have seismographs to measure earthquakes, but is that the best we can do? What about how animals seem to go crazy before an earthquake. Could there be other available data that our spies are not aware of or are not currently able to capture? Even if you come up with a highly accurate formula for predicting earthquakes, they happen so rarely that it makes it really challenging to fine tune it.

So, with mathematical modeling:

1. Spies figure out what information is needed and acquire it
2. Analysts take the given information and work with it

Structure Visualization
Here are visual representations of the steps. Realize that the spies and analysts are often the same single person or group of people in the context of solving a math problem.

First comes the spies who figure out what information is needed and acquire it.

Next comes the analysts who take the given information and work with it to figure out what to keep and how to use what remains.

Now you have a mathematical model created by the analysts using the information acquired by the spies. At this point, you can use the model and verify if it works. For example, does it actually predict the weather or earthquakes?

If it needs modifications (and almost all real mathematical models do) then it’s time to start the process over again. The spies will verify that they have the right information and the analysts will consider other ways of using the provided information.

Conclusion
I’m extremely curious about how this structure can help us make better sense of mathematical modeling and how it might prevent us from getting so far from what what we’re after. Accordingly, I’m starting a series of posts called “Math Modeling Can…” that will explore what this looks like in the context of actual math modeling problems. You can read the first one right now about how Target figured out a 17-year-old girl was pregnant and sent her diaper coupons before her father even knew she was going to have a baby! I hope you’ll be curious about this too and let me know where you think I’m on the right track and what concerns you have in the comments below.

1. Joshua Nelson says:

This is really great and has helped me wrap my mind around SMP4. I haven’t read the target article, but it is an enticing cliff hanger. I did, however, recently read Moneyball by Michael Lewis and about 50 pages in it finally dawned on me that they were creating a model. In fact, they use the word model frequently when discussing baseball stats, because that is basically what they are; de-contextualized numbers we can gather from real players and toy with in the hopes of winning more games!

• Robert Kaplinsky says:

Thanks Joshua. Yes, I reference Moneyball in a presentation I do as they created a mathematical model to figure out which baseball players are better than people realize. Based on my current understanding of mathematical modeling though, I would not consider the individual stats to be the model but rather a data point that spies might pass to the analysts when creating the model.

2. What if we, as educators, consider ourselves both spies and analysts, building models of what students understand? Spies, in that we are observing mathematical behaviors of students; analysts, in that we take that information and work with it. A large part of quality teaching is working with students where they are, not just shooting a bunch of knowledge out into the void and hoping kids catch it. Building “second-order-models” of student understanding is vital, but this takes a lot of questioning, listening, observing, and analyzing in order to make good instructional decisions.
I like your analogy and how it leads to building mathematical models. Would love to know your thoughts on applying the same analogy to building models of student understanding.

• Robert Kaplinsky says:

I don’t see why it couldn’t work. For example, as teachers we are making real-time decisions as to where and how we spend our time an energy when we are with the classrooms. Without realizing it, we are constantly adjusting the relative weighting of what we see. For example, in some situations, a kid who is off task might be something we address. Other days we might have too many things going on and focus elsewhere.

Nice call.

• Kim Morrow-Leong says:

Grading is definitely a mathematical model of performance, something that occurred to me out of the blue one day.
Of course it is. We assign values to an assignment based on its importance and then put student performance on a scale. The percentage generated comes straight out of those choices.
For this reason, 0.01 or 0.1 difference between an A and a B is meaningless, as models can be easily tweaked.
Are assessment and teaching models? I have to give that more thought.

• Jennifer Cassini says:

I feel like grading is less a model then it could be. It’s really just an accumulation of data points. Grades could be the data spies gather but then someone would need to analyze them to create something new from that to consider it a model.

3. John A. Bell, J.D. says:

It is interesting that you make reference to mathematics and World War II. While many are aware of John Von Neumann’s mathematical contributions to work on the Manhattan Project. However, few are aware that he created mathematical models to plan out the path the bombers carrying the bombs would take to minimize their chances of being shot down.

• Robert Kaplinsky says:

4. Josh Rosen says:

Hi Robert-
I have really enjoyed reading your posts about mathematical modeling. I was wondering if you could give an example of what modeling would look like in the primary grades. Thanks.

5. Kim Morrow-Leong says:

I want to dig into the Spies/Analysts/Modeling phase. The work of Analysts needs some unpacking because the decisions made here impact the results so profoundly.
It reminds me of a Thompson’s grain silo problem. Thompson posed a group of 5th graders from a farming community the conundrum of deciding and quantifying when and how silo explosions happened. The kids knew that grain dust, dryness, and the height of the grain were factors. Step 1 is quantifying and naming the units. I don’t remember the exact details, but all of the factors they noted could be measured.
Grain dust can be measured as parts per million.
Humidity has a measure and scale.
Height of the grain can be it’s volume. Possibly, or the volume of the space between the grain and the top of the silo.
No matter the details, the students recognized that low humidity was a contributing factor so the model reflects that. The “dustiness” of some grains over others contributes more to the risk of explosion, and having less space in the silo above the grain also contributes to the risk. The model generates A SINGLE number that reflects the the risk, which at first has no context. Through experience, the number generated can be placed on a scale of Grain Silo Explosion Risk.
This makes me think of other scales that acquire meaning only with experience with the model: SAT scores, Apgar scores, many of the stats generated from blood tests such as white blood cell count, etc.
In any case, my original point is that forming the mathematical model expresses relationships quantitatively by collecting “extensive“ quantities and marrying them together with some operation to create “intensive” quantities. This is a multiplicative relationship. What if we wanted factors to have less dependence on each other – would an additive relationship better represent what’s happening? What if the relationship is better expressed as an exponential one? The huge questions about operations and operation sense that arrive out of this issue are really interesting.

• Robert Kaplinsky says:

Thanks Kim. I love your questions. So one thing is that in the way I conceive of Spies and Analysts, I would consider the “quantifiying and naming the units” to be part of the Spies’ job. Any data collection including what data should be collected is part of the Spies. Deciding what to do with the data is the Analysts.

That being said, yes, figuring out what to pay attention to and what to ignore is some of the most challenging aspect of the Analyst work. I don’t think we give students enough time playing around with huge questions like the one you shared.

6. Scott Litterer says:

I enjoyed this post and especially enjoyed the link about potential terrorists attending flight schools. One of my favorite resources is The Numbers Behind Numb3rs: Solving Crime with Mathematics by Devlin https://www.goodreads.com/book/show/914664.The_Numbers_Behind_Numb3rs. This book has been a great resource in the my statistics classes and fits in well with your spies & analysts examples. Also, you may have a broken link to the information about animals going crazy before earthquakes. I was not able to get it to work.

• Robert Kaplinsky says:

Thanks Scott. I appreciate this connection. I fixed the link for the animals as well. Apparently the site was updated.

7. Anna Tait says:

It is similar to the difference between understanding how to do a problem with the available information and an in-depth understanding of how the concepts fit into the overall scheme of mathematics and the limitations and links arising from this.

There are many teachers and students who limit the definition of understanding to being able to apply a process correctly in problem-solving. It ignores the context and development of the processes and hence the links and limitations.