3 Steps To Increase A Math Problem’s DOK Level

I asked educators on Twitter about the questions they have regarding Depth of Knowledge (DOK) in mathematics.

The questions I received ranged across the spectrum, but one of the most common was about how to change a problem’s Depth of Knowledge level.  As an example, how do you take a problem that is DOK 1 and make it DOK 2 or DOK 3?

Here’s my first attempt at articulating my thought process behind increasing a procedural problem’s Depth of Knowledge level.  It’s not my intention to say that this is the only way to change the level.  This is just one way.  Also, I am only addressing the DOK level of the math content, not the conversation that students may have around the problem.

I’m starting out with something really basic: single operation problems.  Specifically, I want to discuss how to take a problem that is primarily a routine procedure and make it more rigorous.  In an effort to make this applicable for a K-12 audience, my examples include addition, subtraction, multiplication, square root, exponents, and trigonometry.  Clearly six problems won’t apply to every grade level, but I hope they are close enough to what you teach so that you can imagine a version you could use.

I would greatly appreciate feedback in the comments on:

• What steps are not clear enough to be actionable?
• What operations should I be including to make this more applicable for all grade levels?
• What explanations are not detailed enough?

Step one begins with finding a problem to modify.  In this case, I’m looking for a problem with one operation.  I consider all six of the problems in this section to be Depth of Knowledge level one because they require little to no mental effort beyond remembering and applying the correct procedure.  Simply put, you can solve these with a calculator even if you don’t know what you’re doing.

To be clear, I’m not saying that problems with more operations are a higher DOK level.  I’m just trying to make these particular examples as simple as possible so that the differences are easier to spot.

Here they are:

Solve.

41 + 36 = ___

Solve.

821 – 357 = ___

Solve.

3/7 x 2/9 = ___

Solve.

√81 = ___

Solve.

3^4 = ___

Solve.

sin(π/3) = ___

Two of my favorite techniques to increase a problem’s rigor from DOK 1 to DOK 2 are:

• Strategically remove some information from the problem (to prevent immediate calculation)
• Raise the quantity of solutions needed (to increase the need to look for patterns)

To make this more concrete, I’ve taken each of the six examples above and modified them using the techniques I listed.  Notice how the amount of thinking needed to solve them has increased.  Also notable is that problems can no longer be solved by a traditional calculator.

Here are the examples followed by my thoughts:

Using the digits 1 to 9, at most one time each, fill in the blanks to make two different pairs of two-digit numbers that have a sum of 71.

__ __ + __ __ = 71

Thoughts

By removing both addends and providing the sum, this is no longer a routine procedure.  Students may initially try guessing and checking, but will inevitably start thinking about the attributes of each addend.  For example, “If the sum is 71, then there’s no way I can use an 8 or 9 in either of the ten’s places.”

Note that there are far more than two different pairs of numbers that have a sum of 71.  This problem could also be modified by changing the sum, changing the operation (such as multiplication or subtraction), or changing the number of digits in each number.  For example, making it one digit and limiting the numbers to between 1 and 5 would make it more appropriate for Kindergarten.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two different pairs of three-digit numbers that form a true number sentence.

__ __ __ – 291 = __ __ __

Thoughts

The basic structure of this problem is very similar to the one for adding two-digit numbers.  In this case, I removed the difference and the number being subtracted from instead of both addends.  Again, students may initially try guessing and checking, but will inevitably start thinking about the attributes of each number.  For example, “If I’m subtracting 291 and the difference is a three-digit number, then I’m not going to have a 1, 2, or 3 in the hundred’s place.”

Note that there are far more than two different pairs of numbers that will make this a true number sentence.  This problem could also be modified by changing the difference, changing the operation (for example multiplication or division), or changing the number of digits in each number.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two different pairs of fractions that have a product of 2/3.

__ / __   x   __ / __ = 2/3

Thoughts

Again, the basic structure of this problem is very similar to the two prior problems.  By removing the numerators and denominators but providing the product, this is no longer a routine procedure.  Students may initially try guessing and checking, but will inevitably start thinking about the attributes of each fraction being multiplied.  For example, “If the product is 2/3, then I’ll need smaller numbers in the numerator and larger numbers in the denominators.”

Note that there are more than two different pairs of numbers that have a product of 2/3.  This problem could also be modified by changing the product, changing the operation (for example subtraction or division), or changing the number of digits in the numerators or denominators.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two true number sentences.

√__ __ = __

Thoughts

By removing the numbers, students cannot immediately jump into computation.  Students first have to realize that not every two-digit number is going to produce a single-digit root.  They may initially discover that √16 is 4, but requiring a second solution helps them think about patterns they may lead them to realize that some perfect squares won’t work.  For example, 25 and 36 don’t work because the digits 5 and 6 are already used.

All of this requires students to think deeply about which numbers will make this problem easier to solve.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two true number sentences.

__ ^ __ = 64

Thoughts

Once more, students cannot immediately plug numbers into their calculator without thinking.  Randomly selecting numbers and checking them will take a long time.  Students may initially discover that 8^2 makes a true number sentence, but requiring a second solution helps them think about patterns they may not normally notice.

To solve this problem, students will have to think about what numbers are options, perhaps based on the factors of 64.  This process is slightly more rigorous than simply computing numbers.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two true number sentences.

sin(__π/__) = 1

Thoughts

The DOK 1 version of this problem required almost no thinking.  You could plug the answer into a calculator or look up the answer in a chart.  While this problem is similar, most calculators will only find one value.  So, adding the requirement for a second answer is less about making a student do extra work and more about having students think about patterns and how they know what they know.

Students will have to think more deeply to realize that there are an infinite number of solutions to this problem that are all 2π (360°) greater or less than another answer (such as π/2, 5π/2, 9π/2, 13π/2, etc.).

Another level of rigor is introduced when you require students to optimize their problem solving, taking it from DOK 2 to DOK 3.  This optimization may include asking students to:

• make the greatest or least product/sum/difference/quotient/answer.*
• make an answer that is closest to a specific value.

* Note that I often use the less precise terms of largest and smallest (instead of greatest and least) as “least sum” sounds strange as compared to “smallest sum”.  This is generally discussed before we start the problem.

This optimization introduces the need for strategic thinking.  Specifically, while you could randomly assign numbers, thinking deeply about what will happen as a result of placing a number in slot will make problem solving much easier.  Put simply, students will ask themselves, “Where do I put the 9?  If I put the 9 there, where do I put the 8?”  These thoughts take time before making the first calculation, but are worth the effort because they make it much easier to solve.

I’ve updated each of the six examples and modified them using the techniques I listed.  When you read through a problem, think about how there are many possible answers that correctly complete the problem but only few (or even one) that is optimal.  As such, the problems are accessible to most students yet simultaneously challenge the most gifted students.  Also notable is that again, these problems can no longer be solved by a traditional calculator.

Here are the examples followed by my thoughts:

Using the digits 1 to 9, at most one time each, fill in the blanks to make the smallest sum.

__ __ + __ __ = __ __

Thoughts

This might be a second grade problem, but even adults will need a little time to think about this one.  Students will likely begin by considering which of the nine numbers they want to use.  Once they’ve chosen them, next they’ll think about placement.  Is putting them in order like 12 + 35 = 47 even a good idea?  If not, what should they do?

By removing the addends and sum, many answers are possible.  This gives struggling students an entry point.  The hard part is trying to find the optimal solution.  This problem could also be modified by changing the optimization goal (for example largest or closest to a specific number), changing the operation (for example multiplication or subtraction), changing the number of digits in each number, or removing the requirement for using some of the numbers to make the sum (see the DOK 3 Subtracting Three-Digit Numbers example for a better idea of what I mean).

Using the digits 1 to 9, at most one time each, fill in the blanks to make a difference that is as close to 329 as possible.

__ __ __ – __ __ __ =

Thoughts

This problem has a notable difference from the one for adding two-digit numbers: in this problem, the numbers 1 to 9 do not need to be used in the difference.  This problem is very easy to enter as any numbers will provide a difference.  However much more thinking is needed to find a difference that is as close to 329 as possible.  Again, students may initially try guessing and checking, but will inevitably start thinking about the attributes of each of the numbers being subtracted.  For example, “If the difference has a 9 in the one’s place, how can I subtract two numbers so that the difference in ones is 9?”

Note that while there are optimal answers, there are still many answers that are relatively close to 329.  This problem could also be modified by changing the target number, changing the operation (for example multiplication or division), or changing the amount of digits in each number.

Using the digits 1 to 9, at most one time each, fill in the blanks to make two fractions that have a product that is as close to 4/11 as possible.

__ / __   x   __ / __ =

Thoughts

It takes much more thinking to solve this problem and verify how close it is than it does for the DOK 2 version.  While any pair of fractions can be multiplied, what numbers will give you a product of 4/11?  If you can’t get 4/11 exactly, how do you determine which fraction is closest?

I asked this question to my fellow math educators on Twitter, and it led to a really rich conversation.  Other version of this problem include asking for the largest product (would students use the four largest numbers or realize that they want the numerator to be large and denominator to be small), smallest product, or closer to a different fraction.  In this case, I picked 4/11 because I wanted a prime number denominator that was more than 10 (so not trivial) but not intimidating (like 237/469).

Using the digits 1 to 9, at most one time each, fill in the blanks to make a result that is as close to √32 as possible.

√__ __  = __

Thoughts

This problem builds off of the DOK 2 version and adds optimization to increase the rigor.  With the DOK 2 version, there were many answers that would result in a true number sentence.  In this case, we have to find the one best result.

To solve this problem students will again need to realize that they have to use perfect squares to get a single digit root.  Next they may choose either √25 = 5 or √36 = 6, only to realize that both of those break the rule of repeating a digit.  Finally, they should end up considering √16 = 4 or √49 = 7.  Ultimately, they should determine that √16 = 4 is slightly closer.  Justifications for this could including using a double number line with square roots on one and integers on the other or converting all the roots to decimals and subtracting.

Using the digits 1 to 9, at most one time each, fill in the blanks to make a result that has the greatest value possible.

__ ^ __ = __ __ __

Thoughts

Students will likely go through several stages in solving this problem.  The first stage involves picking the two largest numbers (8 and 9) for their base and power.  Unfortunately, that results in a nine-digit number, not a three-digit number.  The next stage likely involves figuring out a base and power that will result in a three-digit number.  Eventually students will need to find the largest result, and all the while avoiding using any number more than once.

These restrictions are not trivial but force students to think about how a base is affected by a power and what patterns might make problem solving more efficient.

Using the digits 1 to 9, at most one time each, fill in the four blanks to make a result that has the greatest possible value.

sin(__π/__) = (√__)/__

Thoughts

This problem has similarities to the DOK 1 and DOK 2 version, but requiring students to find the largest possible value requires strategic thinking about the possibilities.  Consider the stages that students might go through in solving the problem.  First, students need to realize what solutions are even possible.  For example, while sin(π/2) = 1, in this format it would be written as sin(1π/2) = (√1)/1 and that is not allowed because the digit 1 is used more than once.

Next students may use a unit circle to discover that sin(5π/6) = (√1)/2.  This is a valid solution and initially looks like the greatest possible value.  But is it?  Students need to realize that the sine function has its maximum value at π/2, so sine of something closer to π/2 would have an even greater value.  In this case, sin(4π/6) = sin(2π/3) = (√3)/2, so sin(4π/6) = (√3)/2 gives a greater value.

This may beg the wonderful question of whether sin(4π/6) is even optimal.  For example, 4π/9 and 5π/9 are even closer to π/2, so maybe sin(4π/9) or sin(5π/9) would provide an even greater value.  The question then becomes whether the value can be written in the (√__)/__ format.

You might be wondering about where DOK 4 is.  Depth of Knowledge level 4 is generally represented by problem-based lessons or performance tasks like my real-world problems.  Those don’t fit well into a structure like this so I’ve left them out.

My hope is that these steps have been practical and understandable.  If you like it, feel free to download a printable cheat sheet below.  If not, please let me know where I can do a better job of tightening up the language or examples.  Do you have other methods for increasing the depth of knowledge level of the content (not the conversation)?  If so, please let me know what those are as well.