Observations of Numeracy Errors and Success

The best motivators for reform in education are the experiences we have that show us what needs to be learned, what people have failed to learn, or what learning leads to greater learning. Below are some of my experiences that led me to rethink what math education should be.

Errors and what we can learn from them

Arithmetic with large numbers and our perception of the world

I was once talking with an art teacher, and the discussion turned a bit political. The year was 1992. She said that if our government eliminated the space program we would solve the federal deficit. She justified this claim by saying that ". You always hear about NASA making mistakes that cost a billion dollars." She further went on to say that we would then have the money to spend on important programs. I have no desire to argue her values or perception of the accomplishments of the space program. This discussion is limited to an example of innumeracy.

At the time of this discussion, the federal deficit had been averaging over $200 billion dollars annually. Over the previous ten years NASA had had about 5 tragedies each costing roughly $1 billion. The total federal deficit over this decade was about $2 trillion, the cost of NASA's major errors was less than $10 billion. Could the rest of the NASA budget for the decade have really been nearly $2 trillion The problem she presented could be dealt with using simple estimation and either subtraction or division. No advanced math (higher than seventh grade skill level) was needed.

What did her perceptions of NASA and the federal deficit suggest about her math education ? She didn't appear to have a grasp of large numbers - what is the difference between a million, a billion, and a trillion? Does it really matter? She didn't appear to be able to make estimates using incomplete information. By saying that the money could first be used to solve the deficit and then also used to create more programs, she didn't appear to be aware of the zero-sum nature of a budget.

Think back over your math education. Was estimating with real data ever emphasized? Were clear methods for distinguishing between very large numbers ever emphasized? Was interpreting data ever emphasized? For each of these most people will answer, "no." From seventh grade through college most of us studied the procedures of higher level math, but how many learned how to make sense of real data? The contrast here motivating educational reform is the distinction between memorizing procedures and interpreting real numbers.

Statistics and interpreting the world around us

Most Americans have some concern about the state of the world - social, economic, or environmental. When they discuss their concern with someone who has a different value system the process is quite predictable. They present some data on which their concern is based. They rarely acknowledge that their data is statistical in nature. When somebody suggests alternate data, and even worse uses the word, "statistics," they respond with, "people can make statistics prove anything," or "that's not what I've experienced." Similarly, I have seen activists of opposite camps use the same set of statistics to argue opposite viewpoints. I am not challenging the validity of personal experience, activism, or even statistics here. Instead, for these situations, I'm asking what math was not learned, or was learned incorrectly.

The errors seem to occur in the following forms:

  • first people appear to believe that in math (and consequently data) there is always one right method and one right answer. This belief that is not valid for statistics. This perception is probably rooted in the way we were taught arithmetic.
  • Second, yet related, is the way we impose absolute interpretations on our statistics. Frequently the numbers are right, but the interpretations are wrong. Correlation is not causality. Similarly, the beliefs of the majority do not constitute truth (Ref: De Tocqueville's Democracy in America discussion on Tyranny of the majority ).
  • third, and very important, is that average and median can be useless information without the more important, and rarely reported, idea of distribution or variance. Nobody is exactly equal to the average! America does not have a family with 2.3 children. People do not suddenly drop dead when they reach the life expectancy!
  • Finally, the smaller (and less random) the sample the greater the probable error. Everybody's life constitutes a small data space, nonrandom sampling, of of the world as a whole. Nobody's personal experience constitutes a good description of the world as a whole. (See a simulation of this effect.)

What does all this suggest about education reform? Students need to learn what to expect from small data samples, and nonrandom samples. Students need to learn to interpret distribution or variance, and think about distribution in data when its not reported. Students need to learn that correlation is not causality, and students need to learn the difference between the statistics and the interpretation of the statistics. Look back over your math education. Which of these things were emphasized? Do statistics make you so angry that you say, "Liars figure, and figures lie?" If you had learned with these suggested reforms, you would probably not say that.

Seeing the bigger picture

All too often a big problem is the sum total of small problems and needs to be understood as such. A minor example comes from my experience living in a fraternity house. Complaints would arise about who was messing up the kitchen. On one occasion I pointed out to a complainer that he had left a pan in the sink, to which he responded, "That's only one pan!" I countered that there are 12 rooms in the house and if each room misses one item a day then the kitchen will be a disaster in only two days. Another countered that there are 25 people living in the house .....

Much more prevalent, we read letters for political action that after after a long moral argument state that their cause is justified because it will only cost $0.94 per tax payer. Regardless of whether the moral arguments are justified their cost argument is invalid. A more accurate cost argument would state that there are over 1000 causes of similar importance and cost, and that means each taxpayer must pay over $1000 to support each of these causes.

Many more examples of this type could be cited. They all lead to the same elements missing from math education - making estimates with incomplete data, and generalizing from a small piece of data to the whole set of data. They all imply the same consequence - most people severely underestimate their own impact on their environment. At a young age, I sat with a friend who spoke a strong social conscience. To my horror, at the end of the night, he smashed the entire case of empty beer bottles in the river. If each young American makes this error just once while growing up, how many beer bottles will be in our rivers? If each young person in your neighborhood makes that error just once how many bottles will be in your river? What do you think this implies about corrective actions?

Last Modified 1/03/09



More Numeracy Errors

Readings and Links


Generalizing understanding new problems from old problems

Polya (Ref ) says that the last step to problem-solving is generalizing what was learned to new problems. We teach algebra in all our schools because algebra is about generalizing. Equations that apply to rabbit populations apply to interest on bank accounts, and current in electronic amplifiers. If we know about one situation we can understand others, even though the problems appear quite different.

Most industrial temperature controllers rely on three ways of analyzing data known as PID - process, integral, derivative. Process asks, "How far is the temperature from where we need it to be?" Derivative asks, "How is the temperature changing?" And Integral asks, "How long has it been away from where it should be?"

While working at a company that produced temperature controllers, we frequently observed various control situations and how PID affected them. Occasionally my supervisor would discuss how PID can be used to understand many other things, one such extreme being personalities. A person who is very reactive, is operating on high derivative control, a person who holds a grudge or changes very slowly is operating on high integral control. He was able to understand how formulas for one situation applied to other quite different situations.

Biological research has shown that many of the processes related to our senses to work on derivative control, a few appear to work on integral control.

Seeing order in Randomness

I once worked at a company that made radiation data-logging devices. We tested one overnight and observed the results in the morning. We noticed that about 3:00 A.M. there was a very large spike. We were all worried by what this could mean. Did somebody come to work in the middle of the night and set a source near the detector? Did somebody break in and do something? Did we have a software glitch? While most of us worried, and discussed the problem a senior engineer walked off to do some calculating.

Radiation is a random process that has been well characterized. In random processes extremes are to be expected, although rare. The senior engineer came back with calculations that showed that a spike of the size we observed had better than a 60% chance of happening over the course of our test. To our eyes it appeared out of character, but, in fact it, wasn't.

How many other experiences in life are probable but seem unlikely?



In each of the example of successes, somebody was able to think about what it meant, think about what the answer might look like, and ask what the answer should look like. This does not reflect math education as we know. The focus of the thinking was not on doing procedures. The focus of the thinking was on what concepts and ideas apply to what situations.

In each of the examples of failure, the person in question was unable to apply the number concepts to life experiences. They demonstrated no ability to connect the concept and the procedures. Overall, these examples seem to reflect that they had never learned to apply mathematics to meaningful situations.

The NCTM Standards for reform in math education reflect these observations. The standards say less focus on procedures and more focus on concepts. Can we reason about the problem, can we develop solutions for real problems, can we communicate those solutions and problems? What do we need procedures for, if we can't apply them to real problems?


Return to: