Topic 1e
Numerical "problems"

Learning Outcomes

By the end of this topic students should be able to:

• Explain overflow errors.
• Explain truncation (round-off) errors.
• Discuss situations where inexactness of computer calculations matter (and situations where it doesn't matter) explaining the source of the inexactness.

Learning Materials

• Readings
  • Truncation Errors - pp 66-68
  • Overflow Errors - pp 59-60
• Videos
  • Dr. Schafer explains how computers can "make mistakes" - different kinds of errors in the computer

 

Checking for Understanding

1. Which of these two error types is caused by a limit in the number of bits at the low end of the bit string?
2. Which of these two error types is caused by a limit in the number of bits at the high end of the bit string?
3. Which of these two error types is most likely to occur when we add two bit strings together?
4. Which of these two error types is most likely to occur when we attempt to encode a number like PI into binary?
5. Which of these two error types is most likely to occur when we store decimal (base 10) numbers like money values in a binary encoding?

Answers & Guidance

• The answers to the previous questions.