Understanding Why Computers Struggle with Precise Math and Floating Point Errors

Explore what floating point numbers are, why they cause rounding errors in programming, and understand the limitations of computers in performing exact mathematical calculations.

Understanding Why Computers Struggle with Precise Math and Floating Point Errors
Fireship
8.7M views β€’ Jun 15, 2021
Understanding Why Computers Struggle with Precise Math and Floating Point Errors

About this video

What is a floating point number? And why does it produce rounding errors for programmers?

#shorts #compsci #math

Learn something? Keep the content flowing https://github.com/sponsors/codediodeio

Resources:

https://engineering.fb.com/2018/11/08/ai-research/floating-point-math/
https://0.30000000000000004.com/
https://floating-point-gui.de/formats/fp/

Tags and Topics

Browse our collection to discover more content in these categories.

Video Information

Views

8.7M

Likes

405.5K

Duration

0:54

Published

Jun 15, 2021

User Reviews

4.9
(1743)
Rate:

Related Trending Topics

LIVE TRENDS

Related trending topics. Click any trend to explore more videos.