In more detail
Defined by IEEE 754, a 32-bit float has 1 sign bit, 8 exponent bits and 23 fraction bits (about 7 decimal digits); a 64-bit double has 1, 11 and 52 (about 16 digits). The value is roughly sign × 1.fraction × 2^exponent, so the gaps between representable numbers grow with the size of the number. Special values include signed zero, infinities and NaN (not a number).
Why it matters
Most decimal fractions, such as 0.1, have no exact binary form, so results carry tiny rounding errors and comparisons with == can fail. Programmers compare with a tolerance and use integers or decimal types for money. Inspect the exact bits and value of any number with the IEEE 754 Float Converter.
Example
In double precision 0.1 + 0.2 equals 0.30000000000000004, not 0.3, because neither 0.1 nor 0.2 is stored exactly. The stored value of 0.1 is 0.1000000000000000055511151231257827….