Raul Miller - Precision
The Array Cast
00:00
The Problem With Floating Point Types
You can choose any base you want but like the base that we're talking about is a fixed point type with a base 10 And so I mean no matter what you choose there's always gonna be a higher resolution. You expect to get like three cents like point oh three, but you end up getting point oh oh two. At a high level fixed point exists and that at a very detailed level there's issues with this stuff everywhere.
Transcript
Play full episode
