Raul Miller - Precision
The Array Cast
The Problem With Floating Point Types
You can choose any base you want but like the base that we're talking about is a fixed point type with a base 10 And so I mean no matter what you choose there's always gonna be a higher resolution. You expect to get like three cents like point oh three, but you end up getting point oh oh two. At a high level fixed point exists and that at a very detailed level there's issues with this stuff everywhere.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
