AWS Glue with Johnny Chivers
The Engineering Side of Data
00:00
AWS Crawlers
The promise of the crawler seems awesome, right? That this idea that you could just say, hey, fire off these crawlers. The only time I trust the crawler is when it is a JDBC connection into some sort of relational database because it's effectively crawling the Schema there. If it's not set up in an exact format with folders, subfolders, popular with headers correctly and delimited, it will bring back some nonsense for me. So from that point of view, then I operate in the tables three code for anything on S3.
Transcript
Play full episode
