Full interview on the HOSS talks FOSS podcast: https://www.youtube.com/watch?v=8e3IC-lAgXs
Vitess: https://vitess.io/
Planetscale: https://planetscale.com/
Vitess was created in 2010 to solve the MySQL scalability challenges that the team at YouTube faced. This section briefly summarizes the sequence of events that led to Vitess' creation:
- YouTube’s MySQL database reached a point when peak traffic would soon exceed the database’s serving capacity. To temporarily alleviate the problem, YouTube created a master database for write traffic and a replica database for read traffic.
- With demand for cat videos at an all-time high, read-only traffic was still high enough to overload the replica database. So YouTube added more replicas, again providing a temporary solution.
- Eventually, write traffic became too high for the master database to handle, requiring YouTube to shard data to handle incoming traffic. As an aside, sharding would have also become necessary if the overall size of the database became too large for a single MySQL instance.
- YouTube’s application layer was modified so that before executing any database operation, the code could identify the right database shard to receive that particular query.
Vitess let YouTube remove that logic from the source code, introducing a proxy between the application and the database to route and manage database interactions. Since then, YouTube has scaled its user base by a factor of more than 50, greatly increasing its capacity to serve pages, process newly uploaded videos, and more. Even more importantly, Vitess is a platform that continues to scale.
