Going cloud for your storage needs comes with some baggage. On the one hand, it’s cheap, elastic, and convenient – it just works. On the other hand, it’s messy, especially if you are used to working with data management systems like databases and data warehouses.
Unlike those systems, cloud storage was not designed with things such as transactional support or metadata in mind. If you work with data at scale, these are pretty important features. This is why Databricks introduced Delta Lake to add those features on top of cloud storage back in 2017.
