Your Next Gen Data Architecture: Data Lakes
Software Engineering as a field thrives on ever-changing keywords. In the last decade it was "big data" and "noSQL". Every engineer was a big data engineer and every one was working on noSQL. Similarly, now the world is crazy about Data Science and Machine Learning. Half of my connections on LinkedIn have become "data scientists" overnight. Somewhere in between there was another keyword which made a splash (no pun intended!) in the data warehousing world. That keyword was "Data Lakes". In this post, we will talk about what a data lake is, what are the reasons to have a data lake and what are the things you should keep in mind while designing a data lake for your data architecture. Why Data Lakes? The two biggest reasons to have a data lake in your DWH architecture are: scalability and support for unstructured data. A data lake allows your data to grow, enormously. Without any major change in the storage strategy. In this age of big data,...