The article is talking about Google engineers made snapshots of tables every few minutes to track dimension history instead of spending time to write complex SQL MERGE statements which seems to be more proper. The author concluded that it makes more sense for Google as the cost of storage of those snapshots is much lower than the salary of the engineers.
mighty Data Engineer’s time is more valuable to move fast and create business value, than to waste their valuable time writing MERGE statements
In recent years, especially in the cloud era, more people prefer to buy bigger machines or add more instances over spending time to optimise the code. I saw new programmers always use simple data structure like array or map in all scenario. They don't have the concept of Big O notation and use brute force to solve problems. I still remember the good old days that we need to optimize the C program with byte alignment, memory pool, and use ++i instead of i++ for the little gain of performance.
Time flies. Now it may not bt economically efficient to spend time to optimize your code. But we need to consider sustainable engineering and carbon friendly coding. Think about that your bad code may exist for 10 years and keep consuming extra energy which can be reduce if you spend some time to optimise it in first place. In addition, optimizing code is fun. Identifying and fixing performance issues in a program is a challenging and rewarding process. It requires us to use analytical and critical thinking skills to find creative solutions, which can be a satisfying experience. I will spend hours in LeetCode in order to beat 100% of submissions, which I don't need to, but is fun.
No comments:
Post a Comment