Up until recently, it was easy to define the responsibilities of a given IT role. After all, most IT experts were just plain software engineers. However, the last decade witnessed the development of multiple new expert roles, like the DevOps Engineer, Cloud Developer, Data Engineer, Data Scientist, and more. This article will help you to understand them.
In the late 1980s, IBM researchers developed the idea of the “business data warehouse”. Since then, data warehouses (DWHs) have become an increasingly important data repository. Yet, the market also offers alternatives – like the data lake. In this blog post, we’re going to compare these two systems to help you pick the one best suited to your needs.
The Data Lake isn't new – the term itself has been first used in 2010. After its premiere, it's been met with mixed reviews. The late Dave Needle (Amiga 1000 co-chief architect) characterized the “so-called data lakes” as “one of the more controversial ways to manage big data”. And in 2016, Forbes published an article entitled “Why Data Lakes Are Evil”. However, the response to data lakes hasn’t been universally bad. Many have viewed data lakes as a sort of data panacea. So, which perspective is closer to the truth in 2021? Read this article to find out!
Data engineers have a wide array of responsibilities. Usually, their main job is to make data useful and accessible to other data professionals. Yet, none of their tasks can’t be performed without the proper tooling. That’s why in this article, our data expert is taking you on a journey through the core data engineering tools. With these tools, you’ll be well-equipped to complete all of the key data engineering endeavours.
Did you know that Netflix saves $1 billion annually thanks to big data? From that perspective, it’s not surprising that 97.2% of organizations are already investing in that field. However, with this rapid growth also comes a rising threat – security issues. To fight off these malicious activities, you need protection. So, in this article, we'll explore why big data security is important and how to set it up.