Hadoop, why I need it

I graduate in computer science with Artificial Intelligence specialization, however during my academic path I was never taught how to use the most common frameworks and environment. When I stared looking for my first job, I immediately went for machine learning and data analysis junior positions. I was sure to be qualified for them, however the truth is that probably I’m not (yet). Most of positions are strongly related to the so called Industry 4.0 as a new way to manage industrial machines with IoT and Big Data technologies. This is when my attention focused on a couple names that I maybe heard only once or twice so far: Hadoop and Spark.

Big Data do not necessarily relate to machine learning but you usually end up with running prediction and learning algorithms on them. Conducting Big Data analysis is almost never faisable on a single machine and instead requires many of them, and here is where Hadoop makes his entrance. Continue reading “Hadoop, why I need it”