About

Harnoor Singh is an experienced Senior Data Engineer with a strong background in Big Data Analytics, Python, Pyspark, Tableau, Java, and Hadoop. He has a demonstrated history of working in the information technology and services industry, with a relevant experience of 2.35 years. Harnoor has worked on various Machine learning models based on Regression, Classification, Clustering, and NLP and Big Data Tools such as HDFS, YARN, and Pyspark. He is skilled in collaborating with various teams and management to understand the requirement and design the complete system. Currently, Harnoor is working as a Senior Data Engineer at dunnhumby, where he is responsible for developing and implementing big data pipelines with real-time processing. He has also worked as an Associate Consultant Data Engineer at KPMG and as a Data Engineer at Capgemini. At Capgemini, he worked as a Big Data Engineer in the Data Lake Team, where the client wanted to store, process & manage a vast amount of daily data collected from various sources. Harnoor has experience working with a wide range of technologies, including all major Realtime, batch ETL Integration, DWH, Data Lake, PySpark, Hive, and distributed cloud-based systems, including AI/ML for data science roadmap using python PANDAS, NUMPY. Harnoor holds a BCA degree in Core java, SQL from Guru Nanak Dev University and has completed an Immersive Data Science Engineering Program focused on Data Science from Grey Atom. He is skilled in Data Engineering, Big Data, GCP, ML, Data Scientist, Python, HDFS, NLP, Hadoop, Spark, Java, Pandas, SQL, AI, Scikit, MySQL, Kafka, ETL, Pytorch, Integration, and Apache Hive. He has processed large amounts of data up to 700GB in Datalake using Pyspark and has worked with millions of trading database records daily, finding common errors and bad data patterns and fixing them using python. Harnoor has also created customized reports in Tableau for data visualization and implemented data wrangling, cleaning, transforming, merging, and reshaping data frames.