About

Shivang Agarwal is a Data Engineer II at QuantumBlack, AI by McKinsey with over 4 years of relevant experience in the field of Data Engineering, Big Data, and Machine Learning. He is a Databricks Certified Spark Developer and possesses a strong Computer Science background. Shivang is a big data enthusiast who loves exploring new technologies and enjoys finding creative solutions to complex problems. In his current role at McKinsey, Shivang closely collaborates with clients within EU and US markets, helping with Big Data solutions in the Healthcare industry. He has developed robust data ingestion pipelines using PySpark, SQL, and Cloud services and performed feature engineering for commercial use cases, enabling data scientists to develop an engagement plan for the sales team. Shivang has also worked as a Data Engineer I at McKinsey, where he gained experience in developing data products from ingestion, extract-transform-load, auditing, reporting, and automation of various processes using Apache Spark, Python, and Shell Scripting. He created an automated Data Quality Management (DQM) solution that identifies the quality issue at various ETL stages, reducing manual interventions by 80%. Shivang also developed a scalable Operation Metrics solution using Spark that helps users in monitoring security architecture and alerting data owners about unauthorized access. Shivang holds a Bachelor of Technology in Computer Science from Guru Gobind Singh Indraprastha University. His tech stack includes Data Engineering, Databricks, AI, Spark, Big Data, SQL, ML, Software Engineering, Python, and ETL. Shivang is a skilled and dedicated professional who is passionate about his work and strives to deliver the best solutions for his clients.