About

Ishant Kaushik is a skilled Data Engineer with over 6.6 years of experience in software development and 3+ years of experience as a Big Data developer. He is currently serving his notice period at IHS Markit and has expertise in developing and maintaining big data projects, data pipelines, and performance optimization using Hadoop ecosystem and Apache Spark. He is proficient in programming languages like Scala, Python, and SQL and has worked with various file formats like Orc, Avro, and Parquet. Ishant has extensive exposure to big data on cloud platforms like AWS EMR, AWS GLUE, AWS ATHENA, and AWS S3. He has implemented partitioning and bucketing for best practices and created Hive-Hbase special tables for faster access to customer lookup data. He has also improved the performance of Hive queries and written PySpark code using Spark and Spark SQL. Ishant has experience in performance optimization and has used techniques like Broadcast join, partitioning, and bucketing on join columns. He has written Spark SQL queries to use Hash Aggregate over Sort Aggregate and used re-partitioning to increase parallelism. He has also used Dataset API to leverage Tungsten Optimization provided by Spark and used ReduceByKey over GroupByKey. Ishant has worked with companies like Publicis Sapient and Metadesign Solutions in the past, where he has executed end-to-end responsibility from requirement gathering to delivery for a stream of projects. He has exhibited formidable teamwork and collaboration and provided feasible alternatives to meet tough deadlines. Ishant holds a Bachelor of Technology degree in Computer Science from Krishna Institute of Engineering and Technology, Ghaziabad. He has a strong tech stack that includes AWS, Spark, SQL, Amazon Athena, Azure, Python, Scala, AWS Glue, Hadoop, Apache Hive, and Databricks. Overall, Ishant Kaushik is a highly skilled Data Engineer with relevant experience of 7.27 years and is always eager to take up new challenges in the field of big data and cloud computing.