About

Ankit Kumar is a Senior Data Engineer with over 4 years of experience in Application Development. With a demonstrated ability to develop various ETL and Real-time streaming jobs to solve problems and deal with ambiguity, Ankit is capable of building strategic relations with customers and management. Ankit's skillset includes Big Data, Scala, Java, Spark, Kafka, Hive, Sql Azure, Hadoop, Azure Databricks, Azure Data Factory, and Data Engineering. He is currently working as a Senior Data Engineer at Societe Generale, where he has collaborated with engineers and data teams to monitor, maintain, and leverage data for a $100M+ contract. Ankit has developed effective and optimized data pipelines to process daily and monthly datasets, which helped financial institutes calculate credit risk. He has also built automation monitoring, deployment, and notification scripts to automate the development and deployment process using shell scripts, Ansible. Ankit has developed an automation tool to extract data from multiple data sources and save the data into CSV using Spark, Scala, Ansible, and shell script. He is responsible for the deployment of the latest modules to the higher environment using Ansible, Oozie, and Jenkins. Ankit has previously worked as a Software Engineer at Mphasis, where he built real-time data processing pipelines to parse JSON data using Scala API's Spark Structured Streaming, Databricks, and ADLS Gen2. He has developed and designed pipelines to orchestrate Databricks Notebook, Jar on Azure Databricks using Azure Data Factory. Ankit has built Azure data factory ETL pipelines to read data from multiple sources and load data into ADLS Gen2 using copy data and data flow activities. He has also built delta tables on Azure notebooks to maintain data. Ankit has migrated and converted existing on-premises Spark Application to Azure Databricks Spark Jobs and Java Microservice Application to Spark-Structured Streaming Application. He has set up Databricks for data processing and Azure Data factory for orchestrating the Databricks Jar/Notebook on Databricks. Ankit has optimized and tuned Spark Job for better performance and developed SQL ad-hoc scripts for all Spark Job's performance comparisons. He has also developed Linux shell scripts. Ankit holds a Bachelor of Technology in Computer Science Engineering from Amity University and has completed his Intermediate and All India Senior Secondary Education from RBS College and DAV Public School, respectively. His technical skills include Data Engineering, Spark, Azure, D