About

Divyanshu Gupta is a highly skilled Senior Data Engineer with 6 years of experience in the Big Data arena. He has a proven track record of effectively administering components and architecture and managing file distribution systems. Divyanshu is proficient in collaborating with key stakeholders to conceptualize and execute solutions for resolving systems architecture-based technical issues. He is highly skilled in processing complex data and has expertise in Python, Hadoop, Apache Spark, ETL tools, Unix Scripting, Data Analytics, and SQL scripting. Currently, Divyanshu is working as a Senior Data Engineer at JPMorgan Chase & Co. where he has developed data pipelines from scratch, optimized data aggregation from 10+ data sources with about 1000+ datasets, and built a solution using Hadoop Ecosystem (HDFS, YARN) and Spark. He has interacted with different business users to know about their use cases, ingestion tables, PI data, and built data models. Divyanshu has taken complete ownership and worked closely with business teams to understand the requirements and deliver bug-free deliverables. Additionally, he has worked on a migration project involving migrating 2000+ files from on prime SQL loader jobs in legacy to cloud-based Kubernetes platform. Divyanshu has also developed an application for automating database objects deployment using Liquibase and CICD. This framework helped other business units for smooth ingestion of newly onboarded Kubernetes platform from legacy systems. Divyanshu has directly worked with Feature PMs, Senior Leadership, and several other stakeholders. Prior to JPMorgan Chase & Co., Divyanshu worked as an Application Development Analyst at Accenture where he deployed an end-to-end solution for a leading US payment company. He worked closely with the client to get business requirements and developed and enhanced the ETL process involving Abinito, SQL codes, and Unix Shell scripting. Divyanshu overcame challenges of storing and processing voluminous structured/semi-structured data via Big Data Hadoop Framework and developed a data pipeline for one-time data load which involves spark and Hadoop platform. Divyanshu holds a Bachelor of Technology (B.Tech.) degree from KIIT University and completed his schooling from Delhi Public School. He has a diverse tech stack, including SQL, Data Engineering, Hadoop, Spark, Kubernetes, Big Data, ETL, HDFS, Python, Docker, AWS, Apache Hive, MySQL, Jenkins, DevOps, and Senior Software Engineer