About

Jyoti Dubey is a highly skilled Data Engineer with over 6 years of experience in the field. She is currently working as a Software Engineer 2 at Microsoft, where she is responsible for the development, testing, orchestration, and documentation of data pipelines using Python, SQL, Shell Scripting, Redshift, EMR, S3, Spark, Airflow, Hybris, and SAP HANA. Jyoti has taken complete ownership of her work and has worked closely with business teams to understand their requirements and perform initial analysis to identify the gaps between requirements and raw data. She has developed a generic component to ingest full/incremental raw data from any database and has designed denormalized tables on top of raw data to generate useful insights for the business. Prior to joining Microsoft, Jyoti worked as a Senior Data Engineer at Accenture, where she worked on the Jarvis Data Lake project for the BU Ecommerce/Retail. She was responsible for end-to-end data integration and orchestration of more than 40 source tables from 10 different data sources. She collaborated with Data Science teams to integrate models with daily processed data to generate highly critical insights for the business. Jyoti also contributed to building a working mechanism for CI/CD, where each week's production migration time was reduced by ~5 hrs. Before Accenture, Jyoti worked as a Business Technology Associate Consultant at ZS Associates, where she worked on the Data Hub project for the BU Healthcare. She migrated the existing Data Warehouse to a Big Data Cloud platform using Python, SQL, Shell Scripting, Teradata, Athena, EMR, S3, SNS, Spark, Airflow, and Oracle. She led the Dev Team of 5 members and was responsible for analyzing, estimating, designing, implementing, and reviewing the Use Cases. Jyoti holds a Bachelor of Technology in Computer Science and Engineering from Jaypee University of Engineering and Technology. She is skilled in Data Engineering, Software Engineering, SQL, Python, Spark, Apache Airflow, CI/CD, Integration, Big Data, Data Science, AWS, Amazon Athena, Data Warehouse, MapReduce, Hadoop, ETL, and Apache Hive. Jyoti is a strong business development professional who takes pride in her work and is always looking for ways to improve her skills and knowledge.