About

Sulbha Goyal is an experienced Senior Data Engineer with a strong background in Big Data Development. With over 7.78 years of experience in the information technology and services industry, she has a demonstrated history of working with various Big Data technologies. Sulbha is proficient in Apache Spark, Apache Hadoop, Scala, Core Java, YARN, Kafka, Spark SQL, Aerospike, Cassandra, Delta lake, S3, Airflow, Sqoop, AWS, EMR, Data Bricks, NoSQL Hbase, and Apache Hive. Currently working as a Senior Data Engineer at MakeMyTrip, Sulbha is responsible for developing various batch and real-time pipelines to capture all booking level information across all lines of business for both brands (MMT and GI) using Spark, Scala, Kafka, Spark Streaming, S3, Delta/Parquet. She has also built data migration utility to sync MySQL table data to S3 on an hourly or daily basis using Spark, Scala. Sulbha has migrated 150+ tables from Hadoop Sqoop to AWS env and created airflow dags to launch EMR clusters and schedule ETL jobs on the basis of user-provided configurations. She has explored different types of instance groups and fleets for cost optimization of EMR cluster in AWS env. Sulbha has also developed various ETL jobs to compute business metrics (ATV, GMV, GR, SP) from multiple sources to easily track daily business numbers using Spark, Scala, and AWS services. She has built various dashboards to track the quality of master datasets using Glue and AWS Athena. Sulbha has moved the Adobe metrics capture process from a file-based system to API to reduce processing time and easily track the daily shopper's numbers for analyses. She has also built Compute Data Source utility so that users can schedule their complex Spark SQL queries on existing datasets and can persist the result to S3. Prior to working at MakeMyTrip, Sulbha was a Big Data Engineer at Hewlett Packard where she developed various data pipelines to aggregate and transform the incoming sensors data with the help of big data technologies. She also owned maintenance of multiple pipelines and ISHD support and explored new technologies like Linkedin Pinot to increase the performance of the overall product. Sulbha has also worked as a Senior System Engineer at Infosys where she modified Apache Spark to handle custom queries for authorization and developed features like saving the result of the query on spark, Hive tables