About

Pavan Yadav is an experienced Data Engineer currently working at Jio. He holds a Master's degree in Computer Science from VIT University (Chennai Campus) and has over 5 years of experience in the IT industry. Pavan has a strong background in Scala development, with skills in Spark, Kafka, Hadoop, Docker, Google Cloud Platform, and Hive. He has experience in implementing, developing, and maintaining various applications using Spark, Scala, Hadoop Ecosystem, Kafka, Elasticsearch, and Kibana. He also possesses good knowledge of Hadoop architecture and daemons such as HDFS, Hive, and Spark. Pavan has expertise in configuring Hadoop, Kafka, Spark, and Elasticsearch clusters. He has developed a data pipeline using Kafka to store data into Elasticsearch, MySQL, and HDFS. He is also experienced in setting up MongoDB Shard Cluster and Hadoop Cluster on Docker using the Linux environment. Prior to his current role at Jio, Pavan worked as a Scala Developer at Aaum Research and Analytics, where he was responsible for creating and maintaining optimal data pipeline architecture, automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability. He also used SparkSql to aggregate data, set up and used MongoDB and MongoDB Shard Cluster, and was responsible for creating, managing, and monitoring the Hadoop Cluster. Pavan's education history includes a Master's degree in Computer Science and a Bachelor of Computer Application from Dr. Bhim Rao Ambedkar University. He also holds a second Master's degree from VIT University. Pavan's tech stack includes Data Engineering, Hadoop, Spark, Kafka, Scala, Elastic Search, HDFS, Docker, MongoDB, MySQL, Search, Apache Hive, and infra. He is a strong engineering professional with a demonstrated history of working in the management consulting industry.