About

Divay Garg is a Cloudera Certified Spark & Hadoop Developer with 4.78 years of relevant experience in the field of Big Data technologies. He has a strong background in working with Scala, Python, Hbase, Kafka, Hive, and Apache Spark. Divay has extensive experience in design and development of multiple projects using Agile Methodology. He is skilled in extracting data from diverse sources including Structured, Unstructured, and Semi-structured data like Hbase, SQL, Hive, Impala, RDBMs, and MongoDB. He also has scripting and application development experience in Shell Scripts, Python, Github, Bitbucket, Cloudera, and Data Pipeline. Currently, Divay is working as a Senior System Engineer at Infosys. In his current role, he is responsible for developing end-to-end programs for client Citi Bank to store, process, and manage huge amounts of data in day-to-day and real-time operations collected from various heterogeneous sources. He has worked on several projects, including creating a Spark batch job to read XML files from a file path, enriching and extracting critical data, moving file details from HDFS to Kafka, developing a Spark streaming job which processes real-time messages from Kafka in JSON format, developing a Spark job to match and enrich data from different sources and sending data through Kafka, and creating and maintaining HBase Table to store data from different sources. Divay has also worked on performance optimization of Impala view and Spark Job, reducing job time by 200%. He has developed HQL Script for analysis of data in Hive Tables, performed data enrichment such as filtering, format modeling, sorting, and aggregation, and developed managed and external Hive tables with appropriate partitioning. Divay holds a Bachelor of Technology degree in Electrical, Electronics and Communications Engineering from Maharaja Agrasen Institute of Technology. He has also completed his science degree from Bal Baharati Public School. His technical skills include Spark, Hadoop, Kafka, Data Engineering, Big Data, SQL, Apache Hive, Python, Scala, MongoDB, HDFS, and ETL.