About

Shivank Garg is a highly skilled Tech Lead with a strong background in data processing, governance, security, architecture, and visualization. He is currently serving as a Senior Software Engineer (SE IN4) at Walmart, where he is responsible for developing and implementing data engineering solutions using Spark Streaming for client-side events and session data, Kafka as a source for streaming jobs, and Grafana, Prometheus, Graphite Exporter for dash-boarding. Shivank has also worked with Filestreams (Spark DStreams) and external Offset Management in GCP, implemented his own streaming source for pushing metrics, and built a generic data validation module and plug-and-play generic mail agent for Airflow jobs. Prior to joining Walmart, Shivank worked as a Software Engineer at Paytm, where he owned the entire pipeline and API for Movies and Bus Cancellation Protect (CP) and developed an insight generation pipeline in Spark processing over 12,000+ dimensions and 500+ metrics. He has also reduced the first request time in Movies/Bus API to move them under ALB for autoscaling. Shivank holds a Bachelor's degree in Computer Science from Birla Institute of Technology and Science, Pilani. He is also proficient in various technologies such as Spark, Apache Airflow, Kafka, and GCP. Shivank has 6.20 years of relevant experience in the industry and is highly passionate about everything related to data. He is always eager to learn and implement new technologies to improve the overall data processing and governance.