About

Vinay Kumar is a highly skilled Big Data Consultant with over 10 years of experience in software development and 3+ years of experience as a Big Data Developer. He has a strong background in developing and maintaining big data projects and data pipelines, with a focus on performance optimization using Hadoop Ecosystem and Apache Spark. Vinay has also gained exposure to big data on cloud with AWS EMR, AWS Redshift, AWS Glue, and AWS S3. Throughout his career, Vinay has held various roles and responsibilities, including using Sqoop to efficiently transfer data between MySQL database and HDFS with incremental load, working on various file formats such as Avro, Orc, and Parquet, and implementing partitioning and bucketing for best practices and improving performance of Hive queries. He has also been involved in creating Hive-Hbase special table for faster access of customers lookup data and has exposure to Kafka cluster, involved in connecting various Kafka Topics. Vinay has a strong technical skill set that includes writing Scala code using Spark, Spark SQL, and Spark Streaming. He has also been involved in creating Jars and deploying code using spark-submit. Vinay's experience also includes performance optimization techniques such as using Broadcast Join, Partitioning & Bucketing on join columns, and using Dataset API to leverage Tungsten Optimization provided by Spark. Vinay has worked with several companies throughout his career, including Amazon, Wells Fargo, and Accenture. He is currently working as a Big Data Consultant at Amazon. Vinay holds a Bachelor's degree in Information Technology from Synergy Institute of Engineering & Technology. Vinay has a strong tech stack that includes AWS, Big Data, Spark, Kafka, Apache Airflow, Hadoop, Scala, SQL, Python, Apache Hive, AWS Glue, Software Engineer, HDFS, and MySQL. With 11.52 years of relevant experience, Vinay is a valuable addition to any team looking to optimize their big data projects and data pipelines.