About

Vishal Khawarey is an accomplished Engineering Leader with over 12 years of experience in architecting full stack services and big data pipelines in the cloud. Currently serving as the Lead Software Engineer at 23andme, he has proven expertise in Agile Sprint planning and Project management, working with stakeholders across multiple teams. At 23andme, Vishal developed a microservice to query user relatives from a database with trillions of rows, designed a circuit breaker with RESTful APIs for limiting queries to the query server, and integrated this with a bulk email notification microservice. He also architected the data ingestion pipeline with automated backups and ingestion into multiple mirror clusters on AWS EMR and tuned the JVM Garbage collector (G1) in HBase. Additionally, Vishal developed a microservice to perform user segmentation on a Petabyte scale data lake, using Spark ETL pipelines to perform Machine learning computes such as regression, principal components, decision trees. Vishal’s experience with Docker container based deployments, running CI/ CD pipelines on Jenkins, and Elastic Kubernetes service is extensive. He has implemented monitoring and logging for all his services and led the on-call operations. Currently, he is working on a full-stack product deploying serverless Docker micro-services on ECS (Elastic Container Service) Fargate cluster. Before joining 23andme, Vishal served as a Staff Software Engineer at MapR Technologies, where he developed a service to provision complex Hadoop topologies on hundreds of machines and worked with Solution architects for deployments at customers like Audi, Ericsson, etc. He also supported customizations to HDFS, Hive, Drill, HBase, Elasticsearch, etc. Vishal holds a Master's Degree in Electrical Engineering from North Carolina State University and a Bachelor's Degree in Electronics & Communication Engg. from NIT Warangal. He is proficient in AWS, Software Engineering, Big Data, Microservices, Docker, ETL, Hadoop, ECS, Spark, MapReduce, Java, Kubernetes, ML, Jenkins, HDFS, Ansible, Elastic Search, Azure, and Apache Hive. Overall, he has 14.62 years of relevant experience and is a Full Stack, Big Data, AWS Cloud Leader.