About

Piyush Vaishya is a Data Engineer with a passion for distributed systems, scalable solutions, and automation. With 2.68 years of relevant experience, he is skilled in Azure Databricks, Spark, Python, SQL, and Automation. He is currently working as a Data Engineer at Neal Analytics, where he is responsible for designing fault-tolerant ADF pipelines that ingest data from various components like SQL Server, ADLS, and SFTP. Piyush's past experience includes working as a Specialist Programmer and Digital Specialist Engineer at Infosys Ltd. During his tenure at Infosys, he led the development of the ETL pipeline that automated the manual workflow using Azure Databricks. He also co-authored a SQL Server, Scala-Spark, and ADF-based Data Ingestion-Data Quality Framework that enabled metadata-based generic validation of the data. Additionally, he revamped an existing service using multithreading, which improved the response time by 21%. Piyush holds a Bachelor of Technology in Computer Science from Poornima University, where he graduated in 2020. He is skilled in Data Engineering, SQL, Databricks, Spark, Azure, Scala, Big Data, ETL, Python, Web, and Spring. He has strong analytical thinking skills and excellent storytelling skills, which enable him to communicate effectively. Piyush is a self-motivated individual who excels in working in an agile scrum methodology. He frequently interacts with clients for requirements gathering, designing solutions, status calls, and feedback. He has won the RISE INSTA Award for stepping up and delivering the work under stringent timelines to meet the project's major milestone. Piyush is dedicated to his work and has managed two projects simultaneously, receiving appreciation for his efforts.