About

Shivam Shrivastava is an experienced Data Engineer with over 3 years of experience in creating data pipelines, enhancing existing processes, automating tasks, and performing exploratory data analysis to provide insights to clients. He is skilled in Python, SQL, Spark, AWS, and Databricks. Shivam is currently working at Fractal as a Data Engineer. In his current role, Shivam has achieved several key accomplishments. He converted a data lake to a relational database, which saved $28K in yearly costs. The table that was converted was used extensively on a daily basis, and the querying cost was high since the data was stored in an unstructured format. Converting it into delta tables enhanced the query time by a factor of 300, hence the computation cost decreased. Shivam has also developed multiple automations to complete day-to-day tasks, which helped in saving manual labor by 3-4 hours per day. Shivam is interested in exploring the field of Machine Learning and has relevant experience in Data Engineering, AWS, SQL, Databricks, Python, Spark, ML, Software Engineering, Apache Airflow, Amazon Athena, Perceptron, and Frontend. Before joining Fractal, Shivam worked at ZS Associates as a Business Technology Solutions Associate. He was responsible for creating data pipelines to transfer data between upstream and downstream systems, automating day-to-day tasks, and analyzing data sets to provide insights upon them. Shivam converted a data lake to a relational database, which saved $28K in yearly costs. He also developed multiple automations to complete day-to-day tasks, which helped in saving manual labor by 3-4 hours per day. Shivam has a Bachelor of Engineering degree in Computer Science from Shri Ram Institute of Technology. In his free time, he writes blogs on WordPress and Medium. You can find his Medium blog at https://intellectualappetizer.medium.com/ and his WordPress blog at https://evisagecanvas.wordpress.com/.