About

Swapnik Chimalamarri is an experienced End to End Data Scientist with over 6 years of experience in Big Data technologies, Machine Learning Algorithms, and Query Languages. He is a Solutions Lead who is looking for a challenging role at an organization in the fields of Data Science and Big Data. Currently, he is working as a Data Engineer III at slice. Swapnik has a strong background in designing and implementing CDC workflows using AWS DMS, SQS, Lambda, and S3. He has handled streaming data using autoloaders in Databricks, which reduced the end report refresh rate by 30%. He has also utilized Spark Streaming with Kinesis to transform and load data to multiple sinks (Redshift and S3). He is solely responsible for the end-to-end architecture for handling CDC data from various sources like MySQL, Postgres, DynamoDB, and has designed and triggered corresponding ETL jobs using Databricks using pool instances to reduce the starting time of a job. Swapnik has closely worked with the Machine Learning team to build feature stores in Druid and connecting to data with Databricks. He has handled ML Engineering Pipelines in Databricks using MLFlow where models are continuously logged for different metrics. He is responsible for optimizing existing spark jobs, which reduced the infrastructure cost by 15% by utilizing the correct resources in Databricks. Swapnik has also reduced the failure times and cost of infrastructure by 30% by porting On-Prem Kafka Dynamo DB pipelines to Kinesis and consequently push to Kinesis Firehose to S3. He has created Databricks ETL jobs and scheduled them for different data sources, including S3, EFS, and DynamoDB. He has also orchestrated Databricks pipelines using SQS, Lambda, and Airflow, trying to move existing Databricks architecture to AWS Glue. Swapnik has a Master of Science in Data Science from Liverpool John Moores University and a Post Graduate Diploma in Data Science from IIIT Bangalore. He also has a Master of Science in Integrated Circuit Design from Nanyang Technological University and Technical University Munich. Additionally, he has a Bachelor's degree in Electrical, Electronics, and Communications Engineering from Jawaharlal Nehru Technological University, Kakinada. He has also completed the PGDDS C6 Course 4 & 5 from IIIT Bangalore. Swapnik is proficient in AWS, ML, Data Engineering, Databricks, Big Data, Spark,