About

Amish Agarwal is a highly skilled Data Engineer with over 4 years of experience in designing and developing on-premise and cloud-based Big Data pipelines. He is an expert in Data Ingestion, Transformation, Extraction, and Cleansing, ensuring Data quality and Code quality. Amish has expertise in performance tuning of Hadoop and Spark applications and has worked with AWS cloud technologies like EC2, EMR, S3, and Redshift. He is able to assess business rules, collaborate with stakeholders and provide end-to-end solutions. Amish has worked with structured and semi-structured data formats like Parquet, ORC, XML, and CSV. He has worked with a wide variety of tools and technologies such as Hadoop, Spark, HIVE, Map Reduce, HDFS, YARN, Shell Scripting, languages like Python and Scala, database like SQL server, data warehouse like Redshift. He is experienced in working with Atlasian Suite, which involves JIRA, BitBucket, Nexus, Bamboo, and Confluence, with Git as version control. Amish has performed BI reporting or dash boarding using Spotfire and has expertise in working with job schedulers like BMC Control-M and SOS-berlin Job Scheduler. He has excellent communication, interpersonal, analytical skills, and a strong ability to perform as part of a team or lead the Team. He has an exceptional ability to learn new concepts, is hardworking, and enthusiastic. Amish is trained on SCRUM and Agile Methodology. Amish is currently working as a Data Engineer at iptiQ by Swiss Re. He has previously worked as a Data Engineer at Syntel and as a Big Data Developer at Perkinelmer, Inc. At Syntel, he developed Spark ETL jobs and shell scripts for data loading and transformation from scratch. He was responsible for requirement gathering from business and defining end-to-end architecture. He worked with different file formats like Parquet, XML, and CSV, and also performed Data warehousing and Data modelling. Amish developed and presented reports and dashboards in Spotfire and managed code, repository, version control, and auto production deployment using Bitbucket, GIT, and Bamboo CI/CD pipeline. He actively participated and contributed in all scrum ceremonies and documented automated systems, user guides, and instructions on Wiki (Confluence). At Perkinelmer, Inc., Amish worked with AWS cloud technologies (S3, EC2, EMR