About

Karthik Shetty is an Associate Technical Architect at Quantiphi with over 6 years of experience in Business Intelligence and Analytics. He is an Amazon Certified Solution Architect and Amazon Certified Big Data Specialty professional who has worked extensively in Media & Entertainment and Manufacturing domains. Karthik is a team player who believes in collaborative work for successful product delivery. He has strong customer interaction, visibility, and stakeholder management experience. Karthik is a skilled Data Engineer with good experience in data pipelining and various ELT/ETL processes. He is proficient in PySpark for structured and semi-structured data processing and has hands-on experience in building and managing a 1.3PB Data-Warehouse. He is proficient in SQL and has hands-on experience with storage tiers like AWS Redshift, Redshift Spectrum, and AWS Dynamo DB with performance tuning, cost optimization, and workload management. Karthik has worked on On-prem Databases like IBM DB2 and Microsoft SQL Server. Karthik has hands-on experience in ETL tools like IBM Datastage and Data Visualization tools like IBM Cognos, Periscope, Apache Superset, and Tableau. He has sound knowledge in UNIX and UBUNTU. Karthik has worked on AWS Athena, EMR, Step function, AWS Lambda, and SQS. At Quantiphi, Karthik has designed and developed end-to-end near real-time architecture, a pro-serve engagement with AWS to bring their fabrication unit data from across the globe to the cloud with an SLA of 30 mins. He has also developed and implemented an end-to-end decoupled AWS event-driven architecture, starting from landing bucket through S3 event notifications for pipeline triggers, transformation through optimized EMR cluster configuration and lambda's driven by configuration, orchestration through step function to load into Redshift and use DynamoDB tables for logging and monitoring. Previously, Karthik worked as a Senior Data Engineer at ardentisys - supercharging software and data. He designed & developed scalable, highly available, and fault-tolerant systems on AWS for managing 1.3PB data warehouse using Amazon Redshift and managing user, roles, permissions, snapshots. He also has experience in almost all redshift cluster types including DS28XL, DC28XL, and RA316XL. He has hands-on experience with serverless AWS Athena for adhoc-query analysis and AWS Glue Data Catalog along with Crawler. Karth