About

Anil Kumar Bairy is a highly experienced Senior Software Engineer with over 8 years of experience in the field of Big Data. Currently working at Emirates, Anil has a strong technical background in various data ingestion tools such as Sqoop and Nifi. He has also written optimized Spark applications in Scala for performing aggregation using DataFrame and Dataset API's and loading data into Hive. Anil has experience working with Apache Kafka and various file formats for storing data in Hive/Hadoop. He is skilled in loading streaming data into Spark from Kafka as a data source and implementing transformations using Spark Dataset API. Anil has also worked on Logstash and Filebeat to capture streaming files and sending them to Kafka. Anil has a proven track record of working on complex projects. His most notable projects include Payments Analytics and Apache Tomcat Web Server Log Analyzer. In Payments Analytics, he wrote Sqoop commands to load data from Oracle/SQL Server into Hive tables and performed data cleansing and data quality checks. He implemented transformations using Spark Dataset API and ensured timely checking for quality of data. Anil also automated regular jobs using Python and reviewed and managed Hadoop log files. In Apache Tomcat Web Server Log Analyzer, he created different topics in Kafka and wrote Spark Streaming applications to collect data from Kafka topics. He created dashboards in Apache Zeppelin for the data stored in Hive. Anil has worked with various companies in the past, including ACL Digital and Neosoft. At ACL Digital, he worked on technologies like Hadoop, Spark, Kafka, Nifi, Filebeats, and Logstash. At Neosoft, he worked on multiple projects, including Payments Analytics and Apache Tomcat Web Server Log Analyzer. Anil holds an Engineer’s Degree in Computer Science and Engineering from SR Engineering College, Warangal. He also has a strong skill set in software engineering, Spark, Kafka, Big Data, Apache Hive, Hadoop, SQL, Python, Scala, HDFS, and Web. With his relevant experience and technical expertise, Anil is a valuable asset to any organization he works for.