About

Rayudu S is a highly skilled Sr Big Data Engineer with almost 10 years of experience in the field of Data Engineering and Big Data. He is currently working at TCS as a Sr Big Data Engineer and is responsible for the Comcast Data Engineering Analyses Platform project. Rayudu has extensive knowledge of Hadoop and its various components such as HDFS, MapReduce, Hive, Hbase, Pig, Sqoop, Flume, Oozie, Core Java, MySQL, Kafka, MongoDB, Spark, and Scala. In his current role, Rayudu has been involved in exploring and debugging Pig scripts and creating new Pig scripts to filter data from HDFS logs. He has also created Pentaho jobs and trans for loading data into Oracle tables and has loaded HDFS data to AWS S3 using Spark. Rayudu has been responsible for preparing test cases for testing the data loaded and has been involved in the entire process from data cleansing to data visualization using Tableau. In his previous role as a Software Engineer at TCS, Rayudu was responsible for loading files to HDFS and writing Hive queries to process required data. He maintained and administered the 6-node Hadoop cluster and worked on setting up Hadoop over multiple nodes. Rayudu has designed and developed Java MapReduce jobs and has been involved in loading data from the Linux file system to HDFS. He has implemented test scripts to support test-driven development and continuous integration and has managed data coming from different sources. Rayudu has a Bachelor's degree in Electronics and Communications Engineering from JNTU and has a strong understanding of Data Engineering, Big Data, Hadoop, MapReduce, Spark, Apache Hive, AWS, Java, MySQL, MongoDB, Kafka, and Scala. He has experience in managing and reviewing Hadoop log files and has worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files. Rayudu has also managed and scheduled Jobs on a Hadoop cluster and has installed and configured Hadoop MapReduce, HDFS, and developed multiple MapReduce jobs in Java for data cleaning and preprocessing. He has imported and exported data into HDFS and Hive using Sqoop and has written Hive queries for data analysis to meet business requirements.