Around 8+ years of professional experience in Software development with 5+years of experience in Bigdata technologies including Hadoop and Spark.
• Professional Java developer with strong expertise in data engineering and big data technologies.
• Extensively worked on Spark, Hive, Pig, MapReduce, Sqoop, Kafka, Oozie, HBase, Impala and Yarn.
• Hands on experience in programming using Java, Python, Scala and SQL.
• Sound knowledge of architecture of Distributed Systems and parallel processing frameworks.
• Designed and implemented end-to-end data pipelines to processes and analyze massive amounts of data.
• Experienced working with Hadoop distributions both on-prem (CDH, HDP) and in cloud (AWS).
• Good experience working with various data analytics and big data services in AWS Cloud like EMR, Redshift, S3, Athena, Glue etc.,
• Experienced in developing production ready spark application using Spark RDD Apis, Data frames, Spark-SQL and Spark-Streaming API's.
• Worked extensively on fine tuning spark applications to improve performance and troubleshooting failures in spark applications.
• Strong experience in using Spark Streaming, Spark Sql and other components of spark like accumulators, Broadcast variables, different levels of caching and optimization techniques for spark jobs
• Proficient in importing/exporting data from RDBMS to HDFS using Sqoop.
• Used hive extensively to performing various data analytics required by business teams.
• Solid experience in working various data formats like Parquet, Orc, Avro, Json etc.,
• Experience automating end-to-end data pipelines with strong resilience and recoverability.
• Strong knowledge of NoSQL databases and worked with HBase, Cassandra and Mongo DB.
• Extensively used various IDE's like IntelliJ, NetBeans and Eclipse
• Expert in SQL, extensively worked RDBMSs like Oracle, SQL Server, DB2, MySQL and Teradata
• Worked with Apache Nifi to ingest the data into HDFS from variety of sources
• Proficient and Worked with GIT, Jenkins and Maven.
• Good understanding and Experience with Agile and Waterfall methodologies of Software Development Life Cycle (SDLC).
• Highly motivated, self-learner with a positive attitude, willingness to learn new concepts and accepts challenges. Big Data Ecosystems : HDFS, MapReduce, YARN, Hive, Sqoop, Pig, Spark HBase, Oozie.