Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Hadoop Big Data Engineer @ D2C Insurance Broking

Home >

 Hadoop Big Data Engineer

Job Description

    As an experienced Big Data Engineer with 6 to 8 years of experience, you will be responsible for designing, developing, and maintaining robust and scalable data processing applications using Apache Spark with Scala. Your key responsibilities include implementing and managing complex data workflows, conducting performance optimization and memory tuning for Spark jobs, and working proficiently with Apache Hive for data warehousing and HDFS for distributed storage within the Hadoop ecosystem. You will utilize your strong hands-on experience with Apache Spark using Scala, Hive, and HDFS, along with proficiency in Oozie workflows, ScalaTest, and Spark performance tuning. Your deep understanding of Spark UI, YARN logs, and debugging distributed jobs will be essential in diagnosing and resolving complex issues in distributed data systems to ensure data accuracy and pipeline efficiency. In addition, you will demonstrate expertise in writing comprehensive unit tests using ScalaTest and following best practices for building scalable and reliable data pipelines. Your familiarity with Agile/Scrum methodologies will enable you to work within an Agile/Scrum environment, collaborating with cross-functional teams to deliver high-quality data solutions. Moreover, your working knowledge of CI/CD pipelines, GitHub, Maven, and Nexus will be utilized for continuous integration, delivery, and version control. Your ability to write unit tests and follow best practices for scalable data pipelines will contribute to the successful execution of data projects in an enterprise-level distributed data system environment. Overall, you will play a crucial role in the development, optimization, and maintenance of data processing applications, ensuring high performance, reliability, and efficiency in data workflows within the Big Data platform. (Note: This job description is based on the provided information and is intended to give a standard summary description in the second person format. Headers have been omitted for the final JD.),

Employement Category:

Employement Type: Full time
Industry: IT Services & Consulting
Role Category: Not Specified
Functional Area: Not Specified
Role/Responsibilies: Hadoop Big Data Engineer

Contact Details:

Company: Tekit Software
Location(s): Karnataka

+ View Contactajax loader


Keyskills:   Apache Spark Scala Hive HDFS Oozie GitHub Maven YARN Nexus ScalaTest

 Fraud Alert to job seekers!

₹ Not Disclosed

Similar positions

Oracle Database Administrator

  • Solarzo Technology
  • 3 to 7 Yrs
  • Ahmedabad
  • 2 days ago
₹ Not Disclosed

Software Engineer 2

  • Solarzo Technology
  • 3 to 7 Yrs
  • Bilaspur
  • 2 days ago
₹ Not Disclosed

Cloud & AI Solution Engineer - Software,

  • Solarzo Technology
  • 3 to 7 Yrs
  • karnataka
  • 2 days ago
₹ Not Disclosed

Sr. Quality Engineer w/ TestCafe

  • NTT DATA
  • 2 to 6 Yrs
  • All India
  • 2 days ago
₹ Not Disclosed

D2C Insurance Broking

ABOUT RENEWBUY  Launched in 2015, by industry veterans, RenewBuy.Com is the fastest growing online aggregator in India. A pure digital insurance distribution business, it has become the 2nd largest and the fastest-growing Insure-Tech player by building an extremely differentiated approach from inc...