Lead the design, development, and implementation of big data solutions using Apache Spark and Databricks.
Architect and optimize data pipelines and workflows to process large volumes of data efficiently.
Utilize Databricks features such as Delta Lake, Databricks SQL, and Databricks Workflows to enhance data processing and analytics capabilities.
Collaborate with data engineers, data scientists, and business stakeholders to understand data requirements and deliver high-quality data solutions.
Implement best practices for data engineering, including data quality, data governance, and data security.
Monitor and troubleshoot performance issues in Spark jobs and Databricks clusters.
Mentor and guide junior engineers in the team, promoting a culture of continuous learning and improvement.
Stay up-to-date with the latest advancements in Spark and Databricks technologies and incorporate them into the team's practices.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Analytics - OtherRole: Data Science & Analytics - OtherEmployement Type: Full time