Sr. Databricks Developer
- Newscape Consulting
- 7 - 9 years
- Pune
- 4 days ago
- Email to a friend
- Report this job
Job Description
We are looking for a highly skilled Senior Databricks Developer to join our data engineering team. You will be responsible for building scalable and efficient data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure/AWS/GCP). You will work closely with data architects, data scientists, and business stakeholders to deliver high-performance, production-grade solutions.
Key Responsibilities :
- Design, build, and maintain scalable and efficient data pipelines on Databricks using PySpark, Spark SQL, and optionally Scala.
- Work with Databricks components including Workspace, Jobs, DLT (Delta Live Tables), Repos, and Unity Catalog.
- Implement and optimize Delta Lake solutions aligned with Lakehouse and Medallion architecture best practices.
- Collaborate with data architects, engineers, and business teams to understand requirements and deliver production-grade solutions.
- Integrate CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or similar for Databricks deployments.
- Ensure data quality, consistency, governance, and security by using tools like Unity Catalog or Azure Purview.
- Use orchestration tools such as Apache Airflow, Azure Data Factory, or Databricks Workflows to schedule and monitor pipelines.
- Apply strong SQL skills and data warehousing concepts in data modeling and transformation logic.
- Communicate effectively with technical and non-technical stakeholders to translate business requirements into technical solutions.
Required Skills and Qualifications :
- Hands-on experience in data engineering, with specifically in Databricks.
- Deep expertise in Databricks Workspace, Jobs, DLT, Repos, and Unity Catalog.
- Strong programming skills in PySpark, Spark SQL; Scala experience is a plus.
- Proficient in working with one or more cloud platforms : Azure, AWS, or GCP.
- Experience with Delta Lake, Lakehouse architecture, and medallion architecture patterns.
- Proficient in building CI/CD pipelines for Databricks using DevOps tools.
- Familiarity with orchestration and ETL/ELT tools such as Airflow, ADF, or Databricks Workflows.
- Strong understanding of data governance, metadata management, and lineage tracking.
- Excellent analytical, communication, and stakeholder management skills.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Newscape ConsultingLocation(s): Pune
Keyskills: Azure Data Factory Pyspark Azure Databricks ETL Delta Lake Azure Data Lake Apache Data Bricks
Similar positions
Newscape Consulting
We create beautiful experiences that drive successful businesses!Change, Transform, Innovate, Scale. - These words embody the essence of what we do help top organizations stay competitive, win new markets and increase shareholder value by leveraging the full potential of the latest disruptive techn...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in