Software Engineer - Data Engineer
- Beyond Key
- 4 - 9 years
- Indore
- 24 days ago
- Email to a friend
- Report this job
Job Description
We are looking for a Data Engineer with strong expertise in Databricks, PySpark, SQL, and Azure Data Factory (ADF) to design and optimize scalable data pipelines. Experience with Snowflake and DBT is a plus. The ideal candidate will have a proven track record in building efficient ETL/ELT processes, data warehousing, and cloud-based data solutions.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks.
- Process and transform large datasets using PySpark (DataFrames, Spark SQL, optimizations) in Databricks.
- Write and optimize complex SQL queries for data extraction, transformation, and loading.
- Implement data lakehouse architectures using Delta Lake in Databricks.
- (Optional) Manage and optimize Snowflake data warehouses (table structures, performance tuning).
- (Optional) Use DBT (Data Build Tool) for modular and scalable data transformations.
- Ensure data quality, governance, and monitoring across pipelines.
- Collaborate with data analysts and business teams to deliver actionable insights.
Qualifications:
- 4+ years of hands-on experience in Databricks, PySpark, SQL, and ADF.
- Strong expertise in Databricks (Spark, Delta Lake, Notebooks, Job Scheduling).
- Proficiency in Azure Data Factory (ADF) for pipeline orchestration.
- Advanced SQL skills (query optimization, stored procedures, partitioning).
- Experience with data warehousing concepts (ETL vs. ELT, dimensional modeling).
- (Good to Have) Familiarity with Snowflake (warehouse management, Snowpipe).
- (Good to Have) Knowledge of DBT (Data Build Tool) for transformations.
- (Bonus) Python scripting for automation and data processing.
Preferred Qualifications:
- Certifications (Databricks Certified Data Engineer, Azure Data Engineer).
- Experience with CI/CD pipelines (Azure DevOps, GitHub Actions).
- Knowledge of streaming data (Kafka, Event Hubs) is a plus.
Job Classification
Industry: Software ProductFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data Platform EngineerEmployement Type: Full timeContact Details:
Company: Beyond KeyLocation(s): Indore+ View Contact
Keyskills: Performance tuning Automation Scheduling Data quality Stored procedures microsoft Monitoring Software services SQL Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in