Senior Data Engineer
- Impetus Technologies
- 5 - 10 years
- United Arab Emirates
- 15 days ago
- Email to a friend
- Report this job
Job Description
The Opportunity:We are seeking a highly motivated and technically strong Module Lead Software Engineer with significant expertise in Python, PySpark, and Palantir Foundry. In this role, you will be responsible for the end-to-end technical ownership, design, and delivery of a specific module or component within our enterprise data platform. You will combine hands-on development with technical leadership, ensuring the highest standards of code quality, performance, and reliability.
Key Responsibilities:
- Module Technical Leadership & Ownership: Take full technical ownership of a specific module or component within the data platform on Palantir Foundry. This includes defining its technical roadmap, architecture, design patterns, and ensuring its integration into the broader data ecosystem.
- Hands-on Development and Complex Problem Solving: Act as a lead individual contributor, developing sophisticated data pipelines, transformations, and applications using Python and PySpark within Palantir Foundry's various tools (e.g., Code Workbook, Pipeline Builder). Tackle the most challenging technical problems and implement core functionalities for the module.
- Quality Assurance and Best Practices Advocacy: Drive and enforce high standards for code quality, test coverage, documentation, and operational excellence within your module. Conduct rigorous code reviews, provide constructive feedback, and mentor engineers within your immediate scope to elevate their technical skills.
- Cross-Functional Collaboration and Module Integration: Collaborate extensively with other module leads, architects, data scientists, and business stakeholders to ensure seamless integration of your module's deliverables. Proactively identify and manage technical dependencies and ensure the module aligns with overall project goals and architectural vision.
- Performance Optimization and Troubleshooting: Continuously monitor and optimize the performance of your module's data pipelines and applications. Efficiently troubleshoot and resolve complex technical issues, data quality concerns, and system failures specific to your module.
Required Qualifications:
- Experience: 6-8 years of progressive experience in software development with a strong focus on data engineering.
- Python Proficiency: Expert-level proficiency in Python, including advanced programming concepts, data structures, and performance optimization techniques.
- PySpark Expertise: Strong experience with PySpark for large-scale distributed data processing, transformations, and analytics.
- Palantir Foundry: Proven, hands-on experience designing, developing, and deploying solutions within Palantir Foundry is essential.
- Deep familiarity with Foundry's data integration capabilities, Code Workbook, Pipeline Builder, Data Health checks, and Ontology modeling.
- Experience with Foundry's approach to data governance and versioning.
- SQL Skills: Excellent SQL skills for complex data querying, manipulation, and optimization.
- Data Warehousing/Lakes: Solid understanding of data warehousing concepts, data lake architectures, and ETL/ELT principles.
- Cloud Platforms: Experience with at least one major cloud platform (AWS, Azure, GCP), particularly with data-related services.
- Version Control: Strong experience with Git and collaborative development workflows.
Preferred Qualifications (Nice-to-Have):
- Experience mentoring or leading small technical teams/pods.
- Familiarity with containerization technologies (Docker, Kubernetes).
- Experience with streaming data technologies (e.g., Kafka, Kinesis).
- Understanding of CI/CD pipelines for data solutions.
- Knowledge of data governance, data quality, and metadata management best practices.
- Experience in [specific industry, e.g., Financial Services, Manufacturing, Healthcare].
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Big Data EngineerEmployement Type: Full timeContact Details:
Company: Impetus TechnologiesLocation(s): United Arab Emirates
Keyskills: Pyspark Data Engineering Palantir Foundry Python Spark Data Warehousing SQL
Similar positions
Impetus Technologies
Impetus Technologies Impetus Technologies is a software products and services company focused on creating powerful and intelligent enterprises through deep data awareness, data integration and advanced data analytics. Our products and services are designed to empower the real-time data driven en...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in