Python ETL Automation 3 to 6 yrs (Pune)
- Krivi's Cafe
- 3 to 7 Yrs
- 1 month ago
- Email to a friend
- Report this job
Job Description
- We are seeking a skilled Python ETL Engineer to join our team. The successful candidate will focus on automating data ingestion, processing, and validation workflows using PySpark, ensuring robust, scalable, and efficient data pipelines. You will work closely with data engineers, analysts, and stakeholders to deliver impactful data solutions in a fast-paced environment. Job Location - Pune Key Responsibilities Collaborate with data engineers and stakeholders to define requirements and deliver automated solutions. Design, develop, and maintain scalable and efficient PySpark-based automation for data ingestion, processing and calculation. Automate the reading and integration of data from multiple sources, such as databases, APIs, flat files, and cloud storage. Implement and optimize ETL (Extract, Transform, Load) workflows for high-performance data pipelines. Ensure data quality by incorporating validation checks and handling exceptions in data processing pipelines. Troubleshoot and resolve issues in data pipelines to maintain operational efficiency. Required Skills and Qualifications 3+ years strong proficiency in Python for data handling and validation. Strong experience in Python libraries like, pandas, NumPy, duckdb etc. Familiarity with cloud platforms such as AWS, Azure, or GCP (e.g., S3, Databricks, or BigQuery). Experience with data pipeline orchestration tools such as Apache Airflow, or similar. Proficiency in SQL for querying and manipulating data. Experience in handling structured, semi-structured, and unstructured data. Familiarity with CI/CD processes and version control tools such as Git. Knowledge of performance tuning and optimization techniques in PySpark. Strong analytical and problem-solving skills. NICE TO HAVE knowledge of Spark architecture, including RDDs, DataFrames, and Spark SQL Knowledge of Keyword driven automation framework. Quality Engineering background with prior experience of building automation solutions for data heavy applications. Familiarity with REST APIs and data integration techniques. Understanding of data governance, compliance, and security principles. Looking for immediate joiners ONLY,
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: Not SpecifiedFunctional Area: Not SpecifiedRole/Responsibilies: Python ETL Automation 3 to 6 yrs (Pune)Contact Details:
Company: SpurqlabsLocation(s): Maharashtra,+ View Contact
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in