Data Engineer
- Clairvoyant India
- 5 - 9 years
- Hyderabad
- 3 months ago
- Email to a friend
- Report this job
Job Description
Role: Data Engineer
Experience: 6+ Years
Location: Pune, Gurgaon & Bangalore Hybrid
Shift Time: 12:00 PM - 10:00 PM
Must have:
- Experience working in AWS, Redshift, Glue
- Prior exposure to Data Ingestion and Curation work (such as working with Data Lakehouse)
- Knowledge in SQL for purpose of data analysis/investigation
- Work with the Data Product Owners and Data Analysts to help manage and deliver on the product technical roadmap for the data platform
Good to have:
- Degree in computer science, statistics, or related discipline
- 6+ years as a data engineer
- Comfortable making decisions and leading
- Familiar with version control and relational databases
- Superior communication skills both oral and written
- Positive contributor, strong team member, loves to work with and empower others
- Collaborating with a team
- Time management skills
Responsibilities:
- Coordinate data scientists/analysts, product managers and business leaders to understand data needs and deliver on those needs and set expectations with Product Managers/Analysts.
- Build the infrastructure for optimal extraction, transformation and loading data from a wide variety of data sources using big data technologies and integration across different systems and platforms
- Develop, build, optimize and implement pipelines, data ingestion, storage, wrangling, cataloguing, quality, security features from various data sources on cloud
- Automate jobs (ingestion & pipelines), notifications and reports
- Write quality code and participate in code reviews
- Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD
- Continuously improving systems through performance enhancements and cost reductions in compute and storage
- identify opportunities for improvement in engineering practices and software delivery
- Implement end to end Machine Learning workflows using MLFlow, including model training tracking and deployment ensuring scalability and performance optimization
- Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs
- Prioritizing to manage ad-hoc requests in parallel with ongoing sprints
- Participate with the team to execute sound solutions and approaches to meet business expectations in an efficient manner; deliver and execute the technical and product roadmap for data engineering across a variety of technologies.
- Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Machine LearningRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Clairvoyant IndiaLocation(s): Hyderabad+ View Contact
Fraud Alert to job seekers!
₹ 15-25 Lacs P.A
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in