Lead Data engineer Job in Cynosure Corporate
- Cynosure Corporate
- 8 to 12 Yrs
- Chennai
- 5 months ago
- Email to a friend
- Report this job
Job Description
- Working with clients to understand their data.
- Basedontheunderstanding you will be building the data structures and pipelines.
- You will be working on the application from end to end collaborating with UI and other development teams.
- You will be responsible for building the data pipelines to migrate and load the data into the HDFSeither on-prem or in the cloud.
- Developing Data ingestion/process/integration pipelines effectively.
- Creating Hive data structures,metadata and loading the data into data lakes / Big Data warehouse environments.
- Optimized(Performance tuning) many data pipelines effectively to minimize cost.
- Codeversioning control and git repository is up to date.
- You will be responsible for building and maintaining CI/CD of the data pipelines.
- You will be managing the unit testing of all data pipelines
- Bachelors degree in computer science or related field.
- Minimum of 5+years working experience with Spark, Hadoop eco systems.
- Minimum of 4+years working experience on designing data streaming pipelines.
- Should be an expertin either Python/Scala/Java.
- Should have experience in Data Ingestion and Integration into data lake using hadoop ecosystem tools such as Sqoop, Spark, SQL, Hive, Airflow, etc.
- Should have experience optimizing (Performance tuning) data pipelines.
- Minimum experience of 3+ years on NoSQL and Spark Streaming.
- Knowledge of Kubernetes and Docker is a plus.
- Should have experience with Cloud services either Azure/AWS.
- Should have experience with on-prem distribution such as Cloudera/HortonWorks/MapR.
- Basic understanding of CI/CD pipelines.
- Basic knowledge of Linux environment and commands.
Key responsibilities:
Skills & Experience:
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: IT Operations / EDP / MISFunctional Area: Not SpecifiedRole/Responsibilies: Lead Data engineer Job in Cynosure CorporateContact Details:
Company: Cynosure CorporateLocation(s): Chennai+ View Contact
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Cynosure Corporate
Cynosure Corporate Solutions is a human capital services company. Our focus is to provide Executive Search, Recruitment, Training, Temporary Staffing services, Statutory Compliance's and other HR aligned services to client organizations. We understand the business goals of our clients and their ne...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in