Data Engineer
- Atyeti
- 5 - 10 years
- Noida, Gurugram
- 1 day ago
- Email to a friend
- Report this job
Job Description
Role & responsibilities
Key Responsibilities
- Design, build, and maintain scalable and efficient data pipelines to move data between cloud-native databases (e.g., Snowflake) and SaaS providers using AWS Glue and Python
- Implement and manage ETL/ELT processes to ensure seamless data integration and transformation
- Ensure information security and compliance with data governance standards
- Maintain and enhance data environments, including data lakes, warehouses, and distributed processing systems
- Utilize version control systems (e.g., GitHub) to manage code and collaborate effectively with the team
Primary Skills:
- Enhancements, new development, defect resolution, and production support of ETL development using AWS native services
- Integration of data sets using AWS services such as Glue and Lambda functions.
- Utilization of AWS SNS to send emails and alerts
- Authoring ETL processes using Python and PySpark
- ETL process monitoring using CloudWatch events
- Connecting with different data sources like S3 and validating data using Athena.
- Experience in CI/CD using GitHub Actions
- Proficiency in Agile methodology
- Extensive working experience with Advanced SQL and a complex understanding of SQL.
Secondary Skills:
- Experience working with Snowflake and understanding of Snowflake architecture, including concepts like internal and external tables, stages, and masking policies.
Competencies / Experience:
- Deep technical skills in AWS Glue (Crawler, Data Catalog): 5 years.
- Hands-on experience with Python and PySpark: 3 years.
- PL/SQL experience: 3 years
- CloudFormation and Terraform: 2 years
- CI/CD GitHub actions: 1 year
- Experience with BI systems (PowerBI, Tableau): 1 year
- Good understanding of AWS services like S3, SNS, Secret Manager, Athena, and Lambda: 2 years
- Additionally, familiarity with any of the following is highly desirable: Jira, GitHub, Snowflake
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: AtyetiLocation(s): Noida, Gurugram+ View Contact
Keyskills: Pyspark Aws Lambda Amazon Redshift Aws Glue Python
Fraud Alert to job seekers!
₹ 20-25 Lacs P.A
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in