Azure Data Engineer
- Cloudaeon
- 5 - 10 years
- Pune
- 3 months ago
- Email to a friend
- Report this job
Job Description
Sr Azure Data Engineer
About Cloudaeon:
Cloudaeon is a global technology consulting and services company. We support companies in managing cloud infrastructure and solutions with the help of big data, DevOps and analytics. We offer first-class solutions and services that use big data and always exceed customer expectations. Our deep vertical knowledge, combined with expertise in several enterprise- class big data platforms, helps develop targeted solutions to meet our customers' business needs. Our global team consists of experienced professionals with experience in various tech stacks. Every member of our team is very active and committed to helping our customers achieve their goals.
Job Role:
We are looking for a Senior Azure Data Engineer with overall 5+ years of experience to join our team. The ideal candidate should have expertise in Azure Data Factory (ADF), Databricks, SQL, Python, and experience working with SAP IS-Auto as a data source. This role involves data modeling, systematic layer modeling, and ETL/ELT pipeline development to enable efficient data processing and analytics. You will use various methods to transform raw data into useful data systems. Overall, you will strive for efficiency by aligning data systems with business goals.
Responsibilities:
- Develop & Optimize ETL Pipelines: Build robust and scalable data pipelines using ADF, Databricks, and Python for data ingestion, transformation, and loading.
- Data Modeling & Systematic Layer Modeling: Design logical, physical, and systematic data models for structured and unstructured data.
- Integrate SAP IS-Auto: Extract, transform, and load data from SAP IS-Auto into Azure-based data platforms.
- Database Management: Develop and optimize SQL queries, stored procedures, and indexing strategies to enhance performance.
- Big Data Processing: Work with Azure Databricks for distributed computing, Spark for large-scale processing, and Delta Lake for optimized storage.
- Data Quality & Governance: Implement data validation, lineage tracking, and security measures for high-quality, compliant data.
- Collaboration: Work closely with business analysts, data scientists, and DevOps teams to ensure data availability and usability.
Requirements:
- Azure Cloud Expertise: Strong experience in Azure Data Factory (ADF), Databricks, and Azure Synapse.
- Programming: Proficiency in Python for data processing, automation, and scripting.
- SQL & Database Skills: Advanced knowledge of SQL, T-SQL, or PL/SQL for data manipulation.
- SAP IS-Auto Data Handling: Experience integrating SAP IS-Auto as a data source into data pipelines.
- Data Modeling: Hands-on experience in dimensional modeling, systematic layer modeling, and entity-relationship modeling.
- Big Data Frameworks: Strong understanding of Apache Spark, Delta Lake, and distributed computing.
- Performance Optimization: Expertise in query optimization, indexing, and performance tuning.
- Data Governance & Security: Knowledge of RBAC, encryption, and data privacy standards.
- Strong problem-solving skills coupled with good communication skills.
- Open minded, inquisitive, life-long learner.
- Good conversion of high-level business & technical requirements into technical specs.
- Feeling comfortable in using Azure cloud technologies.
- Customer centric, passionate about delivering great digital products and services.
Preferred Qualifications:
- Experience with CI/CD for data pipelines using Azure DevOps.
- Knowledge of Kafka/Event Hub for real-time data processing.
- Experience with Power BI/Tableau for data visualization (not mandatory but a plus).
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: CloudaeonLocation(s): Pune
Keyskills: Azure Data Factory Etl Pipelines Sap Is Azure Cloud Azure Databricks Pyspark Ci Cd Pipeline Event Hub Azure Data Lake SQL Azure Python
Similar positions
Cloudaeon
We are CirrusLabs. Our vision is to become the world\'s most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and mak...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in