">
Design, develop, and maintain scalable data pipelines and data integration processes to extract, transform, and load (ETL) data from various sources into our data warehouse or data lake.
Job responsibilities:
Collaborate with stakeholders to understand data requirements and translate them into efficient and scalable data engineering solutions.
Optimize data models, database schemas, and data processing algorithms to ensure efficient and high-performance data storage and retrieval.
Implement and maintain data quality and data governance processes, including data cleansing, validation, and metadata management.
Work closely with data scientists, analysts, and business intelligence teams to support their data needs and enable data-driven decision-making.
Develop and implement data security and privacy measures to ensure compliance with regulations and industry best practices.
Monitor and troubleshoot data pipelines, identifying and resolving performance or data quality issues in a timely manner.
Stay up to date with emerging technologies and trends in the data engineering field, evaluating and recommending new tools and frameworks to enhance data processing and analytics capabilities.
Collaborate with infrastructure and operations teams to ensure the availability, reliability, and scalability of data systems and infrastructure.
Mentor and provide technical guidance to junior data engineers, promoting best practices and knowledge sharing."
Desired Skills:
Apache Spark, Python
Azure experience (Data Bricks, Docker, Function App)
Git
Working knowledge of Airflow
Knowledge of Kubernetes and Docker
Power BI
Data Visualization: Proficient in creating interactive and visually appealing dashboards and reports using Power BI.
Data Modeling: Experience in designing and implementing data models, including relationships, hierarchies, and calculated columns/measures.
DAX (Data Analysis Expressions): Strong knowledge of DAX for creating complex calculations and aggregations.
Power Query: Skilled in using Power Query for data transformation and preparation.
Integration: Ability to integrate Power BI with various data sources such as SQL databases, Excel, and cloud services.
Performance Optimization: Experience in optimizing Power BI reports for performance and scalability.
Security: Knowledge of implementing row-level security and managing user access within Power BI.
Collaboration: Experience in sharing and collaborating on Power BI reports and dashboards within an organization.
Best Practices: Familiarity with Power BI best practices and staying updated with the latest features and updates.
Keyskills: Data analysis metadata Data modeling Consulting power bi Data quality Business intelligence Analytics SQL Python