Skills: Azure Data Factory, Data Lake, Synapse SQL DWH, Databricks
Airflow, Python, Pyspark, SQL, Terraform
Experience in designing and building highly scalable data platforms pipelines
Hands on experience in developing end to end big data pipelines using Azure, AWS and or open source bigdata tools and technologies.
Experience in data lake , data warehousing solutions
Experience in extracting data from APIs, Cloud services Salesforce, Eloqua, S3, SQL and NoSQL onprem cloud databases using Azure Data factory, Glue ,Opensource data ingestion tools
Experience in creating complex data processing pipelines ETLs using PySpark Scala Databricks Glue EMR
In depth knowledge of spark architecture and experience in improving the performance and optimization
Experience using cloud data warehouses e.g. Synapse ,SQL DWH, Redshift, Snowflake, build and manage data models and to present data securely
Good understanding of Distributed Data Processing , MPP
Knowledge of common DevOps skills and methodologies
Experience in Azure DevOps, GitHub, Gitlab
Experience in using Azure ARM Templates or Infrastructure as Code Terraform
Indepth understanding of OLTP, OLAP, Data warehousing, Data modeling and strong analytical and problem solving skills
Hands on experience using MySQL, MS SQL Server, Oracle or similar RDBMS platform.
Highly self motivated, self directed, and attentive to detail
Ability to effectively prioritize and execute tasks.
Additional skillsets
Good to have experience in Docker and Kubernetes
Experience working across Azure IaaS, Azure PaaS, Azure Networking, and other areas of the platform
Databricks delta lake and lake house
Familiarity of Data Quality Management methodology and supporting technology tools.
Familiarity with data visualization tools e.g. PowerBI, Tableau etc
Keyskills: Azure Data Factory Azure Synapse Azure Databricks Azure Data Lake Pyspark Azure Data Warehouse Python
We build DATA + CODE for some of the worlds leading B2B brands. Through pioneering marketing data, product data,customised software and marketing automation, Merit helps brands of all sizes confidently embrace the challenges ofthe future.