Senior Data Engineer
- Maersk Global Services
- 4 - 9 years
- Pune
- 5 years ago
- Email to a friend
- Report this job
Job Description
Roles and Responsibilities
- Project experience using Azure cloud, data lake storage, Data factory, Hadoop 2.x, HDInsight, Spark, Scala, Hive, HDFS. Exp with Spark more relevant.
- Exp on creating Batch and Realtime Data pipelines on Azure Data Platform using Logic Apps, Automation Run Books, Azure Data Factory, Azure Databricks, Azure Event Hub, Kafka, Azure Stream Analytics.
- Experience in performance tuning of complex ETL mappings for relational and non-relational workloads Advanced SQL programming, SparkSQL, PySpark, PL/SQL or T-SQL,
- Hands on Unix scripting or PowerShell, experience with Azure PaaS services
- Hadoop Data Serialization mechanism line PARQUET, AVRO, ORC.
- Hands on experience in Azure, preferably data heavy / analytics applications leveraging relational and NoSQL databases, data warehouse and big data.
- Design DWH model, ensuring data design follows the prescribed reference architecture framework while reflecting appropriate business rules built for logical, physical and conceptual model.
- Experience with enterprise-class data modelling tools (Erwin Data Modeler, Sparx Enterprise Architect etc)
- Facilitate data integration, data conformity, data integrity and ensure data structures are designed for flexibility to support future business needs and enhancements.
- Understand the relationships across business information and units of data, Reverse engineer data from various source systems.
- Ensure that data transformations are well documented follow best practices and understood by business users and developers. Have experience cleansing, manipulating and analysing large datasets both structured and Unstructured. Exceptional data analysis, critical thinking, communication, problem solving and cross-group collaboration skills Experience with Power BI would be an added advantage
Desired Candidate Profile
We are seeking for a Big Data Engineer who has strong experience in DWH, ETL and BI technologies on Big Data Framework/Hadoop Technologies.
One who has been in at least 2+ yrs of experience into AZURE, Sparks & SQL
Candidates must have strong communication skills that allow you to regularly deal with the business, as well as with specialists from related fields including BAs, architects and Front-End Engineers.
Please share your resume at th*********a@ma***k.com
Regards,
Thomas Lakra
Maersk Global shared Services
Job Classification
Industry: Shipping, MarineFunctional Area: IT Software - Application Programming, Maintenance, Role Category: Admin/Maintenance/Security/DatawarehousingRole: Admin/Maintenance/Security/DatawarehousingEmployement Type: Full timeEducation
Under Graduation: B.Tech/B.E. in Any Specialization, BCA in ComputersPost Graduation: Any Postgraduate in Any SpecializationDoctorate: Ph.D in Any SpecializationContact Details:
Company: MaerskGlobalServicesAddress: IT Info Park, 7th D Building, Viman Nagar Ave 1, Near Weikfield, Viman Nagar, Pune, Maharashtra 411014, India & Milestone Buildcon, Bhartiya city, Chokanahalli, Bengaluru, Karnataka 560064, IndiaLocation(s): Pune
Keyskills: Azure Data Factory Hive Hadoop Big Data Azure Databricks Microsoft Azure Spark Avro Azure Devops SQL
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in