Design, develop, test, deploy, and maintain large-scale data pipelines using BigQuery, Dataproc, PubSub, and Cloud Storage on Google Cloud Platform (GCP).
Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
Troubleshoot issues related to data pipeline failures or errors in real-time using logs analysis and debugging techniques.
Develop automation scripts using Python to streamline data processing tasks and improve efficiency.
Ensure compliance with security standards by implementing access controls, encryption, and monitoring mechanisms.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full time