Data Engineer Job in DataToBiz at Other Punjab
- Datatobiz
- 1 to 5 Yrs
- Other Punjab
- 4 months ago
- Email to a friend
- Report this job
Job Description
- Experience Required 2-5 years Skillset Experience with Python, PySpark, Scala, Spark to write data pipelines and data processing layers. Strong SQL skills and proven experience in working with large datasets and relational databases. Experience with Data Governance (Data Quality, Metadata Management, Security, etc.) Understanding of big data file formats like JSON, Avro, Parquet and other prevalent big data file formats Understanding Data Warehouse design principles (Tabular, MD). Minimum 1 years of experience in any cloud (AWS/GCP/Azure) platform. Experience in working with big data tools/environment to automate production pipelines. Experience with data visual analysis and BI tools (Matplotliband Power BI) Responsibilities As a Data Engineer in our team, you should have a proven ability to deliver high quality work on time and with minimal supervision. Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth. Pipelines should be optimised to handle both real time data, batch update data and historical data. Establish scalable, efficient, automated processes for complex, large scale data analysis. Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse/data lake Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques. Participate in data pipelines health monitoring and performance optimisations as well as quality documentation. Adapt best practices and industry standards, ensuring top quality deliverables and playing an integral role in cross-functional system integration. Data integration using enterprise development tool-sets (e.g. ETL, MDM, Quality, CDC, Data Masking, Quality). Design and implement formal data warehouse testing strategies and plans including unit testing, functional testing, integration testing, performance testing, and validation testing. Interact with end users/clients and translate business language into technical requirem Fun Workplace Flat Job Hierarchy Flexible Work Hours Casual Dress Code Disciplined Workplace Environment Learning & Development Team Cohesion Task Autonomy Smooth Reporting Extensive Training HR Policies POSH Act for inclusivity Flexible Leave Policy Monthly Paid Leaves Bonus & Incentives Motivational Team Appreciation Individual Awards Performance Rewards Salary Appraisals,
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: Not SpecifiedFunctional Area: Not SpecifiedRole/Responsibilies: Data Engineer Job in DataToBiz at Other PunjabContact Details:
Company: DataToBizLocation(s): Other Punjab+ View Contact
Keyskills: Python Scala Spark SQL Data Governance JSON Avro Data Warehouse design BI tools
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Datatobiz
DataToBiz is an AI and Data Analytics Services startup. We are a team of young and dynamic professionals looking for an exceptional Full Stack Developers (MEAN stack) to join our team in Chandigarh. We are trying to solve some very exciting business challenges by applying cutting-edge Machine Learni...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in