Lead Data Engineer
- Themathcompany
- 10 - 17 years
- Bengaluru
- 7 months ago
- Email to a friend
- Report this job
Job Description
As a DE Manager, you will be responsible for:
- Leading a team of talented data engineers responsible for designing, building, and maintaining scalable data pipelines and infrastructure
- Work closely with cross-functional teams to ensure client data systems meet the highest standards of quality and performance
- Lead, mentor, and develop a team of data engineers, fostering a collaborative and inclusive team environment
- Conduct regular performance reviews, provide feedback, and set goals for team members
- Identify and address skill gaps, and provide opportunities for professional development
- Plan, execute, and deliver data engineering projects on time and within scope
- Coordinate with stakeholders to gather requirements, set priorities, and define project timelines.
- Ensure projects align with overall business objectives and data strategy
- Oversee the design, development, and maintenance of data pipelines, ETL processes, and data warehouse
- Ensure data quality, integrity, and security across all data engineering projects.
- Identify opportunities for process improvements and drive initiatives to enhance the efficiency and effectiveness of data operations.
- Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing
- Ability to build/drive reusable frameworks that can drive efficiency of the overall data system
- Has executed and lead multiple projects including on - streaming, batch, large data pipelines, etc.
- Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.
Required Tech Stack:
- Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
- Working knowledge of an ETL and/or orchestration tools like IICS, Metallion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
- Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Hive, Big Query, etc.
- Proficient in Spark and can optimize Spark jobs with ease.
- Proficient in at least one programming language used in data engineering, such as Python (Non-negotiable) and Scala/Rust/Java
- Understanding of Medallion architecture pattern
- Has strong SQL knowledge along with optimization skills
Required Non-Tech Stack
- Strong problem-solving skills with an ability to assess the financial impact of decisions, both in running the delivery team and delivering solutions to clients.
- Proficient in written and verbal communication and able to hold conversations with mid-management-level clients.
- Ability to recognize pragmatic alternatives vis- -vis a perfect solution and get the delivery teams on board to pursue them, balancing time priorities with potential business impact.
- Strong people skills, including conflict resolution, empathy, communication, listening, and negotiation.
- Shows proficiency in providing technical guidance and provide leadership and mentorship to the delivery team.
- Self-driven with a strong sense of ownership.
Preferred Educational Qualifications
- B.E/B.Tech, MCA, M.Sc. (Mathematics, Statistics)
Job Classification
Industry: Management ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: ThemathcompanyLocation(s): Bengaluru
Keyskills: Cloud computing NoSQL Data modeling SCALA Software development life cycle Data processing Data quality Unit testing SQL Python
Similar positions
Themathcompany
TheMathCompany is a global data analytics and data engineering firm that partners with Fortune500 and equivalent organizations to enhance their analytics capabilities, using our next-generation, proprietary platform, Co.dx, along with our talent and processes to deliver accelerated and human-centric...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in