Lead Data Engineer - Vitality @ Quantium Analytics

Data Engineering Team

As a Lead Data Engineer for India, you will be accountable for leading the technical aspects of product engineering by being hands on, working on the enhancement, maintenance and support of the product on which your team is working, within your technology area. You will be responsible for your own hands-on coding, provide the design thinking and design solutions, ensuring the quality of your teams output, representing your team in product-level technical forums and ensuring your team provides technical input to and aligns with the overall product road-map.

How will you make an impact?

You will work with Engineers in other technology areas to define the overall technical direction for the product on alignment with Groups technology roadmap, standards and frameworks, with product owners and business stakeholders to shape the product's delivery roadmap and with support teams to ensure its smooth operation. You will be accountable for the overall technical quality of the work produced by India that is in line with the expectation of the stakeholders, clients and Group.

You will also be responsible for line management of your team of Engineers, ensuring that they perform to the expected levels and that their career development is fully supported.

Key responsibilities

o Produce Quality Code

o Code follows team standards, is structured to ensure readability and maintainability and goes through review smoothly, even for complex changes

o Designs respect best practices and are favourably reviewed by peers

o Critical paths through code are covered by appropriate tests

o High-level designs / architectures align to wider technical strategy, presenting reusable APIs where possible and minimizing system dependencies

o Data updates are monitored and complete within SLA

o Technical designs follow team and group standards and frameworks, is structured to ensure reusability, extensibility and maintainability and goes through review smoothly, even for complex changes

o Designs respect best practices and are favourably reviewed by peers

o High-level designs / architectures align to wider technical strategy, presenting reusable APIs where possible and minimizing system dependencies

o Estimates are consistently challenging, but realistic

o Most tasks are delivered within estimate

o Complex or larger tasks are delivered autonomously

o Sprint goals are consistently achieved

o Demonstrate commitment to continuous improvement of squad activities

o The product backlog is consistently well-groomed, with a responsible balance of new features and technical debt mitigation

o Other Engineers in the Squad feel supported in their development

o Direct reports have meaningful objectives recorded in Quantium's Performance Portal, and understand how those objectives relate to business strategy

o Direct reports' career aspirations are understood / documented, with action plans in place to move towards those goals

o Direct reports have regular catch-ups to discuss performance, career development and their ongoing happiness / engagement in their role

o Any performance issues are identified, documented and agreed, with realistic remedial plans in place

o Squad Collaboration

o People Management

o Produce Quality Technical Design

o Operate at high level of productivity

Key activities

Build technical product/application engineering capability in the team by that is in line with the Groups technical roadmap, standards and frameworks

Write polished code, aligned to team standards, including appropriate unit / integration tests

Review code and test cases produced by others, to ensure changes satisfy the associated business requirement, follow best practices, and integrate with the existing code-base

Provide constructive feedback to other team members on quality of code and test cases

Collaborate with other Lead / Senior Engineers to produce high-level designs for larger pieces of work

Validate technical designs and estimates produced by other team members

Merge reviewed code into release branches, resolving any conflicts that arise, and periodically deploy updates to production and non-production environments

Troubleshoot production problems and raise / prioritize bug tickets to resolve any issues

Proactively monitor system health and act to report / resolve any issues

Provide out of hours support for periodic ETL processes, ensuring SLAs are met

Work with business stakeholders and other leads to define and estimate new epics

Contribute to backlog refinement sessions, helping to break down each epic into a collection of smaller user stories that will deliver the overall feature Work closely with Product Owners to ensure the product backlog is prioritized to maximize business value and manage

technical debt

Lead work breakdown sessions to define the technical tasks required to implement each user story

Contribute to sprint planning sessions, ensuring the team takes a realistic but challenging amount of work into each sprint and each team member will be productively occupied

Contribute to the teams daily stand-up, highlighting any delays or impediments to progress and proposing mitigation for those issues

Contribute to sprint review and sprint retro sessions, to maintain a culture of continuous improvement within the team

Coach / mentor more junior Engineers to support their continuing development

Set and periodically review delivery and development objectives for direct reports

Identify each direct reports longer-term career objectives and, as far as possible, factor this into work assignments

Hold fortnightly catch-ups with direct reports to review progress against objectives, assess engagement and give them the opportunity to raise concerns about the product or team

Work through the annual performance review process for all team members

Conduct technical interviews as necessary to recruit new Engineers

The superpowers youll be bringing to the team:

8+ years of experience in design, develop, and implement end-to-end data solutions (storage, integration, processing, access) in Google Cloud Platform (GCP) or similar cloud platforms.

2. Strong experience with SQL

3. Values delivering high-quality, peer-reviewed, well-tested code

4. Create ETL/ELT pipelines that transform and process terabytes of structured and unstructured data in real-time

5. Knowledge of DevOps functions and to contribute to CI / CD pipelines

6. Strong knowledge of data warehousing and data modelling and techniques like dimensional modelling etc

7. Strong hands-on experience with BigQuery/Snowflake, Airflow/Argo, Dataflow, Data catalog, VertexAI, Pub/Sub etc or equivalent products in other cloud platforms

8. Solid grip over programming languages like Python or Scala

9. Hands on experience in manipulating SPARK at scale with true in-depth knowledge of SPARK API

10. Experience working with stakeholders and mentoring experience for juniors in the team is good to have

11. Recognized as a go-to person for high-level designs and estimations

12. Experience working with source control tools (GIT preferred) with good understanding of branching / merging strategies

13. Experience in Kubernetes and Azure will be an advantage

14. Understanding of GNU/Linux systems and Bash/scripting

15. Bachelors degree in Computer Science, Information Technology or a related discipline

16. Comfortable working in a fast moving, agile development environment

17. Excellent problem solving / analytical skills

18. Good written / verbal communication skills

19. Commercially aware, with the ability to work with a diverse range of stakeholders

20. Enthusiasm for coaching and mentoring junior engineers

21. Experience in lading teams, including line management responsibilities