Senior Data Engineer
- Themathcompany
- 5 - 10 years
- Bengaluru
- 1 month ago
- Email to a friend
- Report this job
Job Description
- Responsible for designing, storing, processing, and maintaining of large-scale data and related infrastructure
- Can drive multiple projects both from operational and technical standpoint
- Ideate and build PoV or PoC for new product that can help drive more business
- Responsible for defining, designing, and implementing data engineering best practices, strategies, and solutions
- Is an Architect who can guide the customers, team, and overall organization on tools, technologies, and best practices around data engineering
- Lead architecture discussions, align with business needs, security, and best practices
- Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing
- Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
- Understanding of Medallion architecture pattern
- Has worked on at least one cloud platform
- Has worked as data architect and executed multiple end-end data engineering project
- Has extensive knowledge of different data architecture designs and data modelling concepts
- Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes
Required Tech Stack -
- Strong proficiency in SQL
- Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP
- Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc
- Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.)
- Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
- Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java)
- Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables)
- Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc.
- Has worked on Kafka and real-time streaming
- Has strong execution knowledge of data architecture design patterns (lambda vs kappa architecture, data harmonization, customer data platforms, etc.)
- Has worked on code and SQL query optimization
- Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery
- Has worked on data and networking security (RBAC, secret management, key vaults, vnets, subnets, certificates)
Required Non-Tech Stack-
- Strong problem-solving skills
- Able to analyze problems and quickly develop effective and actionable strategies
- Displays excellent leadership qualities and can effectively lead teams
- Proficient in written and verbal communication
- Demonstrates exceptional stakeholder management abilities
- Exercises independent judgment to determine the best course of action within established practices and procedures
- Shows proficiency in providing technical guidance to junior team members
Job Classification
Industry: Management ConsultingFunctional Area / Department: Engineering - Software & QARole Category: DBA / Data warehousingRole: Database Architect / DesignerEmployement Type: Full timeContact Details:
Company: ThemathcompanyLocation(s): Bengaluru+ View Contact
Keyskills: Cloud computing Version control GIT MySQL Schema Unit testing OLTP SQL Python Data architecture
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Themathcompany
TheMathCompany is a global data analytics and data engineering firm that partners with Fortune500 and equivalent organizations to enhance their analytics capabilities, using our next-generation, proprietary platform, Co.dx, along with our talent and processes to deliver accelerated and human-centric...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in