Lead Data Engineer- Java/Big
- Visa
- 12 to 16 Yrs
- Other Karnataka
- 4 months ago
- Email to a friend
- Report this job
Job Description
- Company Description Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa. Job Description The Universal Data Catalog (UDC) / Visa Data Catalog (VDC) is a comprehensive metadata management platform designed to provide a centralized repository for all data-related information across the organization. The platform enables efficient data discovery, governance, and utilization by offering detailed metadata definitions for tables, columns, and other data assets. It supports various business units by improving data accessibility, enhancing data quality, and facilitating compliance with data governance policies. Our implementation leverages the open-source Datahub project, ensuring a robust, flexible, and scalable solution. Key Responsibilities Develop and maintain the Enterprise Data Catalog / Visa Data Catalog platform, ensuring it meets the organizations evolving needs. Implement and manage metadata ingestion processes to ensure the catalog is up-to-date with the latest data definitions and business context. Collaborate with data stewards, data owners, and other stakeholders to enrich metadata with business definitions, data lineage, and usage context. Enhance the catalogs search and discovery capabilities to provide users with intuitive and efficient access to data assets. Integrate the data catalog with other enterprise systems and tools to support data governance, data quality, and analytics initiatives. Monitor and optimize the performance of the data catalog to ensure scalability and reliability. Provide training and support to users on how to effectively leverage the data catalog for their data-related needs. Actively collaborate with the open-source community to contribute to and leverage the latest advancements in the Datahub project. Analyze industry best practices and keep the catalog functionality up-to-date with feature sets provided by the market, while focusing on Visas scalability requirements. Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant. Spend 70% of the time writing code in different languages, frameworks, and technology stacks. This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager. Qualifications Basic Qualification 12 to 15 yrs+ of experience in architecture design and development of large- scale data management platforms and data application with simple solutions Must have extensive hands-on coding and designing skills on Java/Python for backend MVC (model-view-controller) for end-to-end development SQL/NoSQL technology. Familiar with Databases like Oracle, DB2, SQL Server, etc. Web Services (REST/ SOAP/gRPC) React/Angular for front-end (UI front-end nice to have) Expertise in design and management of complex data structures and data processes Expertise in efficiently leveraging the power of distributed big data systems, including but not limited to Hadoop Hive, Spark, Kafka streaming, etc. Deep knowledge and hands on experience on big data and cloud computing technologies. Strong service architecture and development experience with high performance and scalability Technical background in data with deep understanding of issues in multiple areas such as data acquisition, ingestion and processing, data management, distributed processing, and high availability is required. Strong on driving for results and self-motivated, strong learning mindset, with good understanding of related advanced/new technology. Keep up with the technology development in the related areas in the industry, which could be leveraged to enhance current architectures and build durable new ones. Bachelor's degree in Computer Science or related technical discipline required. Advanced degree is a plus. Strong leadership and team player. Strong skills on mentoring/growing junior people Preferred Qualification Experience with ETL / ELT tools / applications Experience with Apache NiFi and Apache Spark for processing large data sets Experience on Elastic Search Knowledge on Data Catalog tools Experience in building Data Pipeline development tools Experience with Data Governance and Data Quality tools Additional Information ,
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: Not SpecifiedFunctional Area: Not SpecifiedRole/Responsibilies: Lead Data Engineer- Java/BigContact Details:
Company: VisaLocation(s): Other Karnataka+ View Contact
Keyskills: Java Python MVC SQL NoSQL Web Services Angular Hadoop Hive Spark Kafka Cloud Computing ETL Apache Spark Elastic Search Data Governance Data Quality React ELT
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Visa
Enhances Visa Immigration is a immigration consultancy firm with headquarters in Delhi, India, and offices across India and Canada. With more than 10 years of experience in the immigration sector, we specialize in a wide range of immigration options suitable for aspirational individuals and families...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in