Senior Data Engineer
- Visa
- 4 to 8 Yrs
- Other Karnataka
- 4 months ago
- Email to a friend
- Report this job
Job Description
- Company Description Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa. Job Description The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for internal data analysts by more than 90%. Key Responsibilities: Provide Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source-based solutions. Develop and enhance the NLQ Copilot platform, ensuring it meets the evolving needs of multi-tenant environments. Implement Context builder algorithms by leveraging different prompt engineering techniques to generate 100% accurate SQL as per the customer needs. Collaborate with downstream clients to integrate business requirements, adding robust guardrails to prevent unauthorized query generation. Work closely with data scientists, engineers, and product managers to optimize the performance of the NLQ platform. Utilize cutting-edge NLP / LLM and machine learning techniques to improve the accuracy and efficiency of query transformations. Ensure the platform's scalability and reliability through rigorous testing and continuous improvement. Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant. Spend 80% of the time writing code in different languages, frameworks, and technology stacks. This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager. Qualifications Basic Qualification 4 yrs+ of experience in architecture design and development of large-scale data management platforms and data application with simple solutions Bachelor's or masters degree in Computer Science or related technical discipline required Must have extensive hands-on coding and designing skills on Java/Python for backend MVC (model-view-controller) for end-to-end development SQL/NoSQL technology. Familiar with Databases like Oracle, DB2, SQL Server, etc. Web Services (REST/ SOAP/gRPC) React/Angular for front-end (UI front-end nice to have) Expertise in design and management of complex data structures and data processes Expertise in efficiently leveraging the power of distributed big data systems, including but not limited to Hadoop Hive, Spark, Kafka streaming, etc. Strong service architecture and development experience with high performance and scalability Strong on driving for results and self-motivated, strong learning mindset, with good understanding of related advanced/new technology. Keep up with the technology development in the related areas in the industry, which could be leveraged to enhance current architectures and build durable new ones. Strong leadership and team player. Strong skills on mentoring/growing junior people Preferred Qualification Deep knowledge and hands on experience on big data and cloud computing technologies. Experience with LLM / GenAI tools / applications and Prompt Engineering Experience with ETL/ELT tools / applications Experience with Apache NiFi and Apache Spark for processing large data sets Experience on Elastic Search Knowledge on Data Catalog tools Experience in building Data Pipeline development tools Experience with Data Governance and Data Quality tools Additional Information ,
Employement Category:
Employement Type: Full timeIndustry: IT Services & ConsultingRole Category: Not SpecifiedFunctional Area: Not SpecifiedRole/Responsibilies: Senior Data EngineerContact Details:
Company: VisaLocation(s): Other Karnataka
Keyskills: Java Python SQL NoSQL Web Services REST SOAP Angular Hadoop Hive Spark Kafka ETL Apache Spark Elastic Search Data Governance Data Quality React LLM ELT Data Pipeline
Similar positions
Visa
Enhances Visa Immigration is a immigration consultancy firm with headquarters in Delhi, India, and offices across India and Canada. With more than 10 years of experience in the immigration sector, we specialize in a wide range of immigration options suitable for aspirational individuals and families...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in