Senior Data Engineer
- Flexera Software
- 8 - 13 years
- Kolkata
- 2 months ago
- Email to a friend
- Report this job
Job Description
We are seeking a skilled and motivated Senior Data Engineer to join our Automation, AI/ML team. In this role, you will work on designing, building, and maintaining data pipelines and infrastructure to support AI/ML initiatives, while contributing to the automation of key processes. This position requires expertise in data engineering, cloud technologies, and database systems, with a strong emphasis on scalability, performance, and innovation.
Key Responsibilities:
- Identify and automate manual processes to improve efficiency and reduce operational overhead.
- Design, develop, and optimize scalable data pipelines to integrate data from multiple sources, including Oracle and SQL Server databases.
- Collaborate with data scientists and AI/ML engineers to ensure efficient access to high-quality data for training and inference models.
- Implement automation solutions for data ingestion, processing, and integration using modern tools and frameworks.
- Monitor, troubleshoot, and enhance data workflows to ensure performance, reliability, and scalability.
- Apply advanced data transformation techniques, including ETL/ELT processes, to prepare data for AI/ML use cases.
- Develop solutions to optimize storage and compute costs while ensuring data security and compliance.
Required Skills and Qualifications:
- Experience in identifying, streamlining, and automating repetitive or manual processes.
- Proven experience as a Data Engineer, working with large-scale database systems (e.g., Oracle, SQL Server) and cloud platforms (AWS, Azure, Google Cloud).
- Expertise in building and maintaining data pipelines using tools like Apache Airflow, Talend, or Azure Data Factory.
- Strong programming skills in Python, Scala, or Java for data processing and automation tasks.
- Experience with data warehousing technologies such as Snowflake, Redshift, or Azure Synapse.
- Proficiency in SQL for data extraction, transformation, and analysis.
- Familiarity with tools such as Databricks, MLflow, or H2O.ai for integrating data engineering with AI/ML workflows.
- Experience with DevOps practices and tools, such as Jenkins, GitLab CI/CD, Docker, and Kubernetes.
- Knowledge of AI/ML concepts and their integration into data workflows.
- Strong problem-solving skills and attention to detail.
Preferred Qualifications:
- Knowledge of security best practices, including data encryption and access control.
- Familiarity with big data technologies like Hadoop, Spark, or Kafka.
- Exposure to Databricks for data engineering and advanced analytics workflows.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Data Science & AnalyticsRole Category: Data Science & Machine LearningRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Flexera SoftwareLocation(s): Kolkata+ View Contact
Keyskills: Procurement Automation data security SCALA Data processing Apache Operations SQL Python Data extraction
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in