Job Description
Job Description:
As an ELK (Elastic, Logstash & Kibana) Data Engineer, you would be responsible for developing, implementing, and maintaining the ELK stack-based solutions for Kyndryl s clients. This role would be responsible to develop efficient and effective, data & log ingestion, processing, indexing, and visualization for monitoring, troubleshooting, and analysis purposes.
Key Responsibilities:
- Configure Logstash to receive, filter, and transform logs from diverse sources (e.g., servers, applications, AppDynamics, Storage, Databases and so son) before sending them to Elasticsearch.
- Configure ILM policies, Index templates etc.
- Develop Logstash configuration files to parse, enrich, and filter log data from various input sources (e.g., APM tools, Database, Storage and so on)
- Implement techniques like grok patterns, regular expressions, and plugins to handle complex log formats and structures.
- Ensure efficient and reliable data ingestion by optimizing Logstash performance, handling high data volumes, and managing throughput.
- Utilize Kibana to create visually appealing dashboards, reports, and custom visualizations.
- Collaborate with business users to understand their data integration & visualization needs and translate them into technical solutions
- Establishing the correlation within the data and develop visualizations to detect the root cause of the issue.
- Integration with ticketing tools such as Service Now
- Hands on with ML and Watcher functionalities
- Monitor Elasticsearch clusters for health, performance, and resource utilization
- Create and maintain technical documentation, including system diagrams, deployment procedures, and troubleshooting guides
Education, Experience, and Certification Requirements:
BS or MS degree in Computer Science or a related technical field
5+ years overall IT Industry Experience.
3+ years of development experience with Elastic, Logstash and Kibana in designing, building, and maintaining log & data processing systems
3+ years of Python or Java development experience
4+ years of SQL experience (No-SQL experience is a plus)
4+ years of experience with schema design and dimensional data modelling Experience working with Machine Learning model is a plus Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus Elastic Certified Engineer certification is preferrable
Job Classification
Industry: IT Services & Consulting
Functional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Data Engineer
Employement Type: Full time
Contact Details:
Company: Kyndryl
Location(s): Bengaluru
Keyskills:
Elk
Logstash
Elastic Search