Job Summary
We are seeking a highly skilled Principal Infra Developer with 8 to 12 years of experience to join our team. The ideal candidate will have expertise in Splunk Admin SRE Grafana ELK and Dynatrace AppMon. This hybrid role requires a proactive individual who can contribute to our infrastructure development projects and ensure the reliability and performance of our systems. The position does not require travel and operates during day shifts.
Responsibilities
Systems Engineer Splunk or ElasticSearch Admin
Job Requirements
Build Deploy and Manage the Enterprise Lucene DB systems Splunk Elastic to ensure that the legacy physical Virtual systems and container infrastructure for businesscritical services are being rigorously and effectively served for high quality logging services with high availability.
Support periodic Observability and infrastructure monitoring tool releases and tool upgrades Environment creation Performance tuning of large scale Prometheus systems
Serve as Devops SRE for the internal observability systems in Visas various data centers across the globe including in Cloud environment
Lead the evaluation selection design deployment and advancement of the portfolio of tools used to provide infrastructure and service monitoring. Ensure tools utilized can provide the critical visibility on modern architectures leveraging technologies such as cloud containers etc.
Maintain upgrade and troubleshoot issues with SPLUNK clusters.
Monitor and audit configurations and participate in the Change Management process to ensure that unauthorized changes do not occur.
Manage patching and updates of Splunk hosts andor Splunk application software.
Design develop recommend and implement Splunk dashboards and alerts in support of the Incident Response team.
Ensure monitoring team increases use of automation and adopts a DevOpsSRE mentality
Qualification
6plus years of enterprise system logging and monitoring tools experience with a desired 5plus years in a relevant critical infrastructure of Enterprise Splunk and Elasticsearch
5plus yrs of working experience as Splunk Administrator with Cluster Building Data Ingestion Management User Role Management Search Configuration and Optimization.
Strong knowledge on opensource logging and monitoring tools.
Experience with containers logging and monitoring solutions.
Experience with Linux operating system management and administration
Familiarity with LANWAN technologies and clear understanding of basic network concepts services
Strong understanding of multitier application architectures and application runtime environments
Monitoring the health and performance of the Splunk environment and troubleshooting any issues that arise.
Worked in 247 on call environment.
Knowledge of Python and other scripting languages and infrastructure automation technologies such as Ansible is desired
Splunk Admin Certified is a plus
Keyskills: enterprise dynatrace monitoring tools ansible sql elastic search automation java devops linux jenkins scripting languages python elk sre splunk admin application architectures environment system application grafana infrastructure splunk administration logging splunk aws