Bigdata+Scala Professional
- Diverse Lynx
- 4 - 5 years
- Hyderabad
- 2 years ago
- Email to a friend
- Report this job
Job Description
Diverse Lynx is looking for Bigdata+Scala to join our dynamic team and embark on a rewarding career journey- Big Data Architecture:Design, develop, and implement scalable and robust Big Data architectures using technologies such as Apache Hadoop and Apache Spark
- Collaborate with cross-functional teams to integrate Big Data solutions into the overall system architecture
- Scala Programming:Develop and maintain data processing applications using Scala programming language
- Implement efficient and functional code for data processing workflows
- Data Processing Frameworks:Utilize and optimize Big Data processing frameworks, with a focus on Apache Spark
- Design and implement Spark applications for both batch and real-time data processing
- Data Modeling and Storage:Design data models for structured and unstructured data sources
- Work with distributed storage systems, including Hadoop Distributed File System (HDFS) and cloud-based storage solutions
- Data Integration:Develop processes for ingesting and integrating data from diverse sources into Big Data environments
- Ensure data quality, consistency, and reliability in integration workflows
- Performance Tuning:Optimize the performance of Big Data processing workflows and clusters, specifically within the Scala and Spark ecosystem
- Conduct performance testing and implement enhancements for efficiency
- Security and Compliance:Implement security measures for Big Data environments, ensuring adherence to access controls, encryption, and compliance with data privacy regulations
- Contribute to data governance and compliance efforts
- Scripting and Automation:Develop scripts and automation processes to streamline data workflows and system maintenance
- Apply Infrastructure as Code (IaC) principles for automation
- Collaboration with Data Scientists:Collaborate with data scientists to provide the necessary infrastructure for advanced analytics and machine learning models
- Support the integration of analytics results into production systems
- Monitoring and Troubleshooting:Set up monitoring systems to proactively identify and address issues in Big Data processing workflows, particularly those involving Scala
- Troubleshoot and resolve technical challenges and incidents
- Documentation:Create and maintain comprehensive documentation for Big Data architectures, configurations, and Scala code
- Provide documentation for knowledge transfer and training
- Continuous Learning:Stay updated on the latest developments in Big Data technologies, Scala, and industry best practices
- Attend relevant conferences, webinars, and training sessions
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: OtherRole Category: OtherRole: OtherEmployement Type: Full timeContact Details:
Company: Diverse LynxLocation(s): Hyderabad
Keyskills: System architecture Automation Data modeling Machine learning Performance testing Data processing Data quality Troubleshooting Analytics Monitoring
Similar positions
Diverse Lynx
Diverse Lynx is a Go to Place for all your IT needs . We combine industry leading expertise and our personal dedication for all your needs. Established in 2002, we are head quartered in Princeton NJ with an off-shore delivery center in New Delhi, India. The Delivery Center is 250+ associate str...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in