Sr SDE-Spark
- Unify Technologies
- 2 - 6 years
- Hyderabad
- 1 year ago
- Email to a friend
- Report this job
Job Description
Unify Technologies is looking for Sr SDE-Spark to join our dynamic team and embark on a rewarding career journey- We are seeking a highly skilled and experienced Senior Software Development Engineer with expertise in Apache Spark to join our dynamic engineering team
- The ideal candidate will have a strong background in big data processing, distributed computing, and software development
- As a Senior SDE specializing in Apache Spark, you will play a key role in designing, implementing, and optimizing data processing workflows to support our organization's data-driven initiatives
Responsibilities:
- Spark Application Development:Design, develop, and maintain Apache Spark applications for large-scale data processing
- Implement efficient data transformations, aggregations, and analytics using Spark RDDs and DataFrames
- Distributed Computing:Work on distributed computing frameworks and architectures to ensure scalability and performance of Spark applications
- Optimize Spark jobs for resource utilization and execution speed
- Data Processing Workflows:Design end-to-end data processing workflows, integrating Spark with other components of the data pipeline
- Collaborate with data engineers and data scientists to implement data processing requirements
- Optimization and Performance Tuning:Identify and address performance bottlenecks in Spark applications
- Implement optimizations to enhance the efficiency of data processing jobs
- Data Integration:Integrate Spark applications with various data sources and sinks, including databases, data lakes, and external APIs
- Ensure data quality and reliability in integration processes
- Monitoring and Debugging:Implement monitoring solutions to track the performance and health of Spark applications
- Debug and troubleshoot issues to ensure the reliability of data processing workflows
- Code Reviews and Collaboration:Conduct code reviews and provide constructive feedback to peers
- Collaborate with cross-functional teams, including data engineers, data scientists, and software developers
- Documentation:Maintain accurate and up-to-date documentation for Spark applications, workflows, and development processes
- Provide documentation for troubleshooting and knowledge transfer
- Requirements:Bachelor's degree in Computer Science, Information Technology, or a related field
- Master's degree is a plus
- Proven experience as a Software Development Engineer with a focus on Apache Spark
- In-depth knowledge of Spark architecture, internals, and best practices
- Proficiency in programming languages such as Scala or Python for Spark development
- Strong understanding of big data processing concepts and distributed computing
- Experience with Spark SQL, Spark Streaming, and related Spark libraries
- Familiarity with data pipeline orchestration tools (eg, Apache Airflow)
- Knowledge of data integration and ETL processes
- Strong problem-solving and analytical skills
- Excellent communication and collaboration skills
- Ability to work independently and contribute to a collaborative team environment
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Data EngineerEmployement Type: Full timeContact Details:
Company: Unify TechnologiesLocation(s): Hyderabad+ View Contact
Keyskills: Computer science Performance tuning Debugging Application development Apache Information technology Analytics Monitoring SQL Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Unify Technologies
UNIFY TECHNOLOGIES PVT LTD About Unify Technologies: We are a Digital Engineering Services company in developing technology solutions towards imparting greater value and creating collaboration amongst global businesses. We work on Cloud/ Data/ Mobile Technologies. Unify leads the way in changing...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in