Big Data Engineer
- Zinnov Management
- 2 to 5 Yrs
- Bengaluru
- 6 years ago
- Email to a friend
- Report this job
Job Description
We are looking for a Big Data Engineer who will work on collecting, storing, processing, and analyzing large sets of data from different data sources. The primary focus will be on choosing optimal solutions to use for these purposes, then implementing and improving on them. If you do not like to take on new challenges every day, then this is definitely not for you.Responsibilities:
1. Work directly with a seasoned founding team to conceptualize and create the final product
2. Opportunity to create an architecture, implement, maintain, and monitor a generic ETL pipeline that can be used across different types of data sources
3. Working with our research team to validate the defined data models for a variety of data sources and implementing them by integrating any Big Data tools and frameworks required to provide requested capabilities
4. Monitoring performance, optimizing the architecture, and advising any necessary infrastructure changes to suit your application needs
5. Creating a platform on top of stored data sources using a distributed processing environment like Spark, for the users to perform any kind of ad- hoc queries with complete abstraction from the internal data points
6. Defining data retention policies
7. Opportunity to collaborate and work with the team to create Artificial Intelligence Application with the use of all the processed data
Skills:
1. Bachelor s or Master s degree in Computer Science or related streams; equivalent professional experience may be substituted for formal education
2. Proficient understanding of distributed computing principles
3. Proficiency in Apache Spark is a MUST
4. Experience with integration of data from multiple data sources
5. Experience working with NoSQL databases like Cassandra, MongoDB and SQL databases like MySQL
6. Good working knowledge of MapReduce, HDFS
7. Experience with building stream- processing systems, using solutions such as Storm or Spark- Streaming
8. Good knowledge of Big Data querying tools such as Pig, Hive, and Impala
9. Knowledge of various ETL techniques and frameworks such as Flume
10. Experience with various messaging systems such as Kafka or RabbitMQ
11. Ability to communicate complex technical concepts to both technical and non- technical audiences
12. We expect an entrepreneurial mindset, someone who is not afraid to take on new challenges every day, and who considers the product as their own by taking complete ownership of it
,
Employement Category:
Employement Type: Full timeIndustry: Recruitment ServicesRole Category: IT Operations / EDP / MISFunctional Area: Not ApplicableRole/Responsibilies: Big Data EngineerContact Details:
Company Name: Zinnov ManagementLocation(s): Bengaluru+ View Contact
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in