Text copied to clipboard!
Title
Text copied to clipboard!Big Data Engineer
Description
Text copied to clipboard!
We are looking for a Big Data Engineer to join our growing team of analytics experts. The ideal candidate will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams. The Big Data Engineer will support our software developers, database architects, data analysts, and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
The right candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The ideal candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.
Key responsibilities include building and maintaining scalable data pipelines, working with structured and unstructured data, and integrating data from multiple sources. The Big Data Engineer will also be responsible for implementing data governance and security policies, ensuring data quality, and collaborating with data scientists and analysts to deliver actionable insights.
This role requires strong programming skills in languages such as Java, Scala, or Python, as well as experience with big data tools like Hadoop, Spark, Kafka, and Hive. Familiarity with cloud platforms such as AWS, Azure, or Google Cloud is also essential. The candidate should have a solid understanding of data modeling, ETL processes, and distributed systems.
If you are passionate about data, enjoy solving complex problems, and want to be part of a dynamic and innovative team, we encourage you to apply.
Responsibilities
Text copied to clipboard!- Design, construct, install, and maintain large-scale processing systems
- Build high-performance algorithms, predictive models, and prototypes
- Develop and maintain scalable data pipelines and build out new API integrations
- Work with stakeholders to assist with data-related technical issues
- Ensure data architecture supports business requirements
- Implement data governance and security measures
- Optimize data delivery and re-design infrastructure for greater scalability
- Collaborate with data scientists and analysts to improve data systems
- Monitor performance and advise on infrastructure improvements
- Document data processes and architecture
Requirements
Text copied to clipboard!- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
- 3+ years of experience in a Data Engineer or similar role
- Proficient in Java, Scala, or Python
- Experience with big data tools such as Hadoop, Spark, Kafka, and Hive
- Strong understanding of ETL processes and data modeling
- Experience with cloud services (AWS, Azure, GCP)
- Knowledge of relational and non-relational databases
- Excellent problem-solving and communication skills
- Ability to work independently and in a team environment
- Familiarity with data governance and security best practices
Potential interview questions
Text copied to clipboard!- What big data tools and technologies are you most experienced with?
- Can you describe a complex data pipeline you’ve built?
- How do you ensure data quality and integrity?
- What is your experience with cloud-based data platforms?
- How do you handle unstructured data?
- Describe a time you optimized a data system for performance.
- What programming languages do you use for data engineering tasks?
- How do you collaborate with data scientists and analysts?
- What challenges have you faced in data integration projects?
- How do you stay updated with new data engineering trends?
