Data Engineer

Responsibilities

• Test and validate data pipelines for all new releases.
  
  
• Ensure data integrity and completeness after deployments.
  
  
• Debug and troubleshoot pipeline issues.
  
  
• Implement fixes and document root causes.
  
  
• Optimise pipeline performance and efficiency.
  
  
• Write clean, maintainable Python code with unit tests.
  
  
• Collaborate with data scientists, engineers, and stakeholders.
  
  
• Support AWS-based pipeline infrastructure (e.g., S3, Lambda, EC2, CloudWatch).
  
  

Requirements

• Proficient in Python (unit testing, clean code practices).
  
  
• Experience in data pipeline development and troubleshooting.
  
  
• Familiarity with AWS services (S3, Lambda, EC2, CloudWatch).
  
  
• Strong debugging and problem-solving skills.
  
  
• Attention to detail and focus on data quality.
  
  

Good to have

• Basic understanding of data science workflows.
  
  
• Experience with CI/CD and version control (e.g., Git).