Responsibilities
Test and validate data pipelines for all new releases.
Ensure data integrity and completeness after deployments.
Debug and troubleshoot pipeline issues.
Implement fixes and document root causes.
Optimise pipeline performance and efficiency.
Write clean, maintainable Python code with unit tests.
Collaborate with data scientists, engineers, and stakeholders.
Support AWS-based pipeline infrastructure (e.g., S3, Lambda, EC2, CloudWatch).
Requirements
Proficient in Python (unit testing, clean code practices).
Experience in data pipeline development and troubleshooting.
Familiarity with AWS services (S3, Lambda, EC2, CloudWatch).
Strong debugging and problem-solving skills.
Attention to detail and focus on data quality.
Good to have
Basic understanding of data science workflows.
Experience with CI/CD and version control (e.g., Git).
#J-18808-Ljbffr