Summary
We are seeking experienced
Data Engineer
to support a high-impact project involving database migration to a cloud-native data lake environment.
This is a
client-facing, on-site role
based in
Singapore , requiring strong technical expertise, excellent communication skills, and a problem-solving mindset.
The ideal candidates will bring hands-on experience in Python, Apache Spark, SQL optimization, and DevOps practices.
Responsibilities
Lead and execute
relational database migration
to cloud-native data lake platforms
Develop robust
data processing pipelines
using Python (Pandas, PyArrow) and Apache Spark (PySpark)
Optimize complex
SQL queries , perform performance tuning, and implement efficient data models
Collaborate with DevOps teams to build and maintain
Infrastructure-as-Code
(Terraform/CloudFormation) and
CI/CD pipelines
Participate in
hands-on coding sessions , problem-solving discussions, and stakeholder meetings
Communicate technical concepts clearly to non-technical stakeholders and provide project updates
Ensure all deliverables meet quality standards through
unit testing
and reconciliation techniques
Requirements
Python (Pandas, PyArrow)
– Production-grade coding practices: modular, efficient, reusable code
Apache Spark (PySpark)
– Building and optimizing scalable data pipelines
SQL
– Expertise in query optimization, data modeling, and performance tuning
ETL & Data Warehousing
– Proven experience in data migration and warehouse implementations
DevOps
– Hands-on with
Terraform ,
CloudFormation , and setting up
CI/CD pipelines
Strong
problem-solving
abilities and experience breaking down complex data problems
Excellent
communication and stakeholder management
skills (essential for client-facing, onshore role)
Experience with
AWS ,
Snowflake ,
Databricks , or other big data technologies
Understanding of
Master Data Management (MDM)
and its downstream impact
#J-18808-Ljbffr