We are seeking a Hadoop Cloud Integration Specialist to design, implement, and manage hybrid data platforms integrating on-premise Hadoop clusters with cloud environments (AWS, Azure, or GCP).
The role focuses on data migration, automation, and performance optimization across distributed systems.
Key Responsibilities:
Integrate and migrate Hadoop/Cloudera environments to cloud platforms such as AWS EMR, Azure HDInsight, or GCP Dataproc.
Configure and manage HDFS, Hive, Spark, Kafka, and YARN within hybrid or multi-cloud setups.
Design secure data transfer, replication, and backup strategies between on-prem and cloud storage (e.g., S3, ADLS, GCS).
Implement CI/CD pipelines for automated deployment and scaling of Hadoop services.
Ensure data security, compliance, and performance tuning across environments.
Collaborate with data engineering, DevOps, and infrastructure teams for seamless integration and operational reliability.
Required Skills:
Expertise in Hadoop ecosystem administration (Cloudera/Hortonworks/CDP).
Strong knowledge of AWS EMR, S3, IAM, VPC, or equivalent cloud services.
Proficiency in Shell/Python scripting, Jenkins, Git, and Terraform.
Hands-on with Kerberos, Ranger/Sentry, and cluster monitoring tools.
Experience with data migration, replication, and disaster recovery in hybrid platforms.
Certifications Required:
Cloudera Essentials for CDP
AWS Fundamentals for CDP Public Cloud (Cloudera)