Hadoop Big Data Developer

About This Role

We are representing a confidential technology consulting organization in their search for an experienced Hadoop Big Data Developer to join their team on a fully remote basis. This is a direct-hire, W2 position offering the opportunity to design, build, and operate large-scale data processing pipelines and analytics platforms within the Hadoop ecosystem.

In this role, you will be responsible for ingesting, transforming, and analyzing massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and reporting workloads. You'll work with production-scale big data systems and contribute to reliable, performant, and cost-effective data platforms.

Location: 100% Remote (United States)
Position Type: Full-time, Direct W2
Experience Required: 10+ years overall; 5+ years with Hadoop pipelines
Visa: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates may apply. New H-1B sponsorship is not available.

Responsibilities

• Design, build, and maintain large-scale data processing pipelines on Hadoop and related big-data ecosystems
• Develop and optimize Spark jobs (Scala, Python, or Java) for production analytics and machine learning workloads
• Implement and manage streaming data pipelines using Kafka, Spark Streaming, or Flink
• Work with Hive, HDFS, Sqoop, HBase, and the broader Hadoop stack to deliver scalable data solutions
• Orchestrate complex workflows using Airflow, Oozie, or similar tools
• Troubleshoot, debug, and optimize distributed data systems for performance and reliability
• Collaborate with data scientists, analysts, and engineering teams to meet enterprise data needs
• Document architectures, pipelines, and operational procedures

What We're Looking For

Required:

• Bachelor's degree in Computer Science, Engineering, or related technical discipline
• 10+ years of professional experience overall
• 5+ years of hands-on experience designing and operating big-data pipelines on Hadoop
• Strong production expertise with Apache Spark (Scala, Python, or Java)
• Solid experience with Hive, HDFS, Sqoop, HBase, and the Hadoop ecosystem
• Hands-on experience with streaming platforms such as Kafka, Spark Streaming, or Flink
• Strong SQL skills and experience with both relational and NoSQL databases
• Experience with workflow orchestration tools (Airflow, Oozie)
• Solid understanding of distributed systems concepts: partitioning, replication, fault tolerance
• Strong scripting skills in Python or Shell
• Excellent troubleshooting, debugging, and documentation abilities

Preferred:

• Experience with cloud-based Hadoop platforms (AWS EMR, Azure HDInsight, Databricks)
• Familiarity with modern lakehouse formats (Delta, Iceberg, Hudi)
• Exposure to data governance tools such as Apache Atlas or Collibra
• Experience with Kubernetes-based data platforms (Spark-on-K8s, Trino)
• Hands-on experience with CI/CD and infrastructure-as-code in data engineering workflows

How We Work With You

When you apply through Get A Job.ai, a member of our talent team will review your background and schedule an initial screening call. We'll discuss your experience, career goals, and the role in detail. If there's a strong match, we will formally submit your profile to our client for consideration. Please do not contact the client directly—all communication and coordination will flow through our recruiting team to ensure a smooth, professional process.

Pay

Pay via Get A Job.ai: $100,000–$150,000 annually

Equal Employment Opportunity: Get A Job.ai is committed to equal employment opportunity for all applicants regardless of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected characteristic under applicable law.