Senior Data Engineer – AWS + Spark Required

About This Opportunity

We are representing a confidential technology organization seeking an experienced Senior Data Engineer for a remote position based in Costa Rica or El Salvador. This role is ideal for a data professional who excels at designing and implementing scalable data lakehouse architectures using modern cloud-native technologies.

Our client is building next-generation transactional data infrastructure to power enterprise analytics at scale. You'll architect robust batch and streaming pipelines, optimize query performance, and mentor engineering talent while working with cutting-edge AWS services and Apache frameworks.

Important: This position is open only to citizens and permanent residents of Costa Rica or El Salvador who do not require visa sponsorship.

Responsibilities

• Design, implement, and maintain scalable, ACID-compliant data lakehouse architectures utilizing Apache Hudi on AWS S3
• Build robust, fault-tolerant ETL/ELT pipelines using Apache Spark (PySpark/Scala) and AWS Glue for high-volume batch and real-time streaming data
• Write advanced, performant SQL queries and optimize interactive analytical workloads using AWS Athena
• Manage Hudi table operations including upserts, soft/hard deletes, compaction, clustering, time-travel queries, and schema evolution
• Optimize Glue jobs, Spark cluster performance, partitioning strategies, file formats (Parquet/ORC), and query latency across large-scale datasets
• Enforce data quality check frameworks, metadata management, cataloging via AWS Glue Data Catalog, and data security/privacy standards
• Mentor junior and mid-level data engineers, participate in code reviews, and partner with Analytics, Data Science, and Product teams to align data modeling with business goals

What We're Looking For

Required Experience and Skills:

• 5+ years of software or data engineering experience with a track record of architecting distributed data platforms
• Advanced mastery of complex SQL queries, analytical functions, CTEs, and performance tuning techniques
• Deep technical knowledge of Apache Spark architecture (RDDs, DataFrames, Spark SQL, Structured Streaming), memory management, and job optimization
• Hands-on experience implementing transactional data lakes with Apache Hudi (Copy-on-Write vs. Merge-on-Read, indexing strategies, CDC ingestion, compaction)
• Expertise in building serverless AWS Glue ETL pipelines, custom scripts, dynamic frames, Glue Triggers, and managing the AWS Glue Data Catalog
• Strong proficiency with AWS Athena including ad-hoc querying, query execution cost/time optimization, dynamic partitioning, and federated queries
• Proficiency in Python or Scala
• Solid grasp of dimensional modeling, data vault, Star/Snowflake schemas, and database internals
• Excellent verbal and written communication skills
• Experience working within agile environments

Nice to Have:

• Experience setting up CDC (Change Data Capture) pipelines into Apache Hudi
• Experience with Infrastructure as Code (Terraform, AWS CloudFormation, or AWS CDK)
• Knowledge of orchestration tools such as Apache Airflow or AWS Step Functions
• AWS Certified Data Engineer – Associate or AWS Certified Big Data / Data Analytics Specialty

How We Work With You

Our talent team at Get A Job.ai partners with leading organizations to connect exceptional engineering professionals with confidential opportunities. Here's our process:

• Apply directly through the Get A Job.ai platform
• A dedicated recruiter from our team will conduct an initial screening conversation
• We'll prepare and submit your profile to our client for consideration
• Please do not attempt to contact the client directly—all communication flows through Get A Job.ai to protect both candidate and client interests

Pay

Compensation details will be discussed during the screening process and are competitive with senior data engineering roles in the Costa Rica and El Salvador markets.

Equal Opportunity: Get A Job.ai is committed to inclusive recruiting practices. We welcome applications from all qualified candidates regardless of race, color, religion, sex, national origin, age, disability, or any other protected characteristic.