Big Data Consultant

About This Opportunity

We are representing a confidential technology organization seeking a Big Data Consultant to join their team in Singapore. Our talent acquisition team is conducting a targeted search for an experienced data engineering professional who can drive complex big data initiatives and collaborate effectively across global teams.

Responsibilities

• Design, develop, and implement Hadoop-based data solutions using PySpark, Scala, or Java
• Ingest and process large-scale datasets into the big data platform using Spark with Hive
• Develop and optimize SQL scripts, including complex queries for data processing with Slowly Changing Dimension (SCD) handling
• Build and maintain Hive-based data warehouse solutions on HDFS infrastructure
• Create and maintain shell scripts to support data pipeline automation
• Deliver modules on time while meeting quality standards and project scope requirements
• Perform data analysis and troubleshoot issues within SQL environments
• Document technical solutions and maintain clear communication with stakeholders
• Collaborate with offshore teams across different time zones as project needs require

What We're Looking For

Required Qualifications:

• Minimum 5 years of technical experience in data engineering or related fields
• At least 4+ years of hands-on development experience with Hadoop programming (HDFS) using PySpark, Scala, or Java
• Proven experience delivering Hive-based data warehouse projects
• Strong shell scripting capabilities
• Demonstrated ability to write complex SQL scripts and handle data processing requirements
• Experience with SCD (Slowly Changing Dimensions) methodologies
• Excellent verbal and written communication skills
• Strong documentation abilities
• Proven track record as a collaborative team player
• Flexibility to work across regional time zones
• Willingness to pursue Databricks Certified Developer: Apache Spark 3.X certification

Preferred Qualifications:

• At least 1+ years of Module Lead or technical leadership experience
• Familiarity with Autosys, Bitbucket, Jenkins, or similar tools
• Experience with Confluence and JIRA
• Prior exposure to Oracle, SQL Server, or ETL platforms
• Hands-on experience with performance tuning of Hive queries and cluster optimization
• Functional testing experience, including test plan creation and execution
• Proactive problem-solving approach with strong technical communication skills
• Ability to work independently with minimal supervision

How We Work With You

Candidates should apply directly through Get A Job.ai. Our recruiting team will conduct an initial screening to assess fit and discuss your background in detail. We will then submit qualified candidates to our client for consideration. Please do not attempt to contact the client directly, as all communication should flow through our recruitment process to ensure the best experience for everyone involved.

Pay

Compensation details will be discussed with qualified candidates during the screening process and will be competitive with Singapore market rates for senior big data professionals.

Equal Employment Opportunity: Get A Job.ai is committed to inclusive hiring practices and providing equal employment opportunities to all qualified applicants regardless of race, color, religion, sex, national origin, age, disability, or any other protected characteristic.