Job Description

We are seeking a highly skilled Senior Software Engineer (SSE) / Staff Data Engineer to design, build, and optimize large-scale data solutions. The ideal candidate will have strong expertise in Python, Spark, and AWS, with a proven track record in developing robust data pipelines, enabling advanced analytics, and ensuring data governance and security.

n

Key Responsibilities

n

n
- Design and implement scalable ETL pipelines for data ingestion, transformation, and integration.
n
- Ensure data quality, governance, and compliance across all stages.
n
- Develop and optimize solutions using Apache Spark, Hadoop, Hive, and Kafka.
n
- Handle large-scale data processing and real-time streaming.
n
- Build and maintain data solutions on AWS (S3, Glue, Redshift, Athena, IAM).
n
- Implement secure and cost-efficient cloud architectures.
n
- APIs & Automation
n
- Develop RESTful APIs and automation scripts for data services and integrations.
n
- Collaborate with application teams to enable data-driven microservices.
n
- DevOps & CI/CD
n
- Implement CI/CD pipelines using Jenkins, Git, Docker, and Kubernetes.
n
- Ensure smooth deployment and monitoring of data applications.
n
- Security & Compliance
n
- Apply best practices for data governance, risk management, and regulatory compliance.
n




- Work closely with cross-functional squads and stakeholders to deliver data solutions in an Agile environment.
n

n

Required Skills & Experience

n

n
- Solid programming skills in Python (Scala experience is a plus).
n
- Hands‐on experience with Spark, Hadoop, Hive, and Kafka.
n
- Expertise in AWS services (S3, Glue, Redshift, Athena, IAM).
n
- Solid understanding of ETL processes, data modeling, and pipeline orchestration.
n
- Familiarity with RESTful APIs, microservices, and automation scripting.
n
- Knowledge of CI/CD tools, containerization (Docker), and orchestration (Kubernetes).
n
- Understanding of data security, compliance, and governance principles.
n
- Excellent problem‐solving skills and ability to work in a fast‐paced Agile environment.
n

n

Preferred Qualifications

n

n
- Experience with data lake and data warehouse architectures.
n
- Exposure to machine learning pipelines or advanced analytics.
n
- Certification in AWS or Big Data technologies.
n

n

Seniority level

n

Mid‐Senior level

n

Employment type

n

Contract

n

Job function

n

Information Technology

n

Industries

n

Software Development

n

Location: Sydney, New South Wales, Australia

n

Salary: A$140,000 - A$180,000

#J-18808-Ljbffr

📌 Data Engineer (SSE / Staff Engineer) - Python & Spark & AWS (New South Wales)
🏢 N2S.Global
📍 New South Wales