My client is seeking a Senior Data Engineer with solid AWS data engineering experience to design, build and optimise scalable data pipelines across both batch and real-time environments. This role is ideal for someone with expertise in Python, PySpark, Airflow and AWS services, who enjoys working on modern cloud data platforms and high-volume data processing.

Key Responsibilities

- Design, develop and maintain scalable batch and real-time data ingestion frameworks.
- Build and optimise data pipelines using Python, PySpark, SQL and AWS services.
- Develop streaming solutions using AWS Kinesis, Kafka or similar technologies.
- Build and manage workflows using Apache Airflow (MWAA).
- Work with Amazon Redshift, RDS, S3 and NoSQL databases to support data warehousing and analytics.
- Implement data transformation, monitoring and observability solutions.
- Collaborate with cross-functional teams to deliver secure, high-performing data solutions.

Must Have Skills

- Strong commercial experience with AWS data services.
- Advanced Python,



PySpark and SQL skills.
- Experience with Apache Airflow (MWAA).
- Hands‑on experience with Amazon Redshift.
- Experience building batch and real-time data pipelines.
- Strong knowledge of AWS Kinesis, Kafka or other streaming technologies.
- Experience working with Data Warehouses and Data Lake/Lakehouse architectures.
- Experience with relational and NoSQL databases such as Oracle, Snowflake, MongoDB, DynamoDB or Teradata.

Nice to Have

- Experience with AWS SageMaker Unified Studio.
- Exposure to ML or Generative AI pipelines.
- Data modelling experience.
- Infrastructure as Code using Terraform or CloudFormation.
- CI/CD and Git experience.

Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability.

#J-18808-Ljbffr

📌 Data Engineer (City of Sydney)
🏢 Paxus
📍 City of Sydney