We are looking for an experienced Senior Data Engineer / Staff Engineer to join our data engineering team and help build highly scalable, reliable, and cloud-native data platforms on AWS . The role requires strong expertise in Python, SQL, AWS data services, distributed data processing, and modern engineering practices , with the ability to contribute to solution design and technical direction.

Key Responsibilities

- Architect and develop robust data ingestion, transformation, and processing pipelines capable of handling large and complex datasets.
- Develop scalable data platforms using AWS Glue, Amazon Redshift, Amazon Athena, AWS Step Functions , and other AWS-native technologies.
- Design and implement both batch and real-time data processing frameworks using Kafka, Apache Flink , and related technologies.
- Build APIs, automation frameworks, and reusable services to support data engineering and platform capabilities.
- Establish effective practices for data quality, governance, security, lineage, and compliance .
- Develop and optimise logical, semantic, and physical data models based on business and analytical requirements.
- Work closely with architects, product teams, business stakeholders, and engineering squads to translate requirements into scalable technical solutions.
- Drive engineering automation through CI/CD and DevOps practices , leveraging Jenkins, Git, Docker, and Kubernetes.
- Analyse platform performance, identify bottlenecks, and implement improvements for scalability, availability, and reliability.
- Participate in architecture discussions,



technical design reviews, and engineering standards.
- Provide technical leadership and mentorship to other members of the data engineering team.

Essential Skills

- Strong commercial experience in Python and SQL .
- Hands-on experience developing data solutions on AWS .
- Strong experience with AWS Glue, Redshift, Athena, and Step Functions .
- Solid understanding of ETL/ELT, data pipelines, data integration, and distributed processing .
- Practical experience with Kafka and/or Apache Flink .
- Robust knowledge of data modelling, data quality, governance, and security .
- Experience with Jenkins, Git, Docker, Kubernetes, IAM, and Vault .
- Experience working with APIs and automation frameworks.
- Strong understanding of Agile development and modern software engineering practices.

Desirable Experience

- Experience with Apache Iceberg and Data Lakehouse architectures .
- Exposure to real-time analytics and event-driven architectures .
- Experience working with high-volume or enterprise-scale data platforms.
- Background in financial services or other highly regulated environments .
- Experience contributing to cloud data architecture and technical strategy.

What We’re Looking For

The successful candidate will be someone who can operate beyond day-to-day pipeline development, taking ownership of technical design, platform scalability, engineering standards, and complex data problems while working closely with multiple stakeholders and mentoring other engineers.

#J-18808-Ljbffr

📌 AWS Data Engineer (City of Sydney)
🏢 Silverlink Technologies
📍 City of Sydney