14 Sep
|
Umbrella Club
|
New South Wales
14 Sep
Umbrella Club
New South Wales
Job Description
We are looking for an experienced Data Engineer to join a team focused on building and supporting scalable, cloud-native data platforms.
n
What you'll be doing
n
n
Design, develop and support cloud-native data platforms and data pipelines.
n
Build and maintain ETL/ELT pipelines using AWS services.
n
Work extensively with SQL, Python and Spark/PySpark for data processing and transformation.
n
Develop solutions using AWS S3, Glue, EMR, Athena and Aurora PostgreSQL.
n
Apply data modelling and database design principles to support scalable data solutions.
n
Work with Change Data Capture (CDC) and operational databases.
n
Contribute to data governance, monitoring and CI/CD processes.
n
Work with GitHub Actions to support automated deployments and delivery.
n
Collaborate with stakeholders and delivery teams in an Agile environment.
n
Troubleshoot data pipeline and platform issues and contribute to continuous improvement.
n
n
What we're looking for
n
n
Strong commercial experience as a Data Engineer.
n
Hands-on expertise in SQL and Python.
n
Strong experience with AWS, particularly S3, Glue, EMR, Athena and Aurora PostgreSQL.
n
Strong knowledge of data modelling.
n
Hands-on experience with Spark/PySpark.
n
Experience with CDC, data governance, CI/CD and monitoring.
n
Experience working with operational databases.
n
Solid problem-solving and stakeholder engagement skills.
n
Experience working in Agile delivery environments.
n
n
Nice to have
n
n
Kafka / Amazon MSK
n
Terraform
n
Event-driven architectures
n
Snowflake
n
dbt
n
📌 Data Engineer (New South Wales)
🏢 Umbrella Club
📍 New South Wales