02 Aug
|
The HR Ally
|
City of Sydney
02 Aug
The HR Ally
City of Sydney
Location: Sydney, NSW
Experience: 9–14 Years
Employment Type: Full-Time
Eligibility: Australian Citizens & Permanent Residents (PR) Only
Sponsorship: Not Available
About the Role
We are seeking an experienced Data Engineer with expertise in AWS SageMaker Unified Studio to design and build scalable data platforms, real‑time and batch data pipelines, and enterprise data solutions on AWS.
Key Responsibilities
- Design and develop batch and real‑time data ingestion frameworks.
- Build ETL/ELT pipelines using Python, PySpark, SQL, and Apache Airflow (MWAA) .
- Develop streaming solutions using AWS Kinesis, Kafka , or similar technologies.
- Build and manage solutions in AWS SageMaker Unified Studio , including ML pipelines, data processing, governance, and project setup.
- Develop data solutions using Amazon Redshift, S3, Glue, Lambda, and DynamoDB .
- Manage Airflow workflows, monitoring, logging, and performance optimisation.
- Collaborate with Data Engineers, Data Scientists, and Architects to deliver scalable AWS data solutions.
Required Skills
- 9–14 years of Data Engineering experience.
- Solid hands‑on experience with:
- AWS SageMaker Unified Studio
- AWS
- Python
- PySpark
- SQL
- Apache Airflow (MWAA)
- Amazon Redshift
- Experience with AWS Kinesis , Apache Kafka, or similar streaming platforms.
- Strong knowledge of AWS services including S3, Glue, Lambda, DynamoDB, CloudWatch, SQS, SNS, API Gateway, and IAM .
- Experience with relational and NoSQL databases (Oracle, RDS, Redshift, MongoDB, Snowflake, DynamoDB).
- Good understanding of Data Warehousing , Data Lakehouse , Git, CI/CD, Docker, and Infrastructure as Code (Terraform/CloudFormation).
Preferred Skills
- Experience with Machine Learning pipelines, MLOps, and Generative AI in SageMaker.
- Data Modelling experience.
- Enterprise‑scale AWS implementation experience.
Eligibility
- Australian Citizens & Permanent Residents (PR) only.
- Candidates must be currently residing in Australia.
- No visa sponsorship is available.
#J-18808-Ljbffr
📌 Data Engineer – AWS SageMaker Unified Studio-Australia Only (City of Sydney)
🏢 The HR Ally
📍 City of Sydney