20 Aug
|
RESOURCE CORNER
|
New South Wales
20 Aug
RESOURCE CORNER
New South Wales
Job Description
Job Overview
n
Location: Sydney
n
Note: Dependent visa with full work permit OR any visa with unlimited work rights are eligible
n
Key Responsibilities
n
n
Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.
n
Ensure pipelines are resilient, performant, and cost optimised.
n
Implement monitoring, alerting, and automated recovery processes.
n
Apply data quality checks and reconciliation processes to ensure accuracy and completeness.
n
Maintain metadata, lineage, and documentation to support governance requirements.
n
Adhere to role based access control, data privacy, and security standards.
n
Contribute to the development and maintenance of ingestion frameworks, layers, and data models.
n
Implement and maintain CI/CD pipelines for automated testing and deployment.
n
Optimise data storage and retrieval strategies to meet performance needs.
n
Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit for purpose solutions.
n
Participate in code reviews and knowledge sharing sessions to improve team capability.
n
Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.
n
Identify opportunities to automate and streamline workflows.
n
n
Key Skills & Experience
n
n
Proficient in SQL for data wrangling, transformation, and analysis.
n
Robust programming skills in Python, Scala, or Java.
n
Experience building ETL/ELT pipelines in cloud or distributed environments.
n
Familiarity with cloud platforms and services.
n
Knowledge of data modeling concepts.
n
Experience with data orchestration tools.
n
Familiar with version control and CI/CD pipelines.
n
Understanding of data quality, validation, and observability practices.
n
Works independently with minimal supervision on well‐scoped tasks.
n
Collaborates effectively with stakeholders.
n
Communicates clearly and documents work for others to maintain or build upon.
n
Open to code reviews, feedback, and continuous learning.
n
Able to troubleshoot and resolve data pipeline failures or performance issues.
n
n
Qualifications & Experience
n
n
4 years of experience in a data engineering or closely related role.
n
Has delivered production‐ready pipelines end‐to‐end.
n
Demonstrated ability to improve code quality, reliability, or performance.
n
Experience working with real‐time or near real‐time datasets.
n
n
Seniority level
n
n
Mid‐Senior level
n
n
Employment type
n
n
Contract
n
n
Job function
n
n
Information Technology
n
n
Industries
n
n
IT Services and IT Consulting
n
#J-*****-Ljbffr
📌 Senior Data Engineer (New South Wales)
🏢 RESOURCE CORNER
📍 New South Wales