16 Sep
|
Jobgether
|
New South Wales
16 Sep
Jobgether
New South Wales
Job Description
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer based in Australia.
n
This role offers the opportunity to help build, maintain, and modernize a growing data platform in a cloud-first setting. You will design reliable data pipelines and scalable data solutions that enable teams to access and use high-quality information effectively. The role combines modern AWS infrastructure with opportunities to improve and transition legacy data processes. You will work across batch and real-time data workflows, APIs, integrations, and data warehouse solutions. Collaboration with analytics teams and other stakeholders will be central to understanding requirements and delivering practical solutions. This is a hands-on opportunity to improve data reliability, performance, security, and quality while contributing to the evolution of the broader data architecture.
n
Accountabilities
n
n
- Design, develop, and maintain scalable and reliable data pipelines and Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes.
n
- Develop, maintain, and optimize data warehouse solutions and data models.
n
- Build and maintain data infrastructure primarily on Amazon Web Services (AWS), while supporting relevant legacy systems.
n
- Develop APIs and data integrations to enable effective data exchange across systems.
n
- Contribute to real-time data pipelines and processing using Apache Kafka.
n
- Automate and optimize data processes, workflows, and operational tasks.
n
- Monitor, troubleshoot, and continuously improve the reliability, performance, security, and data quality of pipelines and infrastructure.
n
- Work closely with analytics teams and other stakeholders to understand data requirements and deliver effective technical solutions.
n
- Maintain accurate and up-to-date documentation covering pipelines, infrastructure, workflows, and operational processes.
n
- Contribute to the modernization of legacy data processes, infrastructure, and overall data architecture.
n
n
Requirements
n
n
- 2+ years of professional experience in data engineering.
n
- Strong SQL and Python programming skills.
n
- Hands-on experience with AWS services, particularly Amazon S3, Amazon Redshift, AWS Glue, and AWS Lambda.
n
- Practical experience with Apache Airflow for data workflow orchestration.
n
- Strong understanding of data warehousing principles and data modeling.
n
- Experience designing, developing, and maintaining production-grade data pipelines, with a strong focus on reliability, monitoring, and data quality.
n
- Solid understanding of both relational and non-relational databases.
n
- Familiarity with Git and Continuous Integration/Continuous Deployment (CI/CD) practices, preferably using GitLab.
n
- Experience developing APIs and building data integrations.
n
- Strong written and verbal communication skills.
n
- Upper-intermediate English proficiency or higher.
n
- Experience with Kafka and real-time data processing is a plus.
n
- Experience with Microsoft SQL Server and SQL Server Integration Services (SSIS) is a plus.
n
n
Benefits
n
n
- Full-time position.
n
- Opportunity to work within an Analytics-focused environment.
n
- Remote working arrangement.
n
- Opportunity to contribute to up-to-date cloud-based data infrastructure.
n
- Hands-on exposure to AWS, Airflow, Kafka, data warehousing, and data modernization initiatives.
n
- Opportunity to contribute to the improvement of data reliability, quality, performance, and architecture.
n
- Collaborative work with analytics teams and cross-functional stakeholders.
n
📌 Data Engineer (New South Wales)
🏢 Jobgether
📍 New South Wales