Role Description

- Design, develop, and maintain scalable data pipelines and data processing systems.
- Build and optimize ETL/ELT workflows to collect, transform, validate, and deliver data from diverse sources.
- Develop and maintain data warehouses, data lakes, and other data storage solutions to support analytics and business applications.
- Integrate data from databases, APIs, cloud platforms, and external systems while ensuring data consistency and reliability.
- Monitor data pipelines and resolve data quality, performance, and availability issues.
- Implement data validation, quality checks, governance, and security practices to maintain trustworthy datasets.
- Optimize data processing workflows, database queries, and infrastructure for scalability and performance.
- Collaborate with data scientists, analysts, software engineers, and business stakeholders to understand data requirements and deliver effective solutions.
- Develop documentation for data models, pipelines, systems, and technical processes.
- Evaluate and adopt appropriate data engineering technologies, frameworks, and cloud services.
- Contribute to the design of reliable, automated, and maintainable data infrastructure.

Qualifications




- Bachelor’s degree or equivalent education in Computer Science, Data Engineering, Software Engineering, Information Technology, Mathematics, or a related quantitative field.
- Strong proficiency in SQL and at least one programming language such as Python, Java, Scala, or similar.
- Solid understanding of relational and non-relational databases, data modeling, and database optimization.
- Experience with ETL/ELT concepts, data pipelines, data integration, and distributed data processing.
- Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud.
- Knowledge of data engineering tools and technologies such as Apache Spark, Airflow, Kafka, dbt, or equivalent platforms is a plus.
- Understanding of data warehousing, data lakes, APIs, and modern data architecture.
- Strong problem-solving and analytical skills with attention to data quality and system reliability.
- Familiarity with Git, CI/CD, testing, and software development best practices.
- Solid communication and collaboration skills, with the ability to work effectively across technical and business teams.
- Willingness to learn new technologies and continuously improve data engineering practices.

-

📌 Data Enginer (Sydney)
🏢 Planning Engineer Jobs
📍 Sydney