About The Company

Softtest Pays Pty connects skilled technology professionals with opportunities to contribute to complex digital and data initiatives. This role supports work in a cloud-based environment where reliable data engineering helps enable effective analysis, reporting and decision-making. The successful candidate will work with engineering, storage and analytics services and help build data solutions that can handle substantial volumes of information while supporting performance, security and maintainability.

About The Role

We are seeking an experienced Data Engineer to design, develop and improve ETL and ELT processes for complex or large-scale data movement and transformation. You will prepare data for efficient querying and use across cloud computing environments, collaborating with engineering, storage and analytics teams to deliver dependable data products. This position suits a professional who can work across data pipelines, cloud infrastructure and analytical platforms, and who understands how design decisions affect performance, scalability and usability.

Australian citizens residing in Australia are the only candidates eligible to apply. Applicants must hold a current Baseline security clearance or be willing and eligible to obtain one. The role requires sound professional judgement and a commitment to handling data and systems responsibly.

Qualifications

Candidates should demonstrate experience developing ETL or ELT processes that support complex or high-volume data movement, transformation or visualisation, particularly in cloud environments.



You will also bring experience preparing data for query performance in cloud compute engines and related platforms. Relevant technologies and concepts may include distributed computing engines such as Spark, graph databases and Azure SQL.

Experience working with cloud infrastructure services across engineering, storage and analytics is essential. The ability to collaborate with technical stakeholders, understand data requirements and translate them into practical engineering solutions is important. Applicants should be comfortable considering data quality, performance, scalability and operational reliability throughout the data lifecycle.

Experience with Azure Data Factory and Databricks is desirable, as is knowledge of data lakes and lakehouse architectures. Familiarity with SQL, storage services, analytics platforms, databases and cloud security will be valuable. A relevant technical background or equivalent practical experience is expected.

Responsibilities

Design, build, test and maintain scalable ETL and ELT pipelines for data ingestion, transformation and delivery. Work with complex datasets and select suitable processing approaches to support reliable and efficient movement of data.



Prepare datasets and storage structures for effective querying in cloud-based compute environments, taking account of workload characteristics and performance requirements.

Collaborate with engineering, storage and analytics teams to understand requirements and integrate data services across the cloud environment. Apply sound data engineering practices to support data quality, consistency, security and maintainability. Contribute to the development of data solutions using appropriate tools and technologies, including distributed processing platforms where relevant.

Identify bottlenecks, troubleshoot pipeline or query issues, and recommend improvements to data workflows and architecture. Document technical processes and communicate implementation considerations clearly to colleagues and stakeholders.

Benefits

This opportunity offers the chance to contribute to meaningful cloud data engineering work and apply your expertise to large-scale data challenges. You will collaborate with professionals across engineering, storage and analytics, broaden your experience with contemporary data platforms, and support the delivery of solutions designed for performance and practical use. Further engagement details can be discussed during the recruitment process.

Equal Opportunity

Softtest Pays Pty is committed to fair and respectful recruitment. Applicants will be considered on the basis of their relevant skills, experience and eligibility requirements. We welcome applications from qualified candidates and aim to provide an inclusive and professional recruitment experience.

📌 Data Engineer (Australia)
🏢 TaskVerse
📍 Australia