22 Nov
|
TaskVerse
|
Australia
22 Nov
TaskVerse
Australia
About The Company
Softtest pays pty delivers technology and data services for organisations working with complex information and demanding operational requirements. The company supports the design and delivery of secure, scalable data solutions, helping teams make trusted information available for analysis and informed decision-making. Its work brings together data engineering, cloud infrastructure, storage, analytics and security, with a focus on building solutions that can perform reliably as data volumes and business needs grow.
About The Role
Softtest pays pty is seeking an experienced Data Engineer to develop and improve cloud-based data platforms and pipelines. In this role, you will work across the data lifecycle, from ingestion and transformation through to storage, query optimisation and analytics enablement. You will help move complex or large datasets into usable, well-structured forms, applying sound engineering practices to ensure that data is dependable, secure and fit for its intended use.
The successful candidate will bring practical experience with ETL and ELT development in cloud environments and understand how data architecture choices affect performance, scalability and maintainability. You will collaborate with engineering, storage and analytics stakeholders to deliver solutions that meet technical and operational requirements.
Experience with Azure Data Factory and Databricks is desirable, as is familiarity with data lakes and lakehouse architectures.
This opportunity is available to Australian citizens residing in Australia. Applicants must hold a valid Baseline security clearance or be willing and eligible to obtain one.
The role requires a considered approach to information security and the handling of data in environments where access, governance and operational controls matter.
Qualifications
- Demonstrated experience developing ETL or ELT processes for complex and/or large-scale data movement, transformation or visualisation, particularly in cloud environments.
- Experience preparing and structuring data to support efficient query performance in cloud compute engines.
- Practical knowledge of distributed computing technologies such as Spark, and familiarity with relevant database technologies, including Azure SQL or graph databases.
- Experience working with cloud infrastructure services across data engineering, storage and analytics.
- Understanding of data security principles and the importance of applying appropriate controls to data platforms and workflows.
- Experience with Azure Data Factory and Databricks is desirable.
- Knowledge of data lakes and lakehouse architectures is an advantage.
- Australian citizenship and residence in Australia are required; candidates must hold, or be eligible and willing to obtain, a Baseline security clearance.
Responsibilities
- Design, build and maintain robust ETL and ELT pipelines for the movement, transformation and preparation of data.
- Develop solutions that support large or complex data workloads within cloud environments.
- Optimise data structures and processing patterns for query performance across cloud computing and analytics services.
- Use distributed processing approaches, including Spark where appropriate, to handle data efficiently and reliably.
- Work with cloud data storage, engineering and analytics services to deliver integrated platform capabilities.
- Contribute to data lake and lakehouse solutions, supporting organised, accessible and reusable data assets.
- Partner with engineering and analytics colleagues to clarify requirements, resolve data issues and deliver maintainable outcomes.
- Apply security-aware practices throughout data development, processing and storage activities.
- Review and improve existing pipelines and data processes for reliability, scalability and operational effectiveness.
Benefits Join a team working on meaningful cloud data challenges across engineering, storage and analytics. The role offers the chance to apply and extend your expertise with modern data platforms while collaborating with colleagues who contribute different technical perspectives. You will have scope to strengthen your experience in distributed processing, cloud data services and secure data delivery, and to contribute to solutions designed for practical use and long-term performance.
Equal Opportunity
Softtest pays pty is committed to fair and respectful employment practices and welcomes applications from qualified people with diverse backgrounds and experiences. Employment decisions are based on role requirements, relevant capabilities and business needs, without discrimination on grounds protected by applicable law.
📌 Data Engineer (Australia)
🏢 TaskVerse
📍 Australia