14 Oct
|
PaxusBB
|
Melbourne
Job Description
Data Product Development Data Engineer | NV1+
n
Melbourne preferred | On-site 3–5 days per week | NV1 minimum
n
We are seeking an experienced Data Product Development Data Engineer to join a large and complex Federal Government data engineering environment.
n
This role is focused on taking raw data brought into a central data platform, conforming and integrating datasets, developing transformation logic and building reusable data products .
n
You’ll work extensively with Apache Spark, Python, Apache NiFi, Apache Iceberg and Azure DevOps , developing scalable data solutions within a secure enterprise environment.
n
What you’ll be doing
n
n
- Build and maintain data engineering solutions within a large enterprise data platform.
n
- Transform and conform raw data into structured, reusable datasets.
n
- Integrate multiple datasets to develop data products.
n
- Develop distributed data processing solutions using Apache Spark and Python .
n
- Build and maintain integration workflows using Apache NiFi .
n
- Work with Apache Iceberg to support modern data storage and processing.
n
- Use Azure DevOps for source control and engineering delivery.
n
- Develop and optimise transformation and integration logic.
n
- Troubleshoot and enhance existing data engineering solutions.
n
- Work with platform, engineering and technical stakeholders to turn data requirements into working solutions.
n
n
What you’ll need
n
Mandatory:
n
n
- At least 1 year of Apache Spark experience using Python .
n
- At least 1 year of Azure DevOps experience .
n
- At least 1 year of Apache NiFi experience .
n
- Experience with Apache Iceberg .
n
- Current NV1 security clearance or higher .
n
- Ability to work on-site 3–5 days per week .
n
n
Highly regarded:
n
n
- NV2 clearance.
n
- Melbourne-based.
n
- Cloudera exposure.
n
- Microsoft SQL Server experience.
n
- Azure Cloud experience.
n
n
Why consider this opportunity?
n
This is an prospect to work within a significant enterprise data platform and focus on the engineering work that turns raw data into usable, integrated data products.
n
The role would suit an experienced Data Engineer who enjoys solving complex data problems, working with modern distributed processing technologies and building practical data solutions.
n
Apply now with your CV, availability and expected daily rate.
n
To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Rashmi Kapse on + or email and quote the job reference number.
n
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.
📌 Data Product Development Data Engineer+ (Melbourne)
🏢 PaxusBB
📍 Melbourne