22 Sep
|
Paxus
|
Victoria
Job Description
Melbourne preferred | On-site 3-5 days per week | NV1 minimum
n
We are seeking an experienced Data Product Development Data Engineer to join a large and complex Federal Government data engineering environment.
n
This role is focused on taking raw data brought into a central data platform, conforming and integrating datasets, developing transformation logic and building reusable data products.
n
You'll work extensively with Apache Spark, Python, Apache NiFi, Apache Iceberg and Azure DevOps, developing scalable data solutions within a secure enterprise environment.
n
What you'll be doing
n
n
Build and maintain data engineering solutions within a large enterprise data platform.
n
Transform and conform raw data into structured, reusable datasets.
n
Integrate multiple datasets to develop data products.
n
Develop distributed data processing solutions using Apache Spark and Python.
n
Build and maintain integration workflows using Apache NiFi.
n
Work with Apache Iceberg to support modern data storage and processing.
n
Use Azure DevOps for source control and engineering delivery.
n
Develop and optimise transformation and integration logic.
n
Troubleshoot and enhance existing data engineering solutions.
n
Work with platform,
engineering and technical stakeholders to turn data requirements into working solutions.
n
n
What you'll need
n
Mandatory:
n
n
At least 1 year of Apache Spark experience using Python.
n
At least 1 year of Azure DevOps experience.
n
At least 1 year of Apache NiFi experience.
n
Experience with Apache Iceberg.
n
Current NV1 security clearance or higher.
n
Ability to work on-site 3-5 days per week.
n
n
Highly regarded:
n
n
NV2 clearance.
n
Melbourne-based.
n
n
Why consider this opportunity?
n
This is an opportunity to work within a significant enterprise data platform and focus on the engineering work that turns raw data into usable, integrated data products.
n
The role would suit an experienced Data Engineer who enjoys solving complex data problems, working with up-to-date distributed processing technologies and building practical data solutions.
n
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.
📌 Data Product Development Data Engineer | Nv1+ (Victoria)
🏢 Paxus
📍 Victoria