13 Oct
|
PaxusBB
|
Melbourne
Job Description
Data Product Development Data Engineer | NV1+
n
Melbourne preferred | On-site 3–5 days per week | NV1 minimum
n
We are seeking an experienced
Data Product Development Data Engineer
to join a large and complex Federal Government data engineering environment.
n
This role is focused on taking raw data brought into a central data platform,
conforming and integrating datasets, developing transformation logic and building reusable data products
.
n
You'll work extensively with
Apache Spark, Python, Apache NiFi, Apache Iceberg and Azure DevOps
, developing scalable data solutions within a secure enterprise environment.
n
What you'll be doing
n
n
Build and maintain data engineering solutions within a large enterprise data platform.
n
Transform and conform raw data into structured, reusable datasets.
n
Integrate multiple datasets to develop data products.
n
Develop distributed data processing solutions using
Apache Spark and Python
.
n
Build and maintain integration workflows using
Apache NiFi
.
n
Work with
Apache Iceberg
to support modern data storage and processing.
n
Use
Azure DevOps
for source control and engineering delivery.
n
Develop and optimise transformation and integration logic.
n
Troubleshoot and enhance existing data engineering solutions.
n
Work with platform, engineering and technical stakeholders to turn data requirements into working solutions.
n
n
What you'll need
n
Mandatory:
n
n
At least
1 year of Apache Spark experience using Python
.
n
At least
1 year of Azure DevOps experience
.
n
At least
1 year of Apache NiFi experience
.
n
Experience with
Apache Iceberg
.
n
Current
NV1 security clearance or higher
.
n
Ability to work on-site
3–5 days per week
.
n
n
Highly regarded:
n
n
NV2 clearance.
n
Melbourne-based.
n
Cloudera exposure.
n
Microsoft SQL Server experience.
n
Azure Cloud experience.
n
n
Why consider this prospect?
n
This is an opportunity to work within a significant enterprise data platform and focus on the engineering work that turns raw data into usable, integrated data products.
n
The role would suit an experienced Data Engineer who enjoys solving complex data problems, working with modern distributed processing technologies and building practical data solutions.
n
Apply now with your CV, availability and expected daily rate.
n
To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Rashmi Kapse on + or email and quote the job reference number.
n
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.
📌 Data Product Development Data Engineer | Nv1+ (Melbourne)
🏢 PaxusBB
📍 Melbourne