28 Aug
|
Paxus
|
Victoria
Job Description
n
- Build and maintain scalable ETL/ELT workflows using Informatica (mappings, workflows, scheduling and monitoring)
n
- Manage BigQuery datasets and tables, including partitioning and clustering for large-scale analytics
n
- Write and optimise advanced SQL to improve performance and reduce compute cost
n
- Develop Python scripts for file parsing, orchestration and integration with cloud APIs
n
- Automate operational routines using Unix shell scripting (file transfers, pre/post-load steps, batch sequencing)
n
- Schedule, monitor and troubleshoot integrated batch workflows across on-prem and cloud schedulers
n
- Implement robust data validation, error handling and logging to improve data quality and reliability
n
- Diagnose and resolve bottlenecks across Informatica, BigQuery SQL and scripting components
n
- Maintain technical documentation (mapping specs, runbooks, diagrams) and collaborate with analysts/testing/business stakeholders
n
n
Location: Melbourne
n Contract: 12 month
n
Responsibilities
n
n
- Build and maintain scalable ETL/ELT workflows using Informatica (mappings, workflows, scheduling and monitoring)
n
- Manage BigQuery datasets and tables, including partitioning and clustering for large-scale analytics
n
- Write and optimise advanced SQL to improve performance and reduce compute cost
n
- Develop Python scripts for file parsing, orchestration and integration with cloud APIs
n
- Automate operational routines using Unix shell scripting (file transfers, pre/post-load steps, batch sequencing)
n
- Schedule, monitor and troubleshoot integrated batch workflows across on-prem and cloud schedulers
n
- Implement robust data validation, error handling and logging to improve data quality and reliability
n
- Diagnose and resolve bottlenecks across Informatica, BigQuery SQL and scripting components
n
- Maintain technical documentation (mapping specs, runbooks, diagrams) and collaborate with analysts/testing/business stakeholders
n
n
Requirements
n
n
- 4-7 years' hands-on data engineering experience building and supporting production data pipelines
n
- Strong Informatica experience (development, scheduling, monitoring, troubleshooting)
n
- Proven Google BigQuery capability (table design, dataset management, partitioning/clustering, performance optimisation)
n
- Advanced SQL skills (complex joins, window functions, tuning, validation routines)
n
- Python proficiency for data processing, automation and reusable utilities
n
- Robust understanding of operational monitoring, reconciliation, logging and batch workload support
n
- Demonstrated ability to troubleshoot end-to-end data pipeline issues across tools and scripts
n
- Experience producing and maintaining technical documentation and working in delivery teams
n
n
Desirables
n
n
- Unix shell scripting for orchestration/automation in data pipeline operations
n
- Experience supporting cloud migration or hybrid integration (moving legacy on-prem data into cloud platforms)
n
- Familiarity with Agile delivery, version control and release processes across distributed teams
n
- Relevant degree (Computer Science / IT / Data Engineering or similar) and/or cloud/data/Informatica certifications
n
n
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.
n
#J-18808-Ljbffr
📌 Data Engineer (Victoria)
🏢 Paxus
📍 Victoria