Join to apply for the GCP BigData Engineering role at XPT Software

Sydney, Current South Wales, Australia

Top 3 Must‑Have Skillsets

- GCP BigData Engineering (BigQuery + Dataform + Pub/Sub)
- Expert in designing and optimising BigQuery schemas, partitioning/clustering, cost/performance tuning, query optimisation, and policy tag integration.
- Building streaming and batch pipelines using Apache Beam/Dataflow and Pub/Sub with exactly‑once semantics, backpressure handling, and replay strategies.
- Strong experience with Dataform (or similar) for SQL‑based transformations, dependency graphs, unit tests, and multi‑environment deployments.
- Production‑grade Python for ETL/ELT, distributed processing, robust error handling, and testable modular design.
- Designing resilient Airflow DAGs on Cloud Composer: dependency management, retries, SLAs, sensors, service accounts, and secrets.
- Monitoring, alerting, and Cloud Logging/Stackdriver integration for end‑to‑end pipeline observability.
- Data Security & Governance on GCP
- Hands‑on with Dataplex (asset management, data quality, lineage), BigQuery policy tags, Cloud IAM (least privilege, fine‑grained access), KMS (key rotation, envelope encryption), and audit trails via Cloud Logging.
- Practical experience implementing PII controls (data masking, tokenisation, attribute‑based access control) and privacy‑by‑design in pipelines.

Additional Expertise

- Cloud Run & APIs : Building stateless microservices for data access/serving layers, implementing REST/gRPC endpoints, authentication/authorisation, rate limiting.
- Data Modelling : Telecom‑centric event models (e.g., CDRs, network telemetry, session/flow data), star/snowflake schemas,



and lakehouse best practices.
- Performance Engineering : BigQuery slot management, materialised views, BI Engine, partition pruning, cache strategies.
- Secure Source Manager (CI/CD) : Pipeline‑as‑code, automated tests, artifact versioning, environment promotion, canary releases, and GitOps patterns.
- Data Quality & Testing : Great Expectations/Deequ‑like checks, schema contracts, anomaly detection, and automated data validations in CI/CD.
- Streaming Patterns : Exactly‑once delivery, idempotent sinks, watermarking, late data handling, windowing strategies.
- Observability & SRE Practices : Metrics, logs, traces, runbooks, SLIs/SLOs for data platforms, major incident response to support DevOps.
- Cost Governance : BigQuery cost controls, slot commitments/reservations, workload management, storage lifecycle policies.
- Domain Knowledge (Mobile Networks) : Familiarity with 3G/4G/5G network data, OSS/BSS integrations, network KPIs, and typical analytics use cases.

Experience Level

- 7+ years total in data engineering; 4+ years on GCP with production systems
- Evidence of impact:
- • Led end‑to‑end delivery of large‑scale pipelines (batch + streaming) with strict PII governance.
- • Owned performance/cost optimisation initiatives in BigQuery/Dataflow at scale.
- • Implemented CI/CD for data workflows (Secure Source Manager) including automated tests and environment promotion.
- • Drove operational excellence (SLAs, incident management, RTO/RPO awareness, DR patterns).
- Soft skills: Technical leadership, code reviews, mentoring, clear documentation, cross‑functional collaboration with Network/Analytics teams, and a bias for automation & reliability.

#J-18808-Ljbffr

📌 GCP BigData Engineering (City of Sydney)
🏢 XPT Software
📍 City of Sydney