Job Description
Join to apply for the GCP BigData Engineering role at XPT Software
n
Sydney, New South Wales, Australia
n
Top 3 Must‐Have Skillsets
n
n
GCP BigData Engineering (BigQuery + Dataform + Pub/Sub)
n
Expert in designing and optimising BigQuery schemas,
partitioning/clustering,
cost/performance tuning, query optimisation, and policy tag integration.
n
Building streaming and batch pipelines using Apache Beam/Dataflow and Pub/Sub with exactly‐once semantics, backpressure handling, and replay strategies.
n
Strong experience with Dataform (or similar) for SQL‐based transformations, dependency graphs, unit tests, and multi‐environment deployments.
n
Production‐grade Python for ETL/ELT, distributed processing, robust error handling, and testable modular design.
n
Designing resilient Airflow DAGs on Cloud Composer: dependency management, retries, SLAs, sensors, service accounts, and secrets.
n
Monitoring, alerting, and Cloud
Logging/Stackdriver
integration for end‐to‐end pipeline observability.
n
Data Security & Governance on GCP
n
Hands‐on with Dataplex (asset management, data quality, lineage), BigQuery policy tags, Cloud IAM (least privilege, fine‐grained access), KMS (key rotation, envelope encryption), and audit trails via Cloud Logging.
n
Practical experience implementing PII controls (data masking, tokenisation, attribute‐based access control) and privacy‐by‐design in pipelines.
n
n
Additional Expertise
n
n
Cloud Run & APIs: Building stateless microservices for data access/serving layers, implementing REST/gRPC endpoints,
authentication/authorisation,
rate limiting.
n
Data Modelling: Telecom‐centric event models (e.g., CDRs, network telemetry, session/flow data),



star/snowflake schemas, and lakehouse best practices.
n
Performance Engineering: BigQuery slot management, materialised views, BI Engine, partition pruning, cache strategies.
n
Secure Source Manager (CI/CD): Pipeline‐as‐code, automated tests, artifact versioning, workplace promotion, canary releases, and GitOps patterns.
n
Data Quality & Testing: Great Expectations/Deequ‐like checks, schema contracts, anomaly detection, and automated data validations in CI/CD.
n
Streaming Patterns: Exactly‐once delivery, idempotent sinks, watermarking, late data handling, windowing strategies.
n
Observability & SRE Practices: Metrics, logs, traces, runbooks, SLIs/SLOs for data platforms, major incident response to support DevOps.
n
Cost Governance: BigQuery cost controls, slot
commitments/reservations,
workload management, storage lifecycle policies.
n
Domain Knowledge (Mobile Networks): Familiarity with 3G/4G/5G network data, OSS/BSS integrations, network KPIs, and typical analytics use cases.
n
n
Experience Level
n
n
7+ years total in data engineering; 4+ years on GCP with production systems
n
Evidence of impact:
n
• Led end‐to‐end delivery of large‐scale pipelines (batch + streaming) with strict PII governance.
n
• Owned performance/cost optimisation initiatives in BigQuery/Dataflow at scale.
n
• Implemented CI/CD for data workflows (Secure Source Manager) including automated tests and environment promotion.
n
• Drove operational excellence (SLAs, incident management, RTO/RPO awareness, DR patterns).
n
Soft skills: Technical leadership, code reviews, mentoring, clear documentation, cross‐functional collaboration with Network/Analytics teams, and a bias for automation & reliability.
n
#J-*****-Ljbffr

📌 Gcp Bigdata Engineering (New South Wales)
🏢 XPT Software
📍 New South Wales