Senior GCP Data Architect / Data Engineering Lead – Sydney/Melbourne

Delivery Centric is seeking a Senior GCP Data Architect / Data Engineering Lead to design and deliver large-scalereal-time and batch transaction data platforms on Google Cloud. The role combines solution architecture, hands-on data engineering, technical leadership and customer collaboration.

Key Responsibilities

- Lead technical architecture and solution design for large-scale transaction data platforms.
- Provide technical guidance onGCP architecture, data modelling, schema design and engineering best practices .
- Design and implement real-time and batch data ingestion and processing pipelines.
- Develop pipelines includingKafka to Spanner/BigQuery via Dataflow and GCS to Spanner/BigQuery .
- Independently develop, unit test and hand over production-ready data solutions.
- Perform pair programming and provide technical guidance to data engineering teams.
- Lead sprint planning, backlog management, scope definition, risks, issues and delivery timelines.
- Work as a technical workstream or overall project technical lead depending on project scale.
- Design solutions for migrating existing data platforms toGCP cloud-native services and for greenfield GCP implementations.
- Optimise BigQuery queries and data workloads for performance and scalability.
- Develop CI/CD pipelines for data workloads usingCloud Build, Artifact Registry and Terraform .
- Design orchestration workflows usingCloud Composer and Cloud Workflows .
- Support data governance and cataloguing usingDataplex and Data Catalog .
- Apply Generative AI technologies and integrations where appropriate within enterprise data solutions.

Must-Have Skills





- Strong hands-on experience withGoogle Cloud Platform data and analytics technologies .
- Strong experience with GCP Dataflow, BigQuery and Cloud Spanner .
- Experience with real-time/event-driven data ingestion usingConfluent Kafka and Pub/Sub .
- Experience with Cloud Composer and Cloud Workflows .
- Strong experience with Terraform and Cloud Build for data engineering workloads.
- Hands-on programming experience withPython and Java .
- Experience designing and implementing batch and event-driven data pipelines from sources including databases, files, APIs and streaming platforms.
- Experience with BigQuery performance optimisation .
- Experience with CI/CD, infrastructure automation and production data workload deployment.
- Robust understanding of data architecture, solution design and engineering best practices.

Good-to-Have Skills

- DBT or Dataform .
- Experience with Cloud Run and enterprise GCP services.
- Experience with Dataplex and Data Catalog .
- Experience modernising or migrating legacy data platforms to GCP.
- Experience with legacy technologies such asHadoop, Spark, Cloudera/Hortonworks, HBase and Oozie .
- Experience with NoSQL databases such as HBase or MongoDB.
- Knowledge of Spanner, Bigtable, Cloud SQL and database migration approaches.
- Experience modernising legacy data-processing logic such asHive, Teradata and Python into BigQuery SQL.
- Experience applying Generative AI technologies within enterprise environments.
- GCP Data Engineering certification.

Key Skills

GCP | Dataflow | BigQuery | Cloud Spanner | Pub/Sub | Confluent Kafka | Cloud Composer | Cloud Workflows | Terraform | Cloud Build | Python | Java | Real-Time Data | Batch Data | Transaction Data Platforms | Data Architecture | Data Engineering| Technical Leadership

#J-18808-Ljbffr

📌 Senior GCP Data Architect / Data Engineering Lead – Sydney/Melbourne
🏢 Delivery Centric
📍 Sydney