23 Sep
|
FR Consultancy
|
Australia
23 Sep
FR Consultancy
Australia
Status:
AU valid visa with full work rights
Must have:
Programming knowledge and willingness to be hands-on - Python, Java
Key Responsibilities
In-depth knowledge and experience of GCP data and analytics technologies
Designing Large scale Transaction Data Platform architecture for (a) migrating open source or public cloud data platforms to GCP cloud native services (b) designing greenfield data platforms on GCP
Define solution architecture and detailed design, and perform hands-on implementation for data pipelines for batch and event driven ingestion and processing of data from a variety of sources such as on-prem files, on-prem databases, APIs, etc.
Real-time data ingestion and event-driven processing pipelines from Confluent Kafka and GCS to Spanner and BigQuery
Automatic transpilation of legacy code (HIVE, Teradata, python logic etc.) to BQ SQL
CI/CD pipelines for data workloads using Cloud Build, Artifact Registry, Terraform
Orchestration setup and Cloud Composer DAG development for data pipeline workflows
Data governance solutioning using GCP governance tooling (Dataplex, Data Catalog) Experience applying Generative AI technologies and integrations in enterprise settings.
Tools and languages experience Required
J-18808-Ljbffr
📌 Gcp Data Engineer Sydney (Australia)
🏢 FR Consultancy
📍 Australia