21 Nov
|
Infosys APAC
|
Sydney
21 Nov
Infosys APAC
Sydney
Job Description
We are looking for an experienced GCP Data Engineer with robust hands-on expertise in designing, developing, and supporting scalable data solutions on Google Cloud Platform (GCP). /n /n
Key Responsibilities
/n /n
- /n Design and develop scalable ETL/ELT data pipelines on GCP. /n
- /n Build batch and streaming data solutions using BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Dataproc. /n
- /n Develop data transformation and processing solutions using Python, SQL, and Apache Spark. /n
- /n Design and optimize BigQuery datasets, tables, partitioning, clustering, and complex SQL queries. /n
- /n Build ingestion frameworks to integrate data from databases, APIs, files, SaaS platforms, and event streams. /n
- /n Implement workflow orchestration using Cloud Composer / Apache Airflow. /n
- /n Develop robust data quality, reconciliation, logging, monitoring, and exception-handling frameworks. /n
- /n Implement incremental data processing, CDC, and high-volume data ingestion patterns. /n
- /n Work with CI/CD pipelines and infrastructure automation for deploying GCP data solutions. /n
- /n Implement appropriate IAM, security, encryption, and data governance controls. /n
- /n Troubleshoot performance and production issues across data pipelines. /n
- /n Collaborate with data architects, analysts, business stakeholders, and downstream reporting teams. /n
📌 GCP Data Engineer (Sydney)
🏢 Infosys APAC
📍 Sydney