Accenture is seeking a hands-on Data Engineer in Melbourne to build reliable, real-time data pipelines using Amazon Kinesis and Databricks. You will move data from Kinesis to the Databricks Lakehouse via Spark Structured Streaming and Delta tables, optimizing for performance, latency, and cost.
You will ingest streaming data into Delta tables, develop Python and SQL code, and collaborate with batch engineers to ensure seamless integration of streaming and batch workloads.