We’re looking for a hands-on Data Engineer with strong experience building reliable, real-time data pipelines using Amazon Kinesis and Databricks. You’ll work with contemporary AWS data technologies to develop and optimise streaming pipelines, moving high-volume data from Kinesis into the Databricks Lakehouse using Spark Structured Streaming and Delta tables. What you’ll do:
- Build and support real-time data pipelines using Amazon Kinesis Data Streams and Kinesis Data Firehose.
- Develop streaming solutions in Databricks using Spark Structured Streaming.
- Ingest and manage streaming data within Delta tables.
- Optimise pipelines for performance, latency, reliability and cost.
- Build monitoring and alerting using AWS CloudWatch.
- Work with batch data engineers to ensure streaming and batch pipelines integrate effectively.
- Develop clean, testable Python and SQL code.




- Troubleshoot pipeline and data-processing issues and contribute to continuous improvement. What you’ll bring:
- Hands-on production experience with Amazon Kinesis Data Streams and/or Kinesis Data Firehose. Kinesis experience is essential.
- Strong Python and SQL skills.
- Experience with Apache Spark Structured Streaming.
- Hands-on experience with Databricks, ideally including Delta Lake and Unity Catalog.
- Good knowledge of AWS services, particularly IAM, S3 and CloudWatch.
- Understanding of scalable and reliable data-pipeline design.
- Strong troubleshooting and problem-solving skills.
- Experience with Kafka is beneficial

📌 Data Engineer – AWS Kinesis (Canberra)
🏢 Accenture
📍 Canberra