We're looking for a hands-onData Engineerwith strong experience building reliable, real-time data pipelines usingAmazon Kinesis and Databricks.
You'll work with modern AWS data technologies to develop and optimise streaming pipelines, moving high-volume data from Kinesis into the Databricks Lakehouse using Spark Structured Streaming and Delta tables.
What you'll do:
Build and support real-time data pipelines usingAmazon Kinesis Data Streams and Kinesis Data Firehose.
Develop streaming solutions inDatabricks using Spark Structured Streaming.
Ingest and manage streaming data withinDelta tables.
Optimise pipelines for performance, latency, reliability and cost.
Build monitoring and alerting usingAWS CloudWatch.
Work with batch data engineers to ensure streaming and batch pipelines integrate effectively.
Develop clean, testable Python and SQL code.
Troubleshoot pipeline and data-processing issues and contribute to continuous improvement.
What you'll bring:
Hands-on production experience withAmazon Kinesis Data Streams and/or Kinesis Data Firehose.Kinesis experience is essential.
StrongPython and SQLskills.
Experience withApache Spark Structured Streaming.
Hands-on experience withDatabricks,
ideally including Delta Lake and Unity Catalog.
Good knowledge of AWS services, particularlyIAM, S3 and CloudWatch.
Understanding of scalable and reliable data-pipeline design.
Solid troubleshooting and problem-solving skills.
Experience withKafkais beneficial
Additional Requirements:
Security Clearance Requirement:Given the secure nature of this role, we are seeking candidates who are Australian Citizens with an active security clearance.
This requirement ensures that all team members are cleared to handle sensitive information and contribute to maintaining the highest standards of security in our operations.
Benefits of working at Accenture:
18 weeks paid parental leave
Long & short-term career break opportunities
Structured career development program
Local and international career opportunities.
Certified as a Family Inclusive Workplace
Flexible Work Arrangements - centred around Accenture's Truly Human ethos and our commitment to supporting the health and wellbeing of our people.
We are proud to be in the top 3 of last year's Diversity & Inclusion Index!
#LI_GM
📌 Data Engineer – Aws Kinesis & Databricks - Au Citizen (Canberra)
🏢 Accenture
📍 Canberra
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.