Data Engineer – Spark & Kafka

We are looking for a Data Engineer with experience in Spark and Kafka to operate with one of our large Federal Government Clients. As a Data Engineer, you will have the chance to work with cutting‑edge technologies and collaborate with cross‑functional teams to ensure successful data integration.

Responsibilities
Maintain and develop Spark pipelines to process large‑scale data sets.
Ingest tables into our data lake from Kafka using batch processing, micro‑batching, or real‑time processing techniques.
Run Spark jobs for ingestion, curation, and extraction of data.
Alter schemas and configs to meet business requirements and ensure data quality.
Attend meetings regarding data ingestion and collaborate with cross‑functional teams to ensure successful data integration.
Troubleshoot and resolve bugs in the data processing pipelines.
Monitor performance and troubleshoot issues using logging and monitoring tools.
Document best practices and develop standards for data processing pipelines.
Stay up‑to‑date with emerging technologies and industry trends in big data processing and analytics.

Requirements
Bachelor’s or Master’s degree in Computer Science, Software Engineering,



or a related field.
3+ years of experience in data engineering, software development, or a related field.
Strong programming skills in Scala, Python, or Java.
Expertise in Spark, Kafka, and big data processing concepts.
Experience with data storage solutions such as Hadoop HDFS or Amazon S3, and data serialization formats.
Knowledge of database concepts, data modelling, schema design, and normalisation.
Familiarity with monitoring and logging tools, and workflow management tools such as Apache Airflow.
Experience with version control systems such as Git.
Security clearance required – Baseline (NV1 preferred).

Attributes and Aptitude
Strong communication, problem‑solving, and collaboration abilities.
Ability to work independently and in a team‑oriented, collaborative environment.

Advantages

We offer competitive compensation packages and perks, along with opportunities for growth and advancement within the company. As a member of our team, you will have the chance to work on challenging projects, develop your skills, and make a real impact on our business.
J-18808-Ljbffr

📌 Data Engineer Canberra
🏢 Nuvento
📍 Canberra