Data Engineer – Kafka/Flink

Employment Type: Perm/Contract

Note: A minimum of 10 years of experience is mandatory.

Job Summary:

We are looking for a highly skilled Data Engineer with strong experience in Apache Flink and Apache Kafka to build and maintain scalable, high-performance real-time data processing pipelines. The ideal candidate will have expertise in streaming architectures, distributed systems, and data engineering best practices, with the ability to design, develop, and optimize robust data platforms that support business-critical analytics and operational workloads.

This role requires a strong understanding of event-driven architecture, stream processing, and integration of large-scale data systems in enterprise environments.

Key Responsibilities:

Data Pipeline Development

- Design, develop, and maintain real-time and batch data pipelines using Apache Flink , Apache Kafka , and related big data technologies.
- Build scalable and reliable stream processing applications for high-volume, low-latency data workloads.
- Develop data ingestion frameworks to capture, transform, and process structured and unstructured data from multiple sources.
- Implement robust ETL/ELT workflows to support analytics, reporting, and downstream systems.

Streaming & Event-Driven Architecture:

- Design and manage event-driven data architectures using Kafka producers, consumers, topics, partitions, and connectors .
- Develop and optimize Apache Flink jobs for stateful and stateless stream processing, windowing, event-time processing, and complex event handling.
- Ensure high throughput, fault tolerance, and exactly-once or at-least-once processing guarantees where required.
- Monitor and troubleshoot streaming applications to ensure high availability and performance.

Data Platform Engineering:





- Work with distributed systems and cloud/on-premise data platforms to build resilient and scalable data solutions.
- Integrate Kafka and Flink with data lakes, warehouses, APIs, and enterprise platforms.
- Develop reusable frameworks, libraries, and automation scripts for data engineering operations.
- Optimize data storage, partitioning, and pipeline performance for cost and efficiency.

Collaboration & Stakeholder Engagement:

- Collaborate with data architects, software engineers, DevOps teams, analysts, and business stakeholders to understand requirements and deliver effective data solutions.
- Translate business and technical requirements into scalable engineering designs.
- Support data consumers by ensuring data quality, consistency, and accessibility across systems.
- Participate in architecture reviews, sprint planning, and technical design discussions.

Monitoring, Quality & Governance

- Implement logging, alerting, and monitoring for Kafka and Flink-based applications.
- Ensure data quality, lineage, governance, and security standards are maintained across the data platform.
- Perform root cause analysis and resolve production issues related to data ingestion and stream processing.
- Contribute to best practices in coding, testing, deployment, and operational support.

Required Skills & Qualifications

- Bachelor’s degree in Computer Science, Information Technology, Engineering , or a related field.




- Proven experience as a Data Engineer , Big Data Engineer , or Streaming Data Engineer .
- Strong hands-on experience with Apache Kafka and Apache Flink in enterprise-scale environments.
- Solid experience in building real-time streaming pipelines and distributed data processing systems.
- Proficiency in Java, Scala, or Python for data engineering and stream processing development.
- Strong understanding of event-driven architecture , message queues, data streaming concepts, and distributed systems.
- Experience with Kafka topics, partitions, brokers, consumers, producers, Kafka Streams, and Kafka Connect .
- Expertise in Flink DataStream API, state management, checkpoints, windowing, and fault tolerance mechanisms .
- Valuable knowledge of SQL and data modeling concepts.
- Experience with cloud platforms such as AWS, Azure, or GCP is an advantage.
- Familiarity with CI/CD pipelines, containerization, and orchestration tools such as Docker and Kubernetes .
- Strong problem-solving, debugging, and performance tuning skills.
- Knowledge of relational and NoSQL databases.
- Familiarity with cloud platforms such as AWS, Azure, or GCP .
- Experience with workflow orchestration tools like Airflow .
- Understanding of DevOps practices, version control, and CI/CD tools.

Preferred Qualifications

- Experience with Kafka Connect, Kafka Streams, Schema Registry, and Confluent Platform .
- Knowledge of containerization tools such as Docker and Kubernetes .
- Experience working with data lakes and modern cloud data platforms.
- Understanding of monitoring tools such as Prometheus, Grafana, or ELK .
- Bachelor’s degree in Computer Science, Engineering, Information Technology , or related field.

📌 Data Engineer(KAFKA) (Southern Sydney)
🏢 Synechron
📍 Southern Sydney