Role DescriptionA Data Engineer is responsible for designing, building, maintaining, and optimizing data infrastructure, pipelines, platforms, and systems that enable organizations to collect, transform, store, govern, and analyze large volumes of data. The role focuses on creating reliable and scalable data solutions that support analytics, business intelligence, machine learning, reporting, and operational decision-making.

Key responsibilities include designing and developing scalable data pipelines for structured, semi-structured, and unstructured data; collecting data from databases, APIs, applications, cloud platforms, enterprise systems, files, and external sources; building ETL and ELT processes for data extraction, transformation, validation, and loading; developing data warehouses, data lakes, lakehouses, data marts, and analytical data platforms; designing efficient data models, schemas, tables, partitions, indexes, and storage structures; writing and optimizing SQL queries and data-processing code; working with relational and NoSQL databases such as PostgreSQL, MySQL, Oracle, SQL Server, MongoDB, and similar technologies; utilizing data-processing frameworks such as Apache Spark, Kafka, Flink, Airflow, dbt, or equivalent technologies; integrating data from multiple systems and ensuring data consistency, completeness, accuracy, and availability; implementing data-quality checks, validation frameworks, monitoring, logging, alerting, and automated error handling; developing batch and real-time streaming data pipelines; supporting data governance, metadata management, lineage, access controls, privacy, security, and regulatory requirements; collaborating with data scientists, analysts, software engineers, BI teams, product managers, cloud engineers, and business stakeholders; supporting machine-learning and artificial-intelligence initiatives by preparing reliable datasets, feature pipelines, and data infrastructure; designing cloud-based data architectures using AWS, Microsoft Azure, Google Cloud,



or other platforms; implementing infrastructure automation, CI/CD, version control, testing, and deployment practices for data workflows; monitoring pipeline performance, resource utilization, data-processing costs, and system reliability; troubleshooting failed pipelines, data discrepancies, integration issues, performance problems, and production incidents; optimizing storage, compute, query performance, and data-processing workloads; developing technical documentation, data dictionaries, architecture diagrams, pipeline specifications, and operational procedures; supporting data migration, modernization, platform upgrades, and legacy-system transformation initiatives; utilizing artificial intelligence, automation, data observability, and intelligent data-management technologies to improve engineering efficiency; and continuously improving data reliability, scalability, security, performance, accessibility, and overall data-platform capabilities.

Qualifications

- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, Software Engineering, Mathematics, Statistics, or a related discipline.
- Solid understanding of data engineering, database systems, data architecture, ETL/ELT, data modeling, and distributed data processing.
- Strong SQL skills and proficiency in one or more programming languages such as Python, Java, Scala, or similar technologies.
- Knowledge of relational databases, NoSQL databases, data warehouses, data lakes, and lakehouse architectures.
- Familiarity with technologies such as Apache Spark, Kafka, Airflow, Flink, dbt, Hadoop, or equivalent data-engineering platforms.




- Strong understanding of batch processing, real-time streaming, event-driven architectures, APIs, and data integration.
- Familiarity with cloud data services and platforms such as AWS, Microsoft Azure, Google Cloud, Snowflake, Databricks, BigQuery, or equivalent technologies.
- Knowledge of data modeling, dimensional modeling, schema design, partitioning, indexing, query optimization, and data-storage strategies.
- Strong understanding of data quality, data validation, data lineage, metadata, governance, security, and privacy.
- Familiarity with Git, CI/CD, infrastructure-as-code, Docker, Kubernetes, and automated deployment practices is advantageous.
- Knowledge of data observability, monitoring, logging, alerting, pipeline orchestration, and performance optimization.
- Understanding of machine learning, artificial intelligence, feature engineering, vector databases, and AI data pipelines is beneficial.
- Strong analytical, troubleshooting, problem-solving, and critical-thinking skills.
- Ability to design scalable, reliable, maintainable, and cost-efficient data solutions.
- Solid communication and collaboration skills when working with engineers, analysts, data scientists, business stakeholders, and technical teams.
- Ability to manage multiple data projects, pipelines, priorities, and deadlines.
- Understanding of Agile, Scrum, DevOps, DataOps, or other modern development methodologies.
- Strong attention to data accuracy, system reliability, security, performance, and operational excellence.
- Relevant cloud, data engineering, database, analytics, or professional certifications are advantageous.
- High level of technical curiosity, adaptability, accountability, and continuous-learning mindset.
- Strong commitment to staying informed about cloud data platforms, real-time analytics, AI infrastructure, data mesh, lakehouse architectures, automation, and emerging data-engineering technologies.

-

📌 Data Enginer (Australia)
🏢 Aku Lodge
📍 Australia