Role Description A Data Engineer is responsible for designing, building, maintaining, and optimizing data infrastructure and pipelines that enable organizations to collect, process, transform, store, and deliver reliable data for analytics, business intelligence, artificial intelligence, and operational applications. The role works closely with data scientists, analysts, software engineers, BI teams, cloud engineers, and business stakeholders to develop scalable and secure data solutions.

Key responsibilities include designing and developing scalable data pipelines, ETL/ELT workflows, data ingestion processes, transformation frameworks, and data-processing systems; collecting and integrating structured, semi-structured, and unstructured data from databases, APIs, applications, cloud services, files, streaming platforms, and external sources; developing batch and real-time data-processing solutions; designing and maintaining data warehouses, data lakes, lakehouses, data marts, and analytical platforms; implementing data models, schemas, tables, views, data structures, and semantic layers optimized for analytical and operational requirements; writing and optimizing SQL queries, stored procedures, and data-transformation logic; developing data solutions using technologies such as Python, Java, Scala, Spark, Kafka, Airflow, dbt, or equivalent tools; implementing data-quality checks, validation rules, monitoring, reconciliation, and error-handling processes; ensuring data pipelines are reliable, scalable, observable, secure, and maintainable; monitoring pipeline performance, data freshness, processing times, failures, resource utilization, and system availability; troubleshooting data pipeline failures, integration issues, performance bottlenecks, data inconsistencies, and infrastructure problems; designing data integration solutions across cloud, on-premises, hybrid, and distributed environments; working with cloud data platforms such as AWS, Microsoft Azure, Google Cloud, Snowflake, Databricks, BigQuery, Redshift, or equivalent technologies; implementing data governance, metadata management, data lineage, access controls, privacy, security,



and compliance requirements; collaborating with data scientists and analysts to prepare high-quality datasets for analytics, machine learning, reporting, and experimentation; supporting machine-learning and AI workloads through reliable feature pipelines, training datasets, data services, and model-serving infrastructure where applicable; implementing automation, Infrastructure as Code, CI/CD pipelines, version control, testing, and deployment processes for data engineering workflows; utilizing DataOps, observability, orchestration, automation, and AI-assisted engineering tools to improve data-platform reliability and development efficiency; optimizing storage, compute, query performance, processing costs, and data architecture; maintaining technical documentation, data dictionaries, pipeline specifications, architecture diagrams, and operational procedures; supporting disaster recovery, backup, data retention, and business-continuity requirements; participating in data-platform modernization, cloud migration, technology upgrades, and architectural improvement initiatives; collaborating with security, infrastructure, DevOps, application, and business teams; and continuously improving data availability, quality, scalability, performance, security, automation, and overall data-platform capabilities.

Qualifications

- Bachelor's or Master's degree in Computer Science, Data Engineering, Software Engineering, Information Technology, Mathematics, Statistics, or a related discipline.
- Strong understanding of data engineering principles, data architecture, ETL/ELT, data integration, data modeling, and distributed data processing.
- Solid SQL skills and knowledge of relational and non-relational databases.
- Proficiency in Python, Java, Scala, or another relevant programming language.
- Familiarity with Apache Spark, Kafka, Airflow,



dbt, Hadoop, or equivalent data-engineering technologies.
- Strong understanding of data warehouses, data lakes, lakehouses, data marts, and analytical data platforms.
- Knowledge of cloud platforms such as AWS, Microsoft Azure, Google Cloud, Snowflake, Databricks, BigQuery, Redshift, or equivalent technologies.
- Understanding of batch processing, real-time streaming, event-driven architectures, APIs, and data ingestion frameworks.
- Strong knowledge of data quality, validation, monitoring, lineage, metadata, governance, and security practices.
- Familiarity with Git, CI/CD, Docker, Kubernetes, Terraform, and Infrastructure as Code is advantageous.
- Understanding of DataOps, DevOps, automation, workflow orchestration, and software-engineering best practices.
- Knowledge of machine learning, AI data pipelines, feature engineering, model-data requirements, and large-scale datasets is beneficial.
- Familiarity with generative AI, vector databases, retrieval-augmented generation, data pipelines for LLM applications, and AI data infrastructure is advantageous.
- Strong analytical, troubleshooting, debugging, optimization, and problem-solving skills.
- Ability to design scalable, reliable, maintainable, and cost-efficient data architectures.
- Strong understanding of data security, privacy, access controls, encryption, compliance, and responsible data management.
- Ability to collaborate effectively with data scientists, data analysts, software engineers, BI teams, cloud engineers, DevOps teams, and business stakeholders.
- Strong technical documentation and communication skills.
- Familiarity with data observability and monitoring platforms is advantageous.
- Relevant certifications in cloud engineering, data engineering, database technologies, or analytics are beneficial.
- High level of technical curiosity, attention to detail, reliability, accountability, and continuous-learning mindset.
- Strong commitment to staying informed about cloud data platforms, distributed computing, AI infrastructure, automation, data architecture, and emerging data-engineering technologies.

-

📌 Data Enginer (Victoria)
🏢 Aku Lodge
📍 Victoria