24 Aug
|
TalentReady
|
Sydney
24 Aug
TalentReady
Sydney
Sydney
Contract Engagement
We are seeking a skilled Data Engineer with strong Databricks experience to join our growing data team. In this role, you will design, build, and optimize large-scale data pipelines, ensure high data quality, and support analytics a initiatives.
Responsibilities
- Develop and maintain scalable ETL/ELT pipelines using Databricks (Spark, Delta Lake).
- Build and optimize data lakehouse architectures for batch and streaming workloads.
- Collaborate with data analysts, data scientists, and product teams to deliver reliable datasets for reporting and ML models.
- Implement best practices around data quality, observability, governance, and documentation.
- Manage and optimize Databricks clusters, workflows, and job performance.
- Integrate data from various sources (APIs, databases, cloud storage, SaaS platforms).
- Support infrastructure using Azure/AWS and CI/CD for data pipelines.
- Ensure compliance with data privacy and security standards.
Required Qualifications
- 4+ years of experience as a Data Engineer.
- Hands-on experience with Databricks, Apache Spark, and Delta Lake.
- Solid SQL and Python skills.
- Experience building ETL/ELT pipelines at scale.
- Familiarity with cloud platforms (Azure, AWS, or GCP).
- Experience with Azure Databricks or AWS Databricks.
- Understanding of data modeling, distributed computing, and data warehousing concepts.
- Experience with data orchestration tools (Airflow, dbt, ADF, Prefect, etc.).
- Experience with version control (Git) and CI/CD tools.
Additional information
- Contract Engagement
- Tech skills required Python | SQL | Databricks | Dbt
- Immediate Start
📌 Software Engineer - Python | SQL | Databricks | Dbt (Sydney)
🏢 TalentReady
📍 Sydney