Job DescriptionAbout UsHalo Labs is a future focused, end-to-end data solutions firm – transforming tomorrow, today.
Our intelligent and secure technology systems and data driven solutions drive meaningful outcomes and unlock lasting value.Why work at Halo Labs?
Leading Innovation: We don't just solve problems; we illuminate a never-ending stream of innovation.Exceptional Perks: Enjoy outstanding perks like a dedicated learning budget, performance bonuses, and comprehensive wellbeing support.Remote-First Organisation: Experience the perks of remote work while having the flexibility to travel to client locations throughout Australia.Inclusive and Engaging: Celebrate diversity in a welcoming space that thrives on new ideas and open conversations, all in a respectful environment.Inspiring Origins: With a compelling founder story, we are a customer-focused, culture-first organisation.About the roleDesign, develop, and maintain reusable in-house PySpark frameworks to enforce standardised data engineering patterns across the SaaS platformArchitect and implement scalable, production-grade ETL/ELT pipelines across Azure and AWS environmentsBuild distributed data processing solutions using Python and PySpark on DatabricksDevelop batch and near real-time ingestion pipelines integrating third-party clinical systems, healthcare APIs, and external enterprise platformsDesign secure data integration patterns (REST APIs, SFTP, event-driven ingestion, webhooks) ensuring compliance and data integrityWork closely with Software Engineers to embed data services directly into the SaaS product architectureContribute to backend system design discussions to ensure data layer scalability, observability, and performanceImplement CI/CD pipelines using Azure DevOps, Git,



and Azure Pipelines for automated deployment and testing of data workloadsApply infrastructure-as-code and environment management best practices across Azure and AWSOptimise Spark jobs, cluster configurations, and storage strategies for performance and cost efficiencyDesign and maintain robust data models, including dimensional models and SaaS-oriented data schemasImplement data validation, monitoring, and alerting to ensure pipeline reliability and production stabilityProvide technical mentorship and enforce engineering standards across the analytics and data engineering teamAbout youStrong hands-on experience with Azure services including Databricks, Azure Data Factory, Azure SQL, Azure Storage, and Azure DevOpsPractical experience with AWS services relevant to contemporary data platforms (S3, Lambda, RDS, Glue, IAM, etc.)Advanced proficiency in Python, SQL, and PySpark for large-scale distributed data processingDeep experience configuring and managing Databricks clusters for scalable big data workloadsExperience building production-ready data pipelines in a SaaS or product-led engineering environmentStrong understanding of cloud-native data architecture, including data lakes, lakehouse architecture, and modular pipeline designExperience integrating with third-party systems via APIs and secure data exchange mechanismsExposure to healthcare or regulated data environments, including handling sensitive data securelyStrong knowledge of data modelling, metadata management, and data governance principlesExperience implementing automated testing frameworks for data pipelinesSolid understanding of DevOps practices including Git workflows, branching strategies, and CI/CD automationDegree in Computer Science, Engineering, Data Science, or related technical field
#J-*****-Ljbffr

📌 Freelance Data Scraping Engineer (Python) (Adelaide)
🏢 Mindrift
📍 Adelaide