Job Title: Full-Stack Data Engineer (Databricks, FastAPI & Next.js)

Location:
REMOTE or Hybrid from Brisbane QLD 4000 (for Local Candidates) Duration: 3+ Months Initial (Extendable up to 6+ months) Must Have: Unrestricted working rights in Australia. Job Summary We are seeking a highly motivated Full Stack Data Engineer to build modern, scalable, cloud-native applications that leverage Databricks for data processing, Python for backend services and APIs, and Next.js for responsive web applications. The ideal candidate will have experience developing enterprise-grade applications that integrate with analytics platforms, data pipelines, AI/ML services, and cloud-native technologies. The role requires solid backend engineering skills, modern frontend development capabilities, and experience working with Databricks as the compute platform.

Key Responsibilities
Databricks & Compute
• Develop and maintain data processing workloads using Databricks.
• Build PySpark notebooks and workflows for data transformation and analytics.
• Integrate applications with Databricks Jobs, Workflows, SQL Warehouses, and REST APIs.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Work with Delta Lake, Unity Catalog, and Medallion Architecture.
• Develop data APIs and services that consume Databricks datasets. Backend Development (Python)
• Design and develop scalable backend services using Python (FastAPI, Flask, or Django).
• Build RESTful APIs and microservices.
• Develop authentication and authorization mechanisms (OAuth2, JWT, SSO).
• Integrate backend services with Databricks, Azure services, and third-party APIs.
• Implement asynchronous processing and background jobs.
• Write reusable, well-tested, and maintainable code.
• Build logging, monitoring, and error-handling capabilities.



Frontend Development (Next.js)
• Develop responsive web applications using Next.js and React.
• Create reusable UI components and contemporary dashboards.
• Integrate frontend applications with backend APIs.
• Implement authentication and role-based access control.
• Optimize application performance and user experience.
• Ensure responsive design across desktop and mobile devices. Integration & Cloud
• Integrate applications with Azure cloud services.
• Consume APIs from enterprise applications and external systems.
• Implement secure API communication and data exchange.
• Participate in cloud deployment and infrastructure discussions. DevOps & CI/CD
• Use Git for source control and collaborative development.
• Build and maintain CI/CD pipelines using Azure DevOps or GitHub Actions.
• Support automated testing and deployment processes.
• Participate in code reviews and enforce coding standards. Quality & Security
• Develop unit and integration tests.
• Ensure applications meet enterprise security and compliance requirements.
• Implement secure coding practices and vulnerability remediation.
• Troubleshoot production issues and support release activities. Must Have: Data Engineering core concepts
• Databricks
• ETL Pipelines, Medallion Architecture, PySpark
• Databricks Automation Bundles
• Python Backend
• FastAPI (Must)
• REST API Development
• AuthN Basics (JWT, OAuth2)
• Databricks App builder exp including deployment
• General Engineering
• Agile
• Unit Testing
• CI/CD
• Azure DevOps Good to Have:
• Front end design/development
• Next.js or similar
• Html/css framework
• AI Engineering
• ML Engineering
• MLOps ADDITIONAL REQUIREMENTS
• Unrestricted working rights in Australia.

📌 Full-Stack Data Engineer (Australia)
🏢 Celebal Technologies
📍 Australia