02 Nov
|
Tata Consultancy Services
|
Sydney
02 Nov
Tata Consultancy Services
Sydney
Join Tata Consultancy Services, Asia Pacific and be part of an organization committed to sustainable development for our future. TCS follows the Tata group philosophy of building sustainable businesses that are rooted in the community and demonstrate care for the environment. Our unique values position us to combine a purpose-driven worldview with digital innovation, collaborating with customers, communities and governments to lead and shape innovative solutions for a sustainable future. TCS has been carbon neutral in its operations across 11 countries, 12 delivery centres and 18 offices in Asia Pacific since 2022. This is only the initial stage in TCS’ journey as we strive to achieve long-term net zero emissions by 2030.
Corporate sustainability is embedded in our triple-bottom-line, focusing on people, the planet, and our purpose. Our offices are designed with eco-friendly features that significantly reduce our carbon footprint and enhance energy efficiency. We actively champion green initiatives, such as promoting paperless operations, implementing energy-productive practices, and fostering employee engagement in sustainability efforts. When you become part of the TCS family, you will play an essential role dedicated to innovation, excellence, and crafting a brighter, greener future together. Join us and be a part of our mission to drive sustainability through technology and talent at Tata Consultancy Services, APAC today.
Experience: 8+ years
Location: Sydney
Job Type: Full Time (Permanent)
Required Skills:
- Strong hands-on experience designing, developing and supporting scalable enterprise data pipelines.
- Strong Spark-etl experience, including the ability to analyze existing Omnia pipelines,
understand their processing logic and derive new transformation and business rules.
- Strong working knowledge of AWS services, including S3, Glue, EMR, Lambda, Athena, Step Functions, SQS, IAM, Secrets Manager and CloudWatch, combined with hands‑on experience in orchestrating data pipelines using Apache Airflow.
- Practical experience with Snowflake, covering data ingestion, data modelling, transformation and performance optimization.
- Experience using dbt to develop data models, incremental processing, reusable macros, automated tests, documentation and dependency management.
- Excellent SQL skills, with experience in complex data transformations, query optimization, reconciliation and understanding existing business logic.
- Familiarity with Git, automated testing, CI/CD, Jenkins and DevSecOps practices within a data engineering environment.
- Good Python programming skills for data processing, automation, and reusable framework components.
Key Responsibilities :
- Conduct detailed analysis of existing end-to-end data pipelines across multiple platforms, with a particular focus on Omnia - PySpark-based processing.
- Review existing source code, SQL, transformation rules, mappings and data flows to understand the current business and technical logic.
- Derive, document,
and implement new transformation logic based on the analysis of existing pipelines and business requirements.
- Design, develop and maintain reliable, scalable, and reusable data pipelines using AWS, Snowflake and dbt.
- Develop batch and, where required, event-driven data processing solutions using cloud-native and distributed data technologies.
- Translate existing Omnia data processing logic into strategic cloud-based data engineering solutions.
- Build ingestion, transformation, enrichment and data integration pipelines across data lakes, data warehouses and cloud lakehouse platforms.
- Develop complex and performance-optimized SQL for data transformation, reconciliation, validation and analytical processing.
- Analyze end-to-end data flows from source capture through transformation, enrichment and downstream consumption.
- Work with data designers, product owners, and subject matter experts to translate business requirements into technical data rules and durable engineering solutions.
- Implement data quality, reconciliation and balancing controls across data pipelines.
- Troubleshoot data pipeline failures, transformation issues, data discrepancies and performance bottlenecks across multiple environments.
Top Reasons to Join Our Team at TCS Australia
Attractive compensation & rewards
We offer a highly market-competitive salary package aligned with the Australian market, complemented by performance-based incentives, recognition programs, and comprehensive employee benefits. Your achievements are celebrated, and your contributions truly matter.
Global exposure with local impact
At TCS, you’ll work with leading Australian enterprises across diverse industries while collaborating with our gl
📌 Data Engineer (Sydney)
🏢 Tata Consultancy Services
📍 Sydney