Join Tata Consultancy Services, Asia Pacific and be part of an organization committed to sustainable development for our future. TCS follows the Tata group philosophy of building sustainable businesses that are rooted in the community and demonstrate care for the environment. Our unique values position us to combine a purpose-driven worldview with digital innovation, collaborating with customers, communities and governments to lead and shape innovative solutions for a sustainable future. TCS has been carbon neutral in its operations across 11 countries, 12 delivery centres and 18 offices in Asia Pacific since 2022. This is only the initial stage in TCS' journey as we strive to achieve long-term net zero emissions by 2030.

Corporate sustainability is embedded in our triple-bottom-line, focusing on people, the planet, and our purpose. Our offices are designed with eco-friendly features that significantly reduce our carbon footprint and enhance energy efficiency. We actively champion green initiatives, such as promoting paperless operations, implementing energy-efficient practices, and fostering employee engagement in sustainability efforts. When you become part of the TCS family, you will play an essential role dedicated to innovation, excellence, and crafting a brighter, greener future together. Join us and be a part of our mission to drive sustainability through technology and talent at Tata Consultancy Services, APAC today.

Location : Sydney

Years of Experience : 8+Yrs

Job Description :

Must have :

- AWS
- Python
- Airflow
- SQL
- Pyspark
- Proficiency with AWS SageMaker Unified Studio, including its various components Discover, Build and Govern.




- Experience with development in Sagemaker IDE and Application (Jupyterlab, Spaces and Partner AI Apps).
- Data Analysis and Integrations ( Query Editor, Visual ETL Jobs and Data Processing jobs)
- Orchestration of workflows and ML Pipelines
- Understanding and expertise to work with ML and Gen AI tools available in unified studio are ad-on values.
- Setting up projects and data governance in SageMaker Unified studio
- Advanced Data Ingestion & Processing (Real-time & Batch):
- Framework Development: Proven ability to design, develop, and implement highly reusable and adaptable data ingestion frameworks capable of handling diverse source types (e.g., databases, APIs, message queues, file systems).
- Low-Latency Real-time: Deep expertise in building real-time data pipelines with stringent sub-second latency requirements, utilising services like AWS Kinesis (Data Streams/Firehose), Apache Kafka, or similar streaming technologies.
- Batch Processing: Experience with robust batch ingestion patterns and tools (e.g., AWS Glue, Apache Spark) for efficient processing of larger datasets.
- Data Transformation: Strong skills in designing and implementing efficient data transformation logic for both streaming and batch data.
- Programming and Scripting:
- Advanced proficiency in Python, particularly for developing scalable data ingestion and export frameworks, API integration,



and extensive use of the AWS SDK (Boto3).
- Experience with performance optimisation techniques for Python applications in data-intensive environments.
- Familiarity with other relevant languages (e.g., Scala, Spark) for high-performance streaming applications is beneficial.
- AWS Services for Data and Infrastructure:
- In-depth knowledge of core AWS services: Lambda, S3, DynamoDB, CloudWatch, SQS, SNS, and API Gateway.
- Solid understanding of AWS networking (VPC, security groups, private endpoints) and IAM for secure, fine-grained access control.
- Databases (Relational and No-SQL):
- Expertise with Amazon RDS (Relational Database Service) for both real-time data ingestion and efficient batch export. This includes optimising database performance, connection pooling, and transaction management for high-throughput, low-latency operations.
- Proficiency with Amazon Redshift for large-scale data warehousing, including data loading strategies (e.g., COPY command), query optimisation, and managing Redshift clusters for analytical workloads and batch export.
- Experience with No-SQL databases such as Amazon DynamoDB, MongoDB
- for high-performance, low-latency data storage and retrieval, particularly for real-time applications and feature serving.
- Orchestration and Workflow Management:
- Experience with AWS Managed Apache Airflow (MWAA) for orchestrating complex data pipelines, scheduling batch jobs, and managing dependencies between ingestion, processing, and export tasks.
- Ability to write, deploy, and manage Airflow DAGs (Directed Acyclic Graphs) for robust workflow automation.
- Monitoring and Observ

📌 Senior Data Engineer (Sydney)
🏢 Tata Consultancy Services
📍 Sydney