Overview

Opportunity to work with one of the largest and most advanced Data Engineering teams in the country. As a Senior Data Engineer with expertise in software development / programming and a passion for building data-driven solutions, you're ahead of trends and work at the forefront of Big Data and Data Warehouse technologies.

Responsibilities

- Passionate about building next generation data platforms and data pipeline solutions across the bank.
- Execute state-of-the-art coding practices, driving high quality outcomes to solve core business objectives and minimise risks.
- Create both technology blueprints and engineering roadmaps for a multi-year data transformational journey.
- Provide data driven solutions that source data from various enterprise data platforms into a Cloudera Hadoop Big Data setting, using technologies like Spark, MapReduce, Hive, Sqoop, Kafka; transform and process the source data to produce data assets; and transform and egression to other data platforms like Teradata or RDBMS systems.
- Build effective and efficient Big Data and Data Warehouse frameworks, capabilities, and features, using languages such as Scala, Java, or Python, with data quality assurance and security controls.
- Design, build, and deliver optimized enterprise-wide data ingestion, data integration and data pipeline solutions for Big Data and Data Warehouse platforms.
- Lead and mentor other data engineers in a project or initiative.




- Have experience and responsibility for data security and data management.

Technical skills

- Experience designing, building, and delivering enterprise-wide data ingestion, data integration and data pipeline solutions using Scala, Java, or Python in Big Data and Data Warehouse platforms; preferably 5+ years in a Data Engineering role.
- Experience building data solutions on Hadoop with Spark, MapReduce, Sqoop, Kafka and various ETL frameworks for distributed data storage and processing; preferably 5+ years.
- Strong Unix/Linux shell scripting and programming skills in Scala, Java, or Python.
- Proficient in SQL scripting, writing complex SQLs for building data pipelines.
- Experience in leading and mentoring data engineers, including ownership of internal business stakeholder relationships and working with consultants.
- Experience in working in Agile teams, including collaboration with internal business stakeholders.
- Familiarity with data warehousing and/or data mart build experience in Teradata, Oracle or RDBMS systems is a plus.
- Certification on Cloudera CDP, Hadoop, Spark, Teradata, AWS, Ab Initio is a plus.
- Experience with Ab Initio software products (GDE, CoOperating System, ExpressIt, etc.) is a plus.
- Experience with AWS technologies (EMR, Redshift, DocumentDB, S3, etc.) is a plus.

#J-18808-Ljbffr

📌 Senior Data Engineer (City of Sydney)
🏢 PayTech Nexus
📍 City of Sydney