Data Engineer, DaaS

PayPay's growth is driving a rapid expansion of PayPay product teams and the need for a robust Data Engineering Platform to support our growing business needs is more critical than ever. We are looking for a Data Engineer for the DaaS team. The DaaS team’s responsibility is to design, implement, and operate this platform by using cutting edge technologies such as Spark, Hudi, Delta Lake, Scala, and AWS suite of data tools. We are looking for talented Data Engineers to join our team and help us scale our platform across the organizations.

Main Responsibilities

Design, develop, and maintain scalable data ingestion pipelines using Databricks, Airflow, Lambda, and Terraform

Optimize and manage large scale data pipelines to ensure high performance, reliability, and efficiency

Implement data processing workflows using Spark, Delta Lake, PySpark, and Scala

Maintain and enhance Data Lakehouse with Unity Catalog for effective data management and discovery

Build automation, framework, and improved tooling to support end to end data engineering workflows

Collaborate with cross-functional teams to ensure seamless data flow and integration across the organization

Implement best practices for observability, data governance, security, and compliance