Database Developer
Chennai
Engineering – Product Development /
On-site
For nearly 40 years, PDI has helped convenience retailers and petroleum wholesalers adapt to changes in the industry by leveraging the latest technologies. Simplifying the complexity in your world is our main focus. That's why we're delivering an integrated portfolio of global, cloud based solutions and services to meet our customers' needs today and well into the future.
From the back office to fuel logistics and digital commerce, PDI solutions deliver measurable value across the supply chain. We are proud to support over 1,500 customers in 50+ countries, powering 200,000+ sites worldwide.
The Opportunity: We’re looking for a seasoned Data Engineer III who is passionate about building scalable and resilient cloud-native data infrastructure — with a focus on governance, CI/CD, automation, and platform maturity. You will be a key contributor in evolving our modern data stack, ensuring operational excellence and code quality across ETL pipelines, metadata frameworks, and real-time/batch data services.
You’ll work at the intersection of data engineering, DevOps, and governance, setting standards across code repositories, orchestrators (Airflow), compute layers (Glue/EMR), and ingestion tools (DMS, Kafka, etc.)
Key Responsibilities:
- Maintain and evolve OLTP (Postgres) and OLAP (Redshift) data models /data lakes by evaluating new feature requirements, ensuring alignment with dimensional modeling best practices, and executing schema changes via Liquibase pipelines.
- Develop and maintain metadata-driven data pipeline frameworks that support validation, logging, auditing, and job orchestration.
- Standardize and govern Bitbucket/Git repositories, manage branching strategies, enforce code review and CI pipelines for ETL/data jobs.
- Design and implement CI/CD workflows for data services using tools like Jenkins, Liquibase, and Shell/Python scripting.
- Support automated deployment of ETL, Airflow DAGs, Glue jobs, and DB schema changes across environments (QA, Stage, Prod).
- Collaborate with DataOps and DevOps teams to maintain infrastructure as code (IaC) standards and shared configuration patterns.
- Build and scale data quality frameworks, including pre/post validations, job restorability, and alerting (CloudWatch, SNS).
- Implement data masking and access control standards (RBAC, column-level masking, role-based access) across Redshift and Iceberg.
- Optimize DMS/Kafka-based CDC pipelines and help reduce dependency through automation or zero-ETL patterns.
- Define standards for data retention, archival, and operational efficiency across OLTP/OLAP environments.
- Partner with data engineers and analysts to align platform standards with business needs and analytical readiness
Qualifications:
- 8+ years of experience in data engineering or platform engineering with exposure to production-grade data pipelines and systems.
- Deep expertise in Python and SQL, with strong understanding of pipeline design patterns and modular codebases.
- 3+ years of experience with CI/CD tooling (e.g., Jenkins, Liquibase, Bitbucket Pipelines) and managing deployment pipelines for data workloads.
- Solid understanding of AWS cloud services: S3, Glue, Redshift, DMS, Lambda, EMR, IAM, CloudWatch.
- Experience with workflow orchestration tools like Airflow (DAG scheduling, dependency mapping, alerts).
- Hands-on experience maintaining data lakehouse platforms (e.g., Apache Iceberg, Delta Lake) and managing batch vs. streaming ingestion.
- Experience managing schema changes, migrations, and rollback strategies across databases (Postgres, Redshift).
- Strong understanding of data security practices, including PII masking, row/column-level controls, and audit logging.
- Familiarity with dimensional modeling and differences between OLTP vs. OLAP patterns.
- Strong documentation and process-driven mindset to define standards and maintain operational transparency
Behavioral Competencies:
- Ensures Accountability
- Manages Complexity
- Communicates Effectively
- Balances Stakeholders
- Collaborates Effectively
PDI is committed to offering a well-rounded benefits program, designed to support and care for you, and your family throughout your life and career. This includes a competitive salary, market-competitive benefits, and a quarterly perks program. We encourage a good work-life balance with ample time off [time away] and, where appropriate, hybrid working arrangements. Employees have access to continuous learning, professional certifications, and leadership development opportunities. Our global culture fosters diversity, inclusion, and values authenticity, trust, curiosity, and diversity of thought, ensuring a supportive environment for all.
