Data Engineer, Noida
Noida
Lab49 – Software Engineering /
Full-time /
Lab49 has an opportunity available for a Data Engineer to lead work on complex and challenging projects to drive transformative change for our top tier Financial Services clients. Operating in an Agile environment, the Data Engineer will work on building the next generation financial systems.
Required Skills and Experience
- 5+ years experience in any programming language (e.g. Scala, Java, or Python)
- Data warehousing using Redshift with emphasis on Information management domain.
- Data modeling
- AWS - any or all of EC2, S3, EBS, ELB, RDS, Dynamo DB, EMR
- SQL, authoring complex queries and DDL
- Big data querying tools (e.g. Hive, Spark, or Presto)
- Messaging system, such as Kafka
- Monitoring performance and ensuring infrastructure is appropriately matched to support workload and SLAs
- Experience in Linux/Unix
- Proven ability to troubleshoot and resolve production issues under a DevOps model
- Track record of continuously identify opportunities to improve the performance and quality of your ecosystem
- Working knowledge of Data Pipelines tools (e.g. Apache Oozie, LUIGI, or Airflow)
- Broad knowledge of technical solutions, design patterns, and code for medium/complex applications deployed in clustered computing environment
- Experience working in an Agile environment
Desired Skills and Experience
- Experience in Flink, Beam
- Experience in Data streaming
- Experience in Docker, Kubernetes, Openshift
- Real-time integration (using Kafka, Spark streaming or other technologies)
- Critical understanding of Lambda Architecture
- Working knowledge of API-based integration for modern systems
- Exposure to security concepts and best practices
- Knowledge of machine learning (e.g. Spark ML or R)
