Data Engineer (Information Management Systems)
Singapore, Singapore
Software Engineering – Data Engineering /
Full-time /
On-site
CSIT develops digital products to advance the national security interests of Singapore. We are looking for talented software engineers with a growth mindset and a passion for creating great products to improve the effectiveness and productivity of the organisation.
About this role
As a data engineer in the Information Management Systems team, you will be responsible for developing and managing data pipelines that deliver data to downstream business-critical systems. You will have opportunities to create/improve ETL (extract, transform, load) flows that process large volumes of data. You will also build/enhance APIs to serve data to downstream services and end-users.
You will have the autonomy to find the best way to solve problems, such as finding the most suitable technologies to use.
Responsibilities
· Write high-quality code with unit, functional and end-to-end tests
· Design, build, and deploy ETL solutions that are scalable and maintainable
· Triage system issues to ensure the smooth running of data pipelines and systems (detecting/debugging/tracking/resolving production incidents)
· Enable easy access to data for end-users (data scientists, analysts, and software engineers) via APIs with appropriate access control mechanisms
· Prepare data in ready state for AI applications
Requirements
Minimum
· Bachelor's degree in Computer Science, Computer Engineering, Information Systems or a related field
· At least 2 years of hands-on experience in Java programming
· (Fresh grads – project/internship experience will be considered)
· Experience in building ETL flows (e.g. NiFi, Airflow)
· Experience in back-end development using frameworks such as SpringBoot
· Familiarity with microservices architecture & RESTful API design
Preferred
· Software development experience with Agile SCRUM practises, testing, and production error/metrics monitoring
· Experience in building scalable and fault-tolerant data pipelines that are maintainable
· Familiarity with databases (e.g. MongoDB) and object stores (e.g. MinIO)
Experience in the following would be advantageous:
· Search engines (e.g. Elasticsearch, Solr)
· Messaging systems (e.g. RabbitMQ, Kafka)
· Metrics/logging systems (E.g. Elastic Stack, Prometheus/Grafana)
Considerations for senior positions
· Candidates with at least 3 years of experience in software development with proven track record of the above qualifications
· Experience in providing technical mentorship to junior software engineers
· Experience in collaborating across teams to develop and deliver complex systems/products (e.g. communicating and negotiating architectural designs and technical solutions)
As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.
#LI-DNI
