Senior Data Engineer
Old Street, London
Tech and Product – Technology /
Full time /
Hybrid
Senior Data Engineer
We are looking for a technical expert with deep data engineering experience, who thrives on solving complex problems, driving business impact, and coaching other engineers.
Salary: £70k-£90k
Reporting to: François-Xavier Decroix (Engineering Manager)
Location: primarily based in our London Office, with flexible and hybrid working (ideally 2 days per week in the office).
About the role
As a Senior Data Engineer, you will take responsibility for continuously improving critical systems which transport hundreds of gigabytes of data per day and trillions of messages per year. You will work closely with the broader team to bring innovative new datasets and insights to customers in a reliable, scalable, and cost-effective way. Your work will be crucial in our mission to empower safe and sustainable transport around the world through data insights.
You will be the go-to person for our cloud data pipeline & underlying infrastructure, especially Apache Kafka and our PostgreSQL database. You will also work with tech like Kubernetes, Vault, go microservices and other cloud services, as well as on-premise infrastructure. Your technical expertise will be instrumental in developing and operating these systems for our existing and new products and features; you will be crucial in helping us maintain our market-leading position in the UK, and in scaling internationally.
In this role, you will work within the AI and cloud team, establish technical standards, lead architectural decision-making within your area of expertise, and proactively share knowledge with colleagues. This role has the option to grow towards either a principal engineering role (a technical specialist leader) or an engineering manager (a team leader with line management) position at Vivacity, through structured career progression pathways.
About us
At VivaCity, we make cities smarter, safer, and more sustainable. We have over 5,000 AI sensors around the world, gathering real-time anonymous data on transport modes, traffic flow, and travel patterns. This is used to gather actionable insights to support strategic decisions to improve the global transport network.
All our solutions are community-centric, using 'privacy by design' principles. Our ultimate goal is to make the European vision of a Smart City - one which makes the city work effectively, for the community.
We pride ourselves on a collaborative, open culture that fosters innovation, learning and encourages everyone to do their best work, whilst building a sense of community and collaboration. Read more about what it's like to work at VivaCity.
About you
You are a technical expert with a proven ability to creatively solve complex problems across a broad remit, manage production systems reliably, and navigate ambiguity to deliver impactful solutions. You have a good understanding of distributed systems, data pipelines, and cloud infrastructure, and you thrive in an environment that values collaboration, continuous improvement, and technical excellence.
You proactively and effectively communicate with both technical and non-technical colleagues, earning trust and influence through your knowledge and proactive, considerate approach. You are passionate about simplifying complex systems and seek opportunities to balance immediate delivery with long-term improvements. You continue working on problems, even when you are not familiar with the problem space, and you know when and how to ask for help.
Requirements for the role:
- Expertise in Apache Kafka (ideally Kafka Strimzi), including broker management, scaling, upgrades, and integration with high-throughput data pipelines.
- Experience managing PostgreSQL databases, including load analysis, query optimisation, and role management.
- Experience with modern cloud infrastructure as GCP/AWS, Kubernetes, and Docker
- Proficiency in Unix systems, ideally Linux (we use Ubuntu).
- Strong communication skills, with experience mentoring engineers and collaborating with stakeholders.
- Proven ability to resolve technical incidents in unfamiliar production systems.
- Technical and process documentation champion.
- Experience of operationally managing production software components, including observability, logging, metrics, error reporting, debugging, and live incident management.
- 60% - Proactive technical work (e.g. migrating DB hosting provider, new message bus system proof of concept, deploying new algorithms to kubernetes)
- 20% - Reactive technical work (e.g. incident response, unblocking engineers, code review)
- 10% - Technical leadership initiatives (e.g. establishing technical standards, architectural principles, & staff training)
- 10% - Planning and administration
Your time will be spent roughly as follows:
Any of the following would further strengthen an application
- Proficiency with Hashicorp Vault including secret and role management.
- Proficiency in one or more programming languages (we primarily use Go and TypeScript).
- Comfortable working with Infrastructure as Code, ideally Terraform.
- Familiarity with common DevOps tools and CI/CD processes.
- Proficiency with infrastructure management tools (e.g. Ansible, Chef, Puppet)
- Knowledge of information security principles, including identity and access management and authentication/authorisation processes.
- Experience in a start-up or scale-up environment.
- Passion for sustainability, transport or smart cities.
You don't need to have done all of these things before, but to excel in this role, you will need to be keen to learn and comfortable working in a dynamic, fast-paced environment. If you're close to what we're looking for, please consider applying. Experience comes in many forms, skills are transferable, and passion goes a long way.
Hiring process
- 30 minute screening interview.
- 1.5 hour system design interview where you work together with a VivaCity engineer
- 2.5 hours final round interview, split into a 1 hour technical experience interview, 1 hour soft skills interview, and 30 minutes with Shaun, our Head of Engineering
We know that diverse ideas and perspectives drive innovation and make us better. We are creating an environment where everyone, from any background, can do their best work. We're an equal opportunities employer and all applications will receive consideration for employment without regard to ethnicity, religion, gender, gender identity or expression, sexual orientation, nationality, disability, age, or social background. If you need any reasonable accommodations to help you perform at your best during the application process, please let us know.
