Sr. Software Engineer, Linkerd Proxy (Rust)
Hi, we're Buoyant.
We build Linkerd—the lightest, fastest, simplest service mesh for Kubernetes.
Linkerd is an open source project. It has a thriving community of adopters and it powers the production systems of companies around the world. Companies like Microsoft, Nordstrom, Timescale, Expedia, and many, many more use Linkerd to add critical security, reliability, and observability features to their Kubernetes applications.
We’re a small company with an incredibly outsized impact on the world. We’re remote-first and fully distributed with team members all over the world. Our competition is fierce: every day, we take on some of the largest companies in the world with essentially infinite dollars and infinite engineers to devote to their competing projects. Our secret weapon? Focus, and a deep sense of empathy for our users.
Working here is not for the faint of heart.
Imagine taking ultralight, ultrafast L7 "micro-proxies" written in Rust, sticking them next to other people's applications running on other people's clusters, and having them mediate all TCP (and in the future, UDP) communication to and from these apps—no matter what it is. Imagine these proxies upgrading connections from HTTP/1 to HTTP/2, initiating and terminating mutual TLS, retrying requests, emitting fine-grained metrics, issuing CSRs to the local CA every n hours, and so on. And imagine the end user blissfully operating 10,000 of these proxies, as a whole, with an easy-to-use CLI and API. That's a service mesh.
(The best way to learn more is to try it yourself. If you can run Kubernetes, you can get Linkerd up and running in 5 minutes.)
Under the hood Linkerd is extremely sophisticated and deeply technical, but the key to the entire project is simplicity. It's extremely difficult to make a simple service mesh, and very easy to make a complex one. (Heck, look at pretty much any of the other options.) And simplicity in this case means operational simplicity: it doesn't mean that you get a one-click install wizard, it means that whenever Linkerd is running, you can understand what it's doing and predict its behavior. It means no leaky abstractions, no complex tuning, no hidden gotchas. We spend a lot of time and energy in ensuring Linkerd is simple—it's our central promise to our users.
Linkerd is a graduated-tier project of the Cloud Native Computing Foundation, just like Kubernetes, Prometheus, and other defining projects of the cloud native space. It's written in Rust and Go; Go powers (most of) Linkerd's control plane. Rust powers Linkerd2-proxy, the source of Linkerd's true power. We're the only service mesh that uses something like Linkerd2-proxy, and it's a tremendous differentiator.
On the commercial side, we make Buoyant Cloud, a cloud-based management system for Linkerd. Buoyant Cloud makes Linkerd stress-free to operate. You connect your existing deployments, and Buoyant Cloud monitors their health and alerts you if anything looks weird. It takes on the toil of upgrades, data plane management, and more. In short: Buoyant Cloud turns Linkerd into a managed service, even though it's running on your own clusters.
What are you looking for?
In this role, we're looking for people who want to work on Linkerd2-proxy. Linkerd2-proxy is built in Rust on top of frameworks like Hyper, Tower, and Tokio. It's not trivial code: Linkerd2-proxy has to be extremely performant, extremely lightweight, and it has to gracefully handle any network traffic that the application sends to it, no matter what. It has to be capable of scaling up to huge levels of load and handling tens of thousands of concurrent connections, and scaling down to consume the smallest amount of resources in low-traffic situations. And it's not just throughput-sensitive but latency sensitive: every millisecond of additional latency in the proxy means can add 10s of milliseconds of latency to the user's application. And it has to handle all of that gracefully, without requiring complex tuning, without having unpredictable resource usage, and while running in a container on unpredictable hardware with an unpredictable network.
This is a Rust job, but we are not looking for Rust-only idealogues. We're a small company and everyone wears a lot of hats. You will need to write some Go. You will need to interact with actual users and treat them nicely even if they ask silly questions.
Linkerd is an open source project, and it feels like one: we have a lot of users, a continuous stream of GitHub issues and Slack messages asking for help or describing problems or asking when something is going to be ready. The cloud native space is rife with buzz, FOMO, and distraction. You need to be able to be productive in that environment.
We interact with our Linkerd users in the community Slack and we try to engage with them authentically while not being distracted. You don't have to deliver keynotes at Kubecon (unless you want to!), but participating in the community is part of your job.
We’re a team, not a family, but we have families and are the kind of place where work doesn’t get in the way of that. There will be crunch times, but work life balance is important and our p95 work times are pretty good for a startup.
This is an extremely technical project that pushes the boundaries of modern network software. If that sounds fun, this just might be the job for you.
This might be a good role for you if you have:
- Eight or more years of professional software engineering experience developing high-performance, low latency network services.
- Expert-level understanding of Linux operating system fundamentals and network programming.
- Expert-level Rust programming skills, ideally with the Tokio ecosystem.
- The ability to navigate complex software engineering tradeoffs between performance, maintainability, future-proofing, end-user experience, and feature completenessExcellent written communication skills.
- The ability to work with in a distributed team with members across with different timezones.
- Familiarity with Linkerd, Kubernetes, gRPC, and containers.
- A weird but adorable love of software infrastructure.
In this role, you will:
- Write mission-critical, high-performance open source code in Rust and Go. Yes, there will be some Go programming involved. No complaining allowed!
- Design and implement complex features at the intersection of network programming, operating systems, Kubernetes, and cloud native deployment models, in collaboration with other team members.
- Give and receive critical feedback on design and code while maintaining a supportive and friendly environment.
- Ensure performance and correctness of Linkerd across multiple dimensions, including traffic load, protocol choice, resource constraints, and product requirements.
- Develop high-scale reliable test tooling and environments, drawing from experience working on production systems.
- Solve problems (aka fight with) CI/CD tools, Linux systems, networking, and the rest of the complex ecology in which Linkerd lives.
You should be:
- Low-ego, collaborative, and results-oriented.
- Able to give and receive constructive feedback.
- Willing to learn from and mentor teammates.
- Passionate, empathetic, and kind.
Buoyant is a fully distributed, remote-first company. Applicants from anywhere in the world are encouraged to apply. However, working hours must overlap at least partially with US or EU timezones.