Software engineer
San Francisco
Engineering /
Full time /
On-site
While significantly more efficient than humans for many tasks, the cost and latency of LLMs is still prohibitive for most real world applications. At Ntropy, our mission is to make LLMs viable at scale. We are developing a new kind of domain-specific wrappers for language models, that leads to a reduction in the number of queries to the base model by 3-5 orders of magnitude and cost per datapoint by 2-3 orders of magnitude, without impacting accuracy.
The first domain we have been building for is finance. In early 2022, we released the first set of tools for extracting information from bank transactions. This challenge has traditionally demanded human expertise—accountants, underwriters, financial controllers, which restricted applications to only the most substantial transactions and left areas like small business lending and instant consumer credit scoring out of reach.
Using the Ntropy API, financial data can now be processed with super-human accuracy, scaling to cover over 4 billion daily electronic transactions globally. In the near future, we will take what we have built for financial data to other domains, enabling anyone, from single developers to Fortune 500 companies, to run large language models at maximum effectiveness to solve the hardest and most impactful problems in the world. We are just at the beginning of our journey and you will be one of the early members of the team to shape this future with us.
We
- come from various fields - engineering, mathematics, physics and arts.
- are allergic to over-engineering.
- are anarchists at heart and like to hack around the status quo.
- love playing board and video games.
- are radically honest and appreciate challenging one another, rather than giving out “pats on the back”. Yet, we can always rely on each other for support, feedback and results.
- are willing to learn and adapt quickly to new situations and requirements. Languages, frameworks, libraries, compilers, etc. are just tools for a job. A new problem might need a new tool. If it doesn’t exist yet, we will build it.
- have a sense of humor (well, we think we do).
As an early member of our backend engineering team, you will help
- push the limits of distributed computing and orchestrating large-scale machine learning pipelines.
- build the team and drive the direction of the company.
- shape our product and culture.
- experience the real-world impact of what you build.
The following are a big plus
- fluency in Python and Rust.
- experience with multi-GPU environments.
- recognized open-source contributions.
- participation in competitive programming events (CTFs, Putnam, IOI, IMO, Codeforces, etc.).
- experience with infrastructure-as-code, cloud deployments, Kubernetes, containers.
What do typical software engineering tasks at Ntropy look like?
- developing our core system and API in both Python and Rust using technologies such as PostgreSQL, ScyllaDB, Redis and ML using industry standard tools (PyTorch, Nvidia Triton).
- working with highly distributed systems that move and process large quantities of data. Reviewing and changing the architecture of these systems to improve metrics such as throughput and latency.
- developing, testing, and maintaining our large language model platform.
- scaling LLMs to 100M+ requests per day.
- understanding customer problems and designing vertical solutions on top of our core systems.
FAQ
Where is Ntropy located?
We are currently starting our SF office and are hiring there for in-person roles only. We also have hubs in London, UK and Lisbon, Portugal.
Do you consider part-time work?
Not at the moment. Full-time roles only.
How are you funded?
We are backed by some of the top funds in the world and have raised double-digit millions of dollars so far. We can share more details over the call.
Do you already have a product and customers?
Yes. We have been in production since 1st Jan, 2022 and have in the high double digits customers using our APIs in production.
How big is your team?
We are around 20 people at the moment. Mostly engineering and product.
What is the interview process like?
1. Send us an overview of problems you have encountered before and how you approached solving them. Please include as much detail as possible: code, algorithms, derivations, proofs, etc. We will then do a video call to kick things off and go through it (45 mins).
2. We will give you a take-home project related to whatever we are currently working on (3-4 hours). Alternatively, if you have a relevant project that you worked on previously that demonstrates your skills as an engineer, you are welcome to use that instead.
3. We will then do a deep-dive through the project over a call and discuss the implementation, improvements and bottlenecks.
Above all, we respect your time and commitment and will keep you up to speed on where we are at during the whole process.
What are your hiring plans?
We expect to be 40-50 people by the end of 2024.
What is your current stack?
backend - Python, Rust
compute - AWS, GCP
ML - PyTorch, ONNX, Triton, LLMs
Work / life balance
We are a startup which requires you to put in a lot more work and soul than a regular job. We believe, however, that nothing easy is worth doing. We will expect a lot from you, and you should expect a lot from us.
What is the compensation?
approx. $130-170k • 0.1-0.3%
