Founding Engineer (Infra)
San Francisco, CA /
Streaming technologies are changing the data landscape and every application that produces and consumes data. Yet, most machine learning models, whose performance is tightly coupled with data quality and freshness, are still in the batch paradigm.
Claypot unifies streaming and batch systems to make it easier and cheaper for companies to leverage fresh data for online prediction and continuous evaluation. Our solution can be especially helpful for problems in fast-changing environments such as fraud prevention, recommender systems, e-commerce, and logistics.
Claypot AI was founded by Zhenzhong Xu and Chip Huyen. We're well-funded and working with cool companies!
For more discussion on the problem we're tackling, see Machine learning is going real-time and The Four Innovation Phases of Netflix’s Trillions Scale Real-time Data Infrastructure.
We're looking for great ML/Data infrastructure engineers to be the foundation of our engineering team.
What you'll do:
- Implement and optimize our computation workflows, both for latency and cost.
- Design and build the core infrastructure powering our services, which includes container infrastructure, storage, processing, control plane orchestration, and other core data platform capabilities.
- Evaluate tools and services to integrate our backend with.
- Ensure that our platform is scalable, available, and resilient.
- Grow a great engineering team.
We're building a cloud-native, distributed platform at internet scale. We're looking for engineers who are experienced with:
- You are familiar with modern data processing, storage technologies, and database internals such as relational algebra and optimization techniques
- You have built and operated large-scale distributed systems and can reasons about tradeoffs in distributed systems
- You have worked with cloud-native architectures
You'll stand out if you're:
- Familiar with streaming technologies such as Kafka, Flink, Iceberg, DuckDB, Arrow, or alternatives
- Comfortable with both Python and a more infra-oriented language such as Java
- Familiar with the challenges around online, offline, and nearline storage/processing technologies
- Familiar with ML/data engineering workflows
- Product-oriented. Our platform focuses on providing the best experience, so empathy with customers' pain points is a huge plus!
What will you get?
- Work with cutting-edge technologies
- Competitive compensation package
- Flexible remote-friendly culture with options for in-person collaboration
- A trustworthy and high-performing team to grow together with
- Learn how to build a startup from the ground up
- Public speaking opportunities
- An environment for you to grow into the career you want
What makes Claypot AI special?
- A culture of transparency, collaboration, and ownership
- A very high bar for engineering craftsmanship
- Expertise in both distributed systems and machine learning
- A strong community
- An opportunity to win over a large, growing, yet untapped market for a streaming-first computation platform for ML use cases
You should join us if you're:
- A learner (we're in a new space so there is a lot for us to learn!)
- A problem solver who is excited about machine learning and streaming technologies
- Open-minded to different ideas, cultures, and backgrounds
- Ready to take ownership, make decisions, and iterate
The job descriptions below are to give a sense of the challenges we're working on. As the company grows, you can define the role that you want with us. We believe in creating an environment for people to grow to their full potential and create the most impact for the team, not squeezing people to fit into job descriptions.