Software Engineer (Ecosystem)
San Francisco, CA /
At Anyscale, we’re simplifying and democratizing distributed computing. As a software engineer focused on integrations at Anyscale, you will play a pivotal role in shaping the machine learning ecosystem.
One of the things that makes Python great is the fact that it has great libraries. Developers can import numpy and pandas and start building powerful applications. But in the distributed setting today, we don’t have libraries. Instead, we have distributed systems like Spark or TensorFlow Serving. These systems cannot easily be composed together and used as elements of a larger application.
Imagine if you could import powerful distributed libraries from a rich ecosystem and easily compose them together to build cutting edge programs. We’re looking for someone to help us build that ecosystem.
About the role:
This position requires hands-on expertise in building machine learning pipelines along with a breadth of knowledge of the big data and machine learning ecosystem. On a weekly basis, you will be working with different open source communities to expand Ray (and its native libraries) usage within different libraries in the machine learning ecosystem. You will also be facilitating adoption by working with different downstream users in different communities. You may excel at this role if you are self-motivated and excel in cross-functional collaboration. Part of this work will be open source as part of Ray.
We are looking for senior hires as well as less experienced but motivated individuals.
Anyscale provides an application development platform for developers to build distributed applications. We’re commercializing a popular open source project called Ray, which is a framework for distributed computing as well as an ecosystem of libraries for scalable machine learning. Our goal is to build a standardized platform for distributed computing. Ray was developed at UC Berkeley by Robert Nishihara and Philipp Moritz, under the guidance of Ion Stoica and Michael Jordan, and the four of them have co-founded Anyscale. The company raised a $20.6M Series A funding led by Andreessen Horowitz (a16z) with participation from NEA, Intel Capital, Ant Financial, Amplify Partners, 11.2 Capital, and The House Fund.
With Ray, we're making it easy to program at any scale (from your laptop to the datacenter) by providing easy-to-use, general-purpose, and high-performance tools. In addition, we are building a rich ecosystem of libraries (for reinforcement learning, hyperparameter search, experiment management, machine learning training, prediction serving, and more) on top of the core distributed system so that users can rapidly build sophisticated applications. Help us build the future of software development.
We are looking for passionate, motivated people who are excited to build tools to power the next generation of cloud applications.
As part of your role you will:
- Identify and drive new initiatives that will both drive Ray adoption and provide large benefit to the ecosystem at large
- Work with a team of leading distributed systems and machine learning experts
- Collaborate with external open source teams to share your work to a broader audience through talks, tutorials, and blog posts
- Help us to build and shape a world class company
- At least 2 years of relevant work experience
- Expertise programming in Python
- Experience with cloud providers such as AWS, Azure or GCP
- Design and implementation experience with modern machine learning stack (tensorflow, Spark, xgboost, etc)
- (Nice to have) Experience working on open source
Must be willing to work onsite in our office.
We are excited to build a diverse team and encourage all to apply!