San Mateo, California
We're looking for an engineer interested in developing, maintaining, and scaling data pipelines that empower our organization to make data-driven decisions in all areas of business.
You’ll work on huge datasets (more than 150 billion events a month), building systems that make messaging more powerful and relevant for more than 800,000 developers.
We have built a data lake that ingests terabyte-scale data and gets queried by a MPP SQL engine, which is used daily by our internal data users. We plan to extend this usage to our core product and you will take on a core role to scale and architect our data infrastructure.
In addition to the data projects, you will help our internal data users understand, visualize, and track our data in order to cultivate a data-driven culture across the organization.
What you'll do:
- Identify innovative applications of data to enable new features in our core product
- Work closely with the product and growth team to help understand data better
- Maintain and develop data pipelines
- Architect solutions to address our data pipeline scaling needs
- Open source contributions
Skills and experience:
- 3+ years of experience working with the Hadoop ecosystem. (HDFS, Hive)
- 2+ years of experience working with data pipelines (Airflow, Oozie)
- Experience working with MPP databases (Presto, Impala)
- Experience working with stream processing (Kafka)
- Experience writing with at least one statically typed language such as Rust, Java, C++, etc.
- Proficiency in written and oral communications
- Ability to collaborate well on a team
- Can deliver solutions independently
- Friendliness, empathy, modesty, and love of learning
Preferred skills and experience:
- Experience operating data infrastructure on Kubernetes
- Experience writing a custom Presto/Hive plugin
- Experience working with structured data serialization (ProtoBuf, Cap’n Proto)
- Strong understanding of the CQRS pattern/event sourcing
In keeping with our beliefs and goals, no employee or applicant will face discrimination/harassment based on: race, color, ancestry, national origin, religion, age, gender, marital domestic partner status, sexual orientation, gender identity, disability status, or veteran status. Above and beyond discrimination/harassment based on 'protected categories,' we also strive to prevent other, subtler forms of inappropriate behavior (e.g., stereotyping) from ever gaining a foothold in our office. Whether blatant or hidden, barriers to success have no place at OneSignal.