Data Engineer

San Mateo, California /
Engineering /
We're looking for an engineer interested in developing, maintaining, and scaling data pipelines that empower our organization to make data-driven decisions in all areas of business. 

You’ll work on huge datasets (more than 150 billion events a month), building systems that make messaging more powerful and relevant for more than 1 million developers.

We have built a data lake that ingests terabyte-scale data and gets queried by a MPP SQL engine, which is used daily by our internal data users. We plan to extend this usage to our core product and you will take on a core role to scale and architect our data infrastructure. You will also be laying the foundation for customer-facing data features that will help millions of websites and apps to reach their goals.

What you'll do:

    • Identify innovative applications of data to enable new features in our core product
    • Work closely with the product and BI teams to help understand data better
    • Maintain and develop data pipelines
    • Architect solutions to address our data pipeline scaling needs
    • Open source contributions

Skills and experience:

    • 3+ years of experience working with the Hadoop ecosystem. (HDFS, Hive)
    • 2+ years of experience working with data pipelines (Airflow, Oozie)
    • Experience working with MPP databases (Presto, Impala)
    • Experience writing with at least one statically typed language such as Rust, Java, C++, etc.
    • Experience with scripting languages, e.g. Bash, Python,
    • Good to have experience in R, Matlab, VBA, C#, stream processing (Kafka) 
    • Proficiency in written and oral communications
    • Ability to collaborate well on a team
    • Can deliver solutions independently 
    • Friendliness, empathy, modesty, and love of learning

Preferred skills and experience:

    • Experience operating data infrastructure on Kubernetes
    • Experience writing a custom Presto/Hive plugin
    • Experience working with structured data serialization (ProtoBuf, Cap’n Proto)
    • Strong understanding of the CQRS pattern/event sourcing
In keeping with our beliefs and goals, no employee or applicant will face discrimination/harassment based on: race, color, ancestry, national origin, religion, age, gender, marital domestic partner status, sexual orientation, gender identity, disability status, or veteran status. Above and beyond discrimination/harassment based on 'protected categories,' we also strive to prevent other, subtler forms of inappropriate behavior (e.g., stereotyping) from ever gaining a foothold in our office. Whether blatant or hidden, barriers to success have no place at OneSignal.