Data Platform Engineer

San Francisco, CA
Engineering
Full-time
Who We Are:

With industry-leading Electronic Logging Device (ELD) hardware and Fleet management platform- KeepTruckin is bringing the trucking industry online and fundamentally changing the way freight is moved on our roads.

About the job:

We’re looking for passionate engineers to work on KeepTruckin’s Data and Machine Learning group. As an early member of our team, you will spearhead KeepTrucking’s efforts to organize and use petabyte-scale telematics, vehicle data, videos and various customer and derived data through efficient storage, transformation, retrieval, modeling and serving.

Responsibilities:

    • Own KeepTruckin’s data pipeline and processing flow maintaining reliability, scalability and performance
    • Design and implement data models and schema based on engineering and business needs
    • Set up workflows and ETL infrastructure with tooling to enable self-serve data processing
    • Build platform for machine learning model training, batch evaluation, automated re-training, continuous monitoring and deployment in a micro-service architecture
    • Build visibility into machine learning and system performance and optimize if necessary

Qualifications:

    • 4+ years experience in data engineering and building machine learning driven products
    • Extensive experience with Hadoop tools such as HDFS, Hive, Presto, Parquet or similar technology
    • Expertise in data-oriented programming such as Spark, Python, Map-Reduce, SQL with solid understanding of query performance and tuning
    • Experience with workflow management tools such as Airflow, AWS Batch, Luigi or similar technology
    • Knowledge of Amazon Web Services is a big plus
As an equal opportunity employer, we are committed to diversity in the workforce. In accordance with applicable law, we prohibit discrimination against any applicant or employee based on any legally recognized basis, including, but not limited to; race, color, religion, sex (including pregnancy, lactation, childbirth or related medical conditions), sexual orientation, gender identity, age (40 and over), national origin or ancestry, physical or mental disability, genetic information (including testing and characteristics), veteran status, uniformed service member status or any other status protected by federal, state or local law.