Data Engineer

San Francisco
About Intricately
Intricately is the only product of its kind that captures global cloud usage, spend, and engagement by collecting real-time data from across the public Internet. Companies across the cloud and infrastructure ecosystem rely on Intricately's market insight to get visibility into a rapidly expanding market, map and plan territories, and much more.

Engineering at Intricately
Many companies talk about Big Data- we're living it. How would you model cloud/infrastructure spend for the Fortune 500? How would you collect this data? How would you track variations in this dataset? What's noise vs signal? What's the most efficient way to query 30mm+ Internet Gateways? How do you create a front-end to surface insights for customers so they can get the information they need and get on with their day? Intricately engineering is focused on solving these challenges every day. Got strong opinions on the topic? We'd love to hear them.

As an early hire, you will be a core member of the team with a tremendous impact on the product, culture, and trajectory of the company. You'll work closely with the founding team to refine, improve, and ship product. You'll be part of a team creating a service that's heavily used (and loved) by some of the largest cloud and infrastructure companies in the world.

We are looking for a Big Data Engineer to spearhead the collecting, storing, processing, and analyzing of huge sets of data. Your primary focus will be both surfacing actionable insights from our dataset as well as driving the long-term strategy around data collection. You've got strong opinions on data solutions, maintenance, implementation, and monitoring- you will be the go-to resource on big data architecture across the company.

Some context on our scale
- We see Millions of daily real-time events representing the living breathing internet
- We track Billions of digital assets
- We store Petabytes of event-state data


    • Selecting and integrating Big Data tools and frameworks
    • Implementing ETL strategies
    • Monitoring performance and driving the discussion on supporting infrastructure
    • Defining data retention policies


    • Proficient in your understanding of distributed computing principles
    • Proficient with Hadoop v2, MapReduce, HDFS
    • Experienced with building stream-processing systems, using solutions such as Storm or Spark-Streaming, Kafka
    • Experienced with Big Data querying tools, such as Pig, Hive, and Impala
    • Experienced with NoSQL databases, such as HBase, Cassandra, MongoDB
    • Knowledgeable of various ETL techniques and frameworks, such as Flume
    • Experienced with various messaging systems, such as Kafka or RabbitMQ
    • Experienced with Big Data ML toolkits, such as Mahout, SparkML, or H2O
    • Experienced with Lambda Architecture, and understanding of its advantages and drawbacks


    • Front-End : Angular
    • Back-End: Ruby / Rails, Node
    • Data Stores: Postgres, Mongo, ElasticSearch, Dynamo, InfluxDB, Redis, Memcache, S3
    • Clouds: AWS, Heroku, DigitalOcean


    • Comprehensive medical, dental and vision insurance
    • Lunch, snacks, fancy beverages, and more
    • Gym membership or wellness regimen of your choosing – up to $150 a month
    • Awesome team building activities and events - let's hang out
    • Whatever computer and desk setup you want - let's get to work!
Intricately is an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. Diversity and inclusion are at the core of our team's culture. We're proud to welcome people with different backgrounds, perspectives, and experiences — join us and let's make something awesome together.