Senior Engineering Manager - Data
Tech/Product – Tech/Product
Lucid is the world’s largest audience platform for sourcing and understanding real, deterministic data about human behavior at scale. The company has built the largest and most powerful programmatic survey exchange, and has enabled brands, agencies and partners to ask and answer more than 8 billion questions worldwide. With a pool of over 70 million respondents, Lucid delivers unprecedented access to consumers for data-driven decision making – about everything from marketing to product development in real-time. Lucid is headquartered in New Orleans with offices in New York, London, Sydney and Gurgaon.
The Senior Engineering Manager - Data will be a lead engineer responsible for building our data platform that backs Lucid’s core platform and products. The data platform collects, processes and stores all of the data used by our internal and external customers to drive analysis, insights, models and reporting. Do you have extensive experience designing schemas, report generation and turning data in to business value? Are you looking to leverage and expand your skill sets around Cassandra, Amazon Redshift, Apache Kafka, Apache Spark, and other Big Data technologies used in today’s modern Big Data platforms? This is the opportunity you’ve been looking for.
- Collaborate with business units to translate challenges and opportunities into data warehousing/ETL and Stream Processing projects
- Analyze source systems including internal systems and API integrations
- Design and implement ETL systems to source data into Data Warehouse and OLAP DBs
- Perform SQL query performance tuning
- Aid in creation of dimensionally modeled data warehouse
- Create and support mechanisms to ease access and queries for data scientists/data intensive applications
- Create and support data pipelines to ingest and process data according to given specs (3pd and normalisation)
- Move data out of legacy stores to a datalake/dwAssist training ML models Can design and advise on data migrations strategies between data storages taking in account availability, data integrity and recovery.
- 5+ years of experience in development with a focus on ETL and data warehousing
- Strong knowledge of dimensional data modeling/star schemaStrong knowledge of SQL including the ability to analyze execution plan for performance tuning
- Ability to identify and implement new tools as projects dictate
- Experience creating views and stored procedures for supporting various reporting applications
- Experience with source control systems including Github
- Continual learner, always looking for a new and better way to solve a problem
- Knows Python or another programming language, can ship and review code, support the team doing thatKnows data migration tooling for PGSQL, Cassandra and SQLServer
- AWS experience with S3, RDS, Redshift, EMR, and Elasticsearch
- PostgreSQL experienceExperience with NoSQL data sources including Cassandra
- Experience with SQL Server 2014, In-Memory tables, DAX, MDX and Power BI
- Experience with Apache Spark, Kafka, Redis
- Experience creating Data Lakes
At Lucid we foster a collaborative and inspiring workplace. We pride ourselves in doing this by recruiting, hiring and retaining diverse, passionate, and forward-thinking talent. Lucid is committed to and encourages an inclusive environment and we are dedicated to providing equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please let us know.