Data engineer / Senior data engineer
Boston, NYC, or Bay Area /
Biobot Analytics is a wastewater epidemiology company and uses technology developed at MIT to measure human health in sewage to understand population health in cities.
In this role, you will provide critical support in maintaining and improving our existing data analysis and reporting pipelines, as well as contribute to the development of new pipelines as we launch new products. You’ll also work with our software and product team as we work to scale and deploy our infrastructure.
Roles and responsibilities
- Maintain and improve our current codebase for processing and analyzing Covid-19 wastewater data. Improvements might range from adapting the pipeline to changing data inputs, updating reports in response to customer feedback, adding tests to ensure robust and reproducible code, incorporating new analytical models developed by our data science team, and iterating to improve the overall structure and reliability of the codebase.
- In collaboration with our data science and lab teams, implement a new data pipeline to process and analyze opioid data. You'll work with our software engineering team to convert your MVP data pipeline into production-level, automated code that integrates with our entire AWS-hosted product offering.
- Work with our product and engineering team to integrate your data pipeline with Biobot’s APIs and interactive visualization dashboards.
- Senior data engineer: work with our data science and lab teams to design frameworks for building data analysis pipelines across a variety of product offerings and data types. Lead database and codebase design decisions for existing and new data pipelines.
- Senior data engineer: implement best practices for version control, code review, testing, and collaboration in a remote working environment.
Our tech stack
- Our data pipelines are written primarily in Python, with some bash scripting. We currently rely on snakemake as our scientific workflow manager, and are always open to exploring more suitable tools.
- We collaborate on code via GitHub and communicate primarily through Slack. We strive to achieve high standards of documentation, and work together to implement best practices for collaborating on code.
- We have an AWS-hosted infrastructure and use Jenkins to automate our data processing jobs.
Skills and qualifications
- Python experience, with a solid command of data wrangling skills and tidy data concepts (primarily in pandas). Ability to work via the command line with basic bash scripting skills.
- Demonstrated experience working with data analysis pipelines. Bonus if you have experience designing pipelines from scratch, including robust data validations and tests.
- Practical understanding of version control and software collaboration best practices. Ideally, experience implementing version controlling systems for data.
- Willingness and ability to work in a rapidly changing environment, responding to customer feedback and evolving product priorities and adapting codebases accordingly.
- Experience working with biological data is a plus, but not required. Interest and curiosity to learn the basic science behind our data is expected of all Biobot team members.
- Senior data engineer: experience working with cloud-based infrastructures, ideally AWS.
- Senior data engineer: experience designing data pipelines and working with pipelining and automation tools (e.g. Jenkins, Airflow, etc).
About Biobot Analytics
Our mission is to transform wastewater infrastructure into public health observatories. Biobot Analytics is a wastewater epidemiology company and uses technology developed at MIT to measure human health in sewage (based on what is excreted in urine and stool) to understand population health in cities. We first launched an opioid product to support government and public health officials in responding to the opioid epidemic, and this year launched a Covid19 product to estimate the scope of the outbreak in communities.
Inspired by the potential of wastewater epidemiology, we are the first company in the world to bring wastewater epidemiology to market. We are VC backed by top investors including The Engine, DCVC, Y Combinator and Homebrew.
Battling the opioid epidemic and now the Covid19 outbreak is just the beginning - we’re transforming sewage into a data asset and building a public health database. Eventually, Biobot will be an early warning system for disease, a map of nutrition disparities, and more. Headquartered in the Boston area with an office in NYC, we aim to create the bedrock of human health infrastructure and smart cities in countries across all six continents.
At Biobot, we believe that the best technologists can improve society and we strive to build a workplace in which everyone can thrive. Our goal is to be a diverse team that is representative, at all job levels, of the society we live in. We encourage applications from women; non-binary, trans, and gender-non conforming individuals; Black and indigenous individuals, and people from other minoritized racial and ethnic groups; and other groups traditionally underrepresented in technology startups.