Senior Data Pipeline Engineer

São Paulo /
Engineering – Data Engineering /
Migo is a cloud-based platform that enables companies to offer credit to their consumer and small business customers. Leveraging proprietary datasets, Migo builds ML algorithms on transaction data to assess credit risk then offers credit lines to the companies’ customers. This credit line can be used to make purchases from a merchant or withdraw cash without the need for point-of-sale hardware or plastic cards. Because of our proprietary data and innovative technical solutions, Migo is able to extend credit to underbanked customers who are not typically covered by credit bureaus. Migo is headquartered in San Francisco and currently operates in Nigeria and Brazil.

As an experienced data pipeline engineer, we would look to you to design a high-capacity data pipeline architecture as well as design data governance policies while ensuring privacy and security of sensitive data. You will develop scalable data management systems and manage the storage of large datasets. You will be involved with creating performant and reliable ETL jobs, as well as manage external data feeds across on-premise and cloud infrastructure.

In the first 180 days you will:
- Rewrite our big data ETL pipeline to create datasets for our modeling efforts
- Wrangle with raw data from large, diverse data sets from our distribution partners
- Automate data integrity checks
- Connect our various databases and real-time services to our BI platform
- Route platform events to various data partners to drive functionality
- Design and manage A/B test data ingestion and analytics

The opportunities for you to make an impact are limitless. We are looking for candidates who have expertise in:

- ETL and ELT pipelines
- Data processing and job orchestration
- SQL Data warehousing
- Feature engineering for machine learning
- Implementing analytics pipelines to assess machine learning model results
- Setting up data and cloud environments to make data science more efficient
- Quickly learning new tools

To fit best with Migo culturally, you should be a clear and concise communicator, with an ability to communicate ideas to a wide range of stakeholders both technical and non-technical. You appreciate hearing different points of view and wait to hear other’s point of view before offering your own. You have a pragmatic approach to building systems, see multiple ways of solving problems, and are able to discuss the tradeoffs of each solution. You are technology agnostic with broad depth and breadth of experience using many different technologies.

Our interview process begins with an introductory call to help you better understand the opportunity, give us a glimpse into your interests and motivations, and help you decide if Migo is the right place for you to be your happiest and most successful self. From there, we will conduct a technical screen with one of our engineers so you can show us your skills. If all goes well, you will be invited to a virtual onsite to interview with an interview panel from our data science and software engineering teams. Our onsite interview includes 3-4 technical rounds, as well as conversations around what it is like to work at Migo and how you would work with the team on a daily basis. It is designed to assess a broad range of skills so that we can gain a holistic understanding of what you bring to the team and where you shine. We pride ourselves on being transparent throughout the entire interview process with conversations around compensation and the impact you will make here at Migo.

When you come to work at Migo, you can be assured that your work will be deeply meaningful. You will spend your days solving challenging problems alongside smart and capable colleagues. Daily decisions here have tangible and immediate impact on millions of people. You will be given both the respect and the latitude to drive best-practices for building world-class systems. You will be fully supported by executive management, many of whom have engineering backgrounds and will share your concerns if you say we are accumulating too much technical debt.

Our technology stack consists of modern tools; we are open to technologies and pick the right tool for the job:

- Python for Machine Learning e.g. (Scikit-learn and PyTorch)
- Python/Scala for data pipelines
- Snowflake cloud SQL database for data warehousing
- Scala/Java/Python for micro-services and APIs
- Swagger(OpenAPI) for API documentation
- Docker and Kubernetes to package and run services
- AWS for underlying infrastructure
- On-premise servers for data processing and extraction at our partners

- Degree in a relevant technical field or equivalent experience
- Five years of experience building production-quality software infrastructure
- Ability to own and deliver on large, multi-faceted projects with little guidance
- Experience developing ETL jobs
- Fluency in SQL and experience with RDBMSs
- Fluency in Python data tools e.g. Pandas, Dask, or Pyspark
- Experience designing and building big data pipelines
- Experience working on large scale, distributed systems

- Experience working with AWS, GCP, or other cloud-based services
- Experience at a rapidly growing startup or with cutting-edge teams