DataOps Engineer
Seattle, WA
Engineering – Data Engineering and Data Science /
Full time (Remote) /
Remote
Mactores is a trusted leader among businesses in providing modern data platform solutions. Since 2008, Mactores have been enabling businesses to accelerate their value through automation by providing End-to-End Data Solutions that are automated, agile, and secure. We collaborate with customers to strategize, navigate, and accelerate an ideal path forward with a digital transformation via assessments, migration, or modernization.
We are looking for a DataOps Engineer with expertise while operating a data lake. Amazon S3, Amazon EMR, and Apache Airflow for workflow management are used to build the data lake.
You have the experience of building and running data lake platforms on AWS. You have exposure to operating PySpark-based ETL Jobs in Apache Airflow and Amazon EMR. Expertise in monitoring services like Amazon CloudWatch.
If you love solving problems using yo, professional services background, usual and fun office environment that actively steers clear of rigid "corporate" culture, focuses on productivity and creativity, and allows you to be part of a world-class team while still being yourself.
What you will do?
- Operate the current data lake deployed on AWS with Amazon S3, Amazon EMR, and Apache Airflow
- Debug and fix production issues in PySpark.
- Determine the RCA (Root cause analysis) for production issues.
- Collaborate with product teams for L3/L4 production issues in PySpark.
- Contribute to enhance the ETL efficiency
- Build CloudWatch dashboards for optimizing the operational efficiencies
- Handle escalation tickets from L1 Monitoring engineers
- Assign the tickets to L1 engineers based on their expertise
What are we looking for?
- AWS data Ops engineer.
- Overall 5+ years of exp in the software industry Exp in developing architecture data applications using python or scala, Airflow, and Kafka on AWS Data platform Experience and expertise.
- Must have set up or led the project to enable Data Ops on AWS or any other cloud data platform.
- Strong data engineering experience on Cloud platform, preferably AWS.
- Experience with data pipelines designed for reuse and use parameterization.
- Experience of pipelines was designed to solve common ETL problems.
- Understanding or experience on various AWS services can be codified for enabling DataOps like Amazon EMR, Apache Airflow.
- Experience in building data pipelines using CI/CD infrastructure.
- Understanding of Infrastructure as code for DataOps ennoblement.
- Ability to work with ambiguity and create quick PoCs.
You will be preferred if
- Expertise in Amazon EMR, Apache Airflow, Terraform, CloudWatch
- Exposure to MLOps using Amazon Sagemaker is a plus.
- AWS Solutions Architect Professional or Associate Level Certificate
- AWS DevOps Professional Certificate
Life at Mactores
We care about creating a culture that makes a real difference in the lives of every Mactorian. Our 10 Core Leadership Principles that honor Decision-making, Leadership, Collaboration, and Curiosity drive how we work.
1. Be one step ahead
2. Deliver the best
3. Be bold
4. Pay attention to the detail
5. Enjoy the challenge
6. Be curious and take action
7. Take leadership
8. Own it
9. Deliver value
10. Be collaborative
We would like you to read more details about the work culture on https://mactores.com/careers
The Path to Joining the Mactores Team
At Mactores, our recruitment process is structured around three distinct stages:
Pre-Employment Assessment:
You will be invited to participate in a series of pre-employment evaluations to assess your technical proficiency and suitability for the role.
Managerial Interview: The hiring manager will engage with you in multiple discussions, lasting anywhere from 30 minutes to an hour, to assess your technical skills, hands-on experience, leadership potential, and communication abilities.
HR Discussion: During this 30-minute session, you'll have the opportunity to discuss the offer and next steps with a member of the HR team.
At Mactores, we are committed to providing equal opportunities in all of our employment practices, and we do not discriminate based on race, religion, gender, national origin, age, disability, marital status, military status, genetic information, or any other category protected by federal, state, and local laws. This policy extends to all aspects of the employment relationship, including recruitment, compensation, promotions, transfers, disciplinary action, layoff, training, and social and recreational programs. All employment decisions will be made in compliance with these principles.
Note: Please answer as many questions as possible with this application to accelerate the hiring process.
