Software Engineer - Data and Monitoring

Calabasas, CA
Science & Engineering - Materials & Microsystems Laboratory – Materials & Microsystems /
Regular /
Based in Southern California with locations in Malibu, Calabasas and Camarillo; HRL has been on the leading edge of technology, conducting pioneering research and advancing the state of the art. 

General Description:
As a senior software engineer in the Internal Tools group you will be responsible for building tools to collect logs and metrics from various hardware and software components of our quantum infrastructure; examples include making key data searchable, building endpoint monitoring into existing APIs, and maintaining the monitoring software stack across multiple enterprise networks. You will operate as part of a cross-functional team (IT, Physicists, Software Engineers, Data Scientists, and Industrial Technicians) and will often be the main link between otherwise disparate groups. This role is high-impact with high visibility, and the services you build and maintain will be an essential part of the function of the Quantum Software team.

Supervisory Responsibilities:
Expected to display technical leadership
May be required to mentor junior staff and oversee their contributions to specific projects
Essential Duties:
Build and maintain infrastructure for scraping logs, capturing metrics, enriching/processing data, and sending it to Elasticsearch
Build endpoint monitoring into APIs
Profile and tune Elasticsearch deployment to maintain performance
Develop internal tools written in Python as well as various shell-scripting languages (batch, powershell, and bash)
Develop procedures and standards for archival and backup of data stores
Work with developers and end-users to build queries, dashboards, and reports to support business objectives
Work with data scientists and engineers to develop and implement anomaly detection routines and machine-learning algorithms
Maintain a high level of industry knowledge and best practices; stay abreast of new tooling options; provide recommendations for enhancing, upgrading, deprecating monitoring stack as appropriate
Required Skills:
Demonstrated expertise with Elasticsearch ELK stack
Familiarity with medium- to high-volume logging, syslog, and its associated services
Experience writing SQL queries
Experience writing shell scripts
Solid understanding of core computer science fundamentals including data structures, algorithms, and computer architectures
Basic familiarity with statistics and data science
Strong communication skills; ability to write and maintain comprehensive, readable documentation
Required Education:
7+ years relevant software engineering experience.
Physical Requirements:
Strong verbal communication skills
Program a computer via keyboard or some other method that is as fast as a typical programmer on a keyboard
Program a computer with visual feedback from a computer monitor
Long hours sitting or standing to program a computer
Special Requirements:
U.S. citizenship; active security clearance or ability to obtain one. 
Personable and pleasant to have on a team

The base salary range for this full-time position is $145,745 - $186,688 + bonus + benefits. 

Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range during the hiring process. Please note that the compensation details listed reflect the base salary only, and do not include potential bonus or benefits.
Don’t meet every single requirement? Studies have shown that some people are less likely to apply to jobs unless they meet every single desired qualification. At HRL, we are dedicated to building a diverse, inclusive, and authentic workplace, so if you’re excited about this role but your past experience doesn’t align perfectly with every qualification in the job description, we encourage you to apply anyway. You may be just the right candidate for this or other roles.

HRL offers a very competitive compensation and benefits package. Our Regular/Full Time benefits include medical, dental, vision, life insurance, 401K match, gym facilities, PTO, growth potential, and an exciting and challenging work environment.

HRL Laboratories is an Equal Employment Opportunity employer and does not discriminate in recruiting, hiring, training or promoting, on the basis of race, ethnicity, color, creed, religion, sex, sexual orientation, gender, gender identity, genetic information, national origin, physical or mental disability, pregnancy, medical condition, age, U.S. military or protected veteran status, union membership, or political affiliation. We maintain a drug-free workplace and perform pre-employment substance abuse testing.

For our privacy policy please visit :

This position must meet Export Control compliance requirements, therefore a "U.S. Person" as defined by 22 C.F.R. § 120.15 is required. "U.S. Person" includes U.S. Citizen, lawful permanent resident, refugee, or asylee.