Senior Splunk Engineer

Ashburn, VA
Federal / Civilian – Cyber / Security /
Regular /
Remote
At Agile Defense we know that action defines the outcome and new challenges require new solutions. That’s why we always look to the future and embrace change with an unmovable spirit and the courage to build for what comes next.

Our vision is to bring adaptive innovation to support our nation's most important missions through the seamless integration of advanced technologies, elite minds, and unparalleled agility—leveraging a foundation of speed, flexibility, and ingenuity to strengthen and protect our nation’s vital interests.

Requisition #: 303

Job Title: Senior Splunk Engineer

Location: Ashburn, VA

Clearance Level: Active DoD - Secret

Salary Range: $133,000 - $199,500
 
Required Certification(s): 
·       Current Splunk Enterprise Certified Architect certification.

SUMMARY
The Security Operations Center (SOC) is a U.S. Government program responsible to prevent, identify, contain and eradicate cyber threats to the DHS's networks through monitoring, intrusion detection and protective security services to DHS's information systems.  The SOC is responsible for the overall security of the clients Enterprise-wide information systems, and collects, investigates, and reports any suspected and confirmed security violations.
The Splunk Engineer will install and maintain Splunk infrastructure, gather requirements from customers, onboard data, and assist end users with searches, dashboards, reports, and knowledge objects. The Splunk Engineer may be required to interact with senior management, as necessary.  Conceptualize, design, build, and maintain current and future customer-supported tools and platforms  Manage multiple assignments, changing priorities, and work independently with little oversight  Lead team of Splunk engineers in the management of Splunk solution to optimize data availability for SOC and other stakeholders.  Develop data storage, access, and retention strategies for a large IT enterprise using industry standards and best practices to advise Customer executive-level stakeholders  Design, build, implement, and administer Splunk infrastructure in on-prem and cloud environments.

JOB DUTIES AND RESPONSIBILITIES
·       Conceptualize, design, build, and maintain current and future customer-supported tools and platforms
·       Manage multiple assignments, changing priorities, and work independently with little oversight
·       Lead team of Splunk engineers in the management of Splunk solution to optimize data availability for SOC and other stakeholders.
·       Develop data storage, access, and retention strategies for a large IT enterprise using industry standards and best practices to advise Customer executive-level stakeholders
·       Design, build, implement, and administer Splunk infrastructure in on-prem and cloud environments.
·       Create, manage, and support automation solutions for Splunk deployment and orchestration in on-prem and cloud environments
·       Work with existing and custom Splunk applications and add-ons to fulfill customer needs
·       Provide overall engineering and design support for a distributed Splunk environment consisting of heavy forwarders, indexers, and search head servers, spanning security, performance, and operational roles
·       Onboard data to Splunk via forwarder, scripted inputs, TCP/UDP, and modular inputs from various sources
·       Normalize data to ensure CIM compliance, and develop data models to accelerate queries, dashboards, and correlation searches
·       Work closely with all relevant stakeholders to solve technical problems at the network, system, and application levels
·       Conduct periodic architectural reviews of Splunk and related systems to assess effectiveness and propose optimal installation alternatives as required
·       Develop and manage comprehensive documentation, artifacts, procedures, and processes for the optimal management of the Splunk infrastructure.

QUALIFICATIONS

Required Certifications
·       Current Splunk Enterprise Certified Architect certification.

Education, Background, and Years of Experience
·       Minimum of a Bachelor’s degree coupled with 7+ years’ experience in the Information Technology arena.

ADDITIONAL SKILLS & QUALIFICATIONS

Required Skills
·       8-12 of experience with Linux and Windows system administration or an expert understanding of operating systems and common operating environments
·       5+ years of experience with Splunk in distributed deployments
·       Current Splunk Enterprise Certified Architect certification
·       Excellent written and oral skills, ability to work closely with multiple customers, manage expectations, and track engagement scope
·       Experience implementing FISMA, NIST, NSA, and other information security, cybersecurity and CDM-related
·       industry policies, procedures, guidelines, standards, and best practices
·       Expert-level knowledge and ability with Splunk Enterprise Security or integration with other Security Information and Event Management (SIEM) platforms
·       Extensive experience with advanced configuration of Splunk including Indexer Clustering and Search Head Clustering
·       Proficient at data on-boarding activities including routing, parsing, and normalizing events to the Splunk Common Information Model (CIM)
·       Proficiency onboarding data using Splunk-developed add-ons forWindows, Linux, and common third-party devices and applications
·       Experience onboarding data into Splunk via forwarder, scripted inputs, TCP/UDP, and modular inputs from a variety of sources
·       Proficiency managing Splunk using the Splunk command-line interface
·       Proficiency managing Splunk using configuration files
·       Experience collaborating with separate engineering teams to configure data sources for Splunk integration
·       Intermediate understanding of SQL and common SQL dialects
·       Proficiency implementing and onboarding data in Splunk DB Connect
·       Experience with Splunk performing systems administration, including performing installation, configuration, monitoring system performance and availability, upgrades, and troubleshooting
·       General networking and security troubleshooting (firewalls, routing, NAT, etc.)
·       Splunk architecture/design, implementation, and troubleshooting experience
·       Experience in managing, maintaining, and administering multi-site indexer cluster
·       Proficiency developing log ingestion and aggregation strategies per Splunk best practices
·       Perform integration activities to configure, connect, and pull data with 3rd party software APIs
·       Experience implementing and optimizing Splunk data models
·       Proficient in regular expressions
·       Scripting and development skills (Bash, Python, or PowerShell)
·       Ability to autonomously prioritize and successfully deliver across a portfolio of projects
·       Department of Homeland Security ESOC employees are required to obtain an Entry on Duty (EOD) clearance.

WORKING CONDITIONS

Environmental Conditions
·       Department of Homeland Security ESOC employees are required to obtain an Entry on Duty (EOD) clearance.

Strength Demands
·       Light – 20 lbs. Maximum lifting with frequent lift/carry up to 10 lbs.  A job is light if less lifting is involved but significant walking/standing is done or if done mostly sitting but requires push/pull on arm or leg controls.

Physical Requirements
·       Stand or Sit; Walk; Repetitive Motion; Use Hands / Fingers to Handle or Feel; See; Push or Pull
Employees of Agile Defense are our number one priority, and the importance we place on our culture here is fundamental. Our culture is alive and evolving, but it always stays true to its roots. Here, you are valued as a family member, and we believe that we can accomplish great things together. Agile Defense has been highly successful in the past few years due to our employees and the culture we create together. What makes us Agile? We call it the 6Hs, the values that define our culture and guide everything we do. Together, these values infuse vibrancy, integrity, and a tireless work ethic into advancing the most important national security and critical civilian missions. It's how we show up every day. It's who we are.

Happy - Be Infectious.
Happiness multiplies and creates a positive and connected environment where motivation and satisfaction have an outsized effect on everything we do.

Helpful - Be Supportive.
Being helpful is the foundation of teamwork, resulting in a supportive atmosphere where collaboration flourishes, and collective success is celebrated.

Honest - Be Trustworthy.
Honesty serves as our compass, ensuring transparent communication and ethical conduct, essential to who we are and the complex domains we support.

Humble - Be Grounded.
Success is not achieved alone, humility ensures a culture of mutual respect, encouraging open communication, and a willingness to learn from one another and take on any task.

Hungry - Be Eager.
Our hunger for excellence drives an insatiable appetite for innovation and continuous improvement, propelling us forward in the face of new and unprecedented challenges.

Hustle - Be Driven.
Hustle is reflected in our relentless work ethic, where we are each committed to going above and beyond to advance the mission and achieve success.

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities
The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60-1.35(c)