User Manual

Introduction to DefenseStorm

Put simply, DefenseStorm is a network security system for financial institutions. We monitor for suspicious activity, alert you when its found, and work with you to stop any potential attacks and prevent them from reoccurring. Think of a security system for your house, that's what provide for network protection. 

We communicate with your system through the DVM (DefenseStorm virtual machine), or the Windows Agent, depending on your asset type and configuration settings. Once data reaches our cloud, it is pre-processed via classifiers and placed within a datastore. Triggers search through live data to generate an alert, incident, email, or a combination of the three. 

                                                                       

Contact Us

There are a few different ways to contact DefenseStorm. You can contact us through Connect, TRAC Team, or Knowledge Center feedback.

  1. Connect: Technical issues with the product, DVM, or UI.
  2. TRAC Team: Security concerns or potential threats on your network. 
  3. Knowledge Center Feedback: Submit general questions on functionality or request additional documentation.


User Interface

All the great features offered by the DefenseStorm GRID are performed through the user interface. This includes the dashboard and all its features (See Dashboard), as well as Alerts, Policy, Assets, and more! For help logging into the UI, see our Login FAQ.  

Events

Events is an extremely powerful search engine that gives you the ability to investigate log data thoroughly and efficiently. Queries can be as simple or complex as you make them using straightforward search query language. See Events for more detailed information.  

Classifiers

Classifiers allow you to pre-process your data by creating fields, deleting data, and changing field values. See Classifiers for more detailed information. 

Alert Inbox

Alert Inbox is a way to manage the alerts that your triggers generate. It shows useful and actionable information that helps you respond quickly. See Alert Inbox for more detailed information. 

ThreatMatch

ThreatMatch gives you the ability to turn on feeds and use Threat Intelligence Sources to identify risks. See ThreatMatch for more detailed information. 

Tickets

The Tickets section of the dashboard is where you create, monitor, and/or update Incidents. See Tickets for more detailed information.

Compliance

Compliance utilizes built-in tools that link your policies to different internal and government guidelines, system alerts, and helps you define, enforce, and report on your security policies. See our Compliance & Policy Reporting  article for more details.  


Assets

Assets allows you to manage devices that are sending data to the DVM.  The Assets page displays all tracked and untracked assets. See Assets for more detailed information. 

TRAC Team

In dynamic co-management with your resources, the cybersecurity and cybercompliance experts on our TRAC Team monitor and manage your security and compliance  24 x 7 x 365 to identify and resolve millions of potential cyber threats. 

onTRAC Services

  • Trigger development and maintenance. Any trigger you create requires a discussion with TRAC to discuss expectations before it is actively monitored. 
  • Classifier development and maintenance
  • Log analysis
  • Threat Match verification/analysis
  • Gap analysis for potential instrumentation sources to improve visibility
  • Incident Management
    • Analyze and Remediate open incidents
    • Assigning to customer when further action is necessary

Additional Services

  • Virtual CISO: leverage strategic expertise of a Chief Information Security Officer without investing in full-time staff.
  • Vulnerability Management: stay ahead of potential and emerging threats across both your virtual and physical assets and properties.

Unavailable Services

Our TRAC Team can only provide recommendations for network changes, not execution. The following services cannot be completed by our team:

  • Add blocks to your Firewall
  • Block traffic 
  • Management of any device or appliance 
  • Create user accounts

What do we need from you?

 To efficiently monitor and analyze your network, provide the following:

  • Asset List
  • Network diagrams
  • Technical security controls inventory (NAV, AV, etc.)
  • User List (phone & email)
  • Security Policies
  • Incident Response Plan
  • Privileged Account Management
  • Third Party Vendor Management


Service Level Agreements (SLA’S)

The timelines listed in the table below are the maximum response times for our TRAC Team to initiate triage.

Guardian Security Monitoring Services SLA - Triage

System Generated Actionable Item

Severity

Response Time

System Incidents

High

2 hours

Medium

12 hours

Low

Next Biz Day

None

N/A


Contact for Escalation Cases

In order to streamline the triage process, provide our TRAC Team with who to contact and when, by filling out the escalation use case chart. Below is an example of the chart.


The DefenseStorm Virtual Machine

The DefenseStorm Virtual Machine (DVM) integrates your network with the cloud, via outbound HTTPS (port 443), so we can monitor network activity. The DVM accepts Syslog (both formatted and unformatted) for transferring data. 

Have you Deployed the DVM Before?

Deploying the DVM allows logs to be sent from your network to the DefenseStorm GRID for monitoring. After the initial deployment, you are notified via Release Notes when an update is recommended. 

If you have already deployed the dvm and need to upgrade, see the Upgrading section. If you are unsure if an upgrade is recommended, in the main menu of the dvm, select option 8: Get DVM Status and compare your version to the latest version listed here

Deploying

If you have never installed the DefenseStorm virtual machine, follow these steps based on your virtual machine environment. The DefenseStorm Virtual Machine (DVM) image is available for:

  • VMware
  • Hyper-V

VMware

Obtain detailed OVA information directly from VMWare: https://www.vmware.com/support/developer/studio/studio20/va_user.pdf

How to Install DVM VMware Image
  1. Download the OVA
  2. Deploy the OVA image to VSphere/ESXi
  3. Power on the DefenseStorm Virtual Machine
  4. Open the Console to begin configuration
  5. Scroll down to Configuring the DVM, and follow the instructions.


Hyper-V

This section details the minimum recommended specifications for Hyper-V host servers to perform efficiently. We recommend 2012 R2 and above. According to Microsoft, the end of life dates are as follows:

  • Server 2012 R2 end of life is January 2020
  • Server 2016 end of life estimated as 2025-2026
Server Core install options

As an alternative to a paid option, Hyper-V Server can be installed on a host to enable services on a headless server. This is a Server Core-based OS, and is command-line.

Images are provided by Microsoft at the following URLs:

For all other SKUs, the Server Core install option can be used to further reduce the resources used by the host (reduces host OS storage footprint by ~4 GB) as long as no other roles are active on the Windows host; however, the VMs must be managed remotely or through PowerShell if this option is chosen. If this option is used, ensure that remote management is set up as part of provisioning the Hyper-V host.

Windows Server Minimum Recommended Specs

These specifications assume that only remote management and the Hyper-V Server roles are enabled on the Windows Server install. Using this host for additional roles and/or services may require additional resources depending on which roles are enabled.

OS SKU (select one): Hyper-V Server, Server Standard, Server Datacenter, Server Enterprise

CPU: multi-core, 64-bit CPU

  • must support virtualization and DEP (data execution prevention)

RAM: 4 GB

  • 2 GB RAM for hyper-v host, 1-2 GB for DVM

Disk size: 100 GB.

  • Recommend sizing this higher if possible (200GB+) to allow for VM snapshotting and account for troubleshooting scenarios where DVM reprovisioning alongside an existing copy is necessary.
  • 100 GB breakdown: minimum 32 GB for Windows install + 28 GB space for windows updates, 20 GB for DVM, and 30 GB free space for VM image upgrades.

Network adapter: Gigabit ethernet network adapter

  • At least 10 Mb/s peak outbound network bandwidth to internet (Spikes of high event volume may require higher peak upload bandwidth to avoid queueing event data on the DVM).

How to Install the DVM Hyper-V Image

  1. Download the zip file
  2. Deploy the Hyper-V image to the Windows Server Host.
  3. Power on the DefenseStorm Virtual Machine.
  4. Open the Console to begin configuration.
  5. Scroll down to Configuring the DVM, and follow the instructions.

Configuring

After you have installed the DVM via VMware or Hyper-V, follow these instructions to configure your DVM settings.

  1. Log into the DVM
    username: ds / password: defensestorm 
  2. Change the password (forced by system)
  3. Configure Time Zone
    Select option 5 and then answer the questions regarding your local timezone.
  4. Configure Networking
    Selection option 4,  
    • Choose to enable DHCP
    • Set the static IP address.
    • Set the Netmask 
    • Set the Gateway
    • Set the Nameserver 1,2, and 3. If there is no Nameserver 2 or 3, then leave the field blank and select 'OK'.
  5. Set DefenseStorm Credentials
    Select option 1, and then use the instructions below to answer the questions.
    • Input your Administrator email address and password.  Note: These credentials are used for one-time authorization of the DVM and are not stored.
  6. Verify Network Connectivity
    Select option 6: Troubleshooting, then option 1: Connectivity Tests

    Once the network tests have passed, you see the following message:
  7. Verify that the DVM is sending messages to the Console by verifying that the API key displayed through your DVM menu and the web console are the same. Within your DVM, select option 8: Get DVM Status to view your API key.
  8. After you have seen the api key through your DVM, open your DefenseStorm UI to verify that associated events are coming through. Go to Events, and select the API key from the filters drop-down list.


Enabling DVM Event Compression

The DVM supports compression to help with limited egress bandwidth. Compression also helps prevent events from being dropped during ingestion. Sign into the DVM and enter the following information. 

  1. Select option (10) Bash Shell
  2. type  sudo vi /etc/praesidio/praesidio.conf   (Enter DVM logon credentials if prompted)
  3. After config file is displayed, do the following:

    Type this Command

    Action (hit)

    Explanation

    /Flush

    Enter

    With cursor on the /Flush line, type the next command.

    o


    Creates new line for typing

    Compress=True

    Enter, ESC

    This adds the command to the file and exits edit mode.

    :wq

    Hit Enter

    Saves the file and exits the vi

  4. Restart syslog-ng service with the new configuration file upon display of the command prompt.
    Type this CommandAction
    sudo service syslog-ng reloadhit enter
    exithit enter

If syslog restarts without error, the compression feature has been successfully enabled. 


Upgrading

Upgrading your DVM keeps your network protection up to date with the best features and enhancements. To determine if your DVM is eligible for upgrade, select Option 8: Get DVM Status, and view your version number. If your version number does not display, please open a support ticket for assistance. If a version number displays, complete the steps below.  

Pre-Upgrade

There are a few steps required to ensure that your DVM environment is ready for a successful upgrade. 

  • Backup critical files
  • Increase max_connections

Which files need to be backed up?

To prevent any unintended alterations to configuration files, we recommend that you back them up prior to upgrade, and restore them once the upgrade has completed.

As listed in the table, some files may not be present on your network, therefore, they do not need to be backed up.

Configuration File

Description

/etc/praesidio/praesidio.conf

This is the DVM's primary configuration file, and contains the provisioned DVM API key as seen in the web console.

/etc/syslog-ng/conf.d/praesidio.conf

This is the syslog-ng configuration file, generated during DVM servicing by the pConfig script.

--Optional Files--

--The below files may not exist unless customized--

/etc/syslog-ng/conf.d/snmp.conf

/etc/default/snmpd

SNMP configuration files, generated / modified as part of DVM SNMP configuration. (See connect article for details)

/lib/ufw/user.rules

/lib/ufw/user6.rules

User-generated firewall rules files for IPv4 and IPv6.  This is modified during setup of SSH and SNMP, among other protocols.

 

How to back them up through the DVM Menu

The following steps create a folder called "dvm_yyyymmdd" in the ds user's home directory on the DVM; then backup the configurations listed in the table above to the folder. 

In the DVM menu, select option (10) Bash Shell, then do the following:

Type this Command

Action (hit)

Explanation

cd ~

Enter

Navigate to the current user's home directory.

mkdir dvm_20171025

Enter

Make new backup folder (change date portion to current date)

cd dvm_20171025

Enter

Navigate into new backup folder.

sudo cp /etc/praesidio/praesidio.conf .

Enter

Copy the /etc/praesidio/praesidio.conf file to here (the backup folder).

Enter DVM login password if prompted.

 

sudo cp /etc/syslog-ng/conf.d/praesidio.conf .

Enter

Copy the /etc/syslog-ng/conf.d/praesidio.conf file to here (the backup folder).

-- optional section --

 

 --Ignore any copy failures for the below files; may not exist.--

sudo cp /etc/syslog-ng/conf.d/snmp.conf .

Enter

Copy the /etc/syslog-ng/conf.d/snmp.conf file to here (the backup folder).

sudo cp /etc/default/snmpd .

Enter

Copy the /etc/default/snmpd file to here (the backup folder).

sudo cp /lib/ufw/user.rules .

Enter

Copy the /lib/ufw/user.rules file to here (the backup folder).

sudo cp /lib/ufw/user6.rules .

Enter

Copy the /lib/ufw/user6.rules file to here (the backup folder).


Increasing max_connections using Bash Shell

If your host counts exceed 100 (for linux / appliances) or 500 (for NXLog server installs, or Windows Agent installs on Windows workstations / laptops), we recommend increasing the max_connections option on your DVM.

The default ports, connection counts, and port uses are described below:

Port

Max Conn. Count

Port Description

Host Type

TCP 514

100

Standard RFC-compliant Syslog port.

Unix / Linux, appliances

TCP 516

100

Non-strict syslog port.  This is used for devices that send events over syslog, but whose formats do not comply with the RFC format.  Cisco Meraki devices are an example of this.

Appliances (non-compliant)

TCP 601

500

Syslog port used by Windows NXLog clients.

Windows

TCP 1602

500

Syslog port used by the DefenseStorm Windows Agent.

Windows

 

If your host counts exceed 100 (for linux / appliances) or 500 (for NXLog server installs, or Windows Agent installs on Windows workstations / laptops), you should modify the following section to increase the number of maximum connections.  The default CPU and RAM amounts provisioned on the DVM image can support raising these counts up to 1500; if you need more concurrent connections than this, we suggest increasing the resources available to the VM instance first. 

Steps to open the file, navigate to the configuration file section, and change the values below. Select option 10: Bash Shell through the DVM Main Menu, and perform the following command steps: 

Type this Command

Action (hit)

Explanation

cd /etc/praesidio/

Enter

Navigate to the DVM configuration directory.

sudo vi praesidio.conf

Enter

Open the praesidio.conf file in Vi.  Provide DVM login password if prompted.

-- repeat next 5 steps for each count to change --



/tcp###  (e.g. /tcp514)

Enter

Moves the edit cursor to the line for tcp### (if editing TCP514 counts, type /tcp514)

ww

 

Move the cursor two words to the right.  Should be under the count number (100, 500) at this point.

dw

 

Deletes the existing count number.

a

 

Typing mode (append): moves cursor to character right.

(type your count number here)

ESC

Input your new max connection count digits here.  This adds the count to the file and exits edit mode.

-- end repeat section --



:wq

Enter

Saves the file and exits the vi


If you make a mistake and need to revert all changes and restart from the original file, type the following sequence to quit without saving.

Type this Command

Action (hit)

Explanation


ESC

Exit any edit modes

:q!

Enter

Quit without saving.


How to Upgrade your DVM

Once you have backed up all required configuration files and increased the max_connections if necessary, follow the steps listed below to upgrade your DVM. 

  1.  Access your DVM main menu.
  2.  Select Option 7: Update/Upgrade DVM.
  3. Input your DVM console user (ds user) and password when prompted. 
  4. Text scrolls by on the screen during the upgrade process. Always accept the default options.
    Do you want to overwrite praesidio file, default is No, select this option. Deleting this file deletes all previous configurations and files.
  5. Once the upgrade is finished, login to your DefenseStorm UI as usual. 

Dashboards

Once you login to the GRID UI, the first screen you see is a dashboard of charts displaying useful information regarding your network.  Our experts have created this dashboard to provide information useful to all customers regardless of size or devices. Even though it was created with you in mind, feel free to edit this dashboard, or you can clone it to create a new version without disturbing the original.  For a video walkthrough of this feature, see our Release Video.

Quick Facts

  • Last dashboard accessed displays on next login.
  • Selecting Dashboards allows you to create, edit, clone, or delete dashboards.
  • Change the duration by selecting the drop-down arrow next to During.
  • Clicking on the name of a chart takes you to the charts page where the chart itself can be edited. 



Creating a Dashboard

A custom dashboard can be created for any purpose. You can create a dashboard for firewall monitoring, board-level monitoring, or overview statistics. 

  1. Select Dashboards
  2. Click the + sign to create a new Dashboard.
  3. Give the dashboard a name, description, choose the layout, and then add charts.
    *Note: If the chart has not already been created, go to Reports > Charts to create it; then come back and add it. 
  4. Save.


Editing a Dashboard

 To edit the charts themselves,  go to Reports > Charts and edit from there.

  1. Select Dashboards from the top of the screen.
  2. Click the name of the dashboard you want to edit.
  3. Make any desired changes.
  4. Click Save.

Cloning a Dashboard

Cloning a dashboard gives you the ability to edit a copy without disturbing the original. 

  1. Select Dashboards from the top of the screen.
  2. Hover over the name of the dashboard you want to clone, and to the far right, the option to clone displays.
  3. Make any desired changes.
  4. Click Save.

Deleting a Dashboard

This does not delete the charts, only the dashboard.

  1. Select Dashboards from the top of the screen.
  2. Click the checkbox of the dashboard you want to delete.
  3. Click the trashcan icon.


Creating your Profile

Creating your profile through the DefenseStorm UI couldn’t be easier. Click on the Settings logo at the bottom left of your screen > Profile tab > enter profile information.  For information on two-factor authentication, see  Two-Factor Authentication.


Create a New User

  1. As a user with DefenseStorm admin privileges, log into the UI
  2. Select Settings page (gear icon) from the bottom left.
  3. Click on the + icon from the upper right side of the screen.
  4.  Complete the user form with desired information and click Save. After the user is created, they receive an email with simple instructions on how to complete setup.
     The user roles are:
    • Administrator
    •  Power User
    • User
    • Read-Only User



Two-Factor Authentication

During installation, two-factor authentication is automatically set up to provide an extra layer of security against unauthorized access.  Users with DefenseStorm Admin permissions can reset the 2FA settings by selecting Settings > Users > 2FA column > Reset 2FA. If you only have one DefenseStorm Admin and their account requires reset, call DefenseStorm for assistance. To eliminate the need to contact support, have two DefenseStorm Admins so they can reset each others 2FA if necessary.  

Setting up two-factor authentication

To set up two-factor authentication as an individual user:

  1. Go to Settings > Profile and select Set Up.
    Yy17DU6fnGU_IGIHVFU9rpOqcJJYetKmUA.png
  2. Select device type (Android, Blackberry, iOS, Windows Phone). While the UI mentions Google Authenticator, Authy, and Salesforce are two common alternative 2FA applications. 
  3. After completing the instructions for your device, select Enter Generated Code, and enter code generated by your chosen authentication app. *YubiKey Note: To use it with the GRID, go to Settings > Profile > turn on U2F.
  4. Select Verify to enable 2FA. It will be required upon next login. 

Using two-factor authentication

  1. Go to the DefenseStorm UI. 
  2. Enter your username and password. Verification code prompt displays.
  3. Open your authenticator app and enter the 6 to 8 digit verification code associated with your DefenseStorm account. 
  4. Once the code is entered correctly, you are fully logged into the system. 


Reports

When creating a report within the DefenseStorm UI, there are 3 main steps: 

  1. Create a chart 
  2. Organize a template 
  3. Generate a report 

By default the Reports homepage displays all generated reports. You can utilize the filter option to view reports generated from specific templates, and/or status' (active or deleted reports). Hovering over a report name displays options to download, delete, or view the charts within the report. 


Creating a DefenseStorm Report

To create a Report, the first step is to create a chart to gather and display desired data. As soon as a new chart is created, it begins gathering applicable data from the previous 90 days. This gathering of data is strictly behind the scenes, and does not impact GRID performance. 

Once you are satisfied with the charts you've created, organize them into templates.  When putting together your templates, add all desired charts with headings and descriptions specific to the template's purpose. For example, if you create an overview template, the charts and descriptions would be high-level, and as simple as possible. If you're creating an in-depth template, while utilizing some of the same charts, they would have more detailed descriptions, with additional heading 2 charts. 

After you have finalized your templates, determine the report frequency. Reports can be generated as a one-time occurrence or on a schedule.  If you select to generate the report on a schedule, you can add email addresses to receive a link once the report has been generated.


1. Creating a Chart

Charts use search queries to capture data. The process of creating a chart can be started from the Events page or the Reports page.

How to create a chart via Events page

If you are searching through events and want more information on the trends and statistics for this occurrence, select to create a chart directly from the Events screen.

  1. Within the Events page, enter search query. It must be a search query, event results based solely on filters do not apply. 
  2. Once desired results display, select the icon to create a chart.  
  3. The Reports > Chart page opens providing a chart view of your queried data.
  4. To use this chart for reporting,  go to Step 2 of the next section, "How to create a chart via Reports page".  If you do not want to save the chart, select Cancel. 

How to create a chart via Reports page

  1. Select Reports > Charts > blue + icon
  2. Enter required and desired chart information. * indicates  a required field
    Query*: Data being pulled from the GRID. You can enter a new query, or a saved search.
    Name*: What the chart is called.
    Chart Type: Default is a line graph.
    X-Asis Title: Chosen title for the horizontal axis of the graph.
    Y-Axis Title: Chosen title for the vertical axis of the graph.
  3. Select Save.

Example: Failed Admin Logons

  • Query*:  app_name:("microsoft-windows-security-auditing" OR "DefenseStorm Agent") AND (event_id:4625) AND (account_name:"Administrator" OR target_user_name:"Administrator") AND NOT defensestorm_type:alert
  • Name*:  Failed Admin Logins 
  • Chart Type:  Line
  • X-Asis Title:  Time
  • Y-Axis Title:  Count


2. Creating a Template

Templates is where you choose, organize, and give context for the charts in your report. You can create multiple templates with a variation of charts. 

How to create a template

  1. Select Reports > Templates > blue + icon
  2. Enter template title and description.
  3. Select Apply.
  4.  Select 'Add a new fragment here' to add a chart.
  5. Select chart from the drop-down in the Edit Fragment window.
  6. Create a Heading name and Heading Type. Chose either Heading 1 or Heading 2.  Heading 2 is a good option for a secondary, or sub-chart.
  7. Enter a paragraph description. What makes it relevant to your report? The same chart could have different descriptions based on the report audience and timeframe.
  8. Select Apply.  Repeat steps 4-8 to add additional charts.
  9. Click Save. The template is created and ready to be generated as a report. 

Example: Utilizing Heading 1 & Heading 2 when creating a template

For example, one of your boss' wants incredibly detailed information, while the other wants general information. You would create a template consisting of multiple heading 1 and heading 2 charts to provide in-depth information; and another template with a variety of heading 1 charts that show general information.


3. Generating a Report

Since charts and templates are created prior to generating a report, this is the quickest and easiest step in the process.  All you have to decide is if you want the report generated once, or on a schedule. 

How to generate a report 

  1. Select Reports > Templates to view all templates. 
  2. Search or scroll to find the template you wish to generate.
  3. Chose from the options to the right of the template name: Generate New Report, View Previously Generated Reports, Schedule, Clone, or Delete.
  4. Selecting to Generate a New Report > select the date range > Select Create.
  5. Previously generated report takes you to a filtered version of the Reports page.
  6. Schedule a report > add email addresses > determine frequency > Create.
    Email addresses added here receive a URL to download the report once its completed. 
  7. Cloning the template creates an exact copy of the template to allow quick, minor adjustments. 
  8. Deleting the template removes it from the list so future reports cannot be generated. No charts or data is affected by deleting a template. All previously generated reports are still visible.

On-Demand Cybersecurity Report

The Cybersecurity Report is a default report created by the experts at DefenseStorm to make presenting information quick and simple. 

The following information is provided in each report,

  • Incidents
  • Opened Incident Severity Breakdown
  • Most Active Incidents
  • Alerts
  • Most Fired Alerts
  • Events
  • Events by Hour (daily report)
  • Events by Day (weekly report)
  • Events by Date (monthly report)

Generating the On-Demand Report

  1. Go to the Reports page
  2. Select +  to open the Create Custom Report window
  3. Choose 'On-demand Report' from the Template drop-down
  4. Enter any email addresses you want the report sent to
  5. Determine Date Range
  6. Click Create

Events

The Events page is a powerful search engine to investigate network activity. The DefenseStorm GRID displays any activity that generates a log as an event. This includes activity generated from your network, or our GRID. For example, when someone attempts to login to their account, you receive an event; and when a DefenseStorm trigger fires, you also receive an event. Your search queries can be as simple or complex as desired by using any one or a combination of the search methods explained in this article. 

Types of Events

There are three types of events that display in the UI: log, alert, and system events. 

  • Log events are generated through the DVM each time an activity providing log information is performed on your network. 
  • An alert event, or alert derivative, is generated each time an alert is triggered. 
  • System events are created each time an action is performed on the DefenseStorm GRID side.  


Log Events

The DVM takes all logs generated from network activity and creates a searchable event that is displayed through the console. These events can be used to create incidents, alerts, triggers, and charts. For a list of the most common and relevant event fields, see our FAQ, Event Fields.

Alert Derivatives (alert events)

The idea behind alert derivatives is to allow alerts to be searched for, alerted on, and monitored through the events page. This event type is a composite of all events that triggered the alert.

For example, if we had a query for geo_dest.country:china, and two users, Angela and Andrew had connected to a Chinese server in a given hour, the alert event that matched geo_dest.country:china would have both hostname:"John's PC" and hostname:"Angela's PC".   A search in the console for hostname:"Angela's PC" geo_dest.country:china would return both the event that caused the alert to fire and the event for the alert.

Using Alert Derivatives

The alert derivatives functionality creates a centralized and simple way to view alerts through the addition of the alert event. When searching events, alert events can easily be included or excluded by using defensestorm_type:alert. 


Excluding alert events from queries

To exclude Alert Events from your search query, include the following flag:  

-defensestorm_type:alert

Alert Event Fields

In addition to all log event fields, alert events also contain the following:

  • timestamp - the beginning of the interval that we alerted upon
  • defensestorm_type - always "alert"
  • email - the email addresses that were informed of this alert
  • trigger_name - the user-friendly name of the trigger
  • trigger_id - the internal GUID of the trigger
  • trigger_query - the query string for the trigger
  • trigger_interval - the number of seconds in each trigger interval
  • severity - high/medium/low, copied from the trigger
  • event_count - number of hits in the search that fired the trigger

The following Information (found in the original log events) will not be duplicated to the alert event:

  • message
  • timestamp
  • ingest_timestamp
  • event_count
  • unparsed_expanded_message
  • unparsed_message
  • raw_message
  • praesidio_parse_nanos
  • percolator_tag
System Events

In addition to collecting log data for activity on your network, we also have log data for activity performed by the DefenseStorm GRID.  For example, when a new asset is created, or when a new task schedule is created.  

USE CASE 1 : Joe wants an alert when a new Asset is updated by the system. For example, when a new IP address is assigned to a new asset.

 Query:  app_name:"DefenseStorm Audit" category:asset action:"assignedIp"
Trigger Threshold: 1

Searching Events

When searching Events, the default setting is 'Last Hour' which grabs the most up-to-date information each time the page refreshes. Note: This could result in different event data and overall count when search parameters are altered.  In order to freeze your data timeline, utilize the bars in the event graph.  DefenseStorm provides several different methods for searching through events. These search methods can either be used on their own or combined together for the most specified results. 

Freezing Search Timeframe 

Freezing the timeframe works when utilizing any of the search methods, and allows you to keep the same data while altering search parameters, filters, aggregation, etc. 

  1. Enter desired search
  2. (optional) Use the 'During' field to get a close match to your desired data timeframe
  3. Move the bars in the graph to the exact timeframe

The data timeframe is now frozen and any changes made to the search parameters do not default to the most recent data, but adhere to the event graph selection. 

Ways to Search Events

  • Query Syntax
  • Duration Calendar
  • Event Spike
  • Filter Options
  • Aggregation

Query Syntax 

The DefenseStorm Events page follows the format of ElasticSearch  “Query String Queries”.  For details regarding Query Syntax, see the article, DefenseStorm Query Syntax.  For help, select the question mark icon to the right of the search bar.

Duration Calendar

The DefenseStorm Events page provides a calendar to narrow down events by time frame. If you want all events for a specific day, select During drop-down to update the calendar. The calendar also allows you to select a smaller time frame, such as Last Hour, or Last 3 Hours. 

Event Spike

Within the Events page, it displays a graph with all ingested log events. If you see suspicious activity through the graph, you can drag the bars to display only events associated with the activity spike. 

Filter Options

Within the DefenseStorm Events page,  search through events by filtering. To search by filtering, drill down through the filters and select the desired checkboxes to display only associated events.

Aggregation

The Aggregate feature allows you to group your events before searching to greatly reduce the number of events displayed; therefore increasing result speed and accuracy.

  1. Enter an Aggregate field.
  2. In the screenshot below, the events have been grouped by the primary aggregate field: account_domain.

    From here you can either select one of the displayed account domains, or add a secondary field to further narrow down the displayed domain events.


    The default view when both the primary and secondary fields are utilized, is the Spreadsheet View (as shown above). To simplify your view, select Compound from the View drop-down list. The Compound view allows you to better understand the relationship between the values of your two aggregate fields by displaying them side-by-side with their associated event count.
  3. Select the Desired result to view events
    Once you click on the desired compound result, in this example, account_domain:postilionoffice and user_domain:postilionoffice, a filter is applied to all events and the 56 events display. 

Saving a search

Saving your search allows you to reuse all methods of searching without having to fill in the fields. For example, if you used a combination of Aggregation and Query Syntax, saving the search allows you to select the search without having to reapply the primary/second aggregate fields and the search query. The option to save your search is in the top right of the event graph.

To view all previously saved searches, select Saved Searches from the top navigation of the Events page.

The Saved Searches page displays all previously saved searches. To edit the search, click the name of the search, and the Edit window displays. To view all associated events, select the magnifying glass on the far right of the saved search name. If the saved search includes a query, it cannot be copied and pasted into the Events search bar and have the aggregate be included. You must click the magnifying glass to view the entire event list with aggregate included.


Create a new Incident

If the results of an event search are something you would like to further investigate, or send to TRAC, you can create a new incident directly from the Events page. Select Incident > New Incident. This displays the New Incident window where you fill-in the desired fields, and select Create to complete. 

Create a Trigger

To receive an alert the next time an event matching your search parameters comes through the console, create a Trigger. Select Trigger within the events page. This takes you to the Alert > Trigger page where you fill-in desired fields, and select Save to complete. 

Create a Classifier

If the results of your search parameters are something you would like to modify using metadata, create a classifier. Select Classifier from the Events page. This takes you to the Events > Classifier page where you fill-in desired fields, and select Save to complete. 

Download a CSV

If you select to export a CSV from the main Events page, it downloads the first 10,000 event results. To create a CSV export with a smaller number of events, you can export a CSV during any stage of searching or select checkboxes for an even more defined list. The CSV option exports all events matching your search parameters. 


Alert Inbox

The Alert Inbox is home to all alerts generated by DefenseStorm GRID through triggers, PatternScout, or ThreatMatch.  We provide different alert states to help organize and keep your inbox clean. Each alert links to the log data for contributing events by clicking on either Alert Name or Count. Please note that this takes you away from the Alert Inbox and to a filtered view of the Events page.  

Handling an Alert 

Once an alert is fired, it shows up as New in the Alert Inbox.  During alert investigation,  click the ✓ from the top of the Alert to send it to the Acknowledged folder. This signifies that the alert is no longer new, but has not been handled or completed.  Once the investigation has completed,  set the Alert to a Handled State:

  • Escalated generates an incident ticket, which takes over as the final destination for that alert. (Even though it creates an incident, it can still be marked as False Positive if that is the end result.)
  • False Positive means the trigger that generated the alert shouldn't have fired. Maybe the query needs to be tuned, or the thresholds, or maybe anomaly detection found a deviation that wasn't malicious. 
  • Dismissed is the middle ground, where it's not an incident, but also not a false positive.

Alert Process Flowchart

Shows how an alert fires from a query, is placed in the Alert Inbox, and then you handle it from there through investigation and acknowledgment. 


Triggers

Triggers are used to track events based on query strings. You are notified when a condition is met via the Alert Inbox, Email, Incident creation, or a combination of the three. The faster an alert is received, the faster the threat can be identified and stopped. 

Think of the DefenseStorm GRID as a car. Within the cars computer, there are programed triggers to alert you when something may be wrong. For example, if your seatbelt isn't on, you have low tire pressure, low fuel, etc, you receive a notification. Some of the notifications are loud beeping,  a simple message on your dash, or a message on your dash with a light to draw further attention. Triggers serve the same function. You create a trigger, set an interval, and customize the notification method when your query is matched.

The default view of the Trigger homepage displays Active Triggers of all severity levels. You can filter the list to view paused or deleted triggers, and/or only certain severity levels. 


Implementing Triggers

Triggers can be implemented by either creating one or copying one from the default library provided by DefenseStorm. The library was carefully curated by our own security experts on the TRAC Team to provide useful and efficient triggers for targeted alerting. 

Ways to implement triggers
  1. Copy & modify library trigger 
  2. Create a new trigger

Trigger Options

Whether you decide to customize a trigger from the library or create your own, the following fields are available.

Information

These fields are for your records, information, reporting, compliance, etc. 

  • Name: Choose a meaningful name for quick identification. Append any custom triggers with your company initials. 
  • Description: Displays within main trigger list. Make it a short, simple explanation of why you created the trigger.
  • Severity: How critical this trigger is. Options are None, Low, Medium, or High.

Trigger Conditions

Lets us know what events you want us to look for and how the results display.

  • Query: Conditions to search for.  Include -category:alert in the string to prevent duplicate alerts. For more information on query strings, see Query Syntax.
  • Aggregate: Field to group by.  For example, if you want to create a trigger to fire if a larger number of alerts for a specified username go off, you would enter user_name in this field.
  • Function
    • Count (default):  Number of matching events 
    • Count distinct:  Number of different categories of matching events. 
  • The following fields work best when a numeric field is specified. This allows the functions to work correctly and provide accurate information. For example, you can determine the average time it takes backups to run, the sum of bytes, etc.
    • Average
    • Max
    • Min
    • sum

Schedule and Intervals*

How often you want to be notified. 

Schedule 

Monitor activity only during a specific time period to reduce false positives.

  1. Alerts > Schedules > +
  2. Enter information into New Schedule window > Select Create
  3.  Alerts > Triggers > click desired Trigger > edit
  4.  + Add Schedule > select schedule > Click Add Schedule
  5. Save Trigger

Interval (only for Interval based triggers)

It is recommended to select an interval that reduces noise and alert overload. Once a Trigger is saved, you cannot modify the interval. To edit an interval, clone the trigger, adjust the interval, and delete the old trigger.

  • Example: If you set your trigger with a threshold of 1, an interval of 30 minutes AND
    you have 3 matching events from 1:30 pm to 2:00 pm, you receive a single alert at 2:05 pm with all 3 matching events.  

Tags

Type any desired tags for enhanced searching, reporting, and organizing of triggers. Example tags could be FFIEC, Failed Login, Nancy Smith, etc. This allows you to type in any tag into the search portion of Triggers and pull up the list of all triggers with matching tags.

Notifications

Determines the method and person for notifications. 

  • Notify via email: If you want to receive an email, select this checkbox and insert email addresses of anyone who should receive a notification email. 
  • Create an Incident: Once the checkbox is checked, select an owner to oversee the incident. 

Results Fields to include in Email

Add particular fields you want included in your email.  For details on fields that can be included in the email, see Event Fields.

Policy

Add any desired, related policies for compliance, metrics, and reporting. 


Creating a new trigger

  1. Select Alerts > Triggers.
  2. Click the + at the far right of the screen.
  3. Select trigger type: Every Event, Interval, Anomaly
  4. Complete the trigger form by filling in all desired fields based on the information provided above. 
  5. Select Save to enable the trigger.

Copying a trigger from the library

  1. Select Alerts > Library to display the list of trigger groups.
  2. Click on the desired group to display individual triggers.
  3. Select the checkbox(es) of the triggers you wish to copy to your network.
  4. Click Copy Selected.
  5. Go to Alerts > Triggers and click on the trigger copied to your network.
  6. Review and modify the trigger as desired. Select Save to enable the trigger.

Modifying a trigger

  1. Go to Alerts > Triggers.
  2. Select the trigger you wish to modify, select the pencil icon.
  3. Modify fields as desired. Select Save to enable the modified trigger.

Searching triggers

You can search through triggers to create a more manageable and useful list for reporting or tracking purposes.

  1. Alerts > Triggers.
  2. Filter your triggers as desired using the Status/Severity filters and Search tags.
  3. Select the cloud icon to generate the CSV of filtered triggers.
  4. Save as an Excel sheet and format as desired for reporting.

 

Tickets

This article provides definitions, explanations, and recommendations for how to best utilize all the features available on the Tickets page including a use case walk-through. The features include Incidents, Task Schedules, Tasks, CSV downloads, bulk deletion, due dates,  determining responsibility, and much more. 


IncidentsTask SchedulesTasks
DefinitionNetwork activity that requires additional research and possible action.Reccurring actions that can be assigned to a responsible party.One-time action that can be assigned to a responsible party.
Generated bySystem, an alert, ThreatMatch, PatternScout, or userUserSystem, User
ExampleSpike in Logon and Logoff Events for User: ExampleUserNameMonthly Core Systems Report Review1 instance of the Monthly Core Systems Report Review


Incidents

The default Incidents page shows all currently open incidents. This list can be modified by filtering or searching to create a specified list of incidents that can be download to a CSV for your records.  An incident can be created directly from the Events page to grab suspicious activity, or from within the Tickets > Incidents page.  There are several Incident States that explain what stage of the remediation process the incident is in; see the list below for descriptions. 

Incident States
  • Triage - Incident requires further analysis to determine next steps. This is the default state when an incident is created.
  • Analysis - The incident has been escalated and is undergoing further analysis.
  • Remediation - Negative impacts from the incident have been determined and efforts are underway to resolve any residual damage as well as remedy the root cause.
  • Resolved -  There is still action required (updating or documenting), but the issue has been discovered.
  • Closed - No further action needs to be taken.

Creating an Incident from the Tickets page

  1.  Tickets page > Incidents tab (default) > + blue plus sign on the top right. 
  2. Give your incident a title and an owner. (If you want TRAC to review, select TRAC as the owner.)
  3. Update the severity level to Low, Medium, or High based on your concern. (Note: high indicates a 2-hour response, medium is 12 hours, low/none is next business day.)
  4. Add any Watchers and select your desired notification frequency. 
    1. All notifications 
    2. Only when opened and closed
  5. Enter Due Date.
  6. Description should include the following,
    1. Who:  host or ip address experiencing unusual activity
    2. What: What happened to make you believe this is unusual.
    3. When: What time did this occur, has it stopped, is it ongoing.
    4. Optional Links: Links to any reference documentation.
  7. Select Create to save the incident. Once an incident is created, it is automatically placed in the Triage state and the owners/watchers are notified via email.  For incidents and tasks rated as medium and high, please call the OpsGenie number (251-333-6557) to reach the on-call engineer and inform them of the situation.


Creating an Incident through the Events page 

If a search query reveals events that require investigation, create a new incident directly from the Events page to ensure events are captured.

  1. Select the clipboard icon.  A drop-down displays with recently viewed incidents, and at the very bottom, the option to create a New Incident.
  2. Once you select New Incident, the Create Incident window displays. Fill out fields as described above, and select Create.


Updating an Incident

  1. Tickets > Incidents > click an incident to open for updates. 
  2. Add attachments, links, policies, incident state, owner, or severity.
    1. Links are connections to other incidents.
    2. Files allow uploading of any files that were associated with the investigation.
  3. Add any Watchers and select your desired notification frequency. 
    1. All notifications 
    2. Only when opened and closed
  4. Assign a policy to the incident for tracking and reporting purposes.
  5. Select the pencil icon to edit the description or add a new note.  
  6. To view a summary of all actions, select the Activity Log icon on the left.


Filtered CSV Export of Incidents

You can create a filtered CSV export to view a more manageable list of incidents. Follow the steps below.

  1. Go to Tickets > Incidents (default page).
  2. Filter down your incidents as desired by State, Due Date, Severity, Owner, Created By, Created, or a Title Search.
  3. Select the cloud icon to generate your CSV.
  4. Save as an Excel sheet and format as desired.


Closing Multiple Incidents

  1. Go to Tickets > Incidents (default page)
  2. Select checkboxes of incidents you want to close
  3. Select the box icon from the top right


Tasks

Creating a task reminds you of something that needs to be done. For example: within three days, Bob needs to review the report on failed logins that  generated a task today.  The Task homepage displays all open tasks by default, but you can filter, add, and/or update. For compliance purposes, Tasks cannot be deleted, but they can be set to a Closed or Invalid state. 

How to create a Task

  1. Select Tickets > Tasks > the blue + box on the far right.
  2. Fill out the following fields
    1. Title: Subject of the task (Review failed logins report generated on 4/2/2018)
    2. Owner: Person responsible for completing the task 
    3. Add  Watchers (person assigning the task to be completed) and select your desired notification frequency. 
      1. All notifications 
      2. Only when opened and closed
    4. Due Date: When the task needs to be completed by
    5. Description: What the task is. (Review the failed logins report generated yesterday, 4/2/2018. Looked for any user accounts with suspiciously high numbers, and after-hours attempts.) Make a note here with your findings once its completed.


Task Schedules

Creating a task schedule allows you to schedule something that needs to be done on a reoccurring basis and determine a due date. For example: every week, Sally needs to review the report(s) generated on new user accounts created, and she has 5 days to complete.  The Task Schedule homepage displays all schedules by default, but you can filter, add, and/or update schedules. 

When creating a task schedule, a new task is created and displayed in the Task page when the scheduled timeframe is met. For example, if you create a task schedule for every two weeks, a new task is generated in the Task tab every two weeks. The information put into the task schedule is copied over to the Task tab, so the more information you put in your schedule, the more information you'll have for your task. 

Task Schedule ViewTask View


How to create a task schedule

  1. Tickets > Task Schedule > the blue + box on the far right.
  2. Fill out the following fields
    1. Name: Descriptive name. For example, 'Quarterly Information Security Report to Board'.
    2. Owner: Person responsible for completing the task(s)
    3. Description: What you want done
    4. Add Watchers (person making sure its been completed) and select your desired notification frequency. 
      1. All notifications 
      2. Only when opened and closed
    5. Query: Search query that monitors related events
    6. Schedule: How often should tasks be generated?
    7. Due Date: How many days after the task runs does the responsible party have to complete it?
    8. Policies:  If creating this after leaving the Policy page, the current statement is added by default. You can add additional policies if desired.
    9. Tags: Terms you would use to search and/or organize this schedule by.

CSV Export of Task Schedules

To assist with compliance records, you can create a CSV export to view all your task schedules in a single spreadsheet.  Follow the steps below.

  1. Go to Tickets > Task Schedule (default page).
  2. Click the cloud icon to download.

Use Case for Task Schedules: User account created and enabled

Within the description on the Task Schedule be sure to include how the Task is supposed to be completed, documented, what specific things need to be reviewed, etc. 

Example Scenario 

Sara, a manager, requires Justin to verify that all user accounts created within a month are legitimate.  Sara creates a task for Justin to do the verification and requires him to finish the task in 4 days.  

How the fields would be filled out when creating the task schedule

NameUser Account Created 
OwnerJustin (employee doing the task)
DescriptionVerify that all user accounts created were legitimate and necessary accounts. Verify they were created by authorized users. 
WatcherSara (person who created task schedule)
Queryapp_name:(“microsoft-windows-security-auditing” OR “DefenseStorm Agent”) AND (event_id:4720 OR event_id:624 OR event_id:4722 OR event_id:626) AND NOT category:(“alert”)
ScheduleEvery month, on the 1st
Due Date4 days
PoliciesInternal policy to verify user accounts created.
TagsUser Account created, User Account enabled


Compliance

Cybercompliance has become a hot topic item - especially around banking. The utilization of government guidelines such as FFIEC CAT and 20 Critical Security Controls have become industry standards required for customer confidence. To decrease the required workload for compliance, the DefenseStorm GRID contains policy sections for the FFIEC CAT, 20 Critical Security Controls, and custom policies. While the 20 Critical Security Controls and the custom policies are good options, completing the FFIEC CAT provides a strong foundation for compliance on your entire network. 

The Compliance homepage displays a dashboard with useful graphs and insightful information into your network security and its relation to your policies.

  • Incident Statistics gives information on the number of Active and Triaged incidents, as well as ones immediately closed and the average length of time before they are closed. 
  • Incidents is a customizable donut graph that shows comparison and percentage information for incidents within your specified criteria.


Creating a Policy

To create a policy, select Policies from the top navigation, and choose from one of the following,

  • FFIEC (recommended)
  • Custom Policies (internal policies not included in FFIEC) 
  • Critical Security Controls for Effective Cyber Defense (extra defense)


FFIEC CAT

The FFIEC CAT has two main sections, Risk Profile and Maturity Evaluation. The risk profile section determines your risk (how high, or low). The maturity section determines how ready you are to deal with those risks. While you can complete the FFIEC in any order, the guidelines are written with the Risk Profile being completed first.

Risk Profile

This section of the FFIEC CAT determines your network infrastructure's risk. It is much shorter than the Maturity Evaluation, and the UI results are displayed instantaneously. The results include a raw count of your answers and a DefenseStorm score. The score is an algorithm we created that is applied to each category.

Completing the Risk Profile Assessment

  1. Select Risk Profile to display the overview and begin your assessment.
  2. Select Category 1 > answer all questions. Repeat for categories 2 - 5. 
  3. View your results.
    Results is split into two sections, Count and Score. The profile count is the raw number of answers per category. The score section is where DefenseStorm's expertise comes into play. We have utilized our diverse and expansive cybersecurity experiences, and created a "weight" for each answer based on its risk and applied it to the category.

Maturity Evaluation

The Maturity Evaluation section is more complex and lengthy than the Risk Profile. Here are some helpful tips for completing this portion of the assessment efficiently, accurately, and with the ability to provide stellar reporting. 

Answering Statements

The following options are available for each statement:

  • Yes
  • No
  • Yes (C) - which means not exactly, but you do something else to compensate. This option is best if used in conjunction with adding evidence, as described below.

Every answer in a maturity level has to be YES or YES (C) to be compliant. For example, if you have all baseline marked as YES or YES (C), and then have 4 out of 5 in Evolving, you are considered Baseline compliant.

Evidence

Add proof to your statements. There are three forms of evidence that can be combined or used alone. All methods of evidence can selectively be shown in reports. 

  • Comment 
    Statement relevant information. For example, if you select YES (C), you can explain, "I do x, y, and z instead."
  • Tasks
    Ability to assign a one-time job, or task, to somebody in order to comply with a statement. 
  • Task Schedule
    A reoccurring task necessary to comply with a statement. For example, you can create a task schedule to create a report every quarter.

Adding Evidence

To add evidence, select the statement you want to add evidence for, and then chose to add/edit evidence. From the drop-down, select the type of evidence you would like to add: Comments, Tasks, or Task Schedules. 

  • Comments: Select Comments > insert comments > click Save.
  • Tasks: Select Tasks > enter a title, owner, and description > click Create.
  • Task Schedules: Select Task Schedules > Fill out all relevant information > click Create.
    Name: Your name
    Owner: Responsible party
    Description: What you want done
    Query: Search query that monitors events related to this policy.
    Schedule: When you want it done
    Policies: Current statement is added by default. You can add additional policies if desired.
    Tags: Terms you would use to search for and/or organize. 

Viewing / editing evidence

All evidence can be viewed, edited, and reported on. For auditing purposes, no tasks can be deleted, but they can be closed or marked invalid.

Viewing individual statement evidence

To view all evidence associated with a specific statement, go to Policy > Policies > correct Domain > correct Factor > Statement > Select Evidence from under the statement. This displays a drop down list of all evidence added to the statement. To view specific evidence, click on the line item with the evidence name. 

Viewing Comprehensive Evidence

To view all tasks and task schedules, go to Tickets > Task (or Task Schedules). When you open a task or schedule created through FFIEC, you'll see the correct domain and control added as a policy to the task.

When the time comes for a task schedule to be completed, a task is created and becomes visible in the task page. 


Completing the Maturity Evaluation Assessment

  1. Select Maturity Evaluation to display the overview and begin your assessment in Domain 1.
  2. Answer each statement for all five domains and their subsequent factors. 
  3. View Results.
    Results are separated by domain. To view comprehensive results, generate a Maturity Evaluation report.

Generating FFIEC Reports

A major part of compliance is being able to prove it to external auditors. We assist by providing two different report options, FFIEC Report and the FFIEC Evidence Report.


FFIEC Report

This report provides detailed charts and graphical representations of your FFIEC results as well as an overview of the sections and what the charts represent. This works great for board meetings because it provides a good foundation of information and explanations plus critical security statistics. 


FFIEC Evidence Report

This report is great to show auditors proof of your compliance via added evidence. We know that security is key, and have provided the ability to select which sections are populated with evidence. The level of customization is as granule as providing evidence for a single control or as expansive as evidence for the entire report. 

When you select to download the evidence report, it displays in your download list as a zip file with all evidence added as well as a Word file of the entire report. 

Within the report, it lists all evidence associated with each control. It does not provide overview information, explanations, graphs, or charts. It is a list of evidence. 


Custom and Critical Security Control Policies

The option to create custom policies and to enable policies for the top 20 Critical Controls is available through the UI. These are good options for extra security, or for creating policies that are specific to your office only, and not covered in government regulations.

Create a custom policy

  1. To create a custom policy, click the + sign at the top, right side of the Policies page to open the Add a Policy window. 
  2. Insert Name and Description.
  3. Click Add Policy.
  4. Once the policy is added, the following window displays with the option to add Triggers, Incidents, Documents, and additional Control lines to the Policy.

Activate / Decativate Critical Controls Policy

  1. Go to Policy > Polices.
  2. Scroll down to the Critical Security Controls for Effective Cyber Defense section.
  3. Select the desired policy.
  4. Chose either Active or Inactive status.

Adding a Trigger

  1. Go to Alerts > Triggers
  2. Find and click the desired trigger
  3. Click the pencil icon to edit the trigger.
  4. Scroll down to the Policy section. Click the drop-down arrow and add the desired policies.

Adding an Incident

  1. Go to Tickets > Incidents.
  2. Find and click the desired incident.
  3. Scroll down to the Policy section. Click the drop-down arrow and add the desired policies. 

Adding a Document

  1. Go to Policy > Policies and click the desired policy.
  2. Select Documents to view a list of already added documents and the option to upload new ones.

Adding a Control

  1. Go to Policy > Policies and click the desired policy.
  2. Select Add control, insert a title and description, click Add Control to save addition.

Assets

The Assets page automatically displays all assets detected from data sent to the DefenseStorm GRID based on configuration. As a best practice, an organization should regularly review their untracked assets. 

Setup, Configuration, and Importation of Assets

Importing assets is typically done during the DefenseStorm on-boarding process, but there are several other instances where it may be necessary. For example, infrastructure changes such as upgrading hardware, routers, or switches may create the need to import your asset list. If your list of assets becomes unmanageable due to a high number of untracked assets, it may also be best to re-import.

The Assets portion of DefenseStorm GRID was designed to work with your specific network setup. With that in mind, there are different requirements for static, DHCP, or mixed networks. Most networks are considered mixed. 

Special considerations

  • If your network uses DHCP, DHCP logs should be forwarded to GRID.
  • Hostnames include: 
    • Machine hostname and FQDN
    • DNS hostname and FQDN
    • Aliases and FQDN

Importing your Asset List 

  1. Create an Excel file with all assets, and the following information (depending on your network configuration):
    • Mixed Network = Hostnames, IP address, (MAC address required for DHCP host)
    • Static Network = IP, Hostnames, optional MAC address
    • DHCP Network = Hostnames and MAC address 
  2. Upload Excel list to the DefenseStorm GRID via the UI 
  3. Turn on auto discovery
  4. Clean up the list (make sure all information from your Excel sheet transferred)
  5. Additional configurations
    • Insert CIDR ranges = Static IP blocks  
      To ensure your CIDR range is accurate, you can use an online conversion chart such as IPADDRESSGUIDE
    • *Enable detect Bare MAC Addresses. Only select this option if you are tracking every single MAC address. If you select this option and not all MAC addresses are properly listed, it creates untracked assets for new MAC addresses seen which may result in duplicate assets.

Import Recommendations

While only a few fields are required to track an asset, we highly recommend answering as many fields as possible to reduce the possibility for duplicate assets.  Multiple IP and MAC addresses can be associated with a single asset via a comma-separated list, surrounded by quotes. If using Excel to export to CSV, the extra quotes are added automatically.

The following Asset fields are highly recommended, or required, depending on your network configuration:

  • Owner 
  • Asset Name
  • Asset Importance
  • Asset Tag
  • Asset Hostname

Only include the ID field if you are importing assets that have previously been exported. The id number is from the export to ensure you are updating the existing asset as part of your import, rather than importing a new asset. 

Known Limitations

If the import data contains mixed entries (some are only MAC addresses, and some only have IP addresses), null values must be manually inserted into the CSV, as these are read in pairs during import.

Example with three mixed entries:

(11.11.11.11, null)
(22.22.22.23, null)
(null, de:ad:be:ef)

IP address CSV field: “11.11.11.11,22.22.22.23,,”
MAC address CSV field: “,,de:ad:be:ef”

The following line can be used as the first line of a CSV file, then opened in an editor of your choice to continue inputting data. (note: row spans multiple lines on this document).

Name,ID,Owner,Hostname,IP Address,MAC Address,Importance,Description,OS,Product Vendor,Product Name,Product Version,Server Purpose / Notes

Organization of the Assets Page

After you have uploaded and ensured all assets are tracked, use the information in this section to keep an up to date record of your assets and their activity. 

There are eight sortable columns (from left to right):

  • Importance: Displays as either none, one, two, or three dots. Knowing the importance level of an asset helps our TRAC Team provide better monitoring by ensuring your critical systems are always attended. If you update the importance, open a connect ticket so custom triggers can be updated.
  • Name: When you track your asset, the name entered displays here; this is searchable. If an asset has not been added, it displays as untracked.
  • Heartbeat: Last time data was received from the asset. In order to receive heartbeat data, automatic asset detection must be enabled and an IP address must be listed.
    • Grey: Never
    • Green: Within the last 24 hours
    • Yellow: Within the last 7 days, but not the last 24 hours
    • Red: Within the last 30 days, but not the last 7 days
  • Hostname: The hostname you give the asset, this is searchable. (Machine hostname and FQDN, DNS hostname and FQDN, Aliases and FQDN)
  • IP Address: The IP address of the asset, this is searchable.
  • MAC Address: The MAC address of the asset, this is searchable.
  • Last Seen: This is updated when events are matched to the asset via IP address, MAC address, or hostname. Any untracked assets that have not been seen for 30 or more days, are automatically deactivated. This allows the associated MAC and IP addresses to be available for future use.
  • Events: Takes you to the Events page and only displays events for the selected asset.

In addition to the ability to sort columns, you have several other options when organizing your assets.

  • Asset Settings: Number of assets per page,  auto-detection, and CIDR ranges.
  • Trashcan: Bulk deletion of depreciated or invalid assets.
  • Cloud Download: CSV export of selected assets.
  • Cloud Upload: Upload your assets to the UI via CSV file.
  • Plus: Add a new asset.


Managing Untracked assets

An untracked asset means the asset either has not been listed as tracked, or for Windows Machines, does not have the Windows Agent running, but is still sending data. You have three options with an Untracked Asset:

  • Merge into existing asset: This option is good for employees that have more than one asset sending data to the UI or if an asset has multiple interfaces like WiFi and Ethernet. For example, Bob has a laptop, desktop, and a mobile device.
  • Track this asset: Make your asset official. Give the asset a name, IP, hostname, and all other known information.
  • Create incident from this asset: When a suspicious asset displays on your console, create an incident for the TRAC Team to investigate.

To Track an Asset

An asset must be tracked before any changes or updates can be made. Complete the following steps to track your untracked asset.

  1. Click the dropdown arrow to the right of the Name and then select Track This Asset.
  2. The Add Tracked Asset window displays. Add as much information as possible.
  3. After an asset has been added, click on the asset name to display the Asset Details window where you can view, edit, and delete the asset.

CIDR: Auto-detecting Untracked Assets

Including a CIDR range when adding assets helps let us know what IP Addresses we should expect to see on your network. If you aren't sure what an applicable CIDR range would be, you can use an online converter such as IPADDRESSGUIDE.  By default, we autodetect assets from events with hostnames or with mac-addresses. If we receive a DHCP event that links an IP address to a MAC address, we create an untracked asset with both.

If we receive an event that has an IP address, but no MAC address, we will not auto-detect that asset unless the IP falls within an “Included CIDR Range” that has been configured on the Assets page.

To configure the "Included CIDR Range" 


  1. Click the Asset Settings button located at the top right corner of the Assets page.
  2. Set the Asset Auto-Detection field to 'on' 
  3. Set the Included and/or Excluded CIDR Range fields based on your network
  4. Click 'Add' to save the CIDR range(s).
  5. 'Save' to apply all changes to Asset Settings. This auto-detects events coming in from those IP addresses and marks it as an "untracked" asset. Later, you can manage the untracked assets based on your needs.

Managing your Asset List

After the initial upload of your assets, there are a few best practices to keep your asset list tidy and up to date.

High number of untracked assets

  1. Upload your updated Asset list to the console as described earlier in this article. This allows your console to receive a fresh start and gives you a strong foundation for future asset management.
  2. Consider adding a classifier which excludes assets that are not relevant; such as a guest WiFi network or a test network.

Regular Asset Maintenance

The following steps are best performed on a weekly basis to ensure your asset list is up to date and maintained for optimum efficiency. 

  1. Ensure your CIDR range is accurate. (Assets > Settings)
  2. Verify there are no untracked assets. (Search via Untracked, and see that none display.)
  3. If you do see untracked assets, investigate and determine if it needs to be merged into an existing asset, added as a new tracked asset, or if it needs TRAC Team attention.
  4. If it is a duplicate asset, select to merge the asset.
  5. If it is a new asset that has been added to the network since your initial upload, select to track this asset.
  6. If it needs investigation,
    • Select Create Incident from this Asset, and fill it out as such:
      • Title: Unknown Asset
      • Owner: TRAC 
      • Severity: Low
      • Description: What steps you have already taken to figure out what the asset could be, along with your conclusion.

Creating a Filtered CSV Export of Assets

This gives you the ability to filter down your assets to a useful and manageable list to be exported, saved, formatted, and used for reporting.

  1. Select Assets in the left navigation.
  2. Filter your assets as desired by using Tracked status, Importance, Filter by Tag, or Searching options.
  3. Select the cloud icon to generate a CSV of the filtered assets.
  4. Save as an Excel sheet and format as desired. 

 

PatternScout

DefenseStorm GRID's anomaly detection is called PatternScout.  The PatternScout Engine forms a baseline of activity and alerts on deviations.  When a deviation is detected, an alert is sent to the Alert Inbox for investigation. 

Setup PatternScout via Triggers

Utilizing PatternScout is essential to creating constant, around the clock network monitoring. The more a PatternScout trigger is cultivated, the more efficient it becomes. Use PatternScout by either copying or creating a custom PatternScout trigger.

Copy PatternScout Triggers from the Library

The quickest way to utilize a PatternScout trigger is by copying it from the Trigger library. Once a trigger is copied to your network, it is automatically enabled. 

  1. Go to Alerts > Library > Scroll down to PatternScout
  2. Open PatternScout > check desired triggers
  3. Choose 'Copy Selected'. 
  4. To view or edit the trigger, go to Alerts > Triggers. (See Triggers for additional trigger field descriptions.)

Creating a Custom PatternScout Trigger

In addition to enabling PatternScout Triggers via the Trigger Library, you can also create custom triggers to enhance PatternScout functionality with your network. 

  1. Go to Alerts > Triggers
  2. Select the blue + icon on the top right of the page
  3. Select Anomoly
  4. Fill out fields as desired. (based on descriptions from the Triggers article.)
  5. Select either the Dynamic Threshold or Temporal PatternScout checkbox.
    Dynamic Thresholds are best suited to detect sharp differences in queries.
    Temporal anomaly detection is best used for data that has a wall clock pattern to it, such as employee logons, or normal network traffic. 
  6. Select Save

PatternScout Trigger Library 

The library consists of PatternScout Triggers engineered by our TRAC team to best monitor your network.  You can access the full list by going to the GRID UI > Alerts > Library > scroll to PatternScout > open list of triggers.  The following PatternScout triggers are the favorites among our TRAC Team. 

PatternScout Temporal Anomaly

Description

  This looks for changes in each hostname's temporal usage.

Query  _exists_:hostname AND NOT defensestorm_type:alert AND NOT praesidio_skip_ad:true
Schedule  Runs every 30 minutes
Why its important Verify that all activity is within normal time ranges. If not, investigate. For example, Bob usually logs in at 8 am and logs off at 5 pm. Multiple login attempts to Bob's account at 1 am would fire under this trigger. 


PatternScout Geographic Anomaly – Outbound

Description

This looks for changes in how each country interacts with your network.

Query
  _exists_:ip_dest AND NOT defensestorm_type:alert AND NOT praesidio_skip_ad:true
Schedule  Runs every 30 minutes
Why its important Know what is coming out of your network and where it is going. Ports being talked on, IP Address and port they are talking to - are these legit ports and ip addresses? 


PatternScout Lateral Anomaly – Internal Traffic by Host

Description Reports when internal traffic reported by a host is different.
Query  ip_type_src:private AND ip_type_dest:private AND NOT defensestorm_type:alert AND NOT praesidio_skip_ad:true
ScheduleRuns every 30 minutes
Why its important
Allows you to verify activity based on hostnames.


Handling PatternScout Alerts

When a PatternScout alert displays in the Alert Inbox, there are several different 'States' and ways they can be handled. 

Once an alert is fired, it shows up as New in the Alert Inbox.  During alert investigation,  click the ✓ from the top of the Alert to send it to the Acknowledged folder. This signifies that the alert is no longer new, but has not been handled or completed.  Once the investigation has completed,  set the Alert to a Handled State:

  • Escalated generates an incident ticket, which takes over as the final destination for that alert. (Even though it creates an incident, it can still be marked as False Positive if that is the end result.)
  • Dismissed is the middle ground, where it's not an incident, but also not a false positive.
  • False Positive means the trigger that generated the alert shouldn't have fired. Maybe the query needs to be tuned, or the thresholds, or maybe anomaly detection found a deviation that wasn't malicious. (Be careful marking an alert is a false positive because it affects future anomaly detection.)


Integrations

Cloud security has become an essential part of protecting the modern financial institution from cyberattack. Therefore, DefenseStorm offers security of your cloud resources at no additional charge. 

AWS: CloudTrail and ELB

DefenseStorm watches and alerts on parts of AWS that, based on the shared responsibility model, Amazon expects you to monitor. For example:

  • Operating system
  • Network configurations
  • Applications 
  • Access management 

CloudTrail is an Amazon web service that provides visibility into user activity by recording API calls made on your account and delivers log files to an Amazon S3 bucket.  This information helps you track changes made to your AWS resources and to troubleshoot operational issues. If you are using AWS, it is recommended that CloudTrail be enabled. 

The DefenseStorm GRID ingests ELB access logs that capture detailed information about requests sent to the load balancer. Perform the following steps to setup the AWS features through the DefenseStorm UI.

  1. Go to Settings > Integrations and select the Amazon Web Services icon.
  2. Input your Amazon Web Services Account information and select Create.
  3. Connect CloudTrail to DefenseStorm by selecting the gear icon on CloudTrail and follow the instructions displayed.
  4. Connect your AWS ELB to DefenseStorm by selecting the gear icon on Elastic Load Balancing, and then following the instructions displayed. 

OpenDNS

OpenDNS offers network security by reviewing all of your employee network connections on or off the corporate network. Since DefenseStorm is a layer that can “see everything”, we correlate the events OpenDNS captures when users leave the corporate network with the rest of your corporate network. 

Use of this integration requires an "Insights" or "Platform"  OpenDNS subscription; the "Professional" subscription level is incompatible, due to lacking the log export feature below:

  • Retain logs with Amazon Web Services integration using customer-managed or Cisco-managed S3 bucket (source: https://umbrella.cisco.com/products/packages)
  • To enable the OpenDNS functionality, contact DefenseStorm for assistance.

Office 365

Adding Office 365 services is as easy as adding a cloud app. To avoid creating multiple passwords, the administrator uses their Active Directory (AD) credentials to setup Office 365 integration and ingestion of logs. Integration with Office 365 supports the following activities:

  • File and folder
  • Sharing and access request
  • Synchronization
  • Site administration
  • Exchange mailbox
  • User administration
  • Group administration
  • Application administration
  • Role administration
  • Directory administration
  1. Go to Settings > Integrations > Office 365 icon > and select the option to Add Office 365 Account. Follow instructions displayed.
  2. Once you have selected the link and followed the steps provided by Microsoft, all that’s left is to give your Office 365 account a display name. You are redirected back to the DefenseStorm console, where you see the following:
  3. After the display name is added, a message displays saying that you have successfully integrated with Office 365. Note: Auditing on Exchange Mailboxes is off by default. 


ThreatMatch

Have you ever wondered how the FBI or CIA catch cybercriminals? Well, part of it is through their extensive threat-sharing network. DefenseStorm ThreatMatch utilizes that network and gives you access to threat feeds from various companies and government agencies to scan your network for matching activity.

If you want action taken each time a match is found, you can subscribe to the threat feed. Subscribing to a feed allows you to configure options such as email alerts, incident creation, linking policies, assigning severity, etc. If you chose not to subscribe, all relevant data is still displayed, but no automatic action is taken.

In the screenshot below, the blue icons to the left of the feed name indicate you are subscribed. As you can see, all data is still displayed for all feeds, regardless of subscription.


Once within the Alerts > ThreatMatch page, all collected data is displayed per feed. This page gives you the option to subscribe, configure, and investigate your ThreatMatch data.  For assistance utilizing the feature efficiently, here is a description of the data within each column:

  • On/Off Switch: Blue indicates you are subscribed, transparent is unsubscribed. 
  • Name: Unique identifier provided by creator of the feed.
  • Tags:  Type of information searched for threats.
  • Indicators: Number of possible matches. 
  • Matches: Total count of matches, including the same matches found over multiple days.
  • Unique Matches: Deduplicated count of matches.
    For example, you can have 11 matches, 1 unique match. Which means the 1 unique match was found 11 times. 
  • Last updated: The last time the feed was updated.


Configuring ThreatMatch

To apply configuration settings to the feeds you have subscribed to, click the gear icon towards the top of the ThreatMatch page.  

  • Email: If you chose to receive email alerts, only a single email is sent the day a match is detected. 
  • Policy: Associate a policy and control with Threat Match alert findings. 
  • Severity: Give alerts a severity level.

Creating your own Threat Feed

One of the Threat Match features is the blocking and tracking of indicators of compromise (IOC’s).  An IOC is anything from IP addresses, malicious files, or URL’s. If you find specific activity on your network that is an indicator of possible malicious activity, you can upload it to ThreatMatch to begin tracking.

To upload an IOC into ThreatMatch

  1. Go to Alerts > ThreatMatch > Select the blue plus symbol (+).
  2. Fill-in all relevant fields:
    Name: Unique identifier for the IOC
    Description: For example: Malcode creates and maintains a list of domains that are known to host malware and spyware.
    URL: The URL associated with your IOC
    Tags: Type of information
  3. Once a new threat source has been added, a cloud icon displays to the far right of the row; select the icon.
  4. Fill in all relevant fields:
    Indicators
    There are several things to consider when choosing to upload a file instead of typing in each indicator per line:
    - The files must be .txt (notepad, etc)
    - No combination documents. For example, one document is all domains, one document is all IPs, and one document is all hashes.
    - "domain.com" includes only the one domain, no subdomains.
    Description
    Information you find relevant to the threat sharing source, why you needed to add it is an IOC, etc.
    Threat Source
    Name of the threat source.
    URL
    The URL that needs to be alerted on.
    Date Range
    Due to the rapid rate of change in IP addresses, DefenseStorm recommends that you input a date range for IP addresses, that it look no farther than 3 months ahead. We also recommend that it only go back on month. This date range of back 1 month, forward 3 months provides the best range of coverage. Since domains remain constant, putting a date range for them is not necessary and can be removed.  You can use the Pop-up or manually type in the date.  To remove the date range, make sure there is an indicator, and then highlight the date and delete it.
    Threat Indicator Type
    IP Address
    Domain
    Signature (This is a file hash, like an MD5, SHA1, or SHA256) 
  5. Select Save.


ThreatMatch Classifiers

Sometimes ThreatMatch can find matches that have already been denied by your firewall or appropriate system. To dampen the noise while still receiving email alerts, you can exclude events from ThreatMatch by creating a Classifier.

To create a ThreatMatch Classifier

  1. Go to Events and enter the search query you'd like to exclude. For example: app_name:"Cisco ASA" deny
  2. Select "Create Classifier"
  3. Select the Exclude from ThreatMatch checkbox to prevent any alerts on the query.  See Classifiers for details on other classifier fields.
  4. Select Save.

Classifiers

Classifiers are used to manipulate, or preprocess, events before they get to the UI. For example, you can raise or lower the severity of an event, choose to ignore specific events from ThreatMatch, PatternScout, asset detection, or even drop an event before it reaches the GRID.  To be safe and not miss critical data, we recommend sending all data to the GRID.

Classifiers allow you to modify events or change how they're treated by the GRID. 

  • Exclude: No alerts. Data is still saved and searchable.
  • Change attributes: Alter how data is displayed, to make data more useful. 
  • Drop Events:  Discarded before they get to the GRID.  Be careful, it is not searchable or logged.  WARNING: Dropping events means that the event is not logged within the GRID. It is not searchable and does not trigger alerts.

Creating a classifier 

Classifiers have a large impact on the functionality and efficiency of the GRID. Follow the instructions below to ensure your classifiers are syntactically correct and function properly. 

  1. Events > enter search query. All events matching the query display.
  2. Select the Classifier icon to create a classifier of your query.
  3. Fill in Classifier fields.
    Tag Name: Classifiers should have meaningful, easily identifiable names.
    Query String: [Auto-populated from the event search] Conditions the classifier is set to search on.
    Drop Event:  Warning: If drop event is checked, it does not log the event.  Events that match this classifier are not searchable and do not trigger alerts. (Be careful using this!)
    Exclude from ThreatMatch, PatternScout, or Asset Detection: Stop the system from using the event for threat intelligence, anomaly detection, or asset detection, respectively.
    Attributes to Apply: Select the key (data) you'd like to update, and then add the value you want the key to have.  For multiple changes, click the plus sign.
    For example, a group of events have a category of ‘None’ and the organization wants to categorize these events based on information contained within the event. Key = Category and the Value = Research
  4. Select Save to complete and enable your classifier.
  5. After setting up the classifier, you can search for _exists_:tag_queries to display events that have matched one or more classifiers.

Specifications for Classifiers

  • You must select to either drop, exclude, or apply an attribute to create a classifier.
  • If you create a classifier for ‘WDAP’, it matches all events that contain ‘WDAP’. If you create a classifier NOT ‘WDAP’, it matches events that do not contain ‘WDAP’. This is the opposite of writing NOT or - within command line text.
     

Pausing or deleting a classifier

Once the classifier is created, you have the option to pause, edit, or delete it. 

To edit a classifier

  1. Go to Events > Classifiers
  2. Select the classifier you want to edit
  3. Make desired changes
  4.  Click Save.

To pause a classifier

  1.  Click the power icon and it will toggle from blue to gray. 
    1. Blue - active
    2. Grey - paused
  2. Note: Pausing classifiers may take a short time to take effect.

To delete a classifier

  1. Go to Events > Classifiers
  2. Select the classifier you want to delete
  3. Click Delete. Note: Deleting classifiers may take a short time to take effect.

Query Syntax

The query string used for searching events is composed of a series of terms and operators. A term can be a single word — DefenseStorm or Agent — or a phrase, surrounded by double quotes — "DefenseStorm Agent" — which searches for all the words in the phrase, in the same order. Boolean operators (AND, OR, NOT) allow you to customize the search.

Searching

When performing a search in GRID, a single word or phrase is searched for in all available fields such as app_name, message, etc. Searches are not case sensitive but attention needs to be paid to spacing and required double quotes. Terms containing punctuation, such as 127.0.0.1, bro_dns, or microsoft-windows-search, are not considered a phrase and do not need to be placed in double quotes.

Single fields can be isolated and searched for without searching all of the fields. This is a preferred tactic for more specific searches. Be sure to separate the field and term with a colon(:).

For example, app_name:"DefenseStorm Agent"

A space is permitted between the colon and phrase: app_name: "DefenseStorm Agent" although not preferred. Also, app_name:"DefenseStorm Agent" is the same as app_name:"defensestorm agent" as capitalization is not acknowledged.

Watch for synonyms

Synonyms are not understood by the GRID. People may understand wi fi and "wi fi" and "wifi" and “Wi-Fi” to all be the same thing but the system does not. The query string parser would interpret your query as a search for wi fi as "wi OR fi". This likely is not specific enough for your intended search. Be sure what the query is designed for will in fact answer the intended demand.

Wildcards

Wildcard searches can be run on individual terms, using ? to replace a single character, and * to replace zero or more characters:

bro_d?s  would return events containing bro_dns and bro_dxs (if there were such a thing).

bro* would return events under all bro in app_name including bro_weird, bro_files, bro_ssl, etc. This particular search approach is good for confirming large quantities of events such as a firewall or other appliance traffic.

Boolean Operators

Boolean Operators are used to connect and define the relationship between your search terms and phrases. The three Boolean operators are AND, OR and NOT. 

Use AND to narrow your search: all of your search terms must be present in the retrieved records. AND can also be executed as + or &&.

app_name:"DefenseStorm Agent" AND category:Bad_Stuff AND event_id:911

app_name:"DefenseStorm Agent" +category:Bad_Stuff +event_id:911

app_name:"DefenseStorm Agent" && category:Bad_Stuff && event_id:911
(*note using && requires a space between && and the term whereas using + does not)

 These queries require that events contain all three terms in order to be returned.

Use OR to broaden your search by connecting two or more synonyms. OR can also be executed as a double pipe (||), not to be confused with two lowercase Ls as they look the same.

category:Bad_Stuff OR category:Worse_Stuff

category:Bad_Stuff || category:Worse_Stuff
(*note using || requires a space between || and the term)

This query requires either term in order to be returned.

Use NOT to exclude terms from your search results. NOT can also be executed as an exclamation point (!) or a minus sign (-).

category:Bad_Stuff NOT event_id:911

category:Bad_Stuff !event_id:911

category:Bad_Stuff -event_id:911
(*note using ! or - to denote NOT requires no space between it and the term)

This query requires any event_id fields be returned that are not event_id:911.

 

The familiar operators AND, OR, and NOT (also written &&, || and !) are supported and all variations of these can be combined in a single query. However, the effects of these operators are more complicated than is obvious because of the order of precedence, also known as the order of operations. There are predetermined rules that govern the order of which procedures are performed first in order to evaluate a query. NOT takes precedence over AND, which takes precedence over OR. 

NOT > AND > OR        which is the same as    ! > && > ||

While the + and - only affect the term to the right of the operator, AND and OR can affect the terms to the left and right. If this isn’t confusing enough, we have the added dilemma of remembering the spaces.

Another consideration is the use of parentheses which can ease the confusion with the other operators. Syntax contained within parentheses has the highest priority. 

Grouping

Multiple terms or phrases can be grouped together with parentheses to form sub-queries:

(virus OR malware) AND trojan

Reserved Characters

If you need to use any of the characters which function as operators in your query itself (and not as operators), then you should escape them with a leading backslash. For instance, to search for (1+1)=2, you would need to write your query as \(1\+1\)=2.

The reserved characters are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a syntax error which prevents your query from running.

Empty Query

If the query string is empty or only contains whitespaces the query string is interpreted as a no_docs_query and is expected to return no results.

Regular Expressions

Regex searching is supported but use this method against single fields. Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes ("/"):

name:/joh?n(ath[oa]n)/

The supported regular expression syntax is explained later in this document in Appendix A.


Appendix A: Regular Expression Query

Regular Expression syntax

Regular expression queries are supported in the DefenseStorm GRID, under the Events view search. The Lucene regular expression engine is not Perl-compatible but supports a smaller range of operators. We are not going to explain regular expressions, just the supported operators.

An excellent resources for additional information is https://regexr.com.

Standard Operators

Standard operators are always enable through the DefenseStorm Console. For more detailed information about standard operators, see Standard Query Operators Overview from Microsoft.

Anchoring

Most regular expression engines allow you to match any part of a string. If you want the regexp pattern to start at the beginning of the string or finish at the end of the string, then you have to anchor it specifically, using ^ to indicate the beginning or $ to indicate the end.

Lucene’s patterns are always anchored. The pattern provided must match the entire string. 

For string "abcde":

ab.*  # match
abcd  # no match

Allowed characters

Any Unicode characters may be used in the pattern, but certain characters are reserved and must be escaped. The standard reserved characters are:

. ? + * | { } [ ] ( ) " \

If you enable optional features (see below) then these characters may also be reserved:

# @ & < >  ~

Any reserved character can be escaped with a backslash "\*" including a literal backslash character: "\\"

Additionally, any characters (except double quotes) are interpreted literally when surrounded by double quotes:

john"@smith.com"

Match any character

The period "." character can be used to represent any character. 

For string "abcde":

ab...  # match
a.c.e  # match

One-or-more

The plus sign "+" can be used to repeat the preceding shortest pattern once or more times. 

For string "aaabbb":

a+b+  # match
aa+bb+  # match
a+.+  # match
aa+bbb+  # match

 Zero-or-more

The asterisk "*" can be used to match the preceding shortest pattern zero-or-more times. 

For string "aaabbb":

a*b*  # match
a*b*c*  # match
.*bbb.*  # match
aaa*bbb*  # match 

Zero-or-one

The question mark "?" makes the preceding shortest pattern optional. It matches zero or one times.  When no parentheses (groupings) are used, this affects a single character to the left of it.

For string "aaabbb":

aaa?bbb?  # match
aaaa?bbbb?  # match
.....?.?  # match
aa?bb?  # no match

Min-to-max

Curly brackets "{}" can be used to specify a minimum and (optionally) a maximum number of times the preceding shortest pattern can repeat. The allowed forms are:

{5}  # repeat exactly 5 times
{2,5}  # repeat at least twice and at most 5 times
{2,}  # repeat at least twice

For string "aaabbb":

a{3}b{3}  # match
a{2,4}b{2,4}  # match
a{2,}b{2,}  # match
.{3}.{3}  # match
a{4}b{4}  # no match
a{4,6}b{4,6}  # no match
a{4,}b{4,}  # no match 

Grouping

Parentheses "()" can be used to form sub-patterns. The quantity operators listed above operate on the shortest previous pattern, which can be a group.  

For string "ababab":

(ab)+  # match
ab(ab)+  # match
(..)+  # match
(...)+  # no match
(ab)*  # match
abab(ab)?  # match
ab(ab)?  # no match
(ab){3}  # match
(ab){1,2}  # no match

Alternation

The pipe symbol "|" acts as an OR operator. The match succeeds if the pattern on either the left-hand side OR the right-hand side matches. The alternation applies to the longest pattern, not the shortest. 

For string "aabb":

aabb|bbaa  # match
aacc|bb  # no match
aa(cc|bb)  # match
a+|b+  # no match
a+b+|b+a+  # match
a+(b|c)+  # match

Character classes

Ranges of potential characters may be represented as character classes by enclosing them in square brackets "[]". A leading ^ negates (disallows) the character class. 

The allowed forms are:

[abc]  # 'a' or 'b' or 'c'
[a-c]  # 'a' or 'b' or 'c'
[-abc]  # '-' or 'a' or 'b' or 'c'
[abc\-] # '-' or 'a' or 'b' or 'c'
[^abc]  # any character except 'a' or 'b' or 'c'
[^a-c]  # any character except 'a' or 'b' or 'c'
[^-abc]  # any character except '-' or 'a' or 'b' or 'c'
[^abc\-] # any character except '-' or 'a' or 'b' or 'c'

Note that the dash "-" indicates a range of characters, unless it is the first character or if it is escaped with a backslash.

For string "abcd":

ab[cd]+  # match
[a-d]+  # match
[^a-d]+  # no match