User Manual

Introduction to DefenseStorm

Put simply, DefenseStorm is a network security system for financial institutions. We monitor for suspicious activity, alert you when its found, and work with you to stop any potential attacks and prevent them from reoccurring. Think of a security system for your house, that's what provide for network protection. 

We communicate with your system through the DVM (DefenseStorm virtual machine), or the Windows Agent, depending on your asset type and configuration settings. Once data reaches our cloud, it is pre-processed via classifiers and placed within a datastore. Triggers search through live data to generate an alert, incident, email, or a combination of the three. 

 

Contact Us

There are a few different ways to contact DefenseStorm. You can contact us through Connect, TRAC Team, or Knowledge Center feedback.

  • Connect: Technical issues with the product, DVM, or UI.
  • TRAC Team: Security concerns or potential threats on your network. 
  • Knowledge Center Feedback: Submit general questions on functionality or request additional documentation. 

User Interface

All the great features offered by the DefenseStorm GRID are performed through the user interface. This includes the dashboard and all its features (See Dashboard), as well as Alerts, Policy, Assets, and more! For help logging into the UI, see our Login FAQ.  

Events

Events is an extremely powerful search engine that gives you the ability to investigate log data thoroughly and efficiently. Queries can be as simple or complex as you make them using straightforward search query language. See Events for more detailed information.  

Classifiers

Classifiers allow you to pre-process your data by creating fields, deleting data, and changing field values. See Classifiers for more detailed information. 

Alert Inbox

Alert Inbox is a way to manage the alerts that your triggers generate. It shows useful and actionable information that helps you respond quickly. See Alert Inbox for more detailed information. 

ThreatMatch

ThreatMatch gives you the ability to turn on feeds and use Threat Intelligence Sources to identify risks. See ThreatMatch for more detailed information. 

Tickets

The Tickets section of the dashboard is where you create, monitor, and/or update Incidents. See Tickets for more detailed information.

 

Compliance

Compliance utilizes built-in tools that link your policies to different internal and government guidelines, system alerts, and helps you define, enforce, and report on your security policies. See our Compliance & Policy Reporting  article for more details.  


Assets

Assets allows you to manage devices that are sending data to the DVM.  The Assets page displays all tracked and untracked assets. See Assets for more detailed information. 

TRAC Team

In dynamic co-management with your resources, the cybersecurity and cybercompliance experts on our TRAC Team monitor and manage your security and compliance  24 x 7 x 365 to identify and resolve millions of potential cyber threats. 

onTRAC Services

  • Trigger development and maintenance. Any trigger you create requires a discussion with TRAC to discuss expectations before it is actively monitored. 
  • Classifier development and maintenance
  • Log analysis
  • Threat Match verification/analysis
  • Gap analysis for potential instrumentation sources to improve visibility
  • Incident Management
    • Analyze and Remediate open incidents
    • Assigning to customer when further action is necessary

Additional Services

  • Virtual CISO: leverage strategic expertise of a Chief Information Security Officer without investing in full-time staff.
  • Vulnerability Management: stay ahead of potential and emerging threats across both your virtual and physical assets and properties.

Unavailable Services

Our TRAC Team can only provide recommendations for network changes, not execution. The following services cannot be completed by our team:

  • Add blocks to your Firewall
  • Block traffic 
  • Management of any device or appliance 
  • Create user accounts

What do we need from you?

 To efficiently monitor and analyze your network, provide the following:

  • Asset List
  • Network diagrams
  • Technical security controls inventory (NAV, AV, etc.)
  • User List (phone & email)
  • Security Policies
  • Incident Response Plan
  • Privileged Account Management
  • Third Party Vendor Management


Service Level Agreements (SLA’S)

The timelines listed in the table below are the maximum response times for our TRAC Team to initiate triage.

Guardian Security Monitoring Services SLA - Triage

System Generated Actionable Item

Severity

Response Time

System Incidents

High

2 hours

Medium

12 hours

Low

Next Biz Day

None

N/A


Contact for Escalation Cases

In order to streamline the triage process, provide our TRAC Team with who to contact and when, by filling out the escalation use case chart. Below is an example of the chart.



The DefenseStorm Virtual Machine

The DefenseStorm Virtual Machine (DVM) integrates your network with the cloud, via outbound HTTPS (port 443), so we can monitor network activity. The DVM accepts Syslog (both formatted and unformatted) for transferring data. 

Have you Deployed the DVM Before?

Deploying the DVM allows logs to be sent from your network to the DefenseStorm GRID for monitoring. After the initial deployment, you are notified via Release Notes when an update is recommended. 

If you have already deployed the dvm and need to upgrade, see the Upgrading section. If you are unsure if an upgrade is recommended, in the main menu of the dvm, select option 8: Get DVM Status and compare your version to the latest version listed here

Deploying

If you have never installed the DefenseStorm virtual machine, follow these steps based on your virtual machine environment. The DefenseStorm Virtual Machine (DVM) image is available for:

  • VMware
  • Hyper-V

VMware

Obtain detailed OVA information directly from VMWare: https://www.vmware.com/support/developer/studio/studio20/va_user.pdf

How to Install DVM VMware Image
  1. Download the OVA
  2. Deploy the OVA image to VSphere/ESXi
  3. Power on the DefenseStorm Virtual Machine
  4. Open the Console to begin configuration
  5. Scroll down to Configuring the DVM, and follow the instructions.


Hyper-V

This section details the minimum recommended specifications for Hyper-V host servers to perform efficiently. We recommend 2012 R2 and above. According to Microsoft, the end of life dates are as follows:

  • Server 2012 R2 end of life is January 2020
  • Server 2016 end of life estimated as 2025-2026
Server Core install options

As an alternative to a paid option, Hyper-V Server can be installed on a host to enable services on a headless server. This is a Server Core-based OS, and is command-line.

Images are provided by Microsoft at the following URLs:

For all other SKUs, the Server Core install option can be used to further reduce the resources used by the host (reduces host OS storage footprint by ~4 GB) as long as no other roles are active on the Windows host; however, the VMs must be managed remotely or through PowerShell if this option is chosen. If this option is used, ensure that remote management is set up as part of provisioning the Hyper-V host.

Windows Server Minimum Recommended Specs

These specifications assume that only remote management and the Hyper-V Server roles are enabled on the Windows Server install. Using this host for additional roles and/or services may require additional resources depending on which roles are enabled.

OS SKU (select one): Hyper-V Server, Server Standard, Server Datacenter, Server Enterprise

CPU: multi-core, 64-bit CPU

  • must support virtualization and DEP (data execution prevention)

RAM: 4 GB

  • 2 GB RAM for hyper-v host, 1-2 GB for DVM

Disk size: 100 GB.

  • Recommend sizing this higher if possible (200GB+) to allow for VM snapshotting and account for troubleshooting scenarios where DVM reprovisioning alongside an existing copy is necessary.
  • 100 GB breakdown: minimum 32 GB for Windows install + 28 GB space for windows updates, 20 GB for DVM, and 30 GB free space for VM image upgrades.

Network adapter: Gigabit ethernet network adapter

  • At least 10 Mb/s peak outbound network bandwidth to internet (Spikes of high event volume may require higher peak upload bandwidth to avoid queueing event data on the DVM).
How to Install the DVM Hyper-V Image
  1. Download the zip file
  2. Deploy the Hyper-V image to the Windows Server Host.
  3. Power on the DefenseStorm Virtual Machine.
  4. Open the Console to begin configuration.
  5. Scroll down to Configuring the DVM, and follow the instructions.

Configuring the DVM

After you have installed the DVM via VMware or Hyper-V, follow these instructions to configure your DVM settings.

  1. Log into the DVM
    username: ds / password: defensestorm 
  2. Change the password (forced by system)
  3. Configure Time Zone
    Select option 5 and then answer the questions regarding your local timezone.
  4. Configure Networking
    Selection option 4,  
    • Choose to enable DHCP
    • Set the static IP address.
    • Set the Netmask 
    • Set the Gateway
    • Set the Nameserver 1,2, and 3. If there is no Nameserver 2 or 3, then leave the field blank and select 'OK'.
  5. Set DefenseStorm Credentials
    Select option 1, and then use the instructions below to answer the questions.
    • Input your Administrator email address and password.  Note: These credentials are used for one-time authorization of the DVM and are not stored.
  6. Verify Network Connectivity
    Select option 6: Troubleshooting, then option 1: Connectivity Tests

    Once the network tests have passed, you see the following message:

  7. Verify that the DVM is sending messages to the Console by verifying that the API key displayed through your DVM menu and the web console are the same. Within your DVM, select option 8: Get DVM Status to view your API key.
  8. After you have seen the api key through your DVM, open your DefenseStorm UI to verify that associated events are coming through. Go to Events, and select the API key from the filters drop-down list.


Enabling DVM Event Compression

The DVM supports compression to help with limited egress bandwidth. Compression also helps prevent events from being dropped during ingestion. Sign into the DVM and enter the following information. 

  1. Select option (10) Bash Shell

  2. type  sudo vi /etc/praesidio/praesidio.conf   (Enter DVM logon credentials if prompted)
  3. After config file is displayed, do the following:

    Type this Command

    Action (hit)

    Explanation

    /Flush

    Enter

    With cursor on the /Flush line, type the next command.

    o


    Creates new line for typing

    Compress=True

    Enter, ESC

    This adds the command to the file and exits edit mode.

    :wq

    Hit Enter

    Saves the file and exits the vi

  4. Restart syslog-ng service with the new configuration file upon display of the command prompt.
    Type this Command
    Action
    sudo service syslog-ng reload
    hit enter
    exit
    hit enter

If syslog restarts without error, the compression feature has been successfully enabled. 


Upgrading

Upgrading your DVM keeps your network protection up to date with the best features and enhancements. To determine if your DVM is eligible for upgrade, select Option 8: Get DVM Status, and view your version number. If your version number does not display, please open a support ticket for assistance. If a version number displays, complete the steps below.  

Pre-Upgrade

There are a few steps required to ensure that your DVM environment is ready for a successful upgrade. 

  • Backup critical files
  • Increase max_connections

Which files need to be backed up?

To prevent any unintended alterations to configuration files, we recommend that you back them up prior to upgrade, and restore them once the upgrade has completed.

As listed in the table, some files may not be present on your network, therefore, they do not need to be backed up.

Configuration File

Description

/etc/praesidio/praesidio.conf

This is the DVM's primary configuration file, and contains the provisioned DVM API key as seen in the web console.

/etc/syslog-ng/conf.d/praesidio.conf

This is the syslog-ng configuration file, generated during DVM servicing by the pConfig script.

--Optional Files--

--The below files may not exist unless customized--

/etc/syslog-ng/conf.d/snmp.conf

/etc/default/snmpd

SNMP configuration files, generated / modified as part of DVM SNMP configuration. (See connect article for details)

/lib/ufw/user.rules

/lib/ufw/user6.rules

User-generated firewall rules files for IPv4 and IPv6.  This is modified during setup of SSH and SNMP, among other protocols.

 

How do back them up through the DVM Menu

The following steps create a folder called "dvm_yyyymmdd" in the ds user's home directory on the DVM; then backup the configurations listed in the table above to the folder. 

In the DVM menu, select option (10) Bash Shell, then do the following:

Type this Command

Action (hit)

Explanation

cd ~

Enter

Navigate to the current user's home directory.

mkdir dvm_20171025

Enter

Make new backup folder (change date portion to current date)

cd dvm_20171025

Enter

Navigate into new backup folder.

sudo cp /etc/praesidio/praesidio.conf .

Enter

Copy the /etc/praesidio/praesidio.conf file to here (the backup folder).

Enter DVM login password if prompted.

 

sudo cp /etc/syslog-ng/conf.d/praesidio.conf .

Enter

Copy the /etc/syslog-ng/conf.d/praesidio.conf file to here (the backup folder).

-- optional section --

 

 --Ignore any copy failures for the below files; may not exist.--

sudo cp /etc/syslog-ng/conf.d/snmp.conf .

Enter

Copy the /etc/syslog-ng/conf.d/snmp.conf file to here (the backup folder).

sudo cp /etc/default/snmpd .

Enter

Copy the /etc/default/snmpd file to here (the backup folder).

sudo cp /lib/ufw/user.rules .

Enter

Copy the /lib/ufw/user.rules file to here (the backup folder).

sudo cp /lib/ufw/user6.rules .

Enter

Copy the /lib/ufw/user6.rules file to here (the backup folder).


Increasing max_connections using Bash Shell

If your host counts exceed 100 (for linux / appliances) or 500 (for NXLog server installs, or Windows Agent installs on Windows workstations / laptops), we recommend increasing the max_connections option on your DVM.

The default ports, connection counts, and port uses are described below:

Port

Max Conn. Count

Port Description

Host Type

TCP 514

100

Standard RFC-compliant Syslog port.

Unix / Linux, appliances

TCP 516

100

Non-strict syslog port.  This is used for devices that send events over syslog, but whose formats do not comply with the RFC format.  Cisco Meraki devices are an example of this.

Appliances (non-compliant)

TCP 601

500

Syslog port used by Windows NXLog clients.

Windows

TCP 1602

500

Syslog port used by the DefenseStorm Windows Agent.

Windows

 

If your host counts exceed 100 (for linux / appliances) or 500 (for NXLog server installs, or Windows Agent installs on Windows workstations / laptops), you should modify the following section to increase the number of maximum connections.  The default CPU and RAM amounts provisioned on the DVM image can support raising these counts up to 1500; if you need more concurrent connections than this, we suggest increasing the resources available to the VM instance first. 

Steps to open the file, navigate to the configuration file section, and change the values below. Select option 10: Bash Shell through the DVM Main Menu, and perform the following command steps: 

Type this Command

Action (hit)

Explanation

cd /etc/praesidio/

Enter

Navigate to the DVM configuration directory.

sudo vi praesidio.conf

Enter

Open the praesidio.conf file in Vi.  Provide DVM login password if prompted.

-- repeat next 5 steps for each count to change --



/tcp###  (e.g. /tcp514)

Enter

Moves the edit cursor to the line for tcp### (if editing TCP514 counts, type /tcp514)

ww

 

Move the cursor two words to the right.  Should be under the count number (100, 500) at this point.

dw

 

Deletes the existing count number.

a

 

Typing mode (append): moves cursor to character right.

(type your count number here)

ESC

Input your new max connection count digits here.  This adds the count to the file and exits edit mode.

-- end repeat section --



:wq

Enter

Saves the file and exits the vi


If you make a mistake and need to revert all changes and restart from the original file, type the following sequence to quit without saving.

Type this Command

Action (hit)

Explanation


ESC

Exit any edit modes

:q!

Enter

Quit without saving.


How to Upgrade your DVM

Once you have backed up all required configuration files and increased the max_connections if necessary, follow the steps listed below to upgrade your DVM. 

  1.  Access your DVM main menu.
  2.  Select Option 7: Update/Upgrade DVM.
  3. Input your DVM console user (ds user) and password when prompted. 
  4. Text scrolls by on the screen during the upgrade process. Always accept the default options.
    Do you want to overwrite praesidio file, default is No, select this option. Deleting this file deletes all previous configurations and files.
  5. Once the upgrade is finished, login to your DefenseStorm UI as usual. 

Dashboards

Once you login to the GRID UI, the first screen you see is a Default Dashboard. Our experts have created this dashboard to provide information useful to all customers regardless of size or devices. While this was chosen by our experts, feel free to edit this dashboard to further fit your needs, or you can copy it to create a new version to edit. For a video walkthrough of this feature, see our Release Video.

Quick Facts

  • Last dashboard accessed is the one that displays upon next login.
  • Selecting Dashboards allows you to create, edit, clone, or delete dashboards.
  • Change the duration by selecting the drop-down arrow next to During.
  • Clicking on the name of a chart takes you to the charts page where the chart itself can be edited. 


Creating a Dashboard

A custom dashboard can be created for any purpose. You can create a dashboard for firewall monitoring, board-level monitoring, or overview statistics. 

  1. Select Dashboards
  2. Click the + sign to create a new Dashboard.
  3. Give the dashboard a name, description, choose the layout, and then add charts.
    *Note: If the chart has not already been created, go to Reports > Charts to create it; then come back and add it. 
  4. Save.


Editing a Dashboard

 If your desired changes are to the charts themselves,  go to Reports > Charts and edit from there.

  1. Select Dashboards from the top of the screen.
  2. Click the name of the dashboard you want to edit.
  3. Make any desired changes.
  4. Click Save.

Cloning a Dashboard

Cloning a dashboard gives you the ability to edit a copy without disturbing the original. 

  1. Select Dashboards from the top of the screen.
  2. Hover over the name of the dashboard you want to clone, and to the far right, the option to clone displays.
  3. Make any desired changes.
  4. Click Save.

Deleting a Dashboard

This does not delete the charts themselves, only the dashboard.

  1. Select Dashboards from the top of the screen.
  2. Click the checkbox of the dashboard you want to delete.
  3. Click the trashcan icon.


Creating your Profile

Creating your profile through the DefenseStorm UI couldn’t be easier. 

  1. Click on the Settings logo (the gear icon) at the bottom left of your screen
  2. Select the Profile tab. This opens the Edit Profile window where you can fill in all desired information. For information on two-factor authentication, see  Two-Factor Authentication

Create a New User

  1. As a user with DefenseStorm admin privileges, log into the UI
  2. Select Settings page (gear icon) from the bottom left.
  3. Click on the + icon from the upper right side of the screen.
  4.  Complete the user form with desired information and click Save. After the user is created, they receive an email with simple instructions on how to complete setup.
     The user roles are:
    • Administrator 
    • Power User
    • User
    • Read-Only User



Two-Factor Authentication

During installation, two-factor authentication is automatically set up to provide an extra layer of security against unauthorized access. 

Users with DefenseStorm Admin permissions can reset the 2FA settings by selecting Settings > Users > 2FA column > Reset 2FA. If you only have one DefenseStorm Admin and their account requires reset, call DefenseStorm for assistance. To eliminate the need to contact support, have two DefenseStorm Admins so they can reset each others 2FA if necessary.  

Setting up two-factor authentication

To set up two-factor authentication as an individual user:

  1. Go to Settings > Profile and select Set Up.
    Yy17DU6fnGU_IGIHVFU9rpOqcJJYetKmUA.png
  2. Select device type (Android, Blackberry, iOS, Windows Phone). While the UI mentions Google Authenticator, Authy, and Salesforce are two common alternative 2FA applications. 
  3. After completing the instructions for your device, select Enter Generated Code, and enter code generated by your chosen authentication app. *YubiKey Note: To use it with the GRID, go to Settings > Profile > turn on U2F

  4. Select Verify to enable 2FA. It will be required upon next login. 

Using two-factor authentication

  1. Go to the DefenseStorm UI. 
  2. Enter your username and password. Verification code prompt displays.
  3. Open your authenticator app and enter the 6 to 8 digit verification code associated with your DefenseStorm account. 
  4. Once the code is entered correctly, you are fully logged into the system. 


Reports

DefenseStorm Reports 

When creating a report within the DefenseStorm UI, there are 3 main steps: 

  1. Create a chart 
  2. Organize a template 
  3. Generate a report 

By default the Reports homepage displays all generated reports. You can utilize the filter option to view reports generated from specific templates, and/or status' (active or deleted reports). Hovering over a report name displays options to either download or delete the report from the UI. 


Creating a DefenseStorm Report

To create a Report, the first step is to create a chart. As soon as a new chart is created, it begins gathering applicable data from the previous 90 days. This gathering of data is strictly behind the scenes, and does not impact GRID performance. 

Once you are satisfied with the charts you've created, organize them into templates.  When putting together your templates, add all desired charts with headings and descriptions specific to the template's purpose. For example, if you create an overview template, the charts and descriptions would be high-level, and as simple as possible. If you're creating an in-depth template, while utilizing some of the same charts, they would have more detailed descriptions, with additional heading 2 charts. 

After you have finalized your templates, determine the report frequency. Reports can be generated as a one-time occurrence or on a schedule.  If you select to generate the report on a schedule, you can add email addresses to receive a link once the report has been generated.


1. Creating a Chart

Charts use search queries to capture data. The process of creating a chart can be started from the Events page or the Reports page.

How to create a chart via Events page

If you are searching through events and want more information on the trends and statistics for this occurrence, select to create a chart directly from the Events screen.

  1. Within the Events page, enter search query. It must be a search query, event results based solely on filters do not apply. 
  2. Once desired results display, select the icon to create a chart.  

  3. The Reports > Chart page opens providing a chart view of your queried data.
  4. If you wish to use this chart for reporting, you may go to Step 2 of the next section, "How to create a chart via Reports page".  If you do not want to save the chart, select Cancel. 

How to create a chart via Reports page

  1. Select Reports > Charts > blue + icon
  2. Enter required and desired chart information. * indicates  a required field
    Query*: Data being pulled from the GRID. You can enter a new query, or a saved search.
    Name*: What the chart is called.
    Chart Type: Default is a line graph.
    X-Asis Title: Chosen title for the horizontal axis of the graph.
    Y-Axis Title: Chosen title for the vertical axis of the graph.
  3. Select Save.
Example: Failed Admin Logons
  • Query*:  app_name:("microsoft-windows-security-auditing" OR "DefenseStorm Agent") AND (event_id:4625) AND (account_name:"Administrator" OR target_user_name:"Administrator") AND NOT defensestorm_type:alert
  • Name*:  Failed Admin Logins 
  • Chart Type:  Line
  • X-Asis Title:  Time
  • Y-Axis Title:  Count


2. Creating a Template

Templates is where you choose, organize, and give context for the charts in your report. You can create multiple templates with a variation of charts. 

How to create a template

  1. Select Reports > Templates > blue + icon
  2. Enter template title and description.
  3. Select Apply.
  4.  Select 'Add a new fragment here' to add a chart.
  5. Select chart from the drop-down in the Edit Fragment window.
  6. Create a Heading name and Heading Type. Chose either Heading 1 or Heading 2.  Heading 2 is a good option for a secondary, or sub-chart.
  7. Enter a paragraph description. What makes it relevant to your report? The same chart could have different descriptions based on the report audience and timeframe.
  8. Select Apply.  Repeat steps 4-8 to add additional charts.
  9. Click Save. The template is created and ready to be generated as a report. 
Example: Utilizing Heading 1 & Heading 2 when creating a template

For example, one of your boss' wants incredibly detailed information, while the other wants general information. You would create a template consisting of multiple heading 1 and heading 2 charts to provide in-depth information; and another template with a variety of heading 1 charts that show general information.


3. Generating a Report

Since charts and templates are created prior to generating a report, this is the quickest and easiest step in the process.  All you have to decide is if you want the report generated once, or on a schedule. 

How to generate a report 

  1. Select Reports > Templates to view all templates. 
  2. Search or scroll to find the template you wish to generate.
  3. Chose from the options to the right of the template name: Generate New Report, View Previously Generated Reports, Schedule, Clone, or Delete.
  4. Selecting to Generate a New Report > select the date range > Select Create.
  5. Previously generated report takes you to a filtered version of the Reports page.
  6. Schedule a report > add email addresses > determine frequency > Create.
    Email addresses added here receive a URL to download the report once its completed. 
  7. Cloning the template creates an exact copy of the template to allow quick, minor adjustments. 
  8. Deleting the template removes it from the list so future reports cannot be generated. No charts or data is affected by deleting a template. All previously generated reports are still visible.

On-Demand Cybersecurity Report

The Cybersecurity Report is a default report created by the experts at DefenseStorm to make presenting information quick and simple. 

The following information is provided in each report,

  • Incidents
  • Opened Incident Severity Breakdown
  • Most Active Incidents
  • Alerts
  • Most Fired Alerts
  • Events
  • Events by Hour (daily report)
  • Events by Day (weekly report)
  • Events by Date (monthly report)

Generating the On-Demand Report

  1. Go to the Reports page
  2. Select +  to open the Create Custom Report window
  3. Choose 'On-demand Report' from the Template drop-down
  4. Enter any email addresses you want the report sent to
  5. Determine Date Range
  6. Click Create


Events

The Events page is a powerful search engine to investigate network activity. The DefenseStorm GRID displays any activity that generates a log as an event. This includes activity generated from your network, or our GRID. For example, when someone attempts to login to their account, you receive an event; and when a DefenseStorm trigger fires, you also receive an event. Your search queries can be as simple or complex as desired by using any one or a combination of the search methods explained in this article. 

Types of Events

There are three types of events that display in the UI: log, alert, and system events. 

  • Log events are generated through the DVM each time an activity providing log information is performed on your network. 
  • An alert event, or alert derivative, is generated each time an alert is triggered. 
  • System events are created each time an action is performed on the DefenseStorm GRID side.  


Log Events

The DVM takes all logs generated from network activity and creates a searchable event that is displayed through the console. These events can be used to create incidents, alerts, triggers, and charts. For a list of the most common and relevant event fields, see our FAQ, Event Fields.

Alert Derivatives (alert events)

The idea behind alert derivatives is to allow alerts to be searched for, alerted on, and monitored through the events page. This event type is a composite of all events that triggered the alert.

For example, if we had a query for geo_dest.country:china, and two users, Angela and Andrew had connected to a Chinese server in a given hour, the alert event that matched geo_dest.country:china would have both hostname:"John's PC" and hostname:"Angela's PC".   A search in the console for hostname:"Angela's PC" geo_dest.country:china would return both the event that caused the alert to fire and the event for the alert.

Using Alert Derivatives

The alert derivatives functionality creates a centralized and simple way to view alerts through the addition of the alert event. When searching events, alert events can easily be included or excluded by using defensestorm_type:alert. 


Excluding alert events from queries

To exclude Alert Events from your search query, include the following flag:  

-defensestorm_type:alert

Alert Event Fields

In addition to all log event fields, alert events also contain the following:

  • timestamp - the beginning of the interval that we alerted upon
  • defensestorm_type - always "alert"
  • email - the email addresses that were informed of this alert
  • trigger_name - the user-friendly name of the trigger
  • trigger_id - the internal GUID of the trigger
  • trigger_query - the query string for the trigger
  • trigger_interval - the number of seconds in each trigger interval
  • severity - high/medium/low, copied from the trigger
  • event_count - number of hits in the search that fired the trigger

The following Information (found in the original log events) will not be duplicated to the alert event:

  • message
  • timestamp
  • ingest_timestamp
  • event_count
  • unparsed_expanded_message
  • unparsed_message
  • raw_message
  • praesidio_parse_nanos
  • percolator_tag

System Events

In addition to collecting log data for activity on your network, we also have log data for activity performed by the DefenseStorm GRID.  For example, when a new asset is created, or when a new task schedule is created.  

USE CASE 1 : Joe wants an alert when a new Asset is updated by the system. For example, when a new IP address is assigned to a new asset.

 Query:  app_name:"DefenseStorm Audit" category:asset action:"assignedIp"
Trigger Threshold: 1

Searching Events

DefenseStorm provides several different methods for searching through events. These search methods can either be used on their own or combined together for the most specified results. The methods of searching are:

  • Query Syntax
  • Time frame
  • Event spike
  • Filter options
  • Aggregation

Query Syntax 

The DefenseStorm Events page follows the format of ElasticSearch  “Query String Queries”.  For details regarding Query Syntax, see the article, DefenseStorm Query Syntax.  For help, select the question mark icon to the right of the search bar.

Time Frame

The DefenseStorm Events page provides a calendar to narrow down events by time frame. If you want all events for a specific day, select During drop-down to update the calendar. The calendar also allows you to select a smaller time frame, such as Last Hour, or Last 3 Hours. 

 

Event Spike

Within the Events page, it displays a graph with all ingested log events. If you see suspicious activity through the graph, you can drag the bars to display only events associated with the activity spike. 

 

Filter Options

Within the DefenseStorm Events page,  search through events by filtering. To search by filtering, drill down through the filters and select the desired checkboxes to display only associated events.

 

Aggregation

The Aggregate feature allows you to group your events before searching to greatly reduce the number of events displayed; therefore increasing result speed and accuracy.

  1. Enter an Aggregate field.
  2. In the screenshot below, the events have been grouped by the primary aggregate field: account_domain.

    From here you can either select one of the displayed account domains, or add a secondary field to further narrow down the displayed domain events.


    The default view when both the primary and secondary fields are utilized, is the Spreadsheet View (as shown above). To simplify your view, select Compound from the View drop-down list. The Compound view allows you to better understand the relationship between the values of your two aggregate fields by displaying them side-by-side with their associated event count.
  3. Select the Desired result to view events
    Once you click on the desired compound result, in this example, account_domain:postilionoffice and user_domain:postilionoffice, a filter is applied to all events and the 56 events display. 

Saving a search

Saving your search allows you to reuse all methods of searching without having to fill in the fields. For example, if you used a combination of Aggregation and Query Syntax, saving the search allows you to select the search without having to reapply the primary/second aggregate fields and the search query. The option to save your search is in the top right of the event graph. 


To view all previously saved searches, select Saved Searches from the top navigation of the Events page. 

The Saved Searches page displays the name of your search, the date it was created, and gives you the option to edit your search. Selecting a search takes you to the Events page and applies the saved search parameters.

Create a new Incident

If the results of an event search are something you would like to further investigate, or send to TRAC, you can create a new incident directly from the Events page. Select Incident > New Incident. This displays the New Incident window where you fill-in the desired fields, and select Create to complete. 


Create a Trigger

To receive an alert the next time an event matching your search parameters comes through the console, create a Trigger. Select Trigger within the events page. This takes you to the Alert > Trigger page where you fill-in desired fields, and select Save to complete. 


Create a Classifier

If the results of your search parameters are something you would like to modify using metadata, create a classifier. Select Classifier from the Events page. This takes you to the Events > Classifier page where you fill-in desired fields, and select Save to complete. 


Download a CSV

If you select to export a CSV from the main Events page, it downloads the first 10,000 event results. To create a CSV export with a smaller number of events, you can export a CSV during any stage of searching or select checkboxes for an even more defined list. The CSV option exports all events matching your search parameters. 


Alert Inbox

Alert Inbox is home to all alerts generated by DefenseStorm GRID through triggers, PatternScout, or ThreatMatch.  We provide different alert states to help organize and keep your inbox clean.  

Once an alert is fired, it shows up as New in the Alert Inbox.  During alert investigation,  click the ✓ from the top of the Alert to send it to the Acknowledged folder. This signifies that the alert is no longer new, but has not been handled or completed.  Once the investigation has completed,  set the Alert to a Handled State:

  • Escalated generates an incident ticket, which takes over as the final destination for that alert. (Even though it creates an incident, it can still be marked as False Positive if that is the end result.)
  • False Positive means the trigger that generated the alert shouldn't have fired. Maybe the query needs to be tuned, or the thresholds, or maybe anomaly detection found a deviation that wasn't malicious. (Be careful marking an alert is a false positive because it affects future anomaly detection.)
  • Dismissed is the middle ground, where it's not an incident, but also not a false positive.


Viewing Associated Events

Within the Alert, clicking the Alert name or Count number opens the Events page with all events linked to the alert. This shows you all log data for each individual event that contributed to the GRID firing the alert .  You can also view a chart of the data to see the statistical graph form.  


Triggers

Triggers are used to track events based on query strings. You are notified when a condition is met via the Alert Inbox, Email, Incident creation, or a combination of the three. The faster an alert is received, the faster the threat can be identified and stopped. 

Think of the DefenseStorm GRID as a car. Within the cars computer, there are programed triggers to alert you when something may be wrong. For example, if your seatbelt isn't on, you have low tire pressure, low fuel, etc, you receive a notification. Some of the notifications are loud beeping,  a simple message on your dash, or a message on your dash with a light to draw further attention. Triggers serve the same function. You create a trigger, set an interval, and customize the notification method when your query is matched.

The default view of the Trigger homepage displays Active Triggers of all severity levels. You can filter the list to view paused or deleted triggers, and/or only certain severity levels. 


Implementing Triggers

DefenseStorm has created a default library of triggers that are fully functional and ready to implement. There are two ways to add triggers to your network. 

  1. Modify library trigger: Copy a trigger and edit desired fields. 
  2. Create new trigger: Enter all fields. 

Trigger Options

Whether you decide to customize a default trigger from the library or create your own, the following fields are available.

Information

These fields are for your records, information, reporting, compliance, etc. 

  • Name: Choose a meaningful name for quick identification. Append your custom trigger with your company initials. 
  • Description: Displays within your list of triggers, so make it a short, simple explanation of why you created this trigger.
  • Severity: How critical this trigger is. Options are None, Low, Medium, or High.

Trigger Conditions

Lets us know what events you want us to look for and how the results display.

  • Query: Conditions to search for.  Include -category:alert in the string to prevent duplicate alerts. For more information on query strings, see Query Syntax.
  • Aggregate: Field to group by.  For example, if you want to create a trigger to fire if a larger number of alerts for a specified username go off, you would enter user_name in this field.
  • Function
    • Count (default):  Number of matching events 
    • Count distinct:  Number of different categories of matching events. 
  • The following fields work best when a numeric field is specified. This allows the functions to work correctly and provide accurate information. For example, you can determine the average time it takes backups to run, the sum of bytes, etc.
    • Average
    • Max
    • Min
    • sum

Schedule and Intervals*

How often you want to be notified. 

Schedule 

Monitor activity only during a specific time period to reduce false positives.

  1. Alerts > Schedules > +
  2. Enter information into New Schedule window > Select Create
  3.  Alerts > Triggers > click desired Trigger > edit
  4.  + Schedule > select schedule > Click Add Schedule
  5. Save Trigger
Interval (only for Interval based triggers)

It is recommended to select an interval that reduces noise and alert overload. Once a Trigger is saved, you cannot modify the interval. To edit an interval, clone the trigger, adjust the interval, and delete the old trigger.

  • Example: If you set your trigger with a threshold of 1, an interval of 30 minutes AND
    you have 3 matching events from 1:30 pm to 2:00 pm, you receive a single alert at 2:05 pm with all 3 matching events.  

Tags

Type any desired tags for enhanced searching, reporting, and organizing of triggers. Example tags could be, FFIEC, Failed Login, Nancy Smith, etc. This allows you to type in any tag into the search portion of Triggers and pull up the list of all triggers with matching tags.

Notifications

Determines the method and person for notifications. 

  • Notify via email: If you want to receive an email, select this checkbox and insert email addresses of anyone who should receive a notification email. 
  • Create an Incident: Once the checkbox is checked, select an owner to oversee the incident. 

Results Fields to include in Email

Add particular fields you want included in your email.  For details on fields that can be included in the email, see Event Fields.

Policy

Add any desired, related policies for compliance, metrics, and reporting. 


Creating a new trigger

  1. Select Alerts > Triggers.
  2. Click the + at the far right of the screen.
  3. Select trigger type: Every Event, Interval, Anomaly
  4. Complete the trigger form by filling in all desired fields based on the information provided above. 
  5. Select Save to enable the trigger.

Adding a trigger from the library

  1. Select Alerts > Library to display the list of trigger groups.
  2. Click on the desired group to display individual triggers.
  3. Select the checkbox(es) of the triggers you wish to copy to your network.
  4. Click Copy Selected.
  5. Go to Alerts > Triggers and click on the trigger copied to your network.
  6. Review and modify the trigger as desired. Select Save to enable the trigger.

Modifying a trigger

  1. Go to Alerts > Triggers.
  2. Select the trigger you wish to modify, select the pencil icon.
  3. Modify fields as desired. Select Save to enable the modified trigger.

Searching Triggers

You can search through triggers to create a more manageable and useful list for reporting or tracking purposes.

  1. Alerts > Triggers.
  2. Filter your triggers as desired using the Status/Severity filters and Search tags.
  3. Select the cloud icon to generate the CSV of filtered triggers.
  4. Save as an Excel sheet and format as desired for reporting.

 

Tickets

The Tickets page is used to track Incidents, Tasks, and Task Schedules. Incidents track network activity that requires additional research and possible action. Incidents are either generated by the system, through Alerts, ThreatMatch, PatternScout, or user. Tasks and Task Schedules are used for reminders to review log data, or provide compliance to auditors. Tasks are one-time actions that can be assigned to a responsible party. Task Schedules are reoccurring actions that can also be assigned to a responsible party.  Think of a Task Schedule as a recurring meeting on your calendar, and the Task, as the actual calendar event for one of those meetings.

The Tickets page defaults to the Incidents tab which displays all open Incidents.  Other tab options include Tasks, and Task Schedules.  


Incidents

The default Incidents page shows all currently open incidents. This list can be modified by filtering or searching to create a specified list of Incidents that can then be download to a CSV for your records.  

Creating an Incident from the Tickets page

Incidents can either be created from this page, or the Events Page. Follow the instructions below to create an Incident from within the Incidents page.

  1. Select the Tickets page on the left-hand side of the screen. This defaults you to the Incidents tab. 
  2. Click on the blue plus sign on the top right to display the New Incident window.
  3. Give your incident a title and an owner. (If you want TRAC to review, select TRAC as the owner.)
  4. Update the severity level to Low, Medium, or High based on your concern. (Note: high indicates a 2-hour response, medium is 12 hours, low/none is next business day.)
  5. Description should include the following fields:
    Who:  host or ip address experiencing unusual activity
    What: What happened to make you believe this is unusual.
    When: What time did this occur, has it stopped, is it ongoing.
    Optional Links: Links to any reference documentation.
  6. Select Create to save the incident. Once an incident is created, it is automatically placed in the Triage state and the owners/watchers are notified via email.  For incidents and tasks rated as medium and high, please call the OpsGenie number (251-333-6557) to reach the on-call engineer and inform them of the situation.


Creating an Incident through the Events page 

If a search query reveals events that require investigation, create a new incident directly from the Events page to ensure all concerning events are captured in the Incident.

  1. Select the clipboard icon.  A drop-down displays with recently viewed incidents, and at the very bottom, the option to create a New Incident.
  2. Once you select New Incident, the Create Incident window displays. Fill out fields as described above, and select Create.


Updating an Incident

Within the Tickets > Incidents page, click an Incident to open for updates. From this screen you can add watchers, attach documents, edit search queries, link an incident and/or policy, and add to the activity log through notes. The icons to the left jump to specific incident details. 

  1. From this screen, add attachments, links, policies, watchers, incident state, owner, or severity.
    Incident States are:
    • Triage - Incident requires further analysis to determine next steps. This is the default state when an incident is created.
    • Analysis - The incident has been escalated and is undergoing further analysis.
    • Remediation - Negative impacts from the incident have been determined and efforts are underway to resolve any residual damage as well as remedy the root cause.
    • Resolved -  There is still action required (updating or documenting), but the issue has been discovered.
    • Closed - No further action needs to be taken.
      Other update options
    • Links are connections to other incidents.
    • Files allow uploading of any files that were associated with the investigation.
    • You can assign other watchers or choose to add yourself as a watcher. A watcher is someone being cc'd on an incident so they can be kept up to date on status via email.  If you prefer to not receive emails, you can log into the UI and view updates that way.
    • You can assign a policy to the incident for tracking and reporting purposes.
  2. Select the pencil icon to edit the description or add a new note.  New notes display in the Activity Log section of the Incident with the date/time, and user who made the note.
  3. To view a summary of all actions, select the Activity Log icon on the left.


How to Create a Filtered CSV Export

You can create filtered CSV export of incidents to view a more manageable list of incidents. Follow the steps below:

  1. Go to Tickets > Incidents (default page).
  2. Filter down your incidents as desired by State, Owner, Created By, or a Title Search.
  3. Select the cloud icon to generate your CSV.
  4. Save as an Excel sheet and format as desired.


How to Close Multiple Incidents

  1. Go to Tickets > Incidents (default page).
  2. Select checkboxes of incidents you want to close
  3. Select the Trashcan icon from the top right.


Tasks

Creating a task allows you to be reminded of something that needs to be done. For example: tomorrow, Bob needs to review the report on failed logins generated today.  The Task homepage displays all open tasks by default, but you can filter the tasks displayed, add and/or update Tasks. For compliance purposes, Tasks cannot be deleted, but they can be set to a Closed or Invalid state. 


How to create a Task

  1. Select Tickets > Tasks > the blue + box on the far right.
  2. Fill out the following fields:
    Title: Subject of the task (Review failed logins report generated on 4/2/2018)
    Owner: Person responsible for completing the task (Bob)
    Description: What the task is (Review the failed logins report generated yesterday, 4/2/2018. Look for any user accounts with suspiciously high numbers, and for after-hours attempts. Make a note here with your findings once its completed.)


Task Schedules

Creating a task schedule allows you to be schedule something that needs to be done on a reoccurring basis. For example: every week, Sally needs to review the report(s) generated on new user accounts created.  The Task Schedule homepage displays all schedules by default, but you can filter the schedules displayed, add and/or update schedules. 

When creating a task schedule, a new task is created and displayed in the Task page when the scheduled timeframe is met. For example, if you create a task schedule for every two weeks, a new task is generated in the Task tab every two weeks. The information put into the task schedule is copied over to the Task tab, so the more information you put in your schedule, the more information you'll have for your task. 

Task Schedule View

Shows one-time Tasks and Tasks created from Schedules


How to create a task schedule

  1. To create a task schedule, select Task Schedule > the blue + box on the far right.
    Fill out the following fields:
    Name: Descriptive name. For example, 'Quarterly Information Security Report to Board'.
    Owner: Responsible party
    Description: What you want done
    Query: Search query that monitors related events
    Schedule: How often should tasks be generated?
    Policies:  If creating this after leaving the Policy page, the current statement is added by default. You can add additional policies if desired.
    Tags: Terms you would use to search and/or organize this schedule by.


Use Case for Task Schedules

User Account Created & Enabled

Within the description on the Task Schedule be sure to include how the Task is supposed to be completed, documented, what specific things need to be reviewed, etc. 

Example on how to fill out Task Schedule Fields

NameUser Account created & enabled
OwnerSteve
DescriptionVerify that all user accounts created were legitimate and necessary accounts. Verify they were created by authorized users. 
Queryapp_name:(“microsoft-windows-security-auditing” OR “DefenseStorm Agent”) AND (event_id:4720 OR event_id:624 OR event_id:4722 OR event_id:626) AND NOT category:(“alert”)
ScheduleEvery month, on the 1st
PoliciesInternal policy to verify user accounts created.
TagsUser Account created, User Account enabled


Compliance

Cybercompliance has become a hot topic item - especially around banking. The utilization of government guidelines such as FFIEC CAT and 20 Critical Security Controls have become industry standards required for customer confidence. To decrease the required workload for compliance, the DefenseStorm GRID contains policy sections for the FFIEC CAT, 20 Critical Security Controls, and custom policies. While the 20 Critical Security Controls and the custom policies are good options, completing the FFIEC CAT provides a strong foundation for compliance on your entire network. 

The Compliance homepage displays a dashboard with useful graphs and insightful information into your network security and its relation to your policies.

Incident Statistics gives information on the number of Active and Triaged incidents, as well as ones immediately closed and the average length of time before they are closed. 

Incidents is a customizable donut graph that shows comparison and percentage information for incidents within your specified criteria. 


Creating a Policy

To create a policy, select Policies from the top navigation, and choose from one of the following,

  • FFIEC (recommended)
  • Custom Policies (internal policies not included in FFIEC) 
  • Critical Security Controls for Effective Cyber Defense (extra defense)


FFIEC CAT

The FFIEC CAT has two main sections, Risk Profile and Maturity Evaluation. The risk profile section determines your risk (how high, or low). The maturity section determines how ready you are to deal with those risks. While you can complete the FFIEC in any order, the guidelines are written with the Risk Profile being completed first.

Risk Profile

This section of the FFIEC CAT determines your network infrastructure's risk. It is much shorter than the Maturity Evaluation, and the UI results are displayed instantaneously. The results include a raw count of your answers and a DefenseStorm score. The score is an algorithm we created that is applied to each category.

Completing the Risk Profile Assessment

  1. Select Risk Profile to display the overview and begin your assessment.
  2. Select Category 1 > answer all questions. Repeat for categories 2 - 5. 
  3. View your results.
    Results is split into two sections, Count and Score. The profile count is the raw number of answers per category. The score section is where DefenseStorm's expertise comes into play. We have utilized our diverse and expansive cybersecurity experiences, and created a "weight" for each answer based on its risk and applied it to the category.

Maturity Evaluation

The Maturity Evaluation section is more complex and lengthy than the Risk Profile. Here are some helpful tips for completing this portion of the assessment efficiently, accurately, and with the ability to provide stellar reporting. 

Answering Statements

The following options are available for each statement:

  • Yes
  • No
  • Yes (C) - which means not exactly, but you do something else to compensate. This option is best if used in conjunction with adding evidence, as described below.

Every answer in a maturity level has to be YES or YES (C) to be compliant. For example, if you have all baseline marked as YES or YES (C), and then have 4 out of 5 in Evolving, you are considered Baseline compliant.

Evidence

Add proof to your statements. There are three forms of evidence that can be combined or used alone. All methods of evidence can selectively be shown in reports. 

  • Comment 
    Statement relevant information. For example, if you select YES (C), you can explain, "I do x, y, and z instead."
  • Tasks
    Ability to assign a one-time job, or task, to somebody in order to comply with a statement. 
  • Task Schedule
    A reoccurring task necessary to comply with a statement. For example, you can create a task schedule to create a report every quarter.
Adding Evidence

To add evidence, select the statement you want to add evidence for, and then chose to add/edit evidence. From the drop-down, select the type of evidence you would like to add: Comments, Tasks, or Task Schedules. 

  • Comments: Select Comments > insert comments > click Save.
  • Tasks: Select Tasks > enter a title, owner, and description > click Create.
  • Task Schedules: Select Task Schedules > Fill out all relevant information > click Create.
    Name: Your name
    Owner: Responsible party
    Description: What you want done
    Query: Search query that monitors events related to this policy.
    Schedule: When you want it done
    Policies: Current statement is added by default. You can add additional policies if desired.
    Tags: Terms you would use to search for and/or organize. 
Viewing / editing evidence

All evidence can be viewed, edited, and reported on. For auditing purposes, no tasks can be deleted, but they can be closed or marked invalid.

Viewing individual statement evidence

To view all evidence associated with a specific statement, go to Policy > Policies > correct Domain > correct Factor > Statement > Select Evidence from under the statement. This displays a drop down list of all evidence added to the statement. To view specific evidence, click on the line item with the evidence name. 


Viewing Comprehensive Evidence

To view all tasks and task schedules, go to Tickets > Task (or Task Schedules). When you open a task or schedule created through FFIEC, you'll see the correct domain and control added as a policy to the task.

When the time comes for a task schedule to be completed, a task is created and becomes visible in the task page. 


Completing the Maturity Evaluation Assessment

  1. Select Maturity Evaluation to display the overview and begin your assessment in Domain 1.
  2. Answer each statement for all five domains and their subsequent factors.
  3. View Results.
    Results are separated by domain. To view comprehensive results, generate a Maturity Evaluation report.

Generating FFIEC Reports

A major part of compliance is being able to prove it to external auditors. We assist by providing two different report options, FFIEC Report and the FFIEC Evidence Report.


FFIEC Report

This report provides detailed charts and graphical representations of your FFIEC results as well as an overview of the sections and what the charts represent. This works great for board meetings because it provides a good foundation of information and explanations plus critical security statistics. 


FFIEC Evidence Report

This report is great to show auditors proof of your compliance via added evidence. We know that security is key, and have provided the ability to select which sections are populated with evidence. The level of customization is as granule as providing evidence for a single control or as expansive as evidence for the entire report. 

When you select to download the evidence report, it displays in your download list as a zip file with all evidence added as well as a Word file of the entire report. 

Within the report, it lists all evidence associated with each control. It does not provide overview information, explanations, graphs, or charts. It is a list of evidence. 


Custom and Critical Security Control Policies

The option to create custom policies and to enable policies for the top 20 Critical Controls is available through the UI. These are good options for extra security, or for creating policies that are specific to your office only, and not covered in government regulations.

Create a custom policy

  1. To create a custom policy, click the + sign at the top, right side of the Policies page to open the Add a Policy window.  
  2. Insert Name and Description.
  3. Click Add Policy.
  4. Once the policy is added, the following window displays with the option to add Triggers, Incidents, Documents, and additional Control lines to the Policy.

Activate / Decativate Critical Controls Policy

  1. Go to Policy > Polices.
  2. Scroll down to the Critical Security Controls for Effective Cyber Defense section.
  3. Select the desired policy.
  4. Chose either Active or Inactive status.


Adding a Trigger

  1. Go to Alerts > Triggers
  2. Find and click the desired trigger
  3. Click the pencil icon to edit the trigger.
  4. Scroll down to the Policy section. Click the drop-down arrow and add the desired policies.

Adding an Incident

  1. Go to Tickets > Incidents.
  2. Find and click the desired incident.
  3. Scroll down to the Policy section. Click the drop-down arrow and add the desired policies. 

Adding a Document

  1. Go to Policy > Policies and click the desired policy.
  2. Select Documents to view a list of already added documents and the option to upload new ones.


Adding a Control

  1. Go to Policy > Policies and click the desired policy.
  2. Select Add control, insert a title and description, click Add Control to save addition.



Assets

The Assets page automatically displays all assets sending data to the DefenseStorm GRID via IP address, MAC address, and hostnames. If an asset displays as untracked, the Windows Agent has not been installed, and it must be manually listed as tracked. For best results, track/merge or investigate all untracked assets.

Setup, Configuration, and Importation of Assets

Importing your assets is typically done during the DefenseStorm on-boarding process, but there are several other instances where it may be necessary. For example, infrastructure changes such as upgrading hardware, routers, or switches may create the need to import your asset list. If your list of assets becomes unmanageable due to a high number of untracked assets, it may also be best to re-import.

The Assets portion of DefenseStorm GRID was designed to work with your specific network setup. With that in mind, there are different requirements for static, DHCP, or mixed networks. Most networks are considered mixed. 

Special considerations

  • If your network uses DHCP, DHCP logs should be forwarded to GRID.
  • Hostnames include: 
    • Machine hostname and FQDN
    • DNS hostname and FQDN
    • Aliases and FQDN

Importing your Asset List 

  1. Create an Excel file with all assets on your network 
    • Mixed Network = Hostnames, IP address, (MAC address required for DHCP host)
    • Static Network = IP, Hostnames, optional MAC address
    • DHCP Network = Hostnames and MAC address 
  2. Upload Excel list into the DefenseStorm GRID via the UI 
  3. Turn on auto discovery
  4. Clean up the list by making sure all information from step 1 is there for each asset. 
  5. Additional configurations
    • Insert CIDR ranges for static IP blocks
    • Enable detect Bare MAC Addresses - Only select this option if you are tracking every single MAC address. If you select this option and not all MAC addresses are properly listed, it creates untracked assets for new MAC addresses seen which may result in duplicate assets.

Import Recommendations

While only a few fields are required to track an asset, we highly recommend answering as many fields as possible to reduce the possibility for duplicate assets.  Multiple IP and MAC addresses can be associated with a single asset via a comma-separated list, surrounded by quotes. If using Excel to export to CSV, the extra quotes are added automatically.

The following Asset fields are highly recommended, or required, depending on your network configuration:

  • Owner - Always required
  • Asset Name
  • Asset Importance
  • Asset Tag
  • Asset Hostname

Only include the ID field if you are importing assets that have previously been exported. The id number is from the export to ensure you are updating the existing asset as part of your import, rather than importing a new asset. 

Known Limitations

If the import data contains mixed entries (some are only MAC addresses, and some only have IP addresses), null values must be manually inserted into the CSV, as these are read in pairs during import.

Example with three mixed entries:

(11.11.11.11, null)
(22.22.22.23, null)
(null, de:ad:be:ef)

IP address CSV field: “11.11.11.11,22.22.22.23,,”
MAC address CSV field: “,,de:ad:be:ef”

The following line can be used as the first line of a CSV file, then opened in an editor of your choice to continue inputting data. (note: row spans multiple lines on this document).

Name,ID,Owner,Hostname,IP Address,MAC Address,Importance,Description,OS,Product Vendor,Product Name,Product Version,Server Purpose / Notes

Organization of the Assets Page

After you have uploaded and ensured all assets are tracked, use the information in this section to keep an up to date record of your assets and their activity. 

There are eight sortable columns (from left to right):

  • Importance: Displays as either none, one, two, or three dots. Knowing the importance level of an asset helps our TRAC Team provide better monitoring by ensuring your critical systems are always attended. If you update the importance, open a connect ticket so custom triggers can be updated.
  • Name: When you track your asset, the name entered displays here; this is searchable. If an asset has not been added, it displays as untracked.
  • Heartbeat: Last time data was received from the asset. In order to receive heartbeat data, automatic asset detection must be enabled and an IP address must be listed.
    • Grey: Never
    • Green: Within the last 24 hours
    • Yellow: Within the last 7 days, but not the last 24 hours
    • Red: Within the last 30 days, but not the last 7
  • Hostname: The hostname you give the asset, this is searchable.
  • IP Address: The IP address of the asset, this is searchable.
  • MAC Address: The MAC address of the asset, this is searchable.
  • Last Seen: This is updated when events are matched to the asset via IP address, MAC address, or hostname. Any untracked assets that have not been seen for 30 or more days, are automatically deactivated. This allows the associated MAC and IP addresses to be available for future use.
  • Events: Takes you to the Events page and only displays events for the selected asset.

In addition to the ability to sort columns, you have several other options when organizing your assets.

  • Asset Settings: Change the number of assets per page, disable auto-detection, and include/exclude CIDR ranges.
  • Trashcan: Bulk deletion of depreciated or invalid assets.
  • Cloud Download: CSV export of selected assets.
  • Cloud Upload: Upload your assets to the UI via CSV file.
  • Plus: Add a new asset.


Managing Untracked assets

An untracked asset means the asset does not have the Windows Agent running, but is still sending data. You have three options with an Untracked Asset:

  • Merge into existing asset: This option is good for employees that have more than one asset sending data to the UI or if an asset has multiple interfaces like WiFi and Ethernet. For example, Bob has a laptop, desktop, and a mobile device.
  • Track this asset: Make your asset official. Give the asset a name, IP, hostname, and all other known information.
  • Create incident from this asset: When an unknown asset displays on your console, create an incident for the TRAC Team to investigate.

To Track an Asset

An asset must be tracked before any changes or updates can be made. Complete the following steps to track your untracked asset.

  1. Click the dropdown arrow to the right of the Name and then select Track This Asset.
  2. The Add Tracked Asset window displays. Add as much information as possible.
  3. After an asset has been added, click on the asset name to display the Asset Details window where you can view, edit, and delete the asset.

CIDR Auto-detecting Untracked Assets

By default, we autodetect assets from events with hostnames or with mac-addresses. If we receive a DHCP event that links an IP address to a MAC address, we create an untracked asset with both.

If we receive an event that has an IP address, but no MAC address, we will not auto-detect that asset unless the IP falls within an “Included CIDR Range” that has been configured on the Assets page.

To configure the "Included CIDR Range" 


  1. Click the Asset Settings button located at the top right corner of the Assets page.
  2. Set the Asset Auto-Detection field to 'on' 
  3.  include the IP addresses in the "Included CIDR Range" field
  4. click 'Add' and 'Save' to apply the changes.
     This auto-detects the events coming in from those IP addresses and marks it as an "untracked" asset. Later, you can manage the untracked assets based on your needs.

Managing your Asset List

After the initial upload of your assets, there are a few best practices to keep your asset list tidy and up to date.

High number of untracked assets

  1. Upload your updated Asset list to the console as described earlier in this article. This allows your console to receive a fresh start and gives you a strong foundation for future asset management.
  2. Consider adding a classifier which excludes assets that are not relevant; such as a guest WiFi network or a test network.

Regular Asset Maintenance

  1. Each week verify there are no untracked assets. Search via Untracked, and see that none display.
  2. If you do see untracked assets, investigate and determine if it needs to be merged into an existing asset, added as a new tracked asset, or if it needs TRAC Team attention.
  3. If it is a duplicate asset, select to merge the asset.
  4. If it is a new asset that has been added to the network since your initial upload, select to track this asset.
  5. If it needs investigation,
    • Select Create Incident from this Asset, and fill it out as such:
      • Title: Unknown Asset
      • Owner: TRAC 
      • Severity: Low
      • Description: What steps you have already taken to figure out what the asset could be, along with your conclusion.

Creating a Filtered CSV Export of Assets

This gives you the ability to filter down your assets to a useful and manageable list to be exported, saved, formatted, and used for reporting.

  1. Select Assets in the left navigation.
  2. Filter your assets as desired by using Tracked status, Importance, Filter by Tag, or Searching options.
  3. Select the cloud icon to generate a CSV of the filtered assets.
  4. Save as an Excel sheet and format as desired. 

 


PatternScout

DefenseStorm GRID's anomaly detection capabilities are called PatternScout.  PatternScout forms a baseline of activity and looks for deviations that could be malicious.  Due to PatternScout's machine learning, each time an alert is handled within the Alert Inbox, the GRID learns and adapts its responses. 

There are two types of PatternScout, Dynamic Thresholds and Temporal.  Dynamic Thresholds are best suited to detect sharp differences in queries. (If the query is strongly associated with the day cycle, anomaly detection may result in false positives at the start of the day.) Temporal anomaly detection is best used for data that has a wall clock pattern to it, such as employee logons, or normal network traffic. (It does not work well when there are level shifts that do not correlate with time.) 

Anomaly Detection via PatternScout Triggers

Utilizing PatternScout is essential to creating constant, around the clock network monitoring. The more PatternScout is cultivated, the more efficient it becomes. PatternScout is utilized through triggers. Each trigger, either from the library or custom created, has the option to enable a form of PatternScout. The following are the different ways PatternScout is utilized through triggers:

  • Enable from the Library
  • Edit a trigger 
  • Create a custom a trigger 

Copy PatternScout Triggers from the Library

Utilizing their many years of cybersecurity expertise, the DefenseStorm TRAC Team designed a list of triggers specifically for effective and efficient alerting on anomalous activity. 

  1. Go to Alerts > Library > Scroll down to PatternScout
  2. Open PatternScout > select desired checkbox(es) 
  3. Choose to Copy Selected. After they have been copied to your network, they are enabled.
  4. To view or edit the trigger, go to Alerts > Triggers. (See Triggers for additional trigger field descriptions.)

Editing a Trigger to Enable PatternScout

  1. Go to Alerts > Triggers
  2. Select the Trigger you want to encompass PatternScout
  3. Select the pencil icon to edit 
  4. Scroll down to PatternScout and chose anomaly detection type: Dynamic Thresholds or Temporal
  5. Select Save

Creating a Custom PatternScout Trigger

In addition to enabling PatternScout Triggers via the Trigger Library, you can also create custom triggers to enhance PatternScout functionality with your network. 

  1. Go to Alerts > Triggers
  2. Select the blue + icon on the top right of the page
  3. Fill out fields as desired. (based on descriptions from the Triggers article.)
  4. Select either Dynamic Threshold or Temporal PatternScout checkboxes
  5. Select Save



Integrations

Cloud security has become an essential part of protecting the modern financial institution from cyberattack. Therefore, DefenseStorm offers security of your cloud resources at no additional charge. 


AWS: CloudTrail and ELB

DefenseStorm watches and alerts on parts of AWS that, based on the shared responsibility model, Amazon expects you to monitor. For example:

  • Operating system
  • Network configurations
  • Applications 
  • Access management 

CloudTrail is an Amazon web service that provides visibility into user activity by recording API calls made on your account and delivers log files to an Amazon S3 bucket.  This information helps you track changes made to your AWS resources and to troubleshoot operational issues. If you are using AWS, it is recommended that CloudTrail be enabled. 

The DefenseStorm GRID ingests ELB access logs that capture detailed information about requests sent to the load balancer. Perform the following steps to setup the AWS features through the DefenseStorm UI.

  1. Go to Settings > Integrations and select the Amazon Web Services icon.
  2. Input your Amazon Web Services Account information and select Create.
  3. Connect CloudTrail to DefenseStorm by selecting the gear icon on CloudTrail and follow the instructions displayed.
  4. Connect your AWS ELB to DefenseStorm by selecting the gear icon on Elastic Load Balancing, and then following the instructions displayed.

OpenDNS

OpenDNS offers network security by reviewing all of your employee network connections on or off the corporate network. Since DefenseStorm is a layer that can “see everything”, we correlate the events OpenDNS captures when users leave the corporate network with the rest of your corporate network.

Use of this integration requires an "Insights" or "Platform"  OpenDNS subscription; the "Professional" subscription level is incompatible, due to lacking the log export feature below:

  - Retain logs with Amazon Web Services integration using customer-managed or Cisco-managed S3 bucket (source: https://umbrella.cisco.com/products/packages)

To enable the OpenDNS functionality, contact DefenseStorm for assistance.


Office 365

Adding Office 365 services is as easy as adding a cloud app. To avoid creating multiple passwords, the administrator uses their Active Directory (AD) credentials to setup Office 365 integration and ingestion of logs. Integration with Office 365 supports the following activities:

  • File and folder
  • Sharing and access request
  • Synchronization
  • Site administration
  • Exchange mailbox
  • User administration
  • Group administration
  • Application administration
  • Role administration
  • Directory administration
  1. Go to Settings > Integrations > Office 365 icon > and select the option to Add Office 365 Account. Follow instructions displayed.
  2. Once you have selected the link and followed the steps provided by Microsoft, all that’s left is to give your Office 365 account a display name. You are redirected back to the DefenseStorm console, where you see the following:
  3. After the display name is added, a message displays saying that you have successfully integrated with Office 365. Note: Auditing on Exchange Mailboxes is off by default. 


ThreatMatch

Have you ever wondered how the FBI or CIA catch cybercriminals? Well, part of it is through their extensive threat-sharing network. DefenseStorm ThreatMatch utilizes that network and gives you access to threat feeds from various companies and government agencies to scan your network for matching activity.

If you want action taken each time a match is found, you can subscribe to the threat feed. Subscribing to a feed allows you to configure options such as email alerts, incident creation, linking policies, assigning severity, etc. If you chose not to subscribe, all relevant data is still displayed, but no automatic action is taken.

In the screenshot below, the blue icons to the left of the feed name indicate you are subscribed. As you can see, all data is still displayed for all feeds, regardless of subscription.


Once within the Alerts > ThreatMatch page, all collected data is displayed per feed. This page gives you the option to subscribe, configure, and investigate your ThreatMatch data. 

For assistance utilizing the feature efficiently, here is a description of the data within each column:

  • On/Off Switch: Blue indicates you are subscribed, transparent is unsubscribed. 
  • Name: Unique identifier provided by creator of the feed.
  • Tags:  Type of information searched for threats.
  • Indicators: Number of possible matches. 
  • Matches: Total count of matches, including the same matches found over multiple days.
  • Unique Matches: Deduplicated count of matches.
    For example, you can have 11 matches, 1 unique match. Which means the 1 unique match was found 11 times. 
  • Last updated: The last time the feed was updated.


Configuring ThreatMatch

To apply configuration settings to the feeds you have subscribed to, click the gear icon towards the top of the ThreatMatch page.  

  • Email: If you chose to receive email alerts, only a single email is sent the day a match is detected. 
  • Policy: Associate a policy and control with Threat Match alert findings. 
  • Severity: Give alerts a severity level.

Creating your own Threat Feed

One of the Threat Match features is the blocking and tracking of indicators of compromise (IOC’s).  An IOC is anything from IP addresses, malicious files, or URL’s. If you find specific activity on your network that is an indicator of possible malicious activity, you can upload it to ThreatMatch to begin tracking.

To upload an IOC into ThreatMatch

  1. Go to Alerts > ThreatMatch > Select the blue plus symbol (+).
  2. Fill-in all relevant fields:
    Name: Unique identifier for the IOC
    Description: For example: Malcode creates and maintains a list of domains that are known to host malware and spyware.
    URL: The URL associated with your IOC
    Tags: Type of information
  3. Once a new threat source has been added, a cloud icon displays to the far right of the row; select the icon.

  4. Fill in all relevant fields:
    Indicators
    There are several things to consider when choosing to upload a file instead of typing in each indicator per line:
    - The files must be .txt (notepad, etc)
    - No combination documents. For example, one document is all domains, one document is all IPs, and one document is all hashes.
    - "domain.com" includes only the one domain, no subdomains.
    Description
    Information you find relevant to the threat sharing source, why you needed to add it is an IOC, etc.

    Threat Source
    Name of the threat source.

    URL
    The URL that needs to be alerted on.

    Date Range
    Due to the rapid rate of change in IP addresses, DefenseStorm recommends that you input a date range for IP addresses, that it look no farther than 3 months ahead. We also recommend that it only go back on month. This date range of back 1 month, forward 3 months provides the best range of coverage. Since domains remain constant, putting a date range for them is not necessary and can be removed.

    Threat Indicator Type
    IP Address
    Domain
    Signature (This is a file hash, like an MD5, SHA1, or SHA256) 
  5. Select Save.


ThreatMatch Classifiers

Sometimes ThreatMatch can find matches that have already been denied by your firewall or appropriate system. To dampen the noise while still receiving email alerts, you can exclude events from ThreatMatch by creating a Classifier.

To create a ThreatMatch Classifier

  1. Go to Events and enter the search query you'd like to exclude. For example: app_name:"Cisco ASA" deny
  2. Select "Create Classifier"
  3. Select the Exclude from ThreatMatch checkbox to prevent any alerts on the query.  See Classifiers for details on other classifier fields.  
  4. Select Save.



Classifiers

Classifiers are used to manipulate, or preprocess, events before they get to the UI. For example, you can raise or lower the severity of an event, choose to ignore specific events from ThreatMatch, PatternScout, asset detection, or even drop an event before it reaches the GRID.  To be safe and not miss critical data, we recommend sending all data to the GRID.

WARNING: Dropping events means that the event is not logged within the GRID. It is not searchable and does not trigger alerts.

How to use classifiers

Classifiers allow you to modify events or change how they're treated by the GRID. 

  • Exclude: No alerts. Data is still saved and searchable.
  • Change attributes: Alter how data is displayed, to make data more useful. 
  • Drop Events:  Discarded before they get to the GRID.  Be careful, it is not searchable or logged. 

Creating a classifier 

Classifiers have a large impact on the functionality and efficiency of the GRID. Follow the instructions below to ensure your classifiers are syntactically correct and function properly. 

  1. Events > enter search query. All events matching the query display.
  2. Select the Classifier icon to create a classifier of your query.
  3. Fill in Classifier fields.
    Tag Name: Classifiers should have meaningful, easily identifiable names.
    Query String: [Auto-populated from the event search] Conditions the classifier is set to search on.
    Drop Event:  Warning: If drop event is checked, it does not log the event.  Events that match this classifier are not searchable and do not trigger alerts. (Be careful using this!)
    Exclude from ThreatMatch, PatternScout, or Asset Detection: Stop the system from using the event for threat intelligence, anomaly detection, or asset detection, respectively.
    Attributes to Apply: Select the key (data) you'd like to update, and then add the value you want the key to have.  For multiple changes, click the plus sign.
    For example, a group of events have a category of ‘None’ and the organization wants to categorize these events based on information contained within the event. Key = Category and the Value = Research
  4. Select Save to complete and enable your classifier.
  5. After setting up the classifier, you can search for _exists_:tag_queries to display events that have matched one or more classifiers.

Specifications for Classifiers

  • You must select to either drop, exclude, or apply an attribute to create a classifier.
  • If you create a classifier for ‘WDAP’, it matches all events that contain ‘WDAP’. If you create a classifier NOT ‘WDAP’, it matches events that do not contain ‘WDAP’. This is the opposite of writing NOT or - within command line text.
     

Pausing, editing, or deleting a classifier

Once the classifier is created, you have the option to pause, edit, or delete it.  To activate or pause a classifier, simply click the power icon and it will toggle from blue to gray. Once a classifier is active (enabled), a blue power icon displays to the left. If classifiers are paused (disabled), the power icon is gray.

To delete a classifier
  1. Go to Events > Classifiers.
  2. Click on the classifier you want to delete.
  3. Select Delete.

Pausing or deleting classifiers may take a short time to take effect.



Query Syntax

Query Syntax

The query string used for searching events is composed of a series of terms and operators. A term can be a single word — DefenseStorm or Agent — or a phrase, surrounded by double quotes — "DefenseStorm Agent" — which searches for all the words in the phrase, in the same order. Boolean operators (AND, OR, NOT) allow you to customize the search.

Searching

When performing a search in GRID, a single word or phrase is searched for in all available fields such as app_name, message, etc. Searches are not case sensitive but attention needs to be paid to spacing and required double quotes. Terms containing punctuation, such as 127.0.0.1, bro_dns, or microsoft-windows-search, are not considered a phrase and do not need to be placed in double quotes.


Single fields can be isolated and searched for without searching all of the fields. This is a preferred tactic for more specific searches. Be sure to separate the field and term with a colon(:).

For example, app_name:"DefenseStorm Agent"

A space is permitted between the colon and phrase: app_name: "DefenseStorm Agent" although not preferred. Also, app_name:"DefenseStorm Agent" is the same as app_name:"defensestorm agent" as capitalization is not acknowledged.

Watch for synonyms

Synonyms are not understood by the GRID. People may understand wi fi and "wi fi" and "wifi" and “Wi-Fi” to all be the same thing but the system does not. The query string parser would interpret your query as a search for wi fi as "wi OR fi". This likely is not specific enough for your intended search. Be sure what the query is designed for will in fact answer the intended demand.

Wildcards

Wildcard searches can be run on individual terms, using ? to replace a single character, and * to replace zero or more characters:

bro_d?s  would return events containing bro_dns and bro_dxs (if there were such a thing).

bro* would return events under all bro in app_name including bro_weird, bro_files, bro_ssl, etc. This particular search approach is good for confirming large quantities of events such as a firewall or other appliance traffic.

Boolean Operators

Boolean Operators are used to connect and define the relationship between your search terms and phrases. The three Boolean operators are AND, OR and NOT. 

Use AND to narrow your search: all of your search terms must be present in the retrieved records. AND can also be executed as + or &&.

app_name:"DefenseStorm Agent" AND category:Bad_Stuff AND event_id:911

app_name:"DefenseStorm Agent" +category:Bad_Stuff +event_id:911

app_name:"DefenseStorm Agent" && category:Bad_Stuff && event_id:911 
(*note using && requires a space between && and the term whereas using + does not)

 These queries require that events contain all three terms in order to be returned.

Use OR to broaden your search by connecting two or more synonyms. OR can also be executed as a double pipe (||), not to be confused with two lowercase Ls as they look the same.

category:Bad_Stuff OR category:Worse_Stuff

category:Bad_Stuff || category:Worse_Stuff 
(*note using || requires a space between || and the term)

This query requires either term in order to be returned.

Use NOT to exclude terms from your search results. NOT can also be executed as an exclamation point (!) or a minus sign (-).

category:Bad_Stuff NOT event_id:911

category:Bad_Stuff !event_id:911

category:Bad_Stuff -event_id:911 
(*note using ! or - to denote NOT requires no space between it and the term)

This query requires any event_id fields be returned that are not event_id:911.

 

The familiar operators AND, OR, and NOT (also written &&, || and !) are supported and all variations of these can be combined in a single query. However, the effects of these operators are more complicated than is obvious because of the order of precedence, also known as the order of operations. There are predetermined rules that govern the order of which procedures are performed first in order to evaluate a query. NOT takes precedence over AND, which takes precedence over OR. 

NOT > AND > OR        which is the same as    ! > && > ||

While the + and - only affect the term to the right of the operator, AND and OR can affect the terms to the left and right. If this isn’t confusing enough, we have the added dilemma of remembering the spaces.

Another consideration is the use of parentheses which can ease the confusion with the other operators. Syntax contained within parentheses has the highest priority. 

Grouping

Multiple terms or phrases can be grouped together with parentheses to form sub-queries:

(virus OR malware) AND trojan

Reserved Characters

If you need to use any of the characters which function as operators in your query itself (and not as operators), then you should escape them with a leading backslash. For instance, to search for (1+1)=2, you would need to write your query as \(1\+1\)=2.

The reserved characters are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a syntax error which prevents your query from running.

Empty Query

If the query string is empty or only contains whitespaces the query string is interpreted as a no_docs_query and is expected to return no results.

Regular Expressions

Regex searching is supported but use this method against single fields. Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes ("/"):

name:/joh?n(ath[oa]n)/

The supported regular expression syntax is explained later in this document in Appendix A.


Appendix A: Regular Expression Query

Regular Expression syntax

Regular expression queries are supported in the DefenseStorm GRID, under the Events view search. The Lucene regular expression engine is not Perl-compatible but supports a smaller range of operators. We are not going to explain regular expressions, just the supported operators.

An excellent resources for additional information is https://regexr.com.

Standard Operators

Standard operators are always enable through the DefenseStorm Console. For more detailed information about standard operators, see Standard Query Operators Overview from Microsoft.

Anchoring

Most regular expression engines allow you to match any part of a string. If you want the regexp pattern to start at the beginning of the string or finish at the end of the string, then you have to anchor it specifically, using ^ to indicate the beginning or $ to indicate the end.

Lucene’s patterns are always anchored. The pattern provided must match the entire string. 

For string "abcde":

ab.*  # match
abcd  # no match

Allowed characters

Any Unicode characters may be used in the pattern, but certain characters are reserved and must be escaped. The standard reserved characters are:

. ? + * | { } [ ] ( ) " \

If you enable optional features (see below) then these characters may also be reserved:

# @ & < >  ~

Any reserved character can be escaped with a backslash "\*" including a literal backslash character: "\\"

Additionally, any characters (except double quotes) are interpreted literally when surrounded by double quotes:

john"@smith.com"

Match any character

The period "." character can be used to represent any character. 

For string "abcde":

ab...  # match
a.c.e  # match

One-or-more

The plus sign "+" can be used to repeat the preceding shortest pattern once or more times. 

For string "aaabbb":

a+b+  # match
aa+bb+  # match
a+.+  # match
aa+bbb+  # match

 Zero-or-more

The asterisk "*" can be used to match the preceding shortest pattern zero-or-more times. 

For string "aaabbb":

a*b*  # match
a*b*c*  # match
.*bbb.*  # match
aaa*bbb*  # match 

Zero-or-one

The question mark "?" makes the preceding shortest pattern optional. It matches zero or one times.  When no parentheses (groupings) are used, this affects a single character to the left of it.

For string "aaabbb":

aaa?bbb?  # match
aaaa?bbbb?  # match
.....?.?  # match
aa?bb?  # no match

Min-to-max

Curly brackets "{}" can be used to specify a minimum and (optionally) a maximum number of times the preceding shortest pattern can repeat. The allowed forms are:

{5}  # repeat exactly 5 times 
{2,5}  # repeat at least twice and at most 5 times 
{2,}  # repeat at least twice

For string "aaabbb":

a{3}b{3}  # match
a{2,4}b{2,4}  # match
a{2,}b{2,}  # match
.{3}.{3}  # match
a{4}b{4}  # no match
a{4,6}b{4,6}  # no match
a{4,}b{4,}  # no match 

Grouping

Parentheses "()" can be used to form sub-patterns. The quantity operators listed above operate on the shortest previous pattern, which can be a group.  

For string "ababab":

(ab)+  # match
ab(ab)+  # match
(..)+  # match
(...)+  # no match
(ab)*  # match
abab(ab)?  # match
ab(ab)?  # no match
(ab){3}  # match
(ab){1,2}  # no match

Alternation

The pipe symbol "|" acts as an OR operator. The match succeeds if the pattern on either the left-hand side OR the right-hand side matches. The alternation applies to the longest pattern, not the shortest. 

For string "aabb":

aabb|bbaa  # match
aacc|bb  # no match
aa(cc|bb)  # match
a+|b+  # no match
a+b+|b+a+  # match
a+(b|c)+  # match

Character classes

Ranges of potential characters may be represented as character classes by enclosing them in square brackets "[]". A leading ^ negates (disallows) the character class. 

The allowed forms are:

[abc]  # 'a' or 'b' or 'c' 
[a-c]  # 'a' or 'b' or 'c' 
[-abc]  # '-' or 'a' or 'b' or 'c' 
[abc\-] # '-' or 'a' or 'b' or 'c' 
[^abc]  # any character except 'a' or 'b' or 'c' 
[^a-c]  # any character except 'a' or 'b' or 'c' 
[^-abc]  # any character except '-' or 'a' or 'b' or 'c' 
[^abc\-] # any character except '-' or 'a' or 'b' or 'c'

Note that the dash "-" indicates a range of characters, unless it is the first character or if it is escaped with a backslash.

For string "abcd":

ab[cd]+  # match 
[a-d]+  # match 
[^a-d]+  # no match