Collecting Logs and Events from Windows Machines


Overview

Overview

The Windows Agent collects and forwards your local Windows event activity to the DefenseStorm GRID. To ensure all events are logged and tracked, data is still captured when a device is offline.  DefenseStorm recommends using Sysmon along with the Windows Agent to maximize visibility of activity on the device.

The Windows Agent plus Sysmon is for all Windows Servers*, Workstations, and Laptops.  Includes Windows Vista SP2 and higher, Windows 7 SP1 and higher, Windows 8 (all versions, but 8.1 or higher recommended), Windows 10 (all versions), and Servers 2008 and higher.

*If you are currently using NXLog to capture Windows Events, see Moving from NXLog & PWT to the Windows Agent at the end of this article.

Features

  • Auto-updates from the DefenseStorm cloud
    • Checks within 2-4 hours of the service starting, and every 24 hours after that. If the Agent is restarted, the timeline restarts as well.
    • Can be turned off upon installation.
  • Windows based installer supports both GUI and command line installations
  • Full documentation for installer usage with options
    • CLI configuration settings
  • Two-factor authentication
  • Login Options
    •  username/password
    • token/secret/orgId
  • Local event log collection
  • Windows event logging
  • Full offline logging. All available data is sent upon reconnection to the network.
  • Send data via DVM or direct to the Cloud
    • Fallback to the available method

Requirements & Limitations

  • NET 4.5.2, or greater, must be installed
  • User needs administrative permissions
  • Internet connection for authentication during installation
  • Local file logging requires edits to the configuration file, and then a service restart to take effect
  • Requesting information from the Windows event log does incur some load on the system; this load scales with the current size of the event log files.  If your organization has increased the maximum event log sizes from the Microsoft default (25mb) and these files have grown to 1 GB+, you may see high resource utilization during the first event check after Agent initialization.  We are currently investigating ways to reduce this impact on certain hardware configurations.

Recommendations for Windows Audit Logs

Within the Windows Operating system, there are audit logs that can be configured to track success or failure of certain events. For example, once audit logs are activated for logon attempts, you can choose to track if a logon attempt is successful, if it fails, or both. Our TRAC Team has prepared examples for each option within the Windows Operating System to help send the most useful logs to your DefenseStorm DVM and reduce noise on your system. 

The following items can be monitored:

  • Changes to user account and resource permissions.
  • Failed attempts by users to log on.
  • Failed attempts to access resources.
  • Changes to system files. 

This link provides the PDF list of TRAC Recommendations.

Installation

Installation Procedures

When downloading the Windows Agent, we provide a few pre-install procedures, and then offer instructions for a command-line or GUI installation. 

1. Pre-install Procedures 

Before installing the DefenseStorm Windows Agent, make sure you have an msi downloaded on your machine and if you are using the fallback installation, or proxy settings, you also need your DVM IP Address. 

Download an msi 

Required for all installation types.  Follow the steps below.

  1. Obtain DWA installer files > click link for DefenseStorm Windows Agent 
  2. Make note of the path to the msi location on your local machine. The path is used in the install process.
    For example, /Users/joetester/Downloads/DefenesStorm.msi

Obtaining your DVM IP address

Required for fallback installation and entering proxy settings. 

  1. Login to the GRID
  2. Events > search pvm_stats> filter by hostname
  3.  If you see multiple IP Addresses and are not sure which one is correct, contact DefenseStorm Support

Obtaining your Key, Secret, and Org Id

Required for command line installation of the Windows Agent. 

  1. Login to the GRID
  2. Settings > Input Tokens > Get Agent Token
  3. Copy  the Key, Secret, and Org Id into a document
  4. Use the copied key, secret, and org id for agent installation



2. Command-line Installation

Utilizing the command-line installation method allows for silent installation across your network. There are three different methods for sending data to the DefenseStorm GRID. To verify that the Windows Agent was installed correctly, view Windows events in the GRID UI. 

Direct to Cloud (default)

Data comes from the Windows Agent directly to the DefenseStorm Cloud. Enter the following command with your specified network information.

msiexec /package <path to DefenseStorm.msi> APIKEY=key APISECRET=secret ORGID=organization_id /quiet

Through DVM

Data flows from the Windows Agent to the DVM, then to the Cloud.  Enter the following command with your specified network information.

msiexec /package <path to DefenseStorm.msi> APIKEY=key APISECRET=secret ORGID=organization_id SENDEVENTS=dvm DVMHOST=1.1.1.1 /quiet


Fallback from DVM to Cloud

Data flows from the Windows Agent, to the DVM, then to the Cloud unless the DVM is unavailable. In that instance, the data goes directly to the Cloud until the DVM becomes available.  Enter the following command with your specified network information.

msiexec /package <path to DefenseStorm.msi> APIKEY=key APISECRET=secret ORGID=organization_id SENDEVENTS=both DVMHOST=1.1.1.1 /quiet

Command Line Switches
  • UPDATES {bool, default True. Toggles automatic application updates}
  • WINDOWSUSER {string, default LocalSystem. Specifies user account to run the agent under. This is not recommended.}
  • WINDOWSPASSWORD {string, default NULL. Specifies user account to run the agent under's password. This is not recommended}
  • APIUSER {string, default NULL. DefenseStorm Username for API access- must have accompanying APIPASSWORD.}
  • APIPASSWORD {string, default NULL. DefenseStorm Password for API access- must have accompanying APIUSER.}
  • APIKEY {string, default NULL. DefenseStorm Key for API access- must have accompanying APISECRET and ORGID.}
  • APISECRET {string, default NULL. DefenseStorm Secret for API access- must have accompanying APIKEY and ORGID.}
  • ORGID {string, default NULL. DefenseStorm org id for API access- must have accompanying APISECRET and APIKEY.}
  • SENDEVENTS {string, default NULL. DefenseStorm org id for API access- must have accompanying APISECRET and APIKEY.}
    • Inserting BOTH allows for fallback from the DVM to the cloud, DS means your data is sent to the cloud, DVM is only the DVM.
    • If choosing DVM or BOTH, you must enter DVMHOST into the command line. This option defaults to the DVM first, then to the cloud as fallback.
  • PROXY {string, default NULL. DefenseStorm org id for API access- must have accompanying APISECRET and APIKEY.}
    • Example of a Cloud Install with proxy settings: msiexec /package <path to DefenseStorm.msi> /l*v mylog.txt APIKEY=key APISECRET=secret ORGID=<organization id> PROXY=<your proxy HTTP address:port, e.g. "http://10.10.10.10:3128"> /quiet


3. GUI Installation

Follow these instructions to install the Windows Agent manually through the GUI.

  1. Click here to Obtain DWA installer files.
  2. Download > double-click to run. 
  3. Select Next on the following window to begin installation.
  4.  Terms and Conditions. Select the checkbox to agree, then click Next.  
  5. Installation folder. Select Next to keep the default folder location.
  6. Sending data. Chose to either send data 'Directly to DefenseStorm', "Through the DVM', or 'Through the DVM if available, directly otherwise'. If you chose to send data through the DVM only, or through the DVM if available, directly otherwise, you must input your DVM IP Address.
  7. Input Token/Key/OrgID from the GRID or select to input Username/Password.  Select Next.
  8. Select Auto-Updates, click Next.
  9. Complete install. Select Yes on the following pop-up.
  10. The installation completes.
  11. Verify that the Windows Agent is installed correctly by viewing Windows events in the GRID.

4. Mass Installation

Follow these instructions to install the Windows Agent on multiple devices at the same time.

  1. Obtain DWA installer files > click link for DefenseStorm Windows Agent.
  2. Download it to a folder accessible to all machines.
  3. Open Notepad
    1. copy and paste the following cmd
      1. msiexec /package <path to DefenseStorm.msi> APIKEY=key APISECRET=secret ORGID=organization_id /quiet
    2. File > Save As > NameOfYourChoice.bat
  4. Push that script using GPO (GroupPolicy) to all desired machines.  


Additional Options & Configurations

Additional Options and Configurations 

DefenseStorm provides many configuration options to help ensure the Windows Agent service functions efficiently for your specific network needs. You have the ability to add additional events, alter default settings, and add DHCP/DNS Logging.

Note: All Windows configurations can be modified by a Windows Admin, so make sure to view your particular instructions to verify location on your network. 

1. Adding Additional Windows Event Logs

By default, the DefenseStorm Windows Agent collects logs from various sources defined in a JSON file called “LogSources”.  Each of these logs can be viewed in the Windows Event Viewer Desktop application. However, you may choose to expand beyond the sources by modifying the ClientLogSources.json file. View this pdf for a table of the default log sources for the Windows Agent, Default Log Sources.

Note: All Windows configurations can be modified by a Windows Admin, so make sure to view your particular instructions to verify location on your network.

  1. On your local machine, go to C:\Program Files (x86)\DefenseStorm\Data
  2.  Open ClientLogSources.json in notepad, or other applicable application.
  3. Add information within any of these 5 log sources. Below is an example,
    {
        "comment": "All DefenseStorm Logs",
        "log_name": "Application",
        "source": "DefenseStorm",
        "level": [ "Information","error","warning"],
        "event_id": [10001,2000]
      }
    1. "comment" can have any text
    2. "log_name" is the same that appears in the `Log` column. Mandatory field.
    3. "source" is the `Source` column (Omitting means all sources)
    4. "level" column, whose valid values are: Critical, Error, Warning, Information, LogAlways, Verbose (Omitting means all levels)
    5. "event_id" is an array of numbers from the `Event ID` column (Omitting means all events)
  4. Make desired field edits. Save file.
  5. Restart the Windows Agent Service via instructions later in this article to start the changes. 

Adding IIS Logging Events

Note: Check your particular instructions in your IIS service to configure our json file properly.

  1. On your local machine, go to C:\Program Files (x86)\DefenseStorm\Data
  2.  Open config.json in notepad, or other applicable application.
  3.  For IIS, insert the following:
{
"AutoUpdate": true,
"MonitoredFiles": {
"Generic": [ "C:\\Windows\\system32\\LogFiles\\W3SVC1\\u_ex*.log" ],
}
}

Adding Windows DHCP and DNS Logging Events

Note: Check your particular instructions in your DHCP and DNS service to configure our json file properly.

  1. On your local machine, go to C:\Program Files (x86)\DefenseStorm\Data
  2.  Open config.json in notepad, or other applicable application.
  3.  For DHCP IPv4, IPv6, and DNS, insert the following:
    {
    "AutoUpdate": true,
    "MonitoredFiles": {
    "Generic": [ "C:\\Windows\\System32\\dhcp\\DhcpSrvLog-*.log", "C:\\Windows\\System32\\dhcp\\DhcpV6SrvLog-*.log" ],
    "DnsDebugLog": [ "C:\\dns\\dns.log" ]
    }
    }
  4.  If you utilize only DHCP IPv4, insert the following:
    {
     "AutoUpdate": true,
     "MonitoredFiles": {
     "Generic": [ "C:\\Windows\\System32\\dhcp\\DhcpSrvLog-*.log" ]
    }
    }
  5. If you utilize only DHCP IPv6, insert the following:
    {
     "AutoUpdate": true,
     "MonitoredFiles": {
     "Generic": [ "C:\\Windows\\System32\\dhcp\\DhcpV6SrvLog-*.log" ]
    }
    }
  6. If you utilize only DNS, insert the following:
    {
     "AutoUpdate": true,
     "MonitoredFiles": {
     "DnsDebugLog": [ "C:\\dns\\dns.log" ]
    }
    }

2. Windows Agent CLI Options

The Windows Agent comes with a CLI (command-line interface) that allows tweaking of configuration options. The CLI is installed by default in C:\Program Files (x86)\DefenseStorm\Agent\DefenseStorm.exe. 

The examples in this section are written for PowerShell, but the functionality works for either PowerShell or Command Prompt.

  1. Execute PowerShell as an Administrator.  The CLI requires access to the DefenseStorm key in the Windows Registry, which is not accessible to non-admins.
  2. Input the following path
    cd 'c:\Program Files (x86)\DefenseStorm\Agent'
  3.  Run the following to get general usage and details for the CLI 
    .\DefenseStorm.exe

Viewing current configurations

To see the current configuration options, execute defenseStorm list, which displays output as shown below, (some values are redacted for security).

.\DefenseStorm.exe list


Example of how to change a configuration

Use defenseStorm set to change any supported configuration options. Below is an example of changing the ReadEventsIntervalSeconds from 1 to 2.

  1. Insert the following command. To change an alternate option, alter the  option name and values accordingly. 
    .\DefenseStorm set ReadEventsIntervalSeconds 2
  2. Old and new values display
  3. Restart the Windows Agent for the changes to take effect.  Procedure is explained further in the article, or if you're comfortable with CLI, enter the following commands to stop and then restart the Agent services. 
    net stop "DefenseStorm Agent"
    net start "DefenseStorm Agent"

Configuration Help 

  1.  If you omit the parameters, the command will list valid options:
  2. If you make a configuration mistake, the changes are rejected
    In this case, the error is that you need your Windows Agent has been set as 'cloud only'  and a DVM must be set before you can use SendEvents  = BOTH

3. Auditing changes

Every time you successfully use the 'set' command,  the CLI generates an event in the Windows Event Log. Administrators can use the DefenseStorm GRID UI to create a trigger on this event to audit which configuration options have been changed. Note however, that if somebody changes the Windows Registry directly, this event won't be generated.


4. Recommended Performance Settings

The following section provides recommendations for the CLI options and what values to set based on your environment.  

Workstations

  • The default event processing settings should suffice for machines with minimal or light auditing configured.  If running third party applications that generate greater than 1000 events per 30 seconds, use the CLI to define an amount above your observed maximum in order to ensure monitoring fall behind.

Servers

  • Monitoring amounts for servers will depend on your organization's security auditing policies, but these can be broken down into two main categories: high event volume servers, and low event volume servers.
  •  High event volume servers (Domain Controllers, File Servers)
    • Domain controllers and file servers can generate a very high amount of logs if object access auditing is enabled, or in the presence of third-party applications such as the Netwrix Domain Compression Service.
    • For this configuration,  increase the maximum events per interval and maximum DB size, as follows:

---------------------------------------------------------------------------

 Performance Options                 |      Value

---------------------------------------------------------------------------

 ReadEventsIntervalSeconds    |    30

 SendEventsIntervalSeconds    |    30

 ReadEventsMaxEvents               |    12000

 SendEventsMaxEvents               |     12000

 MaxEventsInDb                               |     50000


  • Low event volume servers
    • Servers that generate less than 2000 events per minute should be able to use the default Workstation profile. However, the performance configuration in (1a) above can also be applied to these servers if desired - if there are less than 12000 events remaining to process, the Event Log service will retrieve all events up to the present time.

 

5. Command line samples for scripting use

This sequence invokes the CLI five times to set the values for the “high-volume server” scenario above, then restarts the Agent service to pick up the changes.

cd "c:\Program Files (x86)\DefenseStorm\Agent"
.\DefenseStorm.exe set ReadEventsIntervalSeconds 30
.\DefenseStorm.exe set SendEventsIntervalSeconds 30
.\DefenseStorm.exe set ReadEventsMaxEvents 12000
.\DefenseStorm.exe set SendEventsMaxEvents 12000
.\DefenseStorm.exe set MaxEventsInDb 50000


net stop "DefenseStorm Agent"
net start "DefenseStorm Agent"

Verifying DefenseStorm Agent configuration changes

Configuration changes made through the DefenseStorm CLI generate audit logs, which are visible in the DefenseStorm GRID.  

  1. DefenseStorm GRID UI > Events > Enter query below into Search Query
    app_name:DefenseStorm event_id:0 "DefenseStorm CLI"
    


Restarting the Agent

Restarting the Agent

To restart the Windows Agent, you can either use the Windows Services utility that comes with Windows, PowerShell, or the Command Prompt. 

1. Windows Services Restart

Open the Windows Services application and right-click the DefenseStorm Agent to restart. Depending on your version of Windows, the process and program used to restart the service may vary.

2. Command Line Restart 

The examples in this section are written for PowerShell, but the functionality works for either PowerShell or Command Prompt.

  1. Execute PowerShell  as an Administrator
  2. Enter the following command to stop the agent:
    net stop "DefenseStorm Agent"
  3. Enter the following  command to start the agent: 
    net start "DefenseStorm Agent"


Sysmon

Sysmon

Sysmon does not require an installation package, so the executable can be pushed out over a network and easily scripted.  Sysmon can be placed anywhere on the disk, the locations listed below are TRAC Team recommendations.

Installing Sysmon

  1. Download and save  Sysmon from Microsoft
  2. Unzip the folder, copy the Sysmon64 file to windows\system32
  3. Download the custom sysmon config file from github
  4. Unzip the folder and copy the xml file, sysmonconfig-export, to windows\system32
  5. After copying both files into windows\system32, you should be able to search in the folder to verify.
  6. As admin, open Terminal
    1.  Change directory into the system\32 folder location:
      cd c:\windows\system32
    2. Install the files, by running the following command:
      sysmon64.exe -accepteula -i sysmonconfig-export.xml
  7. The installation completes, and provides a verification message in the Terminal.


Moving from NXLog & PWT

Moving from NXLog & PWT to Windows Agent

  1. Uninstall NXLog  (https://nxlog.co/question/3750/how-uninstall-nxlog)
  2. Uninstall Praesidio Windows Tool 
    1. Add/Remove Programs > right-click Praesidio Windows Tool (PWT) > Uninstall 
  3. Install Windows Agent using the steps provided in this article.  For assistance, please contact support@defensestorm.com.