SMC offers a wide range of configuration options to make integration with your log management solution flexible and easy:
- Support for virtually any external syslog server with configurable field inclusion and ordering in CSV and XML formats
- Easy out-of-the-box integration with popular SIEM systems and formats, including:
- ArcSight CEF
- Q1 Labs LEEF
- McAfee ESM
- NetFlow v9
- TLS protection with optional certificate validation and TLS server identity verification
- Detailed specification of what log data to forward based on local filter elements
- Ability to apply multiple forwarding rulesets simultaneously and send the same log data to multiple destinations
This guide explains step-by-step how to configure SMC to forward log data in your chosen format to external syslog or SIEM servers. It also shows how to optionally customize which fields are included and in what order when forwarding log data in CSV, XML, or ESM formats.
- Log entry types and sources
- Supported forwarding formats
- Configuring log and audit data forwarding
- Step 1: Ensure logging is properly configured
- Step 2: Add forwarding rules
- Step 3: Allow traffic to your external server
- Field customization (CSV, XML, ESM)
- CSV and XML field customization
- ESM field customization
Log entry types and sources
Most log entries are traffic-based events that are logged according to policy rules. An audit log entry is a special type of log entry that is not traffic-based, but instead provides a record of SMC administrative actions and some internal events like element updates and scheduled task executions. Depending on your needs, you may wish to forward audit and non-audit log entries separately to different servers.
Forwarded log entries can come from two sources: Log Servers and Management Servers. A Log Server's forwarding configuration will apply to all log entries stored on that server, including its own audit log entries. Management Servers can only forward their own audit log entries.
Log forwarding must be configured for each individual Log and Management Server. Since the forwarding configuration process for Log and Management Servers is generally the same, this guide treats the two interchangeably. Any exceptions to this generalization are specified.
Important If a firewall and a Log Server are in different Domains the Log Server cannot forward traffic-based logs from that firewall to an external server. The Log Server can forward audit log entries but not traffic log entries. A new Log Server must be installed in the same Domain as the firewall in order to forward traffic log entries from that firewall to an external server.
Supported forwarding formats
Forwarded data in CSV, XML, and ESM formats may be optionally customized to define which fields are included and in what order. See the Field customization (CSV, XML, ESM) section below for more details.
Supported forwarding formats by server type
|Format||Log Server||Management Server|
|Short CSV||X|| |
|LEEF (Q1 Labs)||X||X|
|NetFlow v9||X*|| |
|Note: *NetFlow and IPFIX formats require UDP and do not support audit log entry forwarding. When forwarding traffic logs in NetFlow or IPFIX format, the entry’s Source IP field is the IP address of the Log Server forwarding the log entry. The originating log source, for example an NGFW engine, is identified by the Observation Domain ID (IPFIX) or Source ID (NetFlow) field.|
Configuring log and audit data forwarding
Step 1: Ensure logging is properly configured
|Note: Audit log entries do not require any configuration in your policy logging rules as they are always created for audit events.|
For log entries to be forwardable, you must first ensure that they are generated and properly configured in your policy logging rules. The following requirements should be met for log entries to be forwardable:
Logging Rule Options example
- The Log Level of your logging rule must not be None.
- If you are forwarding logs in NetFlow or IPFIX format, Log Accounting Information must be selected as the Connection Closing option of your logging rule to collect traffic volume information.
- If you have enabled log pruning, the log entries you want to forward must not be removed by an Immediate Discard filter.
Step 2: Add forwarding rules
Forwarding rules define what, where, and how log and audit entries are forwarded. Each Log or Management Server must have its own forwarding rules configured to forward its log and audit entries. Multiple forwarding rulesets may be active simultaneously and the same log data can be forwarded to multiple destinations.
To add forwarding rules for a Log or Management Server:
Log Server Properties, Log Forwarding tab example
- Open the Properties view of your Log or Management Server, select the Log or Audit Forwarding tab, and click Add.
- Enter your forwarding rules, including Target Host, Service, Port, and Format.
|Note: NetFlow and IPFIX formats require UDP.|
- To limit which log entries to forward, select a Data Type and/or a local Filter.
|Tip: See Knowledge Base article 12327 for an example of how to create a forwarding rule filter.|
- (Optional) To enable TLS protection: select TCP with TLS under Service, choose a TLS Profile, and define your preferences under the Server TLS Certificate Used for Forwarding Logs section. You may also optionally define a TLS Server Identity for verifying your external server.
- Click OK. Your new log forwarding rule is activate immediately.
Step 3: Allow traffic to your external server
Log and Management Server forward logs to external server. Thus if your external server and your Log or Management Server are separated by a firewall, you must configure your policy access rules to allow your forwarded data to pass through on the port and protocol selected in your log forwarding rules. Refer to your NGFW Product Guide for more information on configuring policy access rules.
Field customization (CSV, XML, ESM)
Forwarded log and audit data in CSV, XML, and ESM formats may be optionally customized to define which fields are included and in what order. A list of available fields for export and forwarding can be found in the Log Fields Appendix of your NGFW Product Guide, or from your SMC installation's datatype files in <install_dir>/data/fields/datatypes. These datatype files also determine the default field configuration used if no custom template location is defined in your server's configuration file or the template cannot be accessed.
CSV and XML field customization
To choose which log fields are forwarded in CSV or XML format, perform the following for each Log/Management Server that requires field customization:
- Create and customize your template:
- Create a new XML file or copy an existing template from the <install_dir>/data/fields/syslog_templates directory for editing.
|Note: Each Log/Management Server may only use one template file for all log and audit data forwarding in CSV or XML format. If you have a Log Server and a Management Server on the same machine, they may each use a separate template file.|
- Edit the template to suit your needs. The order in which fields appear in your template will determine the order in which fields appear in your forwarded logs.
- Configure your Log/Management server to use your customized template:
- Open your server's configuration file for editing: <install_dir>/data/LogServerConfiguration.txt for Log Servers or <install_dir>/data/SGConfiguration.txt for Management Servers.
- Edit or add the SYSLOG_CONF_FILE parameter to point to the absolute path of your custom template file.
- Restart your Log/Management Server for your changes to take effect.
ESM field customization
To choose which log fields are forwarded in ESM format, perform the following for each Log/Management Server that requires field customization:
- Edit the existing ESM template found at <install_dir>/data/fields/syslog_templates/esm_syslog_conf.xml to suit your needs.
- Restart your Log/Management Server for your changes to take effect.
Important Default templates like the ESM template in the syslog_templates directory may be overwritten during an SMC upgrade. It is recommended to make a backup copy of your custom ESM template for reversion in case your original is lost.
See Knowledge Base article 10010 for additional information on field customization.
Keywords: log forwarding; syslog; siem; cef; leef; csv; xml