mirror of
https://github.com/opnsense/docs
synced 2024-11-18 21:28:29 +00:00
247 lines
12 KiB
ReStructuredText
247 lines
12 KiB
ReStructuredText
=========================
|
||
ET Pro Telemetry edition
|
||
=========================
|
||
|
||
Todays cybersecurity engineers need timely and accurate data about eminent threats and how they spread around the globe.
|
||
With this data cybersecurity researchers and analysts can improve the detection of malicious network traffic.
|
||
The times when we could rely on just firewall rules for our protection are long gone.
|
||
Additional layers of security are desperately needed to guard against these attacks.
|
||
|
||
With growing risks the need to fortify our security is growing for both big enterprises as for SMEs alike, but often
|
||
out of reach for the latter.
|
||
An important extra security addition is an Intrusion Detection and Prevention System (IDS/IPS).
|
||
|
||
The IDS/IPS available in OPNsense is based on Suricata.
|
||
This open source IDS/IPS engine has proven its value in OPNsense, especially in combination with the free Proofpoint ETOpen ruleset.
|
||
|
||
The need for valuable threat detection data and the increasing importance of additional network security
|
||
has brought Proofpoint and OPNsense together.
|
||
Our joined efforts resulted in the ET Pro Telemetry edition.
|
||
|
||
The ET Pro Telemetry edition embraces our vision that sharing knowledge leads to better products.
|
||
|
||
When you allow your OPNsense system to share anonymized information about detected threats - the alerts -
|
||
you are able to use the ET Pro ruleset free of charge.
|
||
|
||
..
|
||
|
||
*The ET Pro ruleset is updated daily and covers more than 40 different categories of network behaviors,
|
||
malware command and control, DoS attacks, botnets, informational events, exploits, vulnerabilities,
|
||
SCADA network protocols, exploit kit activity, and more. If offers a great improvement over the ET open ruleset.*
|
||
|
||
--------------------------------------
|
||
Registration
|
||
--------------------------------------
|
||
|
||
When you register for this (free) service, you will share your basic (company) details with us, Deciso Sales B.V..
|
||
We will register your sensor(s) anonymized at Proofpoint.
|
||
|
||
|
||
.. Note::
|
||
|
||
When ET Pro Telemetry is activated, your OPNsense system sends data to Proofpoint. Proofpoint does not know who you are, they
|
||
only know how many sensors an account owns. Your network statistics received by Proofpoint won’t be shared with us.
|
||
|
||
|
||
Sign up for ET Pro Telemetry edition `here <https://shop.opnsense.com/>`__
|
||
|
||
|
||
--------------------------------------
|
||
Installation
|
||
--------------------------------------
|
||
|
||
After registration, we can proceed to the installation steps, which are described below.
|
||
|
||
.. Note::
|
||
|
||
To use ET Pro Telemetry, you will need to have OPNsense 19.1 or higher installed. When using an older version,
|
||
please upgrade to the latest first.
|
||
|
||
|
||
....................
|
||
plugin
|
||
....................
|
||
|
||
First we need to install the required plugin, which is responsible for collecting the telemetry data and provides access
|
||
to the ET Pro ruleset.
|
||
|
||
1. Go to :menuselection:`System --> Firmware --> Updates`
|
||
2. press "Check for updates" in the upper right corner.
|
||
3. open the tab "Plugins" and search for `os-etpro-telemetry`
|
||
4. when found, click on the [+] sign on the right to install the plugin
|
||
|
||
A screen containing the installation status should appear now and the plugin is ready for use.
|
||
|
||
....................
|
||
register token
|
||
....................
|
||
|
||
Next step is to register your token in OPNsense and enable rulesets.
|
||
|
||
1. Go to :menuselection:`Services --> Intrusion Detection --> Administration`
|
||
2. Click on the "Download" tab, which should show you a list of available rules.
|
||
3. Enable all categories you would like to monitor in the "ET telemetry" section,
|
||
if in doubt enable all and monitor the alerts later (select on the right and use the enable selected button on top)
|
||
4. At the bottom of the page there’s a block containing settings, paste the token code you received via email in **et_telemetry.token**
|
||
5. Press **save** to persist your token code
|
||
6. Press **Download & Update rules** to fetch the current ruleset
|
||
|
||
|
||
....................
|
||
Schedule updates
|
||
....................
|
||
|
||
To download the rulesets automatically on a daily bases, you can add a schedule for this task.
|
||
|
||
1. Go to :menuselection:`Services --> Intrusion Detection --> Administration`
|
||
2. Click on the "Schedule" tab
|
||
3. A popup for the update task appears, enable it using the checkbox on top, and click "save changes"
|
||
|
||
|
||
....................
|
||
Subscription status
|
||
....................
|
||
|
||
To validate your subscription, we recommend to add the widget to the dashboard.
|
||
|
||
1. Go to the dashboard :menuselection:`Lobby --> Dashboard`
|
||
2. Click on "Add widget" in the top right corner, click "Telemetry status" in the list
|
||
3. Close dialog and click "Save settings" on the right top of the dashboard
|
||
4. Open :menuselection:`Lobby --> Dashboard` again to refresh the content
|
||
|
||
When everything is setup properly and the plugin can reach Proofpoint, it will show something like:
|
||
|
||
|
||
.. image:: images/ETPRO_telemetry_widget_active.png
|
||
|
||
The status determines which ruleset your sensor will receive, **ACTIVE** or **DORMANT** your sensor will receive ET Pro ruleset,
|
||
when **DISABLED** the license conditions are not met and the ET Open ruleset will be served.
|
||
|
||
All timestamps underneath the status provide you with information when data was send or received from Proofpoint.
|
||
|
||
.. Note::
|
||
|
||
If your sensor will start sending events and heartbeats, it should switch to active after a certain amount of time.
|
||
|
||
|
||
In case your sensor can't communicate to the outside world, the widget shows an error.
|
||
|
||
.. image:: images/ETPRO_telemetry_widget_error.png
|
||
|
||
|
||
.. Note::
|
||
|
||
The system log (:menuselection:`System --> Log Files --> General`) might contain more information, search for *emergingthreats*
|
||
|
||
|
||
.. Tip::
|
||
|
||
Always check the token code first, a common mistake is adding leading or trailing spaces to the code, which will
|
||
show an error in the log (http_code starting with a 4 usually).
|
||
|
||
--------------------------------------
|
||
Information sent to Proofpoint ©
|
||
--------------------------------------
|
||
|
||
When the intrusion detection system logs events, they will be (partially) sent to Proofpoint in return for using the
|
||
ET Pro Telemetry edition.
|
||
|
||
This paragraph describes the attributes from the
|
||
`eve.json <https://suricata.readthedocs.io/en/suricata-4.1.0/output/eve/eve-json-format.html>`__ log file
|
||
that are collected to improve threat detection and the sensor health data to evaluate if the data is usable.
|
||
|
||
An example of an event is detailed below.
|
||
|
||
.. code-block:: json
|
||
|
||
{
|
||
"event_type": "alert",
|
||
"proto": "IPV6-ICMP",
|
||
"timestamp": "2018-04-17T18:38:04.498109+0200",
|
||
"in_iface": "em1",
|
||
"alert": {
|
||
"category": "Generic Protocol Command Decode",
|
||
"severity": 3,
|
||
"rev": 2,
|
||
"gid": 1,
|
||
"signature": "SURICATA zero length padN option",
|
||
"action": "allowed",
|
||
"signature_id": 2200094
|
||
},
|
||
"src_ip": "xxxx:xxxx:fec0:d65f",
|
||
"flow_id": 982154378249516,
|
||
"dest_ip": "ff01:fe00:1200:8900:0000:f000:0000:0016"
|
||
}
|
||
|
||
|
||
Network addresses are needed to identify hosts which pose a higher risk to your and other peoples network, but your internal
|
||
addresses are kept secret.
|
||
|
||
For this reason we mask the addresses found in the log file and only send the last number(s) to identify the host.
|
||
In the example above the *src_ip* is an internal IPv6 address, for IPv4 we only collect the last number (e.g. 0-255).
|
||
|
||
|
||
Fields collected (unmodified):
|
||
|
||
=====================================================================
|
||
|
||
============== ======================================================
|
||
timestamp Timestamp of the event
|
||
flow_id Internal identifier for this communication flow
|
||
in_iface Interface where the event was captured
|
||
event_type Type of event
|
||
vlan Vlan tag
|
||
src_port Source port number
|
||
dest_port Destination port number
|
||
proto Protocol
|
||
alert Alert details, such as the signature_id, action taken
|
||
and associated message.
|
||
tls TLS details, such as certificate subject and serial.
|
||
http HTTP detail information such as the host, but omitting
|
||
sensitive details such as path and user-agent.
|
||
app_proto Application protocol (if known)
|
||
============== ======================================================
|
||
|
||
*Threats change often, to keep statistics valuable, the list of fields is subject to change*
|
||
|
||
.. Note::
|
||
|
||
The plugin comes with a small script to print eve output yourself, it's called **dump_data.py**, when used with the **-p**
|
||
parameter, it will output the data as it will be sent to Proofpoint.
|
||
All script code can be found in the following directory */usr/local/opnsense/scripts/ids_telemetry/*
|
||
|
||
|
||
Sensor health status collected and send as keep-alive:
|
||
|
||
|
||
=======================================================================================================================
|
||
|
||
======================================== ===========================================================================================
|
||
Unique Sensor ID Unique sensor identification, helps identifying messages from the same system,
|
||
without knowing who is the operator.
|
||
OPNSense Version Current installed software version. This will help both for troubleshooting purposes
|
||
(if a bad update is pushed and Proofpoint notices that deployments running version
|
||
X have an issue) as well for planning, to understand how new features and
|
||
functionality would be adopted.
|
||
Suricata Version Suricata version installed.
|
||
Suricata status Reports if the sensor is active, when not active, no detection/telemetry can be provided.
|
||
System Time If the system time is not correct, it will impact the timestamps of messages,
|
||
so knowing what time the system thinks it has will help reconcile the actual time.
|
||
Active Ruleset Version The active ruleset version should match what is published.
|
||
If sensors do not have the active version then they either haven’t configured
|
||
scheduled updates or there is another issue.
|
||
This will help Proofpoint to identify if there are widespread issues with updates.
|
||
Number of rules enabled Helps to gain a better understanding about the number of rules people use on top of
|
||
the ones provided by Proofpoint.
|
||
Number of ETPro Telemetry Rules enabled Because users can control what rules they enable,
|
||
they may not want to enable all ETPro Telemetry rules,
|
||
if this is the case it would help Proofpoint understand how the rules are being
|
||
leveraged so they can better write / tune rules
|
||
Mode (IDS or IPS) This is helpful to understand how the system is deployed and is useful to
|
||
development purposes to determine what rules we should be focusing on based
|
||
on how our customers are using them.
|
||
Suricata Log Stats For QA purposes, some fields with general stats are collected
|
||
from /var/log/suricata/stats.log (capture.kernel_packets, decoder.pkts, decoder.bytes,
|
||
decoder.ipv4, decoder.ipv6, flow.tcp, flow.udp, detect.alert)
|
||
======================================== ===========================================================================================
|