Configuring Audit Events and Logging
The Session Smart Router can be configured to maintain a history of several different class of events in the event log, which can subsequently be used to support compliance audits, forensics on network issues related to configuration (misapplied or otherwise), and traceability. This document covers:
- Types of events available on the router
- Enabling the Audit events
Event Types
The events generated by the router are classified into the following categories:
Traffic Events
Traffic events are generated as sessions are created on the router. These include details such as the protocol, source address, source port, destination address and destination port. In addition, the success or failure status along with a reason code for failure cases are included in the event.
Administration Events
Various administration actions performed by a user such as SSH login generate this category of events. The events contain the details about the user action, whether or not the action was permitted, and the reason for any failures.
System Events
Various system level events such as service and process restarts are generated by this event category. The details include information about the user and details about the underlying action.
Alarm Events
All the SSR alarms generate an add event when the alarm is raised and a clear event when the alarm is cleared. The alarm events can be used to view the history of the events associated with the alarms. The alarm events are implicit events and cannot be disabled via configuration. See Alarms and Events for more details.
Provisioning Events
The provisioning events are generated for software download and upgrades as well as for configuration changes that are processed on the router. For configuration changes the event contains a diff of the configuration change that triggered the event. These are implicit events and cannot be disabled via configuration.
Basic Configuration
The configuration for audit logging is performed under the system > audit branch in the router hierarchy. In most cases, the only configuration required for enabling audit logging is adding it to the router element for your Authority's conductor. For cases where an SSR router is not managed by a conductor, audit logging configuration is added to the system > audit branch of the router hierarchy.
If auditd fails to start or is prevented from running, an immediate, real-time message is displayed to all users indicating that the audit logging capability is impacted. This message persists until the failure is resolved.
Sample Configuration
Configuration not related to audit logging has been filtered out for illustrative purposes.
Enable Basic Audit Logging
config
authority
router my-router
name my-router
system
audit
administration
enabled true
exit
exit
exit
exit
exit
exit
Set the Disk Full Action
Common Criteria compliance does not permit the system to be operated without audit logging enabled. Configuring the disk-full-action as halt ensures that the system automatically shuts down when the disk has no free space remaining to write audit logs.
config
authority
router my-router
name my-router
system
audit
disk-full-action halt
exit
exit
exit
exit
exit
Storing Events for Short Durations
By default the SSR routers store all events except traffic events for up to six months on the local disk. In some cases it might be desirable to shorten the length of time for these events to minimize the impact on the local disk.
In the following example, all the events available on the SSR router are retained for one day. The retention is of type duration and can take values in hours and days; for example, 24h or 1d.
config
authority
router my-router
system
audit
retention 1d
exit
exit
exit
exit
exit
Sending Traffic Events to a Syslog Server
Traffic events are disabled and not persisted by default because they can produce a large volume of data. However, in situations where the traffic events need to be sent off-box for storage, such as a syslog server, they can be enabled but not persisted to local storage. The following snippet provides an example of that configuration.
config
authority
router my-router
system
audit
traffic
enabled true
persist false
exit
exit
exit
exit
exit
For a detailed explanation of configuring the Monitoring Agent to handle audit events, refer to the SSR Monitoring Agent documentation.
On the syslog server:
-
Generate a private/public key using the utility
ssh-keygen -t rsa -b 4096. -
Open (or create if necessary) a known host file
/root/.ssh/known_hostsand authorize the host; prepend the IP address of the host with the public key from the SSR/etc/ssh/ssh_host_rsa_key.pub:
192.168.1.14 ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABAQC4UZe/Q8jce6c02IfFM64UcSJ/IZu3GQNLuElbzsrVZHEVu3/EfNp10acbx1PqlhSxJSJQwXe1Q1vEq6bMR8/tZU3fa6NwCt8rgGs8BT8NQuVHKj5s2CAKtBqhMHQmtngddbEHAj1WJShe3GBr4Xou1uw6o4SEo+8EjO56L3lzSK60dXOx/vDiuDFsNNUjfqD9qSRuwsHPkzdX5s6P8XTYo4OlvMPRplnhEmgczxjGeMQSPBp+vHf6uMHNOKUQqLQsA0dSVKM1CNApXuMsy7HakP1oOn9eKX/uf4VofNfrOW90PrKNd+E9jUgGiiSVc5H8QbCVmO2KhKmGh4wraGa/
On the SSR
-
Authorize the server public key.
- Copy the
id_rsa.pubfile (/root/.ssh/id_rsa.publocated on the server) and append it to end of theauthorized_keysfile (/home/admin/.ssh/authorized_keys) on the device.
- Copy the
-
Open the file
/usr/lib/128technology/sshd/config.template.fipsand change the settingAllowTcpForwarding nofrom no toAllowTcpForwarding yes. -
Create an event collector input to capture the traffic events. An example input configuration is shown below.
[[inputs.t128_events]]
# It is a best practice to specify a custom index file location
index-file = "/var/lib/128t-monitoring/state/events.index"
topic = "events"
[inputs.t128_events.tagpass]
type = ["traffic"]
Refer to Event Collector for information about creating an event collector.
- Define an output where the events are to be sent. In this example, the events are sent to a syslog server.
[[outputs.syslog]]
address = "udp://<ip>:514"
default_sdid = "128T"
- The input and output are placed in the input and output directories respectively and tied together by the Monitoring Agent configuration. A sample monitoring agent configuration:
enabled: true
inputs:
- name: traffic-events
outputs:
- name: my-syslog
- On the syslog server, start a remote forwarding tunnel to collect the syslog events:
ssh -R 127.0.0.1:514:127.0.0.1:514 admin@192.168.1.14 -o ExitOnForwardFailure=yes -i /root/.ssh/id_rsa
Once these configurations are in place, starting the Monitoring Agent application will send events to syslog.
Remote Logging
Audit logs can be stored off system by configuring a remote logging server. When the IP address and port are configured, audit logs are sent to the remote system for storage and review.
config
authority
router Fabric128
name Fabric128
system
audit
remote-logging-server 1.1.1.1 60
address 1.1.1.1
port 60
exit
exit
exit
exit
exit
Secure Audit Logs Transport
To provide secure transport of audit logs to and from a remote server, use the following procedures:
On the Audit Server:
-
Generate a private/public key using the utility
ssh-keygen -t rsa -b 4096. -
Open (or create if necessary) the known host file
/root/.ssh/known_hostsand authorize the host; prepend the IP address of the host with the public key from the SSR/etc/ssh/ssh_host_rsa_key.pub:
[192.168.1.14 ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABAQC4UZe/Q8jce6c02IfFM64UcSJ/IZu3GQNLuElbzsrVZHEVu3/EfNp10acbx1PqlhSxJSJQwXe1Q1vEq6bMR8/tZU3fa6NwCt8rgGs8BT8NQuVHKj5s2CAKtBqhMHQmtngddbEHAj1WJShe3GBr4Xou1uw6o4SEo+8EjO56L3lzSK60dXOx/vDiuDFsNNUjfqD9qSRuwsHPkzdX5s6P8XTYo4OlvMPRplnhEmgczxjGeMQSPBp+vHf6uMHNOKUQqLQsA0dSVKM1CNApXuMsy7HakP1oOn9eKX/uf4VofNfrOW90PrKNd+E9jUgGiiSVc5H8QbCVmO2KhKmGh4wraGa/
- Configure the audit server to listen on port 60. For example, using
auditd:
- Set
tcp_listen_port = 60in the file/etc/audit/auditd.conf service auditd restart
On the SSR
-
Authorize the server public key.
- Copy the
id_rsa.pubfile (/root/.ssh/id_rsa.publocated on the server) and append it to end of theauthorized_keysfile (/home/admin/.ssh/authorized_keys) on the device.
- Copy the
-
Open the file
/usr/lib/128technology/sshd/config.template.fipsand change the settingAllowTcpForwarding nofrom no toAllowTcpForwarding yes. -
Add the config to point the audit server at
localhost port 60and commit the changes.
*admin@conductor-node-1.Conductor# compare config running candidate
config
authority
router Conductor
name Conductor
system
audit
remote-logging-server 127.0.0.1 60
address 127.0.0.1
port 60
exit
exit
exit
exit
exit
exit
- Return to the audit server and start the port forwarding:
ssh -R 127.0.0.1:60:127.0.0.1:60 admin@192.168.1.14 -o ExitOnForwardFailure=yes -i /root/.ssh/id_rsa
Example Audit Logs
SSH Session Establishment Failure
type=USER_AUTH msg=audit(1709742862.344:2320): pid=13394 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:authentication grantors=? acct="?" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=failed'
type=USER_AUTH msg=audit(1709742864.269:2321): pid=13394 uid=0 auid=4294967295 ses=4294967295 msg='op=password acct="(unknown)" exe="/usr/sbin/sshd" hostname=? addr=172.18.4.99 terminal=ssh res=failed'
SSH Session Establishment Success
type=USER_AUTH msg=audit(1709742929.672:2335): pid=13700 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:authentication grantors=pam_faillock,pam_unix acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=USER_ACCT msg=audit(1709742929.674:2336): pid=13700 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=pam_faillock,pam_unix,pam_localuser acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1709742929.676:2337): pid=13700 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=session fp=? direction=both spid=13701 suid=74 rport=52572 laddr=192.168.1.10 lport=22 exe="/usr/sbin/sshd" hostname=? addr=172.18.4.99 terminal=? res=success'
type=USER_AUTH msg=audit(1709742929.678:2338): pid=13700 uid=0 auid=4294967295 ses=4294967295 msg='op=success acct="centos" exe="/usr/sbin/sshd" hostname=? addr=172.18.4.99 terminal=ssh res=success'
type=CRED_ACQ msg=audit(1709742929.678:2339): pid=13700 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_faillock,pam_unix acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=LOGIN msg=audit(1709742929.678:2340): pid=13700 uid=0 old-auid=4294967295 auid=1000 tty=(none) old-ses=4294967295 ses=36 res=1
type=SYSCALL msg=audit(1709742929.678:2340): arch=c000003e syscall=1 success=yes exit=4 a0=4 a1=7ffe4754f5c0 a2=4 a3=3 items=0 ppid=2007 pid=13700 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=36 comm="sshd" exe="/usr/sbin/sshd" key=(null)
type=PROCTITLE msg=audit(1709742929.678:2340): proctitle=737368643A2063656E746F73205B707269765D
type=USER_START msg=audit(1709742929.686:2341): pid=13700 uid=0 auid=1000 ses=36 msg='op=PAM:session_open grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_lastlog acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1709742929.687:2342): pid=13734 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:f7:68:dd:06:e8:30:8b:1b:3e:73:db:60:e6:34:9b:30:c5:0c:b4:b0:3d:7c:1b:20:d3:c2:84:05:4f:fa:d5:7f direction=? spid=13734 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER msg=audit(1709742929.687:2343): pid=13734 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:4c:2a:c1:e0:b9:fd:ce:16:c3:f0:89:16:f6:2a:70:40:ca:84:13:9c:02:58:91:4d:2a:1a:14:bc:f0:e6:f2:3c direction=? spid=13734 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER msg=audit(1709742929.688:2344): pid=13734 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:a7:25:1c:27:28:d9:a9:cc:7f:2b:6e:c4:e0:61:28:cf:31:15:8d:c5:e5:9b:e5:c5:03:24:46:23:ab:42:04:c1 direction=? spid=13734 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRED_ACQ msg=audit(1709742929.688:2345): pid=13734 uid=0 auid=1000 ses=36 msg='op=PAM:setcred grantors=pam_faillock,pam_unix acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=USER_LOGIN msg=audit(1709742929.730:2346): pid=13700 uid=0 auid=1000 ses=36 msg='op=login id=1000 exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=/dev/pts/1 res=success'
type=USER_START msg=audit(1709742929.730:2347): pid=13700 uid=0 auid=1000 ses=36 msg='op=login id=1000 exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=/dev/pts/1 res=success'
type=CRYPTO_KEY_USER msg=audit(1709742929.732:2348): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:a7:25:1c:27:28:d9:a9:cc:7f:2b:6e:c4:e0:61:28:cf:31:15:8d:c5:e5:9b:e5:c5:03:24:46:23:ab:42:04:c1 direction=? spid=13735 suid=1000 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=SYSCALL msg=audit(1709742940.326:2349): arch=c000003e syscall=159 success=yes exit=0 a0=55e40cbbf980 a1=1 a2=0 a3=55e40e52326c items=0 ppid=1 pid=6697 auid=4294967295 uid=38 gid=38 euid=38 suid=38 fsuid=38 egid=38 sgid=38 fsgid=38 tty=(none) ses=4294967295 comm="ntpd" exe="/usr/sbin/ntpd" key="128T"
SSH Session Termination
type=USER_END msg=audit(1709743019.474:2350): pid=13700 uid=0 auid=1000 ses=36 msg='op=login id=1000 exe="/usr/sbin/sshd" hostname=? addr=? terminal=/dev/pts/1 res=success'
type=USER_LOGOUT msg=audit(1709743019.474:2351): pid=13700 uid=0 auid=1000 ses=36 msg='op=login id=1000 exe="/usr/sbin/sshd" hostname=? addr=? terminal=/dev/pts/1 res=success'
type=CRYPTO_KEY_USER msg=audit(1709743019.475:2352): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:a7:25:1c:27:28:d9:a9:cc:7f:2b:6e:c4:e0:61:28:cf:31:15:8d:c5:e5:9b:e5:c5:03:24:46:23:ab:42:04:c1 direction=? spid=13734 suid=1000 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER msg=audit(1709743019.475:2353): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=session fp=? direction=both spid=13734 suid=1000 rport=52572 laddr=192.168.1.10 lport=22 exe="/usr/sbin/sshd" hostname=? addr=172.18.4.99 terminal=? res=success'
type=USER_END msg=audit(1709743019.478:2354): pid=13700 uid=0 auid=1000 ses=36 msg='op=PAM:session_close grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_lastlog acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=CRED_DISP msg=audit(1709743019.478:2355): pid=13700 uid=0 auid=1000 ses=36 msg='op=PAM:setcred grantors=pam_faillock,pam_unix acct="centos" exe="/usr/sbin/sshd" hostname=172.18.4.99 addr=172.18.4.99 terminal=ssh res=success'
type=CRYPTO_KEY_USER msg=audit(1709743019.479:2356): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:f7:68:dd:06:e8:30:8b:1b:3e:73:db:60:e6:34:9b:30:c5:0c:b4:b0:3d:7c:1b:20:d3:c2:84:05:4f:fa:d5:7f direction=? spid=13700 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER msg=audit(1709743019.479:2357): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:4c:2a:c1:e0:b9:fd:ce:16:c3:f0:89:16:f6:2a:70:40:ca:84:13:9c:02:58:91:4d:2a:1a:14:bc:f0:e6:f2:3c direction=? spid=13700 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER msg=audit(1709743019.479:2358): pid=13700 uid=0 auid=1000 ses=36 msg='op=destroy kind=server fp=SHA256:a7:25:1c:27:28:d9:a9:cc:7f:2b:6e:c4:e0:61:28:cf:31:15:8d:c5:e5:9b:e5:c5:03:24:46:23:ab:42:04:c1 direction=? spid=13700 suid=0 exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
NTP Configuration Change
The command show event type system detail shows an NTP configuration change.
====================================================================
2024-03-14T20:58:24.469Z Change to the running 128T configuration.
====================================================================
Type: admin.running_config_change
Node: test1
User: admin
Collector: configDirector
Description: running config changes are committed
Permitted: True
Source Ip: 172.18.15.253
config
authority
router Fabric128
name Fabric128
system
ntp
delete server force 172.2.10.13
server 172.2.10.14
ip-address 172.2.10.14
exit
exit
exit
exit
exit
exit
Unsuccessful Login Attempt Limit Met or Exceeded
These will appear in the sshd journal for SSH-based logins, or Dredd if it is an API-based login.
Mar 14 18:21:25 t117-dut1.openstacklocal sshd[11536]: pam_faillock(sshd:auth): Consecutive login failures for user test account temporarily locked
Mar 14 18:21:27 t117-dut1.openstacklocal sshd[11536]: Failed password for test from 172.18.15.253 port 61203 ssh2
All Use of Identification and Authentication
This information is found within the journal of sshd.
Mar 14 18:23:23 t117-dut1.openstacklocal sshd[14546]: Accepted password for test from 172.18.15.253 port 61205 ssh2
Banner Information:
$ ssh admin@Conductor
admin@10.22.0.68's password:
Last failed login: Mon Mar 18 04:07:15 UTC 2024 from 172.18.15.253 on ssh:notty
There were 2 failed login attempts since the last successful login.
Last login: Mon Mar 18 04:06:15 2024 from 172.18.15.253
+---------------------------------------+
| |
| Welcome to: |
| |
| | . . ,---. . ,---. ,---. ,--. |
| | | | | | | |---' |---' | |
| | `---' ' ' ' ' `---' ' |
| ---' |
| __ ___ __ __ __ |
| |\ | |_ | | | / \ |__) |_/ (_ |
| | \| |__ | |/\| \__/ | \ | \ __) |
| |
| Session Smart Networking Platform ... |
+---------------------------------------+
admin@conductor-node-1.Conductor#
Password-based Authentication
This information is found within the journal of sshd.
Mar 14 18:23:23 t117-dut1.openstacklocal sshd[14546]: Accepted password for test from 172.18.15.253 port 61205 ssh2
Logs for Manual Software Updates
Logs for SSR software updates can be found at /var/log/install128t/installer.log. An example would be updating from 6.3.0-develop to 6.4.0-develop.
2024-03-14 21:34:32,004: INFO - Version requirement: 6.4.0-1.develop.el7
2024-03-14 21:34:39,218: INFO - Verifying that 128T-0:6.4.0-1.develop.el7.x86_64 will be an upgrade
2024-03-14 21:34:39,218: INFO - Resolving version of Manifest matching 128T-manifest-6.3.0.0.202403021319.develop.el7
2024-03-14 21:34:42,009: INFO - Using Manifest package 128T-manifest-0:6.3.0.0.202403021319.develop.el7-1.x86_64
2024-03-14 21:34:51,737: INFO - Resolving version of Deprecated Packages file 128T-deprecated-packages-6.3.0.0.202403021319.develop.el7
2024-03-14 21:34:55,061: INFO - Using Manifest package 128T-deprecated-packages-0:6.3.0.0.202403021319.develop.el7-1.x86_64
2024-03-14 21:34:56,613: INFO - Resolving version of Manifest matching 128T-manifest-6.4.0.1.develop.el7
2024-03-14 21:35:00,501: INFO - Using Manifest package 128T-manifest-0:6.4.0.1.develop.el7-1.x86_64
2024-03-14 21:35:07,172: INFO - Resolving version of Deprecated Packages file 128T-deprecated-packages-6.4.0.1.develop.el7
2024-03-14 21:35:10,274: INFO - Using Manifest package 128T-deprecated-packages-0:6.4.0.1.develop.el7-1.x86_64