lnav/docs/_posts/2024-01-09-hammered.md

---
layout: post
title: Hammered Challenge
excerpt: Use lnav for the cyberdefenders Hammered challenge
---

I recently stumbled on this nice [review of lnav](https://lopes.id/2023-lnav-test/)
by José Lopes.  They use this [Hammered](https://cyberdefenders.org/blueteam-ctf-challenges/42)
challenge by [cyberdefenders.org](https://cyberdefenders.org) as a way to get to
know how to use lnav.  I thought I would do the same and document the commands
I would use to give folks some practical examples of using lnav.

(Since I'm not well-versed in forensic work, I followed this great
[walkthrough](https://forensicskween.com/ctf/cyberdefenders/hammered/).)

#### Q1: Which service did the attackers use to gain access to the system?

We can probably figure this out by looking for common failure messages
in the logs.  But, first, we need to load the logs into lnav.  You
can load all of the logs by passing the path to the `Hammered` directory
along with the `-r` option to recurse through any subdirectories:

```console
lnav -r Hammered
```

Now that the logs are loaded, you can use the `.msgformats` SQL command
to execute a canned query that finds log messages with a common text
format.  (Unfortunately, this command has suffered from bitrot and is
broken in the current release.  It will be fixed in the next release.
In the meantime, you can copy the [snippet](#msgformatlnav) below
to a file and execute it using the `|` prompt.)  You can enter the
SQL prompt by pressing `;` and then entering the command or statement:

```
;.msgformats
```

The top results I get for this batch of logs look like the following.

```
┏━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃total┃log_line┃       log_time        ┃  duration  ┃    log_formats    ┃                                                   log_msg_format                                                   ┃
┡━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│15179│     798│2010-03-16 08:12:09.000│47d14h59m04s│syslog_log         │#): session closed for user root                                                                                    │
│14500│     817│2010-03-16 08:17:01.000│47d14h54m00s│syslog_log         │#): session opened for user root by (#)                                                                             │
│14480│   29380│2010-04-19 04:36:49.000│7d04h03m45s │syslog_log         │pam_unix(sshd:auth): #; #                                                                                           │
│14478│   29381│2010-04-19 04:36:49.000│7d04h03m45s │syslog_log         │#): #; logname= #                                                                                                   │
│ 6300│   74477│2010-04-20 06:57:11.000│6d03h00m42s │syslog_log         │: [#]: IN=# OUT=# MAC=# SRC=# DST=# LEN=# TOS=# PREC=# TTL=# ID=# PROTO=# SPT=# DPT=# LEN=#                         │
│ 5848│    4695│2010-03-18 11:38:04.000│38d21h13m39s│syslog_log         │#): #; logname= #                                                                                                   │
│ 5479│   16164│2010-03-29 13:23:46.000│27d19h27m58s│syslog_log         │Failed password for root from # port # #                                                                            │
...
```

The `#` in the `log_msg_format` column are the parts of the text
that vary between log messages.  For example, the most interesting
message is "Failed password for root from # port # #".  In that case,
the first `#` would be the IP address and then the port number.  The
first column indicates how many times a message like this was found,
so 5,479 failed password attempts is probably a good sign of a breakin
attempt.

To find out the service that logged this message, you can scroll down
to focus on the message and then press `Shift` + `Q` to return to the
LOG view at the line mentioned in the `log_line` column.  In this case,
line 16,164, which contains:

```
Mar 29 13:23:46 app-1 sshd[21492]: Failed password for root from 10.0.1.2 port 51771 ssh2
```

So, the attack vector is `sshd`.


##### msgformat.lnav

The `;.msgformats` command has been broken for a few releases, but
its functionality can be replicated using the script below.
Copy the following to a file named `msgformat.lnav` and place it in the
`formats/installed` lnav configuration directory.

```
;SELECT count(*) AS total,
       min(log_line) AS log_line,
       min(log_time) AS log_time,
       humanize_duration(timediff(max(log_time), min(log_time))) AS duration,
       group_concat(DISTINCT log_format) AS log_formats,
       log_msg_format
    FROM all_logs
    GROUP BY log_msg_format
    HAVING total > 1
    ORDER BY total DESC
:switch-to-view db
```

#### Q2: What is the operating system version of the targeted system? (one word)

The answer to this question has the form `4.*.*.u3` as given in the
challenge.  You can do a search in lnav by pressing `/` and then
entering a PCRE-compatible regular expression.  In this case,
entering `4\.[^ ]+u3` will locate lines with the desired version
number of `4.2.4-1ubuntu3`.

#### Q3: What is the name of the compromised account?

Using the findings of our initial analysis, the compromised account
is `root`.

#### Q4: Consider that each unique IP represents a different attacker. How many attackers were able to get access to the system?

Answering this question will require analyzing messages in the `auth.log`
file.  Specifically, we will need to find failed password attempts, like
the following one and extract the user ID and IP address:

```
Apr 18 18:22:07 app-1 sshd[5266]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.151.246.140  user=root
```

The failed attempts will give us the attacker IP addresses.  However, we
don't want to confuse attacker IPs with legitimate logins.  So, we'll
need to look for successful login messages like this one:

```
Mar 16 08:26:06 app-1 sshd[4894]: Accepted password for user3 from 192.168.126.1 port 61474 ssh2
```

Analyzing log data in lnav is done through the SQL interface.  The
log messages can be accessed through SQL tables that are automatically
defined for each log format.  However, that is pretty cumbersome
since there would be a lot of regex SQL function calls cluttering up
the queries.  Instead, we can use the [`:create-search-table`](https://docs.lnav.org/en/v0.11.2/usage.html#search-tables)
command to create a SQL table that matches a regular expression
against the log messages and extracts data into column(s).  We can
then write much simpler SQL queries to get the data we're interested
in.

First, lets create an `auth_failures` table for the authentication
failure log messages:

```
:create-search-table auth_failures authentication failure; .* rhost=(?<ip>\d+\.\d+\.\d+\.\d+)\s+user=(?<user>[^ ]+)
```

Now, let's try it out by finding the IPs of failed auth attempts:

```sql
;SELECT DISTINCT ip FROM auth_failures
```

Next, lets create an `auth_accepted` table for the successful
authentications:

```
:create-search-table auth_accepted Accepted password for (?<user>[^ ]+) from (?<ip>\d+\.\d+\.\d+\.\d+)
```

Now that we have these two tables, we can write a query that
gets the IPs of failed auth attempts that eventually
succeeded.  We further filter out low failure counts to
eliminate human error.  The full query is as follows:

```sql
;SELECT ip, count(*) AS co FROM auth_failures WHERE user = 'root' AND ip IN (SELECT DISTINCT ip FROM auth_accepted) GROUP BY ip HAVING co > 10
```

The results are the following six IPs:

```
┏━━━━━━━━━━━━━━━┳━━━━┓
┃      ip       ┃ co ┃
┡━━━━━━━━━━━━━━━╇━━━━┩
│61.168.227.12  │ 386│
│121.11.66.70   │2858│
│122.226.202.12 │ 626│
│219.150.161.20 │3120│
│222.66.204.246 │1016│
│222.169.224.197│ 358│
└━━━━━━━━━━━━━━━┴━━━━┘
```

#### Q5: Which attacker's IP address successfully logged into the system the most number of times?

The attacker IPs were found using the query in the previous
question, but the counts are for the number of failed auth
attempts.  Probably the easiest thing to do is create a SQL
view with the previous query.  That can be done quickly by
pressing `;` and then pressing the up arrow to go back in
the command history.  Then, go to the start of the line and
prepend `CREATE VIEW attackers AS ` before the `SELECT`.
That will create an `attackers` SQL view that we can use
to answer this question.

Now that we can easily get the list of attacker IPs, we
can write a query for the `auth_accepted` table that
finds all the successful auth messages.  We then group
by IP and count to get the data we want:

```sql
;SELECT ip, count(*) AS co FROM auth_accepted WHERE ip IN (SELECT ip FROM attackers) GROUP BY ip ORDER co DESC
```

The results are:

```
┏━━━━━━━━━━━━━━━┳━━┓
┃      ip       ┃co┃
┡━━━━━━━━━━━━━━━╇━━┩
│219.150.161.20 │ 4│
│122.226.202.12 │ 2│
│121.11.66.70   │ 2│
│222.169.224.197│ 1│
│222.66.204.246 │ 1│
│61.168.227.12  │ 1│
└━━━━━━━━━━━━━━━┴━━┘
```

The top IP there is `219.150.161.20`.

#### Q6: How many requests were sent to the Apache Server?

Logs that follow the Apache log format can be accessed by the
`access_log` SQL table.  The following query will count the
log messages in each access log file:

```sql
;SELECT log_path, count(*) FROM access_log GROUP BY log_path
```

The results I get are:

```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃                       log_path                        ┃count(*)┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│/Users/tstack/Downloads/Hammered/apache2/www-access.log│     365│
│/Users/tstack/Downloads/Hammered/apache2/www-media.log │     229│
└━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┴━━━━━━━━┘
```

It seems like they want just what is in the `www-access.log`
file, so the answer is 365.

#### Q7: How many rules have been added to the firewall?

Rules are added by the `iptables -A` command, so we can do a
search for that command and the status bar will show
"6 hits for “iptables -A”".

#### Q9: When was the last login from the attacker with IP 219.150.161.20? Format: MM/DD/YYYY HH:MM:SS AM

Using the `auth_accepted` table we created previously, this is
a pretty simple query for `max(log_time)`:

```sql
;SELECT max(log_time) FROM auth_accepted WHERE ip = '219.150.161.20'
```

The result I get is:

```
✔ SQL Result: 2010-04-19 05:56:05.000
```

#### Q10: The database displayed two warning messages, provide the most important and dangerous one.

The database log messages come out in the syslog with a procname
of `/etc/mysql/debian-start` and are recognized as warnings.
Using this, we can write a [filter expression](https://docs.lnav.org/en/v0.11.2/commands.html#filter-expr-expr)
that filters the log based on SQL expression.  For the syslog
file format, the procname is accessible via the `:log_procname`
variable and the log level is in the `:log_level` variable.
The following command puts this together:

```
:filter-expr :log_procname = '/etc/mysql/debian-start' AND :log_level = 'warning'
```

After running this command, you should only see about 15 lines
of the 100+k that was originally shown.  Taking a look at these
lines, the following line seems pretty bad:

```
Mar 18 10:18:42 app-1 /etc/mysql/debian-start[7566]: WARNING: mysql.user contains 2 root accounts without password!
```

To clear the filter, you can press `CTRL` + `R` to reset the
state of the session.

#### Q12: Few attackers were using a proxy to run their scans. What is the corresponding user-agent used by this proxy?

The user-agent can be retrieved from the `cs_user_agent`
column in the `access_log` table.  The following query
can will get the unique user-agent names:

```sql
;SELECT DISTINCT cs_user_agent FROM access_log
```

The results I get are:

```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                                                         cs_user_agent                                                          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│Apple-PubSub/65.12.1                                                                                                            │
│Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)                                                                              │
│Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)                                                                              │
│iearthworm/1.0, iearthworm@yahoo.com.cn                                                                                         │
│Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1045 Safari/532.5          │
│WordPress/2.9.2; http://www.domain.org                                                                                          │
│Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19                                    │
│Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10│
│Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3                                 │
│pxyscand/2.1                                                                                                                    │
│-                                                                                                                               │
│Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/customer_webtxt_02.jsp)                                            │
│Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 │
│Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1059 Safari/532.5          │
└━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┘
```

The `pxyscand/2.1` name seems to be the one they want.