FreeBSD Manual Pages
LOGSURFER(4) Kernel Interfaces Manual LOGSURFER(4) NAME logsurfer.conf - configuration file for logsurfer SYNOPSIS /usr/local/etc/logsurfer/logsurfer.conf DESCRIPTION The logsurfer.conf file is used to configure the logsurfer program. It contains initial rule definitions. Be careful if you need to debug your configuration file: the logsurfer is able to create or delete rules at runtime --- see NOTES. The configuration file uses regular expressions to specify match pat- terns for message lines. Unfortunately there are a lot of different regular expression definitions available. The logsurfer program uses the definition of regex within egrep(1) as defined by the POSIX stan- dard. For a complete description of the available operators you should read the excellent documentation of the GNU regex-library (which is part of the logsurfer distribution). Lines in the configuration file that have a # in the first column are handled as comments and are not processed. If the character in the first column is a whitespace (space or tab) the line is processed as a continuation of the previous line. All other starting characters are interpreted as a beginning of a new rule. There are three different methods of quoting for all strings: If no quotes are used the string is terminated at the first whitespace char- acter (space or tab). If you use single quotes (') the string is termi- nated at the first single quote following the initial one and the con- tents are used "as is". If you use double quotes (") for a string, the string is terminated by the corresponding quote. In this case the back- slash character is the "escape" character (that is: you can use \" to specify a double quote which is not the end of the string). Be careful to escape all backslashes by an extra backslash. To avoid confusion it is advised to always quote your strings. If you use double quoted strings in the action part (see below), than you can use special vari- ables $0 to $9 to specify submatches within the matching regular ex- pression. $0 refers to the entire message line, $1 to the string that has matched the given regular expression (the match_regex as described later on) and the other variables may reference to submatches within the regular expression (these are regular expressions within the com- plete regular expression, that are enclosed in brackets --- for more information read the documentation on regular expressions). The basic format of a rule definition is: match_regex not_match_regex stop_regex not_stop_regex timeout [con- tinue] action match_regex If this (required) regular expression matches the logline to be processed then the rest of this rule is being parsed. Otherwise the logline does not match this rule and the logsurfer tries to find an- other match in one of the following rule defini- tions. not_match_regex If this regex is not a "-" then it is considered to be a regular expression that must not match against the logline. With the help of this second regex you are able to specify rule like "Match this expres- sion except this other one". stop_regex If the stop_regex is not "-" and the logline matches against this regex then the rule is being deleted from the list of active rules (see also the following not_stop_regex definition). This is espe- cially useful for dynamically created rules that should only be active until another message is found. not_stop_regex If this regex is not "-" then it defines the pat- tern that must not match against the logline. It is only processed if you have given a stop_regex. The combination of stop_regex and not_stop_regex speci- fies a "if the first regex matches and the second one does not: then delete this rule". timeout Another method to generate rules that are only ac- tive for a certain amount of time is to specify the lifetime in seconds. If the timeout value is 0 then the rule will not timeout. Otherwise the integer value specifies the number of seconds this rule should be active. continue If this optional keyword is given, than matching of the logline against rules is not stopped if the current rule matches. The logsurfer program will continue to find another matching rule after pro- cessing the current rule. action The action of a rule is defined by one keyword and optional arguments. The complete syntax of possible actions for rules are described in the next para- graph. Within rules you are able to specify one of the following action types: ignore No further actions are initiated. This can be used to fil- ter some lines with known expressions that you want to ig- nore. exec The first argument is the program that should be invoked. You can specify other arguments which are being passed as arguments to the program. As said in the quoting section you are also allowed to use variables describing submatches of the match_regex if you are using double quotes ("). pipe This is similar to the exec definition except that the in- voked program gets the actual logline from stdin. open With the open command you are able to open a new context. See the following section about contexts for a description of the format. If a context with the given match_regex (part of the following context definition) already exists, then no new context is being opened and the open command does nothing. delete Contexts are currently identified by their exact match_regex string. If you specify an existing match_regex as an argument to the delete definition, then the specified context is being closed and deleted without activating the default_action for the context. report The first argument specifies the external program (incl. options) that should be invoked. All further arguments specify context definitions which are summarized and feeded as standard input to the invoked program. rule This option allows you to create new rules. The first word following the keyword "rule" must be one of the following: "before" (the new rule is inserted before the current rule), "behind" (the new rule is inserted behind the cur- rent rule), "top" (insert at the top of all rules) or "bot- tom" (append at the end of all existing rules). Any follow- ing keywords should again be in the format of the rule def- inition. echo The echo action simply echos the next argument string to stdout. This is useful if you want to output a simple string without invoking a separate process to do so. If the first argument is ">file", or ">>file" then the second ar- gument is written to the specified file, either overwriting or appending the file respectively. syslog This allows you to send messages into syslog. The first keyword following the syslog action is the syslog facility and level in the format (facility):(level) where facility must be one of auth, authpriv, cron, daemon, ftp, kern, lo- cal0, local1, local2, local3, local4, local5, local6, lo- cal7, lpr, mail, news, syslog, user, uucp and level must be one of emerg, alert, crit, err, warn, notice, info, debug. The second argument to the syslog action is a string to send to syslog, which should be surrounded by quotes. Once a logline has matched a rule this rule perform certain actions. Rules form the "active" part of the logsurfer. Contexts are a kind of "big box" where messages that match certain regular expressions are stored. They are "passive" and can be used by other actions like re- ports to retrieve the stored information. The idea is to store all mes- sages (for a certain period of time) in contexts (even if they seem to be unimportant). If you detect some unusual activity later on you might be interested in those messages, that are normally not important. In- stead of re-reading the loginformation (which maybe not possible --- e.g. if you are reading from standard input) you use the created con- texts for the retrieval of the message categories. Example: you might want to put all actions from one ftp-session into one context. If noth- ing "unusual" happens then you are not interested in the complete ses- sion and delete it after the user has logged out. But if the user tries to alter or upload files then you are interested in getting the whole session to see what this user has done before he triggered the alert. Be careful: the order of rules is important because the logsurfer stops finding matches after the first matching rule that has no "continue" keyword. Also the performance of the logsurfer maybe better if you put the rules with the most matches (the rule that is being used for most messages --- e.g. an ignore rule for certain actions) at the top. Contexts have the following format: match_regex match_not_regex line_limit timeout_abs timeout_rel [min_lines] default_action match_regex Similar to the match_regex of rules the first regu- lar expressions must match against the logline to be stored in this context. match_not_regex If this regular expression is not "-" then the log- line must not match against this regex to be stored in this context. line_limit The maximum number of lines that you want to store in this context. It is highly recommended to set a limit on all your contexts to avoid memory problems as a result of an error in the configuration or due to a large amount of logmessages which you haven't seen before. You may want to set the limit to a very high value like 5000 or 10000 so that it usu- ally won't be reached but it is still a protection if something goes really wrong. timeout_abs If you create a context you are able to set a time limit (in seconds) specifying how long this context should be active. If this timeout value is reached then the associated default_action is launched. Ex- ample: you might want to collect everything that is coming from a particular host or domain but forget the collected data after 24 hours (so if you create a report as a result of another important logline and you report all actions coming from this host then you are not interested in old messages from the previous day). timeout_rel In addition to the absolute timeout you are also able to specify a relative timeout specifying the number of seconds since the last message was added to this context. This is a kind of inactive timer you can use to launch the default action if there are no new messages stored in this context for a certain amount of time. min_lines An optional parameter specifying the minimum number of lines matched by a context before the action will be performed. The action will be performed if and only if it has collected min_lines or more log lines, otherwise the context will simply be deleted without any action. Note that min_lines does not in itself trigger the action - only a timeout or max_lines will do so. Default is 0 (no check for min_lines) default_action This is the action that is processed if the linenumber limit or one of the timeouts are reached. The possible actions are the same as de- scribed in the rules definition except the "active" part to modify other rules or contexts: you cannot use "open", "delete" or "rule" actions. All externals programs (for example in the "exec", "pipe" and "report" actions) must be given with a full pathname. They are started in the background and the logsurfer continues to process message lines while the other programs are running. The idea was not to stop message pro- cessing if an external program hangs or takes a long time to finish. If you need to specify a context (for example in the "open", "delete" and "report" actions) you have to do this by giving the exact regular expression that this context uses for matching (match_regex), alterna- tively if you specify "-" as the context, then logsurfer will apply the contents of the context under which the action is being performed. The timeout functions require timestamps for each message that is processed. To be independent of the message format the logsurfer uses the time when he first sees the message as an internal timestamp. Time- out values are always working on those timestamps. This might give some unexpected results if you don't follow the end of a logfile but instead use the logsurfer to process one logfile once. In this case all time- stamps are from the current time instead of the time the message was created. EXAMPLES The logsurfer program was designed to work with any kind of ascii loginformation. Most people may want to use the program to monitor the syslog-messages (for example stored in /var/adm/messages). There are several things you should consider if you use this program to monitor such files. One important thing is to realize that the contents of the logmessages are usually under the control of (possible remote) user. This is especially important if you invoke external programs and use parts of the message as arguments or as standard input (exec or pipe actions). Remember that those messages may contain any arbitrary data and that the invoked program may not be able to handle this or may al- low certain actions that are under control of the person who created the message. This may open a big security hole! For example: do not use /bin/mail or mailx to mail the message or a report to an email-ad- dress. Those programs have special escape-features "~" that can be used to invoke other programs or to modify files! Another known pitfall is to open a context for a hostname. Remember, that a message is stored in this context if the hostname appears any- where in the line. Especially if you forward syslog() messages to an- other host then the messages file may contain the name of the host who forwarded the message to the central loghost. To avoid matching in the hostname part you might want to ignore the first 20 chars using "^.{20,}hostname" or describe the format of the logmessage file in de- tail. Example: ignore the first 16 chars (the timestamp) and the first following word (the hostname) "^.{16}[^ ]*hostname". Another problem is the use of submatches in new definitions of regular expressions. For example: the hostname may contain dots "." which is interpreted (while matching lines against match_regex definitions) as "any character". You will usually not miss entries but also match a little bit more than expected. Let's say you have a hostname like "what.is.this.de" and create a context using this as match_regex. This regex will also match strings like "what-is.this.de" or "what7is+this_de". This maybe called a bug in the program, but correct escaping of regular expressions is not trivial and is currently not im- plemented in the logsurfer. Now to some real example configuration lines: '.*' - - - 0 exec "/bin/echo $0" This is a very simple testrule that matches everything ('.*') without exceptions (the first "-"). It has no "self-destroy matching rules (the next to "-") and no timeout (the "0"). For each message line it invokes the external command "/bin/echo" with the complete message line as an argument. The result will be, that every input line is echoed to the standard output. If you put the example line in a file called "test- conf" then you might want to use "logsurfer -c testconf" to start the logsurfer program with this configuration file. Every line you type in should be echoed. 'ftpd\[([0-9]*)\]: connection from' - - - 0 open "ftpd\\[$2\\]:" - 4000 10800 1800 ignore This line will open a new context for each new ftp connection. The con- text will include every logline that contains "ftpd[<pid>]:" (with pid the process id of the ftp-session) in it. One session has a maximum of 4000 lines and is a maximum of 3 hours long. It is expected that there will be at least every 30 minutes a new ftp-command - otherwise the de- fault action will be executed. Of course you won't use ignore in real life (this was just to shorten the example here). 'ftpd\[([0-9]*)\]: FTP session closed' - - - 0 delete "ftpd\\[$2\\]:" We delete (forget) the collected context if the ftp session ends. 'ftpd\[([0-9]*)\]: failed login from ([^ ]*) \[([0-9.]*)' - - - 0 report "/usr/lib/sendmail -odq root" "ftpd\\[$2\\]:" "^.{19,}$3" "^.{19,}$4" If you get a failed login via FTP you will sent all information about the current ftp-session, about the hostname and about the ip-address via sendmail to the sysadmin (root). This example assumes, that you also have opened contexts for the hostname "^.{19,}$3" and the ip-addr. The string "^.{19,}" was used to not match the hostname in the first 19 chars (in the syslog information this is the host that has generated the syslog message). 'ftpd\[([0-9]*)\]: cmd failure - not logged in' - - - 0 rule before "ftpd\\[($2)\\]: FTP session closed" - '.*' - 1800 report "/usr/lib/sendmail -odq root" "ftpd\\[$2\\]:" If someone has tried to issue an ftp command without being logged in then you add another rule, that waits for the end of the ftp-session, sends the summary of the ftp-session via sendmail to root and deletes itself (we do not need this rule any longer, because the ftp-session has been closed). Remember to put this rule before the general rule that handles the "FTP session closed" message or this rule won't be reached. This configuration examples might look a little bit confusing but if you play a with the configuration facilities you will learn how to use them. FILES /usr/local/etc/logsurfer/logsurfer.conf default configuration file /dev/null dump of the rules and contexts SEE ALSO logsurfer(1), swatch(8) NOTES If you need to debug or control the work of the logsurfer it is impor- tant to be able to get an image of the currently active rules and con- texts. This can be done be sending special signals to the logsurfer program to initiate dumping of the state. This is discussed in the NOTES section of the logsurfer manpage. If you work on messages that were written by the syslog daemon you may loose some information if the daemon summarizes repeated messages. In- stead of the original message you might get a message like "last mes- sage repeated X times". Thu Oct 19 1995 LOGSURFER(4)
NAME | SYNOPSIS | DESCRIPTION | EXAMPLES | FILES | SEE ALSO | NOTES
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=logsurfer.conf&sektion=4&manpath=FreeBSD+Ports+14.3.quarterly>