rwcount(1) SiLK Tool Suite rwcount(1) NAME rwcount - Print traffic summary across time SYNOPSIS rwcount [--bin-size=SIZE] [--load-scheme=LOADSCHEME] [--start-time=START_TIME] [--end-time=END_TIME] [--skip-zeroes] [--bin-slots] [--epoch-slots] [--timestamp-format=FORMAT] [--no-titles] [--no-columns] [--column-separator=CHAR] [--no-final-delimiter] [{--delimited | --delimited=CHAR}] [--print-filenames] [--copy-input=PATH] [--output-path=PATH] [--pager=PAGER_PROG] [--site-config-file=FILENAME] [{--legacy-timestamps | --legacy-timestamps={1,0}}] {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]} rwcount --help rwcount --version DESCRIPTION rwcount summarizes SiLK flow records across time. It counts the records in the input stream, and groups their byte and packet totals into time bins. rwcount produces textual output with one row for each bin. rwcount reads SiLK Flow records from the files named on the command line or from the standard input when no file names are specified and --xargs is not present. To read the standard input in addition to the named files, use "-" or "stdin" as a file name. If an input file name ends in ".gz", the file is uncompressed as it is read. When the --xargs switch is provided, rwcount reads the names of the files to process from the named text file or from the standard input if no file name argument is provided to the switch. The input to --xargs must contain one file name per line. rwcount splits each flow record into bins whose size is determined by the argument to the --bin-size switch. When that switch is not provided, rw- count uses 30-second bins by default. By default, the first row of data rwcount prints is the bin containing the starting time of the earliest record that appears in the input. rwcount then prints a row for every bin until it reaches the bin containing the most recent ending time. Rows whose counts are zero are printed unless the --skip-zero switch is specified. The --start-time and --end-time switches tell rwcount to use a specific time for the first row and the final row. The --start-time switch always sets the time stamp on the first bin to the specified time. With the --end-time switch, rwcount computes a maximum end-time by setting any un- specified hour, minute, second, and millisecond field to its maximum value, and the final bin is that which contains the maximum end-time. When --start-time and --end-time are both specified, rwcount reserves the memory for the bins before it begins processing the records. If the memory cannot be allocated, rwcount exits. If this happens, try reducing the time span or increasing the bin-size. Load Scheme A router or other flow generator summarizes the traffic it sees into records. In addition to the five-tuple (source port and address, destina- tion port and address, and protocol), the record has its start time, end time, total byte count, and total packet count. There is no way to know how the bytes and packets were distributed during the duration of the record: their distribution could be front-loaded, back-loaded, uniform, et cetera. When the start and end times of a individual flow record put that record into a single bin, rwcount can simply add that record's volume (byte and packet counts) to the bin. When the duration of a flow record causes it to span multiple bins, rwcount must to told how to allocate the volume among the bins. The --load-scheme switch determines this, and it has supports the following allocation schemes: time-proportional Each bin a flow spans is allocated a percentage of the flow's volume proportional to the amount of the flow's active time that spans the bin. Specifically, rwcount divides the total volume of the flow by the duration of the flow, and multiplies the quotient by the time spent in the bin. This models a flow where the volume/second ratio is uniform throughout the flow. bin-uniform Each bin a flow spans is allocated an equal portion of the flow's vol- ume. rwcount divides the volume of the flow by the number of bins the flow spans, and adds the quotient to each of the bins. In this scheme, the volume/bin ratio is uniform. start-spike The bin that contains the flow's start time is allocated all of the flow's volume regardless of the flow's duration. rwcount adds the to- tal volume for the flow into the bin containing the start time of the flow. This models a flow that is front-loaded to the point where the entire volume is a single spike occurring in the initial millisecond of flow. middle-spike The bin that contains the midpoint between the flow's start time and end time is allocated all of the flow's volume regardless of the flow's duration. end-spike The bin that contains the flow's end time is allocated all of the flow's volume regardless of the flow's duration. This models a flow that is back-loaded to the point where the entire volume is a single spike occurring in final millisecond of the flow. maximum-volume Each bin the flow spans is allocated all of the flow's volume. rwcount adds the entire volume for the flow into every bin that contains any part of the flow. In theory, the distribution of the bytes in the record could be a spike that occurs at any point during the flow's du- ration. This scheme allows one to determine, in aggregate, the maximum possible volume that could have occurred during this bin. In this scheme, the "Records" column gives the number of records that were ac- tive during the bin. minimum-volume For a record that spans multiple bins, each bin is allocated none of the flow's volume. That is, rwcount acts as though the volume for the flow occurred in some other bin. Since it is possible that a record that spans multiple bins did not contribute any volume to the current bin, this scheme allows one to determine, in aggregate, the minimum possible volume that may have occurred during this bin. The "Records" column in this scheme, as in the "maximum-volume" scheme, gives the number of flow records that were active during the bin. Be aware that the "spike" load-schemes allocate the entire flow to a single bin. This can create the impression that there is more traffic occurring during a particular time window that the physical network supports. The "maximum-volume" and "minimum-volume" schemes are used to compute the maximum and minimum volumes that could have been transferred during any one bin. "maximum-volume" intentionally over-counts the flow volume and "mini- mum-volume" intentionally under-counts. To see the effect of the various load-schemes, suppose rwcount is using 60-second bins and the input contains two records. The first record begins at 12:03:50, ends at 12:06:20, and contains 9,000 bytes (60 bytes/second for 150 seconds). This record may contribute to bins at 12:03, 12:04, 12:05, and 12:06. The second record begins at 12:04:05 and lasts 15 sec- onds; this record's volume always contributes its 200 bytes to the 12:04 bin. The --load-scheme option splits the byte-counts of the records as follows: BIN 12:03:00 12:04:00 12:05:00 12:06:00 time-proportional 600 3800 3600 1200 bin-uniform 2250 2450 2250 2250 start-spike 9000 200 0 0 middle-spike 0 200 9000 0 end-spike 0 200 0 9000 maximum-volume 9000 9200 9000 9000 minimum-volume 0 200 0 0 For the record that spans multiple bins: the "time-proportional" scheme as- sumes 60 bytes/second, the "bin-uniform" scheme divides the volume evenly by the four bins, the "middle-spike" scheme assumes all the volume occurs at 12:05:05, the "maximum-volume" scheme adds the volume to every bin, and the "minimum-volume" scheme ignores the record. OPTIONS Option names may be abbreviated if the abbreviation is unique or is an ex- act match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters. --bin-size=SIZE Denote the size of each time bin, in seconds; defaults to 30 seconds. rwcount supports millisecond size bins; SIZE may be a floating point value equal to or greater than than 0.001. --load-scheme=LOADSCHEME Specify how a flow record that spans multiple bins allocates its bytes and packets among the bins. The default scheme is "time-proportional", which assumes the volume/second ratio of the flow record is constant. See the "Load Scheme" section for additional information on the load-scheme choices. The LOADSCHEME may be one of the following names or numbers; names may be abbreviated to the shortest prefix that is unique. time-proportional,4 Allocate the volume in proportion to the amount of time the flow spent in the bin. bin-uniform,0 Allocate the volume evenly across the bins that contain any part of the flow's duration. start-spike,1 Allocate the entire volume to the bin containing the start time of the flow. middle-spike,3 Allocate the entire volume to the bin containing the time at the midpoint of the flow. end-spike,2 Allocate the entire volume to the bin containing the end time of the flow. maximum-volume,5 Allocate the entire volume to all of the bins containing any part of the flow. minimum-volume,6 Allocate the flow's volume to a bin only if the flow is completely contained within the bin; otherwise ignore the flow. --start-time=START_TIME Set the time of the first bin to START_TIME. When this switch is not given, the first bin is one that holds the starting time of the earli- est record. The START_TIME may be specified in a format of "yyyy/mm/dd[:HH[:MM[:SS[.sss]]]]" (or "T" may be used in place of ":" to separate the day and hour). The time must be specified to at least day precision, and unspecified hour, minute, second, and millisecond values are set to zero. Whether the date strings represent times in UTC or the local timezone depend on how SiLK was compiled, which can be determined from the "Timezone support" setting in the output from rw- count --version. Alternatively, the time may be specified as seconds since the UNIX epoch, and an unspecified milliseconds value is set to 0. --end-time=END_TIME Set the time of the final bin to END_TIME. When this switch is not given, the final bin is one that holds the ending time of the latest record. The format of END_TIME is the same as that for START_TIME. Unspecified hour, minute, second, and millisecond values are set to 23, 59, 59, and 999 respectively. When END_TIME is specified as seconds since the UNIX epoch, an unspecified milliseconds value is set to 999. When both --start-time and --end-time are used, the END_TIME is ad- justed so that the final bin represents a complete interval. --skip-zeroes Disable printing of bins with no traffic. By default, all bins are printed. --bin-slots Use the internal bin index as the label for each bin in the output; the default is to label each bin with the time in a human-readable format. --epoch-slots Use the UNIX epoch time (number of seconds since midnight UTC on 1970-01-01) as the label for each bin in the output; the default is to label each bin with the time in a human-readable format. This switch is equivalent to --timestamp-format=epoch. This switch is deprecated as of SiLK 3.11.0, and it will be removed in the SiLK 4.0 release. --timestamp-format=FORMAT Specify the format and/or timezone to use when printing timestamps. When this switch is not specified, the SILK_TIMESTAMP_FORMAT environ- ment variable is checked for a default format and/or timezone. If it is empty or contains invalid values, timestamps are printed in the de- fault format, and the timezone is UTC unless SiLK was compiled with lo- cal timezone support. FORMAT is a comma-separated list of a format and/or a timezone. The format is one of: default Print the timestamps as "YYYY/MM/DDThh:mm:ss". iso Print the timestamps as "YYYY-MM-DD hh:mm:ss". m/d/y Print the timestamps as "MM/DD/YYYY hh:mm:ss". epoch Print the timestamps as the number of seconds since 00:00:00 UTC on 1970-01-01. When a timezone is specified, it is used regardless of the default timezone support compiled into SiLK. The timezone is one of: utc Use Coordinated Universal Time to print timestamps. local Use the TZ environment variable or the local timezone. --no-titles Turn off column titles. By default, titles are printed. --no-columns Disable fixed-width columnar output. --column-separator=C Use specified character between columns and after the final column. When this switch is not specified, the default of '|' is used. --no-final-delimiter Do not print the column separator after the final column. Normally a delimiter is printed. --delimited --delimited=C Run as if --no-columns --no-final-delimiter --column-sep=C had been specified. That is, disable fixed-width columnar output; if character C is provided, it is used as the delimiter between columns instead of the default '|'. --print-filenames Print to the standard error the names of input files as they are opened. --copy-input=PATH Copy all binary SiLK Flow records read as input to the specified file or named pipe. PATH may be "stdout" or "-" to write flows to the stan- dard output as long as the --output-path switch is specified to redi- rect rwcount's textual output to a different location. --output-path=PATH Write the textual output to PATH, where PATH is a filename, a named pipe, the keyword "stderr" to write the output to the standard error, or the keyword "stdout" or "-" to write the output to the standard out- put (and bypass the paging program). If PATH names an existing file, rwcount exits with an error unless the SILK_CLOBBER environment vari- able is set, in which case PATH is overwritten. If this switch is not given, the output is either sent to the pager or written to the stan- dard output. --pager=PAGER_PROG When output is to a terminal, invoke the program PAGER_PROG to view the output one screen full at a time. This switch overrides the SILK_PAGER environment variable, which in turn overrides the PAGER variable. If the --output-path switch is given or if the value of the pager is de- termined to be the empty string, no paging is performed and all output is written to the terminal. --site-config-file=FILENAME Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, rwcount searches for the site configura- tion file in the locations specified in the "FILES" section. --legacy-timestamps --legacy-timestamps=NUM When NUM is not specified or is 1, this switch is equivalent to --time- stamp-format=m/d/y. Otherwise, the switch has no effect. This switch is deprecated as of SiLK 3.0.0, and it will be removed in the SiLK 4.0 release. --xargs --xargs=FILENAME Read the names of the input files from FILENAME or from the standard input if FILENAME is not provided. The input is expected to have one filename per line. rwcount opens each named file in turn and reads records from it as if the filenames had been listed on the command line. --help Print the available options and exit. --version Print the version number and information about how SiLK was configured, then exit the application. --start-epoch=START_TIME Alias the --start-time switch. This switch is deprecated as of SiLK 3.8.0. --end-epoch=START_TIME Alias the --end-time switch. This switch is deprecated as of SiLK 3.8.0. EXAMPLES In the following examples, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line. To count all web traffic on Feb 12, 2009, into 1 hour bins: $ rwfilter --pass=stdout --start-date=2009/02/12:00 \ --end-date=2009/02/12:23 --proto=6 --aport=80 \ | rwcount --bin-size=3600 Date| Records| Bytes| Packets| 2009/02/12T00:00:00| 1490.49| 578270918.16| 463951.55| 2009/02/12T01:00:00| 1459.33| 596455716.52| 457487.80| 2009/02/12T02:00:00| 1529.06| 562602842.44| 451456.41| 2009/02/12T03:00:00| 1503.89| 562683116.38| 455554.81| 2009/02/12T04:00:00| 1561.89| 590554569.78| 489273.81| .... To bin the records according to their start times, use the --load-scheme switch: $ rwfilter ... --pass=stdout \ | rwcount --bin-size=3600 --load-scheme=1 Date| Records| Bytes| Packets| 2009/02/12T00:00:00| 1494.00| 580350969.00| 464952.00| 2009/02/12T01:00:00| 1462.00| 596145212.00| 457871.00| 2009/02/12T02:00:00| 1526.00| 561629416.00| 451088.00| 2009/02/12T03:00:00| 1502.00| 563500618.00| 455262.00| 2009/02/12T04:00:00| 1562.00| 589265818.00| 489279.00| ... To bin the records by their end times: $ rwfilter ... --pass=stdout \ | rwcount --bin-size=3600 --load-scheme=2 Date| Records| Bytes| Packets| 2009/02/12T00:00:00| 1488.00| 577132372.00| 463393.00| 2009/02/12T01:00:00| 1458.00| 596956697.00| 457376.00| 2009/02/12T02:00:00| 1530.00| 562806395.00| 451551.00| 2009/02/12T03:00:00| 1506.00| 562101791.00| 455671.00| 2009/02/12T04:00:00| 1562.00| 591408602.00| 489371.00| ... To force the hourly bins to run from 30 minutes past the hour, use the --start-time switch: $ rwfilter ... --pass=stdout \ | rwcount --bin-size=3600 --start-time=2002/12/31:23:30 Date| Records| Bytes| Packets| 2009/02/12T00:30:00| 1483.26| 581251364.04| 456554.40| 2009/02/12T01:30:00| 1494.00| 575037453.00| 449280.00| 2009/02/12T02:30:00| 1486.36| 559700466.61| 447700.15| 2009/02/12T03:30:00| 1555.23| 588882400.58| 480724.48| 2009/02/12T04:30:00| 1537.79| 564756248.52| 472003.45| ... ENVIRONMENT SILK_TIMESTAMP_FORMAT This environment variable is used as the value for --timestamp-format when that switch is not provided. Since SiLK 3.11.0. SILK_PAGER When set to a non-empty string, rwcount automatically invokes this pro- gram to display its output a screen at a time. If set to an empty string, rwcount does not automatically page its output. PAGER When set and SILK_PAGER is not set, rwcount automatically invokes this program to display its output a screen at a time. SILK_CLOBBER The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction. SILK_CONFIG_FILE This environment variable is used as the value for the --site-con- fig-file when that switch is not provided. SILK_DATA_ROOTDIR This environment variable specifies the root directory of data reposi- tory. As described in the "FILES" section, rwcount may use this envi- ronment variable when searching for the SiLK site configuration file. SILK_PATH This environment variable gives the root of the install tree. When searching for configuration files, rwcount may use this environment variable. See the "FILES" section for details. TZ When the argument to the --timestamp-format switch includes "local" or when a SiLK installation is built to use the local timezone, the value of the TZ environment variable determines the timezone in which rwcount displays timestamps. (If both of those are false, the TZ environment variable is ignored.) If the TZ environment variable is not set, the machine's default timezone is used. Setting TZ to the empty string or 0 causes timestamps to be displayed in UTC. For system information on the TZ variable, see tzset(3) or environ(7). (To determine if SiLK was built with support for the local timezone, check the "Timezone support" value in the output of rwcount --version.) The TZ environment variable is also used when rwcount parses the timestamp specified in the --start-time or --end-time switches if SiLK is built with local time- zone support. FILES ${SILK_CONFIG_FILE} ${SILK_DATA_ROOTDIR}/silk.conf /data/silk.conf ${SILK_PATH}/share/silk/silk.conf ${SILK_PATH}/share/silk.conf /usr/local/share/silk/silk.conf /usr/local/share/silk.conf Possible locations for the SiLK site configuration file which are checked when the --site-config-file switch is not provided. SEE ALSO rwfilter(1), rwuniq(1), silk(7), tzset(3), environ(7) BUGS Unlike rwuniq(1), rwcount does not support counting the number of distinct IPs in a bin. However, using the --bin-time switch on rwuniq can provide time-based binning similar to what rwcount supports. Note that rwuniq al- ways bins by the each record's start-time (similar to rwcount --load-fac- tor=1), and there is no support in rwuniq for dividing a SiLK record among multiple time bins. SiLK 3.22.2 2026-08-27 rwcount(1)
NAME | SYNOPSIS | DESCRIPTION | OPTIONS | EXAMPLES | ENVIRONMENT | FILES | SEE ALSO | BUGS
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=rwcount&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>