IPAGGCREATE(1) IPAGGCREATE(1) NAME ipaggcreate - produce aggregate statistics of network traffic or trace SYNOPSIS ipaggcreate [-r | -i | --netflow-summary] [--src, --dst, --sport, --dport, ...] [other options] [files or interfaces] DESCRIPTION The ipaggcreate program reads IP packets from one or more data sources, maps each packet to a label (such as "source address 192.4.10.9" or "length 10"), and outputs a simply-formatted "aggregate" file reporting the number of packets or bytes observed per label. The resulting file is easy to process with text-based tools. (But see the --binary option, which gener- ates a compressed, quick-to-process binary file.) Here are a couple lines of ipaggcreate output, from `ipaggcreate -s /home/kohler/largedump.gz': !IPAggregate 1.0 !creator "src/ipaggcreate -s /home/kohler/largedump.gz" !counts packets !times 976937726.638704 977337361.804592 399635.165888 !num_nonzero 1437 !ip 4.2.49.2 1 4.2.49.4 1 4.17.143.9 1 4.21.203.29 104 The `-s' option, which is equivalent to `--src', tells ipaggcreate to cate- gorize each packet by its source IP address. `/home/kohler/largedump.gz' is a compressed tcpdump(1) file. Each data line represents a label; the first field is the label number (here, an IP source address), and the sec- ond field the number of packets that had that label. Labels with 0 counts are not reported. OPTIONS Data Sources Data source options tell ipaggcreate what kind of data source to use: tcp- dump(1) raw-packet files (--tcpdump), live network interfaces (--inter- face), NetFlow summary files (--netflow-summary), ipsumdump output files (--ipsumdump), DAG or NLANR-formatted files (--dag, --nlanr), or others. Non-option arguments specify the files, or interfaces, to read. For exam- ple, `ipaggcreate -r eth0 eth1' will read two tcpdump(1) files, named "eth0" and "eth1"; `ipaggcreate -i eth0 eth1' will read from two live net- work interfaces, "eth0" and "eth1". Options that read files read from the standard input when you supply a sin- gle dash "-" as a filename, or when you give no filenames at all. --tcpdump, -r Read from one or more files produced by tcpdump(1)'s -w option (also known as "pcap files"). Stop when all the files are exhausted. This is the default. Files (except for standard input) may be compressed by gzip(1) or bzip2(1); ipsumdump will uncompress them on the fly. --interface, -i Read from live network interfaces. When run this way, ipsumdump will continue until interrupted with SIGINT or SIGHUP. When stopped, ipsum- dump appends a comment to its output file, indicating how many packets were dropped by the kernel before output. --ipsumdump Read from one or more ipsumdump files. Any packet characteristics not specified by the input files are set to 0. --format=format Read from one or more ipsumdump files, using the specified default for- mat. The format should be a space-separated list of content types; see ToIPSummaryDump(n) for a list. --dag[=encap] Read from one or more DAG-formatted trace files. For new-style ERF dumps, which contain encapsulation type information, just say --dag. For old-style dumps, you must supply the right encap argument: "ATM" for ATM RFC-1483 encapsulation (the most common), "ETHER" for Ethernet, "PPP" for PPP, "IP" for raw IP, "HDLC" for Cisco HDLC, "PPP_HDLC" for PPP HDLC, or "SUNATM" for Sun ATM. See <http://dag.cs.waikato.ac.nz/>. --nlanr Read from one or more NLANR-formatted trace files (fr, fr+, or tsh for- mat). See <http://pma.nlanr.net/Traces/>. --ip-addresses Read files containing IP addresses, one address per line. The label must be either --src or --dst. --tu-summary Read TCP/UDP summary files. Each line represents one packet, and car- ries the following information: timestamp, source address, source port, destination address, destination port, protocol, payload length. For example: 976937735.345744 18.26.4.9 22 64.55.139.202 26876 T 0 976937770.197008 128.10.5.110 63749 64.55.139.202 113 T 5 --bro-conn-summary Read Bro connection summary files. Each line represents one connection attempt, and carries the following information: timestamp, source ad- dress, destination address, direction (inbound/outbound). --netflow-summary Read from one or more NetFlow summary files. These are line-oriented ASCII files; blank lines, and lines starting with '!' or '#', are ig- nored. Other lines should contain 15 or more fields separated by verti- cal bars '|'. Ipsumdump pays attention to some of these fields: Field Meaning Example ----- ---------------------------- ---------- 0 Source IP address 192.4.1.32 1 Destination IP address 18.26.4.44 5 Packet count in flow 5 6 Byte count in flow 10932 7 Flow timestamp (UNIX-style) 998006995 8 Flow end timestamp 998006999 9 Source port 3917 10 Destination port 80 12 TCP flags (OR of all pkts) 18 13 IP protocol 6 14 IP TOS bits 0 --tcpdump-text Read from one or more files containing tcpdump(1) textual output. It's much better to use the binary files produced by 'tcpdump -w', but if someone threw those away and all you have is the ASCII output, you can still make do. Only works with tcpdump versions 3.7 and earlier. Label These options determine how packets are labeled; you can supply at most one. --src, -s Label by IP source address; all packets with the same source address form an aggregate. --dst, -d Label by IP destination address. This is the default. --length, -l Label by IP length. --ip field Label by the named IP field. Examples include "ip src" (equivalent to --src), "ip ttl", "ip off", "udp sport", and so forth. See Aggre- gateIP(1) for a full list. --flows Label by TCP or UDP flow, or, essentially, by end-to-end transport- level connection. Two packets have the same label if and only if they are part of the same TCP or UDP connection. Each flow is assigned its own label. The label number is not meaningful; non-TCP/UDP packets are ignored. --unidirectional-flows Label by unidirectional TCP or UDP flow. Like --flows, but packets from a single connection but heading in different directions are as- signed different labels. --address-pairs Label by address pair. Two packets have the same label if and only if they involve the same pair of IP addresses. The label number is not meaningful. --unidirectional-address-pairs Label by unidirectional address pair. Two packets have the same label if and only if their source addresses match and their destination ad- dress match. Measurement Options These options specify whether ipaggcreate should count packets or bytes. --packets Count packets: the output file will report the number of packets per label. This is the default. --bytes, -B Count bytes: the output file will report the number of bytes per label. This number includes IP and transport headers, but not any link head- ers. Limit and Split Options These options select portions of the trace file, and allow the user to split trace data into multiple aggregate files. --time-offset=time, -T time Ignore the first time worth of packets in the input trace. If the first packet has timestamp T, then all packets (including the first) with timestamp less than T+time are ignored. The time argument can be an absolute number of seconds (938.42), or use suffixes such as "100s", "12ms", "1.5min", "2hr", and so forth. --start-time=time Ignore packets with timestamps less than time. --interval=time, -t time Stop after recording aggregate information for time worth of packets. That is, if the first recorded packet has timestamp T, then ipaggcreate will exit just before the first packet with timestamp T+time, or the end of the trace, whichever comes first. --limit-labels=count Stop after recording information for count distinct labels. That is, exit just before encountering a packet with the count+1 different la- bel, or at the end of the trace, whichever comes first. The four --split options generate multiple aggregate output files based on characteristics of the input. To use --split, you must supply an explicit --output filename containing a "%d"-style template; a file number is plugged in to that template. For example, the template "file%03d.txt" will generate files "file001.txt", "file002.txt", and so forth. --split-time=time Start a new output file every time period. That is, each file will contain data for at most time worth of packets. --split-labels=count Start a new output file every count distinct labels. That is, each file will contain at most count different labels. --split-packets=count Start a new output file every count packets. --split-bytes=count Start a new output file every count bytes. Other Options --output=file, -o file Write the summary dump to file instead of to the standard output. --binary, -b Write the summary dump in binary format. See below for more informa- tion. --write-tcpdump=file, -w file Write processed packets to a tcpdump(1) file -- or to the standard out- put, if file is a single dash "-" -- in addition to the usual summary output. --filter=filter, -f filter Only include packets and flows matching a tcpdump(1) filter. For exam- ple, `ipsumdump -f "tcp && src net 18/8"' will summarize data only for TCP packets from net 18. (The syntax for filter is currently a subset of tcpdump's syntax.) --anonymize, -A Anonymize IP addresses in the output. The anonymization preserves pre- fix and class. This means, first, that two anonymized addresses will share the same prefix when their non-anonymized counterparts share the same prefix; and second, that anonymized addresses will be in the same class (A, B, C, or D) as their non-anonymized counterparts. The anonymization algorithm comes from tcpdpriv(1); it works like `tcpdpriv -A50 -C4'. If --anonymize and --write-tcpdump are both on, the tcpdump output file will have anonymized IP addresses. However, the file will contain ac- tual packet data, unlike tcpdpriv output. --no-promiscuous Do not place interfaces into promiscuous mode. Promiscuous mode is the default. --sample=p Sample packets with probability p. That is, p is the chance that a packet will cause output to be generated. The actual probability may differ from the specified probability, due to fixed point arithmetic; check the output for a `"!sampling_prob"' comment to see the real prob- ability. Strictly speaking, this option samples records, not packets, so for NetFlow summaries without --multipacket, it will sample flows. --multipacket Supply this option if you are reading NetFlow or IP summaries -- files where each record might represent multiple packets -- and you would like the output summary to have one line per packet, instead of the de- fault one line per record. See also --packet-count, above. --collate Sort output packets by increasing timestamp. Use this option when read- ing from multiple tcpdump(1) files to ensure that the output has sorted timestamps. Combine --collate with --write-tcpdump to collate overlap- ping tcpdump(1) files into a single, sorted tcpdump(1) file. --random-seed=seed Set the random seed deterministically to seed, an unsigned integer. By default, the random seed is initialized to a random value using /dev/random, if it exists, combined with other data. The random seed indirectly determines which packets are sampled, and the values of anonymized IP addresses. --quiet, -q Do not print a progress bar to standard error. This is the default when ipsumdump isn't running interactively. --config Do not produce a summary. Instead, write the Click configuration that ipsumdump would run to the standard output. --verbose, -V Produce more verbose error messages. --help, -h Print a help message to the standard output, then exit. --version, -v Print version number and license information to the standard output, then exit. SIGNALS When killed with SIGTERM or SIGINT, ipaggcreate will exit cleanly (and gen- erate an output file). If you want it to flush its buffers without exit- ing, kill it with SIGHUP. BINARY FORMAT Binary ipaggcreate files begin with several ASCII lines, just like regular ipaggcreate files. A line `"!packed_be"' or `"!packed_le"' indicates that the rest of the file, starting immediately after the newline, consists of binary records (in big-endian or little-endian order, respectively). Each record is 8 bytes long, and looks like this: +---------------+---------------+ | label | count | +---------------+---------------+ <---4 bytes---> <---4 bytes---> The initial word of data contains the label number, the second the count. CLICK The ipaggcreate program uses the Click modular router, an extensible system for processing packets. Click routers consist of C++ components called el- ements. While some elements run only in a Linux kernel, most can run ei- ther in the kernel or in user space, and there are user-level elements for reading packets from libpcap or from tcpdump files. Ipaggcreate creates and runs a user-level Click configuration. However, you don't need to install Click to run ipsumdump; the libclick directory contains all the relevant parts of Click, bundled into a library. If you're curious, try running `ipaggcreate --config' with some other op- tions to see the Click configuration ipsumdump would run. This is, I think, a pleasant way to write a packet processor! SEE ALSO tcpdump(1), tcpdpriv(1), click(1), ipsumdump(1) See http://www.pdos.csail.mit.edu/click/ for more on Click. AUTHOR Eddie Kohler <kohler@cs.ucla.edu>, based on the Click modular router. Anonymization algorithm from tcpdpriv(1) by Greg Minshall. Version 1.86 2014-05-02 IPAGGCREATE(1)
NAME | SYNOPSIS | DESCRIPTION | OPTIONS | SIGNALS | BINARY FORMAT | CLICK | SEE ALSO | AUTHOR
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=ipaggcreate&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>