rwpdedupe(1) SiLK Tool Suite rwpdedupe(1) NAME rwpdedupe - Eliminate duplicate packets collected by several sensors SYNOPSIS rwpdedupe { --first-duplicate | --random-duplicate[=SCALAR] } [--threshold=MILLISECONDS] FILE... > OUTPUT-FILE rwpdedupe --help rwpdedupe --version DESCRIPTION Detects and eliminates duplicate records from tcpdump(1) capture files. Duplicate records are defined as having timestamps within a user-config- urable time of each other. In addition, their Ethernet (OSI layer 2) head- ers must match. If they are not IP packets, then their entire Ethernet payload must match. If they are IP packets, then their source and destina- tion addresses, protocol, and IP payload must match. OPTIONS Option names may be abbreviated if the abbreviation is unique or is an ex- act match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters. --threshold=MILLISECONDS Set the maximum number of milliseconds which may elapse between two packets and still have those packets be detected as duplicates. De- fault 0 (exact timestamp match). Must be a value between 0 and 1,000,000 milliseconds. One and only one of the following switches is required: --first-duplicate When selecting between multiple duplicate packets, always choose the packet with the earliest timestamp. Not compatible with --random-du- plicate. --random-duplicate --random-duplicate=SCALAR Select a random packet from the list of duplicate packets. SCALAR is a random number seed, so that multiple runs can produce identical re- sults. --help Print the available options and exit. --version Print the version number and information about how SiLK was configured, then exit the application. EXAMPLES In the following example, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line. Given tcpdump files data1.tcp and data2.tcp, detect and eliminate duplicate packets which occur within one second of each other (when choosing which timestamp to output, pick one randomly.) Store the result file in out.tcp. $ rwpdedupe --threshold=1000 --random-duplicate \ data1.tcp data2.tcp > out.tcp SEE ALSO silk(7), mergecap(1), tcpdump(1), pcap(3) NOTES mergecap(1) can be used to merge two tcpdump capture files without elimi- nating duplicate packets. SiLK 3.22.2 2026-08-27 rwpdedupe(1)
NAME | SYNOPSIS | DESCRIPTION | OPTIONS | EXAMPLES | SEE ALSO | NOTES
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=rwpdedupe&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>