rwpmapbuild(1) SiLK Tool Suite rwpmapbuild(1) NAME rwpmapbuild - Create a binary prefix map from a text file SYNOPSIS rwpmapbuild [--input-path=PATH] [--output-path=PATH] [--mode={ipv4|ipv6|proto-port}] [--dry-run] [--ignore-errors] [--note-add=TEXT] [--note-file-add=FILENAME] [--invocation-strip] rwpmapbuild --help rwpmapbuild --version DESCRIPTION rwpmapbuild reads a white-space-delimited stream of text and writes a bi- nary output stream representing a prefix map. The syntax of this input is described in the "INPUT FILE FORMAT" section below. The textual input is read from the file specified by --input-path or from the standard input when the switch is not provided. The binary output is written to the location named by --output-path or to the standard output when the switch is not provided and the standard output is not connected to a terminal. A prefix map file is a binary file that maps a value (specifically either an IP addresses or a protocol-port pair) to a string label. Once you have created a prefix map file, you may use the file in rwfil- ter(1), rwstats(1), rwuniq(1), rwgroup(1), rwsort(1), or rwcut(1) to parti- tion, count, sort and display SiLK flow records based on the string labels defined in the prefix map. See the pmapfilter(3) manual page for details. To view the contents of a prefix map file, use rwpmapcat(1). To query the contents of a prefix map, use rwpmaplookup(1). The remainder of this section provides example files that could be used as input to rwpmapbuild, and a note on the proper ordering of the input. For details on the syntax of the input, see the "INPUT FILE FORMAT" section that follows the description of the command line "OPTIONS". Sample IPv4 input file The following is a sample input file for rwpmapbuild that describes the registry of special-purpose IPv4 addresses. Any IP address that is not a special-purpose address get the label "Normal". # Prefix map sample input file for special purpose IPv4 addresses map-name ipv4-special mode ipv4 default Normal # Each line has an either a CIDR block or a pair of IP # addresses and then a label for that range 0.0.0.0/8 This host on this network [RFC1122 section 3.2.1.3] 10.0.0.0/8 Private-Use [RFC1918] 100.64.0.0/10 Shared Address Space [RFC6598] 127.0.0.0/8 Loopback [RFC1122 section 3.2.1.3] 169.254.0.0/16 Link Local [RFC3927] 172.16.0.0/12 Private-Use [RFC1918] 192.0.0.0/24 IETF Protocol Assignments [RFC6890 section 2.1] 192.0.0.0/29 IPv4 Service Continuity Prefix [RFC7335] # A range of a single IP address requires a "/32" suffix or # that the IP address be repeated 192.0.0.8/32 IPv4 dummy address [RFC7600] 192.0.0.9/32 Port Control Protocol Anycast [RFC7723] 192.0.0.10/32 Traversal Using Relays around NAT Anycast [draft] # A range may be specified as two IP addresses 192.0.0.170 192.0.0.171 NAT64/DNS64 Discovery [RFC7050 section 2.2] 192.0.2.0/24 Documentation (TEST-NET-1) [RFC5737] 192.31.196.0/24 AS112-v4 [RFC7535] 192.52.193.0/24 AMT [RFC7450] 192.88.99.0/24 Deprecated (6to4 Relay Anycast) [RFC7526] 192.168.0.0/16 Private-Use [RFC1918] 192.175.48.0/24 Direct Delegation AS112 Service [RFC7534] 198.18.0.0/15 Benchmarking [RFC2544] 198.51.100.0/24 Documentation (TEST-NET-2) [RFC5737] 203.0.113.0/24 Documentation (TEST-NET-3) [RFC5737] 240.0.0.0/4 Reserved [RFC1112 section 4] 255.255.255.255/32 Limited Broadcast [RFC919 section 7] Sample IPv6 input file The following input file for rwpmapbuild describes IPv6 address space. The file demonstrates the use of the label statement. It uses a hyphen ("-") as the label for any undefined ranges. # Prefix map sample input file for IPv6 address space map-name iana-ipv6 mode ipv6 label 0 RFC3849 label 1 RFC3879 label 2 RFC4048 label 3 RFC4193 label 4 RFC4291 label 5 RFC4291 Loopback Address label 6 RFC4291 Unspecified Address label 7 RFC4291 IPv4-mapped Address label 8 RFC5180 label 9 RFC6666 label 10 RFC7723 label 11 - default - 0000::/8 4 ::1/128 5 ::/128 6 ::ffff:0:0/96 7 0100::/8 4 0100::/64 9 # RFC6666 0200::/7 2 # RFC4048 0400::/6 4 0800::/5 4 1000::/4 4 2000::/3 4 2001:1::1/128 10 # RFC7723 2001:2::/48 8 # Benchmarking 2001:db8::/32 0 # Documentation 4000::/3 4 6000::/3 4 8000::/3 4 a000::/3 4 c000::/3 4 e000::/4 4 # You may use the label number or the f000::/5 RFC4291 # exact label name, but any other text f800::/6 4 # causes rwpmapbuild to issue an error fc00::/7 RFC4193 fe00::/9 4 fe80::/10 4 fec0::/10 RFC3879 ff00::/8 4 Sample protocol-port input file This is a small sample of a file that could be used to label IP protocols, specific ports within the TCP and UDP protocols, and ICMP type and code values. When ranges overlap or one range is a specialization of another, the wider or more general range should be listed first, followed by the narrower or more specific ranges. map-name protocol-port-example mode proto-port # The range is either a single protocol or a protocol and # a port separated by a slash. 1 1 ICMP # Specify the wider categories first, then specialize 6 6 TCP 6/0 6/1024 TCP/Generic reserved # A range of a single port requires both the starting # value and the ending value 6/21 6/21 TCP/FTP 6/22 6/22 TCP/SSH 6/25 6/25 TCP/SMTP 6/80 6/80 TCP/HTTP 6/443 6/443 TCP/HTTPS 6/6000 6/6063 TCP/X11 17 17 UDP 17/0 17/1024 UDP/Generic reserved 17/53 17/53 UDP/DNS 17/67 17/68 UDP/DHCP 50 50 ESP 58 58 ICMPv6 # For ICMP Type/Code, convert the type and code to a port # value using this expression: type * 256 + code 1/0 1/255 ICMP/Echo Reply 1/768 1/1023 ICMP/Destination Unreachable 1/1024 1/1279 ICMP/Source Quench 1/768 1/768 ICMP/Net Unreachable 1/769 1/769 ICMP/Host Unreachable 1/770 1/770 ICMP/Protocol Unreachable 1/771 1/771 ICMP/Port Unreachable Complete ICMPv4 Prefix Map An ideal candidate for port-based prefix maps is for decoding ICMP types and codes. Although most SiLK commands support a form of ICMP type and code options, these are all based on the actual number values. However, a prefix map may be useful to decode the noun-name of the ICMP types and codes. The following prefix map can be used for that purpose. (Note that ICMP type and code is always in the destination port field, regardless of the traffic direction.) # Identify this as a protocol-port prefix map, rather than an IP-range map mode proto-port # Set a default value for all records 0 255 Other # Set the default value for all ICMP records 1 1 ICMP/Undefined # ICMP specific entries 1/0 1/255 ICMP/Echo Reply 1/768 1/768 ICMP/Destination Unreachable/Net Unreachable 1/769 1/769 ICMP/Destination Unreachable/Host Unreachable 1/770 1/770 ICMP/Destination Unreachable/Protocol Unreachable 1/771 1/771 ICMP/Destination Unreachable/Port Unreachable 1/772 1/772 ICMP/Destination Unreachable/Fragmentation Needed and Don't Fragment was Set 1/773 1/773 ICMP/Destination Unreachable/Source Route Failed 1/774 1/774 ICMP/Destination Unreachable/Destination Network Unknown 1/775 1/775 ICMP/Destination Unreachable/Destination Host Unknown 1/776 1/776 ICMP/Destination Unreachable/Source Host Isolated 1/777 1/777 ICMP/Destination Unreachable/Communication with Destination Network is Administratively Prohibited 1/778 1/778 ICMP/Administratively Prohibited/Communication with Destination Host is Administratively Prohibited 1/779 1/779 ICMP/Administratively Prohibited/Destination Network Unreachable for Type of Service 1/780 1/780 ICMP/Administratively Prohibited/Destination Host Unreachable for Type of Service 1/781 1/781 ICMP/Administratively Prohibited/Communication Administratively Prohibited 1/782 1/782 ICMP/Administratively Prohibited/Host Precedence Violation 1/783 1/783 ICMP/Administratively Prohibited/Precedence cutoff in effect 1/1024 1/1279 ICMP/Source Quench 1/1280 1/1280 ICMP/Redirect/Redirect Datagram for the Network (or subnet) 1/1281 1/1281 ICMP/Redirect/Redirect Datagram for the Host 1/1282 1/1282 ICMP/Redirect/Redirect Datagram for the Type of Service and Network 1/1283 1/1283 ICMP/Redirect/Redirect Datagram for the Type of Service and Host 1/1536 1/1536 ICMP/Alternate Host Address/Alternate Address for Host 1/2048 1/2303 ICMP/Echo 1/2304 1/2304 ICMP/Router Advertisement/Normal router advertisement 1/2320 1/2320 ICMP/Router Advertisement/Does not route common traffic 1/2560 1/2815 ICMP/Router Selection 1/2816 1/2816 ICMP/Time Exceeded/Time to Live exceeded in Transit 1/2817 1/2817 ICMP/Time Exceeded/Fragment Reassembly Time Exceeded 1/3072 1/3072 ICMP/Parameter Problem/Pointer indicates the error 1/3073 1/3073 ICMP/Parameter Problem/Missing a Required Option 1/3074 1/3074 ICMP/Parameter Problem/Bad Length 1/3328 1/3583 ICMP/Timestamp 1/3584 1/3839 ICMP/Timestamp Reply 1/3840 1/4095 ICMP/Information Request 1/4096 1/4351 ICMP/Information Reply 1/4352 1/4607 ICMP/Address Mask Request 1/4608 1/4863 ICMP/Address Mask Reply 1/7680 1/7935 ICMP/Traceroute 1/7936 1/8191 ICMP/Datagram Conversion Error 1/8192 1/8447 ICMP/Mobile Host Redirect 1/8448 1/8703 ICMP/IPv6 Where-Are-You 1/8704 1/8959 ICMP/IPv6 I-Am-Here 1/8960 1/9215 ICMP/Mobile Registration Request 1/9216 1/9471 ICMP/Mobile Registration Reply 1/9984 1/10239 ICMP/SKIP 1/10240 1/10240 ICMP/Photuris/Bad SPI 1/10241 1/10241 ICMP/Photuris/Authentication Failed 1/10242 1/10242 ICMP/Photuris/Decompression Failed 1/10243 1/10243 ICMP/Photuris/Decryption Failed 1/10244 1/10244 ICMP/Photuris/Need Authentication 1/10245 1/10245 ICMP/Photuris/Need Authorization # A few other well-known protocols 6 6 TCP 17 17 UDP 50 50 ESP 51 51 AH Proper Ordering of rwpmapbuild Input When creating the textual input for rwmpabuild, be sure to put the most general attributes first in the list. For example, suppose we administer the address block 12.0.0.0/8, and would like to report on address ranges delegated within the organization, A pre- fix map can be used as follows to show utilization for each address block, as well as unallocated (and presumably unauthorized) usage. Display the contents of input file: $ cat network.pmap.txt 12.0.0.0/8 Assigned, Unallocated 12.1.0.0/16 RESERVED 12.38.0.0/16 Client Network 1 12.127.0.0/16 Data Center (Primary) 12.130.0.0/16 Client Network 2 12.154.0.0/16 Client Network 3 12.186.0.0/16 Data Center (Secondary) 12.210.0.0/16 RESERVED Create the binary prefix map: $ rwpmapbuild --input=network.pmap.txt --output=network.pmap Use rwfilter(1) to select IPs in the 12.0.0.0/8 netblock, and use rwuniq(1) to bin the results according to the prefix map: $ rwfilter --start=2007/07/30:00 --saddr=12.x.x.x --pass=stdout \ | rwuniq --pmap-file=network.pmap --field=sval --value=bytes sval| Bytes| RESERVED| 39749| Data Center (Primary)| 87621| Assigned, Unallocated| 4296212| Client Network 2| 545848| Data Center (Secondary)| 18228| Client Network 1| 112404| Client Network 3| 68820| Suppose the input file had placed the most general entry at the bottom, like so: $ cat network.pmap.txt 12.1.0.0/16 RESERVED 12.38.0.0/16 Client Network 1 12.127.0.0/16 Data Center (Primary) 12.130.0.0/16 Client Network 2 12.154.0.0/16 Client Network 3 12.186.0.0/16 Data Center (Secondary) 12.210.0.0/16 RESERVED 12.0.0.0/8 Assigned, Unallocated The general entry would overwrite the other entries: $ rwpmapbuild --input=network.pmap.txt --output=network.pmap $ rwfilter --start=2007/07/30:00 --saddr=12.x.x.x --pass=stdout \ | rwuniq --pmap-file=network.pmap --fields=sval --bytes sval| Bytes| Assigned, Unallocated| 5168882| This is easy to see by using rwpmapcat(1) to print the contents of the pre- fix map: $ rwpmapcat --map-file=network.pmap ipBlock| label| 0.0.0.0/5| UNKNOWN| 8.0.0.0/6| UNKNOWN| 12.0.0.0/8| Assigned, Unallocated| 13.0.0.0/8| UNKNOWN| 14.0.0.0/7| UNKNOWN| 16.0.0.0/4| UNKNOWN| 32.0.0.0/3| UNKNOWN| 64.0.0.0/2| UNKNOWN| 128.0.0.0/1| UNKNOWN| (rwpmapcat lists all possible addresses from 0.0.0.0 to 255.255.255.255 and their labels. The default label is "UNKNOWN" unless the default is set to something else.) The best way to make sure your entries are properly ordered is to explic- itly order them before compiling the prefix map. When the data uses the CIDR-block format, the UNIX sort(1) command often produces the proper out- put. $ cat network.pmap.txt 12.1.0.0/16 RESERVED 12.38.0.0/16 Client Network 1 12.127.0.0/16 Data Center (Primary) 12.130.0.0/16 Client Network 2 12.154.0.0/16 Client Network 3 12.186.0.0/16 Data Center (Secondary) 12.210.0.0/16 RESERVED 12.0.0.0/8 Assigned, Unallocated Split the input at the "/" and sort the input numerically by the bitmask size. (A small bitmask represents a large netblock.) $ sort -n -k 2 -t "/" network.pmap.txt 12.0.0.0/8 Assigned, Unallocated 12.1.0.0/16 RESERVED 12.127.0.0/16 Data Center (Primary) 12.130.0.0/16 Client Network 2 12.154.0.0/16 Client Network 3 12.186.0.0/16 Data Center (Secondary) 12.210.0.0/16 RESERVED 12.38.0.0/16 Client Network 1 OPTIONS Option names may be abbreviated if the abbreviation is unique or is an ex- act match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters. --input-path=PATH Read the textual input from PATH. You may use "stdin" or "-" to repre- sent the standard input. When this switch is not provided, the input is read from the standard input unless the standard input is a termi- nal. rwpmapbuild will read textual input from the terminal if the standard input is explicitly specified as the input. The input file format is described below. (Added in SiLK 3.17.0 as a replacement for --input-file.) --output-path=PATH Write the binary prefix map to PATH. You may use "stdout" or "-" to represent the standard output. When this switch is not provided, the prefix map is written to the standard output unless the standard output is connected to a terminal. (Added in SiLK 3.17.0 as a replacement for --output-file.) --mode={ipv4|ipv6|proto-port} Specify the type of the input, as if a mode statement appeared in the input stream. The value specified by this switch must not conflict with an explicit mode statement appearing in the input. --dry-run Do not write the output file. Simply check the syntax of the input file. --ignore-errors Write the output file regardless of any errors encountered while pars- ing the input file. --note-add=TEXT Add the specified TEXT to the header of the output file as an annota- tion. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool. --note-file-add=FILENAME Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to en- sure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation. --invocation-strip Do not record the command used to create the prefix map in the output. When this switch is not given, the invocation is written to the file's header, and the invocation may be viewed with rwfileinfo(1). Since SiLK 3.12.0. --input-file=PATH Read the textual input from PATH. An alias for --input-path. Depre- caated as of SiLK 3.17.0. --output-file=PATH Write the binary prefix map to PATH. An alias for --output-path. Dep- recaated as of SiLK 3.17.0. --help Print the available options and exit. --version Print the version number and information about how SiLK was configured, then exit the application. INPUT FILE FORMAT This section describes the format of the textual input file for rwpmap- build. Three example files are shown above in the "DESCRIPTION" section. Blank lines or lines containing only whitespace in the input file are ig- nored. The file may contain comments, and these are ignored. A comment begins with the first "#" character on a line and extends to the end of the line. Note that "#" appearing in a textual label is treated as the beginning of a comment. Each non-blank line in the input file that is not a comment is considered a statement. A statement must be completed on a single line, and only one statement may appear on a line. The delimiter in the input file is whitespace---specifically one or more space and/or tab characters. rwpmapbuild supports five types of statements. Four of those statements begin with a specific keyword: one of mode, map-name, label, and default. Any line that does not begin with with a keyword is expected to contain a range definition, which maps a range to a label. The format of the range definition depends on the mode. The four statement types that begin with a keyword are all optional. They are: mode { ipv4 | ipv6 | proto-port | ip } Specify what types of ranges are defined in the file. The mode state- ment must appear before any ranges are specified. The mode may also be set using the --mode command line switch. When both the mode statement and the --mode switch are given, their values must match. When neither the mode statement nor the --mode switch is provided, rwpmapbuild processes the input in IPv4 address mode. The ip mode is deprecated; it is an alias for ipv4. The mode statement may only appear one time. map-name simple-string Create a name for the data in this prefix map file that other SiLK tools may use to refer to this prefix map file. When the prefix map file is used by rwfilter(1), the simple-string is used to generate the filtering switch names. When the prefix map file is used by rwcut(1), rwgroup(1), rwsort(1), rwstats(1), or rwuniq(1), the simple-string is used to generate the field names. See pmapfilter(3) for details. The simple-string may not contain whitespace, a comma, or a colon. The map-name statement may only appear one time. label num label-text Associate the numeric identifier num with the given label text la- bel-text. By specifying a label statement, the identifier num is ex- panded to label-text when the range definitions are being defined. Either all labels used in the file must appear in label statements, or no label statements may appear in which case rwpmapbuild creates labels as it parses the range definitions. All label statements must appear before the default statement and before the range definitions. label-text is a textual string that begins at the first non-whitespace character and extends to the final non-whitespace character on that line that does not appear in a comment. The label-text may include em- bedded whitespace and non-alphanumeric characters. While a comma (",") is legal in the label-text, using a comma prevents the label from being used by the --pmap-src and --pmap-dest switches in rwfilter(1). If no label statements appear in the input, any text containing at least one non-whitespace character may be used as the label in the de- fault statement and the range definitions. It is an error if num or label-text appear in any other label state- ment. The minimum value for num is 0 and the maximum value is 2147483647. Note that rwpmapbuild creates labels for all numeric iden- tifiers between 0 and the maximum identifier used in the input file, and using an unnecessarily large value creates many empty entries. default label-value Use the label label-value for any ranges not explicitly mentioned in this input file. The label-value text is one of 1. when label statements are used, a numerical label identifier that was specified in one of the statements 2. when label statements are used, a string that is an exact match of the label-text that was specified in one of those statements 3. when label statements are not used, a string that begins at the first non-whitespace character and extends to the final non-white- space character on that line that does not appear in a comment The default statement must appear before the range definitions are specified. If the default statement does not appear in the input, the label "UNKNOWN" is automatically defined and used as the default. As mentioned above, any line that does not begin with one of the above key- words must contain a range definition, and the format of the line depends on the type of data in the input file--that is, the mode of the input. Regardless of the input mode, the final item in each range definition is the label to assign to that range. The label has the same form as that de- scribed for the default statement above, and the label is denoted by la- bel-value in the following. Address Mode When the mode is either ipv4 or ipv6, rwpmapbuild parses the file in ad- dress mode. In address mode, each range definition contains an IP range and a label-value. A range is either a CIDR block or a pair of IP addresses that specify the starting IP and ending IP of the range. To provide a label for a single IP address, you must either use the same IP address as the starting and ending values of the range, append "/32" to a single IPv4 address, or append "/128" to a single IPv6 address. When the mode is ipv4, an IPv6 address in the input file raises an error. rwpmapbuild also accepts integer represen- tations of IP addresses when in ipv4 mode, When ranges overlap or one range is a specialization of another, the wider or more general range should be listed first, followed by the narrower or more specific ranges. cidr-block label-value Associate the given label identifier or label text with this CIDR block. The CIDR block is composed of an IP address in canonical nota- tion (e.g, dotted-decimal for IPv4), a slash "/", and the number of significant bits. low-ip high-ip label-value Associate the given label identifier or label text with this IP range, where low-ip and high-ip are in canonical notation. low-int high-int label-value Treat low-int and high-int as 32-bit values, convert the values to IPv4 addresses, and associate the given label identifier or label text with the IPv4 range. Protocol/Port Mode When the mode is proto-port, rwpmapbuild parses the file in protocol/port mode. In protocol/port mode, each range definition contains a starting value, an ending value, and the label-value. The starting and ending values may both be integers between 0 and 255, in- clusive. The numbers are treated as protocol values, where 6 is TCP, 17 is UDP, and 1 is ICMP. The starting and ending values may also both be a number, a slash ("/"), and a number. The first number is treated as the protocol and the second number as a port number (or service) in that protocol. For example, 6/80 is considered the http service of TCP. When ranges overlap or one range is a specialization of another, the wider or more general range should be listed first, followed by the narrower or more specific ranges. That is, specify the generic protocol first, then port numbers within that protocol. proto/port proto/port label-value Associate the given label identifier or label text with all protocols and port numbers between these two values inclusive. Note that while port is not meaningful for all protocols (specifically, it is meaning- ful for TCP and UDP and may contain type/code information for ICMP), rwpmapbuild accepts port numbers for any protocol. proto proto label-value Associate the given label identifier or label text for all protocols between these two values inclusive. EXAMPLE In the following examples, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line. Reading from and writing to a file: $ rwpmapbuild --input iana-ipv6.txt --output iana-ipv6.pmap Reading from the standard input and writing to the standard output: $ cat ipv4-special.txt \ | rwpmapbuild > ipv4-special.pmap For example input files, see the "DESCRIPTION" section above. ENVIRONMENT SILK_CLOBBER The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction. SEE ALSO pmapfilter(3), rwfilter(1), rwfileinfo(1), rwpmapcat(1), rwpmaplookup(1), rwcut(1), rwgroup(1), rwsort(1), rwstats(1), rwuniq(1), silk(7) SiLK 3.22.2 2026-08-27 rwpmapbuild(1)
NAME | SYNOPSIS | DESCRIPTION | OPTIONS | INPUT FILE FORMAT | EXAMPLE | ENVIRONMENT | SEE ALSO
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=rwpmapbuild&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>