FreeBSD Manual Pages
YAFDPI(1) Yet Another Flowmeter YAFDPI(1) NAME yaf deep packet inspection DESCRIPTION yaf(1) can examine packet payloads, capture useful information for a spe- cific protocol, and export it in a protocol-specific template within yaf's SubTemplateMultiList if yaf was built with plugin support enabled (using the --enable-plugins option to ./configure). It may be necessary to set the LTDL_LIBRARY_PATH environment variable if the plugins were installed in a nonstandard location. The DPI plugin requires payload capture to be enabled with the --max-pay- load option. A minimum payload capture length of 384 octets is recommended for best results. The --applabel option is also required, as the applica- tion label determines how the inspection will execute. DPI in yaf is directly related to application labeling as yaf will only perform DPI if a match was found during the application labeling phase, and it will only execute an inspection specific to the protocol denoted by the application label. In order to enable DPI in yaf the following should be added to the command line: "--plugin-name=/usr/local/lib/yaf/dpacketplugin.la" You can also add the option switch to specify on which protocols to perform DPI: "--plugin-opts="53 80 21"" The above will perform DPI for DNS, HTTP, and FTP. DPI operates differently depending on whether the protocol is plugin-based or regex-based in the yafApplabelRules.conf file. If the protocol uses a regex rule for application labeling, the yafDPIR- ules.conf file has a list of regular expressions that are compared against the captured payload. Any matches are stored and later exported in a se- ries of IPFIX basicLists, where each basicList contains one IPFIX informa- tion element corresponding to one regular expression. If the protocol is based on a plugin rule, the dynamically loaded plugin listed in the yafApplabelRules.conf file stores important information while it is decoding the payload, and later exports that information in cus- tomized templates. See the source code to the plugins included with yaf for details on the specific protocol implementations. Some plugins will allow configurable deep packet inspection from the yafD- PIRules.conf file, such as DNP 3.0, Ethernet/IP, and SCADA. See below for specific information on these particular protocols. In order to perform DPI on DNSSEC resource records, add "DNSSEC" to the --plugin-opts option: "--plugin-opts=DNSSEC" "--plugin-opts="DNSSEC 53"" DPI CONFIG FILE FORMAT The yafDPIRules.conf file should be in the same directory as the yafAppla- belRules.conf file (see applabel(1)), and it follows a format similar to that file. The regex rules in the file contain three parts: a label, an element, and a regular expression rule. Elements may either be defined by YAF or by the user, as described below. If a rule is not properly formatted, all subsequent rules may not be processed. It is acceptable to comment out any yaf DPI rules. yaf rules commented out will not be executed against the payload but they will still exist in the template and record. User-defined information elements are added based on the configuration file at run time. YAF Defined Elements A regex rule for a YAF-defined element has the following form: label <N> yaf <N2> <element-rule> where * <N> is the application label (usually the well-known port) found in the yafApplabelRules.conf file (an unsigned 16-bit decimal integer in the range 0 to 65535). * <N2> is the Information Element ID found in the CERT IPFIX Registry <https://tools.netsa.cert.org/cert-ipfix-registry/cert_ipfix_format- ted.html> and also below. * <element-rule> is a PCRE2 regular expression to apply to the payload, and matching payload will be stored and associated with the element ID pre- ceding it. There can be multiple lines for a single application label, however each should have a different element ID (<N2>). There should be parentheses around the substring you want to capture and store. If there are no paren- theses, the entire matched expression is captured. If there are more than 1 set of parentheses in the regular expression, the outermost set is the substring captured. For efficiency, use non-capturing parentheses for grouping (for example, '"(?:foo|bar)"' to match '"foo"' or '"bar"' without capturing the text). See the PCRE2 documentation for details on regular expressions and substring matching. User Defined Elements To define your own information elements, use the following form: label <N> user <E> name <element-name> <element-rule> where * <N> is the application label found in yafApplabelRules.conf file. * <E> is the Information Element ID in the range of 0 to 65535 to be given to the element upon export. This number should be unique to this file and should NOT be defined in the CERT IPFIX Registry. This element will be added to the template upon processing of this file, and must be added to the yaf collecting process in order to properly decode the IPFIX mes- sage. * <element-name> is the name you want to give to this IPFIX Information El- ement. This name can consist of letters and numbers and underscores; it can not contain special characters or spaces. * <label-rule> is the PCRE2 regular expression and will be stored and asso- ciated with the Information Element ID and name preceding it. There is a limit of 30 additional fields per protocol that YAF will store and execute. To find out if yaf accepted your elements, run yaf with --verbose. All user elements will be exported using the CERT Private En- terprise Number (PEN) 6871. ONLY user labels for protocols FTP, HTTP, RTSP, and SIP will be added. Elements will be added to the template in the order they are listed in the yafDPIRules.conf file in the form of an fbBa- sicList_t. By default, HTTP exports 20 basicLists, FTP exports 5 basi- cLists, RTSP exports 12 basicLists, and SIP exports 7 basicLists. Comments A ""#"" symbol starts a comment for the entire line. Comments are only recognized when ""#"" is the first non-whitespace character on the line. Export Limits Optionally, this file may contain two limit statements to configure the DPI plugin. A limit statement begins with the keyword 'limit', and has the following form: limit [field|total] <limit-value> If the "field" label is present, the <limit-value> will be the number of bytes yaf will export for any given field in this file. This does not af- fect the DNS Deep Packet Inspection or SSL Certificate Capture. For DNS, a domain name can have a maximum of 255 characters, so the limit is not con- figurable. If the "total" field is present, the <limit-value> will be the total number of bytes yaf may export from the DPI plugin. Obviously, this number will not be larger than the --max-payload value yaf is given at run time. Both the field and total limits have a maximum value of 65535. If they are larger, they will revert back to the defaults of 200 for per-field limit and 1000 for total limit. SSL/TLS Export Options There are also 2 configuration parameters related to SSL/TLS export. By default, yaf parses the X.509 certificates and exports the information de- scribed below under SSL/TLS. If the following line is present: cert_export_enabled = 1 yaf will export the full X.509 certificate in the format described below under Full Certificate Export. Setting this variable to 1 disables the traditional SSL certificate decode and export. If the second configuration variable is present: cert_hash_enabled = 1 yaf will export the hash of the X.509 certificate as found in the certifi- cate. This is typically the SHA-256 hash of the binary certificate but it can vary on the hashing algorithm used. The hashing algorithm can be iden- tified by the sslCertSignature field. If both cert_export_enabled and cert_hash_enabled are set to 1, yaf will export both the full X.509 cer- tificate and perform the traditional decode of the X.509 certificate. It is not recommended to do both. If cert_export_enabled is set to 1, super_me- diator(1) can perform the extraction of relevant fields as is done by yaf, plus it provides the option to perform SHA-1 or MD5 hashes of the certifi- cate. DPI in Action Upon yaf startup and capture, you will be able to see if the rule files and their regular expressions were accepted using the --verbose flag. [2013-05-03 19:39:25] DPI Running for ALL Protocols [2013-05-03 19:39:25] Reading packets from packets.pcap [2013-05-03 19:39:25] Initializing Rules from DPI File /usr/lo- cal/etc/yaf/yafDPIRules.conf [2013-05-03 19:39:25] DPI rule scanner accepted 63 rules from the DPI Rule File An unacceptable regular expression will be brought to your attention with the above statements. If you choose certain protocols for inspection using the "--plugin-opts" flag, only the appropriate rule statements will be loaded into the DPI Rule Scanner. Configure Options The following options can be given to ./configure when yaf is built to ex- port DNS authoritative and NXDomain Responses only. --enable-exportDNSAuth Enable export of DNS Authoritative Responses only. The default is to capture and export all DNS Responses. This flag can be used in conjunc- tion with --enable-exportDNSNXDomain. It is only recognized if --plu- gin-name is set to the DPI plugin, application labeling is enabled, and --max-payload is set. --enable-exportDNSNXDomain Enable export of DNS NXDomain Responses only. The default is to capture and export all DNS Responses. This flag can be used in conjunction with --enable-exportDNSAuth. It is only recognized if --plugin-name is set to the DPI plugin, application labeling is enabled, and --max-payload is set. DPI Data Export DPI Templates & Information Elements by Protocol yaf's output consists of an IPFIX message stream. yaf uses a variety of templates for IPFIX data records. As of yaf 2.0, yaf uses a subTemplate- MultiList to export optional information elements, such as Deep Packet In- spection fields, relating to the flow. Below are templates that may appear in this subTemplateMultiList depending on the application label of the flow. For more information on yaf information elements see yaf(1). For more information on IPFIX Structured lists, see the Internet Draft, Export of Structured Data in IPFIX, <RFC 6313>. Most of the elements are exported as a basicList. An IPFIX basicList represents a list of zero or more in- stances of any Information Element (IE 291). FTP File Transfer Protocol (FTP) Deep Packet Inspection is based on RFC 959. The following information elements are exported as a template in the sub- TemplateMultiList as basicLists of variable length elements in the order they are listed in the yafDPIRules.conf file. YAF will always export at least 5 basicLists for FTP, even if not all of the following are enabled. By default, they will be in the following order: ftpReturn CERT (PEN 6871) IE 131, variable length, DPI basicList FTP Commands or Replies. ftpUser CERT (PEN 6871) IE 132, variable length, DPI basicList FTP User Command Argument. This command will normally be the first com- mand transmitted by the user. ftpPass CERT (PEN 6871) IE 133, variable length, DPI basicList FTP Password Command Argument. This command must be preceded by the user name command, and is usually required to complete authentication. ftpType CERT (PEN 6871) IE 134, variable length, DPI basicList FTP Data Representation Type. ftpRespCode CERT (PEN 6871) IE 135, variable length, DPI basicList FTP Reply. This consists of a three digit number followed by some text. HTTP HTTP Deep Packet Inspection is based on RFC 2616. The following informa- tion elements are exported as a template in the subTemplateMultiList as ba- sicLists of variable length elements in the order they are listed in the yafDPIRules.conf file. Some elements are not enabled by default. The tem- plate will always contain at least 20 information elements even if less el- ements are enabled in the configuration file. By default, the following 20 information elements are exported in the following order: httpServerString CERT (PEN 6871) IE 110, variable length, DPI basicList HTTP Server Response-header field. Contains information about the soft- ware used to handle the HTTP Request. httpUserAgent CERT (PEN 6871) IE 111, variable length, DPI basicList HTTP User-Agent Request-header field. Contains information about the user agent originating the request. httpGet CERT (PEN 6871) IE 112, variable length, DPI basicList HTTP Method Command. Retrieves information identified by the following Request-URI. httpConnection CERT (PEN 6871) IE 113, variable length, DPI basicList HTTP Connection header fields. Contains options that are desired for a particular connection. httpReferer CERT (PEN 6871) IE 115, variable length, DPI basicList HTTP Referer request-header field. Address (URI) of the resource which the Request-URI was obtained. httpLocation CERT (PEN 6871) IE 116, variable length, DPI basicList HTTP Location response-header field. Used to redirect the recipient to a location to complete a request or identify a new resource. httpHost CERT (PEN 6871) IE 117, variable length, DPI basicList HTTP Host Request-header. The Internet host and port number of the re- source being requested. httpContentLength CERT (PEN 6871) IE 118, variable length, DPI basicList HTTP Content-Length header. Indicates the size of the entity-body. httpAge CERT (PEN 6871) IE 119, variable length, DPI basicList HTTP Age response-header. Argument is the sender's estimate of the time elapsed since the response. httpResponse CERT (PEN 6871) IE 123, variable length, DPI basicList HTTP Response Status Code. Usually a three-digit number followed by text. httpAcceptLanguage CERT (PEN 6871) IE 121, variable length, DPI basicList HTTP Accept-Language Request-Header field. Restricts the set of natural languages that preferred. httpAccept CERT (PEN 6871) IE 120, variable length, DPI basicList HTTP Accept request-header field. Used to specify certain media types that are acceptable for the response. httpContentType CERT (PEN 6871) IE 122, variable length, DPI basicList HTTP Content Type entity-header field. Indicates the media type of the entity-body. httpVersion CERT (PEN 6871) IE 114, variable length, DPI basicList HTTP Version Number. httpCookie CERT (PEN 6871) IE 220, variable length, DPI basicList HTTP Cookie Header Field. httpSetCookie CERT (PEN 6871) IE 221, variable length, DPI basicList HTTP Set Cookie Header Field. httpAuthorization CERT (PEN 6871) IE 252, variable length, DPI basicList HTTP Authorization Header Field. httpVia CERT (PEN 6871) IE 253, variable length, DPI basicList HTTP Via Header Field. httpX-Forwarded-For CERT (PEN 6871) IE 254, variable length, DPI basicList HTTP X-Forwarded-For Header Field. httpRefresh CERT (PEN 6871) IE 256, variable length, DPI basicList HTTP Refresh Header Field. Optional HTTP Elements The following information elements are defined but not enabled by default. To enable any of the following fields, uncomment the line in the yafDPIR- ules.conf file. httpExpires CERT (PEN 6871) IE 255, variable length, DPI basicList HTTP Expires Header Field. httpIMEI CERT (PEN 6871) IE 257, variable length, DPI basicList HTTP International Mobile Station Equipment Identity ID. httpIMSI CERT (PEN 6871) IE 258, variable length, DPI basicList HTTP International Mobile Subscriber Identity httpMSISDN CERT (PEN 6871) IE 259, variable length, DPI basicList HTTP MSISDN number, a telephone number for the SIM card in a mobile/cel- lular phone. httpSubscriber CERT (PEN 6871) IE 260, variable length, DPI basicList HTTP Mobile Subscriber Information httpAcceptCharset CERT (PEN 6871) IE 261, variable length, DPI basicList HTTP Accept Charset Header Field. httpAllow CERT (PEN 6871) IE 263, variable length, DPI basicList HTTP Accept Encoding Header Field. httpDate CERT (PEN 6871) IE 264, variable length, DPI basicList HTTP Date Header Field. httpExpect CERT (PEN 6871) IE 265, variable length, DPI basicList HTTP Expect Header Field. httpFrom CERT (PEN 6871) IE 266, variable length, DPI basicList HTTP From Header Field. httpProxyAuthentication CERT (PEN 6871) IE 267, variable length, DPI basi- cList HTTP Proxy Authentication Field. httpUpgrade CERT (PEN 6871) IE 268, variable length, DPI basicList HTTP Upgrade Header Field. httpWarning CERT (PEN 6871) IE 269, variable length, DPI basicList HTTP Warning Header Field. httpDNT CERT (PEN 6871) IE 270, variable length, DPI basicList HTTP DNT Header Field. httpX-Forwarded-Proto CERT (PEN 6871) IE 271, variable length, DPI basi- cList HTTP X-Forwarded-Proto Header Field. httpX-Forwarded-Host CERT (PEN 6871) IE 272, variable length, DPI basicList HTTP X-Forwarded-Host Header Field. httpX-Forwarded-Server CERT (PEN 6871) IE 273, variable length, DPI basi- cList HTTP X-Forwarded-Server Header Field. httpX-DeviceID CERT (PEN 6871) IE 274, variable length, DPI basicList HTTP X-Device ID Header Field. httpX-Profile CERT (PEN 6871) IE 275, variable length, DPI basicList HTTP X-Profile Header Field. httpLastModified CERT (PEN 6871) IE 276, variable length, DPI basicList HTTP Last Modified Header Field. httpContentEncoding CERT (PEN 6871) IE 277, variable length, DPI basicList HTTP Content Encoding Header Field. httpContentLanguage CERT (PEN 6871) IE 278, variable length, DPI basicList HTTP Content Language Header Field. httpContentLocation CERT (PEN 6871) IE 279, variable length, DPI basicList HTTP Content Location Header Field. httpX-UA-Compatible CERT (PEN 6871) IE 280, variable length, DPI basicList HTTP X-UA-Compatible Header Field. IMAP IMAP Deep Packet Inspection is based on RFC 3501. The following informa- tion elements are exported as a template in the subTemplateMultiList as ba- sicLists of variable length elements in the order they are listed in the yafDPIRules.conf file. yaf will always export at least 7 fields in the IMAP template and data record. By default, yaf exports the following fields in order: imapCapability CERT (PEN 6871) IE 136, variable length, DPI basicList IMAP Capability Command and Response. Captures the listing of capabili- ties that the server supports. imapLogin CERT (PEN 6871) IE 137, variable length, DPI basicList IMAP Login Command. Arguments are user name and password. imapStartTLS CERT (PEN 6871) IE 138, variable length, DPI basicList IMAP STARTTLS Command. Captures this command only as no arguments or re- sponses are related. imapAuthenticate CERT (PEN 6871) IE 139, variable length, DPI basicList IMAP Authenticate Command. Captures the authentication mechanism name of the server following this command. imapCommand CERT (PEN 6871) IE 140, variable length, DPI basicList Captures a variety of IMAP Commands and their arguments. imapExists CERT (PEN 6871) IE 141, variable length, DPI basicList IMAP Exists Response. Reports the number of messages in the mailbox. imapRecent CERT (PEN 6871) IE 142, variable length, DPI basicList IMAP Recent Response. Reports the number of message with the Recent flag set. RTSP Real Time Streaming Protocol (RTSP) Deep Packet Inspection is based on RFC 2326. The following information elements are exported as a template in the subTemplateMultiList as basicLists of variable length elements in the order they are listed in the yafDPIRules.conf file. yaf will always export at least 12 information elements in the RTSP template and data record. By de- fault, the following information elements are exported in order: rtspURL CERT (PEN 6871) IE 143, variable length, DPI basicList RTSP URL. Captures the address of the network resources requested. rtspVersion CERT (PEN 6871) IE 144, variable length, DPI basicList RTSP Version Number. rtspReturnCode CERT (PEN 6871) IE 145, variable length, DPI basicList RTSP Status-Line. Captures the RTSP Protocol version, numeric status code, and the textual phrase associated with the numeric code. rtspContentLength CERT (PEN 6871) IE 146, variable length, DPI basicList RTSP Content-Length Header Field. Contains the length of the content of the method. rtspCommand CERT (PEN 6871) IE 147, variable length, DPI basicList RTSP Command. Captures the method to be performed and the Request-URI associated with the method. rtspContentType CERT (PEN 6871) IE 148, variable length, DPI basicList RTSP Content Type. rtspTransport CERT (PEN 6871) IE 149, variable length, DPI basicList RTSP Transport request header field. Captures the transport protocol used and the parameters that follow. rtspCSeq CERT (PEN 6871) IE 150, variable length, DPI basicList RTSP CSeq field. Contains the sequence number for an RTSP request-re- sponse pair. rtspLocation CERT (PEN 6871) IE 151, variable length, DPI basicList RTSP Location header field. rtspPacketsReceived CERT (PEN 6871) IE 152, variable length, DPI basicList RTSP Packets Received header field. rtspUserAgent CERT (PEN 6871) IE 153, variable length, DPI basicList RTSP User Agent field. Contains information about the user agent origi- nating the request. rtspJitter CERT (PEN 6871) IE 154, variable length, DPI basicList RTSP Jitter Value. SIP Session Initiation Protocol (SIP) Deep Packet Inspection is based on RFC 3261. The following information elements are exported as a template in the subTemplateMultiList as basicLists of variable length elements in the order listed in yafDPIRules.conf. yaf will always export at least 7 information elements in the SIP template and data record. By default, the following information elements are exported in order: sipInvite CERT (PEN 6871) IE 155, variable length, DPI basicList SIP Invite Method. Contains the SIP address and SIP Version Number. sipCommand CERT (PEN 6871) IE 156, variable length, DPI basicList SIP Command. Contains a SIP Method, SIP address, and SIP Version Number. sipVia CERT (PEN 6871) IE 157, variable length, DPI basicList SIP Via contains the SIP Version Number and the address the sender is ex- pecting to receive responses. sipMaxForwards CERT (PEN 6871) IE 158, variable length, DPI basicList SIP Max Forwards contains the limit of number of hops a request can make on the way to its destination. sipAddress CERT (PEN 6871) IE 159, variable length, DPI basicList SIP Address contains the argument of the To, From, or Contact Header Fields. sipContentLength CERT (PEN 6871) IE 160, variable length, DPI basicList SIP Content Length header field. Contains the byte count of the message byte. sipUserAgent CERT (PEN 6871) IE 161, variable length, DPI basicList SIP User Agent Header Field. Contains information about the User Agent Client originating the request. SMTP Simple Mail Transfer Protocol (SMTP) Deep Packet Inspection is based on RFC 2821. Note: The templates used for SMTP DPI changed in YAF 2.12.0. An SMTP conversation is exported as a nested set of templates in subTem- plateMultiList. The outer template (yaf_smtp) includes the following ele- ments. smtpHello CERT (PEN 6871) IE 162, variable length, string SMTP Hello or Extend Hello command. Captures the command and the domain name of the SMTP client. smtpEnhanced CERT (PEN 6871) IE 170, variable length, string Enhanced SMTP. Contains the ESMTP command with the following argument. smtpMessageSize, CERT (PEN 6871) IE 330, 4, unsigned SMTP Message Size smtpStartTLS, CERT (PEN 6871) IE 326, 1, unsigned START TLS. Element indicating whether or not the SMTP session sent the START TLS command. smtpResponseList, CERT (PEN 6871) IE 331, variable length, basicList A basicList of SMTP Replies, represented by smtpResponse (IE CERT/169, variable length, string) elements. Each smtpResponse contains of a three digit number followed by text. smtpMessageList CERT (PEN 6871) IE 336, variable length, subTemplateList A subTemplateList containing zero (but usually at least one) or more records described by the yaf_smtp_message template. Each record repre- sents a single email message sent during the SMTP conversation. That template contains the following elements: smtpSubject CERT (PEN 6871) IE 166, variable length, string The subject of the message. smtpToList CERT (PEN 6871) IE 332, variable length, DPI basicList The SMTP Recipient (RCPT) Command. A basicList containing smtpTo (IE CERT/164, variable length, string) elements. Each smtpTo captures the command and the forward-path of the recipient of the mail data. smtpFromList CERT (PEN 6871) IE 333, variable length, DPI basicList SMTP Mail Command. A basicList containing smtpFrom (IE CERT/163, vari- able length, string) elements. Each stmpFrom contains the reverse-path of the sender mailbox. smtpFilenameList CERT (PEN 6871) IE 334, variable length, DPI basicList SMTP Filename. A basicList containing smtpFilename (IE CERT/167, vari- able length, string) elements. Each smtpFilename contains the name of a file attached to the mail message, if any. smtpURLList CERT (PEN 6871) IE 335, variable length, DPI basicList SMTP URL. A basicList containing smtpURL (IE CERT/329, variable length, string) elements. Each smtpURL contains a URL captured in the SMTP message body, if any. smtpHeaderList CERT (PEN 6871) IE 337, variable length, DPI subTem- plateList A subTemplateList containing zero or more records described by the yaf_smtp_header template. Each record represents a single header (SMTP field name and body) in the email DATA. That template contains the following elements: smtpKey CERT (PEN 6871) IE 327, variable length, string SMTP Header key string. The name of the header (for example, "To", "From"). smtpValue CERT (PEN 6871) IE 328, variable length, string SMTP Header value string. The value of that header. SSH By default, yaf exports 14 information element's in the SSH template and data record. sshVersion CERT (PEN 6871) IE 171, variable length, DPI basicList SSH Client Version Number sshServerVersion CERT (PEN 6871) IE 472, variable length, string The version string from an SSH server. sshKeyExchangeAlgorithm CERT (PEN 6871) IE 476, variable length The negotiated key exchange algorithm used for an SSH session. sshHostKeyAlgorithm CERT (PEN 6871) IE 477, variable length The negotiated host key algorithm used for an SSH session. sshServerHostKey CERT (PEN 6871) IE 478, variable length The MD5 hash of the public key of the SSH server. sshCipher CERT (PEN 6871) IE 473, variable length The negotiated symmetric encryption algorithm used for an SSH session. sshMacAlgorithm CERT (PEN 6871) IE 474, variable length The negotiated MAC algorithm used for an SSH session. sshCompressionMethod CERT (PEN 6871) IE 475, variable length The negotiated compression algorithm used for an SSH session. sshHassh CERT (PEN 6871) IE 468, variable length The client HASSH MD5 hash of the sshHasshAlgorithms (CERT/469) finger- print for an SSH client. sshServerHassh CERT (PEN 6871) IE 470, variable length The server HASSH MD5 hash (hasshServer) of the sshServerHasshAlgorithms (CERT/471) fingerprint for an SSH server. sshHasshAlgorithms CERT (PEN 6871) IE 469, variable length The SSH client hasshAlgorithms: the concatenated name-lists of the client-to-server algorithms delimited by a semicolon. Element sshHassh (CERT/468) holds the MD5 of this. sshServerHasshAlgorithms CERT (PEN 6871) IE 471, variable length The SSH server hasshServerAlgoritms: the concatenated name-lists of the server-to-client algorithms delimited by a semicolon. Element sshServer- Hassh (CERT/470) holds the MD5 of this. DNS Domain Name System (DNS) Deep Packet Inspection is based on RFC 1035. DNS Information is exported in the yaf subTemplateMultiList as a subTem- plateList of Resource Record Templates. Each resource record entry con- tains generic resource record information such as type, TTL, and name. There is also one element (subTemplateList) that contains resource record specific information based on the type of resource record (A Record vs NS Record, for example). The subTemplateList will contain one entry for each resource record in the packet. Due to alignment issues, the resource record specific element is the first element in the template and is there- fore the first item listed below. DNSSEC information is not exported by de- fault. To export DNSSEC information, run yaf with --plugin-opts=DNSSEC. The following information elements exist in the DNS resource record subTem- plateList: DNS Resource Record The following elements (in order) are contained in the DNS Resource Record Template. subTemplateList IE 292, variable length An IPFIX subTemplateList. This list contains a "DNS Resource Record Type" Template. The type of this template depends on the type (dnsQR- Type) of resource record. See the DNS Resource Record Types listed be- low. dnsQName CERT (PEN 6871) IE 179, variable length A DNS Query or Response Name. This field corresponds with the QNAME field in the DNS Question Section or the NAME field in the DNS Resource Record Section. dnsTTL CERT (PEN 6871) IE 199, 4 octets, unsigned DNS Time To Live. This is an unsigned integer that specifies the time interval, in seconds, that the resource record may be cached for. This will contain a value of zero for DNS Queries. dnsQRType CERT (PEN 6871) IE 175, 2 octets, unsigned DNS Query/Response Type. This corresponds with the QTYPE field in the DNS Question Section or the TYPE field in the DNS Resource Record Sec- tion. This field determines the type of subTemplateList found in this record. dnsQueryResponse CERT (PEN 6871) IE 174, 1 octet, unsigned DNS Query/Response header field. This corresponds with the DNS header one bit field, QR. If the message is a query (0), or a response (1). dnsAuthoritative CERT (PEN 6871) IE 176, 1 octet, unsigned DNS Authoritative header field. This corresponds with the DNS header one bit field, AA. This bit is only valid in responses (when dns- QueryResponse is 1), and specifies that the responding name server is an authority for the domain name in the question section. dnsNXDomain CERT (PEN 6871) IE 177, 1 octet, unsigned DNS NXDomain or Response Code (RCODE). This corresponds with the DNS RCODE header field. This field will be set to 3 for a Name Error, 2 for a Server Failure, 1 for a Format Error, and 0 for No Error. See http://www.iana.org/assignments/dns-parameters for other valid values. dnsRRSection CERT (PEN 6871) IE 178, 1 octet, unsigned DNS Resource Record Section Field. This field will be set to 0 if the information is from the Question Section, 1 for the Answer Section, 2 for the Name Server Section, and 3 for the Additional Section. dnsID CERT (PEN 6871) IE 226, 2 octets, unsigned DNS Transaction ID. This identifier is used by the requester to match up replies to outstanding queries. DNS Resource Record Types * DNS A Resource Record This entry will exist if dnsQRType is 1 and the A Record contains an IP address. sourceIPv4Address IE 8, 4 octets, unsigned IPv4 address of the host. * DNS NS Resource Record This entry will exist if dnsQRType is 2 and the NS Record contains an NSDNAME. dnsNSDName CERT (PEN 6871) IE 183, variable length An authoritative name server domain-name. * DNS CNAME Resource Record This entry will exist if dnsQRType is 5 and the CNAME Record contains an CNAME. dnsCName CERT (PEN 6871) IE 180, variable length A domain-name which specifies the canonical or primary name for the owner. * DNS SOA Resource Record This entry will exist if dnsQRType is 6 and the SOA Record contains at least 1 of the following elements: dnsSOAMName CERT (PEN 6871) IE 214, variable length Corresponds to DNS SOA MNAME Field. dnsSOARName CERT (PEN 6871) IE 215, variable length Corresponds to DNS SOA RNAME Field. dnsSOASerial CERT (PEN 6871) IE 209, 4 octets, unsigned Corresponds to DNS SOA SERIAL Field. dnsSOARefresh CERT (PEN 6871) IE 210, 4 octets, unsigned Corresponds to DNS SOA REFRESH Field. dnsSOARetry CERT (PEN 6871) IE 211, 4 octets, unsigned Corresponds to DNS SOA RETRY Field. dnsSOAExpire CERT (PEN 6871) IE 212, 4 octets, unsigned Corresponds to DNS SOA EXPIRE Field. dnsSOAMinimum CERT (PEN 6871) IE 213, 4 octets, unsigned Corresponds to DNS SOA MINIMUM Field. * DNS PTR Resource Record This entry will exist if dnsQRType is set to 12 and PTRDNAME exists. dnsPTRDName CERT (PEN 6871) IE 184, variable length Corresponds to DNS PTR PTRDNAME Field. * DNS MX Resource Record This entry will exist if dnsQRType is set to 15 and MXExchange exists dnsMXExchange CERT (PEN 6871) IE 182, variable length Corresponds to the DNS MX Exchange field. dnsMXPreference CERT (PEN 6871) IE 181, 2 octets, unsigned Corresponds to the DNS MX Preference field. * DNS TXT Resource Record This entry will exist if dnsQRType is set to 16 and TXT-DATA exists. dnsTXTData CERT (PEN 6871) IE 208, variable length Corresponds to DNS TXT TXT-DATA field. * DNS AAAA Record This entry will exist if dnsQRType is set to 28 and the IPv6 Address ex- ists. See RFC 3596. sourceIPv6Address IE 27, 16 octets, unsigned An IPv6 Address found in the data portion of an AAAA Resource Record. * DNS SRV Record This entry will exist if dnsQRType is set to 33 and at least 1 of the following elements exist. See RFC 2782. dnsSRVTarget CERT (PEN 6871) IE 219, variable length Corresponds to the Target Field in the DNS SRV Resource Record. dnsSRVPriority CERT (PEN 6871) IE 216, 2 octets, unsigned Corresponds to the Priority Field in the DNS SRV Resource Record. dnsSRVWeight CERT (PEN 6871) IE 217, 2 octets, unsigned Corresponds to the Weight Field in the DNS SRV Resource Record. dnsSRVPort CERT (PEN 6871) IE 218, 2 octets, unsigned Corresponds to the Port Field in the DNS SRV Resource Record. * DNSSEC DNSKEY Record This entry will exist if dnsQRType is set to 48 and at least 1 of the following elements exist. See RFC 4034. dnsPublicKey CERT (PEN 6871) IE 232, variable length DNSSEC uses public key cryptography to sign and authenticate DNS re- source record sets. This field holds the public key. The format de- pends on the algorithm of the key. dnsFlags CERT (PEN 6871) IE 241, 2 octets, unsigned The flags field in the DNSKey Resource Record. Certain bits determine if the key is a zone key or should be used for a secure entry point. protocolIdentifier IE 4, 1 octet, unsigned The protocol field in the DNSKEY RR. This should be 3 or treated as invalid. dnsAlgorithm CERT (PEN 6871) IE 227, 1 octet, unsigned Identifies the public key's cryptographic algorithm, which determines it's format. * DNSSEC DS Record This entry will exist if dnsQRType is set to 43, yaf was enabled to ex- port DNSSEC information, and at least 1 of the following elements exist. See RFC 4034. dnsDigest CERT (PEN 6871) IE 231, variable length The digest of the DNSKEY RR. dnsKeyTag CERT (PEN 6871) IE 228, 2 octets, unsigned The Key Tag field in the DS RR. dnsAlgorithm CERT (PEN 6871) IE 227, 2 octets, unsigned The Algorithm number of the DNSKEY RR referred to by the DS Record. dnsDigestType CERT (PEN 6871) IE 238, 1 octet, unsigned The Digest Type field which identifies the algorithm used to construct the digest. * DNSSEC NSEC Record This entry will exist if dnsQRType is set to 47, yaf was enabled to ex- port DNSSEC information, and the following field exists. See RFC 4034. dnsHashData CERT (PEN 6871) IE 234, variable length This item contains the Next Domain Name in the NSEC RR. * DNSSEC NSEC3 or NSEC3PARAM Record This entry will exist if dnsQRType is set to 50 or 51, yaf was enabled to export DNSSEC information, and at least one of the following fields exists. See RFC 5155. dnsSalt CERT (PEN 6871) IE 233, variable length The Salt Field in the DNSSEC NSEC3 or NSEC3PARAM RR. dnsHashData CERT (PEN 6871) IE 234, variable length The Next Hashed Owner Name in the DNSSEC NSEC3 RR. This will be empty for NSEC3PARAM records. dnsIterations CERT (PEN 6871) IE 235, 2 octets, unsigned The Iterations field in the DNSSEC NSEC3 or NSEC3PARAM RR. dnsAlgorithm CERT (PEN 6871) IE 227, 2 octets, unsigned The Hash Algorithm field in the DNSSEC NSEC3 or NSEC3PARAM RR. Values are described in RFC 5155. * DNSSEC RRSIG Record This entry will exist if dnsQRType is set to 46, yaf was enabled to ex- port DNSSEC information, and at least one of the following fields ex- ists. See RFC 4034. dnsSigner CERT (PEN 6871) IE 229, variable length The Signer's Name field in the RRSIG RR. dnsSignature CERT (PEN 6871) IE 230, variable length The Signature field in the RRSIG RR. Contains the cryptographic signa- ture that covers the dnsQName field. dnsSignatureInception CERT (PEN 6871) IE 237, 4 octets, unsigned The Signature Inception field in a RRSIG RR. The Expiration and Incep- tion fields specify a validity period for the signature. dnsSignatureExpiration CERT (PEN 6871) IE 236, 4 octets, unsigned The Signature Expiration field in a RRSIG RR. The Expiration and In- ception fields specify a validity period for the signature. dnsTTL CERT (PEN 6871) IE 199, 4 octets, unsigned The Original TTL Field in the RRSIG RR. dnsKeyTag CERT (PEN 6871) IE 228, 2 octets, unsigned The Key Tag field in a RRSIG RR. dnsTypeCovered CERT (PEN 6871) IE 240, 2 octets, unsigned The Type Covered field in a RRSIG RR. dnsAlgorithm CERT (PEN 6871) IE 227, 1 octet, unsigned The Algorithm Number field in a RRSIG RR. Identifies the algorithm used to create the signature. dnsLabels CERT (PEN 6871) IE 239, 1 octet, unsigned The Labels field in a RRSIG RR. Specifies the number of labels in the original RRSIG resource record owner name. SSL/TLS Secure Socket Layer (SSL)/Transport Layer Security (TLS) Deep Packet In- spection can identify and export handshake and certificate information if it is contained in the payload of the flow. Each certificate identified by yaf is exported as an entry in the subTemplateList field below. Each entry in the subTemplateList has three nested subTemplateLists, one for issuer fields, one for subject fields, and one for extension fields, along with other basic handshake elements such as serial numbers and validity time- stamps. Each of the nested subTemplateLists contain an ID and a value. The IDs correspond to the attributes associated with X.509 Certificates, object identifiers id-ce and id-at. sslCipher CERT (PEN 6871) IE 185, 4 octets, unsigned, DPI basicList sslCipher is exported by yaf as a basicList that contains the list of Ci- pherSuites suggested by the client in the ClientHello Message. sslServerCipher CERT (PEN 6871) IE 187, 4 octets, unsigned sslServerCipher is the CipherSuite chosen by the server in the Server- Hello message. sslClientVersion CERT (PEN 6871) IE 186, 1 octet, unsigned sslClientVersion reflects the major version of the SSL protocol the client used in its initial message: 2 if the client used an SSLv2-style Client Hello or 3 if it used a TLS/SSLv3-style Client Hello. sslCompressionMethod CERT (PEN 6871) IE 188, 1 octet, unsigned sslCompressionMethod is the compression method chosen by the server in the ServerHello message. sslRecordVersion CERT (PEN 6871) IE 288, 2 octets, unsigned sslRecordVersion is the version of TLS or SSL advertised in the Clien- tHello or ServerHello. sslClientJA3 CERT (PEN 6871) IE 463, variable length The JA3 MD5 hash of the sslClientJA3Fingerprint (CERT/464) calculated on the client-side TLS/SSL fingerprint string. sslClientJA3Fingerprint CERT (PEN 6871) IE 464, variable length The JA3 fingerprint string enumerated from the TLS/SSL ClientHello packet. Element sslClientJA3 (CERT/463) holds the MD5 of this. sslServerJA3S CERT (PEN 6871) IE 465, variable length The JA3S MD5 hash of the sslServerJA3SFingerprint (CERT/466) calculated on the server-side TLS/SSL fingerprint string. sslServerJA3SFingerprint CERT (PEN 6871) IE 466, variable length The JA3S fingerprint string enumerated from the TLS/SSL ServerHello packet. Element sslServerJA3S (CERT/465) holds the MD5 of this. subTemplateList IE 292, variable length This contains 0 or more X.509 Certificates as available to yaf in the captured payload. Note that most certificate chains are about 3000 bytes. In order to capture the entire certificate chain, --max-payload should be set appropriately. subTemplateList IE 292, variable length The Issuer field identifies the entity that has signed and issued the certificate. It is encoded as a sequence of Relative Distinguished Names, which are basically type, value pairs. This list will contains zero or more occurrences of the RelativeDistinguishedName id, value pairs pulled from the X.509 Certificate Issuer RDNSequence. There will be one entry in the list for each pair. See below for a common list of attributes. subTemplateList IE 292, variable length The Subject field identifies the entity associated with the public key stored in the subject public key field. It is encoded as a sequence of Relative Distinguished Names, which are basically type, value pairs. This list will contains zero or more occurrences of the RelativeDistin- guishedName id, value pairs pulled from the X.509 Certificate Subject RDNSequence. There will be one entry in the list for each pair. See below for a common list of attributes. subTemplateList IE 292, variable length Extensions are only defined for X.509 v3 certificates and provide meth- ods for associating additional attributes with the Issuer and Subject information. Each extension includes an object identifier and an ASN.1 structure. This list will contain zero or more occurrences of the ob- ject ids and ASN.1 values. yaf will not parse the ASN.1 values for the string objects, it includes the entire ASN.1 structure in the value field. However, it does not contain the entire Extension ID. yaf only parses extensions that are members of the id-ce arc and only exports information about the following objects: id-ce-subjectKeyIdentifier {id-ce 14} id-ce-keyUsage {id-ce 15} id-ce-privateKeyUsagePeriod {id-ce 16} id-ce-subjectAltName {id-ce 17} id-ce-issuerAltName {id-ce 18} id-ce-certificateIssuer {id-ce 29} id-ce-cRLDistributionPoints {id-ce 31} id-ce-certificatePolicies {id-ce 32} id-ce-authorityKeyIdentifier {id-ce 35} id-ce-extKeyUsage {id-ce 37} sslCertSignature CERT (PEN 6871) IE 190, variable length The signature contained in a SSL certificate. This is typically the hashing algorithm identifier. sslCertSerialNumber CERT (PEN 6871) IE 244, variable length The Serial Number from the X.509 certificate. sslCertValidityNotBefore CERT (PEN 6871) IE 247, variable length The notBefore field in the Validity Sequence of the X.509 Certificate. sslCertValidityNotAfter CERT (PEN 6871) IE 248, variable length The notAfter field in the Validity Sequence of the X.509 Certificate. sslPublicKeyAlgorithm CERT (PEN 6871) IE 249, variable length The algorithm, encoded in ASN.1, in the SubjectPublicKeyInfo Sequence of the X.509 Certificate. sslPublicKeyLength CERT (PEN 6871) IE 250, 2 octets, unsigned The length of the public key in the X.509 Certificate. sslCertVersion CERT (PEN 6871) IE 189, 1 octet, unsigned The Certificate Version. This is the value contained in the certificate v1(0), v2(1), v3(2). sslCertificateHash CERT (PEN 6871) IE 295, variable length, optional The hash of the X.509 certificate. This field is only populated if the cert_hash_enabled is present and set to 1. sslServerName CERT (PEN 6871) IE 294, variable length The server name from the SSL/TLS Client Hello. This is typically the name of the server that the client is connecting to. Issuer, Subject, and Extension Templates Each subTemplateList for the above issuer, subject, and extension sequences will contain zero or more entries of the below elements. sslObjectValue CERT (PEN 6871) IE 246, variable length The bit strings associated with the below attribute types. sslObjectType CERT (PEN 6871) IE 245, 1 octet, unsigned Above lists the extension types that yaf will export. For the Issuer and Subject subTemplateLists, yaf only parses objects that are members of the id-at arc {joint-iso-ccitt(2) ds(5) 4}, pkcs-9 {iso(1) mem- ber-body (2) us(840) rsadsi(113459) pkcs(1) 9}, and LDAP dc 0.9.2342.19200300.100.1.25. This field will not contain the full object identifier, it will just contain the member id. For example, for an is- suer common name, sslObjectType will contain 3. Below is a list of com- mon objects in an X.509 RelativeDistinguishedName Sequence for X.509 Certificates: pkcs-9-emailAddress {pkcs-9 1} id-at-commonName {id-at 3} id-at-countryName {id-at 6} id-at-localityName {id-at 7} id-at-stateOrProvinceName {id-at 8} id-at-streetAddress {id-at 9} id-at-organizationName {id-at 10} id-at-organizationalUnitName {id-at 11} id-at-title {id-at 12} id-at-postalCode {id-at 17} 0.9.2342.19200300.100.1.25 {dc 25} id-at-name {id-at 41} Full Certificate Template yaf will export the full X.509 certificate if the cert_export_enabled vari- able is present and set to 1 in the configuration file. The following in- formation is exported as an extra entry in the subTemplateMultiList as a basicList: sslCertificate, CERT (PEN 6871) IE 296, variable length, DPI basicList IRC Internet Relay Chat (IRC) Deep Packet Inspection is based on RFC 2812. The following information element is exported as a template in the subTemplate- MultiList as a basicList of variable length elements in the following or- der: ircTextMessage CERT (PEN 6871) IE 125, variable length, DPI basicList IRC Chat or Join Message. This field contains any IRC Command and the following arguments. NNTP Network News Transfer Protocol (NNTP) Deep Packet Inspection is based on RFC 977. The following information elements are exported as a template in the subTemplateMultiList in the following order: nntpResponse CERT (PEN 6871) IE 172, variable length NNTP Reply. This consists of a three digit status code and text message. nntpCommand CERT (PEN 6871) IE 173, variable length NNTP Command. Contains an NNTP Command and following argument(s). POP3 Post Office Protocol 3 (POP3) Deep Packet Inspection is based on RFC 1939. The following information element is exported as a template in the subTem- plateMultiList as a basicList of variable length elements: pop3TextMessage CERT (PEN 6871) IE 124, variable length, DPI basicList POP3 Command and Replies. Contains any command or reply message found in POP3 payload data. SLP Service Location Protocol (SLP) Deep Packet Inspection is based on RFC 2608. The following information elements are exported as a template in the subTemplateMultiList in the following order: slpString CERT (PEN 6871) IE 130, variable length, DPI basicList Contains the text elements found in an SLP Service Request. slpVersion CERT (PEN 6871) IE 128, 1 octet, unsigned SLP Version Number. slpMessageType CERT (PEN 6871) IE 129, 1 octet, unsigned SLP Message Type. This value should be between 1 and 11 and describes the type of SLP message. TFTP Trivial File Transfer Protocol (TFTP) Deep Packet Inspection is based on RFC 1350. The following information elements are exported as a template in the subTemplateMultiList in the following order: tftpFilename CERT (PEN 6871) IE 126, variable length TFTP Name of File being transferred. tftpMode CERT (PEN 6871) IE 127, variable length Contains the mode of transfer. (Currently supported: netascii, octet, mail). MySQL MySQL Deep Packet Inspection is based on information found at http://forge.mysql.com/wiki/MySQL_Internals_ClientServer_Protocol. MySQL packet capture information is exported in the yaf subTemplateMultiList as a subTemplateList of Command Code, Command Text pairs. subTemplateList IE 292, variable length An IPFIX SubTemplateList. This type represents a list of zero or more instances of a structured data type, where the data type of each list el- ement is the same and corresponds with a single Template Record. In this case, a list of MySQL Command Code, Command Text Pairs. There will be one element in the list for each MySQL Command found. mysqlCommandText CERT (PEN 6871) IE 225, variable length MySQL Command Text. For example, this can be a SELECT, INSERT, DELETE statement. This is the first element in the MySQL subTem- plateList. mysqlCommandCode CERT (PEN 6871) IE 224, 1 octet, unsigned MySQL Command Code. This number should be between 0 and 28. This is the second element in the above MySQL subTemplateList. mysqlUsername CERT (PEN 6871) IE 223, variable length MySQL Login User Name. DNP3 Distributed Network Protocol (DNP3) Deep Packet Inspection is slightly dif- ferent than other plugin-based protocols. YAF will export the following information if the yafDPIRules.conf contain regular expressions with the label ID 20000. The regular expressions are compared against the payload of DNP3 packets starting with the function code in the DNP Application Layer header. YAF will loop through all the the available DNP3 packets contained in the captured payload. For each packet that matches one of the regular expressions listed in yafDPIRules.conf, YAF will include an entry in the exported subTemplateList. The subTemplateMultiList contains the following information elements in the following order: subTemplateList IE 292, variable length An IPFIX SubTemplateList. This type represents a list of zero or more in- stances of a structured data type, where the data type of each list ele- ment is the same and corresponds with a single Template Record. There will be one element in the list for each DNP3 packet that matches one of the DNP3 regular expressions found in the yafDPIRules.conf file. dnp3SourceAddress CERT (PEN 6871) IE 281, 2 octets, unsigned The DNP3 Source Address found in the Data Link Layer of the DNP Header. dnp3DestinationAddress CERT (PEN 6871) IE 282, 2 octets, unsigned The DNP3 Destination Address found in the Data Link Layer of the DNP Header. dnp3Function CERT (PEN 6871) IE 283, 1 octet, unsigned The DNP3 Function Code found in the first byte of the Application Layer. dnp3ObjectData CERT (PEN 6871) IE 284, variable length The pattern captured from the DNP3 regular expression in yafDPIR- ules.conf Modbus Modbus DPI is similar to DNP3 DPI. YAF will export any patterns matched by the regular expressions labeled with the ID 502 found in the yafDPIR- ules.conf file. The regular expressions are compared against the payload of all valid Modbus packets starting right after the MBAP header (offset 7), beginning with the Modbus function code. The information is exported as variable length fields in a single BasicList. All regular expressions for Modbus should use the label 502. No user-defined information elements will be accepted for Modbus. modbusData CERT (PEN 6871) IE 285, variable length, DPI basicList Any patterns captured from the Modbus regular expressions in yafDPIR- ules.conf Ethernet/IP Ethernet/IP DPI is similar to Modbus DPI. YAF will export any patterns matched by the regular expressions labeled with the ID 44818 in the yafD- PIRules.conf file. The regular expressions are compared against the start of the payload of all valid Ethernet/IP packets (Command in the Encapsula- tion Header is the first byte). The matched patterns are exported as vari- able length fields in a single BasicList. All regular expressions for Eth- ernet/IP should use the label 44818. No user-defined information elements will be accepted for Ethernet/IP. ethernetIPData CERT (PEN 6871) IE 286, variable length, DPI basicList The pattern captured from the Ethernet/IP regular expressions in yafDPIR- ules.conf RTP YAF will export the Payload Type in the Real-time Transport Protocol (RTP) header if RTP DPI is enabled (yes by default). The Payload Type indicates the format of the payload and how it should be interpreted by the receiving application. The following two elements will be exported for each flow la- beled as RTP. If the flow is a uniflow, the reverse element will be ex- ported but will contain the value of 0. rtpPayloadType CERT (PEN 6871) IE 287, 1 octet, unsigned The payload type in the RTP header of the first payload in the forward direction. reverseRtpPayloadType CERT (PEN 6871) IE 16672, 1 octet, unsigned The payload type in the RTP header of the first payload in the reverse direction. AUTHORS Emily Sarneso and the CERT Network Situational Awareness Group Engineering Team <netsa-help@cert.org>. SEE ALSO yaf(1), yafscii(1), applabel(1), super_mediator(1), PCRE2 Documentation 2.19.3 4-Aug-2026 YAFDPI(1)
NAME | DESCRIPTION | DPI CONFIG FILE FORMAT | DPI in Action | Configure Options | DPI Data Export | AUTHORS | SEE ALSO
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=yafdpi&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>