silk-plugin(3) SiLK Tool Suite silk-plugin(3) NAME silk-plugin - Creating a SiLK run-time plug-in using C SYNOPSIS sk_cc=`silk_config --compiler` sk_cflags=`silk_config --cflags` $sk_cc $sk_cflags -shared -o FILENAME.so FILENAME.c rwfilter --plugin=FILENAME.so [--plugin=FILENAME.so ...] ... rwcut --plugin=FILENAME.so [--plugin=FILENAME.so ...] --fields=FIELDS ... rwgroup --plugin=FILENAME.so [--plugin=FILENAME.so ...] --id-fields=FIELDS ... rwsort --plugin=FILENAME.so [--plugin=FILENAME.so ...] --fields=FIELDS ... rwstats --plugin=FILENAME.so [--plugin=FILENAME.so ...] --fields=FIELDS --values=VALUES ... rwuniq --plugin=FILENAME.so [--plugin=FILENAME.so ...] --fields=FIELDS --values=VALUES ... DESCRIPTION Several of the SiLK analysis tools allow the user to augment the tools' functionality through the use of plug-ins that get loaded at run-time. These tools are: rwfilter(1) Supports adding new switches to determine whether each SiLK Flow record should be written in the --pass or the --fail output stream. rwcut(1) Supports adding new output fields that, when selected using the --fields switch, appear as a column in the output. rwsort(1) Supports adding new key fields that, when selected using the --fields switch, are used to determine the order in which records are sorted. rwgroup(1) Supports adding new key fields that, when selected using the --id-fields switch, are used to determine how records are grouped. rwuniq(1) Supports adding new key fields that, when selected using the --fields switch, are used to bin (i.e., group) the records. In addition, rwuniq supports adding new aggregate value fields that, when selected using the --values switch, will be computed for each bin. The key and value fields will appear in the output. rwstats(1) Supports adding new key fields that, when selected using the --fields switch, are used to bin (i.e., group) the records. In addition, rw- stats supports adding new aggregate value fields that, when selected using the --values switch, will be computed for each bin and can be used to determine the top-N (or bottom-N) bins. The key and value fields will appear in the output for bins that meet the top-N thresh- old. rwptoflow(1) Supports adding functionality to ignore packets in the pcap(3) input stream or to modify the SiLK Flow records as the records are generated. In addition, all of the above tools support adding new command line switches that can be used to initialize the plug-in itself (for example, to load an auxiliary file that the plug-in requires). The plug-ins for all tools except rwptoflow can be written in either C or using PySiLK (the SiLK Python extension, see pysilk(3)). Although the exe- cution time for PySiLK plug-ins is slower than for C plug-ins, we encourage you to use PySiLK for your plug-ins since the time-to-result can be faster for PySiLK: The faster development time in Python typically more than com- pensates for the slower execution time. Once you find that your PySiLK plug-in is seeing a great deal of use, or that PySiLK is just too slow for the amount of data you are processing, then re-write the plug-in using C. Even when you intend to write a plug-in using C, it can be helpful to pro- totype your plug-in using PySiLK. The remainder of this document explains how to create a plug-in for the SiLK analysis tools (except rwptoflow) using the C programming language. For information on creating a plug-in using PySiLK, see silkpython(3). A template file for plug-ins is included in the SiLK source tree, in the silk-VERSION/src/template/c-plugin.c file. The setup function When you provide "--plugin=my-plugin.so" on the command line to an applica- tion, the application loads the my-plugin.so file and calls a setup func- tion in that file to determine the new switches and/or fields that "my-plu- gin.so" provides. This setup function is called with three arguments: the first two describe the version of the plug-in API, and the third is a pointer that is cur- rently unused. skplugin_err_t SKPLUGIN_SETUP_FN( uint16_t major_version, uint16_t minor_version, void *plug_in_data) { ... } There are several tasks this setup function may do: (1) check the API ver- sion, (2) register new command line switches (if any), (3) register new filters (if any), and (4) register new fields (if any). Let's describe these in more detail. (1) Check the API version The setup function should ensure that the plug-in and the application agree on the API to use. This provides protection in case the SiLK API to plug-ins changes in the future. To make this determination, call the skpinSimpleCheckVersion() function. A typical invocation is shown here, where the "major_version" and "minor_version" were passed into the SKPLU- GIN_SETUP_FN, and "PLUGIN_API_VERSION_MAJOR" and "PLUGIN_API_VERSION_MINOR" are macros defined in the template file to the current version of the API. #define PLUGIN_API_VERSION_MAJOR 1 #define PLUGIN_API_VERSION_MINOR 0 /* Check the plug-in API version */ rv = skpinSimpleCheckVersion(major_version, minor_version, PLUGIN_API_VERSION_MAJOR, PLUGIN_API_VERSION_MINOR, skAppPrintErr); if (rv != SKPLUGIN_OK) { return rv; } (2) Register command line switches If the plug-in wants to define new command line switches, those switches must be registered in the setup function. A typical use of a command line switch is to allow the user to configure the plug-in; for example, the switch may allow the user to specify the location of an auxiliary input file that the plug-in requires, or to set a parameter used by the plug-in. A second use for a command line switch is more subtle. When creating a plug-in for rwfilter, you may want your plug-in to provide several similar features, and only enable each feature when the user requests it via a com- mand line switch. For this case, you want to delay registering the filter until the command line switch is seen, in which case the filter registra- tion function should be invoked in the switch's callback function. Information on registering a command line switch is available below ("Reg- istering command line switches"). (3) Register filters You only need to register filters when the plug-in will be used by rwfil- ter(1). You may choose to register the filters in the setup function; if you do, the filter will always be used when the plug-in is loaded by rwfil- ter. If you the plug-in provides several filtering functions that the user may choose from via command line switches, you should call the filter reg- istration function in the callback function for the command line switch. See "Registering filter functions" for details on registering a function to use with rwfilter. (4) Register fields If you want your plug-in to create a new printable field for rwcut(1), a new sorting field for rwsort(1), a new grouping field for rwgroup(1), rw- stats(1), or rwuniq(1), or a new aggregate value field for rwstats or rwu- niq, you should register those fields in the setup function. (While you can register the fields in a switch's callback function, there is usually little reason to do so.) There are two interfaces to registering a new field: 1. The advanced interface provides complete control over how the field is defined, and allows (or forces) you to specify exactly how to map from a SiLK Flow record to a binary representation to a textual representa- tion. To use the advanced interface you will need to define several functions and fill in a C structure with pointers to those functions. This interface is described in the "Advanced field registration func- tion" section below. 2. The simple interface can be used to define fields that map to an inte- ger value, an IP address, or text that is index by an integer value. To use this interface, you need to define only one or two functions. The simple interface should handle many common cases, and it is de- scribed in "Simple field registration functions". Registering command line switches When you register a switch, the two important pieces of information you must provide are a name for the switch and a callback function. When the application encounters the command line switch registered by your plug-in, the application will invoke the callback function with the parameter that the user provided (if any) to the command line switch. To register a command line switch, call the skpinRegOption2() function: skplugin_err_t skpinRegOption2( const char *option_name, skplugin_arg_mode_t mode, const char *option_help_string, skplugin_help_fn_t option_help_fn, skplugin_option_fn_t opt_process_fn, void *opt_callback_data, int num_fn_mask, ...); /* list of skplugin_fn_mask_t */ The parameters are "option_name" Specifies the command line switch to create. Do not include the lead- ing "--" characters in the name. "mode" Determines whether the switch takes an argument. It should be one of "NO_ARG" when the command line option acts as an on/off switch "OPTIONAL_ARG" when the command line option has a default value, or "REQUIRED_ARG" when the user of the plug-in must provide an argument to the com- mand line option. "option_help_string" This parameter specifies the usage string to print when the user re- quests --help from the application. This parameter may be NULL. Al- ternatively, you may instruct the application to generate a help string by invoking a callback function your plug-in provides, as described next. "option_help_fn" This parameter specifies a pointer to a function that the application will to call to print a help message for the command line switch when the user requests --help from the application. This parameter may be NULL; if it is not NULL, the "option_help_string" value is ignored. The signature of the function to provide is void option_help_fn( FILE *file_handle, const struct option *option, void *opt_callback_data); The "file_handle" argument is where the function should print its help message. The "opt_callback_data" is the value provided to skpinRegOp- tion2() when the option was registered. The "struct option" parameter has two members of interest: "name" contains the number used to regis- ter the option, and "has_arg" contains the "mode" that was used when the option was specified. "opt_process_fn" Specifies the callback function, whose signature is skplugin_err_t opt_process_fn( const char *opt_arg, void *opt_callback_data); The application will call "opt_process_fn(opt_arg,opt_callback_data)" when --option_name is seen as a command line argument. "opt_arg" will be the parameter the user passed to the switch, or it will be NULL if no parameter was given. "opt_callback_data" Will be passed back unchanged to the plug-in as a parameter in the opt_process_fn() and option_help_fn() callback functions. "num_fn_mask" Specifies the number of "skplugin_fn_mask_t" values specified as the final argument(s) to skpinRegOption2(). "..." Specifies a list of "skplugin_fn_mask_t" values. The length of this list must be specified in the "num_fn_mask" parameter. A plug-in file (e.g., my-plugin.so) can be loaded into any SiLK tool that supports plug-ins, but you may want a command line switch to appear only in cer- tain applications. For example, the flowrate(3) plug-in can be used in both rwfilter and rwcut. When used by rwfilter, flowrate provides a --bytes-per-second switch; when used by rwcut, that switch is not available, and instead the bytes/sec field becomes available. This list determines in which applications the switch gets defined, and the list should contain the "SKPLUGIN_FN_*" or "SKPLUGIN_APP_*" macros de- fined in skplugin.h. To make the switch available in all applications, specify "SKPLUGIN_FN_ANY". When skpinRegOption2() is called in an the application that does not match a value in this list, the function re- turns SKPLUGIN_ERR_DID_NOT_REGISTER, indicating that this option is not applicable to the application. Registering filter functions When you register a filter function, you are specifying a function that rw- filter will call for every SiLK Flow record that rwfilter reads from its input files. If the function returns "SKPLUGIN_FILTER_PASS", rwfilter writes the record into the stream(s) specified by --pass. The record goes to the --fail streams if the function returns "SKPLUGIN_FILTER_FAIL". (The previous paragraph is true only when the plug-in is the only filtering predicate. When multiple tests are specified on the rwfilter command line, rwfilter will put the record into the fail destination as soon as any test fails. If there are multiple tests, your plug-in function will only see records that have not yet failed a test. If a plug-in filter function fol- lows your function, it may fail a record that your filter function passed.) To register a filter function, call the following function: skplugin_err_t skpinRegFilter( skplugin_filter_t **filter_handle, const skplugin_callbacks_t *regdata, void *cbdata); "filter_handle" When this parameter is not NULL, skpinRegFilter() will set the location it references to the newly created filter. Currently, no other func- tion accepts the "skplugin_filter_t" as an argument. "cbdata" This parameter will be passed back unchanged to the plug-in as a para- meter in the various callback functions. It may be NULL. "regdata" This structure has a member for every possible callback function the SiLK plug-in API supports. When used by skpinRegFilter(), the follow- ing members are supported. "filter" rwfilter invokes this function for each SiLK flow record. If the function returns SKPLUGIN_FILTER_PASS, the record is accepted; if it returns SKPLUGIN_FILTER_FAIL, the record is rejected. The type of the function is a "skplugin_filter_fn_t", and its signature is: skplugin_err_t filter( const rwRec *rec, void *cbdata, void **extra); where "rec" is the SiLK Flow record, "cbdata" is the "cbdata" spec- ified in skpinRegFilter(), and "extra" will likely be unused. "init" rwfilter invokes this function for all registered filter predi- cates. It is called after argument processing and before reading records. The function's type is "skplugin_callback_fn_t" and the function pointer may be NULL. The callback's signature is skplugin_err_t init( void *cbdata); "cleanup" When this function pointer is non-NULL, rwfilter calls this func- tion after all records have been processed. This function has the same type and signature as the "init" function. The function's return value will be "SKPLUGIN_OK" unless the "filter" mem- ber of the "regdata" structure is NULL. If your plug-in registers a filter function and the plug-in is used in an application other that rwfilter, the call to skpinRegFilter() is a no-op. Simple field registration functions Using a plug-in, you can augment the keys available in the --fields switch on rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1), and provide new aggregate value fields for the --values switch on rwstats and rwuniq. The standard field registration function, skpinRegField(), is power- ful---for example, you can control exactly how the value you compute will be printed. However, that power comes with complexity. Many times, all your plug-in needs to do is to compute a value, and having to write a func- tion to print a number is work with little reward. The functions in this section handle the registration of common field types. All of these functions require a name for the new field. The name is used as one of the arguments to the --fields or --values switch, and the name will also be used as the title when the field is printed (as in rwcut). Field names are case insensitive, and all field names must be unique within an application. You will get a run-time error if you attempt to create a field whose name already exists. (In rwuniq and rwstats, you may have a --fields key and a --values aggregate value with the same name.) The callback functions dealing with integers use "uint64_t" for conve- nience, but internally the value will be stored in a smaller integer field if possible. Specifying the "max" parameter to the largest value you actu- ally use may allow SiLK to use a smaller integer field. The functions in this section return "SKPLUGIN_OK" unless the callback function is NULL. Integer key field The following function is used to register a key field whose value is an unsigned 64 bit integer. skplugin_err_t skpinRegIntField( const char *name, uint64_t min, uint64_t max, skplugin_int_field_fn_t rec_to_int, size_t width); "name" The name of the new key field. "min" A number representing the minimum integer value for the field. "max" A number representing the maximum integer value for the field. If "max" is 0, a value of UINT64_MAX is used instead. "rec_to_int" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents the value of the "name" field for the given record. The signa- ture is uint64_t rec_to_int( const rwRec *rec); "width" The column width to use when displaying the field. If "width" is 0, it will be computed to be the number of digits necessary to display the integer "max". IPv4 key field The following function registers a new key field whose value is an IPv4 ad- dress. skplugin_err_t skpinRegIPv4Field( const char *name, skplugin_ipv4_field_fn_t rec_to_ipv4, size_t width); "name" The name of the new key field. "rec_to_ipv4" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns a 32 bit integer (in host byte order) which repre- sents the IPv4 addresses for the "name" field for the given record. The signature is uint32_t rec_to_ipv4( const rwRec *rec); "width" The column width to use when displaying the field. If "width" is 0, it will be set to 15. IP key field The following function is used to register a key field whose value is any IP address (an "skipaddr_t"). skplugin_err_t skpinRegIPAddressField( const char *name, skplugin_ip_field_fn_t rec_to_ipaddr, size_t width); "name" The name of the new key field. "rec_to_ipaddr" A callback function that accepts a SiLK Flow record and an "skipaddr_t" as arguments. The function should fill in the IP address as required for the "name" field. The signature is void rec_to_ipaddr( skipaddr_t *dest, const rwRec *rec); "width" The column width to use when displaying the field. If "width" is 0, it will be set to 39 when SiLK has support for IPv6 addresses, or 15 oth- erwise. Text key field (from an integer) The following function is used to register a key field whose value is an unsigned 64 bit integer (similar to skpinRegIntField()), but where the printed representation of the field is determined by a second callback function. This allows the plug-in to create arbitrary text for the field. skplugin_err_t skpinRegTextField( const char *name, uint64_t min, uint64_t max, skplugin_int_field_fn_t value_fn, skplugin_text_field_fn_t text_fn, size_t width); "name" The name of the new key field. "min" A number representing the minimum integer value for the field. "max" A number representing the maximum integer value for the field. If "max" is 0, a value of UINT64_MAX is used instead. "value_fn" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents the value of the "name" field for the given record. The signa- ture is uint64_t rec_to_int( const rwRec *rec); "text_fn" A callback function that provides the textual representation of the value returned by "value_fn". The function's signature is void text_fn( char *dest, size_t dest_len, uint64_t val); The callback should fill the character array "dest" with the printable representation of "val". The number of characters in "dest" is given by "dest_len". Note that "dest_len" may be different than the parame- ter "width" passed to skpinRegTextField(), and "text_fn" must NUL-ter- minate the string. "width" The column width to use when displaying the field. Text key field (from a list) The following function is used to register a field whose value is one of a list of strings. The plug-in provides the list of strings and a callback that takes a SiLK Flow record and returns an index into the list of strings. skplugin_err_t skpinRegStringListField( const char *name, const char **list, size_t entries, const char *default_value, skplugin_int_field_fn_t rec_to_index, size_t width); "name" The name of the new key field. "list" List is the list of strings. The list should either be NULL termi- nated, or "entries" should have a non-zero value. "entries" The number of entries in "list". If "entries" is 0, SiLK determines the number of entries by traversing "list" until it finds a element whose value is NULL. "default_value" The value to use when "rec_to_index" returns an invalid value. "rec_to_index" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents an index into "list". If the return value is beyond the end of "list", "default_value" will be used instead. The signature of this callback function is uint64_t rec_to_int( const rwRec *rec); "width" The column width to use when displaying the field. If "width" is 0, it is defaulted to the width of the longest string in "list" and "de- fault_value". Integer sum aggregate value field The following function registers an aggregate value field that maintains a running unsigned integer sum. That is, the values returned by the callback are summed for every SiLK Flow record that matches a bin's key. The sum is printed when the bin is printed. skplugin_err_t skpinRegIntSumAggregator( const char *name, uint64_t max, skplugin_int_field_fn_t rec_to_int, size_t width); "name" The name of the new aggregate value field. "max" A number representing the maximum integer value for the field. If "max" is 0, a value of UINT64_MAX is used instead. "rec_to_int" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents the value of the "name" value field for the given record. The signature is uint64_t rec_to_int( const rwRec *rec); "width" The column width to use when displaying the value. If "width" is 0, it will be computed to be the number of digits necessary to display the integer "max". Integer minimum or maximum aggregate value field The following function registers an aggregate value field that maintains the minimum integer value seen among all values returned by the callback function. skplugin_err_t skpinRegIntMinAggregator( const char *name, uint64_t max, skplugin_int_field_fn_t rec_to_int, size_t width); This function is similar, except it maintains the maximum value. skplugin_err_t skpinRegIntMaxAggregator( const char *name, uint64_t max, skplugin_int_field_fn_t rec_to_int, size_t width); "name" The name of the new aggregate value field. "max" A number representing the maximum integer value for the field. If "max" is 0, a value of UINT64_MAX is used instead. "rec_to_int" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents the value of the "name" value field for the given record. The signature is uint64_t rec_to_int( const rwRec *rec); "width" The column width to use when displaying the value. If "width" is 0, it will be computed to be the number of digits necessary to display the integer "max". Unsigned integer aggregate value field The following function registers an aggregate value field that can be rep- resented by a 64 bit integer. The plug-in must register two callback func- tions. The first takes a SiLK Flow record and returns an integer value; the second takes two integer values (as returned by the first callback function) and combines them to form a new aggregate value. skplugin_err_t skpinRegIntAggregator( const char *name, uint64_t max, skplugin_int_field_fn_t rec_to_int, skplugin_agg_fn_t agg, uint64_t initial, size_t width); "name" The name of the new aggregate value field. "max" A number representing the maximum integer value for the field. If "max" is 0, a value of UINT64_MAX is used instead. "rec_to_int" A callback function that accepts a SiLK Flow record as its sole argu- ment, and returns an unsigned integer (in host byte order) which repre- sents the value of the "name" value field for the given record. The signature is uint64_t rec_to_int( const rwRec *rec); "agg" A callback function that combines (aggregates) two values. For exam- ple, if you wanted to create a new aggregate value that contained a bit-wise OR of the TCP flags seen on every packet, your "agg" function would OR the values. The signature is uint64_t agg( uint64_t current, uint64_t operand); "initial" Specifies the initial value for the aggregate value. The first time the "agg" function is called on a bin, "operand" will be the value re- turned by "rec_to_int", and "current" will be the value given in "ini- tial". The value in "initial" must be less than or equal to the value in "max". "width" The column width to use when displaying the value. If "width" is 0, it will be computed to be the number of digits necessary to display the integer "max". Advanced field registration function When the simple field registration functions do not provide what you need, you can use the skpinRegField() function that gives you complete control over the field. skpinRegField() registers a new derived field for record processing. The plug-in must supply the name of the new field. The name is used as one of the arguments to the --fields switch (for key fields) or --values switch (for aggregate value fields). Field names are case insensitive, and all field names must be unique within an application. You will get a run-time error if you attempt to create a field whose name already exists. (In rwu- niq and rwstats, you may have a --fields key and a --values aggregate value with the same name.) The skpinRegField() function requires you initialize and pass in a struc- ture. In this structure you will specify the callback functions that the application will call, as well as additional information required by some applications. Although the structure is complex, not all applications use all members. If the plug-in is loaded by an application that does not support fields (such as rwfilter), the function is a no-op. The advanced field registration function is skplugin_err_t skpinRegField( skplugin_field_t **return_field, const char *name, const char *description, const skplugin_callbacks_t *regdata, void *cbdata); "return_field" When this value is not NULL, skpinRegField() will set the location it references to the newly created field. "name" This sets the primary name of the field, and by default will be the ti- tle used when printing the field. "description" The "description" provides a textual description of the field. Cur- rently this is unused. "regdata" The "regdata" structure provides the application with the callback functions and additional information it needs to use the plug-in. The members that must be set vary by application. It is described in more detail below. "cbdata" This parameter will be passed back unchanged to the plug-in as a para- meter in the various callback functions. It may be NULL. The structure used by the skpinRegField() (and skpinRegFilter()) functions to specify callback functions is shown here: typedef struct skplugin_callbacks_st { skplugin_callback_fn_t init; skplugin_callback_fn_t cleanup; size_t column_width; size_t bin_bytes; skplugin_text_fn_t rec_to_text; skplugin_bin_fn_t rec_to_bin; skplugin_bin_fn_t add_rec_to_bin; skplugin_bin_to_text_fn_t bin_to_text; skplugin_bin_merge_fn_t bin_merge; skplugin_bin_cmp_fn_t bin_compare; skplugin_filter_fn_t filter; skplugin_transform_fn_t transform; const uint8_t *initial; const char **extra; } skplugin_callbacks_t; All of the callback functions reference in this structure take "cbdata" as a parameter, which is the value that was specified in the call to skpinReg- Field(). The "extra" parameter to the callback functions is used in com- plex plug-ins and can be ignored. The members of the structure are: "init" This specifies a callback function which the application will call when it has determined this field will be used. (In the case of skpinReg- Filter(), the function is called for all registered filters.) The ap- plication calls the function before processing data. It may be NULL; the signature of the callback function is skplugin_err_t init( void *cbdata); "cleanup" When this callback function is not NULL, the application will call it after all records have been processed. It has the same signature as the "init" function. "column_width" The number of characters (not including trailing NUL) required to hold a string representation of the longest value of the field. This value can be 0 if not used (e.g., rwsort does not print fields), or if it will be set later using skpinSetFieldWidths(). "bin_bytes" The number of bytes (octets) required to hold a binary representation of a value of the field. This value can be 0 if not used (e.g., rwcut does not use binary values), or if it will be set later using skpinSet- FieldWidths(). "rec_to_text" The rwcut application uses this callback function to fetch the textual value for the field given a SiLK Flow record. The signature of this function is skplugin_err_t rec_to_text( const rwRec *rec, char *dest, size_t width, void *cbdata, void **extra); The callback function should fill the character array "dest" with the textual value, and the value should be NUL-terminated. "width" speci- fies the overall size of "dest", and it may not have the same value as specified by the "column_width" member. For proper formatting, the callback function should write no more than "column_width" characters into "dest". Note that if an application requires a "rec_to_bin" func- tion and "rec_to_bin" is NULL, the application will use "rec_to_text" if it is provided. The application will use "column_width" as the width for binary values (zeroing out the destination area before it is written to). "rec_to_bin" This callback function is used by the application to fetch the binary value for this field given the SiLK Flow record. The signature of this function is: skplugin_err_t rec_to_bin( const rwRec *rec, uint8_t *dest, void *cbdata, void **extra); The callback function should write exactly "bin_bytes" of data into "dest" (where "bin_bytes" was specified in the call to skpinRegField() or skpinSetFieldWidths()). See also the "rec_to_text" member. "add_rec_to_bin" This callback function is used by rwuniq and rwstats when computing ag- gregate value fields. The application expects this function to get the binary value for this field from the SiLK Flow record and merge it (e.g., add it) to the current value. That is, the function should up- date the value in "current_and_new_value" with the value that comes from the current "rec". The signature is: skplugin_err_t add_rec_to_bin( const rwRec *rec, uint8_t *current_and_new_value, void *cbdata, void **extra); The callback function should write exactly "bin_bytes" of data into "current_and_new_value". "bin_to_text" This callback function is used to get a textual representation of a bi- nary value that was set by a prior call to the "rec_to_bin" or "add_rec_to_bin" functions. The function signature is skplugin_err_t bin_to_text( const uint8_t *bin, char *dest, size_t width, void *cbdata); The binary input value is in "bin", and it is exactly "bin_bytes" in length. The textual output must be written to "dest". The overall size of "dest" is given by "width", which may be different than the "column_width" value that was previously specified. For proper format- ting, the callback function should write no more than "column_width" characters into "dest". "bin_merge" When rwstats and rwuniq are unable to store all values in memory, the applications write their current state to temporary files on disk. Once all input data has been processed, the temporary files are com- bined to produce the output. When a key appears in multiple temporary files, the aggregate values must be merged (for example, the byte count for two keys would be added). This callback function is used to merge aggregate value fields defined by the plug-in. The function signature is below. The "src1_and_dest" parameter will contain a binary aggre- gate value from one of the files, and the "src2" parameter a value from the other. These should be combined and the (binary) result written to "src1_and_dest". The byte length of both parameters is "bin_bytes". skplugin_err_t bin_merge( uint8_t *src1_and_dest, const uint8_t *src2, void *cbdata); "bin_compare" This callback function is used by rwstats when determining the top-N (or bottom-N) bins based on the binary aggregate values. The function accepts two binary values, "value_a" and "value_b", each of length "bin_bytes". The function must set "cmp_result" to an integer less than 0, equal 0, or greater than 0 to indicate whether "value_a" is less than, equal to, or greater than "value_b", respectively. If this function is NULL, memcmp() will be used on the binary values instead. skplugin_err_t bin_compare( int *cmp_result, const uint8_t *value_a, const uint8_t *value_b, void *cbdata); "filter" This callback function is only required when the plug-in will be used by rwfilter, as described above. When defining a field, "filter" is ignored. "transform" This callback function is only required when the plug-in will be used by rwptoflow. This callback allows the plug-in to modify the SiLK Flow record, "rec", before it is written to the output. The callback func- tion should modify "rec" in place; the signature is skplugin_err_t transform( rwRec *rec, void *cbdata, void **extra); "initial" When the "initial" member is not NULL, it should point to a value con- taining at least "bin_bytes" bytes. These bytes will be used to ini- tialize the binary aggregate value. As an example use case, when the plug-in is computing a minimum, it may choose to initialize the field to contain the maximum value. When "initial" is NULL, binary aggregate values are initialized using bzero(). "extra" This member is usually NULL. When not NULL, it points to a NULL-termi- nated constant array of strings representing "extra arguments". These are not often used, and they will not be discussed in this manual page. Once a field is registered, you may make changes to it by calling the addi- tional functions described below. In each of these functions, the "field" parameter is the handle returned when the field was registered. By default, the "name" will also be used as the field's title. To specify a different title, the plug-in may call skplugin_err_t skpinSetFieldTitle( skplugin_field_t field, const char title); To create an alternate name for the field (that is, a name that can be used in the --fields or --values switches) call skplugin_err_t skpinAddFieldAlias( skplugin_field_t field, const char alias); To set or modify the textual and binary widths for a field, use the follow- ing function. This function should called in the field's "init" callback function. skplugin_err_t skpinSetFieldWidths( skplugin_field_t field, size_t field_width_text, size_t field_width_bin); The following table shows when a member of the "skplugin_callbacks_t" structure is required or optional. (Where the table shows "column_width" and "bin_bytes" as required, the values can be set in the structure or via the skpinSetFieldWidths() function.) rwfilter rwcut rwgroup rwsort rwstats rwuniq rwptoflow init r f f f f,a f,a r cleanup r f f f f,a f,a r column_width . F . . F,A F,A . bin_bytes . . F F F,A F,A . rec_to_text . F . . . . . rec_to_bin . . F F F F . add_rec_to_bin . . . . A A . bin_to_text . . . . F,A F,A . bin_merge . . . . A A . bin_compare . . . . A . . initial . . . . a a . filter R . . . . . . transform . . . . . . R extra r f f f f,a f,a r The legend is F required for a key field A required for an aggregate value field R required for a non-field application (e.g., rwfilter) f optional for a key field a optional for an aggregate value field r optional for a non-field application . ignored Miscellaneous functions The following registers a cleanup function for the plug-in. This function will be called by the application after any field- or filter-specific cleanup functions are called. Specifically, this is the last callback that the application will invoke on a plug-in. skplugin_err_t skpinRegCleanup( skplugin_cleanup_fn_t cleanup); The signature of the "cleanup" function is: void cleanup(void); The plug-in author should invoke the following function to tell rwfilter that this plug-in is not thread safe. Calling this function causes rwfil- ter not use multiple threads; as such, this function should only be called when the plug-in has registered an active filter function. void skpinSetThreadNonSafe(void); Compiling the plug-in Once you have finished writing the C code for the plug-in, save it in a file. The following uses the name my-plugin.c for the name of this file. In the following, the leading dollar sign ("$") followed by a space repre- sents the shell prompt. The text after the dollar sign represents the com- mand line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line. When compiling a plug-in, you should use the same compiler and compiler-op- tions as when SiLK was compiled. The silk_config(1) utility can be used to obtain that information. To store the compiler used to compile SiLK into the variable "sk_cc", specify the following at a shell prompt (note that those are backquotes, and this assumes a Bourne-compatible shell): $ sk_cc=`silk_config --compiler` To get the compiler flags used to compile SiLK: $ sk_cflags=`silk_config --cflags` Using those two variables, you can now compile the plug-in. The following will work on Linux and Mac OS X: $ $sk_cc $sk_cflags -shared -o my-plugin.so my-plugin.c For Mac OS X: $ $sk_cc $sk_cflags -bundle -flat_namespace -undefined suppress \ -o my-plugin.so my-plugin.c If there are compilation errors, fix them and compile again. Notes: The preceding assumed you were building the plug-in after having in- stalled SiLK. The paths given by silk_config do not work if SiLK has not been installed. To compile the plug-in, you must have access to the SiLK header files. (If you are using an RPM installation of SiLK, ensure that the "silk-devel" RPM is installed.) Once you have created the my-plugin.so file, you can load it into an appli- cation by using the --plugin switch on the application as shown in the "SYNOPSIS". When loading a plug-in from the current directly, it is best to prefix the filename with "./": $ rwcut --plugin=./my-plugin.so ... If there are problems loading the plug-in into the application, you can trace the actions the application is doing by setting the SILK_PLUGIN_DEBUG environment variable: $ SILK_PLUGIN_DEBUG=1 rwcut --plugin=./my-plugin.so ... EXAMPLES rwfilter Suppose you want to find traffic destined to a particular host, 10.0.0.23, that is either ICMP or coming from 1434/udp. If you attempt to use: $ rwfilter --daddr=10.0.0.23 --proto=1,17 --sport=1434 \ --pass=outfile.rw flowrec.rw the --sport option will not match any of the ICMP traffic, and your result will not contain ICMP records. To avoid having to use two invocations of rwfilter, you can create the following plug-in to do the entire check in a single pass: #include <silk/silk.h> #include <silk/rwrec.h> #include <silk/skipaddr.h> #include <silk/skplugin.h> #include <silk/utils.h> /* These variables specify the version of the SiLK plug-in API. */ #define PLUGIN_API_VERSION_MAJOR 1 #define PLUGIN_API_VERSION_MINOR 0 /* ip to search for */ static skipaddr_t ipaddr; /* * status = filter(rwrec, reg_data, extra); * * The function should examine the SiLK flow record and return * SKPLUGIN_FILTER_PASS to write the rwRec to the * pass-destination(s) or SKPLUGIN_FILTER_FAIL to write it to the * fail-destination(s). */ static skplugin_err_t filter( const rwRec *rwrec, void *reg_data, void **extra) { skipaddr_t dip; rwRecMemGetDIP(rwrec, &dip); if (0 == skipaddrCompare(&dip, &ipaddr) && (rwRecGetProto(rwrec) == 1 || (rwRecGetProto(rwrec) == 17 && rwRecGetSPort(rwrec) == 1434))) { return SKPLUGIN_FILTER_PASS; } return SKPLUGIN_FILTER_FAIL; } /* The set-up function that the application will call. */ skplugin_err_t SKPLUGIN_SETUP_FN( uint16_t major_version, uint16_t minor_version, void *plug_in_data) { uint32_t ipv4; skplugin_err_t rv; skplugin_callbacks_t regdata; /* Check the plug-in API version */ rv = skpinSimpleCheckVersion(major_version, minor_version, PLUGIN_API_VERSION_MAJOR, PLUGIN_API_VERSION_MINOR, skAppPrintErr); if (rv != SKPLUGIN_OK) { return rv; } /* set global ipaddr */ ipv4 = ((10 << 24) | 23); skipaddrSetV4(&ipaddr, &ipv4); /* register the filter */ memset(®data, 0, sizeof(regdata)); regdata.filter = filter; return skpinRegFilter(NULL, ®data, NULL); } Once this file is created and compiled, you can use it from rwfilter as shown here: $ rwfilter --plugin=./my-plugin.so --pass=outfile.rw flowrec.rw Additional examples For additional examples, see the source files in silk-VERSION/src/plugins. ENVIRONMENT SILK_PATH This environment variable gives the root of the install tree. When searching for plug-ins, a SiLK application may use this environment variable. See the "FILES" section for details. SILK_PLUGIN_DEBUG When set to 1, the SiLK applications print status messages to the stan- dard error as they attempt to find and open each plug-in. In addition, when an attempt to register a field fails, the application prints a message specifying the additional function(s) that must be defined to register the field in the application. Be aware that the output can be rather verbose. FILES ${SILK_PATH}/lib64/silk/ ${SILK_PATH}/lib64/ ${SILK_PATH}/lib/silk/ ${SILK_PATH}/lib/ /usr/local/lib64/silk/ /usr/local/lib64/ /usr/local/lib/silk/ /usr/local/lib/ Directories that a SiLK application checks when attempting to load a plug-in. SEE ALSO rwfilter(1), rwcut(1), rwgroup(1), rwsort(1), rwstats(1), rwuniq(1), silk_config(1), rwptoflow(1), pysilk(3), silkpython(3), flowrate(3), silk(7), pcap(3) SiLK 3.22.2 2026-08-27 silk-plugin(3)
NAME | SYNOPSIS | DESCRIPTION | EXAMPLES | ENVIRONMENT | FILES | SEE ALSO
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=silk-plugin&sektion=3&manpath=FreeBSD+Ports+15.1.quarterly>