FreeBSD Manual Pages
hpct-importer(1) General Commands Manual hpct-importer(1) NAME hpct-importer -- HiPerConTracer Trace Data Importer SYNOPSIS hpct-importer importer_configurati-o-nimporter-confiimporter_configuration] [-database_configurati-o-ndatabase-confdatabase_configuration] [-L level | --loglevel level] [-Z on|off | --logcolor on|off] [-O file | --logfile file] [-q | --quiet] [-v | --verbose] [-X mode | --import-mode mode] [-Y depth | --import-max-depth depth] [-I path | --import-file-path path] [-B path | --bad-file-path path] [-G path | --good-file-path path] [-Ffilter_regexp|--import-file-path-filter filter_regexp] [-Q | --quit-when-idle] [--ping-workers number] [--ping-files number] [--traceroute-workers number] [--traceroute-files number] DESCRIPTION hpct-importer imports HiPerConTracer results into an SQL or NoSQL data- base. Currently, the following database backends are supported: MySQL/MariaDB (SQL), PostgreSQL (SQL), MongoDB (NoSQL). Results files imported successfully can be moved into a "good" direc- tory, deleted or kept as-in (for debugging only!). Results files which failed to import a re moved into a "bad" directory. The input of multiple input files can be combined into a single data- base transaction, to improve performance on short files. In case of an import error, each file is imported sequentially to identify faulty ones. ARGUMENTS The following argument may be provided: -L level | --loglevel level Sets the minimum logging level to the provided value: 0=trace, 1=debug, 2=info, 3=warning, 4=error, 5=fatal. -Z on|off | --logcolor on|off Enables (on) or disables (off) the usage of ANSI color escape sequences to colorise the log output according to its level. -O file | --logfile file Sets a file name for appending the log output to. If a file is set, no log output is made to the console. Otherwise, all log output will be written to the console. -q | --quiet Sets the minimum logging level to 3 (warning). -v | --verbose Sets the minimum logging level to 0 (trace). -C importer_configuration | --importer-config importer_configuration Sets the name of the importer configuration file. This option must be provided! See /usr/share/hipercontracer/hipercontracer-importer.conf for an example. -D database_configuration | --database-config database_configuration Sets the name of the database configuration file. This option must be provided! See /usr/share/hipercontracer/hipercontracer-database.conf for an example. -X mode | --import-mode mode Specifies the import mode, overriding the mode read from the database configuration file: DeleteImportedFiles (delete suc- cessfully imported files) MoveImportedFiles (move successfully imported files into "good" directory), KeepImportedFiles (keep successfully imported files where they are). Do not use Keep- ImportedFiles execpt for testing. Otherwise, the next re-run of hpct-importer tries to import them again! They may create du- plicates when the underlying database system (e.g. NoSQL with- out unique indices) does not ensure uniqueness! -Y depth | --import-max-depth depth Specifies the import directory traversal depth, overriding the mode read from the database configuration file. -I path | --import-file-path path Specifies the import file path, overriding the mode read from the database configuration file. This path is used to look for files to import. -B path | --bad-file-path path Specifies the "good" file path, overriding the mode read from the database configuration file. This path is used to move successfully imported files to, when the import mode is set to KeepImportedFiles. -G path | --good-file-path path Specifies the "bad" file path, overriding the mode read from the database configuration file. This path is used to move files to which failed to import. -F filter_regexp | --import-file-path-filter filter_regexp Specifies the import path filter, overriding the mode read from the database configuration file. This filter can be used to only import a subset of the import files for testing. The fil- ter expression is an std::regex regular expression. -Q | --quit-when-idle Quit importer when all workers are idle. This will perform only one import round. --ping-workers number Sets the number of Ping import workers. Default: 1. An import file is distributed to a worker by hashing the source address given in the file name. --ping-files number Sets the number of Ping import files per database transaction. Default: 1. --traceroute-workers number Sets the number of Traceroute import workers. Default: 1. An import file is distributed to a worker by hashing the source address given in the file name. --traceroute-files number Sets the number of Traceroute import files per database trans- action. Default: 1. EXAMPLES hpct-importer -C hipercontracer-importer.conf -D hipercontracer-database.conf hpct-importer --importer-config hipercontracer-importer.conf --database-config hipercontracer-database.conf --verbose hpct-importer -C hipercontracer-importer.conf -D hipercontracer-database.conf -F 1234/.* AUTHORS Thomas Dreibholz https://www.nntb.no/~dreibh/hipercontracer mailto://dreibh@simula.no hpct-importer December 19, 2024 hpct-importer(1)
NAME | SYNOPSIS | DESCRIPTION | ARGUMENTS | EXAMPLES | AUTHORS
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=hpct-importer&sektion=1&manpath=FreeBSD+Ports+14.3.quarterly>