VMOD_STD(3) Library Functions Manual VMOD_STD(3) NAME vmod_std - Vinyl Cache Standard Module SYNOPSIS import std [as name] [from "path"] REAL random(REAL lo, REAL hi) REAL round(REAL r) VOID collect(HEADER hdr, STRING sep) STRING querysort(STRING) STRING toupper(STRING s) STRING tolower(STRING s) STRING strstr(STRING s1, STRING s2) BOOL fnmatch(STRING pattern, STRING subject, BOOL pathname, BOOL noescape, BOOL period) STRING fileread(STRING) BLOB blobread(STRING) BOOL file_exists(STRING path) BOOL healthy(BACKEND be) INT port(IP ip) DURATION duration([STRING s], [DURATION fallback], [REAL real], [INT integer]) BYTES bytes([STRING s], [BYTES fallback], [REAL real], [INT integer]) INT integer([STRING s], [INT fallback], [BOOL bool], [BYTES bytes], [DURATION duration], [REAL real], [TIME time]) IP ip(STRING s, [IP fallback], BOOL resolve, [STRING p]) REAL real([STRING s], [REAL fallback], [INT integer], [BOOL bool], [BYTES bytes], [DURATION duration], [TIME time]) TIME time([STRING s], [TIME fallback], [REAL real], [INT integer]) STRING strftime(TIME time, STRING format) VOID log(STRING s) VOID syslog(INT priority, STRING s) VOID timestamp(STRING s) BOOL syntax(REAL) STRING getenv(STRING name) BOOL cache_req_body(BYTES size) VOID late_100_continue(BOOL late) VOID set_ip_tos(INT tos) VOID rollback(HTTP h) BOOL ban(STRING) STRING ban_error() TIME now() DURATION timed_call(SUB) VOID rfc_ttl() DESCRIPTION vmod_std contains basic functions which are part and parcel of Vinyl Cache, but which for reasons of architecture fit better in a VMOD. NUMERIC FUNCTIONS REAL random(REAL lo, REAL hi) Returns a random real number between lo and hi. This function uses the "testable" random generator in vinyld which enables deterministic tests to be run (See debug.srandom CLI command). This func- tion should not be used for cryptographic applications. Example: set beresp.http.random-number = std.random(1, 100); REAL round(REAL r) Rounds the real r to the nearest integer, but round halfway cases away from zero (see round(3)). STRING FUNCTIONS VOID collect(HEADER hdr, STRING sep=", ") Collapses multiple hdr headers into one long header. The default separator sep is the standard comma separator to use when collapsing headers, with an additional whitespace for pretty printing. Care should be taken when collapsing headers. In particular collapsing Set-Cookie will lead to unexpected results on the browser side. Using hdr from obj.http triggers a VCL failure. Examples: std.collect(req.http.accept); std.collect(req.http.cookie, "; "); STRING querysort(STRING) Sorts the query string for cache normalization purposes. Example: set req.url = std.querysort(req.url); STRING toupper(STRING s) Converts the string s to uppercase. Example: set beresp.http.scream = std.toupper("yes!"); STRING tolower(STRING s) Converts the string s to lowercase. Example: set beresp.http.nice = std.tolower("VerY"); STRING strstr(STRING s1, STRING s2) Returns a string beginning at the first occurrence of the string s2 in the string s1, or an empty string if s2 is not found. Note that the comparison is case-sensitive. Example: if (std.strstr(req.url, req.http.restrict)) { ... } This will check if the content of req.http.restrict occurs anywhere in req.url. BOOL fnmatch(STRING pattern, STRING subject, BOOL pathname, BOOL noescape, BOOL period) BOOL fnmatch( STRING pattern, STRING subject, BOOL pathname=1, BOOL noescape=0, BOOL period=0 ) Shell-style pattern matching; returns true if subject matches pattern, where pattern may contain wildcard characters such as * or ?. The match is executed by the implementation of fnmatch(3) on your system. The rules for pattern matching on most systems include the following: * * matches any sequence of characters * ? matches a single character * a bracket expression such as [abc] or [!0-9] is interpreted as a charac- ter class according to the rules of basic regular expressions (not pcre2(3) regexen), except that ! is used for character class negation in- stead of ^. If pathname is true, then the forward slash character / is only matched literally, and never matches *, ? or a bracket expression. Otherwise, / may match one of those patterns. By default, pathname is true. If noescape is true, then the backslash character \ is matched as an ordi- nary character. Otherwise, \ is an escape character, and matches the char- acter that follows it in the pattern. For example, \\ matches \ when noescape is true, and \\ when false. By default, noescape is false. If period is true, then a leading period character . only matches liter- ally, and never matches *, ? or a bracket expression. A period is leading if it is the first character in subject; if pathname is also true, then a period that immediately follows a / is also leading (as in /.). By de- fault, period is false. std.fnmatch() invokes VCL failure and returns false if either of pattern or subject is NULL -- for example, if an unset header is specified. Examples: # Matches URLs such as /foo/bar and /foo/baz if (std.fnmatch("/foo/\*", req.url)) { ... } # Matches URLs such as /foo/bar/baz and /foo/baz/quux if (std.fnmatch("/foo/\*/\*", bereq.url)) { ... } # Matches /foo/bar/quux, but not /foo/bar/baz/quux if (std.fnmatch("/foo/\*/quux", req.url)) { ... } # Matches /foo/bar/quux and /foo/bar/baz/quux if (std.fnmatch("/foo/\*/quux", req.url, pathname=false)) { ... } # Matches /foo/bar, /foo/car and /foo/far if (std.fnmatch("/foo/?ar", req.url)) { ... } # Matches /foo/ followed by a non-digit if (std.fnmatch("/foo/[!0-9]", req.url)) { ... } FILE(SYSTEM) FUNCTIONS STRING fileread(STRING) Reads a text file and returns a string with the content. The entire file is cached on the first call, and subsequent calls will re- turn this cached contents, even if the file has changed in the meantime. For binary files, use std.blobread() instead. Example: synthetic("Response was served by " + std.fileread("/etc/hostname")); Consider that the entire contents of the file appear in the string that is returned, including newlines that may result in invalid headers if std.fil- eread() is used to form a header. In that case, you may need to modify the string, for example with regsub() (see vcl(7)): set beresp.http.served-by = regsub(std.fileread("/etc/hostname"), "\R$", ""); BLOB blobread(STRING) Reads any file and returns a blob with the content. The entire file is cached on the first call, and subsequent calls will re- turn this cached contents, even if the file has changed in the meantime. BOOL file_exists(STRING path) Returns true if path or the file pointed to by path exists, false other- wise. Example: if (std.file_exists("/etc/return_503")) { return (synth(503, "Vinyl Cache is in maintenance")); } TYPE INSPECTION FUNCTIONS BOOL healthy(BACKEND be) Returns true if the backend be is healthy. INT port(IP ip) Returns the port number of the IP address ip. Always returns 0 for a *.ip variable when the address is a Unix domain socket. TYPE CONVERSION FUNCTIONS These functions all have the same form: TYPE type([arguments], [fallback TYPE]) Precisely one of the arguments must be provided (besides the optional fall- back), and it will be converted to TYPE. If conversion fails, fallback will be returned and if no fallback was spec- ified, the VCL will be failed. DURATION duration([STRING s], [DURATION fallback], [REAL real], [INT inte- ger]) DURATION duration( [STRING s], [DURATION fallback], [REAL real], [INT integer] ) Returns a DURATION from a STRING, REAL or INT argument. For a STRING s argument, s must be quantified by ms (milliseconds), s (sec- onds), m (minutes), h (hours),``d`` (days), w (weeks) or y (years) units. real and integer arguments are taken as seconds. If the conversion of an s argument fails, fallback will be returned if pro- vided, or a VCL failure will be triggered. Conversions from real and integer arguments never fail. Only one of the s, real or integer arguments may be given or a VCL failure will be triggered. Examples: set beresp.ttl = std.duration("1w", 3600s); set beresp.ttl = std.duration(real=1.5); set beresp.ttl = std.duration(integer=10); BYTES bytes([STRING s], [BYTES fallback], [REAL real], [INT integer]) BYTES bytes( [STRING s], [BYTES fallback], [REAL real], [INT integer] ) Returns BYTES from a STRING, REAL or INT argument. A STRING s argument can be quantified with a multiplier (k (kilo), m (mega), g (giga), t (tera) or p (peta)). real and integer arguments are taken as bytes. If the conversion of an s argument fails, fallback will be returned if pro- vided, or a VCL failure will be triggered. Other conversions may fail if the argument cannot be represented, because it is negative, too small or too large. Again, fallback will be returned if provided, or a VCL failure will be triggered. real arguments will be rounded down. Only one of the s, real or integer arguments may be given or a VCL failure will be triggered. Example: std.cache_req_body(std.bytes(something.somewhere, 10K)); std.cache_req_body(std.bytes(integer=10*1024)); std.cache_req_body(std.bytes(real=10.0*1024)); INT integer([STRING s], [INT fallback], [BOOL bool], [BYTES bytes], [DURATION duration], [REAL real], [TIME time]) INT integer( [STRING s], [INT fallback], [BOOL bool], [BYTES bytes], [DURATION duration], [REAL real], [TIME time] ) Returns an INT from a STRING, BOOL or other quantity. If the conversion of an s argument fails, fallback will be returned if pro- vided, or a VCL failure will be triggered. A bool argument will be returned as 0 for false and 1 for true. This con- version will never fail. For a bytes argument, the number of bytes will be returned. This conver- sion will never fail. A duration argument will be rounded down to the number of seconds and re- turned. A real argument will be rounded down and returned. For a time argument, the number of seconds since the UNIX epoch (1970-01-01 00:00:00 UTC) will be returned. duration, real and time conversions may fail if the argument can not be represented because it is too small or too large. If so, fallback will be returned if provided, or a VCL failure will be triggered. Only one of the s, bool, bytes, duration, real or time arguments may be given or a VCL failure will be triggered. Examples: if (std.integer(req.http.foo, 0) > 5) { ... } set resp.http.answer = std.integer(real=126.42/3); IP ip(STRING s, [IP fallback], BOOL resolve=1, [STRING p]) Converts the string s to the first IP number returned by the system library function getaddrinfo(3). If conversion fails, fallback will be returned or VCL failure will happen. The IP address includes a port number that can be found with std.port() that defaults to 80. The default port can be set to a different value with the p argument. It will be overridden if s contains both an IP address and a port number or service name. When s contains both, the syntax is either address:port or address port. If the address is a numerical IPv6 address it must be enclosed between brack- ets, for example [::1] 80 or [::1]:http. The fallback may also contain both an address and a port, but its default port is always 80. If resolve is false, getaddrinfo(3) is called using AI_NUMERICHOST and AI_NUMERICSERV to avoid network lookups depending on the system's getad- drinfo(3) or nsswitch configuration. This makes "numerical" IP strings and services cheaper to convert. Example: if (std.ip(req.http.X-forwarded-for, "0.0.0.0") ~ my_acl) { ... } REAL real([STRING s], [REAL fallback], [INT integer], [BOOL bool], [BYTES bytes], [DURATION duration], [TIME time]) REAL real( [STRING s], [REAL fallback], [INT integer], [BOOL bool], [BYTES bytes], [DURATION duration], [TIME time] ) Returns a REAL from a STRING, BOOL or other quantity. If the conversion of an s argument fails, fallback will be returned if pro- vided, or a VCL failure will be triggered. A bool argument will be returned as 0.0 for false and 1.0 for true. For a bytes argument, the number of bytes will be returned. For a duration argument, the number of seconds will be returned. An integer argument will be returned as a REAL. For a time argument, the number of seconds since the UNIX epoch (1970-01-01 00:00:00 UTC) will be returned. None of these conversions other than s will fail. Only one of the s, integer, bool, bytes, duration or time arguments may be given or a VCL failure will be triggered. Example: if (std.real(req.http.foo, 0.0) > 5.5) { ... } TIME time([STRING s], [TIME fallback], [REAL real], [INT integer]) TIME time([STRING s], [TIME fallback], [REAL real], [INT integer]) Returns a TIME from a STRING, REAL or INT argument. For a STRING s argument, the following formats are supported: "Sun, 06 Nov 1994 08:49:37 GMT" "Sunday, 06-Nov-94 08:49:37 GMT" "Sun Nov 6 08:49:37 1994" "1994-11-06T08:49:37" "784111777.00" "784111777" real and integer arguments are taken as seconds since the epoch. If the conversion of an s argument fails or a negative real or integer ar- gument is given, fallback will be returned if provided, or a VCL failure will be triggered. Examples: if (std.time(resp.http.last-modified, now) < now - 1w) { ... } if (std.time(int=2147483647) < now - 1w) { ... } STRING strftime(TIME time, STRING format) Format the time argument with the format argument using strftime(3) and re- turn the result for the UTC (historically GMT) timezone. The empty string is returned if formatting fails, but may also be returned as a valid result. Example: set req.http.iso = std.strftime(now, "%Y%m%dT%H%M%SZ"); # e.g. 20210521T175241Z LOGGING FUNCTIONS VOID log(STRING s) Logs the string s to the shared memory log, using vsl(7) tag SLT_VCL_Log. Example: std.log("Something fishy is going on with the vhost " + req.http.host); VOID syslog(INT priority, STRING s) Logs the string s to syslog tagged with priority. priority is formed by OR- ing the facility and level values. See your system's syslog.h file for pos- sible values. Notice: Unlike VCL and other functions in the std vmod, this function will not fail VCL processing for workspace overflows: For an out of workspace condition, the std.syslog() function has no effect. Example: std.syslog(9, "Something is wrong"); This will send a message to syslog using LOG_USER | LOG_ALERT. VOID timestamp(STRING s) Introduces a timestamp in the log with the current time, using the string s as the label. This is useful to time the execution of lengthy VCL subrou- tines, and makes the timestamps inserted automatically by Vinyl Cache more accurate. Example: std.timestamp("curl-request"); CONTROL AND INFORMATION FUNCTIONS BOOL syntax(REAL) Returns true if VCL version is at least REAL. STRING getenv(STRING name) Return environment variable name or the empty string. See getenv(3). Example: set req.http.My-Env = std.getenv("MY_ENV"); BOOL cache_req_body(BYTES size) Caches the request body if it is smaller than size. Returns true if the body was cached, false otherwise. Normally the request body can only be sent once. Caching it enables retry- ing backend requests with a request body, as usually the case with POST and PUT. Example: if (std.cache_req_body(1KB)) { ... } Restricted to: vcl_recv. VOID late_100_continue(BOOL late) Controls when vinyld reacts to an Expect: 100-continue client request header. Vinyl Cache always generates a 100 Continue response if requested by the client trough the Expect: 100-continue header when waiting for request body data. But, by default, the 100 Continue response is already generated immediately after vcl_recv returns to reduce latencies under the assumption that the request body will be read eventually. Calling std.late_100_continue(true) in vcl_recv will cause the 100 Continue response to only be sent when needed. This may cause additional latencies for processing request bodies, but is the correct behavior by strict inter- pretation of RFC7231. This function has no effect outside vcl_recv and after calling std.cache_req_body() or any other function consuming the request body. Example: vcl_recv { std.late_100_continue(true); if (req.method == "POST") { std.late_100_continue(false); return (pass); } ... } Restricted to: vcl_recv. VOID set_ip_tos(INT tos) Sets the Differentiated Services Codepoint (DSCP) / IPv4 Type of Service (TOS) / IPv6 Traffic Class (TCLASS) byte for the current session to tos. Silently ignored if the listen address is a Unix domain socket. Please note that setting the traffic class affects all requests on the same http1.1 / http2 TCP connection and, in particular, is not removed at the end of the request. Example: if (req.url ~ "^/slow/") { std.set_ip_tos(0); } Restricted to: client. VOID rollback(HTTP h) Restores the h HTTP headers to their original state. Example: std.rollback(bereq); Restricted to: backend, vcl_recv, vcl_pass, vcl_hash, vcl_purge, vcl_miss, vcl_hit, vcl_deliver, vcl_synth. BOOL ban(STRING) Invalidates all objects in cache that match the given expression with the ban mechanism. Returns true if the ban succeeded and false otherwise. Error details are available via std.ban_error(). The format of STRING is: <field> <operator> <arg> [&& <field> <oper> <arg> ...] * <field>: * string fields: * req.url: The request url * req.http.*: Any request header * obj.status: The cache object status * obj.http.*: Any cache object header obj.status is treated as a string despite the fact that it is actually an integer. * duration fields: * obj.ttl: Remaining ttl * obj.age: Object age * obj.grace: The grace time of the object * obj.keep: The keep time of the object * obj.last_hit: Time since the last hit obj.ttl, obj.age and obj.last_hit are relative to the submission time of the ban, such that they represent a fixed point in time despite be- ing specified as a duration. * <operator>: * for all fields: * ==: <field> and <arg> are equal * !=: <field> and <arg> are unequal strings are compared case-sensitively * for string fields: * ~: <field> matches the regular expression <arg> * !~:<field> does not match the regular expression <arg> * for duration fields: * >: <field> is greater than <arg> * >=: <field> is greater than or equal to <arg> * <: <field> is less than <arg> * <=: <field> is less than or equal to <arg> * <arg>: * for string fields: Either a literal string or a regular expression. Note that <arg> does not use any of the string delimiters like " or {"..."} or """...""" used elsewhere in Vinyl Cache. To match against strings containing whitespace, regular expressions containing \s can be used. * for duration fields: A VCL duration like 10s, 5m or 1h, see vcl(7)_durations Expressions can be chained using the and operator &&. For or semantics, use several bans. The unset <field> is not equal to any string, such that, for a non-existing header, the operators == and ~ always evaluate as false, while the opera- tors != and !~ always evaluate as true, respectively, for any value of <arg>. obj.last_hit can be used almost as a "last accessed" time, so, for example, ban obj.last_hit > 1d removes all objects which were last accessed more than one day ago. Also, it can be used to remove objects stuck at request bans by issuing ban obj.last_hit > X, with X being slightly less than the time since the request ban. obj.last_hit is based on an internal last LRU time, which might not be im- plemented by all storage engines. Where it is not available, ban expres- sions using obj.last_hit evaluate to false, which means that the respective ban behaves as if it was not present. Where implemented, the last LRU time might only get updated by the lru_interval parameter, which therefore is the maximum precision of obj.last_hit bans. STRING ban_error() Returns a textual error description of the last std.ban() call from the same task or the empty string if there either was no error or no std.ban() call. TIME now() Returns the current time. In contrast to the now built-in variable, every call returns a new value. DURATION timed_call(SUB) Call the given SUB and return a high precision measurement of the execution time. VOID rfc_ttl() Restricted to: vcl_backend_response, vcl_backend_refresh. Re-calculate the object timers (beresp.ttl, beresp.grace and beresp.keep) based on the current state of beresp as if it had been processed by core code before vcl_backend_response was called. This does not change beresp.uncacheable. This is useful to get the default ttl calculations after modifications of relevant properties like beresp.status, beresp.http.Date, beresp.http.Age or beresp.http.Cache-Control. SEE ALSO * vinyld(1) * vsl(7) * fnmatch(3) * strftime(3) COPYRIGHT Copyright (c) 2010-2017 Varnish Software AS All rights reserved. Author: Poul-Henning Kamp <phk@FreeBSD.org> SPDX-License-Identifier: BSD-2-Clause Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. VMOD_STD(3)
NAME | SYNOPSIS | DESCRIPTION | NUMERIC FUNCTIONS | STRING FUNCTIONS | FILE(SYSTEM) FUNCTIONS | TYPE INSPECTION FUNCTIONS | TYPE CONVERSION FUNCTIONS | LOGGING FUNCTIONS | CONTROL AND INFORMATION FUNCTIONS | SEE ALSO | COPYRIGHT
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=vmod_std&sektion=3&manpath=FreeBSD+Ports+15.1.quarterly>