Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
VMOD_BLOB(3)		   Library Functions Manual		  VMOD_BLOB(3)

NAME
       vmod_blob - Utilities for the VCL blob type, encoding and decoding

SYNOPSIS
	  import blob [as name]	[from "path"]

	  BLOB decode(ENUM decoding, INT length, STRING	encoded)

	  STRING encode(ENUM encoding, ENUM case, BLOB blob)

	  STRING transcode(ENUM	decoding, ENUM encoding, ENUM case, INT	length,	STRING encoded)

	  BOOL same(BLOB, BLOB)

	  BOOL equal(BLOB, BLOB)

	  INT length(BLOB)

	  BLOB sub(BLOB, BYTES length, BYTES offset)

	  new xblob = blob.blob(ENUM decoding, STRING encoded)

	      BLOB xblob.get()

	      STRING xblob.encode(ENUM encoding, ENUM case)

DESCRIPTION
       This  VMOD  provides  utility  functions	and an object for the VCL data
       type BLOB, which	may contain arbitrary data of any length.

       Examples:

	  sub vcl_init {
	      #	Create blob objects from encodings such	as base64 or hex.
	      new myblob   = blob.blob(BASE64, "Zm9vYmFy");
	      new yourblob = blob.blob(encoded="666F6F", decoding=HEX);
	  }

	  sub vcl_deliver {
	      #	The .get() method retrieves the	BLOB from an object.
	      set resp.http.MyBlob-As-Hex
		  = blob.encode(blob=myblob.get(), encoding=HEX);

	      #	The .encode() method efficiently retrieves an encoding.
	      set resp.http.YourBlob-As-Base64 = yourblob.encode(BASE64);

	      #	decode() and encode() functions	convert	blobs to text and
	      #	vice versa at runtime.
	      set resp.http.Base64-Encoded
		  = blob.encode(BASE64,
				blob=blob.decode(HEX,
						 encoded=req.http.Hex-Encoded));
	  }

	  sub vcl_recv {
	      #	transcode() converts from one encoding to another.
	      #	case=UPPER specifies upper-case	hex digits A-F.
	      set req.http.Hex-Encoded
		  = blob.transcode(decoding=BASE64, encoding=HEX,
				   case=UPPER, encoded="YmF6");

	      #	transcode() from URL to	IDENTITY effects a URL decode.
	      set req.url = blob.transcode(encoded=req.url, decoding=URL);

	      #	transcode() from IDENTITY to URL effects a URL encode.
	      set req.http.url_urlcoded
		  = blob.transcode(encoded=req.url, encoding=URL);
	  }

   ENCODING SCHEMES
       Binary-to-text encoding schemes are specified by	ENUMs  in  the	VMOD's
       constructor,  methods and functions. Decodings convert a	(possibly con-
       catenated) string into a	blob, while encodings convert a	 blob  into  a
       string.

       ENUM values for an encoding scheme can be one of:

        IDENTITY

        BASE64

        BASE64URL

        BASE64URLNOPAD

        BASE64CF

        HEX

        URL

       Empty  strings  are  decoded into a "null blob" (of length 0), and con-
       versely a null blob is encoded as the empty string.

       For encodings with HEX or URL, you may also specify a  case  ENUM  with
       one  of	the  values  LOWER,  UPPER or DEFAULT to produce a string with
       lower- or uppercase hex digits (in [a-f]	or [A-F]). The	default	 value
       for case	is DEFAULT, which for HEX and URL means	the same as LOWER.

       The  case  ENUM is not relevant for decodings; HEX or URL strings to be
       decoded as BLOBs	may have hex digits in either case, or in mixed	case.

       The case	ENUM MUST be set to DEFAULT for	the other  encodings  (BASE64*
       and IDENTITY).  You cannot, for example,	produce	an uppercase string by
       using  the  IDENTITY  scheme  with  case=UPPER. To change the case of a
       string,	use  the  std.toupper()	 or   std.tolower()   functions	  from
       vmod_std(3).

   IDENTITY
       The  simplest encoding converts between the BLOB	and STRING data	types,
       leaving the contents byte-identical.

       Note that a BLOB	may contain a null byte	at  any	 position  before  its
       end; if such a BLOB is decoded with IDENTITY, the resulting STRING will
       have  a	null byte at that position. Since VCL strings, like C strings,
       are represented with a terminating null byte, the string	will be	 trun-
       cated, appearing	to contain less	data than the original blob. For exam-
       ple:

	  # Decode from	the hex	encoding for "foo\0bar".
	  # The	header will be seen as "foo".
	  set resp.http.Trunced-Foo1
	      =	blob.encode(IDENTITY, blob=blob.decode(HEX,
						       encoded="666f6f00626172"));

       IDENTITY	is the default encoding	and decoding. So the above can also be
       written as:

	  # Decode from	the hex	encoding for "foo\0bar".
	  # The	header will be seen as "foo".
	  set resp.http.Trunced-Foo2
	    = blob.encode(blob=blob.decode(HEX,	encoded="666f6f00626172"));

       The case	ENUM MUST be set to DEFAULT for	IDENTITY encodings.

   BASE64*
       The  base64  encoding schemes use 4 characters to encode	3 bytes. There
       are no newlines or maximal line lengths -- whitespace is	not permitted.

       The BASE64 encoding uses	the alphanumeric characters, + and /; and  en-
       coded  strings  are padded with the = character so that their length is
       always a	multiple of four.

       The BASE64URL encoding also uses	the alphanumeric characters, but - and
       _ instead of + and /, so	that an	encoded	string can be used safely in a
       URL. This scheme	also uses the padding character	=.

       The BASE64URLNOPAD encoding uses	the same  alphabet  as	BASE6URL,  but
       leaves out the padding. Thus the	length of an encoding with this	scheme
       is not necessarily a multiple of	four.

       The  BASE64CF` is similar to ``BASE64URL, with the following changes to
       BASE64: + replaced with -, / replaced with ~ and	_ as the padding char-
       acter. It is used by a certain CDN provider who also inspired the name.

       The case	ENUM MUST be set to DEFAULT for	all of the BASE64* encodings.

   HEX
       The HEX encoding	scheme converts	hex strings into blobs and vice	versa.
       For encodings, you may use the case ENUM	to specify upper- or lowercase
       hex digits A through f  (default	 DEFAULT,  which  means	 the  same  as
       LOWER).	A prefix such as 0x is not used	for an encoding	and is illegal
       for a decoding.

       If  a  hex  string to be	decoded	has an odd number of digits, it	is de-
       coded as	if a 0 is prepended to it; that	is, the	first digit is	inter-
       preted  as representing the least significant nibble of the first byte.
       For example:

	  # The	concatenated string is "abcdef0", and is decoded as "0abcdef0".
	  set resp.http.First =	"abc";
	  set resp.http.Second = "def0";
	  set resp.http.Hex-Decoded
	      =	blob.encode(HEX, blob=blob.decode(HEX,
				 encoded=resp.http.First + resp.http.Second));

   URL
       The URL decoding	replaces any %<2-hex-digits> substrings	with  the  bi-
       nary value of the hexadecimal number after the %	sign.

       The URL encoding	implements "percent encoding" as per RFC3986. The case
       ENUM  determines	the case of the	hex digits, but	does not affect	alpha-
       betic characters	that are not percent-encoded.

   BLOB	decode(ENUM decoding, INT length, STRING encoded)
	  BLOB decode(
	     ENUM {IDENTITY, BASE64, BASE64URL,	BASE64URLNOPAD,	BASE64CF, HEX, URL} decoding=IDENTITY,
	     INT length=0,
	     STRING encoded
	  )

       Returns the BLOB	derived	from  the  string  encoded  according  to  the
       scheme specified	by decoding.

       If  length  > 0,	only decode the	first length characters	of the encoded
       string. If length <= 0 or greater than the length of the	 string,  then
       decode the entire string. The default value of length is	0.

       decoding	defaults to IDENTITY.

       Example:

	  blob.decode(BASE64, encoded="Zm9vYmFyYmF6");

	  # same with named parameters
	  blob.decode(encoded="Zm9vYmFyYmF6", decoding=BASE64);

	  # convert string to blob
	  blob.decode(encoded="foo");

   STRING encode(ENUM encoding,	ENUM case, BLOB	blob)
	  STRING encode(
	     ENUM {IDENTITY, BASE64, BASE64URL,	BASE64URLNOPAD,	BASE64CF, HEX, URL} encoding=IDENTITY,
	     ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
	     BLOB blob
	  )

       Returns a string	representation of the BLOB blob	as specified by	encod-
       ing.  case determines the case of hex digits for	the HEX	and URL	encod-
       ings, and is ignored for	the other encodings.

       encoding	defaults to IDENTITY, and case defaults	to  DEFAULT.   DEFAULT
       is  interpreted	as LOWER for the HEX and URL encodings,	and is the re-
       quired value for	the other encodings.

       Example:

	  set resp.http.encode1
	      =	blob.encode(HEX,
			    blob=blob.decode(BASE64, encoded="Zm9vYmFyYmF6"));

	  # same with named parameters
	  set resp.http.encode2
	      =	blob.encode(blob=blob.decode(encoded="Zm9vYmFyYmF6",
						     decoding=BASE64),
				encoding=HEX);

	  # convert blob to string
	  set resp.http.encode3
	      =	blob.encode(blob=blob.decode(encoded="foo"));

   STRING transcode(ENUM decoding,  ENUM  encoding,  ENUM  case,  INT  length,
       STRING encoded)
	  STRING transcode(
	     ENUM {IDENTITY, BASE64, BASE64URL,	BASE64URLNOPAD,	BASE64CF, HEX, URL} decoding=IDENTITY,
	     ENUM {IDENTITY, BASE64, BASE64URL,	BASE64URLNOPAD,	BASE64CF, HEX, URL} encoding=IDENTITY,
	     ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT,
	     INT length=0,
	     STRING encoded
	  )

       Translates  from	 one encoding to another, by first decoding the	string
       encoded according to the	scheme decoding, and then returning the	encod-
       ing of the resulting blob according to the scheme encoding. case	deter-
       mines the case of hex digits for	the HEX	and URL	encodings, and is  ig-
       nored for other encodings.

       As  with	 blob.decode():	 If  length  > 0, only decode the first	length
       characters of the encoded string, otherwise decode the  entire  string.
       The default value of length is 0.

       decoding	 and  encoding	default	 to IDENTITY, and case defaults	to DE-
       FAULT. DEFAULT is interpreted as	LOWER for the HEX and  URL  encodings,
       and is the required value for the other encodings.

       Example:

	  set resp.http.Hex2Base64-1
	       = blob.transcode(HEX, BASE64, encoded="666f6f");

	   # same with named parameters
	   set resp.http.Hex2Base64-2
	      =	blob.transcode(encoded="666f6f",
				    encoding=BASE64, decoding=HEX);

	   # URL decode	-- recall that IDENTITY	is the default encoding.
	   set resp.http.urldecoded
	      =	blob.transcode(encoded="foo%20bar", decoding=URL);

	   # URL encode
	   set resp.http.urlencoded
	       = blob.transcode(encoded="foo bar", encoding=URL);

   BOOL	same(BLOB, BLOB)
       Returns true if and only	if the two BLOB	arguments are the same object,
       i.e. they specify exactly the same region of memory, or both are	empty.

       If the BLOBs are	both empty (length is 0	and/or the internal pointer is
       NULL), then blob.same() returns true. If	any non-empty BLOB is compared
       to an empty BLOB, then blob.same() returns false.

   BOOL	equal(BLOB, BLOB)
       Returns	true if	and only if the	two BLOB arguments have	equal contents
       (possibly in different memory regions).

       As with blob.same(): If the BLOBs are both empty, then blob.equal() re-
       turns true. If any non-empty BLOB is compared to	an  empty  BLOB,  then
       blob.equal() returns false.

   INT length(BLOB)
       Returns the length of the BLOB.

   BLOB	sub(BLOB, BYTES	length,	BYTES offset=0)
       Returns a new BLOB formed from length bytes of the BLOB argument	start-
       ing  at	offset	bytes from the start of	its memory region. The default
       value of	offset is 0B.

       blob.sub() fails	and returns NULL if the	BLOB argument is empty,	or  if
       offset +	length requires	more bytes than	are available in the BLOB.

   new xblob = blob.blob(ENUM decoding,	STRING encoded)
	  new xblob = blob.blob(
	     ENUM {IDENTITY, BASE64, BASE64URL,	BASE64URLNOPAD,	BASE64CF, HEX, URL} decoding=IDENTITY,
	     STRING encoded
	  )

       Creates	an  object  that contains the BLOB derived from	the string en-
       coded according to the scheme decoding.

       Example:

	  new theblob1 = blob.blob(BASE64, encoded="YmxvYg==");

	  # same with named arguments
	  new theblob2 = blob.blob(encoded="YmxvYg==", decoding=BASE64);

	  # string as a	blob
	  new stringblob = blob.blob(encoded="bazz");

   BLOB	xblob.get()
       Returns the BLOB	created	by the constructor.

       Example:

	  set resp.http.The-Blob1 =
	      blob.encode(blob=theblob1.get());

	  set resp.http.The-Blob2 =
	      blob.encode(blob=theblob2.get());

	  set resp.http.The-Stringblob =
	      blob.encode(blob=stringblob.get());

   STRING xblob.encode(ENUM encoding, ENUM case)
	  STRING xblob.encode(
		ENUM {IDENTITY,	BASE64,	BASE64URL, BASE64URLNOPAD, BASE64CF, HEX, URL} encoding=IDENTITY,
		ENUM {LOWER, UPPER, DEFAULT} case=DEFAULT
	  )

       Returns an encoding of BLOB created by the  constructor,	 according  to
       the scheme encoding. case determines the	case of	hex digits for the HEX
       and URL encodings, and MUST be set to DEFAULT for the other encodings.

       Example:

	  # blob as text
	  set resp.http.The-Blob = theblob1.encode();

	  # blob as base64
	  set resp.http.The-Blob-b64 = theblob1.encode(BASE64);

       For  any	 blob.blob()  object,  encoding	 and  case,  encodings via the
       xblob.encode() method and the blob.encode() function are	equal:

	  # Always true:
	  blob.encode(ENC, CASE, blob.get()) ==	blob.encode(ENC, CASE)

       But the xblob.encode() object method is more efficient -- the  encoding
       is  computed  once and cached (with allocation in heap memory), and the
       cached encoding is retrieved on every  subsequent  call.	 The  blob.en-
       code()  function	 computes the encoding on every	call, allocating space
       for the string in Varnish workspaces.

       So if the data in a BLOB	are fixed at VCL initialization	time, so  that
       its  encodings  will  always  be	 the  same,  it	 is better to create a
       blob.blob() object. The VMOD's functions	should be used for  data  that
       are not known until runtime.

ERRORS
       The encoders, decoders and blob.sub() may fail if there is insufficient
       space  to  create the new blob or string. Decoders may also fail	if the
       encoded string is an illegal format for the decoding  scheme.  Encoders
       will  fail  for	the  IDENTITY and BASE64* encoding schemes if the case
       ENUM is not set to DEFAULT.

       If any of the VMOD's methods, functions or constructor fail,  then  VCL
       failure	is invoked, just as if return(fail) had	been called in the VCL
       source. This means that:

        If the	blob.blob() object constructor fails, or  if  any  methods  or
	 functions  fail  during vcl_init{}, then the VCL program will fail to
	 load, and the VCC compiler will emit an error message.

        If a method or	function fails in any  other  VCL  subroutine  besides
	 vcl_synth{},  then  control  is directed to vcl_synth{}. The response
	 status	is set to 503 with the reason string "VCL failed", and an  er-
	 ror message will be written to	the vsl(7) using the tag VCL_Error.

        If  the  failure  occurs  during  vcl_synth{},	 then  vcl_synth{}  is
	 aborted. The response line "503 VCL  failed"  is  returned,  and  the
	 VCL_Error message is written to the log.

LIMITATIONS
       The  VMOD  allocates  memory in various ways for	new blobs and strings.
       The blob.blob() object and its methods allocate memory from  the	 heap,
       and hence they are only limited by available virtual memory.

       The  blob.encode(),  blob.decode() and blob.transcode() functions allo-
       cate Varnish workspace, as does blob.sub() for the newly	created	 BLOB.
       If these	functions are failing, as indicated by "out of space" messages
       in  the Varnish log (with the VCL_Error tag), then you will need	to in-
       crease the varnishd parameters workspace_client and/or  workspace_back-
       end.

       The  blob.transcode()  function also allocates space on the stack for a
       temporary BLOB. If this function	causes stack overflow, you may need to
       increase	the varnishd parameter thread_pool_stack.

SEE ALSO
        varnishd(1)

        vcl(7)

        vsl(7)

        vmod_std(3)

COPYRIGHT
	  This document	is licensed under the same conditions as Varnish itself.
	  See LICENSE for details.

	  SPDX-License-Identifier: BSD-2-Clause

	  Authors: Nils	Goroll <nils.goroll@uplex.de>
		   Geoffrey Simmons <geoffrey.simmons@uplex.de>

								  VMOD_BLOB(3)

Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=vmod_blob&sektion=3&manpath=FreeBSD+Ports+14.3.quarterly>

home | help