Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
NEWSBATCH(8)		    System Manager's Manual		  NEWSBATCH(8)

       sendbatches, batchsplit - news batching to other	sites
       batcher,	batchih, batchsm, batchra - news-batch preparation
       compcun - news-batch compression
       c7encode, bencode - compressed-news-batch encoding
       viauux -	news-batch transmission	via uucp
       viamail - news-batch transmission via mail
       viainews, viarsh	- news-batch transmission by misc. means

       /usr/local/libexec/cnews/batch/sendbatches  [  -p ] [ -c	class ]	[ site
       ... ]

       .../batchsplit batchsize	maxsize	count
       .../batcher [ listfile ]
       .../batchih [ listfile ]
       .../batchsm [ listfile ]
       .../batchra [ listfile ]
       .../compcun [ compressoptions ]
       .../bencode [ file ]
       .../viauux [ -ggrade ] [	-z ] [ -n ] [ -d decompressor ]	[ -c ] [  site
       .../viamail [ -e	] [ 0-@	] [ site ]
       .../viarsh [ site ]

       Sendbatches  administers	 batched  transmission of news to other	sites.
       It should be run	periodically, as the owner of the  news	 database,  by
       cron(8)	or similar means.  It prepares and sends batches of news, sub-
       ject to restrictions on available space and length of outbound queues.

       Normally, sendbatches does locking to ensure that only one  sendbatches
       is running at a time.  The -p option suppresses the locking and permits
       parallel	sendbatches runs, although lower-level locking is done to  en-
       sure that only one is trying to prepare batches for any particular site
       at a given time.	 Parallel sendbatches runs impose  very	 heavy	system
       loads  but may be useful	to systems with	extensive hardware parallelism
       and many	outbound news feeds.

       Each site that can have batches sent to it needs	a site directory under
       /var/news/out.going.   If  sendbatches  is  invoked with	specific sites
       given, it considers batching for	those sites, only, in that order.   By
       default,	sendbatches consults the batchparms file (see below) to	deter-
       mine what to do:	If there is a /default/	 entry	in  batchparms,	 send-
       batches	will  consider batching	for all	sites that have	directories in
       /var/news/out.going, in oldest-first order by modification time of  the
       directory.   If	there  is  no  /default/  entry, sendbatches considers
       batching	for those sites	named in batchparms, in	the order named.

       To use the batcher, names of files to be	sent to	a specific site	should
       be  appended to a togo file in its site directory.  The batcher expects
       the lines in togo to have two fields, a filename	(as a  full  pathname,
       or relative to /var/news) of an article and its size in bytes.  A miss-
       ing size	field is arbitrarily assumed to	be a default average.

       Sendbatches uses	a number of auxiliary programs to do  the  real	 work.
       The  search  path it uses to find them includes,	in order, the site di-
       rectory for the	site  in  question,  /var/lib/news/bin,	 and  /usr/lo-
       cal/libexec/cnews/batch.	  This	permits	per-site and per-news-database
       overrides of the	default	behaviors.   Sendbatches  provides  all	 these
       programs	 with  environment  variables NEWSSITE,	containing the name of
       the site	that batches are being prepared	for, and NEWSSITEDIR, contain-
       ing  the	 full pathname of the site directory, in case these are	useful
       in customization.

       The names of most of the	auxiliary programs, and	some other parameters,
       are  taken  from	the file /var/lib/news/batchparms, an ASCII text file.
       Empty lines and lines starting with `#' are ignored.  Other lines spec-
       ify  the	 behavior  for	sites, one line	per site.  A site line is four
       fields, separated by white space.

       The first field is the site name.  A line whose site name is  /default/
       specifies  what parameters should be used for sites not explicitly men-
       tioned.	(The presence or absence of such a line	 also  influences  the
       behavior	of sendbatches when invoked without arguments; see above.)

       The  second  field is the class of the site.  If	sendbatches is invoked
       with the	-c option, it attempts batching	only for sites of  the	speci-
       fied  class.   A	 class	is a single letter, by convention `u' for UUCP
       feeds and `n' for NNTP feeds; user-defined classes should be  uppercase

       If  the	character  `!' appears anywhere	in the class field of a	batch-
       parms line, that	disables the line.  Any	site whose batching would have
       been controlled by that line will have no batching attempted for	it un-
       der any circumstances.  This provides a way to turn a site  off	tempo-

       The  third field	is the size of batches to be prepared (before compres-
       sion), in bytes.	 It may	optionally be two sizes	separated by  `-',  in
       which case the first is the nominal size	and the	second is the absolute
       maximum.	 If only one size is given, that is the	nominal	size, and  the
       absolute	maximum	is three times that.

       The  fourth  field is the maximum length	of the output queue for	trans-
       mission to that site.  If it is `-', no queue-length limiting is	done.

       The fifth field,	which may contain white	space,	is  the	 command  line
       (normally  a pipeline of	three-four programs, possibly with options) to
       be used to build, compress, and transmit	batches	to that	site.  It  re-
       ceives  the  contents  of  the togo file	on standard input.  It may not
       contain any single quotes (').

       For each	site being considered for batches,  sendbatches	 first	deter-
       mines  whether  there are in fact any articles to be batched.  Assuming
       there are, sendbatches then finds the batchparms	line for that site and
       (if  queue-length limiting is done for that site) invokes queuelen (see
       newsaux(8CN)) to	find out the size of the outbound queue	for the	 site.
       Sendbatches limits the number of	batches	prepared to the	minimum	of the
       limits implied by queue lengths and available space.

       Sendbatches uses	batchsplit as necessary	to slice  chunks  out  of  the
       togo  file, each	chunk containing the togo lines	for a batch limited to
       the nominal size.  Exception:  a	single article bigger than the nominal
       size  will  still  go out as one	batch, provided	it does	not exceed the
       absolute-maximum	size.

       Each chunk is then processed through the	command	 line,	which  usually
       consists	 of  a batch preparer (typically batcher), which assembles the
       articles	into a batch, a	batch compressor  (typically  compcun),	 which
       performs	 compression, possibly a batch encoder,	which encodes the com-
       pressed batch against the vagaries of ill-designed  transmission	 chan-
       nels, and a batch transmitter (typically	viauux), which sends the batch
       on its way (e.g.	enqueues it  for  transmission).   All	are  run  with
       /var/news  as  the  current directory, so non-absolute pathnames	in the
       chunk are valid filenames; NOTE that this represents a change from ear-
       lier releases, in which only the	preparer ran there.

       If  there  is  anything left in the file	togo.leftover after a chunk is
       processed, sendbatches assumes this is the portion of the  chunk	 which
       could not be processed at this time.  If/when this happens, sendbatches
       replaces	the chunk file with the	contents of togo.leftover and attempts
       no further batching to that site	on this	run.

       Batch preparers in the standard distribution are:

	      batcher  Normal batching.

	      batchih  Ihave-sending part of uucp ihave/sendme (not to be con-
		       fused with NNTP).

	      batchsm  Sendme-sending part of ihave/sendme.

	      batchra  Requested-article-sending part of ihave/sendme.

       Batchih,	batchsm, and batchra have to map from the phony	``site	name''
       given  in their batchparms line to the name of the site they should ac-
       tually send to; they do this by stripping off the last `.'  and	every-
       thing  that follows (usually `.ihave' or	`.sendme' respectively,	but on
       machines	which limit the	size of	filenames these	may have to be	short-

       Caution:	 batchih  and batchsm do their work by constructing an article
       and feeding it to inews,	which means that the batch size	must be	within
       what inews will accept, if it imposes limits (see inews(1CN)).

       Batch compressors can include ordinary programs like compress and gzip,
       but one special compressor is provided by the distribution:

	      compcun  Compression with	compress plus the silly	B-news-compat-
		       ible `#!	cunbatch' header.  Options, if any, are	passed
		       to compress.

       Batch encoders supplied with the	distribution are:

	      bencode  Encodes 8-bit data  using  only	the  ASCII  characters
		       ``A'' - ``Z'', ``a'' - ``z'', ``0'' - ``9'', ``+'', and
		       ``-''.  The ASCII characters blank, newline, and	 ``/''
		       also  appear  in	the encoded file, but do not represent
		       encoded bits.  The encoded data is  terminated  with  a
		       byte  count  and	 cyclic	redundancy check for detecting
		       corruption.  This ought to suffice to get data  through
		       almost any network.

	      c7encode Encodes	8-bit  data  into  a  7-bit form optimized for
		       transmission by uucp `f'	 protocol.   The  encoding  is
		       complex and bizarre.  Obsolete; use bencode instead.

	      nencode  Feebly  attempts	to protect text	(not compressed!) data
		       by prepending a blank line to the input and an  `N'  to
		       each  line.   Obsolete and not recommended; use bencode

       Most transmitters take an optional site argument, using	the  value  of
       the  NEWSSITE  variable if no argument is supplied.  Batch transmitters
       in the standard distribution are:

	      viauux	Normal transmission via	UUCP.  -z or -n	may be used to
			feed the corresponding option to uux.  A -g option may
			be used	to feed	the grade to uux (the default is grade
			`d').  A -d option may be used to specify a decompres-
			sor program (e.g., gunzip) to the site's rnews;	beware
			that  old  rnewses  may	not recognize this.  -c	may be
			used to	request	execution of the site's	cunbatch  pro-
			gram rather than rnews.

	      viamail	Mail  the batch	to `site!rnews'.  -e specifies mailing
			to `site!enews'	instead.  -@ specifies use of  @  syn-
			tax, e.g. `rnews@site',	instead	of ! syntax.

	      viainews	Feed  the  batch back to inews (arguments are ignored)
			(normally useful only for ihave/sendme).

	      viarsh	Use rsh	to run rnews on	the site via Ethernet,	Inter-
			net,  etc.  (the directory containing rnews must be in
			the default PATH on site).

       Sendbatches  logs  some	information  about  sites  with	 backlogs   in
       /var/lib/news/batchlog  (see also newsdaily(8CN)).  This	is intended to
       help detection and diagnosis of flow problems.

       /var/lib/news/LOCKbatch	   lock	for sendbatches
       /var/lib/news/LOCKexplode   overall batch-file lock (used by batchsplit)
       /var/lib/news/L.*	   lock	temporaries
       /var/news/out.going/*	   batch directories
       /var/news/out.going/*/LOCK  per-site locks
       /var/news/out.going/*/L.*   per-site lock temporaries
       /var/lib/news/batchparms	   parameter file

       also see	text

       inews(1CN), compress(1),	uux(1),	relaynews(8CN),	rnews(8CN)

       Complaints, if any, from	sendbatches and	its auxiliaries	are mailed via

       Written at University of	Toronto	as part	of the C News project.	A num-
       ber of the ideas	came from Chris	Lewis, who  was	 then  with  Motorola.
       Bencode	written	 at  University	 of  Waterloo  by  Reg Quinton and Ken

       Sendbatches estimates available space without considering  the  effects
       of compression; this is usually too conservative.

       Viarsh does not incorporate a spooling subsystem, so a slow site	stalls
       the entire batching system and a	non-responding site loses news.	 It is
       not  recommended	for bulk transmission or where high reliability	is es-

       There ought to be an encoder using MIME's base64	encoding and  suitable
       descriptive headers.

       The  logging  is	overly simplistic and doesn't work well	when selective
       batching	(controlled by site arguments, classes,	or disabling) is being

				 10 March 1995			  NEWSBATCH(8)


Want to link to this manual page? Use this URL:

home | help