Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
NEWSFEEDS(5)		  InterNetNews Documentation		  NEWSFEEDS(5)

NAME
       newsfeeds - Determine where Usenet articles are sent

IN A NUTSHELL
       The newsfeeds file configures what innd does with incoming accepted
       articles.

       A minimalist newsfeeds file which parameters an outgoing	NNTP feed to
       "news.server.com", using	path identity "server" (as it appears in Path
       header fields), of all the articles you receive is:

	   # Mandatory line, with no exclusion patterns	defined.
	   ME:::

	   # A real-time feed through innfeed of all your articles.
	   news.server.com/server\
	       :*\
	       :Tm:innfeed!

	   # The channel feed entry for	innfeed.
	   # Change "/usr/lib/news/bin"	to the directory path
	   # of	the innfeed program.
	   innfeed!:!*:Tc,Wnm*:/usr/lib/news/bin/innfeed

       In order	to find	an external feed, you can ask in the
       news.admin.peering newsgroup.  Several news administrators will
       certainly respond and gracefully	provide	you with a news	feed.

       Other entries may be added in newsfeeds depending on your needs.	 See
       the examples below and also in the archive(8), controlchan(8),
       news2mail(8), ninpaths(8) and perl-nocem(8) man pages for the most
       widely used.

       If you exclude local hierarchies	or newsgroups from the feed, make sure
       to add "!control,!control.*" so that cancel articles or any kind	of
       control articles	related	to excluded hierarchies	or newsgroups are not
       propagated to the remote	peer.  Its newsfeeds entry then	looks like:

	   news.server.com/server\
	       :*,!local.*,@*bina*,!control,!control.*\
	       :Tm:innfeed!

       After any changes, run "inncheck" to perform basic syntax checks, and
       reload this configuration file with the following command:

	   ctlinnd reload newsfeeds 'new peer'

DESCRIPTION
       The file	pathetc/newsfeeds specifies how	incoming articles should be
       distributed to other programs and files on the server.  It is parsed by
       the InterNetNews	server innd(8) when it starts up, or when directed to
       by ctlinnd(8).  innd doesn't send articles to remote sites itself, so
       newsfeeds doesn't directly determine which remote news servers articles
       are sent	to.  Instead, it specifies what	batch files should be created
       or which	programs should	be run (and what information should be sent to
       them), and then this information	is used	by programs like innxmit(8)
       and innfeed(8) to feed articles to remote sites.

       Used in conjunction with	delayer(1) in the configuration	of the related
       funnel feed, innfeed can	even send a delayed feed to remote peers or to
       another local instance of innd, notably to permit the rejection of spam
       before it actually arrives.

       The newsfeeds file isn't	used solely to set up feeding accepted
       articles	to remote sites	but also to pass them (or bits of information
       about them) to any local	programs or files that want that data.	For
       example,	controlchan(8),	a daemon that processes	incoming control
       messages, runs out of newsfeeds,	as could a news	to mail	gateway.

       The file	is interpreted as a set	of lines, parsed according to the
       following rules:	if a line ends with a backslash, the backslash,	the
       newline,	and any	whitespace at the start	of the next line is deleted.
       This is repeated	until the entire "logical" line	is collected.  If the
       logical line is blank or	starts with a number sign ("#"), it is
       ignored.

       All other lines are interpreted as feed entries.	 An entry should
       consist of four colon-separated fields; two of the fields may have
       optional	sub-fields, marked off by a slash.  Fields or sub-fields that
       take multiple parameters	should be separated by a comma.	 Extra
       whitespace can cause problems and should	be avoided.  Except for	the
       site names, case	is significant.	 The format of an entry	is:

	   sitename[/exclude,exclude,...]\
	       :pattern,pattern...[/distribution,distribution...]\
	       :flag,flag...\
	       :parameter

       Each field is described below.

       The sitename is the name	of the site to which a news article can	be
       sent.  It is used for writing log entries and for determining if	an
       article should be forwarded to a	site.  (A "site" is the	generic	term
       for some	destination of newsfeed	data; it often corresponds to a	remote
       news peer, but doesn't have to.	For example, a local archiving program
       run from	newsfeeds is also a "site".)  If sitename already appears in
       the article's Path header field body, then the article will not be sent
       to the site.  The name is usually whatever the remote site uses to
       identify	itself in the Path header field	body, but can be almost	any
       word.

       Be careful, though, to avoid having the sitename	accidentally match a
       path identity in	the Path header	field unintentionally.	For this
       reason, special local entries (such as archivers	or gateways) should
       probably	end with an exclamation	point to make sure that	they do	not
       have the	same name as any real site.  For example, "gateway" is an
       obvious name for	the local entry	that forwards articles out to a
       mailing list.  If a site	with the name "gateway"	posts an article, when
       the local site receives the article it will see the name	in the Path
       header field and	not send the article to	its own	"gateway" entry.
       Since "gateway!"	can't appear as	an individual path identity in the
       Path header field (as "!" is the	delimiter), that would be a better
       thing to	use for	sitename.

       (Another	way to avoid this problem is with the "Ap" flag; see the
       description below.)

       If an entry has an exclusion sub-field, the article will	not be sent to
       that site if any	of exclude appear in the Path header field body.
       (It's sometimes convenient to have the sitename be an abbreviated form
       of the name of the remote site, since all the sitenames to which	an
       article is sent are written to the log and using	shorter	sitenames can
       therefore improve performance for large servers.	 In this case, the
       path identities of those	sites should be	given as exclude entries and
       the "Ap"	flag used so that the abbreviated sitename doesn't
       accidentally match some other path identities in	Path header fields.)

       The same	sitename can be	used more than once and	the appropriate	action
       will be taken for each entry that should	receive	the article, but this
       is recommended only for program feeds to	avoid confusion.  Case is not
       significant in site names.

       The comma-separated pattern specifies which groups to send to the site;
       it is interpreted to build a "subscription list"	for the	site.  The
       default subscription is to get all groups carried by the	server.	 It is
       a uwildmat pattern supporting poison ("@") wildcards; see the
       libinn_uwildmat man page	for full details on the	pattern	matching
       language.  pattern will be matched against every	newsgroup carried by
       the server and all newsgroups that match	will be	added to the
       subscription list for the site.

       Normally, a given article (or information about it) is sent to a	site
       if any of the newsgroups	to which the article was posted	are in that
       site's subscription list.  If a newsgroup matches an "@"	pattern	in
       pattern,	then not only is it not	added to the subscription list,	but
       any articles crossposted	to that	newsgroup also will not	be sent	to
       that site even if other newsgroups to which it was crossposted are in
       that site's subscription	list.  This is called a	poison pattern
       (because	matching groups	are "poisoned").

       For example, to receive all comp.* groups, but only comp.sources.unix
       within the sources newsgroups, the following pattern can	be used:

	   comp.*,!comp.sources.*,comp.sources.unix

       Note that the trailing ".*" is required;	the pattern has	to match the
       whole newsgroup name.  "comp.sources.*" could be	written
       "comp.sources*" and would exclude the newsgroup comp.sources (if	it
       exists) as well as the groups in	the comp.sources.* hierarchy, but note
       that this would also exclude a newsgroup	named comp.sources-only
       (whereas	the above pattern would	add that group to the site
       subscription list since it matches "comp.*" and none of the other
       patterns).

       For another example, to feed alt.* and misc.* to	a given	site but not
       any articles posted to alt.binaries.warez (even if they're also
       crossposted to other alt.* or misc.* groups), the following pattern can
       be used:

	   alt.*,@alt.binaries.warez,misc.*

       Note, however, that if you reversed the "alt.*" and
       "@alt.binaries.warez" entries, this pattern would be equivalent to
       "alt.*,misc.*", since the last matching pattern determines whether a
       given newsgroup matches and the poison logic only applies if the	poison
       entry is	the last matching entry.

       Control messages	follow slightly	different propagation rules than
       normal articles;	see innd(8) for	the details.  Note that	most
       subscriptions should have "!junk,!control,!control.*" in	their pattern
       list due	to those propagation rules (and	since "junk" is	a special
       internal	newsgroup; see wanttrash in inn.conf(5)	for more details on
       what it's used for) and that the	best way to keep control messages
       local to	a site is with a distribution.

       A subscription can be further modified by specifying distributions that
       the site	should or should not receive.  The default is to send all
       articles	to all sites that subscribe to any of the groups where it has
       been posted, but	if an article has a Distribution header	field and any
       distributions are specified, then they are checked according to the
       following rules:

       1.  If  the  Distribution header	field matches any of the values	in the
	   sub-field, the article is sent.

       2.  If a	distribution starts with an exclamation	point, and it  matches
	   the Distribution header field, the article is not sent.

       3.  If the Distribution header field does not match any distribution in
	   the	site's	entry  and  no negations were used, the	article	is not
	   sent.

       4.  If the Distribution header field does not match any distribution in
	   the site's entry and	any distribution started with  an  exclamation
	   point, the article is sent.

       If  an  article has more	than one distribution specified, then each one
       is handled according according to the  above  rules.   If  any  of  the
       specified  distributions	 indicate  that	the article should be sent, it
       is; if none do, it is not sent.	In other words,	the rules are used  as
       a logical or.

       It  is almost definitely	a mistake to have a single feed	that specifies
       distributions that start	with an	exclamation point along	with some that
       don't.

       Distributions are text words, not patterns; entries like	"*"  or	 "all"
       have no special meaning.

       The  flag  field	 is described in "FLAG VALUES".	 The interpretation of
       the parameter field depends on the type of feed	and  is	 explained  in
       more  detail  in	 "FEED	TYPES".	  It  can be omitted for some types of
       feeds.

       The site	named "ME" is special.	There must be exactly one such	entry,
       and it should be	the first entry	in the file.  If the "ME" entry	has an
       exclusion  sub-field,  incoming articles	are rejected completely	if any
       of the names specified in that exclusion	sub-field appear in their Path
       header field bodies.  If	the "ME" entry has a subscription  list,  that
       list  is	 prepended to the subscription list of all other entries.  For
       example,	"*,!control,!control.*,!junk,!foo.*" could be used to set  the
       default	subscription  list  for	all other feeds	so that	local postings
       are not propagated unless "foo.*"  explicitly  appears  in  the	site's
       subscription  list.   This feature tends	to be somewhat confusing since
       the default subscription	is prepended and can be	 overridden  by	 other
       patterns.

       If  the	"ME"  entry  has  a distribution sub-field, only articles that
       match that distribution list are	accepted and all other articles	with a
       distribution are	rejected.  A common use	for this is to	put  something
       like  "/!local"	in  the	"ME" entry to reject local postings from other
       misconfigured sites.  The distribution sub-field	of "ME"	has no	effect
       on  the	acceptance  or	rejection  of  articles	 that  do  not	have a
       Distribution header field.

       An empty	"ME" entry is possible,	in which case  no  exclusion  patterns
       will be defined.

       Finally,	it is also possible to set variables in	newsfeeds and use them
       later  in  the  file.   A  line starting	with "$" sets a	variable.  For
       example:

	   $LOCALGROUPS=local.*,example.*
	   $CONTROLGROUPS=control,control.*

       This sets the variable "LOCALGROUPS"  to	 "local.*,example.*"  and  the
       variable	 "CONTROLGROUPS"  to  "control,control.*".   They can later be
       used elsewhere in the file, such	as in a	site entry like:

	   news.example.com:$LOCALGROUPS:Tf,Wnm:

       which is	then completely	equivalent to:

	   news.example.com:local.*,example.*:Tf,Wnm:

       Variables aren't	solely simple substitution.   If  either  "!"  or  "@"
       immediately  precedes  the  variable  and  the  value  of  the variable
       contains	commas,	that character will be duplicated before  each	comma.
       This  somewhat  odd-sounding  behavior is designed to make it easier to
       use variables to	construct feed patterns.   The	utility	 becomes  more
       obvious when you	observe	that the line:

	   news.example.net:*,@$LOCALGROUPS,!$CONTROLGROUPS:Tf,Wnm:

       is therefore equivalent to:

	   news.example.net:*,@local.*,@example.*,!control,!control.*:Tf,Wnm:

       which  (as  explained below) excludes all of the	groups in $LOCALGROUPS
       and unwanted control articles from the feed to that site.

FLAG VALUES
       The flags parameter specifies miscellaneous parameters,	including  the
       type  of	 feed,	what  information  should  be  sent to it, and various
       limitations on what articles should be sent to a	 site.	 They  may  be
       specified  in  any order	and should be separated	by commas.  Flags that
       take values should have the value immediately  after  the  flag	letter
       with no whitespace.  The	valid flags are:

       < size
	   An  article	will only be sent to this site if it is	less than size
	   bytes long.	The default is no limit.

       > size
	   An article will only	be sent	to this	site if	 it  is	 greater  than
	   size	bytes long.  The default is no limit.

       A checks
	   An  article	will  only  be	sent  to  this	site  if  it meets the
	   requirements	specified in checks, which should be chosen  from  the
	   following  set.   checks  can  be  multiple letters if appropriate.
	   Note	that this flag is not effective	on funnel targets; it  has  to
	   be used on every funnel entry (for instance,	Af is not effective on
	   the	innfeed!  funnel  target  and therefore	has to be specified on
	   every funnelled news	site).

	   c  Exclude all kinds	of control messages.

	   C  Only send	control	messages, not regular articles.

	   d  Only send	articles with a	Distribution header  field.   Combined
	      with a particular	distribution value in the distribution part of
	      the  site	 entry,	 this  can be used to limit articles sent to a
	      site to just those with a	particular distribution.

	   e  Only  send  articles  where  every  newsgroup  listed   in   the
	      Newsgroups header	field body exists in the active	file.

	   f  Don't  send  articles  rejected by filters.  This	is only	useful
	      when dontrejectfiltered is set to	true in	inn.conf.   With  that
	      variable	set,  this  lets  one  accept  all  articles  but  not
	      propagate	filtered ones to some sites.

	   j  Propagate	articles according to their Newsgroups	header	field.
	      This  is	only useful when wanttrash is set to true in inn.conf.
	      With that	variable set, articles accepted	and  filed  in	"junk"
	      (due  to wanttrash) are fed to peers based on their subscription
	      pattern applied to the Newsgroups	header field  as  though  they
	      were  accepted  and  all	those  groups  were  locally  carried.
	      Otherwise, they are propagated to	sites that receive the	"junk"
	      newsgroup.

	      This  variable  is  useful if you	want to	run INN	with a minimal
	      active file and propagate	all posts.

	   o  Only send	articles for which overview data was stored.

	   O  Send articles to this site that don't have an Injection-Info  or
	      X-Trace header field, even if the	"O" flag is also given.

	   p  Only  check  the	exclusions  against  the  Path header field of
	      articles;	don't check the	site name.  This  is  useful  if  your
	      site names aren't	the same as the	path identities	in Path	header
	      fields  added  by	those remote sites, or for program feeds where
	      the site name is arbitrary and  unrelated	 to  the  Path	header
	      field body.

	   If  both  "c"  and  "C"  are	 given,	 the  last specified one takes
	   precedence.

       B high/low
	   If a	site is	being fed by a file, channel, or exploder (see below),
	   the server will normally start trying to write the  information  as
	   soon	 as  possible.	 Providing  a  buffer  may  give better	system
	   performance and help	smooth out overall load	if a  large  batch  of
	   news	 comes	in.   The  value  of  this  flag should	be two numbers
	   separated by	a slash.  high specifies the point at which the	server
	   can start draining the feed's I/O buffer, and low specifies when to
	   stop	writing	and begin buffering again; the units are  bytes.   The
	   default  is	to  do	no  buffering, sending output as soon as it is
	   possible to do so.

       C count
	   If this flag	is specified, an article will only  be	sent  to  this
	   site	 if  the  number of groups it is posted	to, plus the square of
	   the number of groups	followups would	appear in,  is	no  more  than
	   count.  30 is a good	value for this flag, allowing for instance:

	     crossposts to only 5 groups when followups aren't	set;

	     crossposts  to  up  to  26  groups when followups	are set	to two
	      groups;

	     crossposts to up to 29 groups when followups are set to a	single
	      group;

	     crossposts to up to 30 groups when followups are set to poster.

	   Note	that if	an article  does  not  contain	a  Followup-To	header
	   field, the number of	groups followups would appear in is the	number
	   of groups it	is posted to.

       F name
	   Specifies  the  name	 of  the  file	that  should  be  used if it's
	   necessary to	begin spooling for the site (see below).  If  name  is
	   not an absolute path, it is taken to	be relative to pathoutgoing in
	   inn.conf.   If name is a directory, the file	togo in	that directory
	   will	be used	as the file name.

       G count
	   If this flag	is specified, an article will only  be	sent  to  this
	   site	 if  it	 is posted to no more than count newsgroups.  This has
	   the problem of filtering  out  many	FAQs,  as  well	 as  newsgroup
	   creation postings and similar administrative	announcements.	Either
	   the C flag or the U flag is a better	solution.

       H count
	   If  this  flag  is  specified, an article will only be sent to this
	   site	if it has count	or fewer sites in its Path header field	 body.
	   This	flag should only be used as a rough guide because of the loose
	   interpretation  of  the  Path  header  field;  some	sites  put the
	   poster's name in the	header field body, and some sites  that	 might
	   logically  be  considered to	be one hop become two because they put
	   the posting workstation's name  in  the  header  field  body.   The
	   default  value  for count if	not specified is one.  (Also see the O
	   flag, which is sometimes more appropriate for  some	uses  of  this
	   flag.)

       I size
	   The flag specifies the size of the internal buffer for a file feed.
	   If  there are more file feeds than allowed by the system, they will
	   be  buffered	 internally  in	 least-recently-used  order.   If  the
	   internal  buffer  grows  bigger  than size bytes, however, the data
	   will	be written out to the appropriate file.	 The default value  is
	   16 KB.

       N status
	   Restricts  the  articles  sent  to this site	to those in newsgroups
	   with	the moderation status given by status.	If status is "m", only
	   articles in moderated groups	are  sent;  if	status	is  "u",  only
	   articles in unmoderated groups are sent.

       O originator
	   If  this  flag  is  specified, an article will only be sent to this
	   site	if it contains an Injection-Info header	field (or  an  X-Trace
	   header  field  if  no Injection-Info	header field is	found) and the
	   first field of this header field matches originator.	 originator is
	   a uwildmat expression without commas	or a list of such expressions,
	   separated by	"/".  The article is never sent	if the first character
	   of the pattern begins with "@" and the rest of the pattern matches.
	   One use of this flag	is to restrict the feed	to  locally  generated
	   posts  by  using  an	originator pattern that	matches	the Injection-
	   Info	header field added by the local	server.

       P priority
	   The nice priority that this channel or program feed should receive.
	   This	should be a positive number  between  0	 and  20  and  is  the
	   priority that the new process will run with.	 This flag can be used
	   to  raise  the  priority  to	 normal	 if  you're using the nicekids
	   parameter in	inn.conf.

       Q hashfeed
	   Specifies the hashfeed match	expression for this site.  It must  be
	   in  the form	"value/mod" or "start-end/mod".	 The Message-ID	of the
	   article is hashed using MD5,	which results in a 128-bit hash.   The
	   lowest  32 bits  are	 then  taken  by default as the	hashfeed value
	   (which is an	integer).  If the hashfeed value  modulus  "mod"  plus
	   one	equals	"value"	 or  is	between	"start"	and "end", the article
	   will	be fed to this site.  All these	numbers	must be	integers.

	   It is a deterministic way to	control	the flow of  articles  and  to
	   split a feed.  For instance:

	       Q1/2	 Feeds about 50% of all	articles to this site.
	       Q2/2	 Feeds the other 50% of	all articles.

	   Another example with	three sites:

	       Q1-3/10	 Feeds about 30% of all	articles.
	       Q4-5/10	 Feeds about 20% of all	articles.
	       Q6-10/10	 Feeds about 50% of all	articles.

	   If  this  flag  is  specified  multiple times, the contents will be
	   logically "OR"ed together (just one match is	needed).

	   You can use an extended syntax of the  form	"value/mod_offset"  or
	   "start-end/mod_offset".   As	 MD5 generates a 128-bit return	value,
	   it is possible to specify from which	byte-offset the	32-bit integer
	   used	by hashfeed starts.  The default value for  "offset"  is  "_0"
	   and	thirteen  overlapping  values  from "_0" to "_12" can be used.
	   Only	up to four totally independent values exist: "_0", "_4",  "_8"
	   and "_12".

	   Therefore,  it  allows  generating  a second	level of deterministic
	   distribution.  Indeed, if a news server is fed "Q1/2", it can go on
	   splitting thanks to "Q1-3/9_4" for instance.	 Up to four levels  of
	   deterministic distribution can be used.

	   The algorithm is compatible with the	one used by Diablo 5.1 and up.
	   If  you  want  to use the legacy quickhashing method	used by	Diablo
	   before 5.1, you can put an "@" sign just  after  the	 Q  flag  (for
	   instance  "Q@1-3/10",  but  the distribution	of the messages	is not
	   perfect with	this legacy method whose use is	 discouraged  and  for
	   which offsets cannot	be used).

       S size
	   If  the  amount  of data queued for the site	gets to	be larger than
	   size	bytes, the server will switch to spooling, appending to	a file
	   specified by	the F flag,  or	 pathoutgoing/sitename	if  F  is  not
	   specified.	Spooling  usually happens only for channel or exploder
	   feeds, when the spawned program isn't keeping up with its input.

       T type
	   This	flag specifies the type	of feed	for this site.	type should be
	   a letter chosen from	the following set:

	       c	Channel
	       f	File
	       l	Log entry only
	       m	Funnel (multiple entries feed into one)
	       p	Program
	       x	Exploder

	   Each	feed is	described below	in "FEED TYPES".  The default  is  Tf,
	   for a file feed.

       U count
	   If  this  flag  is  specified, an article will only be sent to this
	   site	if followups to	this article would be posted to	no  more  than
	   count  newsgroups.	(Also see C for	a more complex way of handling
	   this.)

       W items
	   For a file, channel,	or exploder  feed,  this  flag	controls  what
	   information	will  be  sent to this site.  For a program feed, only
	   the asterisk	("*") has any effect.  items should be chosen from the
	   following set:

	   b  Size of the article (in wire format, meaning with	 CRLF  at  the
	      end of each line,	periods	doubled	at the beginning of lines, and
	      ending in	a line with a single period) in	bytes.

	   e  The  time	 the  article will expire as seconds since epoch if it
	      has an Expires header field, 0 otherwise.

	   f  The storage API token of the article (the	 same  as  "n").   The
	      article  can  be	retrieved given	the storage API	token by using
	      sm(1).

	   g  The newsgroup the	article	is in; if cross-posted,	then the first
	      of the groups to which the article was  posted  that  this  site
	      gets.  (The difference from "G" is that this sends the newsgroup
	      to  which	 the  article  was  posted  even  if  it  is a control
	      message.)

	   h  The history hash key of the article (derived  from  the  message
	      ID).

	   m  The message ID of	the article.

	   n  The  storage  API	 token	of  the	 article.   The	article	can be
	      retrieved	given the storage API token by using sm(1).

	   p  The time the article was posted a	seconds	since epoch.

	   s  The site that fed	the article to the server.  This is taken from
	      either the Path header field body	 or  the  IP  address  of  the
	      sending  site  depending	on the value of	logipaddr in inn.conf.
	      If logipaddr is true and the IP address is 0.0.0.0 (meaning that
	      the article was fed from localhost by a program like  rnews(1)),
	      the Path header field body will be sent instead.

	   t  The time the article was received	as seconds since epoch.

	   *  The  names  of the appropriate funnel entries, or	all sites that
	      get the article (see below for more details).

	   D  The body of the Distribution header field	of the article,	or "?"
	      if there is no such header field in the article.

	   G  Where the	article	is stored.  If the newsgroup  is  crossposted,
	      this is generally	the first of the groups	to which it was	posted
	      that  this site receives;	however, control messages are filed in
	      control or control.*  (which is the difference between this item
	      and "g").

	   H  All of the headers, followed by a	blank line.  The  Xref	header
	      field  will  already  be	present,  and  a  Bytes	 header	 field
	      containing the article's size in bytes as	in the "b"  item  will
	      be  added	to the headers.	 If used, this should be the only item
	      in the list.

	   N  The body of the Newsgroups header	field.

	   P  The body of the Path header field.

	   O  Overview data for	the article.

	   R  Information  needed  for	replication  (the  Xref	 header	 field
	      without the site name).

	   More	 than  one  letter  can	 be  given.   If  multiple  items  are
	   specified, they will	be written in the order	specified separated by
	   spaces.  ("H" should	be the only item if given, but if it's	not  a
	   newline  will  be  sent  before the beginning of the	headers.)  The
	   default is Wn.

	   The "H" and "O" items are intended for use by programs that	create
	   news	 overview  databases or	require	similar	information.  WnteO is
	   the flag to generate	input needed by	the overchan(8)	program.

	   The asterisk	("*") has special meaning.  Normally it	expands	 to  a
	   space-separated  list  of  all  sites  that	received  the  current
	   article.  If, however, this site is a target	of a funnel  feed  (in
	   other words,	if it is named by other	sites which have the Tm	flag),
	   then	 the  asterisk	expands	 to the	names of the funnel feeds that
	   received the	article.  Similarly, if	the site is a program feed, an
	   asterisk in the parameter field will	be expanded into the  list  of
	   funnel  feeds that received the article.  A program feed cannot get
	   the site list unless	it is the target of other Tm feeds.

FEED TYPES
       innd provides four basic	 types	of  feeds:  log,  file,	 program,  and
       channel.	  An  exploder	is  a  special	type of	channel.  In addition,
       several entries can feed	into the same feed; these  are	funnel	feeds,
       which  refer  to	an entry that is one of	the other types.  Funnel feeds
       are partially described above with the description of the W*  flag.   A
       funnel  feed  gets every	article	that would be sent to any of the feeds
       that funnel into	it and normally	include	the W* flag in their flags  so
       that  the program processing that feed knows which sites	received which
       articles.  The most common funnel feed is innfeed(8).

       Note that the term "feed" is technically	a misnomer, since  the	server
       doesn't	transfer  articles  itself  and	 only  writes  data to a file,
       program,	or log telling another program to transfer the articles.

       The simplest feed is a log feed (Tl).  Other than a mention in the news
       log file, pathlog/news, no data is written out.	This is	equivalent  to
       a  Tf  entry  writing to	/dev/null, except that no file is ever opened.
       Flushing	a log feed does	nothing.

       A file feed (Tf)	is the next simplest type  of  feed.   When  the  site
       should  receive	an  article,  the specified data is written out	to the
       file named by the parameter field.  If parameter	 is  not  an  absolute
       path,  it  is  taken  to	 be  relative to pathoutgoing in inn.conf.  If
       parameter is not	given, it defaults to pathoutgoing/sitename.  The file
       name should be unique (two file feeds should not	ever point to the same
       file).

       File feeds are designed for use by external programs that  periodically
       process	the  written  data.   To  cooperate  with  innd	properly, such
       external	programs should	first rename the batch file and	 then  send  a
       flush  command  for that	site to	innd using ctlinnd(8).	innd will then
       write out any buffered data, close the file, and	reopen it  (under  the
       original	 name),	 and  the  program can process the data	in the renamed
       file  at	 its  leisure.	 File  feeds  are  most	 frequently  used   in
       combination with	nntpsend(8).

       A  program  feed	(Tp) spawns a given program for	every article that the
       site receives.  The  parameter  field  must  be	the  command  line  to
       execute,	 and should contain one	instance of %s,	which will be replaced
       by the storage API token	of the article	(the  actual  article  can  be
       retrieved  by  the  program using sm(1)).  The program will not receive
       anything	on standard input (unlike earlier versions of INN,  where  the
       article is sent to the program on stdin), and standard output and error
       from  the  program will be set to the error log (pathlog/errlog).  innd
       will try	to avoid spawning a shell if the command has  no  shell	 meta-
       characters;  this  feature can be defeated if necessary for some	reason
       by appending a semi-colon to the	end of the  command.   The  full  path
       name of the program to be run must be specified unless the command will
       be  run by the shell (and it is strongly	recommended that the full path
       name always be specified	regardless).

       If a program feed is the	target of a funnel, and	if W* appears  in  the
       flags  of  the  site, a single asterisk may be present in the parameter
       and will	be replaced by a space-separated list of names	of  the	 sites
       feeding	into  the  funnel which	received the relevant article.	If the
       site is not the target of a funnel, or if the W*	flag is	not used,  the
       asterisk	has no special meaning.

       Flushing	a program feed does nothing.

       For  a  channel	(Tc)  or exploder (Tx) feed, the parameter field again
       names the process to start.  As with program feeds, the	full  path  to
       the  program  must  be  specified.   However,  rather than spawning the
       program for every article, it is	spawned	once  and  then	 whenever  the
       site  receives  an  article,  the  data	specified by the site flags is
       written to the standard input of	the spawned program.  Standard	output
       and error are set as with program feeds.	 If the	process	exits, it will
       be  restarted  automatically.   If  the	process	cannot be started, the
       server will spool input to a file named pathoutgoing/sitename and  will
       try to start the	process	again later.

       When  a	channel	or exploder feed is flushed, the server	closes its end
       of the pipe to the program's standard input.  Any pending data that has
       not been	written	will be	spooled; see the description  of  the  S  flag
       above.	The  server will then spawn a new instance of the program.  No
       signal is sent to the program; it is  up	 to  the  program  handling  a
       channel	or  exploder  feed to notice end of file on its	standard input
       and exit	appropriately.

       Exploders are a special type of	channel	 feed.	 In  addition  to  the
       channel	feed  behavior	described  above,  exploders  can also be sent
       command lines.  These lines start with an exclamation point  and	 their
       interpretation  is  up  to  the	exploder.   The	following commands are
       generated automatically by the server:

	   !newgroup group
	   !rmgroup group
	   !flush
	   !flush site

       These commands are sent whenever	the ctlinnd(8)	command	 of  the  same
       name  is	 received  by  the server.  In addition, the ctlinnd(8)	"send"
       command can be used to send an arbitrary	command	line to	 an  exploder.
       The primary exploder is buffchan(8).

       Finally,	 Tm  feeds  are	the input to a funnel.	The parameter field of
       the site	should name the	site handling articles for all of  the	funnel
       inputs.

EXAMPLES
       The  syntax of the newsfeeds file is so complex because you can specify
       a staggering variety of feeds.  INN is capable of  interacting  with  a
       wide  variety  of  programs  that do various things with	news articles.
       Far and away the	most common two	entries	 in  newsfeeds,	 however,  are
       file feeds for nntpsend(8) and funnel feeds for innfeed(8).

       The former look like this:

	   feed.example.com:*,!control,!control.*,!junk:Tf,Wnm:

       which  generates	 a file	named pathoutgoing/feed.example.com containing
       one line	per article consisting of the storage API token, a space,  and
       the message ID.

       The latter look like this:

	   feed.example.com:*,!control,!control.*,!junk:Tm:innfeed!

       Very  similar,  except  that  this  is the input	to a funnel feed named
       "innfeed!".  One	could also write this as:

	   example/feed.example.com:*,!control,!control.*,!junk:Ap,Tm:innfeed!

       (note the Ap so that articles that contain just "example" in  the  Path
       header  field  body will	still be sent),	which is completely equivalent
       except that this	will be	logged in pathlog/news as going	 to  the  site
       "example" rather	than "feed.example.com".

       The  typical  feed  entry for innfeed(8)	is a good example of a channel
       feed that's the target of various funnel	feeds (make sure the  path  to
       innfeed is properly set):

	   innfeed!:!*:Tc,Wnm*:<pathbin	in inn.conf>/innfeed -y

       Note  that  the	pattern	 for  this  feed is just "!*" so that it won't
       receive any articles directly.  The  feed  should  only	receive	 those
       articles	that would go to one of	the funnel feeds that are feeding into
       it.  innfeed(8) will receive one	line per article on its	standard input
       containing the storage API token, the message ID, and a space-separated
       list of sites that should receive that article.

       Here's a	more esoteric example of a channel feed:

	   watcher!:*:Tc,Wbnm\
	       :exec awk '$1 > 1000000 { print "BIG", $2, $3 }'	\
		>> <pathlog>/bigart

       This  receives the byte size of each article along with the storage API
       token and Message-ID, and appends to a bigart log file a	line for every
       article that's over a million bytes.  This is actually rather a strange
       way to write this since INN can do the size check itself; the following
       is equivalent:

	   watcher!:*:Tc,>1000000,Wbnm\
	       :exec awk '{ print "BIG", $2, $3}' >> <pathlog>/bigart

       Here's a	cute, really simple news to mail gateway that also  serves  as
       an example of a fairly fancy program feed:

	   mailer!:!*:W*,Tp\
	       :sm %s |	innmail	-s "News article" *

       Remember	 that  %s  is  replaced	 by  the  storage  API	token, so this
       retrieves the article and pipes it into innmail (which  is  safer  than
       programs	 like  Mail(1)	because	 it  doesn't  parse the	body for tilde
       commands) with a	given subject line.   Note  the	 use  of  "*"  in  the
       command	line and W* in the flags; this entry is	designed to be used as
       the target of funnel feeds such as:

	   peter@example.com:news.software.nntp:Tm:mailer!
	   sue@example.com:news.admin.misc:Tm:mailer!

       Suppose	that  the  server  receives  an	 article  crossposted  between
       news.admin.misc	and  news.software.nntp.   The server will notice that
       the article should be sent to the site "peter@example.com" and the site
       "bob@example.com", both of which	funnel into "mailer!", so it will look
       at the "mailer!"	site and end up	executing the command line:

	   sm @...@ | innmail -s "News article"	\
	       peter@example.com sue@example.com

       which will mail the article to both Peter and Sue.

       Finally,	another	very useful example of a channel  feed:	 the  standard
       entry for controlchan(8).  Make sure its	path is	properly set.

	   controlchan!\
	       :!*,control,control.*,!control.cancel\
	       :AC,Tc,Wnsm:<pathbin in inn.conf>/controlchan

       This  program only wants	information about articles posted to a control
       newsgroup other than  control.cancel,  which  due  to  the  sorting  of
       control messages	described in innd(8) will send it all control messages
       except  for  cancel  messages.  controlchan gets	the storage API	token,
       the name	of the sending site (for processing old-style ihave and	sendme
       control messages, be sure to read about logipaddr  in  controlchan(8)),
       and the message ID for each article.

       For  many  other	 examples, including examples of the special "ME" site
       entry, see the example newsfeeds	file distributed with INN.   Also  see
       the  install  documentation  that  comes	with INN for information about
       setting up the standard newsfeeds entries used by most sites.

HISTORY
       Written	 by   Rich   $alz   <rsalz@uunet.uu.net>   for	 InterNetNews.
       Reformatted and rewritten in POD	by Russ	Allbery	<eagle@eyrie.org>.

SEE ALSO
       active(5),   buffchan(8),   controlchan(8),   ctlinnd(8),   delayer(1),
       inn.conf(5),  innd(8),  innfeed(8),   innxmit(8),   libinn_uwildmat(3),
       nntpsend(8).

INN 2.8.0			  2024-03-14			  NEWSFEEDS(5)

Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=newsfeeds&sektion=5&manpath=FreeBSD+Ports+14.3.quarterly>

home | help