discuss: Thread: Progress on the Anti-Spam HOWTO

Hi,
I noticed that there was some discussion about writing an Anti-Spam 
HOWTO, around Sept 2002. Has that got off the ground?
If so, I would like to help. If not, I am willing to start writing one.
I had planned to concentrate on the technical issues of setting up 
anti-spam filtering both at the local level,  the MUA level, as well as 
site-wide installations, the MTA tier, both using a mix of current 
best-of-breed open source technologies.
All this information is available in discrete, distributed sources; 
however, I feel, that the time is right fot a LDP doc.

Chris Albert
Enterprise Systems
Network and Communication Services
McGill University
christopher (period) albert (at) mcgill (dot) ca

chris albert wrote:
> Hi,
> I noticed that there was some discussion about writing an Anti-Spam 
> HOWTO, around Sept 2002. Has that got off the ground?
> If so, I would like to help. If not, I am willing to start writing one.
> I had planned to concentrate on the technical issues of setting up 
> anti-spam filtering both at the local level,  the MUA level, as well as 
> site-wide installations, the MTA tier, both using a mix of current 
> best-of-breed open source technologies.
> All this information is available in discrete, distributed sources; 
> however, I feel, that the time is right fot a LDP doc.

this is a very good idea.

spam has always be a problem but it becomes more and more annoying - I 
work with internet and don't bother faking my e-mail, so I receive 
approx a hundred spam a day (more than half my mails).

I use mozilla and it's junk control but, for now, it needs still read 
the mail on my computer. thanks god I hav a dsl line, but even like this 
  it make me loose too many time

yhanks
jdd

-- 
<http://www.dodin.net>
Formation Linux débutants open

On Sat, May 31, 2003 at 07:14:12AM +0200, jdd wrote:
> 
> I use mozilla and it's junk control but, for now, it needs still
> read the mail on my computer. thanks god I hav a dsl line, but even
> like this it make me loose too many time

I have an interest and would consider helping as well. I am doing
postfix+procmail+bogofilter with the goal of getting at least one week
in my life with no spam. 

               Week to Date Spam Summary 
Sun May 25 04:10:00 thru Sat May 31 09:57:44 EDT 2003
 Postfix Rejected:      803
 Procmail Filtered:      35
 Bogofied:               75
 Spams missed:           33
 Total Spams:           946
 Spams Blocked:         913
 Percent Blocked:        97%          


It would be much better than that, but I botched one of my scripts
that dynamically writes to procmailrc and some stuff got through that
shouldn't. That's a home DSL line that hosts a domain. My ISP blocks
200-300 a week for me as well.

-- 
Hal Burgiss
 

Hal ,

>I have an interest and would consider helping as well.
>
What is the best way to do this? Should we get a cvs account right away 
at tldp?

I was just going to use the one of the big howto templates
http://tldp.org/authors/template-ld/big-howto-template-ld.html
follow the authoring guide
http://tldp.org/LDP/LDP-Author-Guide/index.html
using style suggestions from the gnome guide
http://developer.gnome.org/documents/style-guide/ .

Locally I'll import and export the sgml files from LyX so I wont need to 
worry to much about sgml syntax.
Since I am in Quebec, I will also be able to get the doc translated into 
French.
Since I am on dozens of anti-spam mailing lists, I thought once the doc 
had some meat on it I would try and get
some help on specific technologies, since there really is too much going 
on right now for one person to master.

> I am doing
>postfix+procmail+bogofilter with the goal of getting at least one week
>in my life with no spam. 
>  
>
It can be done! Certainly at the local level, but it is harder site wide 
because of false positives,
and varying ham/spam profiles among a large user base.

Have you thought of combining bogofilter with spamassassin?
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/bogofilter/bogofilter/doc/bogofilter-faq.html#spamassassin
http://www.randomhacks.net/2002/09/23/
http://hotwired.lycos.com/webmonkey/03/06/index2a_page6.html?tw=backend

Let me know how you want to proceed.

Thanks,

Chris

On Sat, May 31, 2003 at 10:46:32AM -0400, chris albert wrote:

> What is the best way to do this? Should we get a cvs account right
> away at tldp?

CVS would be good for a group effort like this. AFAIK, what we need is
a) a fixed doc title for cvs purposes and then b) each author needs an
ldp/cvs account to access cvs (I have one now).

So I think the first order of business:

 - Decide on Doc title
 - Decide doc format (sgml, xml, etc)
 - Decide doc scope (eg usenet spam, etc....)
 - See who has what areas of interest and divide responsibility among 
   authors
 - Make sure we are listed as a "work in progress" with tldp (?).
 - Do an outline
 - Write ;-)
 
> I was just going to use the one of the big howto templates
> http://tldp.org/authors/template-ld/big-howto-template-ld.html
> follow the authoring guide
> http://tldp.org/LDP/LDP-Author-Guide/index.html
> using style suggestions from the gnome guide
> http://developer.gnome.org/documents/style-guide/ .

Just a personal note ... but I have worked very little with xml, and
would like to learn that, and maybe this is a good oppurtunity!

> Locally I'll import and export the sgml files from LyX so I wont need to 
> worry to much about sgml syntax.
> Since I am in Quebec, I will also be able to get the doc translated into 
> French.
> Since I am on dozens of anti-spam mailing lists, I thought once the doc 
> had some meat on it I would try and get
> some help on specific technologies, since there really is too much going 
> on right now for one person to master.

Can you suggest a few lists that aren't real high volume? 
 
> >I am doing
> >postfix+procmail+bogofilter with the goal of getting at least one week
> >in my life with no spam. 
> > 
> >
> It can be done! Certainly at the local level, but it is harder site
> wide because of false positives, and varying ham/spam profiles among
> a large user base.

Yes, of course. The best spam blocking is always the narrowest scope.
One of my best rules is if html, probably is spam. That wouldn't work
for most, I would guess though.
 
> Have you thought of combining bogofilter with spamassassin?

No, not really, but I should look into it. Right now I am spam free
for about 3 days though :)))))))))))))))

> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/bogofilter/bogofilter/doc/bogofilter-faq.html#spamassassin
> http://www.randomhacks.net/2002/09/23/
> http://hotwired.lycos.com/webmonkey/03/06/index2a_page6.html?tw=backend
> 
> Let me know how you want to proceed.

-- 
Hal Burgiss
 

One minor point to add - usually on the "In Progress" list there is at
least one contact shown - you'll have to decide who wants that
responsibility!

Tab


On Sat, 2003-05-31 at 08:23, Hal Burgiss wrote:
> On Sat, May 31, 2003 at 10:46:32AM -0400, chris albert wrote:
> 
> > What is the best way to do this? Should we get a cvs account right
> > away at tldp?
> 
> CVS would be good for a group effort like this. AFAIK, what we need is
> a) a fixed doc title for cvs purposes and then b) each author needs an
> ldp/cvs account to access cvs (I have one now).
> 
> So I think the first order of business:
> 
>  - Decide on Doc title
>  - Decide doc format (sgml, xml, etc)
>  - Decide doc scope (eg usenet spam, etc....)
>  - See who has what areas of interest and divide responsibility among 
>    authors
>  - Make sure we are listed as a "work in progress" with tldp (?).
>  - Do an outline
>  - Write ;-)
>  
> > I was just going to use the one of the big howto templates
> > http://tldp.org/authors/template-ld/big-howto-template-ld.html
> > follow the authoring guide
> > http://tldp.org/LDP/LDP-Author-Guide/index.html
> > using style suggestions from the gnome guide
> > http://developer.gnome.org/documents/style-guide/ .
> 
> Just a personal note ... but I have worked very little with xml, and
> would like to learn that, and maybe this is a good oppurtunity!
> 
> > Locally I'll import and export the sgml files from LyX so I wont need to 
> > worry to much about sgml syntax.
> > Since I am in Quebec, I will also be able to get the doc translated into 
> > French.
> > Since I am on dozens of anti-spam mailing lists, I thought once the doc 
> > had some meat on it I would try and get
> > some help on specific technologies, since there really is too much going 
> > on right now for one person to master.
> 
> Can you suggest a few lists that aren't real high volume? 
>  
> > >I am doing
> > >postfix+procmail+bogofilter with the goal of getting at least one week
> > >in my life with no spam. 
> > > 
> > >
> > It can be done! Certainly at the local level, but it is harder site
> > wide because of false positives, and varying ham/spam profiles among
> > a large user base.
> 
> Yes, of course. The best spam blocking is always the narrowest scope.
> One of my best rules is if html, probably is spam. That wouldn't work
> for most, I would guess though.
>  
> > Have you thought of combining bogofilter with spamassassin?
> 
> No, not really, but I should look into it. Right now I am spam free
> for about 3 days though :)))))))))))))))
> 
> > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/bogofilter/bogofilter/doc/bogofilter-faq.html#spamassassin
> > http://www.randomhacks.net/2002/09/23/
> > http://hotwired.lycos.com/webmonkey/03/06/index2a_page6.html?tw=backend
> > 
> > Let me know how you want to proceed.
-- 
Tabatha Marshall
Web: www.merlinmonroe.com
Linux Documentation Project Review Coordinator (http://www.tldp.org)
Linux Counter Area Manager US:wa (http://counter.li.org)

Hal

>CVS would be good for a group effort like this. AFAIK, what we need is
>a) a fixed doc title for cvs purposes and then b) each author needs an
>ldp/cvs account to access cvs (I have one now).
>
>So I think the first order of business:
>
> - Decide on Doc title
>
Why not the Anti-Spam HOWTO ?
It would be the first thing I searched for, and has the advantage of 
being obvious.

> - Decide doc format (sgml, xml, etc)
>
I am agnostic here, though as I mentioned earlier, I'm using LyX which 
has built in support for linuxdoc sgml.
I spent many years writing TeX documents on emacs, and I'm thinking 
those days are over. It seems I can import
and export docbook xml 4.2 with a db2lyx add on, and so I'm open to xml. 
It really depends on a group consensus,
since I'm sure several people will get involved.

> - Decide doc scope (eg usenet spam, etc....)
>
I was thinking of looking at spam fighting from 3 perspectives: MUA, 
MTA, Network Protocols .
Obviously there is a 'higher' level of Legal Issues; though that is 
beyond my capabilities, and as long
as there are open smtp relays in countries not subject to the laws in 
question, I think practical spam fighting
will need to be done at the three 'technological' tiers mentioned above.

Usenet spam is an interesting and problematic case. First of all, almost 
all the research and spam filtering techniques
have concentrated on email. The usenet users population, of which I am 
one, is trivial compared to email users. Getting spam filtering right at 
the newsreader level could be a daunting task since some people 
subscribe to newsgroups like 
free.porno.flicks.erik-mouse.duct-tape.k-y-jelly , who might also be 
subscribed to , say comp.unix.admin. How to get the filtering right?
Since I never have administered a nntp server, I'm not sure how some 
standard methods, like using DNSBL or RBLs might work.
IMHO flaming maniacs are more of a threat to unmoderated newsgroups than 
spam, but if someone had things to about the usenet spam problem, they 
would be welcome.

> - See who has what areas of interest and divide responsibility among 
>   authors
> - Make sure we are listed as a "work in progress" with tldp (?).
> - Do an outline
>
This latter step seems important, since structural changes in the doc 
are likely to cause the most headaches with cvs commits.

What is the best way to discuss the outline?
In private emails, on this list?
I manage the group that runs the list server at McGill university, and I 
could get created a mailing list for the contributors, if that would be 
useful.
Eventually, I could find web space for the project once it was up and 
running.

Chris

P.S I'll put newsgroups/lists/bookmark sections in the first draft.

On Sat, May 31, 2003 at 02:30:01PM -0400, chris albert wrote:
> >
> >- Decide on Doc title
> >
> Why not the Anti-Spam HOWTO ?

OK. Looking at cvs, this would mean an entry like:

 Anti-Spam-HOWTO

and doc sources as Anti-Spam-HOWTO.sgml. I tried to create it this way
in cvs but failed. I guess I lack permissions to create new entries.

> >- Decide doc format (sgml, xml, etc)
> >
> I am agnostic here, though as I mentioned earlier, I'm using LyX which 
> has built in support for linuxdoc sgml.
> I spent many years writing TeX documents on emacs, and I'm thinking 
> those days are over. It seems I can import
> and export docbook xml 4.2 with a db2lyx add on, and so I'm open to xml. 
> It really depends on a group consensus,
> since I'm sure several people will get involved.

I really don't want to do linuxdoc sgml. Docbook/sgml is fine or xml
is fine with me. 

 
> Usenet spam is an interesting and problematic case. First of all, almost 

I agree, and only mentioned this to make sure we define our mission.
Email spam only then?
 
> What is the best way to discuss the outline?
> In private emails, on this list?
> I manage the group that runs the list server at McGill university, and I 
> could get created a mailing list for the contributors, if that would be 
> useful.

I think at some point, a separate list is indeed a good idea (if not
too much trouble).


-- 
Hal Burgiss
 

On Sat, 31 May 2003 23:13:38 -0400 Hal Burgiss ####@####.#### wrote:

> On Sat, May 31, 2003 at 02:30:01PM -0400, chris albert wrote:
> > >
> > >- Decide on Doc title
> > >
> > Why not the Anti-Spam HOWTO ?
> 
> OK. Looking at cvs, this would mean an entry like:
> 
>  Anti-Spam-HOWTO
> 
> and doc sources as Anti-Spam-HOWTO.sgml. I tried to create it this way
> in cvs but failed. I guess I lack permissions to create new entries.

I created a directory for you and stuck a README file in there.

  <cvs_root>LDP/howto/docbook/Anti-Spam-HOWTO

I also added the entry to our in progress list. I noted Hal as
the point of contact, although I can add more names or change
what's there.

I've been following your discussion with interest as this
concept of 'collaborative authoring' using CVS is exactly
what I had hoped for (a direction the LDP should move in). 
Keep up the good work and best of luck in developing this 
very important HOWTO.

When you have a draft prepared, simply let me/us know.
--
Greg

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hal

|I really don't want to do linuxdoc sgml. Docbook/sgml is fine or xml
|is fine with me.  

If you dont mind, I'd like to go with dockbook sgml . Apparently it is 
not hard to convert
that to xml if we want to at some point.
Should I use the big-howto-template, or do you have other design 
preferences?

|I agree, and only mentioned this to make sure we define our mission.
|Email spam only then?

That's big enough, and what bugs millions of people right now.

|
|I think at some point, a separate list is indeed a good idea (if not
|too much trouble).

I can create a list tomorrow for you and I. I will make us both owners.
What email address to you want me to use?

I made a request for a cvs account, which had a strange slavic twist :

|Contents / Tre[:
|
|>>
|>> CVS Account request for:
|>> ####@####.####
|>>
|>> The reason is:
|>> Creation of the Anti-Spam-howto

Although now that Greg Ferguson has set you up, I guess I'll just need 
rights to that node.

Chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+2itRkRKXIlZkCr8RAmS9AJ9Xd3QARSdA2BjIowvrvtYAxpwKVgCaA5iw
K7MB/X1OQhvNw0BOg58ds+M=
=teT7
-----END PGP SIGNATURE-----