MailCleaner OpenSource Edition Forum

It is currently Fri Sep 10, 2010 7:48 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: sa-learn consuming cpu resources
PostPosted: Thu Feb 18, 2010 12:11 pm 
Offline
User avatar

Joined: Mon Nov 13, 2006 2:30 pm
Posts: 293
Location: India
I have a master and a slave Mailcleaner server. There are not many domains in the server and daily email traffic is less than 30,000 domains. However, every now and then there is a heavy load on both the master and the slave server. I eventually have to restart them to bring down the load, however, till then the emails are delivered after a long period. When I checked the list of running processes using top command, I saw a bunch of sa-learn processes which were consuming the cpu. How can I bring down the load and ensure that sa-learn does not consume cpu?

_________________
Promise only what you can deliver...


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Fri Feb 26, 2010 12:47 pm 
Offline
User avatar

Joined: Mon Nov 13, 2006 2:30 pm
Posts: 293
Location: India
Can somebody help me?

_________________
Promise only what you can deliver...


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Fri Feb 26, 2010 1:32 pm 
Offline
Site Admin
User avatar

Joined: Thu Jan 01, 1970 1:00 am
Posts: 949
mabye you could find help on SpamAssassin's web site: http://www.spamassassin.org, as this is not directly related to MailCleaner.

And as usual, it is not as easy as it seems to get a good and efficient bayesian database. It has to be carefully built and fed, and regularly cleaned up.
Otherwise, it is possible to end up with a corrupted, inefficient database that will only load your system.


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Tue Mar 09, 2010 2:14 pm 
Offline
User avatar

Joined: Mon Nov 13, 2006 2:30 pm
Posts: 293
Location: India
I seem to have found and resolved the problem. I have disabled the auto learn function from the Mailcleaner admin interface. Doing this stopped all the sa-learn processes and the Mailcleaner cluster is working fine. There is no more heavy load and the emails are delivering fast. I have also disabled Razor and Pyzor to bring down the load. Now my only concern is whether these changes will increase the spam on the server? Can Olivier suggest?

_________________
Promise only what you can deliver...


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Tue Mar 09, 2010 3:02 pm 
Offline
Site Admin
User avatar

Joined: Thu Jan 01, 1970 1:00 am
Posts: 949
this is always the same old problem of any automated database building.
For decent performance, efficiency, stability and security, you just can't rely on automated processes.
I know it's sad, because building and keeping such databases and rulesets up-to-date requires time, but believe me, it is really important.

It may seem at first sight, that MailCleaner Community Edition will just work fine out-of-the-box. But time will show that you will have to take care of the different anti-spam modules. And I don't say that to make advertisement for the Enterprise Edition, but rather because I have heard so many time people saying "MailCleaner was good, but now it's crappy and doesn't block spam anymore. It just doesn't work anymore". And when you look at it, they just never set up a single rule...

This is very important to understand that this not a problem of MailCleaner, which still work the way it should. There is unfortunately no magic inside MailCleaner that will trigger new spam as they come in. Not only spam is a moving target, with ever changing new technics, but it's also very different on each installation.

So yes, you can find and download some rulesets, databases and so from many places on the Internet. It may work for some time. But it will never reflect your real traffic, and efficiently block spam. Worst, it will probably be far more sensible to false positives.

Concerning the auto-learn functionnality, it should only be used on a previously manually built database (more than 100'000 samples are advised), and only for some short time periods. Otherwise it may well corrupt your database. Moreover, as you noticed, it may become very cpu intensive (although this is a good sign of a bad database).


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Wed Jun 16, 2010 11:57 am 
Offline
User avatar

Joined: Mon Nov 13, 2006 2:30 pm
Posts: 293
Location: India
If it is a bad database, then how can I delete it and build a new database?

_________________
Promise only what you can deliver...


Top
 Profile  
 
 Post subject: Re: sa-learn consuming cpu resources
PostPosted: Wed Jun 16, 2010 1:00 pm 
Offline
Site Admin
User avatar

Joined: Thu Jan 01, 1970 1:00 am
Posts: 949
this is the same as for building a baysian database. You will find many information in the forum and on the SpamAssassin's website. I strongly suggest you hake a look at it. This is important.

for exemple: http://spamassassin.apache.org/full/3.3.x/doc/sa-learn.html


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group