100% Spams "whitelisted by system"

[meat1oaf]

Having a pretty weird problem - I have some mails that come in and are scoring pretty high, and are still being "whitelisted by the system". I have seen a few posts here in the forums that are similar, but say "white-listed by domain". I have had no luck searching for my specific error since your BB software seems to ignore the word "system".

While I do have the Trusted Sources module enabled, I don't have anything in my global whitelist as far as I can tell.

1) How can I check/edit my global whitelist?
2) This mail is marked as spam when I run is_spam. Does this app take into account the global and domain whitelists?

I am a bit as a loss - I would prefer not to turn off trusted sources, since it has saves us some hassle in the past.

Any ideas would be greatly appreciated.
-Jason

[del]

1: Have a look at Configuration->Anti-Spam. "Global Settings" is the global whitelist. You also can add/edit domain-whitelists in Configuration->Domains; Domain->Filtering ("Enable user's whitelist")
2: Go to Management->Tracing, select Domain and click refresh. Search for the mail, expand and post the logs here.

[meat1oaf]

Thanks for the quick response... to answer your questions:

1) I DO have some trusted domains - namely the other domains on this mailcleaner server. They are denoted as *domain.com

2) domain changed to protect the innocent

Code: Select all

Incoming MTA stage:    2013-04-27 06:50:51 1UW2iR-0006l1-G1 DKIM: d=boptin.com s=k1 c=relaxed/relaxed a=rsa-sha1 i=contact@boptin.com [verification succeeded]
2013-04-27 06:50:51 1UW2iR-0006l1-G1 <= bounce-34413-13549067314-joe=domain.com@boptin.com H=(smtp.boptin.com) [66.199.244.190] P=esmtp S=7869 id=0.0.13549067314.90947327965c32vslpo56391.0@boptin.com
2013-04-27 06:50:51 1UW2iR-0006l1-G1 => joe@domain.com R=filter_forward T=local_smtp S=8014 H=127.0.0.1 [127.0.0.1] C="250 OK id=1UW2iR-0006l5-K2"
2013-04-27 06:50:51 1UW2iR-0006l1-G1 Completed
Filtering MTA stage:    2013-04-27 06:50:51 1UW2iR-0006l5-K2 <= bounce-34413-13549067314-joe=domain.com@boptin.com H=(thinkdomain.com) [127.0.0.1] P=esmtp S=7869 id=0.0.13549067314.90947327965c32vslpo56391.0@boptin.com
Filtering Engine:    Apr 27 06:50:53 localhost MailScanner[23135]: <A> tag found in message 1UW2iR-0006l5-K2 from bounce-34413-13549067314-joe=domain.com@boptin.com
Apr 27 06:50:53 localhost MailScanner[23135]: HTML Img tag found in message 1UW2iR-0006l5-K2 from bounce-34413-13549067314-joe=domain.com@boptin.com
Apr 27 06:50:53 localhost MailScanner[23135]: NiceBayes result is not spam (75.7%%) for 1UW2iR-0006l5-K2
Apr 27 06:50:57 localhost MailScanner[23135]: UriRBLs result is spam (boptin.com:SPAMHAUSDBL) for 1UW2iR-0006l5-K2
Apr 27 06:50:57 localhost MailScanner[23135]: Message 1UW2iR-0006l5-K2 from 66.199.244.190 (bounce-34413-13549067314-joe=domain.com@boptin.com) to domain.com is spam, UriRBLs (boptin.com:SPAMHAUSDBL)
Apr 27 06:50:57 localhost MailScanner[23135]: Profiled SpamCheck for message 1UW2iR-0006l5-K2: (ClamSpam_Check:0.0172s) (NiceBayes_Check:0.0071s) (Prefilters:4.0591s) (SpamCacheCheck:0.0005s) (TrustedSources_Check:0.0278s) (UriRBLs_Check:4.0057s)
Apr 27 06:50:57 localhost MailScanner[23135]: Spam Actions: message 1UW2iR-0006l5-K2 actions are deliver
Apr 27 06:50:57 localhost MailScanner[23135]: Count updated for 1UW2iR-0006l5-K2
Apr 27 06:50:57 localhost MailScanner[23135]: Count updated in daemon for 1UW2iR-0006l5-K2
Outgoing stage:    2013-04-27 06:50:57 1UW2iR-0006l5-K2 => joe <joe@domain.com> R=filter_checkspam T=spam_store S=8173
2013-04-27 06:50:57 1UW2iR-0006l5-K2 Completed
Spam handling stage:    2013-04-27 06:50:59 (5) 626128: message 1UW2iR-0006l5-K2 ready to be delivered with new id: 1UW2iZ-0006lO-DN
2013-04-27 06:50:59 (5) 626128: message 1UW2iR-0006l5-K2 R:<joe@domain.com> S:<bounce-34413-13549067314-joe=domain.com@boptin.com> score: 4 status: is whitelisted (1)
Final outgoing stage:    2013-04-27 06:50:59 1UW2iZ-0006lO-DN DKIM: d=boptin.com s=k1 c=relaxed/relaxed a=rsa-sha1 i=contact@boptin.com [verification failed - body hash mismatch (body probably modified in transit)]
2013-04-27 06:50:59 1UW2iZ-0006lO-DN <= bounce-34413-13549067314-joe=domain.com@boptin.com H=(localhost.localdomain) [127.0.0.1] P=esmtp S=8441 id=0.0.13549067314.90947327965c32vslpo56391.0@boptin.com
2013-04-27 06:51:00 1UW2iZ-0006lO-DN => joe@domain.com R=filter_forward T=remote_smtp S=8673 H=X.X.231.50 [X.X.231.50] C="250 2.6.0 <0.0.13549067314.90947327965c32vslpo56391.0@boptin.com> [InternalId=12548] Queued mail for"
2013-04-27 06:51:00 1UW2iZ-0006lO-DN Completed

[del]

Hmmmmm....
Is NiceBayes decisive? Try without being decisive.
Is Spamc enabled and decisive?

Are you able to log in to the user menu? Have a look at his personal whitelist.

[olivier]

"whitelisted by the system" means that the sender address (or domain) is whitelist at the global level. (Configuration -> Anti-Spam -> Global settings).
"white-listed by domain" means that the sender address (or domain) is whitelist at the recipient's domain level. (Configuration -> Domains -> [domain] -> Filtering).

Nothing to do with TrustedSources or so. It is just a plain whitelisting. Even possibly with something like *@gmail.com or so...
Basically, this is why whitelists are bad. people tend to put all their known addresses in there and spammers just have to fake the sender address of their spam to have their content delivered.

[meat1oaf]

Now here is a thought I was hoping someone could help me gauge the validity of...

I currently have *domain.com in the whitelist for the system.
I also noticed in the headers I posted, the from address is: joe=domain.com@boptin.com

Is this what is triggering the white-listing? I would assume that my string would require the from to END in "domain.com" - since there is no wildcard character (*) AFTER domain.com. It does appear to be triggering the rule if the from only contains my string.

Is there a way I could rewrite this rule to eliminate this false negative? Maybe *@domain.com?

Oliver - I do appreciate your support - but whitelists are necessary. Even in our Enterprise server (not this one...) if we didn't have whitelists, we would have issues. I completely understand how it would be better to report them rather than whitelist them, but end users want immediate results. Bad practice? yes. Unavoidable at times due to customer expectations? Yes as well...
Thanks again!!!!

[olivier]

if you want to whitelist a domain, use the syntax *@domain.tld . Otherwise the expression will match anywhere in the string (this allows far more powerfull rules but requires a bit more attention).

Regarding the whitelist, no, it is possible to reach a workable degree of trust between MailCleaner and the users (even the most picky ones). But of course this requires a big effort at the first step to make good datasets and fix the very first mistakes. Then you have to be reactive on any error reported to make sure it won't happen again. All the tools are here and provided into MailCleaner, but yes, this requires some amount of work that could be "bypassed" in the short term by using "wrong" solutions like whitelists. On the long run, you'll have lost far more time, but you're right that this is really appealing.