Clamspam definitions via sanesecurity

Discuss here all what concerns the MailCleaner anti-spam efficiency, share your rulesets and tips for SpamAssassin !

Moderators: FlorianB, Pascal, bourgeois, mentor

Pentangle
Posts: 80
Joined: Thu Jan 31, 2013 1:20 am
How did you hear about Mailcleaner: Googling

Clamspam definitions via sanesecurity

Postby Pentangle » Wed Jan 13, 2016 3:47 pm

Hi all,

(if this is too much info to read, the question is at the very bottom!)

A while ago when setting up the sanesecurity downloads and updates of the spam definitions, I followed a process listed somewhere in this forum and set it up and it's been working fine for a while until recently, when I noticed that we had a lot of compromised word doc files come through.

I've worked out what was going on and noticed that "badmacro.ndb" wasn't in the /sanesecurity/clamav-unofficial-sigs.conf (it might have been an oversight when the guy was writing the scripts), so I added it in (which appeared straightforward).

I just received another word document with virus through, which according to sanesecurity is in today's badmacro.ndb definition file, so I went looking and found that the update process in crontab says:

Code: Select all

45 * * * * root /sanesecurity/clamav-unofficial-sigs.sh -c /etc/clamav-unofficial-sigs.conf >> /var/log/clamav-unofficial-sigs.log 2>&1


...which to my untrained eye looks like it references /etc/clamav-unofficial-sigs.conf rather than the version in /sanesecurity

This wouldn't be a problem if both configuration files looked like they behaved the same way or were both documented, but that's not the case. Hence the current sanesecurity version looks like:

Code: Select all

# This file contains user configuration settings for clamav-unofficial-sigs.sh
###################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
##################
#
# Script updates can be found at: https://github.com/extremeshok/clamav-unofficial-sigs
#
# Originially based on:
# Script provide by Bill Landry (unofficialsigs@gmail.com).
#
# License: BSD (Berkeley Software Distribution)
#
##################
#
# NOT COMPATIBLE WITH VERSION 3.XX CONFIG
#
################################################################################

# Edit the quoted variables below to meet your own particular needs
# and requirements, but do not remove the "quote" marks.

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/clamav/bin"
export PATH

# Set the appropriate ClamD user and group accounts for your system.
# If you do not want the script to set user and group permissions on
# files and directories, comment the next two variables.
#clam_user="clam"
clam_user="clamav"

#clam_group="clam"
clam_group="clamav"

# If you do not want the script to change the file mode of all signature
# database files in the ClamAV working directory to 0644 (-rw-r--r--):
#
# owner: read, write
# group: read
# world: read
#
# as defined in the "clam_dbs" path variable below, then set the following
# "setmode" variable to "no".
setmode="yes"

# Set path to ClamAV database files location.  If unsure, check
# your clamd.conf file for the "DatabaseDirectory" path setting.
clam_dbs="/var/mailcleaner/spool/clamspam"

# Set path to clamd.pid file (see clamd.conf for path location).
clamd_pid="/var/mailcleaner/run/clamav/clamspamd.pid"
#clamd_pid="/var/run/clamd.pid"

# To enable "ham" (non-spam) directory scanning and removal of
# signatures that trigger on ham messages, uncomment the following
# variable and set it to the appropriate ham message directory.
#ham_dir="/var/lib/clamav-unofficial-sigs/ham-test"

# If you would like to reload the clamd databases after an update,
# change the following variable to "yes".
reload_dbs="yes"

# Top level working directory, script will attempt to create them.
work_dir="/sanesecurity/cache"   #Top level working directory

# Log update information to '$log_file_path/$log_file_name'.
enable_logging="yes"
log_file_path="/var/log"
log_file_name="clamav-unofficial-sigs.log"


# =========================
# MalwarePatrol : https://www.malwarepatrol.net
# MalwarePatrol 2015 free clamav signatures
#
# 1. Sign up for a free account : https://www.malwarepatrol.net/signup-free.shtml
# 2. You will recieve an email containing your password/receipt number
# 3. Enter the receipt number into the config: replacing YOUR-RECEIPT-NUMBER with your receipt number from the email

malwarepatrol_receipt_code="YOUR-RECEIPT-NUMBER"

# =========================
# SecuriteInfo : https://www.SecuriteInfo.com
# SecuriteInfo 2015 free clamav signatures
#
# 1. Sign up for a free account : https://www.securiteinfo.com/clients/customers/signup
# 2. You will recieve an email to activate your account and then a followup email with your login name
# 3. Login and navigate to your customer account : https://www.securiteinfo.com/clients/customers/account
# 4. Click on the Installation tab
# 5. You will need to get your unique identifier from one of the download links for manual setup, you will need your 128 character authorisation signature.
#   eg. https://www.securiteinfo.com/get/signatures/bb9ktrbv30cagin7hajfa5ft6dwpvljsv7b7yob9igpv5175dew05rirq3r5kgnhbb9ktrbv30cagin7hajfa5ft6dwpvljsv7b7yob9igpv5175dew05rirq3r5kgnh/securiteinfo.hdb
#   Your 128 character authorisation signature would be : bb9ktrbv30cagin7hajfa5ft6dwpvljsv7b7yob9igpv5175dew05rirq3r5kgnhbb9ktrbv30cagin7hajfa5ft6dwpvljsv7b7yob9igpv5175dew05rirq3r5kgnh
# 6. Enter the authorisation signature into the config securiteinfo_authorisation_signature: replacing YOUR-SIGNATURE-NUMBER with your authorisation signature from the link

securiteinfo_authorisation_signature="YOUR-SIGNATURE-NUMBER"

# ========================
# Database provider update time
# ========================
# Since the database files are dynamically created, non default values can cause banning, change with caution

securiteinfo_update_hours="4"   # Default is 4 hours (6 downloads daily).
linuxmalwaredetect_update_hours="6"   # Default is 6 hours (4 downloads daily).
malwarepatrol_update_hours="24"   # Default is 24 hours (1 downloads daily).

# ========================
# Sanesecurity Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable usage of any of the Sanesecurity distributed database files
# shown, remove the database file name from the quoted section below.
# To disable usage of all Sanesecurity distributed databases, comment
# all of the quoted lines below.  Only databases defined as "low" risk
# have been enabled by default (for additional information about the
# database ratings, see: http://www.sanesecurity.com/clamav/databases.htm).
# Only add signature databases here that are "distributed" by Sanesecuirty
# as defined at the URL shown above.  Database distributed by others sources
# (e.g., SecuriteInfo & MalewarePatrol, can be added to other sections of
# this config file below).  Finally, make sure that the database names are
# spelled correctly or you will experience issues when the script runs
# (hint: all rsync servers will fail to download signature updates).

sanesecurity_dbs=" # BEGIN SANESECURITY DATABASE
### SANESECURITY http://sanesecurity.com/usage/signatures/
## REQUIRED, no not disable
sanesecurity.ftm  #REQUIRED Message file types, for best performance
sigwhitelist.ign2 #REQUIRED Fast update file to whitelist any problem signatures
## LOW
junk.ndb  #LOW  General high hitting junk, containing spam/phishing/lottery/jobs/419s etc
badmacro.ndb #MED doc and xls with macros
jurlbl.ndb #LOW Junk Url based
phish.ndb #LOW Phishing
rogue.hdb  #LOW Malware, Rogue anti-virus software and Fake codecs etc.  Updated hourly to cover the latest malware threats
scam.ndb #LOW Spam/scams
spamimg.hdb #LOW Spam images
spamattach.hdb #LOW Spam Spammed attachments such as pdf/doc/rtf/zip
blurl.ndb  #LOW Blacklisted full urls over the last 7 days, covering malware/spam/phishing. URLs added only when main signatures have failed to detect but are known to be "bad"
## MED
#spear.ndb  #MED Spear phishing email addresses (autogenerated from data here)
#lott.ndb  #MED Lottery
#spam.ldb  #MED Spam detected using the new Logical Signature type
#spearl.ndb  #MED Spear phishing urls (autogenerated from data here)
#jurlbla.ndb #MED Junk Url based autogenerated from various feeds

### FOXHOLE http://sanesecurity.com/foxhole-databases/
## LOW
malwarehash.hsb  #LOW Malware hashes without known Size
## MED
#foxhole_generic.cdb #MED See Foxhole page for more details
#foxhole_filename.cdb #MED See Foxhole page for more details
## HIGH
#foxhole_all.cdb  #HIGH See Foxhole page for more details

### OITC http://www.oitc.com/winnow/clamsigs/index.html
### Note: the two databases winnow_phish_complete.ndb and winnow_phish_complete_url.ndb should NOT be used together.
# LOW
winnow.attachments.hdb  #LOW Spammed attachments such as pdf/doc/rtf/zip
winnow_malware.hdb  #LOW Current virus, trojan and other malware not yet detected by ClamAV.
winnow_malware_links.ndb #LOW Links to malware
winnow_extended_malware.hdb  #LOW contain hand generated signatures for malware
winnow_bad_cw.hdb #LOW md5 hashes of malware attachments acquired directly from a group of botnets
# MED
#winnow_phish_complete_url.ndb #Med Similar to winnow_phish_complete.ndb except that entire urls are used
#winnow.complex.patterns.ldb  #MED contain hand generated signatures for malware and some egregious fraud
#winnow_extended_malware_links.ndb #MED contain hand generated signatures for malware links
#winnow_spam_complete.ndb  #MED Signatures to detect fraud and other malicious spam
# HIGH
#winnow_phish_complete.ndb #HIGH Phishing and other malicious urls and compromised hosts **DO NOT USE WITH winnow_phish_complete_url**

### SCAMNAILER http://www.scamnailer.info/
# MED
scamnailer.ndb  #MED Spear phishing and other phishing emails

### BOFHLAND http://clamav.bofhland.org/
# LOW
bofhland_cracked_URL.ndb  #LOW Spam URLs
bofhland_malware_URL.ndb  #LOW Malware URLs
bofhland_phishing_URL.ndb #LOW Phishing URLs
bofhland_malware_attach.hdb #LOW Malware Hashes

### CRDF https://threatcenter.crdf.fr/
# LOW
crdfam.clamav.hdb #LOW List of new threats detected by CRDF Anti Malware

### Porcupine
# LOW
porcupine.ndb  #LOW Brazilian e-mail phishing and malware signatures
phishtank.ndb  #LOW Online and valid phishing urls from phishtank.com data feed
porcupine.hsb #LOW SHA256 hashes of VBS and JSE kept for 7 days
" # END SANESECURITY DATABASES

# ========================
# SecuriteInfo Database(s)
# ========================
# Only active when you set your securiteinfo_authorisation_signature
# Add or remove database file names between quote marks as needed.  To
# disable any SecuriteInfo database downloads, remove the appropriate
# lines below.  To disable all SecuriteInfo database file downloads,
# comment all of the following lines.
securiteinfo_dbs="
### Securiteinfo http://www.securiteinfo.com/services/clamav_unofficial_malwares_signatures.shtml
## REQUIRED, no not disable
securiteinfo.ign2
# LOW
securiteinfo.hdb #LOW Malwares in the Wild
javascript.ndb  #LOW Malwares Javascript
securiteinfohtml.hdb  #LOW Malwares HTML
securiteinfoascii.hdb  #LOWe Text file malwares (Perl or shell scripts, bat files, exploits, ...)
securiteinfopdf.hdb #LOW Malwares PDF
# HIGH
#spam_marketing.ndb #HIGH Spam Marketing /  spammer blacklist
# PREMIUM
#spam_marketing_agressif.ndb #HIGH++ Spam Marketing aggressive
# UNKNOWN IF THESE ARE AVAILABLE
#honeynet.hdb  #LOW Old malwares not detected
#securiteinfoelf.hdb #LOW Malwares ELF (Linux executables)
#securiteinfosh.hdb #LOW Malwares SHELL (Linux)
#securiteinfooffice.hdb #LOW Malwares Macros Office
#securiteinfodos.hdb #LOW Malwares MS-DOS
#securiteinfobat.hdb #LOW Malwares BAT
" #END SECURITEINFO DATABASES

# ========================
# Linux Malware Detect Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable any SecuriteInfo database downloads, remove the appropriate
# lines below.  To disable all Linux Malware Detect database file downloads,
# comment all of the following lines.
linuxmalwaredetect_dbs="
### Linux Malware Detect https://www.rfxn.com/projects/linux-malware-detect/
# LOW
rfxn.ndb #LOW HEX Malware detection signatures
rfxn.hdb  #LOW MD5 malware detection signatures
" #END LINUXMALWAREDETECT DATABASES

# =========================
# MalwarePatrol Database
# =========================
# Only active when you set your malwarepatrol_receipt_code
## REQUIRED, no not disable
malwarepatrol_db="malwarepatrol.db" #LOW URLs containing of Viruses, Trojans, Worms, or Malware

# =========================
# Additional signature databases
# =========================
# Additional signature databases can be specified here in the following
# format: PROTOCOL://URL-or-IP/PATH/TO/FILE-NAME (use a trailing "/" in
# place of the "FILE-NAME" to download all files from specified location,
# but this *ONLY* works for files downloaded via rsync).  For non-rsync
# downloads, curl is used.  For download protocols supported by curl, see
# "man curl".  This also works well for locations that have many ClamAV
# servers that use 3rd party signature databases, as only one server need
# download the remote databases, and all others can update from the local
# mirrors copy.  See format examples below.  To use, remove the comments
# and examples shown and add your own sites between the quote marks.
#add_dbs="
#   rsync://192.168.1.50/new-db/sigs.hdb
#   rsync://rsync.example.com/all-dbs/
#   ftp://ftp.example.net/pub/sigs.ndb
#   http://www.example.org/sigs.ldb
#" #END ADDITIONAL DATABASES


# ==================================================
# ==================================================
# A D V A N C E D   O P T I O N S
# ==================================================
# ==================================================

# Enable or disable download time randomization.  This allows the script to
# be executed via cron, but the actual database file checking will pause
# for a random number of seconds between the "min" and "max" time settings
# specified below.  This helps to more evenly distribute load on the host
# download sites.  To disable, set the following variable to "no".
enable_random="yes"

# If download time randomization is enabled above (enable_random="yes"),
# then set the min and max radomization time intervals (in seconds).
min_sleep_time="60"    # Default minimum is 60 seconds (1 minute).
max_sleep_time="600"   # Default maximum is 600 seconds (10 minutes).

# Set the reload or restart option if the "reload_dbs" variable above
# is set to "yes" (only select ONE of the following variables or the
# last uncommented variable option will be the one used).
# - The next variable signals clamd daemon to reload databases (this is the recommended reload option)
reload_opt="/usr/mailcleaner/etc/init.d/clamspamd restart"  # Default
# - The next variable signals clamds Process ID (PID) to reload databases
#reload_opt="kill -USR2 `cat $clamd_pid`"
# - The next variable signals linux based systems to do a full clamd service stop/start
#reload_opt="service clamd restart"
# - The next variable signals linux based systems to do a full clamd service reload
#reload_opt="service clamd reload"
# - Use the next variable to set a custom or system specific reload/restart option
#reload_opt=""

# If running clamd in "LocalSocket" mode (*NOT* in TCP/IP mode), and
# either "SOcket Cat" (socat) or the "IO::Socket::UNIX" perl module
# are installed on the system, and you want to report whether clamd
# is running or not, uncomment the "clamd_socket" variable below (you
# will be warned if neither socat nor IO::Socket::UNIX are found, but
# the script will still run).  You will also need to set the correct
# path to your clamd socket file (if unsure of the path, check the
# "LocalSocket" setting in your clamd.conf file for socket location).
#clamd_socket="/tmp/clamd.socket"
clamd_socket="/var/mailcleaner/run/clamav/clamspamd.sock"

# If you would like to attempt to restart ClamD if detected not running,
# uncomment the next 2 lines.  Confirm the path to the "clamd_lock" file
# (usually can be found in the clamd init script) and also enter the clamd
# start command for your particular distro for the "start_clamd" variable
# (the sample start command shown below should work for most linux distros).
# NOTE: these 2 variables are dependant on the "clamd_socket" variable
# shown above - if not enabled, then the following 2 variables will be
# ignored, whether enabled or not.
#clamd_lock="/var/lock/subsys/clamd"
#start_clamd="service clamd start"

# Set rsync connection and data transfer timeout limits in seconds.
# The defaults settings here are reasonable, only change if you are
# experiencing timeout issues.
rsync_connect_timeout="30"
rsync_max_time="90"

# Set curl connection and data transfer timeout limits in seconds.
# The defaults settings here are reasonable, only change if you are
# experiencing timeout issues.
curl_connect_timeout="30"
curl_max_time="90"

# Set working directory paths (edit to meet your own needs). If these
# directories do not exist, the script will attempt to create them.
# Sub-directory names:
sanesecurity_dir="$work_dir/dbs-ss"        # Sanesecurity sub-directory
securiteinfo_dir="$work_dir/dbs-si"        # SecuriteInfo sub-directory
linuxmalwaredetect_dir="$work_dir/dbs-lmd"      # Linux Malware Detect sub-directory
malwarepatrol_dir="$work_dir/dbs-mbl"      # MalwarePatrol sub-directory
config_dir="$work_dir/configs"   # Script configs sub-directory
gpg_dir="$work_dir/gpg-key"      # Sanesecurity GPG Key sub-directory
add_dir="$work_dir/dbs-add"      # User defined databases sub-directory

# If you would like to make a backup copy of the current running database
# file before updating, leave the following variable set to "yes" and a
# backup copy of the file will be created in the production directory
# with -bak appended to the file name.
keep_db_backup="no"

# If you want to silence the information reported by curl, rsync, gpg
# or the general script comments, change the following variables to
# "yes".  If all variables are set to "yes", the script will output
# nothing except error conditions.
silence_ssl="yes" # Default is "yes" ignore ssl errors and warnings
curl_silence="no"      # Default is "no" to report curl statistics
rsync_silence="no"     # Default is "no" to report rsync statistics
gpg_silence="no"       # Default is "no" to report gpg signature status
comment_silence="no"   # Default is "no" to report script comments

# If necessary to proxy database downloads, define the rsync and/or curl
# proxy settings here.  For rsync, the proxy must support connections to
# port 873.  Both curl and rsync proxy setting need to be defined in the
# format of "hostname:port".  For curl, also note the -x and -U flags,
# which must be set as "-x hostname:port" and "-U username:password".
rsync_proxy=""
curl_proxy=""

# After you have completed the configuration of this file, set the
user_configuration_complete="yes"

# ========================
# Database provider URLs, do not edit.
sanesecurity_url="rsync.sanesecurity.net"
sanesecurity_gpg_url="http://www.sanesecurity.net/publickey.gpg"
securiteinfo_url="https://www.securiteinfo.com/get/signatures/"
linuxmalwaredetect_url="http://cdn.rfxn.com/downloads/"
malwarepatrol_url="https://lists.malwarepatrol.net/cgi/getfile?product=8&list=clamav_basic"

# ========================
# do not edit
configuration_version="4.6"

# https://eXtremeSHOK.com ##############################################################
root@mailcleaner:/sanesecurity#


and the version in /etc looks like:

Code: Select all

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/clamav/bin"
export PATH
clam_user="clamav"
clam_group="clamav"
setmode="yes"
clam_dbs="/var/mailcleaner/spool/clamspam"
clamd_pid="/var/mailcleaner/run/clamav/clamspamd.pid"
reload_dbs="yes"
reload_opt="/usr/mailcleaner/etc/init.d/clamspamd restart"
clamd_socket="/var/mailcleaner/run/clamav/clamspamd.sock"
enable_random="yes"
min_sleep_time="60" # Default minimum is 60 seconds (1 minute).
max_sleep_time="600" # Default maximum is 600 seconds (10 minutes).
ss_dbs="
blurl.ndb
junk.ndb
jurlbl.ndb
phish.ndb
rogue.hdb
sanesecurity.ftm
scam.ndb
sigwhitelist.ign2
spamattach.hdb
spamimg.hdb
winnow.attachments.hdb
winnow_bad_cw.hdb
winnow_extended_malware.hdb
winnow_malware.hdb
winnow_malware_links.ndb
doppelstern.hdb
bofhland_cracked_URL.ndb
bofhland_malware_attach.hdb
bofhland_malware_URL.ndb
bofhland_phishing_URL.ndb
crdfam.clamav.hdb
phishtank.ndb
porcupine.ndb
"
si_dbs="
honeynet.hdb
securiteinfo.hdb
securiteinfobat.hdb
securiteinfodos.hdb
securiteinfoelf.hdb
securiteinfohtml.hdb
securiteinfooffice.hdb
securiteinfopdf.hdb
securiteinfosh.hdb
"
si_update_hours="4" # Default is 4 hours (6 update checks daily).
mbl_dbs="
mbl.ndb
"
mbl_update_hours="6" # Default is 6 hours (4 downloads daily).
rsync_connect_timeout="15"
rsync_max_time="60"
curl_connect_timeout="15"
curl_max_time="90"
work_dir="/sanesecurity/cache"
ss_dir="$work_dir/ss-dbs" # Sanesecurity sub-directory
si_dir="$work_dir/si-dbs" # SecuriteInfo sub-directory
mbl_dir="$work_dir/mbl-dbs" # MalwarePatrol sub-directory
config_dir="$work_dir/configs" # Script configs sub-directory
gpg_dir="$work_dir/gpg-key" # Sanesecurity GPG Key sub-directory
add_dir="$work_dir/add-dbs" # User defined databases sub-directory
keep_db_backup="no"
curl_silence="no" # Default is "no" to report curl statistics
rsync_silence="no" # Default is "no" to report rsync statistics
gpg_silence="no" # Default is "no" to report gpg signature status
comment_silence="no" # Default is "no" to report script comments
enable_logging="no"
log_file_path="/var/log"
log_file_name="clamav-unofficial-sigs.log"
rsync_proxy=""
curl_proxy=""
user_configuration_complete="yes"
root@mailcleaner:/etc#


So as you can see i've added the badmacro.ndb into the /sanesecurity one but am unsure if I can just add it without any other options or other code etc into the /etc one??
vid99
Posts: 3
Joined: Thu Jan 07, 2016 9:30 pm
How did you hear about Mailcleaner: http://www.happymac.info/cms/knowledge-base/102-ma

Re: Clamspam definitions via sanesecurity

Postby vid99 » Thu Jan 14, 2016 3:14 am

oh, an edit...Your first file is the .sh file and the second one is .conf file. I leave the .sh file alone, make changes to the .conf file in the sanesecurity folder and then copy it to /etc.

In the directions here:
viewtopic.php?f=3&t=2015

The one line is:
cp /sanesecurity/clamav-unofficial-sigs.conf /etc/

I made this mistake as well, updated some things in the sanesecurity folder, but then forgot to re-run that command, so my updates didn't take affect.

Think you just need to redo that cp command.
Pentangle
Posts: 80
Joined: Thu Jan 31, 2013 1:20 am
How did you hear about Mailcleaner: Googling

Re: Clamspam definitions via sanesecurity

Postby Pentangle » Fri Jan 15, 2016 1:23 pm

Thanks! that's exactly what I needed to know :)
Pentangle
Posts: 80
Joined: Thu Jan 31, 2013 1:20 am
How did you hear about Mailcleaner: Googling

Re: Clamspam definitions via sanesecurity

Postby Pentangle » Sat Jan 16, 2016 4:37 pm

Just to follow up on this, whilst vid99 helped me to get to the solution he wasn't fully correct. Actually BOTH files i'd pasted were conf files, but whilst the lower one was from the "quickstart" guide that he linked to, the upper one was an original extremeSHOK updater from another guide i'd checked here.

What's more important though, is that when I'd worked all this out and worked out that the clamav log file is at /var/log/clamav-unofficial-sigs.log

I checked this logfile out and saw that for the previous few months i'd been getting the error:

**Your configuration version is not compatible with this version** (and hence my spam filtering was les than optimal since then)

So i've spent this saturday working out precisely what was needed, which appeared to be a new version of the ExtremeSHOK updater scripts, found at: https://github.com/extremeshok/clamav-unofficial-sigs

There are a few variables needing editing in the .conf file, but none in the .sh or logrotate files, and I think that's all that's needed to grab again. Edit the variables to match the mailcleaner requirements, and run the updater and BOOM, down it loads and I now get the newer definitions (and hopefully happier customers with less spam!)

This all stemmed from the Sanesecurity Blog at: http://sanesecurity.blogspot.co.uk/ which has been listing without fail almost ALL my niggling spam on a daily basis, and it appears that badmacro.ndb is the definitions file which the old script didn't include.

Hence - my conclusion would be that ensuring you include badmacro.ndb into your definitions mix will really do a great deal towards fighting the spam that otherwise gets through Mailcleaner!

Hope this helps.
Mike.
florima
Posts: 3
Joined: Fri Nov 11, 2016 7:41 pm
How did you hear about Mailcleaner: web

Re: Clamspam definitions via sanesecurity

Postby florima » Sat Nov 12, 2016 2:46 am

Pentangle wrote:Just to follow up on this, whilst vid99 helped me to get to the solution he wasn't fully correct. Actually BOTH files i'd pasted were conf files, but whilst the lower one was from the "quickstart" guide that he linked to, the upper one was an original extremeSHOK updater from another guide i'd checked here.

What's more important though, is that when I'd worked all this out and worked out that the clamav log file is at /var/log/clamav-unofficial-sigs.log

I checked this logfile out and saw that for the previous few months i'd been getting the error:

**Your configuration version is not compatible with this version** (and hence my spam filtering was les than optimal since then)

So i've spent this saturday working out precisely what was needed, which appeared to be a new version of the ExtremeSHOK updater scripts, found at: https://github.com/extremeshok/clamav-unofficial-sigs

There are a few variables needing editing in the .conf file, but none in the .sh or logrotate files, and I think that's all that's needed to grab again. Edit the variables to match the mailcleaner requirements, and run the updater and BOOM, down it loads and I now get the newer definitions (and hopefully happier customers with less spam!)

This all stemmed from the Sanesecurity Blog at: http://sanesecurity.blogspot.co.uk/ which has been listing without fail almost ALL my niggling spam on a daily basis, and it appears that badmacro.ndb is the definitions file which the old script didn't include.

Hence - my conclusion would be that ensuring you include badmacro.ndb into your definitions mix will really do a great deal towards fighting the spam that otherwise gets through Mailcleaner!

Hope this helps.
Mike.


mike,

can you post your modified files or one min how to change to update sanesecurity because im trying to make it work but everytime im stack with errors like this one
Loading config: /sanesecurity/clamav-unofficial-sigs.conf
=========================================================
===================================================
ERROR: Config file/s are missing important contents
===================================================
Note: Possible fix would be to point the script to the dir with the configs


thanks for help
Pentangle
Posts: 80
Joined: Thu Jan 31, 2013 1:20 am
How did you hear about Mailcleaner: Googling

Re: Clamspam definitions via sanesecurity

Postby Pentangle » Thu Nov 24, 2016 1:01 am

Sorry I didn't see this until now. Here you go. This is the .conf file

Code: Select all

# This file contains user configuration settings for clamav-unofficial-sigs.sh
###################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
##################
#
# Script updates can be found at: https://github.com/extremeshok/clamav-unofficial-sigs
#
# Originially based on:
# Script provide by Bill Landry (unofficialsigs@gmail.com).
#
# License: BSD (Berkeley Software Distribution)
#
##################
#
# NOT COMPATIBLE WITH VERSION 3.XX CONFIG
#
################################################################################

# Edit the quoted variables below to meet your own particular needs
# and requirements, but do not remove the "quote" marks.

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/clamav/bin"
export PATH

# Set the appropriate ClamD user and group accounts for your system.
# If you do not want the script to set user and group permissions on
# files and directories, comment the next two variables.

clam_user="clamav"

clam_group="clamav"

# If you do not want the script to change the file mode of all signature
# database files in the ClamAV working directory to 0644 (-rw-r--r--):
#
# owner: read, write
# group: read
# world: read
#
# as defined in the "clam_dbs" path variable below, then set the following
# "setmode" variable to "no".
setmode="yes"

# Set path to ClamAV database files location.  If unsure, check
# your clamd.conf file for the "DatabaseDirectory" path setting.
clam_dbs="/var/mailcleaner/spool/clamspam"

# Set path to clamd.pid file (see clamd.conf for path location).
clamd_pid="/var/mailcleaner/run/clamav/clamspamd.pid"

# To enable "ham" (non-spam) directory scanning and removal of
# signatures that trigger on ham messages, uncomment the following
# variable and set it to the appropriate ham message directory.
#ham_dir="/var/lib/clamav-unofficial-sigs/ham-test"

# If you would like to reload the clamd databases after an update,
# change the following variable to "yes".
reload_dbs="yes"
reload_opt="/usr/mailcleaner/etc/init.d/clamspamd restart"

# Top level working directory, script will attempt to create them.
work_dir="/sanesecurity/cache"   #Top level working directory

# Log update information to '$log_file_path/$log_file_name'.
enable_logging="no"
log_file_path="/var/log"
log_file_name="clamav-unofficial-sigs.log"


# =========================
# MalwarePatrol : https://www.malwarepatrol.net
# MalwarePatrol 2015 free clamav signatures
#
# 1. Sign up for a free account : https://www.malwarepatrol.net/signup-free.shtml
# 2. You will recieve an email containing your password/receipt number
# 3. Enter the receipt number into the config: replacing YOUR-RECEIPT-NUMBER with your receipt number from the email

malwarepatrol_receipt_code="f1452952545"
# Set to no to enable the commercial subscription url.
malwarepatrol_free="yes"

# =========================
# SecuriteInfo : https://www.SecuriteInfo.com
# SecuriteInfo 2015 free clamav signatures
#
#Usage of SecuriteInfo 2015 free clamav signatures : https://www.securiteinfo.com
# - 1. Sign up for a free account : https://www.securiteinfo.com/clients/customers/signup
# - 2. You will recieve an email to activate your account and then a followup email with your login name
# - 3. Login and navigate to your customer account : https://www.securiteinfo.com/clients/customers/account
# - 4. Click on the Setup tab
# - 5. You will need to get your unique identifier from one of the download links, they are individual for every user
# - 5.1. The 128 character string is after the http://www.securiteinfo.com/get/signatures/
# - 5.2. Example https://www.securiteinfo.com/get/signatures/your_unique_and_very_long_random_string_of_characters/securiteinfo.hdb
#   Your 128 character authorisation signature would be : your_unique_and_very_long_random_string_of_characters
# - 6. Enter the authorisation signature into the config securiteinfo_authorisation_signature: replacing YOUR-SIGNATURE-NUMBER with your authorisation signature from the link

securiteinfo_authorisation_signature="94e2a532f4acdf12bf817cbff1e1823df26936f5bbb775c41849f2bd4b375b37212427d976799cd2fc8526f36cd87c359c6fa5298cc6db54314d871c511c9e52"

# ========================
# Database provider update time
# ========================
# Since the database files are dynamically created, non default values can cause banning, change with caution

securiteinfo_update_hours="4"   # Default is 4 hours (6 downloads daily).
linuxmalwaredetect_update_hours="6"   # Default is 6 hours (4 downloads daily).
malwarepatrol_update_hours="24"   # Default is 24 hours (1 downloads daily).
yararules_update_hours="24"   # Default is 24 hours (1 downloads daily).

# ========================
# Enabled Databases
# ========================
# Set to no to disable an entire database.
sanesecurity_enabled="yes"   # Sanesecurity
securiteinfo_enabled="yes"   # SecuriteInfo
linuxmalwaredetect_enabled="yes"   # Linux Malware Detect
malwarepatrol_enabled="yes"   # Malware Patrol
yararules_enabled="no"   # Yara-Rule Project, requires clamAV 0.99+

# ========================
# Sanesecurity Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable usage of any of the Sanesecurity distributed database files
# shown, remove the database file name from the quoted section below.
# Only databases defined as "low" risk have been enabled by default
# for additional information about the database ratings, see:
# http://www.sanesecurity.com/clamav/databases.htm
# Only add signature databases here that are "distributed" by Sanesecuirty
# as defined at the URL shown above.  Database distributed by others sources
# (e.g., SecuriteInfo & MalewarePatrol, can be added to other sections of
# this config file below).  Finally, make sure that the database names are
# spelled correctly or you will experience issues when the script runs
# (hint: all rsync servers will fail to download signature updates).

sanesecurity_dbs=" # BEGIN SANESECURITY DATABASE
### SANESECURITY http://sanesecurity.com/usage/signatures/
## REQUIRED, Do NOT disable
sanesecurity.ftm  #REQUIRED Message file types, for best performance
sigwhitelist.ign2 #REQUIRED Fast update file to whitelist any problem signatures
## LOW
junk.ndb  #LOW  General high hitting junk, containing spam/phishing/lottery/jobs/419s etc
jurlbl.ndb #LOW Junk Url based
phish.ndb #LOW Phishing
rogue.hdb  #LOW Malware, Rogue anti-virus software and Fake codecs etc.  Updated hourly to cover the latest malware threats 
scam.ndb #LOW Spam/scams 
spamimg.hdb #LOW Spam images
spamattach.hdb #LOW Spam Spammed attachments such as pdf/doc/rtf/zip
blurl.ndb  #LOW Blacklisted full urls over the last 7 days, covering malware/spam/phishing. URLs added only when main signatures have failed to detect but are known to be "bad" 
## MED
spear.ndb  #MED Spear phishing email addresses (autogenerated from data here)
lott.ndb  #MED Lottery 
spam.ldb  #MED Spam detected using the new Logical Signature type
spearl.ndb  #MED Spear phishing urls (autogenerated from data here)   
jurlbla.ndb #MED Junk Url based autogenerated from various feeds
badmacro.ndb #MED Detect dangerous macros

### FOXHOLE http://sanesecurity.com/foxhole-databases/
## LOW
malwarehash.hsb  #LOW Malware hashes without known Size
## MED
#foxhole_generic.cdb #MED See Foxhole page for more details
#foxhole_filename.cdb #MED See Foxhole page for more details
## HIGH
#foxhole_all.cdb  #HIGH See Foxhole page for more details 

### OITC http://www.oitc.com/winnow/clamsigs/index.html
### Note: the two databases winnow_phish_complete.ndb and winnow_phish_complete_url.ndb should NOT be used together. 
# LOW
winnow.attachments.hdb  #LOW Spammed attachments such as pdf/doc/rtf/zip
winnow_malware.hdb  #LOW Current virus, trojan and other malware not yet detected by ClamAV.
winnow_malware_links.ndb #LOW Links to malware
winnow_extended_malware.hdb  #LOW contain hand generated signatures for malware
winnow_bad_cw.hdb #LOW md5 hashes of malware attachments acquired directly from a group of botnets
# MED
winnow_phish_complete_url.ndb #Med Similar to winnow_phish_complete.ndb except that entire urls are used 
winnow.complex.patterns.ldb  #MED contain hand generated signatures for malware and some egregious fraud 
winnow_extended_malware_links.ndb #MED contain hand generated signatures for malware links
winnow_spam_complete.ndb  #MED Signatures to detect fraud and other malicious spam
# HIGH
#winnow_phish_complete.ndb #HIGH Phishing and other malicious urls and compromised hosts **DO NOT USE WITH winnow_phish_complete_url**

### SCAMNAILER http://www.scamnailer.info/
# MED
scamnailer.ndb  #MED Spear phishing and other phishing emails

### BOFHLAND http://clamav.bofhland.org/
# LOW
bofhland_cracked_URL.ndb  #LOW Spam URLs 
bofhland_malware_URL.ndb  #LOW Malware URLs
bofhland_phishing_URL.ndb #LOW Phishing URLs
bofhland_malware_attach.hdb #LOW Malware Hashes

###  RockSecurity http://rooksecurity.com/
#LOW
hackingteam.hsb #LOW Hacking Team hashes

### CRDF https://threatcenter.crdf.fr/
# LOW
crdfam.clamav.hdb #LOW List of new threats detected by CRDF Anti Malware 

### Porcupine
# LOW
porcupine.ndb  #LOW Brazilian e-mail phishing and malware signatures
phishtank.ndb  #LOW Online and valid phishing urls from phishtank.com data feed
porcupine.hsb  #LOW Sha256 Hashes of VBS and JSE malware, kept for 7 days

### Sanesecurity YARA Format rules
### Note: Yara signatures require ClamAV 0.99 or newer to work
#Sanesecurity_sigtest.yara #LOW Sanesecurity test signatures
#Sanesecurity_spam.yara #LOW detect spam

" # END SANESECURITY DATABASES

# ========================
# SecuriteInfo Database(s)
# ========================
# Only active when you set your securiteinfo_authorisation_signature
# Add or remove database file names between quote marks as needed.  To
# disable any SecuriteInfo database downloads, remove the appropriate
# lines below.
securiteinfo_dbs="
### Securiteinfo https://www.securiteinfo.com/services/improve-detection-rate-of-zero-day-malwares-for-clamav.shtml
## REQUIRED, Do NOT disable
securiteinfo.ign2
# LOW
securiteinfo.hdb #LOW Malwares in the Wild
javascript.ndb  #LOW Malwares Javascript
securiteinfohtml.hdb  #LOW Malwares HTML
securiteinfoascii.hdb  #LOW Text file malwares (Perl or shell scripts, bat files, exploits, ...)
securiteinfopdf.hdb #LOW Malwares PDF
# HIGH
#spam_marketing.ndb #HIGH Spam Marketing /  spammer blacklist
" #END SECURITEINFO DATABASES

# ========================
# Linux Malware Detect Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable any SecuriteInfo database downloads, remove the appropriate
# lines below.
linuxmalwaredetect_dbs="
### Linux Malware Detect https://www.rfxn.com/projects/linux-malware-detect/
# LOW
rfxn.ndb #LOW HEX Malware detection signatures
rfxn.hdb  #LOW MD5 malware detection signatures
" #END LINUXMALWAREDETECT DATABASES

# =========================
# MalwarePatrol Database
# =========================
# Only active when you set your malwarepatrol_receipt_code
## REQUIRED, Do NOT disable
malwarepatrol_db="malwarepatrol.db" #LOW URLs containing of Viruses, Trojans, Worms, or Malware 

# ========================
# Yara Rules Project Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable any Yara Rule database downloads, remove the appropriate
# lines below.
yararules_dbs="
### Yara Rules https://github.com/Yara-Rules/rules
# LOW
antidebug.yar #LOW anti debug and anti virtualization techniques used by malware
malicious_document.yar #LOW documents with malicious code
# MED
#packer.yar #MED well-known sofware packers
# HIGH
#crypto.yar #HIGH detect the existence of cryptographic algoritms
" #END YARARULES DATABASES


# =========================
# Additional signature databases
# =========================
# Additional signature databases can be specified here in the following
# format: PROTOCOL://URL-or-IP/PATH/TO/FILE-NAME (use a trailing "/" in
# place of the "FILE-NAME" to download all files from specified location,
# but this *ONLY* works for files downloaded via rsync).  For non-rsync
# downloads, curl is used.  For download protocols supported by curl, see
# "man curl".  This also works well for locations that have many ClamAV
# servers that use 3rd party signature databases, as only one server need
# download the remote databases, and all others can update from the local
# mirrors copy.  See format examples below.  To use, remove the comments
# and examples shown and add your own sites between the quote marks.
#add_dbs="
#   rsync://192.168.1.50/new-db/sigs.hdb
#   rsync://rsync.example.com/all-dbs/
#   ftp://ftp.example.net/pub/sigs.ndb
#   http://www.example.org/sigs.ldb
#" #END ADDITIONAL DATABASES




# ==================================================
# ==================================================
# A D V A N C E D   O P T I O N S
# ==================================================
# ==================================================

# Enable or disable download time randomization.  This allows the script to
# be executed via cron, but the actual database file checking will pause
# for a random number of seconds between the "min" and "max" time settings
# specified below.  This helps to more evenly distribute load on the host
# download sites.  To disable, set the following variable to "no".
enable_random="yes"

# If download time randomization is enabled above (enable_random="yes"),
# then set the min and max radomization time intervals (in seconds).
min_sleep_time="60"    # Default minimum is 60 seconds (1 minute).
max_sleep_time="600"   # Default maximum is 600 seconds (10 minutes).

# Set the clamd_restart_opt if the "reload_dbs" variable above is set
# Command to do a full clamd service stop/start
# clamd_restart_opt="service clamd restart"

# If running clamd in "LocalSocket" mode (*NOT* in TCP/IP mode), and
# either "SOcket Cat" (socat) or the "IO::Socket::UNIX" perl module
# are installed on the system, and you want to report whether clamd
# is running or not, uncomment the "clamd_socket" variable below (you
# will be warned if neither socat nor IO::Socket::UNIX are found, but
# the script will still run).  You will also need to set the correct
# path to your clamd socket file (if unsure of the path, check the
# "LocalSocket" setting in your clamd.conf file for socket location).
#clamd_socket="/tmp/clamd.socket"
clamd_socket="/var/mailcleaner/run/clamav/clamspamd.sock"

# If you would like to attempt to restart ClamD if detected not running,
# uncomment the next 2 lines.  Enter the clamd service stop and  start command
# for your particular distro for the "start_clamd" "stop_clamd" variables
# (the sample start command shown below should work for most linux distros).
# NOTE: these 2 variables are dependant on the "clamd_socket" variable
# shown above - if not enabled, then the following 2 variables will be
# ignored, whether enabled or not.
#clamd_start="service clamd start"
#clamd_stop="service clamd stop"

# Set rsync connection and data transfer timeout limits in seconds.
# The defaults settings here are reasonable, only change if you are
# experiencing timeout issues.
rsync_connect_timeout="30"
rsync_max_time="90"

# Set curl connection and data transfer timeout limits in seconds.
# The defaults settings here are reasonable, only change if you are
# experiencing timeout issues.
curl_connect_timeout="30"
curl_max_time="90"

# Set working directory paths (edit to meet your own needs). If these
# directories do not exist, the script will attempt to create them.
# Sub-directory names:
sanesecurity_dir="$work_dir/dbs-ss"        # Sanesecurity sub-directory
securiteinfo_dir="$work_dir/dbs-si"        # SecuriteInfo sub-directory
linuxmalwaredetect_dir="$work_dir/dbs-lmd"      # Linux Malware Detect sub-directory
malwarepatrol_dir="$work_dir/dbs-mbl"      # MalwarePatrol sub-directory
yararules_dir="$work_dir/dbs-yara"      # Yara-Rules sub-directory
config_dir="$work_dir/configs"   # Script configs sub-directory
gpg_dir="$work_dir/gpg-key"      # Sanesecurity GPG Key sub-directory
add_dir="$work_dir/dbs-add"      # User defined databases sub-directory

# If you would like to make a backup copy of the current running database
# file before updating, leave the following variable set to "yes" and a
# backup copy of the file will be created in the production directory
# with -bak appended to the file name.
keep_db_backup="no"

# If you want to silence the information reported by curl, rsync, gpg
# or the general script comments, change the following variables to
# "yes".  If all variables are set to "yes", the script will output
# nothing except error conditions.
silence_ssl="yes" # Default is "yes" ignore ssl errors and warnings
curl_silence="no"      # Default is "no" to report curl statistics
rsync_silence="no"     # Default is "no" to report rsync statistics
gpg_silence="no"       # Default is "no" to report gpg signature status
comment_silence="no"   # Default is "no" to report script comments

# If necessary to proxy database downloads, define the rsync and/or curl
# proxy settings here.  For rsync, the proxy must support connections to
# port 873.  Both curl and rsync proxy setting need to be defined in the
# format of "hostname:port".  For curl, also note the -x and -U flags,
# which must be set as "-x hostname:port" and "-U username:password".
rsync_proxy=""
curl_proxy=""

# After you have completed the configuration of this file, set the
user_configuration_complete="yes"

# ========================
# Database provider URLs, do not edit.
sanesecurity_url="rsync.sanesecurity.net"
sanesecurity_gpg_url="http://www.sanesecurity.net/publickey.gpg"
securiteinfo_url="https://www.securiteinfo.com/get/signatures/"
linuxmalwaredetect_url="http://cdn.rfxn.com/downloads/"
malwarepatrol_free_url="https://lists.malwarepatrol.net/cgi/getfile?product=8&list=clamav_basic"
malwarepatrol_subscription_url="https://lists.malwarepatrol.net/cgi/getfile?product=15&list=clamav_basic"

yararules_url="https://raw.githubusercontent.com/Yara-Rules/rules/master/"

# ========================
# do not edit
config_version="53"

# https://eXtremeSHOK.com ##############################################################
cglmicro
Posts: 257
Joined: Thu Mar 07, 2013 2:12 am
How did you hear about Mailcleaner: google

Re: Clamspam definitions via sanesecurity

Postby cglmicro » Fri Nov 25, 2016 10:13 pm

Thanks Pantangle.

I understand that malwarepatrol offer now a "free" mode that won't require me to enter my receipt_code (that I also obtained for free) like before ? In other words: Can I use your malwarepatrol_receipt_code="f1452952545" ?

Same question for securiteinfo_authorisation_signature="94e2a532f4acdf12bf817cbff1e1823df26936f5bbb775c41849f2bd4b375b37212427d976799cd2fc8526f36cd87c359c6fa5298cc6db54314d871c511c9e52", can I also use yours or should I replace it with mine ?

EDITED: OUPS !! Now I receive this error:

Code: Select all

root@mailcleaner:/sanesecurity# sh clamav-unofficial-sigs.sh
======================================================================
 eXtremeSHOk.com ClamAV Unofficial Signature Updater
 Version: v4.4.3 (14 May 2015)
 Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
======================================================================
clamav-unofficial-sigs.sh: 554: Bad substitution


Any idea why ?
cglmicro
Posts: 257
Joined: Thu Mar 07, 2013 2:12 am
How did you hear about Mailcleaner: google

Re: Clamspam definitions via sanesecurity

Postby cglmicro » Sat Nov 26, 2016 3:15 am

I've resolved BAD SUBSTITUTION by removing SH before my script name.

But now I get:

Code: Select all

root@mailcleaner:~# /sanesecurity/clamav-unofficial-sigs.sh
======================================================================
 eXtremeSHOk.com ClamAV Unofficial Signature Updater
 Version: v4.4.3 (14 May 2015)
 Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
======================================================================
*** Your configuration version is not compatible with this version ***


I've downloaded the new script from https://github.com/extremeshok/clamav-u ... al-sigs.sh but I get:

Code: Select all

root@mailcleaner:~# /sanesecurity/clamav-unofficial-sigs.sh
################################################################################
 eXtremeSHOK.com ClamAV Unofficial Signature Updater
 Version: v5.4.1 (20 July 2016)
 Required Configuration Version: v65
 Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
=============================================
ERROR: Config file/s could NOT be read/loaded
=============================================


I also tried adding the variable default_config="/sanesecurity/clamav-unofficial-sigs.conf" in the 5.4.1 script.

Can you share also the script you are using in combination with this CONF file please ?

... and thanks !

Return to “Filter efficiency”

Who is online

Users browsing this forum: No registered users and 2 guests