Plugins

Overview

The fuglu core does nothing except receiving mails from postfix and sending them back. All functionality is written in plugins which can be enabled or disabled at will. Fuglu provides plugins for the most common mail filtering requirements, but if some functionality is missing, it is easy to add new plugins without knowing all of fuglu’s internals.

Types of plugins

Scanner Plugins

Scanner plugins are the most important type of plugins. They do the actual mail content filtering. Plugins are run in the order specified in the plugins= configuration option. Each plugin returns an action for the message:

  • DUNNO : This plugin decides not to take any final action, continue with the next plugin (this is the most common case)

  • ACCEPT : Whitelist this message, don’t run any remaining plugins

  • DELETE : Silently delete this message (The sender will think it has been delivered)

  • DEFER : Temporary Reject (4xx error), used for error conditions in after-queue mode or things like greylisting in before-queue mode

  • REJECT : Reject this message, should only be used in before-queue mode (in after-queue mode this would produce a bounce / backscatter)

If one of the plugins returns something other than DUNNO, all remaining scanner plugins are skipped. If all plugins return DUNNO, the message is accepted and re-injected to postfix.

Prepender Plugins

Prepender plugins run before the scanner plugins and have the ability to alter the list of scanner plugins to be run. This can be used for example to have different plugins run for incoming or outgoing mails or to skip whitelisted messages.

Appender Plugins

Appender plugins are run after the scanner plugins when the message has already been re-injected into postfix (or rejected or deleted or…) so they don’t increase your scanning time . They are mostly used for statistical tasks, updating counters in a database etc.

Plugin configuration

Each plugin has its own configuration section in /etc/fuglu/fuglu.conf or any *.conf in /etc/fuglu/conf.d. The section is usually named like the plugin itself.

For example, the Spamassassin Plugin’s Name is SAplugin, so it would search for a [SAPlugin] config section.

Suspect Filters

SuspectFilter are special rule files used by many fuglu plugins. Often they define actions a plugin should take based on a message header or suspect tag.

The format is : <property> <regex> <optional argument>

<property> can be any of…
  • a email header name, e.g. Received, To, From, Subject … also supports ‘*’ as wildcard character

  • mime:headername to get mime Headers in the message payload eg: mime:Content-Disposition

  • one of fuglus builtin names to get envelope data:

  • envelope_from or from_address (the envelope from address)

  • from_domain (domain part of envelope_from)

  • envelope_to or to_address (envelope to address)

  • to_domain (domain part of envelope_to)

  • a message Tag prepended by the @ symbol, eg. @incomingport

  • body:raw to match the the decoded message body (only applies to text/* parts)

  • body:stripped or just body to match the the message body (only applies to text/* parts), with stripped tags and newlines replaced with space (similar to SpamAssassin body rules)

  • body:full to match the full body

<regex> is a standard python regular expression applied to <property>. if the regex is not enclosed by forward slashes, be sure not to include any whitespace in the regex, it must be one continous string. To match whitespace, use \s. all regexes that are not enclosed in forward slashes are automatically case insensitive and support multiple lines (re.DOTALL is enabled, so the newline character is matched by a dot). with slashes, the regex flags can be defined manually, eg.

Subject /hello world/i match hello world, case insensitive

<optional argument> depends on the plugin that reads this header filter file, some don’t need arguments at all. Please refer to the plugin documentation.

Filter files are automatically reloaded if you change anything, you don’t need to restart fuglu.

Example:

#normal header test
Subject hello   Hello in the subject!
MIME-Version ^1\.0$ Mime Version is 1.0

#builtin special fields
to_domain       (\.)?fuglu\.org$        Sent to fuglu.org or any subdomain
envelope_from ^sender@example\.com$

#match a tag from a previous plugin
@SAPlugin.report MISSING_HEADER

#wildcard
X-Spam-* .*     a X-Spam-<something> header exists

#decoded body text parts
body    Viagra

#full body
body:full ^--SPAMMY-MIME-BOUNDARY

#mime-headers
mime:Content-Type ^application\/x-msdos-program$

see Debug a suspect filter for info on how to debug suspect filter files on specific messages.

Template Variables

Some plugins allow you to create templates (errormessages in bounces, reject reasons, …). Here is a list of commonly used template variables that should be available in those templates. Plugins may not support all those variables and can add more variables not listed here. If unsure, check the plugin documentation.

  • ${from_address} : Envelope Sender Address

  • ${to_address} : Envelope Recipient Address

  • ${from_domain} : Envelope Sender Domain

  • ${to_domain} : Envelope Recipient Domain

  • ${timestamp} : Unix Timestamp when the message was received

  • ${subject} : Message subject

  • ${date} : Current date

  • ${time} : Current time

  • ${blockinfo} : Reason why this message was blocked (available in the attachment plugin)

Plugins included in Fuglu

SpamAssassin

Plugin: fuglu.plugins.sa.SAPlugin

This plugin passes suspects to spamassassin daemon.

Prerequisites: SPAMD must be installed and running (not necessarily on the same box as fuglu)

Notes for developers:

if forwardoriginal=False, the message source will be completely replaced with the answer from spamd.

Tags:

  • reads SAPlugin.skip, (boolean) skips scanning if this is True

  • reads SAPlugin.tempheader, (text) prepends this text to the scanned message (use this to pass temporary headers to spamassassin which should not be visible in the final message)

  • sets spam['spamassassin'] (boolean)

  • sets SAPlugin.spamscore (float) if possible

  • sets SAPlugin.skipreason (string) if the message was not scanned (fuglu >0.5.0)

  • sets SAPlugin.report, (string) report from spamd or spamheader (where score was found) depending on forwardoriginal setting

Configuration

[SAPlugin]
#hostname where spamd runs
host=localhost

#tcp port number or path to spamd unix socket
port=783

#how long should we wait for an answer from sa
timeout=30

#maximum size in bytes. larger messages will be skipped
maxsize=256000

#enable scanning of messages larger than maxsize. all attachments will be stripped and only headers, plaintext and html part will be scanned. If message is still oversize it will be truncated.
strip_oversize=1

#how often should fuglu retry the connection before giving up
retries=3

#how long should fuglu wait in seconds before retryng the connection
retry_sleep=1

#should we scan the original message as retreived from postfix or scan the current state
#in fuglu (which might have been altered by previous plugins)
#only set this to disabled if you have a custom plugin that adds special headers to the message that will be
#used in spamassassin rules
scanoriginal=True

#forward the original message or replace the content as returned by spamassassin
#if this is enabled, no spamassassin headers will be visible in the final message.
#"original" in this case means "as passed to spamassassin", eg. if 'scanoriginal' above is disabled this will forward the
#message as retreived from previous plugins
forwardoriginal=False

#what header does SA set to indicate the spam status
#Note that fuglu requires a standard header template configuration for spamstatus and score extraction
#if 'forwardoriginal' is set to 0
#eg. start with _YESNO_ or _YESNOCAPS_ and contain score=_SCORE_
spamheader=X-Spam-Status

#tells fuglu what spamassassin prepends to its headers. Set this according to your spamassassin config especially if you forwardoriginal=0 and strip_oversize=1
spamheader_prepend=X-Spam-

#enable user_prefs in SA. This hands the recipient address over the spamd connection which allows SA to search for configuration overrides
peruserconfig=True

#lowercase user (envelope rcpt) before passing it to spamd
lowercase_user=True

#spamscore threshold to mark a message as high spam
highspamlevel=15

#what should we do with high spam (spam score above highspamlevel)
highspamaction=DEFAULTHIGHSPAMACTION

#what should we do with low spam (eg. detected as spam, but score not over highspamlevel)
lowspamaction=DEFAULTLOWSPAMACTION

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#reject message template if running in pre-queue mode
rejectmessage=message identified as spam

#sqlalchemy db connect string, e.g. mysql:///localhost/spamassassin
sql_blocklist_dbconnectstring=

#Suspect tags to attach as text part to message for scanning
attach_suspect_tags=

Clam AV

Plugin: fuglu.plugins.clamav.ClamavPlugin

This plugin passes suspects to a clam daemon.

Actions: This plugin will delete infected messages. If clamd is not reachable or times out, messages can be DEFERRED.

Prerequisites: You must have clamd installed (for performance reasons I recommend it to be on the same box, but this is not absoluely necessary)

Notes for developers:

Tags:

  • sets virus['ClamAV'] (boolean)

  • sets ClamavPlugin.virus (list of strings) - virus names found in message

Configuration

[ClamavPlugin]
#hostname where clamd runs
host=localhost

#tcp port number or path to clamd.sock for unix domain sockets
#example /var/lib/clamav/clamd.sock or on ubuntu: /var/run/clamav/clamd.ctl
port=3310

#socket timeout
timeout=30

#*EXPERIMENTAL*: Perform multiple scans over the same connection. May improve performance on busy systems.
pipelining=False

#maximum message size, larger messages will not be scanned.
#should match the 'StreamMaxLength' config option in clamd.conf
maxsize=22000000

#how often should fuglu retry the connection before giving up
retries=3

#action if infection is detected (DUNNO, REJECT, DELETE)
virusaction=DEFAULTVIRUSACTION

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#reject message template if running in pre-queue mode and virusaction=REJECT
rejectmessage=threat detected: ${virusname}

#*EXPERIMENTAL*: fallback to clamscan if clamd is unavailable. YMMV, each scan can take 5-20 seconds and massively increase load on a busy system.
clamscanfallback=False

#the path to clamscan executable
clamscan=/usr/bin/clamscan

#process timeout
clamscantimeout=30

#define AVScanner engine names causing current plugin to skip if they found already a virus
skip_on_previous_virus=none

#path to file with signature names that can be skipped if message is welcomelisted. list one signature per line, signature is case sensitive
skiplist_file=

Attachment Blocker

Plugin: fuglu.plugins.attachment.FiletypePlugin

This plugin checks message attachments. You can configure what filetypes or filenames are allowed to pass through fuglu. If a attachment is not allowed, the message is deleted and the sender receives a bounce error message. The plugin uses the ‘’’file’’’ library to identify attachments, so even if a smart sender renames his executable to .txt, fuglu will detect it.

Attachment rules can be defined globally, per domain or per user.

Actions: This plugin will delete messages if they contain blocked attachments.

Prerequisites: You must have the python file or magic module installed. Additionaly, for scanning filenames within rar archives, fuglu needs the python rarfile module.

The attachment configuration files are in /etc/fuglu/rules. You should have two default files there: default-filenames.conf which defines what filenames are allowed and default-filetypes.conf which defines what content types a attachment may have.

For domain rules, create a new file <domainname>-filenames.conf / <domainname>-filetypes.conf , eg. fuglu.org-filenames.conf / fuglu.org-filetypes.conf

For individual user rules, create a new file <useremail>-filenames.conf / <useremail>-filetypes.conf, eg. oli@fuglu.org-filenames.conf / oli@fuglu.org-filetypes.conf

To scan filenames or even file contents within archives (zip, rar), use <...>-archivefilenames.conf and <...>-archivefiletypes.conf.

The format of those files is as follows: Each line should have three parts, seperated by tabs (or any whitespace): <action> <regular expression> <description or error message>

<action> can be one of:
  • allow : this file is ok, don’t do further checks (you might use it for safe content types like text). Do not blindly create ‘allow’ rules. It’s safer to make no rule at all, if no other rules hit, the file will be accepted

  • deny : delete this message and send the error message/description back to the sender

  • delete : silently delete the message, no error is sent back, and ‘blockaction’ is ignored

<regular expression> is a standard python regex. in x-filenames.conf this will be applied to the attachment name . in x-filetypes.conf this will be applied to the mime type of the file as well as the file type returned by the file command.

Example of default-filetypes.conf :

allow    text        -
allow    script    -
allow    archive        -
allow    postscript    -
deny    self-extract    No self-extracting archives
deny    executable    No programs allowed
deny    ELF        No programs allowed
deny    Registry    No Windows Registry files allowed

A small extract from default-filenames.conf:

deny    \.ico$            Windows icon file security vulnerability
deny    \.ani$            Windows animated cursor file security vulnerability
deny    \.cur$            Windows cursor file security vulnerability
deny    \.hlp$            Windows help file security vulnerability

allow    \.jpg$            -
allow    \.gif$            -

Note: The files will be reloaded automatically after a few seconds (you do not need to kill -HUP / restart fuglu)

Per domain/user overrides can also be fetched from a database instead of files (see dbconnectstring / query options). The query must return the same rule format as a file would. Multiple columns in the resultset will be concatenated.

The default query assumes the following schema:

CREATE TABLE `attachmentrules` (
  `rule_id` int(11) NOT NULL AUTO_INCREMENT,
  `action` varchar(10) NOT NULL,
  `regex` varchar(255) NOT NULL,
  `description` varchar(255) DEFAULT NULL,
  `scope` varchar(255) DEFAULT NULL,
  `checktype` varchar(20) NOT NULL,
  `prio` int(11) NOT NULL,
  PRIMARY KEY (`rule_id`)
)

action: allow, deny, or delete

regex: a regular expression

description: description/explanation of this rule which is optionally sent back to the sender if bounces are enabled

scope: a domain name or a recipient’s email address

checktype: one of filename,``contenttype``,``archive-filename``,``archive-contenttype``

prio: order in which the rules are run

The bounce template (eg /etc/fuglu/templates/blockedfile.tmpl ) should start by defining the headers, followed by a blank line, then the message body for your bounce message. Something like this:

To: ${from_address}
Subject: Blocked attachment

Your message to ${to_address} contains a blocked attachment and has not been delivered.

${blockinfo}

${blockinfo} will be replaced with the text you specified in the third column of the rule that blocked this message.

The other common template variables are available as well.

Configuration

[FiletypePlugin]
#Mail template for the bounce to inform sender about blocked attachment
template_blockedfile=/etc/fuglu/templates/blockedfile.tmpl

#inform the sender about blocked attachments.
#If a previous plugin tagged the message as spam or infected, no bounce will be sent to prevent backscatter
sendbounce=True

#directory that contains attachment rules
rulesdir=/etc/fuglu/rules

#what should the plugin do when a blocked attachment is detected
#REJECT : reject the message (recommended in pre-queue mode)
#DELETE : discard messages
#DUNNO  : mark as blocked but continue anyway (eg. if you have a later quarantine plugin)
blockaction=DELETE

#sqlalchemy connectstring to load rules from a database and use files only as fallback. requires SQL extension to be enabled
dbconnectstring=

#sql query to load rules from a db. #:scope will be replaced by the recipient address first, then by the recipient domain
#:check will be replaced 'filename','contenttype','archive-filename' or 'archive-contenttype'
query=SELECT action,regex,description FROM attachmentrules WHERE scope=:scope AND checktype=:checktype ORDER BY prio

#enable scanning of filenames within archives (zip,rar). This does not actually extract the files, it just looks at the filenames found in the archive.
checkarchivenames=False

#extract compressed archives(zip,rar) and check file content type with libmagics
#note that the files will be extracted into memory - tune archivecontentmaxsize  accordingly.
#fuglu does not extract archives within the archive(recursion)
checkarchivecontent=False

#only extract and examine files up to this amount of (uncompressed) bytes
archivecontentmaxsize=5000000

#recursive extraction level for archives. Undefined or negative value means extract until it's not an archive anymore
archiveextractlevel=1

#comma separated list of archive extensions. do only process archives of given types.
enabledarchivetypes=

Archive

Plugin: fuglu.plugins.archive.ArchivePlugin

This plugins stores a copy of the message if it matches certain criteria (Suspect Filter). You can use this if you want message archives for your domains, need a quarantine or to debug problems occuring only for certain recipients. The architecture allows to store data in various backends (databases), either simultaneously or using one backend as main and others as fallback in case the main backend is unavailable. Currently the following backends are supported: - localdir: save mail to a local directory structure. This backend is stable and in active use. - lmtp: pass message to LMTP server. This backend is stable and in active use. - elastic: store message in elasticsearch. This backend is in development (2021). - cassandra: store message in cassandra. This backend is no longer maintained and published mainly for code archival reasons. YMMV.

Examples for the archive.regex filter file:

Archive messages to domain ‘’test.com’’:

to_domain test\.com

Archive messages from oli@fuglu.org:

envelope_from oli@fuglu\.org

you can also append “yes” and “no” to the rules to create a more advanced configuration. Lets say we want to archive all messages to sales@fuglu.org and all regular messages support@fuglu.org except the ones created by automated scripts like logwatch or daily backup messages etc.

` envelope_from logwatch@.*fuglu\.org   no envelope_to sales@fuglu\.org yes from backups@fuglu\.org no envelope_to support@fuglu\.org      yes `

Archive/Quarantine messages that are marked as spam: ` @spam['spamassassin'] True @archive.spam True `

Note: The first rule to match in a message is the only rule that will be applied. Exclusion rules should therefore be put above generic/catch-all rules.

Configuration

[ArchivePlugin]
#Archiving SuspectFilter File
archiverules=/etc/fuglu/archive.regex

#comma separated list of backends to use. available backends: localdir, lmtp, elastic, cassandra
archivebackends=localdir

#set to True to store mail in all enabled backends. set to False to only use primary and fallback to other backends on error
multibackend=False

#Name of header containing alternative Fuglu ID that overrides storage key
fugluid_headername=

#skip archiving if fugluid_headername is not set
fugluid_headername_skipmissing=True

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#storage for archived messages
local_archivedir=/tmp

#subdirectory within archivedir
local_subdirtemplate=${to_domain}/${to_localpart}/${date}

#filename template for the archived messages
local_filenametemplate=${archiveid}.eml

#if true/1/yes: store original message
#if false/0/no: store message probably altered by previous plugins, eg with spamassassin headers
local_useoriginal=True

#change owner of saved messages (username or numeric id) - this only works if fuglu is running as root (which is NOT recommended)
local_chown=

#change group of saved messages (groupname or numeric id) - the user running fuglu must be a member of the target group for this to work
local_chgrp=

#set file permissions of saved messages
local_chmod=

#comma separated list of LMTP target hostname, hostname:port or path to local LMTP socket (path must start with /)
lmtp_hosts=

#LMTP auth user. leave empty if no authentication is needed
lmtp_user=

#LMTP auth password. leave empty if no authentication is needed
lmtp_password=

#LMTP envelope sender. Leave empty for original SMTP envelope sender
lmtp_sender=

#should we store the original message as retreived from postfix or store the
#                                current state in fuglu (which might have been altered by previous plugins)
lmtp_useoriginal=True

#Name of header containing Fuglu ID when storing via LMTP
lmtp_headername=X-Fuglu-ID

#comma separated list of ElasticSearch host definition (hostname, hostname:port, https://user:pass@hostname:port/)
elastic_uris=

#Name of ElasticSearch index in which document will be stored. Template vars (e.g. ${to_domain} or ${date}) can be used.
elastic_index=fugluquar-${date}

#comma separated list of additional fields to be added to document. Any fuglu Suspect variable is permitted (e.g. to_address)
elastic_extrafields=

#should we store the original message as retreived from postfix or store the
#                                current state in fuglu (which might have been altered by previous plugins)
elastic_useoriginal=True

#quarantine cassandra hostnames, separated by comma
cassandra_hosts=

#quarantine cassandra keyspace
cassandra_keyspace=fugluquar

#ttl for quarantined files in seconds
cassandra_ttl=1209600

#should we store the original message as retreived from postfix or store the
#                                current state in fuglu (which might have been altered by previous plugins)
cassandra_useoriginal=True

Action Override

Plugin: fuglu.plugins.decision.ActionOverridePlugin

Override actions based on a Suspect Filter file. For example, delete all messages from a specific sender domain.

Configuration

[ActionOverridePlugin]
#Rules file
actionrules=/etc/fuglu/actionrules.regex

delete Message

Plugin: fuglu.plugins.decision.KillerPlugin

DELETE all mails (for special mail setups like spam traps etc)

Configuration

[KillerPlugin]

Sophos AV

Plugin: fuglu.plugins.sssp.SSSPPlugin

This plugin scans the suspect using the sophos SSSP protocol.

Prerequisites: Requires a running sophos daemon with dynamic interface (SAVDI)

Configuration

[SSSPPlugin]
#hostname where the SSSP server runs
host=localhost

#tcp port or path to unix socket
port=4010

#socket timeout
timeout=30

#maximum message size, larger messages will not be scanned.
maxsize=22000000

#how often should fuglu retry the connection before giving up
retries=3

#action if infection is detected (DUNNO, REJECT, DELETE)
virusaction=DEFAULTVIRUSACTION

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#reject message template if running in pre-queue mode and virusaction=REJECT
rejectmessage=threat detected: ${virusname}

F-Prot AV

Plugin: fuglu.plugins.antivirus.FprotPlugin

This plugin passes suspects to a f-prot scan daemon

Prerequisites: f-protd must be installed and running, not necessarily on the same box as fuglu though.

Notes for developers:

Tags:

  • sets virus['F-Prot'] (boolean)

  • sets FprotPlugin.virus (list of strings) - virus names found in message

Configuration

[FprotPlugin]
#hostname where fpscand runs
host=localhost

#fpscand port
port=10200

#network timeout
timeout=30

#Always send data over network instead of just passing the file name when possible. If fpscand runs on a different host than fuglu, you must enable this.
networkmode=False

#additional scan options  (see `man fpscand` -> SCANNING OPTIONS for possible values)
scanoptions=

#maximum message size to scan
maxsize=10485000

#maximum retries on failed connections
retries=3

#plugin action if threat is detected
virusaction=DEFAULTVIRUSACTION

#plugin action if scan fails
problemaction=DEFER

#reject message template if running in pre-queue mode and virusaction=REJECT
rejectmessage=threat detected: ${virusname}

ICAP AV

Plugin: fuglu.plugins.antivirus.ICAPPlugin

ICAP Antivirus Plugin This plugin allows Antivirus Scanning over the ICAP Protocol (http://tools.ietf.org/html/rfc3507 ) supported by some AV Scanners like Symantec and Sophos. For sophos, however, it is recommended to use the native SSSP Protocol.

Prerequisites: requires an ICAP capable antivirus engine somewhere in your network

Configuration

[ICAPPlugin]
#hostname where the ICAP server runs
host=localhost

#tcp port or path to unix socket
port=1344

#socket timeout
timeout=10

#maximum message size, larger messages will not be scanned.
maxsize=22000000

#how often should fuglu retry the connection before giving up
retries=3

#action if infection is detected (DUNNO, REJECT, DELETE)
virusaction=DEFAULTVIRUSACTION

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#reject message template if running in pre-queue mode and virusaction=REJECT
rejectmessage=threat detected: ${virusname}

#ICAP Av scan service, usually AVSCAN (sophos, symantec)
service=AVSCAN

#name of the virus engine behind the icap service. used to inform other plugins. can be anything like 'sophos', 'symantec', ...
enginename=icap-generic

DrWeb AV

Plugin: fuglu.plugins.antivirus.DrWebPlugin

This plugin passes suspects to a DrWeb scan daemon

EXPERIMENTAL Plugin: has not been tested in production.

Prerequisites: Dr.Web unix version must be installed and running, not necessarily on the same box as fuglu though.

Notes for developers:

Tags:

  • sets virus['drweb'] (boolean)

  • sets DrWebPlugin.virus (list of strings) - virus names found in message

Configuration

[DrWebPlugin]
#hostname where fpscand runs
host=localhost

#DrWeb daemon port
port=3000

#network timeout
timeout=30

#maximum message size to scan
maxsize=22000000

#maximum retries on failed connections
retries=3

#plugin action if threat is detected
virusaction=DEFAULTVIRUSACTION

#plugin action if scan fails
problemaction=DEFER

#reject message template if running in pre-queue mode and virusaction=REJECT
rejectmessage=threat detected: ${virusname}

DKIM Sign

Plugin: fuglu.plugins.domainauth.DKIMSignPlugin

Add DKIM Signature to outgoing mails

Setting up your keys:

mkdir -p /etc/fuglu/dkim
domain=example.com
openssl genrsa -out /etc/fuglu/dkim/${domain}.key 1024
openssl rsa -in /etc/fuglu/dkim/${domain}.key -out /etc/fuglu/dkim/${domain}.pub -pubout -outform PEM
# print out the DNS record:
echo -n "default._domainkey TXT  \"v=DKIM1; k=rsa; p=" ; cat /etc/fuglu/dkim/${domain}.pub | grep -v 'PUBLIC KEY' | tr -d '\n' ; echo ";\""

If fuglu handles both incoming and outgoing mails you should make sure that this plugin is skipped for incoming mails

Configuration

[DKIMSignPlugin]
#Location of the private key file. supports standard template variables plus additional ${header_from_domain} which extracts the domain name from the From: -Header
privatekeyfile=/etc/fuglu/dkim/${header_from_domain}.key

#Type of header canonicalization (simple or relaxed)
canonicalizeheaders=relaxed

#Type of body canonicalization (simple or relaxed)
canonicalizebody=relaxed

#selector to use when signing, supports templates
selector=default

#comma separated list of headers to sign. empty string=sign all headers
signheaders=From,Reply-To,Subject,Date,To,CC,Resent-Date,Resent-From,Resent-To,Resent-CC,In-Reply-To,References,List-Id,List-Help,List-Unsubscribe,List-Subscribe,List-Post,List-Owner,List-Archive

#include l= tag in dkim header
signbodylength=False

DKIM Verify

Plugin: fuglu.plugins.domainauth.DKIMVerifyPlugin

This plugin checks the DKIM signature of the message and sets tags… DKIMVerify.sigvalid : True if there was a valid DKIM signature, False if there was an invalid DKIM signature the tag is not set if there was no dkim header at all

DKIMVerify.skipreason: set if the verification has been skipped

The plugin does not take any action based on the DKIM test result since a failed DKIM validation by itself should not cause a message to be treated any differently. Other plugins might use the DKIM result in combination with other factors to take action (for example a “DMARC” plugin could use this information)

It is currently recommended to leave both header and body canonicalization as ‘relaxed’. Using ‘simple’ can cause the signature to fail.

Configuration

[DKIMVerifyPlugin]
#File containing a list of domains (one per line) which are not checked
skiplist=

SPF Check

Plugin: fuglu.plugins.domainauth.SPFPlugin

This plugin performs SPF validation using the pyspf module https://pypi.python.org/pypi/pyspf/ by default, it just logs the result (test mode)

to enable actual rejection of messages, add a config option on_<resulttype> with a valid postfix action. eg:

on_fail = REJECT

on_{result} = … valid {result} types are: ‘pass’, ‘permerror’, ‘fail’, ‘temperror’, ‘softfail’, ‘none’, and ‘neutral’ you probably want to define REJECT for fail and softfail

I want to reject all hard fails and accept all soft fails:
  • do not set domain_selective_spf_file

  • set selective_softfail to False

  • set on_fail to REJECT and on_softfail to continue

I want to reject all hard fails and all soft fails:
  • do not set domain selective_spf_file

  • set selective_softfail to False

  • set on_fail to REJECT and on_softfail to REJECT

I only want to reject select hard and soft fails
  • set a domain_selective_spf_file and list the domains to be tested

  • set selective_softfail to False

  • set on_fail to REJECT and on_softfail to REJECT

I want to reject all hard fails and only selected soft fails:
  • set a domain_selective_spf_file and list the domains to be tested for soft fail

  • set selective_softfail to True

  • set on_fail to REJECT and on_softfail to REJECT

I want to reject select hard fails and accept all soft fails:
  • do not set domain selective_spf_file

  • set selective_softfail to True

  • set on_fail to REJECT and on_softfail to continue

This plugin checks the SPF status and sets tag ‘SPF.status’ to one of the official states ‘pass’, ‘fail’, ‘neutral’, ‘softfail, ‘permerror’, ‘temperror’ or ‘skipped’ if the SPF check could not be peformed. Tag ‘SPF.explanation’ contains a human readable explanation of the result. Additionally information to be used by SA plugin is added

The plugin does not take any action based on the SPF test result since. Other plugins might use the SPF result in combination with other factors to take action (for example a “DMARC” plugin could use this information)

However, if mark_milter_check=True then the message is marked as spam if the milter stage check would reject this (fail or softfail). This feature is to avoid rejecting at milter stage but mark later in post-queue mode as spam.

Configuration

[SPFPlugin]
#(milter) file containing a list of IP adresses or CIDR ranges to be exempted from SPF checks. 127.0.0.0/8 is always exempted
ip_whitelist_file=

#(milter) if this is non-empty, all except sender domains in this file will be checked for SPF. define exceptions by prefixing with ! (e.g. example.com !foo.example.com). define TLD wildcards using * (e.g. example.*)
domain_whitelist_file=

#(milter) if this is non-empty, only sender domains in this file will be checked for SPF. define exceptions by prefixing with ! (e.g. example.com !foo.example.com). define TLD wildcards using * (e.g. example.*)
domain_selective_spf_file=

#(milter) evaluate all senders for hard fails (unless listed in domain_whitelist_file) and only evaluate softfail for domains listed in domain_selective_spf_file
selective_softfail=False

#(milter) apply checks to subdomain of whitelisted/selective domains
check_subdomain=False

#SQLAlchemy Connection string, e.g. mysql://root@localhost/spfcheck?charset=utf8. Leave empty to disable SQL lookups
dbconnection=

#get from sql database :domain will be replaced with the actual domain name. must return field check_spf
domain_sql_query=SELECT check_spf from domain where domain_name=:domain

#Action for SPF fail. (DUNNO, DEFER, REJECT)
on_fail=DUNNO

#If Suspect/Session tag is set, return DUNNO on fail
on_fail_dunnotag=

#Action for SPF softfail. (DUNNO, DEFER, REJECT)
on_softfail=DUNNO

#If Suspect/Session tag is set, return DUNNO on softfail
on_softfail_dunnotag=

#reject message template for policy violators
messagetemplate=SPF ${result} for domain ${from_domain} from ${client_address} : ${explanation}

#maximum number of lookups (RFC defaults to 10)
max_lookups=10

#always consider pass if mail is sent from servers specified, MX points to this server, and SPF record contains MX directive
hoster_mx_exception=.google.com .protection.outlook.com

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

#(eom) File containing a list of domains (one per line) which are not checked
skiplist=

#maximum number of retries on temp error
temperror_retries=3

#waiting interval between retries on temp error
temperror_sleep=3

#(eom) check milter setup criterias, mark as spam on hit
mark_milter_check=False

Sender Rewrite Scheme

Plugin: fuglu.plugins.domainauth.SenderRewriteScheme

SRS (Sender Rewriting Scheme) Plugin This plugin encrypts envelope sender and decrypts bounce recpient addresses with SRS As opposed to postsrsd it decides by RECIPIENT address whether sender address should be rewritten. This plugin only works in after queue mode

Required dependencies:
  • pysrs

Recommended dependencies:
  • sqlalchemy

Configuration

[SenderRewriteScheme]
#SQLAlchemy Connection string. Leave empty to rewrite all senders
dbconnection=mysql://root@localhost/spfcheck?charset=utf8

#get from sql database :domain will be replaced with the actual domain name. must return field use_srs
domain_sql_query=SELECT use_srs from domain where domain_name=:domain

#the new envelope sender domain
forward_domain=example.com

#cryptographic secret. set the same random value on all your machines
secret=

#maximum lifetime of bounces
maxage=8

#size of auth code
hashlength=8

#SRS token separator
separator==

#set True to rewrite address in To: header in bounce messages (reverse/decrypt mode)
rewrite_header_to=True

DomainAuth

Plugin: fuglu.plugins.domainauth.DomainAuthPlugin

EXPERIMENTAL This plugin checks the header from domain against a list of domains which must be authenticated by DKIM and/or SPF. This is somewhat similar to DMARC but instead of asking the sender domain for a DMARC policy record this plugin allows you to force authentication on the recipient side.

This plugin depends on tags written by SPFPlugin and DKIMVerifyPlugin, so they must run beforehand.

Configuration

[DomainAuthPlugin]
#File containing a list of domains (one per line) which must be DKIM and/or SPF authenticated
domainsfile=/etc/fuglu/auth_required_domains.txt

#action if the message doesn't pass authentication (DUNNO, REJECT)
failaction=DUNNO

#reject message template if running in pre-queue mode
rejectmessage=sender domain ${header_from_domain} must pass DKIM and/or SPF authentication

Delay Message by 0.000001

Plugin: fuglu.plugins.delay.DelayPlugin

Sleep for a given time (debugging)

Configuration

[DelayPlugin]
#execution time of the examine function
delay=1e-06

#frequency of writing a log message while waiting in the examine function
logfrequency=0.0001

IMAPCopyPlugin

Plugin: fuglu.plugins.mailcopy.IMAPCopyPlugin

This plugins stores a copy of the message to an IMAP mailbox if it matches certain criteria (Suspect Filter). The rulefile works similar to the archive plugin. As third column you have to provide imap account data in the form:

<protocol>://<username>:<password>@<servernameorip>[:port]/<mailbox>

<protocol> is one of:
  • imap (port 143, no encryption)

  • imap+tls (port 143 and StartTLS, only supported in Python 3)

  • imaps (port 993 and SSL)

Configuration

[IMAPCopyPlugin]
#IMAP copy suspectFilter File
imapcopyrules=/etc/fuglu/imapcopy.regex

#if true/1/yes: store original message
#if false/0/no: store message probably altered by previous plugins, eg with spamassassin headers
storeoriginal=True

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

MailFeed

Plugin: fuglu.plugins.mailcopy.MailFeed

Send a copy of a message to a certain target server

Configuration

[MailFeed]
#file with feed targets.
#                format: target.server[;opt=val;opt=val] domain1 domain2 or * for all domains
#                possible options: port=587;tls=True;xclient=True;user=user;pass=pass;from=<>;to=*;timeout=30
#                options tls, xclient, and from will override the global defaults defined in plugin config
targetfile=/etc/fuglu/mailfeeds.txt

#comma separated list of mail classes to be delivered: any, ham, spam, virus, blocked
mail_types=any

#envelope sender to be used. set to <> for empty envelope sender, set to * to use original sender
from_address=<>

#template for envelope recipient to be used. set to * to use original recipient. will always be set to @target if domain is equals to fuglu hostname
to_address=${to_localpart}@${target}

use_tls=True

use_xclient=False

RSpamd

Plugin: fuglu.plugins.rspamd.RSpamdPlugin

Scan messages with rspamd through rspamd’s RESTful interface.

This plugin aims to mimick the behaviour of the fuglu stock SA plugin as closely as possible.

Configuration

[RSpamdPlugin]
#hostname where rspamd runs
host=127.0.0.1

#port number of rspamd TCP socket ("normal" worker)
port=11333

#how long should we wait for an answer from rspamd
timeout=30

#maximum size in bytes. larger messages will be skipped
maxsize=256000

#should we scan the original message as retreived from postfix or scan the current state
#in fuglu (which might have been altered by previous plugins)
#only set this to disabled if you have a custom plugin that adds special headers to the message that will be
#used in rspamd rules
scanoriginal=True

#forward the original message or add spam report
#if this is enabled, no rspamd spam report headers will be visible in the final message.
#"original" in this case means "as passed to rspamd", eg. if 'scanoriginal' above is disabled this will forward the
#message as retreived from previous plugins
forwardoriginal=False

#spamscore threshold to mark a message as high spam
highspamlevel=15

#what should we do with high spam (spam score above highspamlevel)
highspamaction=DEFAULTHIGHSPAMACTION

#what should we do with low spam (eg. detected as spam, but score not over highspamlevel)
lowspamaction=DEFAULTLOWSPAMACTION

#action if there is a problem (DUNNO, DEFER)
problemaction=DEFER

#reject message template if running in pre-queue mode
rejectmessage=message identified as spam

ScriptFilter

Plugin: fuglu.plugins.script.ScriptFilter

This plugin executes scripts found in a specified directory. This can be used to quickly add a custom filter script without changing the fuglu configuration.

Filterscripts must be written in standard python but with the file ending .fgf (“fuglu filter”)

Scripts are reloaded for every message and executed in alphabetic order. You do not need to restart fuglu to load any changes made to these files.

The API is basically the same as for normal plugins within the examine() method, with a few special cases:

there is no ‘self’ which means:

  • access the configuration by using config directly (instead of self.config)

  • use debug('hello world') instead of self._logger().debug('hello world'), this will also automatically write to the message debug channel

the script should not return anything, but change the available variables action and message instead (DUNNO, REJECT, DEFER, ACCEPT, DELETE are already imported)

use stop(action=DUNNO, message='') to exit the script early

Example script: (put this in /etc/fuglu/scriptfilter/99_demo.fgf for example)

#block all messages from evilsender.example.com
if not suspect.from_domain=='evilsender.example.com':
    suspect.add_header("x-fuglu-SenderDomain",suspect.from_domain,immediate=True)
    stop()
debug("hello world")
action=REJECT
message="you shall not pass"

Configuration

[ScriptFilter]
#Dir that contains the scripts (*.fgf files)
scriptdir=/etc/fuglu/scriptfilter

URIExtract

Plugin: fuglu.plugins.uriextract.URIExtract

Extract URIs from message bodies and store them as list in tag body.uris

Configuration

[URIExtract]
#Domain skip list
domainskiplist=/etc/fuglu/extract-skip-domains.txt

#Max. time after which extraction will be stopped (approximate only, <0.0:infinite)
timeout=-1

#Maximum size of processed mail parts/attachments.
maxsize=10485000

#Maximum size of string to analyze in bytes.
maxsize_analyse=2000000

#print extracted uris in fuglu log
loguris=no

#Use extra hacks (int level) trying to parse uris (0: no hacks)
usehacks=0

#List with headers to check for uris
uricheckheaders=

EmailExtract

Plugin: fuglu.plugins.uriextract.EmailExtract

Configuration

[EmailExtract]
#Domain skip list
domainskiplist=/etc/fuglu/extract-skip-domains.txt

#Max. time after which extraction will be stopped (approximate only, <0.0:infinite)
timeout=-1

#Maximum size of processed mail parts/attachments.
maxsize=10485000

#Maximum size of string to analyze in bytes.
maxsize_analyse=2000000

#print extracted uris in fuglu log
loguris=no

#Use extra hacks (int level) trying to parse uris (0: no hacks)
usehacks=0

#List with headers to check for uris
uricheckheaders=

#comma separated list of headers to check for adresses to extract
headers=Return-Path,Reply-To,From,X-RocketYMMF,X-Original-Sender,Sender,X-Originating-Email,Envelope-From,Disposition-Notification-To

#comma separated list of headers with email adresses that should be skipped in body search
skipheaders=X-Original-To,Delivered-To,X-Delivered-To,Apparently-To,X-Apparently-To

#include envelope sender address as header address
with_envelope_sender=True

DomainAction

Plugin: fuglu.plugins.uriextract.DomainAction

Perform Action based on Domains in message body

Configuration

[DomainAction]
#Domainmagic RBL lookup config file
blacklistconfig=/etc/fuglu/rbl.conf

#check subdomains as well (from top to bottom, eg. example.com, bla.example.com, blubb.bla.example.com
checksubdomains=yes

#action on hit (dunno, reject, delete, etc)
action=DUNNO

#message template for rejects/ok messages
message=5.7.1 black listed URL ${domain} by ${blacklist}

#maximum number of domains to check per message
maxlookups=10

#randomise domain list before performing lookups
randomise=False

#if set to True do not abort on first hit, instead continue until maxlookups reached
check_all=False

#path to file with extra TLDs (2TLD or inofficial TLDs)
extra_tld_file=

#test record that should be included in at least one checked rbl (only used in lint)
testentry=

#path to file containing domains that should not be checked (one per line)
exceptions_file=

#evaluate URIs listed in given tags (list tags white space separated)
suspect_tags=body.uris

EmailAction

Plugin: fuglu.plugins.uriextract.EmailAction

Configuration

[EmailAction]
#Domainmagic RBL lookup config file
blacklistconfig=/etc/fuglu/rbl.conf

#action on hit (dunno, reject, delete, etc)
action=DUNNO

#message template for rejects/ok messages
message=5.7.1 black listed email address ${address} by ${blacklist}

#maximum number of domains to check per message
maxlookups=10

#randomise domain list before performing lookups
randomise=False

#if set to True do not abort on first hit, instead continue until maxlookups reached
check_all=False

#test record that should be included in at least one checked rbl (only used in lint)
testentry=

#path to file containing email addresses that should not be checked (one per line)
exceptions_file=

#evaluate URIs listed in given tags (list tags white space separated)
suspect_tags=body.uris

#path to file containing a list of domains. if specified, only query email addresses in these domains.
domainlist_file=

Vacation

Plugin: fuglu.plugins.vacation.VacationPlugin

Sends out-of-office reply messages. Configuration is trough a sql database. Replies are only sent once per day per sender. The plugin will not reply to any ‘automated’ messages (Mailingslists, Spams, Bounces etc)

Requires: SQLAlechemy Extension

Required DB Tables:
  • vacation (fuglu reads this table only, must be filled from elsewhere)

    • id int : id of this vacation

    • created timestamp : creation timestamp

    • enabled boolean (eg. tinyint) : if disabled, no vacation reply will be sent

    • start timestamp: replies will only be sent after this point in time

    • end timestamp: replies will only be sent before this point in time

    • awayuser varchar: the email address of the user that is on vacation

    • subject: subject of the vacation message

    • body : body of the vacation message

    • ignoresender: whitespace delimited list of domains or email addresses that should not receive vacation replies

  • vacationreply (this table is filled by fuglu)

    • id int: id of the reply

    • vacation_id : id of the vacation

    • sent timestamp: timestamp when the reply was sent

    • recipient: recipient to whom the reply was sent

SQL Example for mysql:

CREATE TABLE `vacation` (
  `id` int(11) NOT NULL auto_increment,
  `created` timestamp NOT NULL default now(),
  `start` timestamp NOT NULL,
  `end` timestamp NOT NULL ,
  `enabled` tinyint(1) NOT NULL default 1,
  `awayuser` varchar(255) NOT NULL,
  `subject` varchar(255) NOT NULL,
  `body` text NOT NULL,
  `ignoresender` text NOT NULL,
  PRIMARY KEY  (`id`),
  UNIQUE(`awayuser`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;


CREATE  TABLE `vacationreply` (
  `id` int(11) NOT NULL auto_increment,
  `recipient` varchar(255) NOT NULL,
  `vacation_id` int(11) NOT NULL,
     `sent` timestamp not null default now(),
  PRIMARY KEY  (`id`),
  KEY `vacation_id` (`vacation_id`),
  CONSTRAINT `vacation_ibfk_1` FOREIGN KEY (`vacation_id`) REFERENCES `vacation` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Configuration

[VacationPlugin]
#sqlalchemy connectstring to load vacations
dbconnectstring=

Address Check

Plugin: fuglu.plugins.call_ahead.AddressCheck

Configuration

[AddressCheck]
#the storage backend, one of sql, redis, memory. memory is local only.
cache_storage=sql

#the config backend, currently only sql
config_backend=sql

#SQLAlchemy connection string for sql backend and sql config
dbconnection=mysql://root@localhost/callahead?charset=utf8

#redis backend database connection: redis://host:port/dbid
redis_conn=redis://127.0.0.1:6379/1

#redis backend timeout in seconds
redis_timeout=2

#password to connect to redis database. leave empty for no password
redis_password=

#memory backend cleanup interval
cleanupinterval=300

#set this to true to disable the skiplisting of servers that don't support recipient verification
always_assume_rec_verification_support=False

#Set this to always return 'continue' but still perform the recipient check and fill the cache (learning mode without rejects)
always_accept=False

#how long should expired positive cache data be kept in the table history [days] (sql only)
keep_positive_history_time=30

#how long should expired negative cache data be kept in the table history [days] (sql only)
keep_negative_history_time=1

#reject message template for previously cached responses
messagetemplate=previously cached response: ${message}

#reject message template in positive directory mode
messagetemplate_directory_only=User unknown

#always accept recipients if they match this regex. leave empty to omit
always_accept_regex=

#enable recipient verification
enabled=True

#if set to True, do not perform call-ahead and consider missing entries in database to be negative/rejectable. This is useful if the entries for this domain are added via alternative means to the database (e.g. hardcoded or imported from a database/active directory/user listing)
positive_directory_only=False

#socket timeout
timeout=30

#test user that probably does not exist
testuser=rbxzg133-7tst

#how long should we skiplist a server if it doesn't support recipient verification [seconds]
test_server_interval=3600

#how long should we cache existing addresses [seconds]
positive_cache_time=604800

#how long should we keep negative cache entries [seconds]
negative_cache_time=14400

#how should we retrieve the next hop?
server=mx:${domain}

#if first server fails, try fallback relays?
test_fallback=False

#${bounce}
sender=${bounce}

#use opportunistic TLS if supported by server. set to False to disable tls
use_tls=True

#check on this TCP port. defaults to 25 (smtp)
smtp_port=25

#accept mail on temporary error (4xx) of target server, DEFER otherwise
accept_on_tempfail=True

#defer instead of accept if target server has no A records defined.
defer_on_hostnotfound=False

#defer instead of reject if target server says "Relay access denied"
defer_on_relayaccessdenied=False

#action if we don't find a server to ask
no_valid_server_fail_action=DEFER

#how long should we skiplist a recipient domain if we don't find a server to ask [seconds]
no_valid_server_fail_interval=3600

#message template template if we don't find a server to ask
no_valid_server_fail_message=${errormessage}

#action if we can't resolve target server hostname
resolve_fail_action=DEFER

#how long should we skiplist a server if we can't resolve target server hostname [seconds]
resolve_fail_interval=3600

#message template if we can't resolve target server hostname
resolve_fail_message=${errormessage}

#action if we encounter a failure before connecting to target server (continue, tempfail, reject)
preconnect_fail_action=continue

#how long should we skiplist a server if we encounter a failure before connecting to target server [seconds]
preconnect_fail_interval=3600

#message template if we encounter a failure before connecting to target server
preconnect_fail_message=

#action if we cannot connect to the target server (continue, tempfail, reject)
connect_fail_action=continue

#how long should we skiplist a server if we cannot connect to the target server [seconds]
connect_fail_interval=3600

#message template if we cannot connect to the target server
connect_fail_message=

#HELO name for smtp test (empty: uses socket.getfqdn(), string: use string directly, string starting with '$' will get environment variable
helo_name=

#action if the target server does not accept our HELO (continue, tempfail, reject)
helo_fail_action=continue

#how long should we skiplist a server if the target server does not accept our HELO [seconds]
helo_fail_interval=3600

#message template if the target server does not accept our HELO (continue, tempfail, reject)
helo_fail_message=

#action if the target server does not accept our from address
mail_from_fail_action=continue

#how long should we skiplist a server if the target server does not accept our from address [seconds]
mail_from_fail_interval=3600

#message template if the target server does not accept our from address
mail_from_fail_message=

#action if the target server show unexpected behaviour on presenting the recipient address (continue, tempfail, reject)
rcpt_to_fail_action=continue

#how long should we skiplist a server if the target server show unexpected behaviour on presenting the recipient address [seconds]
rcpt_to_fail_interval=3600

#message template if the target server show unexpected behaviour on presenting the recipient address
rcpt_to_fail_message=

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

OriginPolicy

Plugin: fuglu.plugins.originpolicy.OriginPolicy

This plugin allows to configure from which hosts you are willing to accept mail for a given domain.

Check by recipient domain (MX Rules): This can be useful if you provide shared hosting (= many domains on one mail server) and some of the domains use a cloud based spam filter (= MX records not pointing directly to your hosting server). You can reject mail coming from unexpected hosts trying to bypass the spam filter.

Check by sender domain (SPF Rules): Some domains/freemailers do not have an SPF record, although their domains are frequently forged and abused as spam sender. This plugin allows you to build your own fake SPF database.

Check forward block (FWD Rules): Some users forward abusive amounts of unproperly filtered mail. This mail is hard to filter as it’s delivered through an additional relay, leading to unnecessary high amounts of false negatives. To protect recipients and spam filter reputation such mail can be blocked.

Configuration

[OriginPolicy]
#recipient domain based rule file
datafile_mx=/etc/fuglu/conf.d/enforcemx.txt

#reject message template for mx policy violators
messagetemplate_mx=We do not accept mail for ${to_address} from ${reverse_client_address}. Please send to MX records!

#sender domain based rule file
datafile_spf=/etc/fuglu/conf.d/fakespf.txt

#reject message template for fake spf policy violators
messagetemplate_spf=We do not accept mail for ${from_domain} from ${client_address} with name ${reverse_client_name}. Please use the official mail servers!

#sender domain based rule file
datafile_fwd=/etc/fuglu/conf.d/forwardblock.txt

#reject message template for forward policy violators
messagetemplate_fwd=We do not accept forwarded mail for ${to_address} from ${reverse_client_name}.

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

SenderDomainRules

Plugin: fuglu.plugins.outpolicy.SenderDomainRules

Configuration

[SenderDomainRules]
#set to true to only log. set to false to actually reject policy violations
testmode=False

#SQLAlchemy Connection string
dbconnection=mysql://root@localhost/outbound?charset=utf8

#reject message template for policy violators
rejectmessage=${from_domain} is not in my list of allowed sender domains for account ${sasl_user}

#Interval until listings are refreshed
reloadinterval=300

#Block bounces for selected sasl users
bounceblock=True

#tagname in case of WL hit (empty: don't set, skipmplugins to skip milter plugins)
wltagname=skipmplugins

#tag content in case of WL hit (empty: don't set)
wltagvalue=

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

NoBounce

Plugin: fuglu.plugins.outpolicy.NoBounce

do not send bounces to certain recipient domains (e.g. to prevent listing on backscatter rbls)

Configuration

[NoBounce]
#list of domains to which bounces will be disallowed
nobouncefile=/etc/fuglu/nobounce.txt

#reject message template for policy violators
rejectmessage=${to_domain} does not accept bounces

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

MilterData2Header

Plugin: fuglu.plugins.outpolicy.MilterData2Header

Save specific postfix environment data in a header. Currently only supports saving sasl login user name. Run this plugin in a milter mode fuglu to read data in e.g. a subsequently running after queue fuglu. Consider removing headers after reinjection into postfix.

Configuration

[MilterData2Header]
#Name of header to store sasl login user name
headername_sasluser=X-SASL-Auth-User

AccessRestrictions

Plugin: fuglu.plugins.restrictions.AccessRestrictions

Configuration

[AccessRestrictions]
#access restrictions yaml-file with "restrictions"-array and "setup"-dict
restrictionfile=/etc/fuglu/accessrestrictions.yml

#Delay reject to this state, empty means immediate reject
delayreject=rcpt

#If defined, only run plugin end-of-message if header is present
eom_trigger_header=

#comma/space separated list of milter states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh)
state=connect,helo,mailfrom,rcpt,header,eoh

EnforceTLS

Plugin: fuglu.plugins.tlspolicy.EnforceTLS

set TLS policy by recipient domain. Allows to enforce TLS on per-recipient base

Configuration

[EnforceTLS]
#
#                if this is empty, all recipient domains will be forced to use TLS
#                txt:<filename> - get from simple textfile which lists one domain per line
#                sql:<statement> - get from sql database :domain will be replaced with the actual domain name. must return field enforce_inbound_tls
#
domainlist=

#SQLAlchemy Connection string
dbconnection=mysql://root@localhost/enforcetls?charset=utf8

#Action if connection is not TLS encrypted. set to continue, tempfail, reject
action=tempfail

#reject message template for policy violators
messagetemplate=Unencrypted connection. This recipient requires TLS

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

TLSSender

Plugin: fuglu.plugins.tlspolicy.TLSSender

set TLS policy by sender domain. Allows to enforce TLS on per-sender base

Configuration

[TLSSender]
#path to file with sender domains that must use TLS
domains_file=/etc/fuglu/tls-senders.txt

#Action if connection is not TLS encrypted. set to continue, tempfail, reject
action=reject

#reject message template for policy violators
messagetemplate=Unencrypted connection. This sender must use TLS

#comma/space separated list states this plugin should be applied (connect,helo,mailfrom,rcpt,header,eoh,eob)
state=rcpt

Block and welcome list evaluation plugin

Plugin: fuglu.plugins.p_blwl.BlockWelcomeList

This plugin evaluates block and welcome lists, e.g. from spamassassin userprefs. Respective tags if a sender/recipient combination is welcome listed or block listed are written. use e.g. p_skipper.PluginSkipper to skip certain checks or create a custom plugin to decide further action

Configuration

[BlockWelcomeList]
#comma separated list of backends to use. available backends: userpref, filtersettings
blwl_backends=

#sqlalchemy db connect string, e.g. mysql:///localhost/spamassassin
userpref_dbconnection=

#Use Mem Cache. This is recommended. However, if enabled it will take up to userpref_cache_ttl seconds until listing changes are effective.
userpref_usecache=True

#how long to keep userpref data in memory cache
userpref_cache_ttl=300

userpref_block_before_welcome=False

userpref_eval_order=user,domain,global

#name of config section
section_name=FilterSettings

#options that act like "welcomelist_to" (boolean values only)
welcome_options=disable_filter

#options that enable or disable a feature (boolean values only)
yesno_options=deliver_spam, deliver_highspam, block_before_welcome

#options that set a level or threshold (numeric values only)
level_options=subject_tag_ext_level

#options that describe or configure other settings (text values, allows template variables like ${to_address})
value_options=spam_recipient, subject_tag_ext, subject_tag_spam

#default subject tag for tagging external messages
subject_tag_ext=[EXTERNAL]

#when to tag external messages: 0 never, 1 always, 2 when sender domain equals recipient domain
subject_tag_ext_level=0

#default subject tag for tagging spam messages
subject_tag_spam=[SPAM]

#default subject tag for tagging spam messages
spam_recipient=${to_address}

Plugin Skipper

Plugin: fuglu.plugins.p_skipper.PluginSkipper

Skips plugins based on standard filter file

This can be used for example to skip spam filters on outgoing messages. e.g. put this in /etc/fuglu/skipplugins.regex:

@incomingport 1099 SAPlugin

Configuration

[PluginSkipper]
#path to file containing scanner plugin skip regex rules
filterfile=/etc/fuglu/skipplugins.regex

Plugin Fraction

Plugin: fuglu.plugins.p_fraction.PluginFraction

Runs only a fraction of loaded scanner plugins based on standard filter file Use this if you only want to run a fraction of the standard plugins on a specific port for example eg. put this in /etc/fuglu/pluginfraction.regex:

@incomingport 1100 SAPlugin,AttachmentPlugin

Configuration

[PluginFraction]
#path to file containing scanner plugin fraction regex rules
filterfile=/etc/fuglu/pluginfraction.regex

Debugger

Plugin: fuglu.plugins.p_debug.MessageDebugger

Message Debugger Plugin (Prepender).

This plugin enables the fuglu_debug functionality. Make sure fuglu listens on the debug port configured here.

Configuration

[MessageDebugger]
#messages incoming on this port will be debugged to a logfile
#Make sure the debugport is also set in the incomingport configuration option in the main section
debugport=10888

#debug log output
debugfile=/tmp/fuglu_debug.log

#debugged message can not be bounced
nobounce=True

#don't re-inject debugged messages back to postfix
noreinject=True

#don't run appender plugins for debugged messages
noappender=True

Statsd Sender: Plugin Time

Plugin: fuglu.plugins.a_statsd.PluginTime

EXPERIMENTAL: Send Plugin execution time to a statsd server

Configuration

[PluginTime]
#statsd host
host=127.0.0.1

#statsd port
port=8125

Statsd Sender: Global Message Status

Plugin: fuglu.plugins.a_statsd.MessageStatus

EXPERIMENTAL: Send message status to a statsd server

Configuration

[MessageStatus]
#statsd host
host=127.0.0.1

#statsd port
port=8125

Statsd Sender: Per Recipient Message Status

Plugin: fuglu.plugins.a_statsd.MessageStatusPerRecipient

EXPERIMENTAL: Send per recipient stats to a statsd server

Configuration

[MessageStatusPerRecipient]
#statsd host
host=127.0.0.1

#statsd port
port=8125

#domain: send stats per recipient domain. email: send stats per recipient email address
level=domain

ElasticLogger

Plugin: fuglu.plugins.a_logging.ElasticLogger

write fuglu log data directly to elasticsearch all data related to a suspect is written to one elasticsearch document, what data exactly will be logged can be configured to a certain degree if write to elasticsearch fails, optionally fallback logging to local json files can be enabled. these json files can later be reimported.

Configuration

[ElasticLogger]
#comma separated list of ElasticSearch host definition (hostname, hostname:port, https://user:pass@hostname:port/)
elastic_uris=

#Name of ElasticSearch index in which document will be stored. Template vars (e.g. ${to_domain} or ${date}) can be used.
elastic_index=fuglulog-${date}

log_headers=from:addr,to:addr,reply-to:addr,subject,subject:hash:md5,message-id

log_tags=injectqueueid,fuzor.digest,log.scanhost,log.decision,log.clienthelo,log.clientip,log.clienthostname,archived,Attachment.bounce.queueid,spf.result,dkim.result,arc.result,dmarc.result,log.real_from_address

#log raw headers. set to "all" for all headers or comma separated list of header names to be logged
log_raw_headers=all

#log URIs listed in given tags
log_email_tags=body.emails

#log URIs listed in given tags
log_uri_tags=body.uris,uris.safelinks,headers.uris

#log attachment information
log_attachments=True

#path to file with extra TLDs (2TLD or inofficial TLDs)
extra_tld_file=

#path to directory where logs are stored in case of elasticsearch connection/indexing failure
fallback_logdir=/usr/local/fuglu/maillog/

Extra plugins

Note, in addition to the plugins included in the fuglu distribution there are additional contributed plugins available in the fuglu-extra-plugins repository: https://gitlab.com/fumail/fuglu-extra-plugins

Writing your own plugins

Assuming you know python basics, writing plugins for fuglu is very easy. All you have to do is create a new class which extends from ScannerPlugin, override __str__ to provide a nice human readable name and override examine to do the actual work of your plugins. examine should return one of the action codes above (DUNNO, DEFER, DELETE, ….) and optionally a reason for the action. (e.g. return DEFER, 'please try again later' In plugin you usually only have to import things from fuglu.shared , so it’s probably a good idea to get familiar with that module.

This is a quick example of how your plugin code skeleton would look like:

from fuglu.shared import ScannerPlugin,DUNNO

class DemoPlugin(ScannerPlugin):
    """Copy this to make a new plugin"""
    def __init__(self,config,section=None):
        ScannerPlugin.__init__(self,config,section)
        #config example
        #self.requiredvars={
        #    'maxsize':{
        #        'default':'1337',
        #        'description':'Maximum message size',
        #    }
        #}

    def examine(self,suspect):
        #config Example
        #maxsize=self.config.getint(self.section, 'maxsize')

        #debug example
        #self._logger().debug('hello world from DemoPlugin')

        #PUT PLUGIN CODE HERE

        return DUNNO

First of all, you need a few imports. ScannerPlugin (so you can extend from it), and possible return values for your Plugin, DUNNO might be enough depending on what your plugin does.

from fuglu.shared import ScannerPlugin,DUNNO

in __init__ you only call BasicPlugin’s __init__ for now. This sets self.config and self.section on the object where you later can read config options (eg. self.config.get(self.section,'somevalue'). Do NOT load the plugin configuration here. __init__ is only called once when fuglu starts up. Always load plugin config in examine.

def __init__(self,config,section=None):
    ScannerPlugin.__init__(self,config,section)

then code your examine method. You will have to work with The Suspect class, which is a representation of the message being analyzed. The suspect has tags that are read and written by plugins. You can tag a message as virus, as spam, define your own tags, read tags from previous plugins… it’s probably a good idea to look at the Suspect class in fuglu.shared to get a list of things you can do with the suspect.

Common Tasks (“API” FAQ)

Define configuration options for your plugin

In order to make ‘lint’ and ‘fuglu_conf’ work with your plugin it should tell the core what config options it expects. this is done by creating a dictionary named ‘requiredvars’ in the plugins init:

Example:

def __init__(self,config,section=None):
        ScannerPlugin.__init__(self,config,section)
        self.requiredvars={
            'host':{
                'default':'localhost',
                'description':'hostname',
            },

            'username':{
                'default':'root',
            },

            'password':{
                'default':``,
                'confidential':True,
            },

       }

This would tell fuglu that your plugin has three config options: host, username and password.

The ‘dict of dicts’ uses your config option name as key for the outer dict. The inner dict supports the following keys:

  • default - default value, used if the option is not specified in the config file

  • section - config section to check. by default fuglu assumes that the plugin reads its own config section. override this if your plugin requires a config option from a different plugin or from the main config

  • confidential - set this to True if fuglu_conf should treat this option confidential and redact it in ‘fuglu_conf -n’ (passwords etc)

  • validator - function that should be called to validate if the configured value is valid. the function will receive the value as argument and must return True or False

  • deprecated - mark a config option as deprecated

Read the config

Configs in fuglu are stored in /etc/fuglu/fuglu.conf (or any .conf file in /etc/fuglu/conf.d ) in ini-style format. Your plugin gets its own section named like the plugin class.

Example:

[DemoPlugin]
maxsize=10382

You can then read the config in your plugin with the usual methods from a python ConfigParser object ( http://docs.python.org/library/configparser.html )

maxsize=self.config.getint(self.section, 'maxsize')

Important: always load the configs in examine (and not in init !). Reading the config in init breaks loading default values and prevents on-the-fly config reload

Get the message source

Use suspect.get_source() to get the message source. The maxbytes option allows you to get only a part of the source. Reading the whole source can slow down scanning of big messages.

Add headers

suspect.add_header(headername,headervalue)

By default, headers are added to the message shortly before it is re-injected into postfix. add immediate=True to add the header immediately, so other plugins can see it.

Replace headers

Here’s an example on how to change the subject of a message. Note: It’s generally not recommended to change existing headers, this breaks DKIM.

msgrep = suspect.get_message_rep()
msgrep.replace_header('Subject', 'Scanned: '+msgrep['Subject'])
suspect.set_message_rep(msgrep)

Write to the log

Your plugin has a _logger method that returns a standard python logger object, so you can use the info / debug / error / fatal methods.

Example:

self._logger().info('hello world from DemoPlugin')

Write debug info

To make the plugin write special debug info when fuglu_debug is used, use:

suspect.debug('debug info from DemoPlugin!')

Make plugin ‘–lint’-able

By default, lint() only validates the plugin’s configuration settings from self.requiredvars. You can override lint() to do more stuff.

  • use simple print in this method, no logging stuff.

  • if you override lint() you should run self.checkConfig() to test the configuration

  • lint() must return True or False

Example of a plugin that would check if an imap account is available:

def lint(self):
    allok=(self.checkConfig() and self.lint_imap())
    return allok

def lint_imap(self):
    try:
        M=imaplib.IMAP4(self.config.get(self.section, 'host'))
        (type,data)=M.login(self.config.get(self.section, 'user'),self.config.get(self.section, 'password'))
        if type!='OK':
            print('Could not login to imap review account: %s',data)
            return False
        return True
    except Exception, e:
        print "Could not login to imap host:%s - Error %s"%(self.config.get(self.section, 'host'),e)
    return False

Use the ‘SuspectFilter’

SuspectFilters are a common way for all plugins to perform an action based on conditions defined in a filterfile . These files are automatically re-loaded if changed.

  • import SuspectFilter from fuglu.shared

  • define a config variable for your plugin which holds the name of the filter file (not strictly required, you could hardcode the path)

  • create a plugin property for the filter

from fuglu.shared import ScannerPlugin,SuspectFilter
[...]

class MyPlugin(ScannerPlugin):
    def __init__(self,config,section=None):
        ScannerPlugin.__init__(self,config,section)

        self.requiredvars={
            'myrulesfile':{
                'default':'/etc/fuglu/mypluginrules.regex',
                'description':'Filter file for my plugin',
            },

           [...]
        }
        self.filter=None

In examine create the filter if necessary

if self.filter==None:
    self.filter=SuspectFilter(self.config.get(self.section, 'myrulesfile'))

Run the filter in examine: (match,arg)=self.filter.matches(suspect) match is a boolean, telling you if one of the rules matched arg` is an additional argument which have been appended to the filter rule in the config. lets say, the filter rule reads to_address example@fuglu.org hello world!, you would get match=True and arg=’Hello world!’ if the message is sent to example@fuglu.org

(match,arg)=self.filter.matches(suspect)
if match:
    if arg!=None:
        self.logger.debug("""we got a match with arg %s"""%arg)
    else:
        self.logger.debug("""We got a match without an arg""")
else:
    suspect.debug("no rule matches")

Use the sql extension

TODO (DBFiles, sqlalchemy connections)

Debugging

Get a stacktrace

if something went wrong you should see a stacktrace in the fuglu log (/var/log/fuglu/fuglu.log) with fuglu >=0.6.0 you can also get the most recent exceptions with the following command:

fuglu_control exceptionlist

Debug the plugin while fuglu is runnig

run fuglu --console to enter an interactive python console after fuglu startup. Your plugin is then available via the list mc.plugins

Debug a plugin without running fuglu

plugdummy.py is a tool that makes plugin development and testing much easier by creating a minimal fuglu environment for the plugin to run. it doesn’t require a running fuglu or postfix. it will create a dummy suspect, call the plugin’s examine method and print the result (and debug output).

the generated input messag is stored as: /tmp/fuglu_dummy_message_in.eml

if your plugin modified the message source, the resulting message can be found at /tmp/fuglu_dummy_message_out.eml

plugdummy.py is located in the develop/scripts directory.

simple usage:

assuming your plugin file (‘myplugin.py’) is in /usr/local/fuglu/plugins you can run plugdummy.py <pluginname>

#./plugdummy.py myplugin.ExamplePlugin

INFO:root:Input file created as /tmp/fuglu_dummy_message_in.eml
INFO:root:Running plugin: ExamplePlugin
INFO:fuglu.plugin.ExamplePlugin:sender@fuglu.local greets recipient@fuglu.local: hello world!
INFO:root:Result: DUNNO
INFO:root:Suspect a7babc1e4cfe49c36710065966e6ed0a: from=sender@fuglu.local to=recipient@fuglu.local size=231 spam=no virus=no modified=no tags={'virus': {}, 'spam': {}, 'highspam': {}}

Advanced usage:

run plugdummy.py --help to get a list of all options

Examples:

Running plugins from a different directory

./plugdummy.py -p /tmp/ myplugin.ExamplePlugin

Change sender / recipient the ‘-s’ and ‘-f’ options change the envelope sender/recipient. -r can be specified multiple times to simulate a multi-recipient message

./plugdummy.py -s me@example.org -r you@example.net  myplugin.ExamplePlugin
INFO:root:Input file created as /tmp/fuglu_dummy_message_in.eml
INFO:root:Running plugin: ExamplePlugin
INFO:fuglu.plugin.ExamplePlugin:me@example.org greets you@example.net: hello world!
INFO:root:Result: DUNNO
INFO:root:Suspect 423e02e1461cd1c314ac9a409176c4f4: from=me@example.org to=you@example.net size=221 spam=no virus=no modified=no tags={'virus': {}, 'spam': {}, 'highspam': {}}

Adding headers to the input message

./plugdummy.py -h 'subject:yo! whassup' myplugin.ExamplePlugin
[...]
cat /tmp/fuglu_dummy_message_in.eml
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: sender@fuglu.local
To: recipient@fuglu.local
Subject: yo! whassup
Date: Fri, 01 Jun 2012 12:58:34 -0000

hello, world!

Adding tags:

./plugdummy.py -t 'mytag:myvalue' myplugin.ExamplePlugin
INFO:root:Input file created as /tmp/fuglu_dummy_message_in.eml
INFO:root:Running plugin: ExamplePlugin
INFO:fuglu.plugin.ExamplePlugin:sender@fuglu.local greets recipient@fuglu.local: hello world!
INFO:root:Result: DUNNO
INFO:root:Suspect 168268d6ff2c2748454183efcb554242: from=sender@fuglu.local to=recipient@fuglu.local size=231 spam=no virus=no modified=no tags={'spam': {}, 'virus': {}, 'mytag': 'myvalue', 'highspam': {}}

Setting a config option:

./plugdummy.py -o 'greeting:go away!'  myplugin.ExamplePlugin
INFO:root:Input file created as /tmp/fuglu_dummy_message_in.eml
INFO:root:Running plugin: ExamplePlugin
INFO:fuglu.plugin.ExamplePlugin:sender@fuglu.local greets recipient@fuglu.local: go away!
[...]

Setting the message body:

#set body
./plugdummy.py -b 'hi there, whassup!' myplugin.ExamplePlugin

#read body from file
./plugdummy.py -b bla.txt myplugin.ExamplePlugin

#read headers & body from eml file
./plugdummy.py -e /tmp/bla.eml myplugin.ExamplePlugin

Running a interactive console in the dummy enrivonment:

./plugdummy.py -c  myplugin.ExamplePlugin
INFO:root:Input file created as /tmp/fuglu_dummy_message_in.eml
INFO:root:Running plugin: ExamplePlugin
INFO:fuglu.plugin.ExamplePlugin:sender@fuglu.local greets recipient@fuglu.local: hello world!
INFO:root:Result: DUNNO
INFO:root:Suspect 3cf496cbe2a1097abc37ebda5a645cd2: from=sender@fuglu.local to=recipient@fuglu.local size=231 spam=no virus=no modified=no tags={'virus': {}, 'spam': {}, 'highspam': {}}
Fuglu Interactive Console started

pre-defined locals:
{'config': <ConfigParser.ConfigParser instance at 0x1ac9e60>, 'suspect': <fuglu.shared.Suspect object at 0x1ac8750>, 'result': 0, 'plugin': <myplugin.ExamplePlugin object at 0x1ac8290>}

>>> plugin.requiredvars
{'greeting': {'default': 'hello world!', 'description': 'greeting the plugin should log to the console'}}
>>> plugin.examine(suspect)
INFO:fuglu.plugin.ExamplePlugin:sender@fuglu.local greets recipient@fuglu.local: hello world!
0
>>> config.set('ExamplePlugin','greeting','Greetings, Earthling')
>>> suspect.from_address='me@example.org'
>>> plugin.examine(suspect)
INFO:fuglu.plugin.ExamplePlugin:me@example.org greets recipient@fuglu.local: Greetings, Earthling
0
>>>

Deploying Plugins

  • If there is no plugindir set in fuglu.conf yet, define a new directory for custom plugins. eg /usr/local/fuglu/plugins.

  • Copy your plugin file to this directory

  • Depending on the type of your plugin, add it to the plugin/prependers/appenders config option. Eg. if your scanner plugin class is MyHeloPlugin in the file myplugin.py you would add myplugin.MyHeloPlugin to the plugins config

  • If your plugin reads configuration entries, make sure those are present in fuglu.conf or in a custom conf-file in /etc/fuglu/conf.d

  • Run fuglu --lint to check if fuglu is happy with your new plugin

  • (Re)start fuglu