How Personalization Works

Different people want different things from their inbox. A newsletter one person reads every day is junk to someone else. This page explains how SpamFoo adjusts to each user, so you can understand results and answer questions about personal data.

The Two Parts of Personalization

  • Spam Rules: Instructions people set by hand, such as "mail from this sender is always spam". They are simple and predictable.
  • Pattern memory: What SpamFoo learns from a user's corrections. It is built for you automatically when you mark spam, rescue mail from spam, or move a message to a different classification.

Both are stored on your server and apply only to the people they were made for.

Spam Rules

  • A Spam Rule says mail from a sender is always Spam or always Not Spam.
  • A rule can match one email address, a whole domain, or a wildcard pattern. Administrators can also match IP address ranges.
  • Users manage their own rules in the Spam Rules list on the My Rules page. The same list shows the Domain Spam Rules and System Spam Rules that apply to the user, marked under Set By. Only administrators can change those.
  • Spam Rules only decide spam or not spam. They don't choose an Email Classification.
  • When several rules match, a rule made by a system administrator beats one made by a domain administrator, which beats one made by a user. See When Rules Disagree.

To add or change rules, see Manage Your Rules.

Pattern Memory

Spam Rules only cover senders someone named. Pattern memory covers everything else.

  • When a user corrects a message, SpamFoo stores a fingerprint of it with the correct answer. A fingerprint is a list of numbers that captures what the message is about. The message's words can't be read back from it.
  • When new mail arrives for that user, SpamFoo compares it to the stored fingerprints. If it closely matches messages the user already corrected, SpamFoo follows those corrections.
  • Pattern memory updates immediately, so a correction applies to the next similar message in that mailbox.

Spam Memory

  • One correction is enough for SpamFoo to handle similar mail from the same sender the same way.
  • Mail from other senders is only affected when it is nearly identical to messages the user corrected. Even then, the spam models usually still have a say.

Classification Memory

  • Moving a message to Primary, Transactions, Updates, or Promotions teaches SpamFoo where similar mail from that sender belongs, and very similar mail from other senders can follow.
  • Other kinds of mail from the same sender aren't affected. Moving an Amazon newsletter to Updates doesn't move Amazon receipts.
  • When a message marked as spam is reported as a classification in the SpamFoo dashboard, spam memory also learns that it isn't spam.

How Memory Stays Current

  • Each user's memory holds a limited number of corrections. When it is full, the oldest are dropped, so the memory reflects recent preferences.
  • If a user later corrects a message the opposite way, older memories that disagree lose weight and are eventually removed.

Where Personalization Fits

  • A user's own Spam Rules and Domain Spam Rules are checked first and decide right away.
  • Pattern memory is checked after the phishing checks and before sender reputation and the spam models.
  • System Spam Rules are applied last. A System Spam Rule that says Spam always wins. One that says Not Spam gives way when the user's pattern memory for that same sender decided the message.

For the full order, see How SpamFoo Classifies Email.

See and Clear What SpamFoo Learned

  • The My Rules page lists what SpamFoo learned under Trained Rules. You can remove any item so SpamFoo stops using it.
  • Clear All Personalization Data on the same page removes everything SpamFoo learned for the user. It doesn't remove the user's Spam Rules or message history.
  • Developers can clear a user's data through the API. See Manage Personal Data.

Where the Data Is Stored

  • Pattern memory is stored in the personalization folder of the SpamFoo data directory. When SmarterMail runs SpamFoo, it is stored in a spamfoo folder inside each user's SmarterMail folder instead.
  • Spam Rules are stored in SpamFoo's local database, admin.db, in the data directory.
  • Corrected messages: SpamFoo also keeps a copy of a limited number of recently corrected messages in the correction_archive folder of the data directory. When the limit is reached, the oldest copies are removed. Clearing personalization data doesn't remove these copies.

None of this is sent to SpamFoo's servers as part of personalization. To keep it across a reinstall, back up the data directory. See Update and Back Up.