How Personalization Works

Different people want different things from their inbox. A newsletter one person reads daily is junk to someone else. SpamFoo learns each user's preferences from their corrections and applies them the next time mail arrives. All personalization data is stored locally on your server.

Personalization uses two mechanisms: rules and pattern memory. This page explains what each one does and how they are created, so you can answer user questions and manage the data.

Where Personalization Fits

When SpamFoo classifies a message for a user, it checks in this order. The first match wins:

  1. Phishing checks (see Phishing Protection)
  2. The user's rules. A matching rule decides the result immediately, overriding the ML models.
  3. The user's pattern memory. If the message strongly resembles messages the user has corrected before, it follows those corrections.
  4. The global ML models. Used when nothing personal matches.

The API response shows when personalization decided the result: bypassReason is user_rule when a rule matched, or knn_memory when pattern memory matched.

Rules

A rule is a simple, predictable instruction: messages matching this pattern always get this result. There are four rule types, checked in this order:

Type What It Matches
Thread Replies in a specific email conversation
List Messages from a specific mailing list
Template Messages with the same structure and formatting (such as a recurring automated notice)
Sender Messages from a specific sender

For spam filtering, a sender rule normally matches the sender's whole domain. Senders from free mail providers (such as public webmail services) are matched by their full address instead, so one bad sender does not block the whole provider. Classification sender rules always match the full address.

How Rules Are Created

  • Directly. A user or your mail server creates a rule through the Corrections API, or an admin manages rules in the admin console.
  • Automatically. When a user corrects the same kind of message several times in a row with the same answer (by default, three matching corrections within seven days), SpamFoo creates the rule for them. This way users get rules without thinking about them.

A user can undo a classification sender rule by setting the message classification back to automatic, which removes the rule and returns that sender to normal model-based classification.

Pattern Memory

Rules only cover exact patterns. Pattern memory covers everything else. When a user corrects a message, SpamFoo stores a compact mathematical fingerprint of it (an embedding) along with the correct answer. The original message text is not stored and cannot be rebuilt from the fingerprint.

When new mail arrives, SpamFoo compares it to the user's stored fingerprints. If it closely resembles messages the user already corrected, it follows those corrections.

  • Spam memory compares only against corrected messages from the same sender. This makes it precise: a sender the user marked as spam several times stays out, even when the message text varies.
  • Classification memory compares across all of the user's corrections. Moving a few newsletters to Updates teaches SpamFoo where similar newsletters belong, even from new senders.

Each user's memory holds a limited number of entries. When it is full, the oldest entries are dropped, so the memory reflects the user's recent preferences.

Storage and Privacy

Personalization data lives under {data-dir}/personalization/ on your server. It contains rules, pattern fingerprints, and hashes. It does not contain readable email content. Include this directory in backups if you want personalization to survive a reinstall.

Managing Personalization Data

  • List or remove a user's rules through the Corrections API or the admin console.
  • Delete all of a user's personalization data with DELETE /personalization/data (spam) and DELETE /personalization/tabs/data (classification). Use these for data deletion requests.
  • Check the size of a user's pattern memory with GET /personalization/knn/stats and GET /personalization/tabs/knn/stats.

Next Steps