Profile functions

📘
Behavior of historical functions
The result of historical functions is always relative to the time of the message that is being evaluated. During live processing, this means the latest possible information is available. However, during a backtest, these functions only take into account messages that are seen prior to that point in time. If there's not enough data, some fields like .prevalence may be "unknown". This behavior ensures that during a backtest there's never access to "future" data, which would lead to incorrect results and a false sense of confidence in the efficacy of a rule.
Results are typically and deliberately delayed by several hours, so that the prevalence of a sender can remain as "new" for approximately 8-12 hours.

`profile.by_sender`

profile.by_sender() -> SenderProfile

profile.by_sender uses previously ingested inbound messages to build a profile for messages received from a matching Sender. This profile captures information like the .prevalence of the sender domain within your environment to assess how common or uncommon it is across messages. It also captures information about flagged messages, such as false positives or true positives.

For the profile.by_sender function, the list $free_email_providers is used to determine whether a sender means a matching email or domain. If the value of sender.email.domain.domain is in $free_email_providers, then sender.email.email is used to determine a matching Sender. Otherwise, all messages with a matching sender.email.domain.domain are considered to be from the same Sender. This ensures that for profile.by_sender, a matching Sender covers messages from an organization, instead of an individual.

Using profile.by_sender() to find a first-time sender:

type.inbound
and profile.by_sender().prevalence == "new"

Using lists to find a first-time sender is the same but more verbose:

type.inbound
and (
  (
    sender.email.domain.root_domain in $free_email_providers
    and sender.email.email not in $sender_emails
  )
  or (
    sender.email.domain.root_domain not in $free_email_providers
    and sender.email.domain.domain not in $sender_domains
  )
)

To check against the historical reputation for a sender, check whether a sender has sent at least 1 message flagged as malicious or spam but no confirmed false positives.


type.inbound
and profile.by_sender().any_messages_malicious_or_spam
and not profile.by_sender().any_messages_benign

// Additional logic on the suspicious sender.
and ...

Two more sender profile functions: profile.by_sender_domain and profile.by_sender_email exist if the automatic switching between email and domain is not preferred.

`profile.by_sender_domain`

profile.by_sender_domain() -> SenderProfile

profile.by_sender_domain uses previously ingested inbound messages to build a profile for messages received from a matching sender.email.domain.domain.

type.inbound

// filter by first-seen domains or anomalous domains in your environment
and profile.by_sender_domain().prevalence in ("outlier", "new")

// scrutinize PDF attachments, for example
and any(attachments, .file_extension == "pdf" and ...)

`profile.by_sender_email`

profile.by_sender_email() -> SenderProfile

profile.by_sender_email uses previously ingested inbound messages to build a profile for messages received from a matching sender.email.email.

type.inbound

// filter by first-seen or anomalous email addresses in your environment
and profile.by_sender_email().prevalence in ("outlier", "new")

Together, profile.by_sender_domain and profile.by_sender_email can be used to tell when a domain is common but the sending email address is new:

type.inbound

// not a free email provider
and sender.email.domain.domain not in $free_email_providers

// domain is common in your environment
and profile.by_sender_domain().prevalence == "common"

// but this is the first time you've received messages from this sender
and profile.by_sender_email().prevalence == "new"