Beta functions

`beta.ml_extract_sensitive_information`

beta.ml_extract_sensitive_information(text: string) -> SensitiveInformationResult

Analyzes input text and identifies sensitive entities like financial account numbers, national ID and tax numbers, passports, driver's licenses, healthcare identifiers, credentials/secrets, and a few personal identifiers.

Each identification comes with a confidence level. Input under 20 characters returns no matches, unless text is one of the message's core content fields (body.current_thread.text, body.html.display_text, body.plain.raw, subject.subject, _meta.canonical_id). Input over 100,000 characters is truncated before scanning; check .truncated on the result if completeness matters for a given rule.

// flag a US Social Security number, medium or high confidence
any(beta.ml_extract_sensitive_information(body.current_thread.text).elements,
  .type == "us_social_security_number" and .confidence in ("medium", "high")
)

// flag a credit card number, crypto wallet address, US bank account number, or IBAN
any(beta.ml_extract_sensitive_information(body.current_thread.text).elements,
  .type in ("credit_card_number", "crypto_wallet_address", "us_bank_account", "iban")
  and .confidence in ("medium", "high")
)

SensitiveInformationResult object

.elements array of objects
- .text string - the exact substring that matched
- .type string - the entity type detected (see table below)
- .confidence string - low, medium, or high
- .context string - surrounding lines containing the match
.success boolean - whether the scan completed
.truncated boolean - whether input was cut to the 100,000-character limit before scanning

Supported entity types

at_vat
au_bank_account
au_business_number
au_driver_license
au_medicare_number
au_passport_number
au_tax_file_number
aws_access_key_id
aws_secret_access_key
be_vat
blood_test_term
ca_bank_account
ca_driver_license
ca_health_service_number
ca_passport_number
ca_personal_health_identification_number
ca_social_insurance_number
cn_resident_identity_card_number
credit_card_number
crypto_wallet_address
date_of_birth
de_driver_license
de_identity_card_number
de_passport_number
de_tax_identification_number
de_vat
email_address
es_bank_account
es_national_id
es_passport_number
fda_drug_name
fi_european_health_insurance_number
fi_personal_identity_code
fr_bank_account
fr_driver_license
fr_identity_card_number
fr_passport_number
fr_social_security_number
fr_vat
github_token
google_api_key
http_authorization_header
hu_social_security_number
hu_vat
iban
ip_address
jp_driver_license
jp_individual_number
jp_passport_number
json_web_token
kr_resident_registration_number
lu_legal_entity_identification_number
mac_address
medical_condition
medical_specialty
nl_citizen_service_number
nl_vat
nz_bank_account
nz_national_health_index_number
pem_certificate
phone_number
pl_business_registry_number
pl_driver_license
private_key
se_personal_identity_number
sg_foreign_identification_number
sg_national_id
sg_unique_entity_number
slack_access_token
sublime_test_string
surgical_procedure
swift_bic
uk_driver_license
uk_electoral_roll_number
uk_national_insurance_number
uk_nhs_number
uk_passport_number
uk_unique_taxpayer_reference
us_aba_routing_number
us_bank_account
us_dea_number
us_disability_impairment
us_driver_license
us_icd10
us_icd9
us_individual_taxpayer_identification_number
us_medicare_beneficiary_identifier
us_national_provider_identifier
us_passport_number
us_physical_address
us_social_security_number
vehicle_identification_number