Obscured Email Addresses

• Table of Contents
• Why use obscured email addresses?
• What does this protect?
• Obscured Email Address
• Limitations
  • Marketing Studio Remote List Event Tracking
  • Bad Address Suppression
  • Could Not Process Bounce Messages
  • Event Processor
  • Redis
  • Per Envelope Recipient Tracking
• Internal Error

The email address written to GreenArrow’s log files and database can be replaced with an obscured address.

• For SimpleMH, this is done with the X-GreenArrow-Obscured-Email header.
• For Marketing Studio Remote Lists, this is done with the obscured_email column.

When providing an obscured address, the real recipient address will be delivered to as normal. However, all logging to disk and the GreenArrow database will be done using the obscured email address.

Why use obscured email addresses?

For some installations, you may want to limit the amount of recipient data written to disk by GreenArrow.

Some examples of why you may want to use this feature:

• To ease GDPR compliance.
• You want to deliver email without increasing the surface area of software that stores your recipient addresses.

What does this protect?

When an obscured email is provided, that address will be used (instead of the recipient’s real email address) in the following areas:

• Email addresses stored by GreenArrow Engine:
  • Delivery attempt logs
  • Web server logs
  • Bounce processor logs
  • Event Processor logs
  • Other service logs – excluding SMTP server logs which only log email addresses if configured
  • SimpleMH and Studio Remote List recipient addresses
  • SimpleMH click and open tracking
  • The bounce_bad_addresses table
  • Bounce processor repeat bounce tracking
  • Events
  • Send Summary files
• Email addresses stored by GreenArrow Studio:
  • Campaign statistics for Remote Lists
  • Special Sending Rule errors

The real subscriber email address can appear on disk briefly over the course of delivery in the following areas:

• The disk-queue
  • Adjust the Queue Lifetime setting to reduce the amount of time the address can appear on disk.
• The Redis AOF and RDB files in /var/hvmail/data/redis
  • Redis is used during Studio and SimpleMH delivery. As soon as the first delivery attempt occurs, the address is removed from Redis. However, it can take some time before the removal is reflected on disk. See the “Limitations” section below for information on reducing how long it is before these files are rewritten.
• Marketing Studio campaign log files in /var/hvmail/var/tmp
  • Most campaigns delete these files after they finish.
  • Campaigns that fail due to software errors can retain their log files on disk.

Real subscriber email addresses that exist in the ram-queue and bounce-queue are moved from RAM to the following disk locations when GreenArrow is stopped, and back to RAM when GreenArrow is started:

• /var/hvmail/qmail-ram/savedqueue
• /var/hvmail/qmail-bounce/savedqueue

The above temporary disk locations may end up in backups, if a backup is taken while they are present.

Obscured Email Address

The obscured email can be provided in a few different formats, depending on what data you wish to retain in GreenArrow’s logging and database. The table below describes what happens in each possible situation.

The resulting email that is used for logging and in the database always ends in either @_anon_ or ._anon_.

Limitations

Marketing Studio Remote List Event Tracking

In order for events (clicks, opens, unsubs, etc) to be tracked properly in Marketing Studio while using this feature, you should include a distinct_id in your remote query.

The Event Notification System will deliver its events including both the obscured email address and this distinct_id.

Bad Address Suppression

SimpleMH Bad Address Suppression does not work on messages injected using this feature. This is because SimpleMH Bad Address Suppression works using the recipient’s email address, but data is written to bounce_bad_addresses using the obscured email.

Marketing Studio Suppression Lists work for Studio Remote Lists, but we recommend writing your suppression lists using MD5 or SHA256 suppression to avoid writing those addresses to disk.

Could Not Process Bounce Messages

The GreenArrow bounce processor will ordinarily write messages which it cannot process to disk.

See the write_could_not_process_bounce_messages configuration directive to disable this behavior.

Event Processor

The Event Notification System can include recipient email addresses in the bounce_text column. This applies to events of type bounce_all and bounce_bad_address.

To mitigate this:

• Set bouncemaxbytes to 1 byte (since 0 means unlimited).
• Set store_bounce_details_length to 0.

Redis

GreenArrow uses an internal Redis server for its delivery system. Redis’s data can temporarily contain recipient email addresses.

To mitigate this, you can instruct Redis to rewrite its AOF file more frequently, causing this data to be retained on disk for less time.

cat >> /var/hvmail/control/redis.conf <<EOF
auto-aof-rewrite-percentage 50
auto-aof-rewrite-min-size 10mb
EOF

Per Envelope Recipient Tracking

When using obscured email addresses and the simplemh_track_per_envelope_recipient directive together, only messages to the first recipient email address (typically the first TO address) will be obscured. Secondary recipients (i.e. subsequent envelope recipients such as CC or BCC) will not be obscured.

Internal Error

If an internal error is encountered while processing obscured email addresses, a bounce will be written to the address [email protected].

This error comes as a result of file corruption or a communication error within GreenArrow. This is not a normal situation, but is documented here to aid in troubleshooting this situation.