GreenArrow Concepts

• Table of Contents
• Queues
  • ram-queue
  • disk-queue
  • bounce-queue
• Send
• SendID
  • Format of the SendID for Email Sent from GreenArrow Studio
  • Format of the SendID for Email Processed by SimpleMH
  • Setting the SendID
• ListID
  • Format of the ListID When Using GreenArrow Studio
  • Setting the ListID
• MTAid
  • Format of the MTAid When Using GreenArrow Studio
• Internally Created Messages
• Message Unique ID
• Bounces, Deferrals and Throttles

This page documents a number of GreenArrow-specific concepts, which are key to understanding the rest of the documentation.

Queues

GreenArrow Engine has three different queues for storing messages.

ram-queue

The ram-queue is where messages injected into GreenArrow Engine go. It handles the first delivery attempts of messages.

If a delivery attempt is deferred or throttled, then the message is moved to the disk-queue for long-term storage and later retry. (Except for deliveries to local domain names, which are not moved to the disk-queue.)

The ram-queue is stored on a RAM disk, which makes GreenArrow Engine very fast at delivering messages.

disk-queue

The disk-queue is where messages that were not able to be delivered on the first attempt are placed. The disk-queue is stored in the filesystem for long-term storage.

bounce-queue

The bounce-queue is where internally created messages are placed for their first delivery attempt.

If a delivery attempt is deferred or throttled, then the message is moved to the disk-queue for long-term storage and later retry. (Except for deliveries to local domain names, which are not moved to the disk-queue.)

The bounce-queue is stored on a RAM disk, which makes GreenArrow Engine very fast at delivering messages.

Send

Messages are grouped together into a “send” for reporting purposes.

A send is a set of messages that should be grouped together, such as one sending of one newsletter to a set of subscribers, or transactional mail for a period of time.

Each message in a send will have the same SendID.

SendID

The SendID is an identifier in GreenArrow Engine for what “send” the message belongs to. A send is a set of messages that should be grouped together, such as one sending of one newsletter to a set of subscribers, or transactional mail for a period of time. Each send should have a unique SendID.

You don’t want to create too many specific SendID values. For example, don’t set a per-recipient value in the SendID field. If you do this, a new stats tracking database will be created for each email address, potentially overloading the system. It is no problem to have thousands of unique SendID values per day on a high volume server.

GreenArrow will group messages together with the same SendID for up to five months, even if the messages are injected on different days. However, after five months the same SendID will create a new campaign in Engine’s statistics.

Delivery attempt, bounce, spam complaint, click, open, and unsubscribe events for GreenArrow Engine will contain the SendID of the message.

The SendID value is publicly exposed. A malicious user could send a bounce message to GreenArrow Engine with an arbitrary SendID.

The SendID value is used case-insensitively within GreenArrow, so do not rely on case differences to differentiate campaigns.

Format of the SendID for Email Sent from GreenArrow Studio

GreenArrow Studio’s SendIDs will always use the prefix a, and the remainder of the SendID will depend on the origin of the email:

• Campaign Emails: the SendID is set to the StatID of the campaign with the prefix a.
• Autoresponder Emails: the SendID is formatted as aIDaYYMMDD where ID is the autoresponder’s ID, and YYMMDD is the 2-digit year, 2-digit month, and 2-digit day.
• Campaign Preview, Autoresponder Preview, and Seed Emails: the SendID is formatted as a0aYYYYMMDD where YYYYMMDD is the 4-digit year, 2-digit month, and 2-digit day.

Examples:

Campaign Emails sent by the campaign with StatID 9012 will have the SendID a9012.

Autoresponder Emails sent on May 1st 2017 by the autoresponder with ID 1074 will have the SendID a1074a170501.

Format of the SendID for Email Processed by SimpleMH

There are a few different SendID formats generated for emails created by SimpleMH in GreenArrow, depending on how you would like your statistics to be grouped. See Statistic Report Grouping for descriptions of the possible variations.

In general, it’s preferable to use the mailclass and instanceid fields available on events and processed logfiles rather than parse the SendID.

Setting the SendID

You can set the SendID using the GREENARROW_SENDID environment variable when injecting mail via the command-line.

If you’re using Raw Injection, the SendID can be set by using the X-GreenArrow-SendID header.

If you’re using GreenArrow Studio or SimpleMH, the SendID is set for you.

ListID

The ListID is an identifier that uniquely identifies the source of the email address. In most cases, the ListID can be thought of as a “Mailing List Identifier”. For example, if your database uses mailing lists to group subscribers, then the primary key that’s used for each mailing list is a potential ListID value.

Each table or list in your database that can independently contain an email address should probably be considered a distinct mailing list with its own ListID. For example, your customers table is probably a mailing list (since it is the source of the email addresses when you mail your customers), even through you might not think about it that way.

Repeat counting for multiple soft and hard bounces in the bounce processing system is done separately for each ListID. The benefit is that if you are an ESP, and you’re mailing on behalf of different customers, then deactivating an email address for Customer A will be because of what Customer A emailed, not because of what Customer B emailed.

Events that are logged by the Event Notification System will contain the ListID of the message.

The ListID value is publicly exposed. A malicious user could send a bounce message to GreenArrow Engine with an arbitrary ListID.

The ListID can be up to 20-characters long, but we recommend keeping it as short as possible in order to reduce overhead.

The ListID may contain the follow characters:

• ASCII alphanumeric characters (a-z, A-Z and 0-9)
• Underscores (_)
• Plus signs (+)

The a and A ListIDs, and ListIDs starting with an a or A, immediately followed by a digit, such as a1mail are reserved, and should not be used.

If you have multiple sources of mail with potential to have overlapping ListIDs, then ListID prefixes can be used. A ListID prefix consists of the leading alphabetical character(s) in the ListID. A ListID of a1234 has a ListID prefix of a. A ListID of trans1234a has a ListID prefix of trans.

The ListID value is used case-insensitively in GreenArrow – so do not rely on case differences of ListID to differentiate lists.

For example, GreenArrow Studio uses the prefix a in ListIDs. By assigning mail from your application a ListID prefix different to a you can prevent it from using a ListID that’s in use by Studio.

Please see the Bounce Processor Concepts documentation for caveats regarding long ListID values.

Format of the ListID When Using GreenArrow Studio

When using GreenArrow Studio, the ListID is set to the primary key of the mailing list that this email address came from preppended with the letter a.

Setting the ListID

You can set the ListID using the GREENARROW_LISTID environment variable when injecting mail via the command-line.

If you’re using SimpleMH, each Mail Class has a default ListID which can be controlled by editing the Mail Class. This default ListID can be overwritten by setting the X-GreenArrow-ListID header. The Mail Class can be selected by using the X-GreenArrow-MailClass header.

If you’re using Raw Injection, the ListID can be set by using the X-GreenArrow-ListID header.

MTAid

The MTAid sets the VirtualMTA (also known as the Mail Route) that mail will be sent out through. A VirtualMTA can be an IP Address, Relay Server, or Routing Rule. A Routing Rule splits messages between one or more IP Addresses and/or Relay Servers.

This can be set to the name of either an IP Address, Relay Server, or Routing Rule. See the Name column in the VirtualMTAs (formerly called Mail Routes) configuration page in GreenArrow Engine. GreenArrow Engine prevents you from having more than one VirtualMTA use the same name.

The MTAid can also be set to the numeric primary key of an IP Address, Relay Server, or Routing Rule. You can see the primary keys by looking at the number at the end of the URL for the View page of each VirtualMTA in the GreenArrow Engine web interface, or by looking at the id column of the mr_ipaddr, mr_relay, or mr_virtualmta tables in PostgreSQL.

If the MTAid is blank, or if it does not match any of the above values, then the email will be sent through the default VirtualMTA of the server.

Format of the MTAid When Using GreenArrow Studio

When using GreenArrow Studio, the MTAid is set to the primary key of the VirtualMTA that you choose to send through in GreenArrow Studio.

Internally Created Messages

Sometimes in the process of delivering one message, we have to create another message. These are called internally created messages. There are two cases where this happens:

1. When a message fails permanently we have to send a bounce messages back to the sender, which is a new message.
2. When a message is delivered to an address that forwards to another address, a new message is created to the new recipient.

Internally created messages all go to the bounce queue. This queue is dedicated for this purpose for performance reasons.

Message Unique ID

Each message as a unique ID which is internally generated by GreenArrow Engine.

The unique ID is only included in the delivery attempt events, not the bounce, click, open, unsubscribe, and spam complaint events.

Bounces, Deferrals and Throttles

A bounce is a delivery failure. An example is a user does not exist message. Bounces are described in more detail on the Bounce Processor page. The hvmail_report command reports bounced delivery attempts as having a failure status.

A deferral occurs when a receiving mail server refuses to accept a message, but indicates in its response that another delivery attempt should be made later. An example is a Messages from 1.2.3.4 temporarily deferred message.

A throttle occurs when GreenArrow Engine’s Throttling Settings prevent a message from being sent. The hvmail_report command reports throttled delivery attempts as having a connmaxout status.

Throttled and deferred delivery attempts will be retried later unless the message’s queue_lifetime parameter has expired. If the queue lifetime has expired, and a message is either throttled or deferred, GreenArrow Engine stops attempting to deliver the message, and generates an Other bounce.