GreenArrow Email Software Documentation

Delivery Attempt Message Formats

Introduction

When GreenArrow receives a message for a remote delivery attempt, it processes the response and can use it in a variety of different formats. This document describes the possible formats that can either be matched on using a directive like override_smtp_result, smtp_match_begin_backoff_mode, or written to disk as a log file format.

Formats

Raw SMTP Response

This is the raw response as received from the remote SMTP server, with only one modification: the SMTP line termation sequence of CR (0x0D) LF (0x0A) is replaced with a single LF character (0x0A) to follow UNIX newline semantics.

Multiline replies in SMTP have the three-digit response code followed by a space or hyphen (-) on every line, just as it appears in the SMTP multiline reply received from the server. Single line replies start with the three-digit response code followed by a space, just as it appears in the SMTP reply received from the server.

Each newline that ends a line is preserved. If not blank, the string will always end in a newline.

The raw response from the server can be observed by enabling the directive log_smtp_commands and tailing the /var/hvmail/log/rspawn-limiter/current log file.

This format can be used by the override_smtp_result or smtp_match_begin_backoff_mode configuration directives by adding the match_format=raw option.

When a delivery attempt is concluded without an SMTP response from an SMTP server, then the “Raw SMTP Response” is a zero length string. Here are some examples of how this can happen:

  • When unable to establish a connection to the SMTP server (which shows up as Sorry, I wasn’t able to establish an SMTP connection. (#4.4.1) in the “queue logfile format” defined below).
  • When delivery_override is used to prevent SMTP delivery attempts.

Examples:

452-4.2.2 The email account that you tried to reach is over quota. Please direct
452-4.2.2 the recipient to
452 4.2.2  https://support.google.com/mail/?p=OverQuotaTemp h13si10345783qkj.633 - gsmtp


550 5.1.1 <[email protected]> recipient rejected

Queue Logfile Format

This format appears in the following locations:

  • The raw log files used by GreenArrow in /var/hvmail/log/{disk,ram,bounce}-qmail-send
  • The output of the hvmail_report --raw-logfile command
  • The output of the hvmail_report --processed-logfile --human command
  • Delivery Attempt Log (JSON, processed, and human formats)
  • Delivery attempt events
  • Engine’s SMTP server stats reporting

In this format:

  • All bytes less than 33, greater than 126, or equal to % are replaced with _.
  • Newline characters (0x0A) are replaced with / (0x2F)
  • Multiline SMTP responses are modified as follows:
    • On the first line, the hyphen (-) following the three digit SMTP response-code is changed to a space.
    • On all other lines, the three digit SMTP response-code and the space or hyphen (-) following it are removed.
  • If an SMTP response was received, a prefix is added by GreenArrow describing the context of the SMTP response such as 142.250.123.26 does not like recipient./Remote host said:
  • Sometimes this format does not include an SMTP response. For example:

This is the format used by default by the override_smtp_result and smtp_match_begin_backoff_mode configuration directives (or can be used by specifying the match_format=logfile option).

Examples (newlines are added below for readability, not present in the actual format):

142.250.123.26 does not like recipient./Remote host said: 452 4.2.2 The email account
that you tried to reach is over quota. Please direct/4.2.2 the recipient 
to/4.2.2  https://support.google.com/mail/?p=OverQuotaTemp h13si10345783qkj.633 - gsmtp/


142.250.123.26 does not like recipient./Remote host said: 550 5.1.1 <[email protected]> recipient rejected/

Processed Logfile Format

This format appears in the following location:

  • Output of the hvmail_report --processed-logfile command

In this format, the “Queue Logfile Format” described above is taken and horizontal tabs (0x09) are replaced with spaces (0x20).

This format is further described in Delivery Event Processed Logfile Format.

This format can be used by the override_smtp_result or smtp_match_begin_backoff_mode configuration directives by adding the match_format=processed option.

Web Format

This format is used by the drill-down in the “Send Statistics” and “SMTP server stats” tabs in Engine’s web user interface, as well as the Stats API.

The following transformation is performed from the Queue Logfile Format (defined above) to get the Web Format.

$web_format = $queue_logfile_format;

$web_format =~ s/ mta\S+\.yahoo\.com/HOSTNAME/gi;
$web_format =~ s/$email_local_quotemeta/EMAIL_LOCAL/gi;
$web_format =~ s/$domain_quotemeta/DOMAIN/gi;
$web_format =~ s/EMAIL_LOCAL\@DOMAIN/EMAIL/gi;
$web_format =~ s{(http://\S+\?)\S+}{${1}QUERY_STRING}gi;
$web_format =~ s/^\d+\.\d+\.\d+\.\d+ /IPADDR /;
$web_format =~ s/^Connected to \d+\.\d+\.\d+\.\d+ /Connected to IPADDR /;
$web_format =~ s/Giving up on \d+\.\d+\.\d+\.\d+\.\/$/Giving up on IPADDR\.\//;
$web_format =~ s/ [a-zA-Z0-9]+.[0-9]+ - gsmtp/ UNIQUE_IDENTIFIER - gsmtp/;
$web_format =~ s/[a-zA-Z0-9.-]+\.protection\.outlook\.com/NAME.protection.outlook.com/g;
$web_format =~ s/ 550 Access denied\.\.\.[0-9a-zA-Z]+\.\.\./ 550 Access denied...UNIQUE_IDENTIFIER.../; # netzero

This format can be used by the override_smtp_result or smtp_match_begin_backoff_mode configuration directives by adding the match_format=web option.

 Delivery Attempt Log (Deprecated)
Delivery Event Processed Logfile Format 

Copyright © 2012–2024 GreenArrow Email