GreenArrow Email Software Documentation

Speed Considerations

Benchmarking

Test Lists

A list of 10,000 addresses that a default GreenArrow configuration will deliver to a dummy SMTP service is attached here.

You can also generate your own test list by using addresses at subdomains of discardallmail.drh.net. For example:

[email protected]
[email protected]

GreenArrow Sink Services

GreenArrow includes an SMTP sink service that can be used for speed testing. The service accepts and discards all messages delivered to 127.0.0.1 on port 226.

GreenArrow Sink can also simulate network latency, deferrals, and bounces for the SMTP, QMQP, and HTTP injection methods.

The rest of this section shows how to install and configure GreenArrow Sink on a server that’s not running GreenArrow.

Copy the executable

Copy /var/hvmail/libexec/greenarrow-go from an up-to-date copy of GreenArrow to the server you’d like to use as greenarrow-sink. The instructions below assume you place it as /usr/local/bin/greenarrow-sink.

Create the launcher

Edit the parameters shown below to be appropriate for your testing needs.

cat > /usr/local/bin/greenarrow-sink-wrapper <<'EOF'
#!/bin/bash

ulimit -Hn 1048576
ulimit -Sn 1048576

# COMMA_SEPARATED_IP_ADDRESSES defines the IP addresses on which the sink will
# listen. You may specify one or more IPs, separated by commas.
COMMA_SEPARATED_IP_ADDRESSES="0.0.0.0"

# PORT_NUMBER is the port number on which to listen. This must be a single, valid port number.
PORT_NUMBER="2525"

# MAX_PAUSE_DURATION defines the longest time deferrals and failures will delay
# the SMTP session. A random number of seconds between 0 and the specified
# value will be used. The average session will be delayed by half of MAX_PAUSE_DURATION.
MAX_PAUSE_DURATION="4"

# FIXED_PAUSE_DURATION defines a fixed time deferrals and failures will delay
# the SMTP session.
FIXED_PAUSE_DURATION="0"

# SERVER_NAME defines the string used in the greeting.
SERVER_NAME="greenarrowsink"

# Each of the values below defines a proportional chance of each event. By
# setting CHANCE_ACCEPT=50 & CHANCE_DEFER_AFTER_DATA=50, half of all messages
# would be accepted and half would be deferred after receiving the message
# data. These numbers do not need to add up to 100.
CHANCE_ACCEPT="100"
CHANCE_DEFER_BEFORE_DATA="0"
CHANCE_DEFER_AFTER_DATA="0"
CHANCE_FAIL_BEFORE_DATA="0"
CHANCE_FAIL_AFTER_DATA="0"

# Create a certificate to use for greenarrow-sink. If you want to use your own
# certificate and key file, replace the two names here and delete the "if" block below.
# Alternatively, you can disable TLS in the sink by adding the "-no-starttls"
# option to the command below.
TLS_CRT_PATH="/tmp/greenarrow-sink.tls.crt"
TLS_KEY_PATH="/tmp/greenarrow-sink.tls.key"
if [ ! -f "$TLS_CRT_PATH" ] || [ ! -f "$TLS_KEY_PATH" ]; then
  TLS_CSR_PATH="/tmp/greenarrow-sink.tls.csr"
  openssl req -nodes -newkey rsa:1024 -keyout "$TLS_KEY_PATH" -out "$TLS_CSR_PATH" -subj "/C=GA/ST=GA/L=GA/O=GA/OU=GA/CN=GA"
  openssl x509 -req -days 3650 -in "$TLS_CSR_PATH" -signkey "$TLS_KEY_PATH" -out "$TLS_CRT_PATH"
fi

exec /usr/local/bin/greenarrow-sink                           \
  -chance-accept           "$CHANCE_ACCEPT"                   \
  -chance-defer-before     "$CHANCE_DEFER_BEFORE_DATA"        \
  -chance-defer-after      "$CHANCE_DEFER_AFTER_DATA"         \
  -chance-fail-before      "$CHANCE_FAIL_BEFORE_DATA"         \
  -chance-fail-after       "$CHANCE_FAIL_AFTER_DATA"          \
  -max-pause-duration      "$MAX_PAUSE_DURATION"              \
  -fixed-pause-duration    "$FIXED_PAUSE_DURATION"            \
  -bind-ip                 "$COMMA_SEPARATED_IP_ADDRESSES"    \
  -port                    "$PORT_NUMBER"                     \
  -tls-private-key-path    "$TLS_KEY_PATH"                    \
  -tls-certificate-path    "$TLS_CRT_PATH"                    \
  -servername              "$SERVER_NAME"
EOF
chmod 755 /usr/local/bin/greenarrow-sink-wrapper

By default, greenarrow-sink supports the STARTTLS command. If you would like to disable this, to prevent clients from using it, add the -no-starttls option to the command script above.

Create the service entry

cat > /etc/systemd/system/greenarrow-sink.service <<EOF
[Unit]
Description=greenarrow-sink
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/greenarrow-sink-wrapper
Restart=on-abort

[Install]
WantedBy=multi-user.target
EOF


systemctl enable greenarrow-sink
systemctl start greenarrow-sink
systemctl status greenarrow-sink

Updating settings

Following the instructions below will install greenarrow-sink as a systemd service. If you need to change the settings defined in /usr/local/bin/greenarrow-sink-wrapper, run systemctl restart greenarrow-sink to load the new settings.

Certificate and key file

The sink, by default, supports STARTTLS. If you would like to disable it, add -no-starttls to the greenarrow-sink command.

By default, it will look for certificate and key files at GreenArrow’s default HTTP TLS paths (/var/hvmail/var/tls/default_certificates/sink.crt and /var/hvmail/var/tls/default_certificates/sink.key). To override this, see the -tls-certificate-path and -tls-private-key-path options.

The example wrapper script above generates its own key file automatically.

Speed Testing

GreenArrow includes a command hvmail_speed_test for generating email for speed testing. It accomplishes this by injecting messages as fast as possible (or up to a specified speed, if one is given).

When running hvmail_speed_test, it will print current statistics on injection speed.

Here’s an example of running a test using SMTP:

$ hvmail_speed_test --num-send 100 \
  --mailclass default --auto-instance-id \
  --proto smtp --host localhost --port 25 \
  --message-file ~/email-3.txt \
  --realistic-subdomain-sink sink.example.com
automatically generated X-GreenArrow-InstanceID: 2019012416231601453
sent     92 k msg/hour,    25 msg/sec in last 1.015 seconds using 1 threads ( 102.43% of requested)
sent     89 k msg/hour,    24 msg/sec in last 1.041 seconds using 1 threads (  99.90% of requested)
sent     89 k msg/hour,    24 msg/sec in last 1.006 seconds using 1 threads (  99.37% of requested)

This command accepts the following arguments:

--host [HOST OR IP]

This host address will be used when connecting via SMTP or QMQP.

--port [PORT NUMBER]

This port will be used when connecting via SMTP or QMQP.

--speed [NUMBER]

The speed of messages injected per second. Specifying a speed can be useful for the case where you are trying to optimize your system for a particular delivery speed.

--speed-per-hour [NUMBER]

The speed of messages injected per hour.

--sender [SENDER]

The envelope sender address.

--recipient [RECIPIENT]

The envelope recipient to use. Do not use if using --realistic-subdomain-sink.

--mailclass [MAIL CLASS]

Set an X-GreenArrow-MailClass header with this value. This value should be a valid mail class as configured in GreenArrow - but hvmail_speed_test does not validate that this is the case.

--message-file [PATH]

This message file will be used for speed testing. It should contain both raw headers and body. It should be formatted using \n for line breaks (not \r\n).

If the file cannot be opened, hvmail_speed_test will exit with an appropriate error.

This option does not apply when --proto is set to http. For that case, see the --http-html and --http-text options.

--proto [PROTO NAME]

Specify how the mail should be injected.

This can be:

  • smtp - use a standard SMTP connection.
  • qmqp - use a QMQP connection.
  • http - use the HTTP Submission API.
  • do_nothing - no email will actually be delivered.
  • direct - a direct connection to GreenArrow Engine, for DRH internal testing.

--num-send [NUMBER]

The total number of messages that should be sent before terminating the speed test.

Only one of --num-send and --time-to-send may be specified.

--time-to-send [NUMBER]

The number of seconds that the speed test should be run.

Only one of --num-send and --time-to-send may be specified.

--realistic-subdomain-sink [DOMAIN]

This option allows you to send to generated domains using a realistic distribution. The argument specifies the root of the domains that will be generated.

If you specify --realistic-subdomain-sink foo.example.com, it will deliver to domains in the following example patterns:

  • s0.g0.foo.example.com
  • s0.g1.foo.example.com
  • s0.g2.foo.example.com
  • s0.g3.foo.example.com
  • s1.g3.foo.example.com

The distribution of domains will be roughly:

  • The top domain will receive roughly 36% of the emails.
  • Second domain receives roughly 16%.
  • Third domain receives roughly 9%.
  • Fourth domain receives roughly 3%.
  • Fifth domain receives roughly 1%.
  • From there, a long tail of domains will receive a wide distribution of the remaining emails.

--unix-socket [SOCKET]

When used instead of --host and --port, a UNIX domain socket will be used for delivery. This only applies when --proto is set to smtp or qmqp.

--direct-injection-batch-size [NUMBER]

The number of messages to include in a single batch when --proto is set to direct.

--threads [NUMBER]

The number of concurrent injection threads to use. Each thread is a UNIX process. A higher number of threads can increase overall throughput.

We do not recommend setting this value to higher than the number of CPU cores in your system.

--http-username [USERNAME]

When combined with --proto set to http, this username will be used for HTTP basic authentication.

--http-password [PASSWORD]

When combined with --proto set to http, this password will be used for HTTP basic authentication.

--http-url [URL]

When combined with --proto set to http, this is the URL used for HTTP submission.

--http-html [FILENAME]

When combined with --proto set to http, this is the filename that will be read for the HTML part of the submitted message.

--http-text [FILENAME]

When combined with --proto set to http, this is the filename that will be read for the HTML part of the submitted message.

--http-from-email [EMAIL]

When combined with --proto set to http, this is the From Email that will be used in the HTTP submission.

--http-subject [SUBJECT]

When combined with --proto set to http, this is the Subject that will be used in the HTTP submission.

--http-batch-size [SIZE]

When combined with --proto set to http, the number of messages to write in a single HTTP submission. When using http injection for speed testing, we strongly recommend use of the --http-batch-size option to get the best performance.

--instance-id [INSTANCE ID]

This value will be used for the X-GreenArrow-InstanceID header.

See SimpleMH Headers for more information.

--auto-instance-id

A randomly generated (prefixed with today’s date and time) instance ID will be generated for this test.

The generated instance ID will be of the formated YYYYMMDDHHMMSSRRRRR where RRRRR is a 5-digit random integer.

See SimpleMH Headers for more information.

 QMQP Streaming Protocol
Obscured Email Addresses 

Copyright © 2012–2024 GreenArrow Email