Arnie — SMTP Buffer Server

Created 17 Oct 2021 by Andrew Stuart andrew.stuart@supercoders.com.au

License: MIT

Requires minimum Python 3.8

---

⚠️ SECURITY WARNING — INTERNAL USE ONLY ⚠️

Arnie is an intentionally open mail relay with no authentication.

It is designed exclusively for use as a localhost or internal-network service on a trusted server. It must NEVER be exposed to the public internet or any untrusted network.

You MUST ensure:

• The listen port (default 8025) is not reachable from any external network.
• Arnie runs behind a firewall, in a private subnet, or bound to 127.0.0.1.
• Only trusted local applications (your web app, scripts, etc.) connect to it.

If Arnie's port is reachable from the internet, it will be used as a spam relay.

---

What is Arnie and why should you use it?

Arnie is a server that has the single purpose of buffering outbound SMTP emails.

A typical web SaaS needs to send emails such as signup/signin/forgot password etc. The web page code itself should not directly write to an SMTP server. Instead they should be decoupled. If there is an error sending the email, the whole thing falls over if that send was executed by the web page code — there's no chance to resend because the web request has completed. Also, execution of an SMTP request by a web page slows the response time of that page. So when you send SMTP email from your web application, the most performant and safest way is to buffer them for sending. The buffering server will queue them and send them and handle retries if the target SMTP server is down or throttled.

There are other ways to solve this problem — you can set up a local email server configured for relaying, or use Celery. Both have many more features than needed and can be complex to configure/run/troubleshoot.

Arnie is intended for small-scale usage — for example a typical web server for a simple SaaS application.

Project status

This is a small personal project — not battle-tested production software. Use at your own risk.

---

How it works

Arnie uses a file-based queue with atomic rename() operations for crash safety:

1. Incoming SMTP → message written atomically to outbox/
2. Sender picks up message → atomic move to processing/
3. On success → atomic move to sent/ (or deleted if save_sent_mail = false)
4. On failure → atomic move back to outbox/ with incremented retry count
5. After max retries → atomic move to failed/

Envelope metadata (MAIL FROM, RCPT TO, retry count) is persisted in .meta.json files alongside each .eml, so a crash/restart does not lose routing information or reset retry counts.

All directories (outbox/, processing/, sent/, failed/) must be on the same filesystem for os.rename() atomicity.

---

Configuration

Arnie uses a config.ini file in the same directory as arniesmtpbufferserver.py. On first run, a default config.ini is created automatically.

Example config.ini:

[server]
host = 127.0.0.1
port = 8025
files_directory = ~/.arniesmtpbufferserver
max_message_size = 104857600
queue_size = 1000
disk_health_check_interval = 30
disk_high_watermark_percent = 95
disk_low_watermark_percent = 90
max_messages_per_hour = 100
max_bytes_per_hour = 52428800

[sender]
smtp_host = email-smtp.eu-west-1.amazonaws.com
smtp_port = 587
smtp_user = YOUR_SMTP_USERNAME
smtp_password = YOUR_SMTP_PASSWORD
smtp_timeout = 60
use_tls = false
start_tls = true
save_sent_mail = true
max_retry_attempts = 5
initial_retry_delay = 60
max_retry_delay = 3600
circuit_breaker_threshold = 5
circuit_breaker_timeout = 300
max_concurrent_retries = 10

Configuration reference

[server] section:

Key	Description	Default
host	Listen address. Use 127.0.0.1 for localhost only.	127.0.0.1
port	Listen port for inbound SMTP.	8025
files_directory	Where Arnie stores queued email files. ~ is expanded.	~/.arniesmtpbufferserver
max_message_size	Maximum message size in bytes.	104857600 (100 MB)
queue_size	Maximum number of messages in the in-memory queue.	1000
disk_health_check_interval	Seconds between disk health checks.	30
disk_high_watermark_percent	Disk/inode usage % to stop accepting mail.	95
disk_low_watermark_percent	Disk/inode usage % to resume accepting mail.	90
max_messages_per_hour	Per-client-IP rate limit.	100
max_bytes_per_hour	Per-client-IP byte rate limit.	52428800 (50 MB)

[sender] section:

Key	Description	Default
smtp_host	Upstream SMTP server hostname.	127.0.0.1
smtp_port	Upstream SMTP server port.	1025
smtp_user	SMTP username (leave empty for no auth).	(empty)
smtp_password	SMTP password.	(empty)
smtp_timeout	Connection timeout in seconds.	60
use_tls	Implicit TLS (connect with TLS from start).	false
start_tls	STARTTLS (upgrade plaintext to TLS). Cannot be true if use_tls is true.	true
save_sent_mail	Keep a copy of sent messages in sent/.	true
max_retry_attempts	Max delivery attempts before moving to failed/.	5
initial_retry_delay	First retry delay in seconds (doubles each retry).	60
max_retry_delay	Maximum retry delay in seconds.	3600
circuit_breaker_threshold	Consecutive failures to trip circuit breaker.	5
circuit_breaker_timeout	Seconds before circuit breaker allows a retry.	300
max_concurrent_retries	Max retry tasks running simultaneously.	10

---

Installing Arnie

sudo mkdir /opt/arniesmtpbufferserver
sudo chown -R :ubuntu /opt/arniesmtpbufferserver/
sudo chmod -R g+rwx /opt/arniesmtpbufferserver/

Check Python version (3.8+):

python3 -V

Download files:

cd /opt/arniesmtpbufferserver/
curl -O https://raw.githubusercontent.com/bootrino/arniesmtpbufferserver/master/arniesmtpbufferserver.py
curl -O https://raw.githubusercontent.com/bootrino/arniesmtpbufferserver/master/requirements.txt

Create and activate a venv:

python3 -m venv venv3
source venv3/bin/activate

Install dependencies:

pip install -r requirements.txt

Run Arnie:

python3 arniesmtpbufferserver.py

On first run, a default config.ini will be created. Edit it with your upstream SMTP server details, then restart Arnie.

---

Sending a test email

printf "To: foo@example.org\r\nFrom: foo@example.org\r\nSubject: A test from Arnie\r\n\r\nI'll be back." | curl smtp://127.0.0.1:8025 --mail-from foo@example.org --mail-rcpt foo@example.org -T -

Or use the included test script:

python3 send_email_smtplib2.py

---

Running as a systemd service

sudo cp etc/systemd/system/arniesmtpbufferserver.service /etc/systemd/system/

Review and edit the service file for your setup (paths, user, ReadWritePaths):

sudo nano /etc/systemd/system/arniesmtpbufferserver.service

Then:

sudo systemctl daemon-reload
sudo systemctl start arniesmtpbufferserver.service
sudo journalctl -fu arniesmtpbufferserver.service
sudo systemctl enable arniesmtpbufferserver.service

The service will automatically restart on crashes (Restart=on-failure).

---

Credits

Credit to Lynn Root for Arnie's shutdown code https://www.roguelynn.com/words/asyncio-graceful-shutdowns/