Skip to content

Debugging Django Email with Gmail in a Dockerized App

This post summarizes a recent troubleshooting session where I debugged email sending issues in a Django application. The app was deployed in a Docker container built with a multi-stage Dockerfile.prod file.

The Problem

I was able to send emails successfully from my local development environment, but emails were not being sent from the production environment, which was Dockerized and deployed on a remote server.

Initial Steps

I started by examining the Django code responsible for sending emails, ensuring that the email settings were configured correctly in the settings.py file, and verifying that the correct App Password was being used for Gmail SMTP authentication.

Local dev test

# settings.py
# EMAIL
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_HOST = "smtp.gmail.com"
EMAIL_HOST_USER = os.environ.get("EMAIL_HOST_USER")
EMAIL_HOST_PASSWORD = os.environ.get("EMAIL_HOST_PASSWORD")
EMAIL_PORT = 587
EMAIL_USE_TLS = True
EMAIL_USERS_TO_NOTIFY = os.environ.get("EMAIL_USERS_TO_NOTIFY").split(" ")

Remote Prod test

To test on the server, I created a small script to test sending emails from the server:

import os
import smtplib
from email.mime.text import MIMEText

EMAIL_HOST = "smtp.gmail.com"
EMAIL_PORT = 587
EMAIL_HOST_USER = "user@gmail.com"
EMAIL_HOST_PASSWORD = "psw-from-app-password"  # Generated from Gmail account: https://myaccount.google.com/apppasswords
EMAIL_TO = "test_recipient@example.com"
EMAIL_FROM = os.environ.get("EMAIL_HOST_USER")

try:
  with smtplib.SMTP(EMAIL_HOST, EMAIL_PORT) as server:
    server.starttls()
    server.login(EMAIL_HOST_USER, EMAIL_HOST_PASSWORD)

    msg = MIMEText("This is a test email.")
    msg["Subject"] = "Test Email"
    msg["From"] = EMAIL_FROM
    msg["To"] = EMAIL_TO

    server.send_message(msg)
    print("Email sent successfully!")
except Exception as e:
  print(f"Error sending email: {e}")

Running the script from the server:

```sh
root@site-name:~/repo-name# python3 django-app/send_email.py
#... output showing successful email sending...

The output shows that the email was sent successfully when you hardcoded the credentials. This confirms that the problem is not with your Gmail account, App Password, or network configuration. The issue was with how my Django application was accessing the environment variables or something else in the Dockerized app.

Docker and Environment Variables

We then shifted our focus to the Docker environment, analyzing the docker-compose.yml file and the entrypoint.sh script to ensure that environment variables, especially those containing sensitive information like email credentials, were being loaded and passed correctly to the Django application container.

docker-compose exec django-app sh
# Inside the container:
echo $EMAIL_HOST_USER
user@gmail.com
echo $EMAIL_HOST_PASSWORD
dhnz xxxx xxxx xxxx

The output from docker-compose exec confirms that the environment variables EMAIL_HOST_USER and EMAIL_HOST_PASSWORD are correctly set inside the django-app container.

But I was getting a (334, b'UGFzc3dvcmQ6') error, which suggested that the problem is almost certainly with the App Password, Gmail part. The SMTP server (Gmail) is requesting authentication, but it's not receiving the correct password, though it is loaded in the django-app container. The 334 code is an SMTP response that usually means "Continue with authentication," and the b'UGFzc3dvcmQ6' part is the base64-encoded string "Password:". This confirms the authentication issue.

SSL Certificate Verification

Then I encountered a ssl.SSLCertVerificationError, indicating problems with SSL certificate verification.

docker-compose exec django-app sh
python manage.py sendtestemail user@gamil.com

Which returned:

user@gmail.com
Traceback (most recent call last):
  File "/app/manage.py", line 22, in <module>
  ...
    ~~~~~~~~~~~~~~~~~^^
  File "/python/cpython-3.13.2-linux-x86_64-gnu/lib/python3.13/ssl.py", line 1372, in do_handshake
    self._sslobj.do_handshake()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1028)

The ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] error means that your Python environment is unable to verify the authenticity of the SSL certificate presented by the Gmail SMTP server (smtp.gmail.com). This is often caused by a missing or outdated CA (Certificate Authority) certificate bundle in your Docker container.

I checked for the presence of the ca-certificates package using dpkg -l ca-certificates, but it was not fully installed.

dpkg -l ca-certificates

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-=================================
un  ca-certificates <none>       <none>       (no description available)

The output of dpkg -l ca-certificates shows that the ca-certificates package is in an un (unpacked) state, which means it's not fully installed. This is likely why I am still getting the ssl.SSLCertVerificationError.

Manual Installation as a Workaround

As a temporary workaround, I manually installed ca-certificates package in the Docker image to provide the necessary root certificates.

docker-compose exec --user root django-app sh
apt-get purge ca-certificates
apt-get install ca-certificates

🎉 This allowed emails to be sent successfully. 🥳

This confirmed that the problem was with the installation process during the Docker build of Dockerfile.prod.

So, I added the installation of the ca-certificates in my Dockerfile.pro file in my third build step:

#... other stages...

# ###############################################################################
# Stage 3: Building environment
# ###############################################################################
FROM python-base AS builder-base

#... other commands...

# Install ca-certificates and other dependencies
RUN apt-get update && apt-get install -y \
    ca-certificates \
    #... other packages... \
    && rm -rf /var/lib/apt/lists/*

#... other commands...

Persistent SSL Certificate issue

Despite adding ca-certificates to the Dockerfile.pro, the issue persisted 😢. Indeed, after every rebuild of the container, the email sending did not work anymore.

I delved deeper into the Docker build process, analyzing the verbose output of docker-compose build and using intermediate images to pinpoint the exact step where the installation was failing.

I explored various possibilities, including conflicts with other packages, issues with the base image, and problems with the apt package manager. I also considered manually copying certificates as a last resort.

The Manual Installation as a Workaround resolves the issue but I was not going to manually reinstall the ca-certificates after every build...🙄 since after every build when I check:

docker-compose -f docker-compose-prod.yml exec --user root django-app dpkg -l ca-certificates

I get

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-=================================
un  ca-certificates <none>       <none>       (no description available)

Resolution

After carefully reading the Dockerfile.prod noticed that the issue - the ca-certificates package was being installed in the builder-base stage, but it's not being carried over to the final webapp stage since theDockerfile.prod starting fresh from linux-base.

So, I moved the ca-certificates installation to the linux-base stage, which all other stages inherit from.

# syntax=docker.io/docker/dockerfile:1.7-labs

# ###############################################################################
# Stage 1: General debian environment
# ###############################################################################
FROM debian:stable-slim AS linux-base

# Assure UTF-8 encoding is used.
ENV LC_CTYPE=C.utf8
# Location of the virtual environment
ENV UV_PROJECT_ENVIRONMENT="/venv"
# Location of the python installation via uv
ENV UV_PYTHON_INSTALL_DIR="/python"
# Byte compile the python files on installation
ENV UV_COMPILE_BYTECODE=1
# Python verision to use
ENV UV_PYTHON=python3.13
# Tweaking the PATH variable for easier use
ENV PATH="$UV_PROJECT_ENVIRONMENT/bin:$PATH"

ENV APP_HOME=/app/django-app

# Update debian and install base packages
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install --no-install-recommends -y \
    tzdata \
    ca-certificates && \
    update-ca-certificates && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*


# ###############################################################################
# Stage 2: Python environment
# ###############################################################################
FROM linux-base AS python-base

# Install debian dependencies
RUN apt-get update && \
    apt-get install --no-install-recommends -y build-essential gettext && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Install uv
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

# Create virtual environment and install dependencies
COPY pyproject.toml ./
COPY uv.lock ./
RUN uv sync --all-extras --frozen --no-dev --no-install-project


# ###############################################################################
# Stage 3: Building environment
# ###############################################################################
FROM python-base AS builder-base

WORKDIR /app
COPY . /app

# Add build-time environment variables
ENV DJANGO_ALLOWED_HOSTS="localhost 127.0.0.1"
ENV CSRF_TRUSTED_ORIGINS="http://localhost https://localhost"
ENV EMAIL_USERS_TO_NOTIFY="dummy-email-value@gmail.com"

# Then collect all static files
RUN python manage.py collectstatic --noinput -v 2


# ###############################################################################
# Stage 4: Webapp environment
# ###############################################################################
FROM linux-base AS webapp

# Create a non-root user
RUN useradd -m appuser

# Copy python, virtual env and app
COPY --from=builder-base $UV_PYTHON_INSTALL_DIR $UV_PYTHON_INSTALL_DIR
COPY --from=builder-base $UV_PROJECT_ENVIRONMENT $UV_PROJECT_ENVIRONMENT
COPY --from=builder-base /app /app

# Change ownership of the application files to the non-root user
RUN chown -R appuser:appuser /app $UV_PROJECT_ENVIRONMENT $UV_PYTHON_INSTALL_DIR \
    && chmod +x /app/entrypoint.sh

# Switch to the non-root user
USER appuser

WORKDIR /app

RUN mkdir -p $APP_HOME/mediafiles

EXPOSE 8000

ENTRYPOINT ["/app/entrypoint.sh"]

The ca-certificates package should be properly installed in the final image since it's included in the base image that all stages inherit from.

Finale Check

docker-compose -f docker-compose-prod.yml exec --user root django-app dpkg -l ca-certificates

Returns

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-=================================
ii  ca-certificates 20230311     all          Common CA certificates

Lessons Learned

  • Environment Variables in Docker: Properly managing and passing environment variables to Docker containers is crucial.
  • SSL Certificate Verification: Ensure that your Docker images have the necessary CA certificates.
  • Debugging Docker Builds: Analyzing the verbose output of docker-compose build and using intermediate images can help pinpoint issues.
  • Manual Intervention for Debugging: Manually installing packages in a running container can be a useful debugging technique.
  • Dockerfile Stages and Dependencies: Understand how dependencies are carried over between stages in a multi-stage Docker build.

Conclusion

Debugging email sending issues in a Dockerized Django application can be complex, involving various layers of configuration and dependencies. By systematically analyzing the code, environment variables, SSL certificates, and the Docker build process, you can effectively identify and resolve the root cause of the problem.