How do I account for sending multiple (or no) attachments (via request.FILES
) in Django using the Gmail API, so I can store the legacy message ID (ie "FBf…MiD") for future retrieval to reply in the same thread/thread_ID? I am switching from the SMTP (to be fully deprecated by Google) by Django's email.send(), which appears to be significantly different in how it handles file types from Gmail's API upload.
This already works with Django's email. How can I convert it to use Gmail's API for multiple-type attachments?
I am specifically struggling to find out how to attach multiple files of different types to the Gmail API. (unsure if I am supposed to serialize the EmailMultiAlternatives, or how to do so)
This is where the magic/headaches happens, and have issues. I am not sure why I feel like I'm reinventing the wheel, and can't find any packages for the server side that make it easier. I can get the 'attachments' to 'attach' to the email and show up in the UI, but the files are not clickable because they are 0 bytes, and sometimes empty MIME types given their actual type.
def create_body_message_with_attachments(
sender,
to,
subject,
msgHtml,
msgPlain,
attachmentFiles,
cc=None,
bcc=None,
):
"""Create a message for an email.
Args:
sender: Email address of the sender.
to: Email address of the receiver.
subject: The subject of the email message.
msgHtml: Html message to be sent
msgPlain: Alternative plain text message for older email clients
attachmentFile: The path to the file to be attached.
Returns:
An object containing a base64url encoded email object.
"""
# allow either one recipient as string, or multiple as list
if isinstance(to, list):
to = ", ".join(to)
if cc:
if isinstance(cc, list):
cc = ", ".join(cc)
if bcc:
if isinstance(bcc, list):
bcc = ", ".join(bcc)
message = MIMEMultipart("mixed")
message["to"] = to
message["from"] = sender
message["subject"] = subject
…
# allow either one attachment as string, or multiple as list
if not isinstance(attachmentFiles, list):
attachmentFiles = [attachmentFiles]
# attachmentFiles
# [
# <InMemoryUploadedFile: Screenshot.jpg (image/jpg)>,
# <InMemoryUploadedFile: Screenshot2.png (image/png)>,
# <_io.BufferedReader name='/path/to/quote.pdf'>
# ]
messageA = MIMEMultipart("alternative")
messageR = MIMEMultipart("related")
messageR.attach(MIMEText(msgHtml, "html"))
messageA.attach(MIMEText(msgPlain, "plain"))
messageA.attach(messageR)
message.attach(messageA)
for attachment in attachmentFiles:
# Trying to separate the filename from the file content for different types
# File Name
if hasattr(attachment, "temporary_file_path"):
filename = os.path.basename(attachment.temporary_file_path())
elif hasattr(attachment, "name"):
filename = os.path.basename(attachment.name)
else:
filename = os.path.basename(attachment)
# File Contents
# Content Data
if isinstance(attachment, str) and os.path.exists(attachment):
content_type, _ = mimetypes.guess_type(attachment) or (content_type, None)
with open(attachment, "rb") as f:
file_data = f.read()
# Handle BufferedReader (BytesIO)
elif isinstance(attachment, io.BytesIO):
file_data = attachment.getvalue() # Ensure correct byte data is read
# Handle Django InMemoryUploadedFile
elif isinstance(attachment, InMemoryUploadedFile):
content_type = attachment.content_type or content_type
file_data = attachment.read()
# Type
# I have tried different ways to get the MIME type,
# but am unsure of the most pythonic way to do so, many opions out there
mim = magic.Magic(mime=True)
try:
c_t = mim.from_file(filename)
except OSError as e:
# Magic needs to differentiate?! between files and streams
c_t = mim.from_buffer(attachment.read(2048))
# Magic often returns 'application/x-empty'
print(f"file: {attachment} with {content_type=} and {c_t=}")
main_type, sub_type = content_type.split("/", 1)
if main_type == "text":
# Unsure if I have to open and close for each type
# fp = open(attachment, "rb")
msg_attachment = MIMEText(file_data, _subtype=sub_type)
# fp.close()
elif main_type == "image":
# Unsure if I have to open and close for each type
# fp = open(attachment, "rb")
msg_attachment = MIMEImage(file_data, _subtype=sub_type)
# fp.close()
elif main_type == "audio":
msg_attachment = MIMEAudio(file_data, _subtype=sub_type)
elif main_type == "application" and sub_type == "pdf":
msg_attachment = MIMEApplication(attachment.read(), _subtype=sub_type)
else:
msg_attachment = MIMEBase(main_type, sub_type)
msg_attachment.set_payload(attachment.read())
encoders.encode_base64(msg_attachment)
msg_attachment.add_header(
'Content-Disposition', 'attachment', filename=f'{filename}'
)
message.attach(msg_attachment)
raw = base64.urlsafe_b64encode(message.as_bytes())
raw = raw.decode()
body = {"raw": raw}
return body
Trying to passing the EmailMultiAlternatives directly to the Gamil API I get RFC822 payload message string or uploading message via /upload/* URL required", 'domain': 'global', 'reason': 'invalidArgument'
and trying to encode the message to JSON I get errors similar to 'utf-8' codec can't decode byte 0x93 in position 10: invalid start byte
. Not sure how to pass correctly encoded attachments and a message to the API.
Notes(that help differentiate other SO questions)
- Not using SMTP, as that has already begun sundowning/deprecating, and I am are trying to avoid using the google account setting of "less secure apps"
- Using
django.core.mail
's EmailMultiAlternatives to compose and send, whichmail.send(fail_silently=True)
does not return the message, or give an ID on the gmail server, so finding the exact email is not functional/deterministic given the smilarities - In Django, when the upload files come through the
request.FILES
, they are a of typeInMemoryUploadedFile
fromdjango.core.files.uploadedfile
¿with the potential to be different MIME Types?, but the internal attachment of a generated PDF is of type_io.BufferedReader
. This is presumably causing my headaches - using Python 3.11.7 in dev and Python 3.12.3 in prod, and aware that in 3.13 that
mimetypes.guess_type()
will be deprecated per this - Sending the EmailMultiAlternatives directly to the Gmail API
send().execute()
I get an error ofObject of type EmailMultiAlternatives is not JSON serializable
- The MIME information from the received email is:
.
--===============2268240127970561189==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 1.54.01PM.png"
--===============2268240127970561189==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 4.44.11PM.png"
--===============2268240127970561189==
Content-Type: application/octet-stream
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Q-DFD-12345-031525-0XEH-1153.pdf"
--===============2268240127970561189==--
- I have searched many SO questions, walkthroughs, API guides, etc…. I believe there not to be any current examples that satisfy all the following: Django InMemoryUploadedFile, Gmail API, multiple attachments and of different types, non SMTP, traceable email/threadIDs, and mostly updated python. Feel free to share if you find one.
Other files that may help understand flow
view.py
class MakeQuoteWithItems(…, CreateView):
def post(self, request, *args, **kwargs):
# init a Django Model
quote = QuoteClass(request)
# with a generated PDF, and send the associated email
quote.make_and_email_quote()
Current working way (SMTP)
from django.core.mail import EmailMultiAlternatives
def emailTheQuote(quote, attachments=None):
from_email = …
subject = …
email_context = …
html_content = render_to_string("email/tempalte.html", email_context, quote.request)
to_emails = … (string, [] or whatever format needed)
…
#
# Current working way sending through SMTP
#
# Django settings somehow gets the email out
# EMAIL_USE_TLS = False
# EMAIL_HOST = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_HOST")
# EMAIL_HOST_USER = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_EMAIL")
# EMAIL_HOST_PASSWORD = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_PASSWORD")
# EMAIL_PORT = 587
#
mail = EmailMultiAlternatives(
subject,
strip_tags(html_content),
to=to_emails,
bcc=bcc_emails,
from_email=from_email,
)
mail.attach_alternative(html_content, "text/html")
mail.attach(
quote.pdf_canvas._filename, open(quote.canvas._filename, "rb").read(), "application/pdf"
)
if quote.attachments:
for attachment in quote.attachments:
mail.attach(attachment._name, attachment.read(), attachment.content_type)
mail.send(fail_silently=False)
# But can't get a hold of the mail going out to get it's ID to reply to later
Proposed way (Gmail API)
def emailTheQuote(quote, attachments=None):
from_email = …
subject = …
# Instead -
gmail_email = create_and_send_gmail_message(
from_email,
to_emails,
subject,
html_content,
text_content,
quote.request,
attachmentFiles=attachments, # user uploaded and the generated PDF
)
# Some Django Model to query to get the email.message_id and email.thread_id
GmailEmailLog.objects.create(
gmail_email=gmail_email,
message_id=gmail_email.message_id,
# Thread ids are the same for the first message in a thread, but remain
# the same for all subsequent messages sent within the thread
thread_id=gmail_email.thread_id,
quote_id=quote.id
…
)
return gmail_email
helper_functions.py
def create_and_send_gmail_message(
sender,
to,
subject,
msgHtml,
msgPlain,
request,
attachmentFiles=None,
cc=None,
bcc=None,
):
if attachmentFiles:
message_body = create_body_message_with_attachments(sender,to,subject,msgHtml,msgPlain,attachmentFiles,cc,bcc,)
else:
# This is not an issue
message_body = create_messag_body_html(sender,to,subject,msgHtml,msgPlain,cc,bcc,)
result = SendMessageInternal(message_body, request)
return result
def SendMessageInternal(incoming_message_body, request):
credentials = get_credentials(request)
service = discovery.build("gmail", "v1", credentials=credentials)
user_id = settings.EMAIL_GMAIL_USERID
try:
msg = (
service.users()
.messages()
.send(
userId=user_id,
body=incoming_message_body,
)
.execute()
)
print(f"Message Id: {msg['id']}")
return msg
except errors.HttpError as error:
print(f"An error occurred: {error}")
return f"{error=}"
return "OK"
EMAIL Raw files
THIS IS THE PREVIOUS WORKING EMAIL, sudo-anonymised
Delivered-To: [email protected]
Received: by 20:…:ca with SMTP id sp132931;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Received: by 20:…:5c with SMTP id 20….….…06;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
ARC-Seal: i=3; a=rsa-sha256; t=1676522553; cv=pass;
d=google; s=arc-20160816;
b=FBh… …zfk==
ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google; s=arc-20160816;
h=message-id:date:to:from:subject:mime-version:dkim-signature
:delivered-to;
bh=Mn…tY=;
b=zA…fO+wb…9Z+X+GZ…l8+QC…w3+rs…RW+ch…DQ==
ARC-Authentication-Results: i=3; mx.google;
dkim=pass [email protected] header.s=emailProvider header.b=er…YA;
arc=pass (i=2 dkim=pass dkdomain=production_domain);
spf=neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain) smtp.mailfrom="my_=gmailAccount=gmail@personal_test_domain"
Return-Path: <my_=gmailAccount=gmail@personal_test_domain>
Received: from mail-sor-f41.google (mail-sor-f41.google. [209.….41])
by mx.google with SMTPS id a5…23.ip.ad.dr.e.ss
for <[email protected]>
(Google Transport Security);
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
Received-SPF: neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain) client-ip=209.….41;
Authentication-Results: mx.google;
dkim=pass header.i=@production_domain header.s=emailProvider header.b=er…YA;
arc=pass (i=2 dkim=pass dkdomain=production_domain);
spf=neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain) smtp.mailfrom="my_=gmailAccount=gmail@personal_test_domain"
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100; s=20210112;
h=message-id:date:to:from:subject:mime-version:dkim-signature
:delivered-to:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=f3lQdqf4Na3Uj8i45MnNgmpkfOMWo5t8aOYO8suzazE=;
b=6m1jghQ3ciL+qfXsAIFeM5EZ54BIjxX5aCebYBX/neCEaKXoVycDZAC0bAl4FpeiNv
UwkST9cVeWQweICf6HKwQ1J2rQSELlhRLjTqNvM5pBbPMQZXc+g/wrATZd+2botCqZO/
Y6zog9xQWHs/IXeZYV2T+H1AoBZIow9DiYhvl9nD8/zjwAsC5BfvANVQVhpmERKPksYN
9T0L9SX83HokibmO3bZzb5DTK1eJGQDeysgznNHDERZIHTF7W6rq+7lVoqG3wj7auX3F
jsVllk6E7yXxtuBeJ3PQO9ldtaNU/TxaLy3u7Cq2sqlaR5ttqS003cIO/M1IZo/Kr3oT
NtqQ==
X-Gm-Message-State: AO0yUKUng8IxDTR5Pa4seNHrOauqQx1ULgwNakQLuDabR5Df/CR+pbfh 52r6R/0O8UXEuIp4MustAWlEXSMAeWz8hcEWmUwGn5aF1s8PEz6f+UvEcEg=
X-Received: by 2002:a50:bb48:0:b0:4ac:b8e1:7410 with SMTP id y66-20020a50bb48000000b004acb8e17410mr2364158ede.6.1676522552160;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Forwarded-To: [email protected]
X-Forwarded-For: chris@personal_test_domain [email protected]
Delivered-To: jerome@personal_test_domain
Received: by 2002:a54:2789:0:b0:1f9:34b:de17 with SMTP id n9csp225579ecp;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Google-Smtp-Source: AK7set9WzXuuJYVtYFjdtn1LrQnmLKtM2tv4acM4vnzclcNgqHEAS0FQQMXr004S9ccLFIJOWep/
X-Received: by 2002:a17:902:e5c3:b0:19a:a80e:a6d5 with SMTP id u3-20020a170902e5c300b0019aa80ea6d5mr5626264plf.23.1676522545554;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
ARC-Seal: i=2; a=rsa-sha256; t=1676522545; cv=pass;
d=google; s=arc-20160816;
b=fWktl8SgvQeHJZHh5vkX/H+BDnrVsEHD2gJQ7z5kAAcO+0G3MzKJiksm5Q3Tma46/s
vPBk1I9HeFFlmOVDNfZzrpSqNtKzrbRh6KDSFgumiAl/IYWzyul/Y9izd3uWs0IoQhBT
+SutRjEE5ZqgR5bLbNbBBaAkpVIWIbj3PEHxHR3fIrykqReaC0S9x/IlcTBRXdji0I/Y
HbVFL9oiyLU3JoV5HUuU//oQbT648XPTZeawUxP41Hz8PJDYj3riyo32XmlxRNLXRvTZ
+Zb2x6EPOQezwDXb7XR8CgAIQ4KJvxIl7IuArmOXQTRf45gCZywhEMHUxKtW0o/IkZNw
qzjA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google; s=arc-20160816;
h=message-id:date:to:from:subject:mime-version:dkim-signature;
bh=f3lQdqf4Na3Uj8i45MnNgmpkfOMWo5t8aOYO8suzazE=;
b=ED7g4/sG1svFF6LH/QjWHutvwM/kYqlW3n6IUmuCdvqUHsRR9JFqwwE4Sj/1Xjf8qA
gUUUWgGWSxsVC6Oqoqt48PjmRGuVq8y5LYIIGNHgfe/FOScOYl2w1mJup16MwTrXlq51
QF9jJe6fGH9P/uBLUC0QwpwFhAmHVjbwMXsw1zoobjmkKNHRERJWUTzLjNWiiVYmeVog
CvwzW49kRjiapIlQnGCnIje7c4ywLtsU9z6g6VIxwyHoJHEWMO4HdHbGsiwx6LL3VT5O
rv0bJ5lHZnpZnnhWZES+Q8ewr/BcKB/0bSFclfDMPBtbKWM4AVF1dfNcIjRTh8cRdPV2
/LNQ==
ARC-Authentication-Results: i=2; mx.google;
dkim=pass header.i=@production_domain header.s=emailProvider header.b=er…YA;
arc=pass (i=1);
spf=neutral (google: 23.83.212.48 is neither permitted nor denied by best guess record for domain of development-quote@production_domain) smtp.mailfrom=DEVELOPMENT-quote@production_domain
Return-Path: <DEVELOPMENT-quote@production_domain>
Received: from dog.elm.relay.mailchannels (dog.elm.relay.mailchannels. [23.83.212.48])
by mx.google with ESMTPS id e10-20020a170902784a00b00194a1f665b5si293781pln.570.2023.02.15.20.42.23
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
Received-SPF: neutral (google: 23.83.212.48 is neither permitted nor denied by best guess record for domain of development-quote@production_domain) client-ip=23.83.212.48;
X-Sender-Id: emailProvider|x-authsender|[email protected]_domain
Received: from relay.mailchannels (localhost [127.0.0.1]) by relay.mailchannels (Postfix) with ESMTP id 3E82F8810AA; Thu, 16 Feb 2023 04:42:23 +0000 (UTC)
Received: from pdx1-sub0-mail-a306.emailProvider (unknown [127.0.0.6]) (Authenticated sender: emailProvider) by relay.mailchannels (Postfix) with ESMTPA id 8A…14; Thu, 16 Feb 2023 04:42:22 +0000 (UTC)
ARC-Seal: i=1; s=arc-2022; d=mailchannels; t=76…54; a=rsa-sha256; cv=none; b=qk…nC/bA…de/ Dc…mf/s8…3u+XQ…Wx+8c/D…g/SR…up+zl…zf…==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels; s=arc-2022; t=16…25; h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
dkim-signature; bh=lQ…sY=; b=eK…+/Qu…+m2CZ9JA7 lp…+tY…+lvjD 7T…+b…JK…+dh…+D/yytOf Yt…+ht/Kf0rAlyIeuNn7tT9Wu1au4/dR…+ch…w1/gz…+7Z==
ARC-Authentication-Results: i=1; rspamd-b9c55767f-ngqs4; auth=pass smtp.auth=emailProvider smtp.mailfrom=DEVELOPMENT-quote@production_domain
X-Sender-Id: emailProvider|x-authsender|[email protected]_domain
X-MC-Relay: Neutral
X-MailChannels-SenderId: emailProvider|x-authsender|[email protected]_domain
X-MailChannels-Auth-Id: emailProvider
X-Relation-Continue: 21…07
X-MC-Loop-Signature: 22…30:69…94
X-MC-Ingress-Time: 65…54
Received: from pdx1-sub0-mail-a306.emailProvider (pop.emailProvider [ip.ad.dr.es.s1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by ip.ad.dr.es.s9 (trex/6.7.1); Thu, 16 Feb 2023 04:42:23 +0000
Received: from 1.0.0…0.0.0.0.0.0.ip6.arpa (cpe-71-68-87-31.carolina.res.rr [ip.ad.dr.es.s]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: [email protected]_domain) by pdx1-sub0-mail-a306.emailProvider (Postfix) with ESMTPSA id Mk…Qz; Wed, 15 Feb 2023 19:45:01 -0500 (ET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=production_domain; s=emailProvider; t=16…25; bh=lQ…az=; h=Content-Type:Subject:From:To:Date; b=er…YA…gV/wq…9G
Ky…z3/iA…QG/eb…we+aY/4p…cD/y8…7d
WtajPHs+5Z…YB+xX…5D
Z6…UP+j2…KJ
I2…ue/6H…l6/Rx…83+RZ…8D/zD…mP
jB…iC/w==
Content-Type: multipart/mixed; boundary="===============867…5309=="
MIME-Version: 1.0
Subject: Quote - QUOTE-FSL-021523-2342
From: DEVELOPMENT-quote@production_domain
To: other@personal_test_domain, second@personal_test_domain, third@personal_test_domain, [email protected]
Date: Thu, 16 Feb 2023 04:42:19 -0000
Message-ID: <867…5309@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa>
--===============45…72==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
High Level Notes:
quote with attachments
Purchase Order: na
Regular thank you text.
Regular body Text.
--===============45…72==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="/path/to/quote.pdf"
--===============45…72==
Content-Type: image/jpeg
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="_bosch_cp1.jpeg"
--===============45…72==
Content-Type: image/jpeg
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="_cr_valve_bosch.jpeg"
--===============45…72==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="prod_SERVER_quote.pdf"
--===============45…72==--
THIS IS THE GMAIL API MESSAGE, received with three "empty" attachments, sudo-anonymised
Received: from 21…17 named unknown by gmailapi.google with
HTTPREST; Mon, 17 Mar 2025 09:34:56 -0500
Received: from 21…17 named unknown by gmailapi.google with HTTPREST; Mon, 17 Mar 2025 09:34:56 -0500
Content-Type: multipart/mixed; boundary="===============50…13=="
MIME-Version: 1.0
From: [email protected]
To: q@dev_domain
Cc: cc@dev_domain
Subject: Quote - Q-DFD-12345-031725-L0KQ-1034
Date: Mon, 17 Mar 2025 09:34:56 -0500
Message-Id: <oB…[email protected]>
--===============50…13==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
DEFAULT MESSAGE No message body text was provided - Have a nice day
--===============50…13==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 4.44.11PM.png"
--===============50…13==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 1.54.01PM.png"
--===============50…13==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Q-DFD-12345-031725-UNIQ-1034.pdf"
--===============50…13==--
Thanks in advance :)
How do I account for sending multiple (or no) attachments (via request.FILES
) in Django using the Gmail API, so I can store the legacy message ID (ie "FBf…MiD") for future retrieval to reply in the same thread/thread_ID? I am switching from the SMTP (to be fully deprecated by Google) by Django's email.send(), which appears to be significantly different in how it handles file types from Gmail's API upload.
This already works with Django's email. How can I convert it to use Gmail's API for multiple-type attachments?
I am specifically struggling to find out how to attach multiple files of different types to the Gmail API. (unsure if I am supposed to serialize the EmailMultiAlternatives, or how to do so)
This is where the magic/headaches happens, and have issues. I am not sure why I feel like I'm reinventing the wheel, and can't find any packages for the server side that make it easier. I can get the 'attachments' to 'attach' to the email and show up in the UI, but the files are not clickable because they are 0 bytes, and sometimes empty MIME types given their actual type.
def create_body_message_with_attachments(
sender,
to,
subject,
msgHtml,
msgPlain,
attachmentFiles,
cc=None,
bcc=None,
):
"""Create a message for an email.
Args:
sender: Email address of the sender.
to: Email address of the receiver.
subject: The subject of the email message.
msgHtml: Html message to be sent
msgPlain: Alternative plain text message for older email clients
attachmentFile: The path to the file to be attached.
Returns:
An object containing a base64url encoded email object.
"""
# allow either one recipient as string, or multiple as list
if isinstance(to, list):
to = ", ".join(to)
if cc:
if isinstance(cc, list):
cc = ", ".join(cc)
if bcc:
if isinstance(bcc, list):
bcc = ", ".join(bcc)
message = MIMEMultipart("mixed")
message["to"] = to
message["from"] = sender
message["subject"] = subject
…
# allow either one attachment as string, or multiple as list
if not isinstance(attachmentFiles, list):
attachmentFiles = [attachmentFiles]
# attachmentFiles
# [
# <InMemoryUploadedFile: Screenshot.jpg (image/jpg)>,
# <InMemoryUploadedFile: Screenshot2.png (image/png)>,
# <_io.BufferedReader name='/path/to/quote.pdf'>
# ]
messageA = MIMEMultipart("alternative")
messageR = MIMEMultipart("related")
messageR.attach(MIMEText(msgHtml, "html"))
messageA.attach(MIMEText(msgPlain, "plain"))
messageA.attach(messageR)
message.attach(messageA)
for attachment in attachmentFiles:
# Trying to separate the filename from the file content for different types
# File Name
if hasattr(attachment, "temporary_file_path"):
filename = os.path.basename(attachment.temporary_file_path())
elif hasattr(attachment, "name"):
filename = os.path.basename(attachment.name)
else:
filename = os.path.basename(attachment)
# File Contents
# Content Data
if isinstance(attachment, str) and os.path.exists(attachment):
content_type, _ = mimetypes.guess_type(attachment) or (content_type, None)
with open(attachment, "rb") as f:
file_data = f.read()
# Handle BufferedReader (BytesIO)
elif isinstance(attachment, io.BytesIO):
file_data = attachment.getvalue() # Ensure correct byte data is read
# Handle Django InMemoryUploadedFile
elif isinstance(attachment, InMemoryUploadedFile):
content_type = attachment.content_type or content_type
file_data = attachment.read()
# Type
# I have tried different ways to get the MIME type,
# but am unsure of the most pythonic way to do so, many opions out there
mim = magic.Magic(mime=True)
try:
c_t = mim.from_file(filename)
except OSError as e:
# Magic needs to differentiate?! between files and streams
c_t = mim.from_buffer(attachment.read(2048))
# Magic often returns 'application/x-empty'
print(f"file: {attachment} with {content_type=} and {c_t=}")
main_type, sub_type = content_type.split("/", 1)
if main_type == "text":
# Unsure if I have to open and close for each type
# fp = open(attachment, "rb")
msg_attachment = MIMEText(file_data, _subtype=sub_type)
# fp.close()
elif main_type == "image":
# Unsure if I have to open and close for each type
# fp = open(attachment, "rb")
msg_attachment = MIMEImage(file_data, _subtype=sub_type)
# fp.close()
elif main_type == "audio":
msg_attachment = MIMEAudio(file_data, _subtype=sub_type)
elif main_type == "application" and sub_type == "pdf":
msg_attachment = MIMEApplication(attachment.read(), _subtype=sub_type)
else:
msg_attachment = MIMEBase(main_type, sub_type)
msg_attachment.set_payload(attachment.read())
encoders.encode_base64(msg_attachment)
msg_attachment.add_header(
'Content-Disposition', 'attachment', filename=f'{filename}'
)
message.attach(msg_attachment)
raw = base64.urlsafe_b64encode(message.as_bytes())
raw = raw.decode()
body = {"raw": raw}
return body
Trying to passing the EmailMultiAlternatives directly to the Gamil API I get RFC822 payload message string or uploading message via /upload/* URL required", 'domain': 'global', 'reason': 'invalidArgument'
and trying to encode the message to JSON I get errors similar to 'utf-8' codec can't decode byte 0x93 in position 10: invalid start byte
. Not sure how to pass correctly encoded attachments and a message to the API.
Notes(that help differentiate other SO questions)
- Not using SMTP, as that has already begun sundowning/deprecating, and I am are trying to avoid using the google account setting of "less secure apps"
- Using
django.core.mail
's EmailMultiAlternatives to compose and send, whichmail.send(fail_silently=True)
does not return the message, or give an ID on the gmail server, so finding the exact email is not functional/deterministic given the smilarities - In Django, when the upload files come through the
request.FILES
, they are a of typeInMemoryUploadedFile
fromdjango.core.files.uploadedfile
¿with the potential to be different MIME Types?, but the internal attachment of a generated PDF is of type_io.BufferedReader
. This is presumably causing my headaches - using Python 3.11.7 in dev and Python 3.12.3 in prod, and aware that in 3.13 that
mimetypes.guess_type()
will be deprecated per this - Sending the EmailMultiAlternatives directly to the Gmail API
send().execute()
I get an error ofObject of type EmailMultiAlternatives is not JSON serializable
- The MIME information from the received email is:
.
--===============2268240127970561189==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 1.54.01PM.png"
--===============2268240127970561189==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 4.44.11PM.png"
--===============2268240127970561189==
Content-Type: application/octet-stream
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Q-DFD-12345-031525-0XEH-1153.pdf"
--===============2268240127970561189==--
- I have searched many SO questions, walkthroughs, API guides, etc…. I believe there not to be any current examples that satisfy all the following: Django InMemoryUploadedFile, Gmail API, multiple attachments and of different types, non SMTP, traceable email/threadIDs, and mostly updated python. Feel free to share if you find one.
Other files that may help understand flow
view.py
class MakeQuoteWithItems(…, CreateView):
def post(self, request, *args, **kwargs):
# init a Django Model
quote = QuoteClass(request)
# with a generated PDF, and send the associated email
quote.make_and_email_quote()
Current working way (SMTP)
from django.core.mail import EmailMultiAlternatives
def emailTheQuote(quote, attachments=None):
from_email = …
subject = …
email_context = …
html_content = render_to_string("email/tempalte.html", email_context, quote.request)
to_emails = … (string, [] or whatever format needed)
…
#
# Current working way sending through SMTP
#
# Django settings somehow gets the email out
# EMAIL_USE_TLS = False
# EMAIL_HOST = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_HOST")
# EMAIL_HOST_USER = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_EMAIL")
# EMAIL_HOST_PASSWORD = os.getenv("EMAIL_PROVIDER_SMTP_NOTIFIER_PASSWORD")
# EMAIL_PORT = 587
#
mail = EmailMultiAlternatives(
subject,
strip_tags(html_content),
to=to_emails,
bcc=bcc_emails,
from_email=from_email,
)
mail.attach_alternative(html_content, "text/html")
mail.attach(
quote.pdf_canvas._filename, open(quote.canvas._filename, "rb").read(), "application/pdf"
)
if quote.attachments:
for attachment in quote.attachments:
mail.attach(attachment._name, attachment.read(), attachment.content_type)
mail.send(fail_silently=False)
# But can't get a hold of the mail going out to get it's ID to reply to later
Proposed way (Gmail API)
def emailTheQuote(quote, attachments=None):
from_email = …
subject = …
# Instead -
gmail_email = create_and_send_gmail_message(
from_email,
to_emails,
subject,
html_content,
text_content,
quote.request,
attachmentFiles=attachments, # user uploaded and the generated PDF
)
# Some Django Model to query to get the email.message_id and email.thread_id
GmailEmailLog.objects.create(
gmail_email=gmail_email,
message_id=gmail_email.message_id,
# Thread ids are the same for the first message in a thread, but remain
# the same for all subsequent messages sent within the thread
thread_id=gmail_email.thread_id,
quote_id=quote.id
…
)
return gmail_email
helper_functions.py
def create_and_send_gmail_message(
sender,
to,
subject,
msgHtml,
msgPlain,
request,
attachmentFiles=None,
cc=None,
bcc=None,
):
if attachmentFiles:
message_body = create_body_message_with_attachments(sender,to,subject,msgHtml,msgPlain,attachmentFiles,cc,bcc,)
else:
# This is not an issue
message_body = create_messag_body_html(sender,to,subject,msgHtml,msgPlain,cc,bcc,)
result = SendMessageInternal(message_body, request)
return result
def SendMessageInternal(incoming_message_body, request):
credentials = get_credentials(request)
service = discovery.build("gmail", "v1", credentials=credentials)
user_id = settings.EMAIL_GMAIL_USERID
try:
msg = (
service.users()
.messages()
.send(
userId=user_id,
body=incoming_message_body,
)
.execute()
)
print(f"Message Id: {msg['id']}")
return msg
except errors.HttpError as error:
print(f"An error occurred: {error}")
return f"{error=}"
return "OK"
EMAIL Raw files
THIS IS THE PREVIOUS WORKING EMAIL, sudo-anonymised
Delivered-To: [email protected]
Received: by 20:…:ca with SMTP id sp132931;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Received: by 20:…:5c with SMTP id 20….….…06;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
ARC-Seal: i=3; a=rsa-sha256; t=1676522553; cv=pass;
d=google; s=arc-20160816;
b=FBh… …zfk==
ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google; s=arc-20160816;
h=message-id:date:to:from:subject:mime-version:dkim-signature
:delivered-to;
bh=Mn…tY=;
b=zA…fO+wb…9Z+X+GZ…l8+QC…w3+rs…RW+ch…DQ==
ARC-Authentication-Results: i=3; mx.google;
dkim=pass [email protected] header.s=emailProvider header.b=er…YA;
arc=pass (i=2 dkim=pass dkdomain=production_domain);
spf=neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain.) smtp.mailfrom="my_=gmailAccount=gmail@personal_test_domain."
Return-Path: <my_=gmailAccount=gmail@personal_test_domain.>
Received: from mail-sor-f41.google (mail-sor-f41.google. [209.….41])
by mx.google with SMTPS id a5…23.ip.ad.dr.e.ss
for <[email protected]>
(Google Transport Security);
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
Received-SPF: neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain.) client-ip=209.….41;
Authentication-Results: mx.google;
dkim=pass header.i=@production_domain header.s=emailProvider header.b=er…YA;
arc=pass (i=2 dkim=pass dkdomain=production_domain);
spf=neutral (google: 209.….41 is neither permitted nor denied by best guess record for domain of my_=gmailAccount=gmail@personal_test_domain.) smtp.mailfrom="my_=gmailAccount=gmail@personal_test_domain."
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100; s=20210112;
h=message-id:date:to:from:subject:mime-version:dkim-signature
:delivered-to:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=f3lQdqf4Na3Uj8i45MnNgmpkfOMWo5t8aOYO8suzazE=;
b=6m1jghQ3ciL+qfXsAIFeM5EZ54BIjxX5aCebYBX/neCEaKXoVycDZAC0bAl4FpeiNv
UwkST9cVeWQweICf6HKwQ1J2rQSELlhRLjTqNvM5pBbPMQZXc+g/wrATZd+2botCqZO/
Y6zog9xQWHs/IXeZYV2T+H1AoBZIow9DiYhvl9nD8/zjwAsC5BfvANVQVhpmERKPksYN
9T0L9SX83HokibmO3bZzb5DTK1eJGQDeysgznNHDERZIHTF7W6rq+7lVoqG3wj7auX3F
jsVllk6E7yXxtuBeJ3PQO9ldtaNU/TxaLy3u7Cq2sqlaR5ttqS003cIO/M1IZo/Kr3oT
NtqQ==
X-Gm-Message-State: AO0yUKUng8IxDTR5Pa4seNHrOauqQx1ULgwNakQLuDabR5Df/CR+pbfh 52r6R/0O8UXEuIp4MustAWlEXSMAeWz8hcEWmUwGn5aF1s8PEz6f+UvEcEg=
X-Received: by 2002:a50:bb48:0:b0:4ac:b8e1:7410 with SMTP id y66-20020a50bb48000000b004acb8e17410mr2364158ede.6.1676522552160;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Forwarded-To: [email protected]
X-Forwarded-For: chris@personal_test_domain. [email protected]
Delivered-To: jerome@personal_test_domain.
Received: by 2002:a54:2789:0:b0:1f9:34b:de17 with SMTP id n9csp225579ecp;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
X-Google-Smtp-Source: AK7set9WzXuuJYVtYFjdtn1LrQnmLKtM2tv4acM4vnzclcNgqHEAS0FQQMXr004S9ccLFIJOWep/
X-Received: by 2002:a17:902:e5c3:b0:19a:a80e:a6d5 with SMTP id u3-20020a170902e5c300b0019aa80ea6d5mr5626264plf.23.1676522545554;
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
ARC-Seal: i=2; a=rsa-sha256; t=1676522545; cv=pass;
d=google; s=arc-20160816;
b=fWktl8SgvQeHJZHh5vkX/H+BDnrVsEHD2gJQ7z5kAAcO+0G3MzKJiksm5Q3Tma46/s
vPBk1I9HeFFlmOVDNfZzrpSqNtKzrbRh6KDSFgumiAl/IYWzyul/Y9izd3uWs0IoQhBT
+SutRjEE5ZqgR5bLbNbBBaAkpVIWIbj3PEHxHR3fIrykqReaC0S9x/IlcTBRXdji0I/Y
HbVFL9oiyLU3JoV5HUuU//oQbT648XPTZeawUxP41Hz8PJDYj3riyo32XmlxRNLXRvTZ
+Zb2x6EPOQezwDXb7XR8CgAIQ4KJvxIl7IuArmOXQTRf45gCZywhEMHUxKtW0o/IkZNw
qzjA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google; s=arc-20160816;
h=message-id:date:to:from:subject:mime-version:dkim-signature;
bh=f3lQdqf4Na3Uj8i45MnNgmpkfOMWo5t8aOYO8suzazE=;
b=ED7g4/sG1svFF6LH/QjWHutvwM/kYqlW3n6IUmuCdvqUHsRR9JFqwwE4Sj/1Xjf8qA
gUUUWgGWSxsVC6Oqoqt48PjmRGuVq8y5LYIIGNHgfe/FOScOYl2w1mJup16MwTrXlq51
QF9jJe6fGH9P/uBLUC0QwpwFhAmHVjbwMXsw1zoobjmkKNHRERJWUTzLjNWiiVYmeVog
CvwzW49kRjiapIlQnGCnIje7c4ywLtsU9z6g6VIxwyHoJHEWMO4HdHbGsiwx6LL3VT5O
rv0bJ5lHZnpZnnhWZES+Q8ewr/BcKB/0bSFclfDMPBtbKWM4AVF1dfNcIjRTh8cRdPV2
/LNQ==
ARC-Authentication-Results: i=2; mx.google;
dkim=pass header.i=@production_domain header.s=emailProvider header.b=er…YA;
arc=pass (i=1);
spf=neutral (google: 23.83.212.48 is neither permitted nor denied by best guess record for domain of development-quote@production_domain) smtp.mailfrom=DEVELOPMENT-quote@production_domain
Return-Path: <DEVELOPMENT-quote@production_domain>
Received: from dog.elm.relay.mailchannels (dog.elm.relay.mailchannels. [23.83.212.48])
by mx.google with ESMTPS id e10-20020a170902784a00b00194a1f665b5si293781pln.570.2023.02.15.20.42.23
(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
Wed, 15 Feb 2023 19:45:01 -0500 (ET)
Received-SPF: neutral (google: 23.83.212.48 is neither permitted nor denied by best guess record for domain of development-quote@production_domain) client-ip=23.83.212.48;
X-Sender-Id: emailProvider|x-authsender|[email protected]_domain
Received: from relay.mailchannels (localhost [127.0.0.1]) by relay.mailchannels (Postfix) with ESMTP id 3E82F8810AA; Thu, 16 Feb 2023 04:42:23 +0000 (UTC)
Received: from pdx1-sub0-mail-a306.emailProvider (unknown [127.0.0.6]) (Authenticated sender: emailProvider) by relay.mailchannels (Postfix) with ESMTPA id 8A…14; Thu, 16 Feb 2023 04:42:22 +0000 (UTC)
ARC-Seal: i=1; s=arc-2022; d=mailchannels; t=76…54; a=rsa-sha256; cv=none; b=qk…nC/bA…de/ Dc…mf/s8…3u+XQ…Wx+8c/D…g/SR…up+zl…zf…==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels; s=arc-2022; t=16…25; h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
dkim-signature; bh=lQ…sY=; b=eK…+/Qu…+m2CZ9JA7 lp…+tY…+lvjD 7T…+b…JK…+dh…+D/yytOf Yt…+ht/Kf0rAlyIeuNn7tT9Wu1au4/dR…+ch…w1/gz…+7Z==
ARC-Authentication-Results: i=1; rspamd-b9c55767f-ngqs4; auth=pass smtp.auth=emailProvider smtp.mailfrom=DEVELOPMENT-quote@production_domain
X-Sender-Id: emailProvider|x-authsender|[email protected]_domain
X-MC-Relay: Neutral
X-MailChannels-SenderId: emailProvider|x-authsender|[email protected]_domain
X-MailChannels-Auth-Id: emailProvider
X-Relation-Continue: 21…07
X-MC-Loop-Signature: 22…30:69…94
X-MC-Ingress-Time: 65…54
Received: from pdx1-sub0-mail-a306.emailProvider (pop.emailProvider [ip.ad.dr.es.s1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by ip.ad.dr.es.s9 (trex/6.7.1); Thu, 16 Feb 2023 04:42:23 +0000
Received: from 1.0.0…0.0.0.0.0.0.ip6.arpa (cpe-71-68-87-31.carolina.res.rr [ip.ad.dr.es.s]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: [email protected]_domain) by pdx1-sub0-mail-a306.emailProvider (Postfix) with ESMTPSA id Mk…Qz; Wed, 15 Feb 2023 19:45:01 -0500 (ET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=production_domain; s=emailProvider; t=16…25; bh=lQ…az=; h=Content-Type:Subject:From:To:Date; b=er…YA…gV/wq…9G
Ky…z3/iA…QG/eb…we+aY/4p…cD/y8…7d
WtajPHs+5Z…YB+xX…5D
Z6…UP+j2…KJ
I2…ue/6H…l6/Rx…83+RZ…8D/zD…mP
jB…iC/w==
Content-Type: multipart/mixed; boundary="===============867…5309=="
MIME-Version: 1.0
Subject: Quote - QUOTE-FSL-021523-2342
From: DEVELOPMENT-quote@production_domain
To: other@personal_test_domain., second@personal_test_domain., third@personal_test_domain., [email protected]
Date: Thu, 16 Feb 2023 04:42:19 -0000
Message-ID: <867…5309@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa>
--===============45…72==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
High Level Notes:
quote with attachments
Purchase Order: na
Regular thank you text.
Regular body Text.
--===============45…72==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="/path/to/quote.pdf"
--===============45…72==
Content-Type: image/jpeg
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="_bosch_cp1.jpeg"
--===============45…72==
Content-Type: image/jpeg
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="_cr_valve_bosch.jpeg"
--===============45…72==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="prod_SERVER_quote.pdf"
--===============45…72==--
THIS IS THE GMAIL API MESSAGE, received with three "empty" attachments, sudo-anonymised
Received: from 21…17 named unknown by gmailapi.google with
HTTPREST; Mon, 17 Mar 2025 09:34:56 -0500
Received: from 21…17 named unknown by gmailapi.google with HTTPREST; Mon, 17 Mar 2025 09:34:56 -0500
Content-Type: multipart/mixed; boundary="===============50…13=="
MIME-Version: 1.0
From: [email protected]
To: q@dev_domain
Cc: cc@dev_domain
Subject: Quote - Q-DFD-12345-031725-L0KQ-1034
Date: Mon, 17 Mar 2025 09:34:56 -0500
Message-Id: <oB…[email protected]>
--===============50…13==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
DEFAULT MESSAGE No message body text was provided - Have a nice day
--===============50…13==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 4.44.11PM.png"
--===============50…13==
Content-Type: image/png
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Screenshot 2025-03-12 at 1.54.01PM.png"
--===============50…13==
Content-Type: application/pdf
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="Q-DFD-12345-031725-UNIQ-1034.pdf"
--===============50…13==--
Thanks in advance :)
Share Improve this question edited Mar 18 at 22:53 chris Frisina asked Mar 15 at 16:15 chris Frisinachris Frisina 19.3k23 gold badges91 silver badges172 bronze badges 7 | Show 2 more comments4 Answers
Reset to default 1 +500import base64
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.base import MIMEBase
from email import encoders
from googleapiclient.discovery import build
from google.oauth2.credentials import Credentials
import mimetypes
def create_message_with_attachments(sender, to, subject, message_text, files):
message = MIMEMultipart()
message['to'] = to
message['from'] = sender
message['subject'] = subject
msg = MIMEText(message_text)
message.attach(msg)
# Attach files
for file in files:
file.seek(0) # Ensure you're starting at the beginning of the file
file_content = file.read()
mime_type, encoding = mimetypes.guess_type(file.name)
main_type, sub_type = (mime_type or 'application/octet-stream').split('/', 1)
part = MIMEBase(main_type, sub_type)
part.set_payload(file_content)
encoders.encode_base64(part)
part.add_header('Content-Disposition', f'attachment; filename="{file.name}"')
message.attach(part)
raw_message = base64.urlsafe_b64encode(message.as_bytes()).decode()
return {'raw': raw_message}
def send_message(service, user_id, message):
try:
sent_message = (service.users().messages().send(userId=user_id, body=message)
.execute())
print(f'Message Id: {sent_message["id"]}')
return sent_message
except Exception as error:
print(f'An error occurred: {error}')
return None
# Assuming you have the Gmail API setup and credentials
creds = Credentials.from_authorized_user_file('path_to_credentials.json')
service = build('gmail', 'v1', credentials=creds)
message = create_message_with_attachments(
sender="[email protected]",
to="[email protected]",
subject="Your Subject",
message_text="Email body goes here.",
files=request.FILES.values()
)
send_message(service, "me", message)
I tried to find the simplest working solution that requires the least amount of new code. For this purpose I adapted your previous code in which you used SMTP
and mail.send(...)
. I took your emailTheQuote
function as a basis and modified it a bit to work with GmailAPI
. The working version will look like this:
def emailTheQuote(quote, attachments=None):
from_email = ...
subject = ...
email_context = ...
html_content = render_to_string(
"email/tempalte.html",
email_context,
quote.request,
)
to_emails = ... # (string, [] or whatever format needed)
# Your other code
mail = EmailMultiAlternatives(
subject,
strip_tags(html_content),
to=to_emails,
bcc=bcc_emails,
from_email=from_email,
)
mail.attach_alternative(html_content, "text/html")
mail.attach(
filename=quote.pdf_canvas._filename,
content=open(quote.canvas._filename, "rb").read(),
mimetype="application/pdf"
)
if quote.attachments:
for attachment in quote.attachments:
mail.attach(
# Use public attribute `name` instead of `_name`
filename=attachment.name,
content=attachment.read(),
mimetype=attachment.content_type,
)
message_bytes = mail.message().as_bytes(linesep='\r\n')
raw_message = base64.urlsafe_b64encode(message_bytes)
gmail_response = SendMessageInternal(
incoming_message_body={'raw': raw_message.decode()},
request=quote.request,
)
print(gmail_response)
# Now save some data from `gmail_response` to DB
As you can see, there are only a few new lines of code here:
message_bytes = mail.message().as_bytes(linesep='\r\n')
raw_message = base64.urlsafe_b64encode(message_bytes)
gmail_response = SendMessageInternal(
incoming_message_body={'raw': raw_message.decode()},
request=quote.request,
)
Essentially when you create an instance of EmailMultiAlternatives
you are using a convenient wrapper that encapsulates all the low-level logic. When calling the EmailMultiAlternatives(...).send
method, the _send
method will eventually be called is what ultimately gives us a clue as to what methods will be called when the email is actually sent. All we need to do is use similar logic, call EmailMultiAlternatives(...).message()
convert the message to a set of bytes and encode the bytes using base64.urlsafe_b64encode
(this is described in documentation).
"raw": "A String", # The entire email message in an RFC 2822 formatted and base64url encoded string. Returned in
messages.get
anddrafts.get
responses when theformat=RAW
parameter is supplied.
Then delegate the sending of the message to the Gmail
client. The actual description of the message
method can be found here, here is a small part of its description from the documentation:
message()
constructs adjango.core.mail.SafeMIMEText
object (a subclass of Python’sMIMEText
class) or adjango.core.mail.SafeMIMEMultipart
object holding the message to be sent.
Now it should work correctly as before, when using SMTP
. But since you didn't provide some of your functions, I wrote some of my own code to test this. If anyone wants to test the code below, but has never dealt with Oauth
and GmailAPI
before, here's my answer it's a bit different, but it has a lot of useful links that may give you a good idea of the correct path and a general understanding of how it works.
# views.py
import base64
from collections.abc import Iterator
from pathlib import Path
from typing import TypeAlias
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient import discovery
from googleapiclient.errors import HttpError
from django import forms
from django.core.files.uploadedfile import UploadedFile
from django.core.mail import EmailMultiAlternatives
from django.http import JsonResponse
from django.shortcuts import render
from django.template.loader import render_to_string
from django.urls import path
FilesIt: TypeAlias = Iterator[UploadedFile]
PARENT_DIR_PATH = Path(__file__).resolve().parent
SCOPES = [
'https://mail.google/',
]
def create_gmail_client(port: int) -> discovery.Resource:
# Path to file with google Oauth2 credentials
credentials_file = PARENT_DIR_PATH / 'credentials.json'
flow = InstalledAppFlow.from_client_secrets_file(
str(credentials_file),
SCOPES,
)
creds = flow.run_local_server(port=port)
try:
# Create and return the Gmail API client
return discovery.build(
serviceName='gmail',
version='v1',
credentials=creds,
)
except HttpError as error:
print(f"An error occurred: {error}")
raise
def create_and_send_email_using_gmail_api(request_files: FilesIt) -> dict:
from_email = ...
subject = 'Message Title'
to_emails = [
# designate recipients
]
html_content = render_to_string(template_name='test_email.html')
mail = EmailMultiAlternatives(
subject=subject,
body=html_content,
to=to_emails,
from_email=from_email,
)
mail.attach_alternative(html_content, 'text/html')
for file in request_files:
with file.open(mode='rb') as f:
mail.attach(
filename=file.name,
content=f.read(),
mimetype=file.content_type,
)
pdf_filename = 'dummy.pdf'
test_local_pdf_file = PARENT_DIR_PATH / pdf_filename
mail.attach(
filename=pdf_filename,
content=test_local_pdf_file.read_bytes(),
mimetype='application/pdf',
)
message_bytes = mail.message().as_bytes(linesep='\r\n')
raw_message = base64.urlsafe_b64encode(message_bytes)
service = create_gmail_client(port=60257)
gmail_response = (
service
.users()
.messages()
.send(userId='me', body={'raw': raw_message.decode()})
.execute()
)
print(gmail_response)
# {
# 'id': '195b44fed...',
# 'threadId': '195b44fed...',
# 'labelIds': ['UNREAD', 'SENT', 'INBOX']
# }
return gmail_response
def test_gmail_service(request):
if request.method == 'POST':
gmail_response = create_and_send_email_using_gmail_api(
request_files=request.FILES.values(),
)
return JsonResponse(data=gmail_response, status=201)
return render(
request=request,
template_name='test_template.html',
context={'form': FilesForm()},
)
class FilesForm(forms.Form):
file1 = forms.FileField()
file2 = forms.FileField()
{#test_email.html#}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<h1>Hello World!</h1>
<p><b>Lorem Ipsum</b> is simply dummy text of the printing and typesetting industry.
Lorem Ipsum has been the industry's standard dummy text ever since the 1500s,
when an unknown printer took a galley of type and scrambled it to make a type
specimen book. <i>It has survived not only</i> five centuries, but also the leap into
electronic typesetting, remaining essentially unchanged. It was popularised
in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages,
and more recently with desktop publishing software like Aldus PageMaker
including versions of Lorem Ipsum.
</p>
</body>
</html>
{#test_template.html#}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
</head>
<body>
<div>
<main>
<form method="post" enctype="multipart/form-data">
{% csrf_token %}
{{ form.as_p }}
<button type="submit">SEND FILES</button>
</form>
</main>
</div>
</body>
</html>
Everything is as simple as possible - one view
function that accepts files from the form (don't fet to add it to urlpatterns
). Also a create_and_send_email_using_gmail_api
function that copies almost exactly the behavior of your emailTheQuote
function. And two simple html
templates. I also put two files (at the same level where views.py
is): a file with google credentials
and a simple pdf
for testing.
This has been tested with different file types, different file sizes, and it gives me the correct results. Here is an example of a message with attachments inside that I received.
Here are some helpful links:
- Documentation client to work with
GmailAPI
. - Example from the documentation on sending emails (there is an example in
python
). - Guide from the getting started documentation with
python
. - API documentation for the
users.messages.send
method (you can check sending directly from the page).
And finally I would like to add this piece:
If you're trying to send a reply and want the email to thread, make sure that:
- The
Subject
headers match- The
References
andIn-Reply-To
headers follow the RFC 2822 standard.
I think this might also be useful for you if you plan to use threadId
to reply to a message. As a reminder, you can pass additional headers this way:
EmailMultiAlternatives(headers={'References': ...})
I don't know if I got what you need exactly, but yes you can send multiple file types in the same gmail message...
I have some code which I use to prepare such messages:
"""
This module represents a class that is responsible for sending emails all over the
project.
"""
import logging
from smtplib import SMTPException
from django.core.mail import EmailMessage
from django.core.mail.backends.smtp import EmailBackend
from apps.core.models import EmailSettings
from ..enum.security_option_choices import SecurityOptionChoices
from .constants import (
EMAIL_SENT_SUCCESSFULLY,
FAILED_TO_SEND_EMAIL,
NO_EMAIL_SETTINGS,
NOT_SET_SETTINGS_MESSAGE,
SMTP_ERROR_MESSAGE,
)
logger = logging.getLogger("emails")
class CustomEmailMessage(EmailMessage):
"""
A class that subclasses the container for email information.
It customizes the attachment structure. The original structure of an attachment is a
tuple, it re-anizes its fields in a dictionary with clear keys: filename,
content and mimetype.
"""
def __init__(
self,
subject="",
body="",
from_email=None,
to=None,
bcc=None,
connection=None,
attachments=None,
headers=None,
cc=None,
reply_to=None,
):
"""
Overrides __init__ method in EmailMessage class to customize the attachments
structure.
The structure of an attachment is a tuple, so we anize its fields in a
dictionary with clear keys: filename, content and mimetype then we unpack that
dictionary to fit the original __init__ method.
"""
custom_attachments = []
if attachments:
for attachment in attachments:
filename = attachment["filename"]
content = attachment["content"]
mimetype = attachment["mimetype"]
attachment_tuple = (filename, content, mimetype)
custom_attachments.append(attachment_tuple)
super().__init__(
subject=subject,
body=body,
from_email=from_email,
to=to,
bcc=bcc,
connection=connection,
attachments=custom_attachments,
headers=headers,
cc=cc,
reply_to=reply_to,
)
class CustomEmailBackend(EmailBackend):
"""
A custom email backend for Django that retrieves email settings dynamically
from the database. It allows for flexible configuration of SMTP settings
such as host, port, username, password, and security protocol (TLS/SSL/None).
"""
def __init__(self, email_settings=None, **kwargs):
if email_settings is None:
try:
email_settings = EmailSettings.objects.get(id=1).__dict__
except Exception:
logger.error(
msg=NO_EMAIL_SETTINGS,
exc_info=True,
)
raise Exception(NOT_SET_SETTINGS_MESSAGE)
super().__init__(
host=email_settings["host"],
port=email_settings["port"],
username=email_settings["host_user"],
password=email_settings["host_password"],
use_tls=True
if email_settings["security_option"] == SecurityOptionChoices.TLS
else False,
use_ssl=True
if email_settings["security_option"] == SecurityOptionChoices.SSL
else False,
)
def _send(self, email_message):
"""
Sends an email message using the configured SMTP settings.
Sets the 'from_email' attribute of the email message to the username configured
for SMTP authentication before sending.
"""
email_message.from_email = self.username
return super()._send(email_message)
class EmailWrapper:
"""
Class to be used to define general send_mail logic.
"""
@staticmethod
def send_mail(subject, body, recipients, attachments=None, email_settings=None):
"""
Send mail is a wrapper to be used widely in core app to send various types of
emails like HealthChecks or ReportExport.
:param subject: subject of the mail
:param body: the html to be included in the mail.
:param recipients: list of emails to be sent to.
:param attachments: list of files to be included in the email.
:param email_settings: is the connection we are using. If it is not passed then
we should use the default one.
"""
attachment_names = []
if attachments:
attachment_names = [
attachment.get("filename") for attachment in attachments
]
try:
connection = CustomEmailBackend(email_settings)
connection.open()
mail = CustomEmailMessage(
subject=subject,
body=body,
to=recipients,
attachments=attachments,
connection=connection,
)
mail.content_subtype = "html"
# Send the email
mail.send()
connection.close()
except SMTPException as smtp_exception:
logger.exception(
msg=SMTP_ERROR_MESSAGE,
extra={"subject": subject, "attachments": attachment_names},
exc_info=True,
)
raise smtp_exception
except Exception as e:
logger.error(
msg=FAILED_TO_SEND_EMAIL,
extra={"subject": subject, "attachments": attachment_names},
exc_info=True,
)
raise e
else:
logger.info(
EMAIL_SENT_SUCCESSFULLY,
extra={"subject": subject, "attachments": attachment_names},
)
I am using dynamic mail settings (not static), but you can ignore that.
this is how I use this EmailWrapper:
body = render_to_string(TEMPLATE_PATH, CONTEXT_VARIABLES)
recipients = ...
subject = ...
message_id = EmailWrapper.send_mail(
subject=subject,
body=body,
recipients=recipients,
attachments=attachments,
)
This sends emails via Gmail API in Django, grabbing attachments from request.FILES.getlist()
It Returns the message ID for threading replies. In that solution, you'll need to pass csrf_token
directly to the creds
arg.
from googleapiclient.discovery import build
from google.oauth2.credentials import Credentials
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.base import MIMEBase
from email import encoders
import base64
import mimetypes
import os
from django.http import HttpResponse
def send_email_with_attachments(sender, to, subject, msg_html, msg_plain, files=None, cc=None, bcc=None):
creds = Credentials.from_authorized_user_file('csrf_token.json', ['https://www.googleapis/auth/gmail.send'])
service = build('gmail', 'v1', credentials=creds)
msg = MIMEMultipart('mixed')
msg['From'] = sender
msg['To'] = ', '.join(to) if isinstance(to, list) else to
msg['Subject'] = subject
if cc: msg['Cc'] = ', '.join(cc) if isinstance(cc, list) else cc
if bcc: msg['Bcc'] = ', '.join(bcc) if isinstance(bcc, list) else bcc
alt = MIMEMultipart('alternative')
rel = MIMEMultipart('related')
rel.attach(MIMEText(msg_html, 'html'))
alt.attach(MIMEText(msg_plain, 'plain'))
alt.attach(rel)
msg.attach(alt)
if files:
if not isinstance(files, list): files = [files]
for file in files:
filename = os.path.basename(file.name if hasattr(file, 'name') else str(file))
file_data = file.read() # bytes
content_type = mimetypes.guess_type(filename, strict=False)[0] or 'application/octet-stream'
main_type, sub_type = content_type.split('/', 1)
part = MIMEBase(main_type, sub_type)
part.set_payload(file_data)
encoders.encode_base64(part) # Handles bytes directly
part.add_header('Content-Disposition', 'attachment', filename=filename)
msg.attach(part)
raw = base64.urlsafe_b64encode(msg.as_bytes()).decode()
message = service.users().messages().send(userId='me', body={'raw': raw}).execute()
return message['id']
def email_view(request):
if request.method == 'POST':
files = request.FILES.getlist('attachments') if 'attachments' in request.FILES else None
msg_id = send_email_with_attachments(
sender='[email protected]',
to=request.POST.get('to', '[email protected]'),
subject=request.POST.get('subject', 'Test'),
msg_html=request.POST.get('body', '<p>Hi</p>'),
msg_plain=request.POST.get('body', 'Hi'),
cc=request.POST.get('cc'),
bcc=request.POST.get('bcc'),
files=files
)
return HttpResponse(f'Sent! ID: {msg_id}')
return HttpResponse('POST with "to", "subject", "body", optional "cc", "bcc", "attachments".')
emailTheQuote
(with gmail api) function the variable input isattachments
but you passattachmentFiles=email_attachments
whereemail_attachments
defined – mehdi ahmadi Commented Mar 17 at 23:34[<InMemoryUploadedFile: Screenshot.jpg (image/jpg)>, <InMemoryUploadedFile: Screenshot2.png (image/png)>, <_io.BufferedReader name='/path/to/quote.pdf'>]
. some are user uploaded, one is the generated pdf. Ill update the argument var name to match. – chris Frisina Commented Mar 18 at 3:06