This is part 4 of my explanation of my gmail_imap (python) example library, please refer to parts 1, 2, 3.
The worst point of the module currently is the method to get a full message … gird your loins, then take a look:
#gmail_messages.py
def getMessage(self, server, mailbox, uid):
if(not server.loggedIn):
server.login()
server.imap_server.select(mailbox)
status, data = server.imap_server.uid('fetch',uid, 'RFC822')
The only thing strange in this code is the use of the imap_server.uid() function, which is a wrapper function for all the standard commands listed, and the use of a magic constant ‘RFC822′ which tells the IMAP server to send all the standard mail components.
#Taken from http://discussion.forum.nokia.com/forum/showpost.php?p=640492&postcount=10
messagePlainText = ''
messageHTML = ''
for response_part in data:
if isinstance(response_part, tuple):
msg = email.message_from_string(response_part[1])
# http://docs.python.org/library/email.parser.html#email.message_from_string
for part in msg.walk(): #http://docs.python.org/library/email.message.html
if str(part.get_content_type()) == 'text/plain':
messagePlainText = messagePlainText + str(part.get_payload())
if str(part.get_content_type()) == 'text/html':
messageHTML = messageHTML + str(part.get_payload())
#create new message object
message = gmail_message()
... populate message fields ...
return message
The specifics of those steps are available at the urls provided and are fairly readable. Once again the response back from the fetch call is a tuple (status, email_string) which the python package email can interpret with email.message_from_string(email_string). The end result is pushed into a gmail_message object, and returned to the caller.
A quick note about gmail_message.py which encapulates the gmail_message structure:
The boiler plate __init__ / __repr__ of the gmail_message code is a bit too long to post. Needless to say, it’s all pretty readable, the only interesting part is that the gmail_message.Subject member defaults to ‘( no subject )’ instead of None. This may be a bad call, but it makes life simpler for the moment, as this means we don’t need to check if a subject exists when using a message later; every message is contracted to have a subject.
Phew…several posts later, what do we have? A half-way viable method to interact with gmail imap in python, without needing fragile libraries, without downloading every email in our gmail account, and without insulting the hard working people of the Python libraries or Google. Well, maybe not that last part.
If you download the module and collect all the files together, you will be able to make calls like:
if __name__ == '__main__':
import getpass
gmail = gmail_imap(getpass.getuser(),getpass.getpass())
gmail.mailboxes.load()
print gmail.mailboxes
gmail.messages.process("INBOX")
print gmail.messages
#Print the full message text for the first two messages in the box
for message_stub in gmail.messages[0:2]:
message = gmail.messages.getMessage(message_stub.uid)
print message
print message.Body
gmail.logout()
You can download all the code in this example, from the
github code repo. Feel free to fork it.
And leave me a comment to know you put up with my snark all the way to the end.
How does one get the message to be output as the original text and not HTML?
Excellent question friend,
By default, the snippet I pulled returns the HTML mime section of a message if one exists. If you’d rather always get the plain text version, do the following:
1. Grab a copy of my files.
2. Open file:
gmail_messages.py
3. Comment out ( add a # before ) lines 99,100,101:
if(messageHTML != '' ): message.Body = messageHTML else:4. You may also need to de-indent line 102:
message.Body = messagePlainText5. Save, and it should just work.
Thanks for playing.
Verpa, thanks for your hint — it worked.
I have one more question. I would like to get, for each selected message, its complete text, including all the headers, such as saved by mailers like Thunderbird.
– tsf
That’s a bit different from where I was going with my code … you’d need some substantial changes. Unfortunately I wouldn’t have time to test these sort of changes out until the weekend…but if you ping me again on friday I’ll try to find time to give it a whirl.
What’d you need to do is:
(gmail_messages.py) line 51:
change ‘BODY.PEEK[HEADER.FIELDS (FROM SUBJECT DATE)]‘ to either:
BODY.PEEK[] or
BODY.PEEK[HEADER.FIELDS]
not sure which … you’d need to look it up in the IMAP documentation. Good luck!
Then you’d need to add a new field to the object in ‘gmail_message.py’ probably would call it ‘headers’.
Then add a new line to the gmail_messages.py around line 58, saying message.headers = headers
Finally, you’d modify the gmail_message.py print routine to dump that headers dictionary when outputting a message, if you want. If you do that, be sure to remove the current headers being shown ‘From’,'Subject’,'Date’ to avoid duplication.
From then on, after you’d done the prelim work ( find a folder, load the messages, etc ) you should get a dump of the headers.
Like I said, not sure when I’d get around to it, since I’ve moved on a bit. But if you have further questions or really need someone else to try, remind me and I’ll try to make time.
Thanks for your hints. I will try it one of these days!
– Tad
For anyone who gets this far…note that Google has added OAuth as valid method of logging into IMAP now:
http://googlecode.blogspot.com/2010/03/oauth-access-to-imapsmtp-in-gmail.html
It’s not a web based API, but it’s a start!
Thanks for the solution! I’ve adapted this for my code, you saved me hours of pain.
One thing I added was support for attachments. In the gmail_messages.getMessage routine, when we iterate through parts of the message we can detect attachments by looking at filename:
if part.get_filename():
attachments.append((part.get_filename(), base64.decodestring(part.get_payload())))
Hello Vlad,
Thanks for the comments. Since you and Tad both mentioned wanting to change the code a bit, what would you think if I moved this from Google’s hosting to GitHub or BitBucket? Let me know.
Verpa.
Sure, why not. A fresh place for your perfectly working code is in any case better than a directory named “failed_ideas”
I’ll branch off and check in my changes once you publish the new location.
Thanks!
Hello again,
I think I got this setup right in GitHub:
http://github.com/drewbuschhorn/gmail_imap
I’ll change the links around these posts to point over there this weekend. This was in the failed_ideas branch, since *originally* before Google Wave was declared ka-poot, I wanted to make a wave robot like this project, Emaily . That got me into the poor python imap libraries, and it seemed like an interesting thing to kvetch about. Guess it was a bit, as this series is the only thing on the blog that gets any real traffic
To anyone who comes along, the links should be changed. If you find one pointing to googlecode, let me know and I’ll change it.
[...] Python Gmail IMAP : part 4 January 2010 11 comments 3 [...]
Is it possible to pass the username and password from the command line?
Yes, the code on github uses the getpass python standard library.
http://docs.python.org/library/getpass.html
just wanted to thank you guys. I made it through the quick tutorial and it was hugely helpful.
Got the below error when trying the sample program above.
Traceback (most recent call last):
File “test.py”, line 6, in
gmail = gmail_imap(‘myuserid@gmail.com’,getpass.getpass())
TypeError: ‘module’ object is not callable
Very useful. Thanks.
Would this be able to return the unread message count?
Thanks!
–Dan