Making your webapp react to emails with Lamson, pt 2

This is part 2 in a series, here is part 1.

In the last section, you should have got Zed Shaw’s LamsonProject running successfully through its unit tests.

Now I’ll show you how to make what maybe the *simplest* application with Lamson that you’ll ever see. All our application is going to do is:

1. Receive emails with Lamson.
2. Pull the content from the body and subject lines using python.
3. Send the data to Google Calendar to be turned into a calendar entry using the quickAdd call, again using python.

Because Lamson is built to interact easily with Python code, this is snap.

Read Full Post »

Making your webapp react to emails with Lamson, pt 1

(quick install instructions)

If you’re a web programmer … and if you’re not, why are you reading this? … at some point you’ve wished you could integrate email into your webapp.  Now, by integrate, I don’t mean sending emails … that’s trivial.  Any language you can think of, there’s a library that will let you fire off all the html gibberish that fills your little hearts with joy.

The challenge comes when the little light goes off in your head and you think, “How hard could it be to have my app react to incoming email?”  Maybe you’d like people to make forum posts by email, or create mailing lists ( listserv ) dynamically.

The answer used to be hard … very hard.  A nightmarish mishmash of postfix, dovecot, procmail, pipes, and the odd custom script would get you something that might, maybe, let you fire off a script when a new email came in.  Actually passing email data to your app? Forget it.  And after all that, most people would rightly wash their hands of it and say ‘let’s just poll the mail server periodically’ … ick.

Why can’t I just convert emails to HTTP POST’s sent to an endpoint on my webapp?  Oh … wait, I can, thanks to Zed Shaw’s LamsonProject.

But first, before going to the trouble of setting LamsonProject up, please be aware that there exist several semi-respectable SAAS companies trying to do this for you:

• SendGrid Parse API : Email to POST.  I’ve never heard anything bad about SendGrid ( a fair amount of good stuff ) but the it is a sideline to their main business of outsourcing your SMTP server for you.
• Email Yak : Email to XML, Email to JSON, Email to POST.  In private beta.  Don’t confuse with yak mail or yak messenger.
• Cloudmailin : Email to POST … “just like a webhook”.  In beta.

Honestly, if I had two nickels to rub together I’d probably use SendGrid, but I don’t … so here we are.  On with the show!

Read Full Post »

Python Gmail IMAP : part 4

This is part 4 of my explanation of my gmail_imap (python) example library, please refer to parts 1, 2, 3.

The worst point of the module currently is the method to get a full message … gird your loins, then take a look:

        #gmail_messages.py
        def getMessage(self, server, mailbox, uid):
            if(not server.loggedIn):
                server.login()
            server.imap_server.select(mailbox)

            status, data = server.imap_server.uid('fetch',uid, 'RFC822')

Read Full Post »

Python Gmail IMAP : part 3

This is part 3 of my explanation of my gmail_imap (python) example library, please refer to parts 1, 2, 4.

Next we want to load brief forms of our messages so that we could, say, display a list of messages in our inbox.

    gmail.messages.process("INBOX")
    print gmail.messages

Unfortunately, the code is a bit too long post, so instead, pull the code for this file up here, and I’ll post the important snippets as we go along.

Reading through the code, the first weird bit is:

            self.metadataExtracter = re.compile(r'(?P\d*) \(UID (?P\d*) FLAGS \((?P.*)\)\s')

What huh…?

Read Full Post »

Python Gmail IMAP : part 2

This is part 2 of my explanation of my gmail_imap (python) example library, please refer to parts 1, 3, 4.

On to how to get our mailboxes from Gmail.  We want to be able to say something like this:

    gmail.mailboxes.load()
    print gmail.mailboxes

so that gmail.mailboxes will behave as a simple list of mailboxes names.

Read Full Post »

Python Gmail IMAP : part 1

This is part 1 of my explanation of my gmail_imap (python) example library, please refer to parts 2, 3, 4.

As I said before mocking Google for the lack of a Gmail API, we’re forced to turn to IMAP.  This being an exercise in python, I turned to the standard library docs, and got no help … worse than no help, confusing ‘help’.  Next up, I think we should repent and turn to Google to help us find example code that will show us the proper way to deal with imaplib.

To summarize the results … crap.  Some decent attempts, but mostly, as you can see in my references, crap.  Don’t trust my opinion?  Fine, I never liked you anyway … here’s a quote from the ActiveState IMAP MailWatcher recipe:

rough edges. The [example] code is organized in an awkward way. Presentation does not cleanly separate from the core code. This showed painfully when I wanted to deploy this recipe on a box where tcl/Tk was not available. Thus I wanted to replace Tkinter calls so that simply an external program gets called if there is a new mail. Ideally, this should be the matter of replacing a presentation class or function call with another one, leaving IMAP related code intact. But the Mailwatcher class just blends everything together with no concern, so I had to actually understand the IMAP code to achieve what I want, and alltogether [sic], work more on it.

Harsh, but fair.  I hope that I acquit myself better.  Code follows.

Read Full Post »

Python IMAP and other horrors

The state of IMAP access in the main python library is appalling.  I don’t mean poor quality so much as thin; the documentation is thin, the examples are thin, and the code itself is a thin layer of sugar over the raw strings being burped out by the IMAP server.  I know I can’t fix it, because well, no one cares, but I can show you how I’ve gotten it working with gmail.

Read Full Post »

Why does Gmail hate us

I want a decent Gmail API, and the gods of Google are letting me down. Accessing your email data is basically a necessity, and Gmail does its own thing so much that any current mail protocol is basically helpless to deal with it. POP is a joke (a bad one, at that) but in its defense, it’s older than any three of my students combined. IMAP is the magical white knight to solve our problems, but like all true knights, it’s more concerned with the wenches than with making sure it can actually accomplish anything in a timely fashion … I think there was a valid metaphor in there somewhere.

No big deal, if there isn’t a ‘sanctioned’ way to do this, then surely someone must have a hack.  Ah ha, I’ve found it … gmail.py it’s magical, it’s mystical … it’s broken since 2006.  Ok … how about libgmail … it’s magical, it’s myst…broken since 2008.  Oh crap…

Sunday December 13th 2009
Unfortunately the GmailFS project has come to an end. libgmail has ceased being maintained by its developers, and as a result libgmail no longer works with the latest Gmail interface (and has not done so for many weeks). Without a working libgmail, GmailFS does not function, so the end of libgmail also spells the end of GmailFS.

So where does that leave us … screwed.  That’s where.

I don’t like it any more than you, but we’re going to have to go the IMAP route (see the entry on that) and god help us all.

If you think this is an unacceptable situation, let google know by starring the issue and posting (star is next to the issue title).

For the completeness buffs out there, you could also check out the GMail Atom Feed … if you’re, you know, only interested in the last 20 unread messages and no way to find out what mailboxes exist.  Bloody useless.  Google apparently thinks you should go piss on a dog.

Read Full Post »