E-mail in Emacs with mu4e on OS X

For anyone using our repo for packages on OS X (more details here), it's as easy as sudo pkgin -y in offlineimap :) For other OS's, it should be available via your package manager. It should build from source fine too, if you want to take that route.

Once you've got it installed, you can set your config up in ~/.offlineimaprc The syntax is pretty self explanatory, here's mine:

[general]
ui = TTY.TTYUI
accounts = GMail
pythonfile=~/.offlineimap.py
fsync = False

[Account GMail]
localrepository = GMail-Local
remoterepository = GMail-Remote
status_backend = sqlite

[Repository GMail-Local]
type = Maildir
localfolders = ~/.mail/gmail
nametrans = lambda folder: {'drafts':   'Drafts',
                            'sent':     '[Google Mail]/Sent Mail',
                            'trash':    '[Google Mail]/Bin',
                            'archive':  '[Google Mail]/All Mail',
                            'personal': 'Personal',
                            'linkedin': 'Personal/LinkedIn',
                            'shopping': 'Personal/Shopping',
                            'amazon':   'Personal/Shopping/Amazon',
                            'ebay':     'Personal/Shopping/Ebay',
                            'paypal':   'Personal/Shopping/PayPal',
                            'twitter':  'Personal/Twitter',
                            }.get(folder, folder)

[Repository GMail-Remote]
maxconnections = 1
type = Gmail
remoteuser = <redacted>@gmail.com
remotepasseval = get_keychain_pass(account="<redacted>@gmail.com", server="imap.gmail.com")
realdelete = no
ssl = yes
sslcacertfile = /usr/pkg/share/ncat/ca-bundle.crt
nametrans = lambda folder: {'Drafts':                   'drafts',
                            '[Google Mail]/Sent Mail':  'sent',
                            '[Google Mail]/Bin':        'trash',
                            '[Google Mail]/All Mail':   'archive',
                            'Personal':                 'personal',
                            'Personal/LinkedIn':        'linkedin',
                            'Personal/Shopping':        'shopping',
                            'Personal/Shopping/Amazon': 'amazon',
                            'Personal/Shopping/Ebay':   'ebay',
                            'Personal/Shopping/PayPal': 'paypal',
                            'Personal/Twitter':         'twitter',
                            }.get(folder, folder)
folderfilter = lambda folder: folder not in ['[Google Mail]/Bin',
                                             '[Google Mail]/Important',
                                             '[Google Mail]/Spam',
                                             ]

You'll probably have noticed pythonfile=~/.offlineimap.py. This is a really nice feature provided by offlineimap, it allows you to specify an external Python script to call to retrieve information (such as credentials).

The offlineimap.py script used here is for retrieving credentials from the OS X keychain. See here for several examples of retrieving credentials from other various sources, such as the Gnome keyring, or PGP encrypted files, etc.

I found the offlineimap.py script floating around in a few places on the web, though, each instance seemed to contain static data, such as the user's username in the script… There's nothing really wrong with that, but, hey it's Python, why not get that information in a more portable manner? So, I chucked in the user and keychain params to make it truly portable:

#! /usr/bin/env python
import getpass, re, subprocess
from os.path import expanduser, join
def get_keychain_pass(account=None, server=None):
    params = {
        'security': '/usr/bin/security',
        'command': 'find-internet-password',
        'user': getpass.getuser(),
        'account': account,
        'server': server,
        'keychain': join(expanduser('~'), 'library/Keychains/login.keychain'),
    }
    command = "sudo -u %(user)s %(security)s -v %(command)s -g -a %(account)s -s %(server)s %(keychain)s" % params
    output = subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
    outtext = [l for l in output.splitlines()
               if l.startswith('password: ')][0]

    return re.match(r'password: "(.*)"', outtext).group(1)

Note: This is specifically for the OS X keychain, it can be modified in several ways to interact with other sources, as mentioned above

So, once you've got offlineimap installed, your config and credential retrieval method in place, you can kick off your first sync by just running offlineimap in the shell. This could take a while, depending on the amount of e-mail you've got - future runs of offlineimap can be run with the -q flag, for quick retrieval - I'll be noting how this is tied in with mu4e later on.

Once offlineimap is done running, take a look in your localfolders directory (set to ~/.mail/gmail in the above example), you should see some familiar directories:

$ ls ~/.mail/gmail/
INBOX		archive		ebay		paypal		sent		trash
amazon		drafts		linkedin	personal	shopping	twitter

Provided everything is in order, you're all set with your Maildir! Next up, getting mu.

When I first set up mu, I just built from source. Easy enough, just grab the code from djcb's Github, you'd need clang or gmake to compile, along with build and run deps:

• pkg-config
• automake / autoconf / autoreconf
• sqlite3
• glib2
• gmime
• xapian

All of which are available through the Save OS X repo :)

The above works fine, and is currently how I tackled it on Fedora at work. As for OS X, it turned out there was a pkgsrc port for it in wip. Rather than dirty our 2014Q4 repo, I figured I'd just roll my own. That way, if there's anything else I want from wip, or if I want to tinker with some ports, I can just chuck binary distributions into my own personal repo.

So, I deployed a zone under SmartOS, got nginx up and running, and placed a build of mu (which I updated to the more recent 0.9.11 release) there. In order to use the package in my repo, you'll first need to import my PGP key: 1F77 49AB 744A 7E15, as I sign my packages. Then, you can either opt to add my repo (http://pkgsrc.cmacr.ae/Darwin/wip/x86_64/All) to your /usr/pkg/etc/pkgin/repositories.conf, or, provided you're all set up with Save OS X, simply run: sudo pkg_add http://pkgsrc.cmacr.ae/Darwin/wip/x86_64/All/mu-0.9.11.tgz

I haven't added any of mu's dependencies to my repo, so you'll need to have the deps installed, or have our main repo (http://pkgsrc.saveosx.org/Darwin/2014Q4/x86_64/All/) set up - easily done: Save OS X - Download & Install mu will be available in our main repo eventually though!

The binary package on my repo mentioned above is built with mu4e support, so, will also install the necessary elisp so you can use the beautiful front-end in Emacs.

If you opted to install from source, I believe mu4e support is a default compile time option - a quick note on this: if you're having trouble getting it to play nice with your site-lisp directory, when you run the configure script, prepend it by defining the EMACS variable, like so: EMACS=/Applications/Emacs.app/Contents/MacOS/Emacs ./configure This should place the elisp in /usr/local/share/emacs/site-lisp/mu4e.

If you've opted to use my binary package for mu, elisp is distributed to the pkgsrc prefix, so ends up in /usr/pkg/share/emacs/site-lisp/mu4e. My binary package has Emacs 24 as a dependency, so, upon installation will install the emacs24-24.4 package from our main repo. This shouldn't be a problem, as when using Save OS X, PATH evaluation is set up so /usr/pkg/{bin,sbin}/ are evaluated after other binary paths on your system. That means, if you have Emacs installed elsewhere that you run in the shell, provided its binary is somewhere in the PATH before the one installed from our repo, that one will be used.

Once you've gotten mu installed, you'll need to add the following somewhere in your Emacs config:

(add-to-list 'load-path "/usr/pkg/share/emacs/site-lisp/mu4e")
(require 'mu4e)
(setq mu4e-mu-binary "/usr/pkg/bin/mu")

Of course, substituting the above paths depending on how you installed. If you installed from source, replace /usr/pkg in the above with /usr/local (or whatever path you used with the --prefix option).

The above config is the bare minimum needed to use mu4e, but likely not enough to get the real benefits of its functionality. Here's part of my mail.el, where I place all my e-mail related config:

;; -------------------------------------
;; MAIL
;; -------------------------------------

;; mu4e
(add-to-list 'load-path "/usr/pkg/share/emacs/site-lisp/mu4e")
(require 'mu4e)
(setq mu4e-mu-binary "/usr/pkg/bin/mu")
(setq mu4e-maildir "~/.mail/gmail")
(setq mu4e-view-show-images t)
(setq mu4e-html2text-command "w3m -dump -T text/html")
(setq mu4e-view-prefer-html t)
(setq mu4e-use-fancy-chars t)
(setq mu4e-headers-skip-duplicates t)
(setq mu4e-get-mail-command "offlineimap -q")
(setq mu4e-update-interval 300)
(setq mu4e-attachment-dir  "~/downloads")
(setq mu4e-sent-messages-behavior 'delete)
(setq message-kill-buffer-on-exit t)
(setq mu4e-hide-index-messages t)
(add-hook 'mu4e-compose-mode-hook 'flyspell-mode)
(setq
 user-mail-address "<redacted>@gmail.com"
 user-full-name  "Calum MacRae"
 mu4e-compose-signature
 (concat
  "Kind Regards,\n"
  "Calum MacRae\n"))

This ended up being a bit longer than I expected. It's funny, looking back at how much I've written would probably give the impression that set-up is complex… it really, really isn't. Once your config is in place, and hopefully I've helped provide some skeleton configuration, it's smooth sailing and just pure enjoyment.

I'm glad I took the time to write about mu4e, it really is brilliant. The author, djcb, did such an awesome job on it, it's easily one of my favourite components of my Emacs config. He's also written sauron, an Emacs event log, with trigger actions! Really cool, I've been using it for watching IRC & Jabber mentions, but plan on throwing in e-mail alerts from mu4e for important stuff, which I'll definitely post about.

I hope I've convinced some of you to try mu4e out! Questions and feedback are, as always, very welcome :)