Home > software > Full-text search in mutt: alternative notmuch integration

Full-text search in mutt: alternative notmuch integration

February 12th, 2012 Leave a comment Go to comments

If some feature is too slow, you end up conciously avoiding it and losing productivity. This is one of the reasons that we emphasize so much that Git is as fast as it is – you end up using it more because of that. One thing I always found very frustrating was full-text search in mutt; it takes _minutes_ on my mailbox and I end up trying many different header-based queries instead in order to find the mail. But today, I have finally set up notmuch, a very nice and fast mail indexer.

Unfortunately, there was no satisfying way of integrating notmuch with mutt! There is a notmuch-mutt script which creates a temporary maildir with results and moves me there. This was not going to work for me – you cannot make any changes in the “search results list” like deleting mails (I wonder if status would carry if I reply to mails; I suspect not) and in order to get back to your mails, you need to switch mailbox – which implies that your previous position is not restored and that it’s quite slow (few seconds – too much!).

What I envisioned instead was something like the ‘l’imit function that I use very much, just faster. ;-) It turns out that mutt can match message-ids in the limit query and that notmuch can output a list of message-ids of matched mails. Therefore, the most hackish approach is simply to use notmuch to generate a limit specification and perform that – and it turns out that this is good enough (in my scenario)!

Just put these two bindings (or only the first one) in your .muttrc:

# 'L' performs a notmuch query, showing only the results
macro index L "<enter-command>unset wait_key<enter><shell-escape>read -p 'notmuch query: ' x; echo \$x >~/.cache/mutt_terms<enter><limit>~i \"\`notmuch search --output=messages \$(cat ~/.cache/mutt_terms) | head -n 600 | perl -le '@a=<>;chomp@a;s/\^id:// for@a;$,=\"|\";print@a'\`\"<enter>" "show only messages matching a notmuch pattern"
# 'a' shows all messages again (supersedes default <alias> binding)
macro index a "<limit>all\n" "show all messages (undo limit)"

Perhaps sometime in the future, we will get native libnotmuch support in mutt, but I think this is a pretty good substitute for now. :-)

TODO list

  • the way this snippet prompts using a temporary file is completely absurd; mutt needs to get a builtin prompt function for its macros
  • only the most recent 600 search hits are shown, since…
  • …the filtering is grossly inefficient; it is still very fast on my computer, but if mutt could just directly get a list of message ids and match them, things would be much nicer than me abusing the regex matching machinery
  • the 600 search hits limit is global over all folders, therefore if you have a lot of mails and a lot of folders, searching for a common word may hide even some recent results
  • notmuch cannot search for substrings, apparently, only whole words
  • notmuch does not deal with diacritics and other locale transliteration character classes
Categories: software Tags: , ,
  1. February 13th, 2012 at 02:42 | #1

    You know… Thinking about this, I’d almost suggest to make an Akonadi backend for Mutt and use Nepomuk to do these queries :D

    While KMail is not particularly fast, the fact that Akonadi and Nepomuk do fine on a mobile phone proves that they are more than capable of running fast and with a low footprint – and they are designed exectly for the usecase you have (and then some).

    But of course it’s a crazy idea and I would never actually suggest it…


  2. September 25th, 2012 at 18:51 | #2

    Hi, I’m using these macros and know nothing about perl. Notmuch searching works great, but when I use certain words these macros fail. For example:

    “schedule” fails

    “caleb” passes

    “subject: schedule” fails

    “subject: caleb” passes

    Any idea what’s going on here?

  3. September 25th, 2012 at 18:52 | #3

    @Conner McDaniel
    The error looks like this: mutt(51551) malloc: *** error for object 0x7fff5fbfd540: pointer being reallocated was not allocated

  4. pasky
    October 5th, 2012 at 01:07 | #4

    @Conner McDaniel
    Ouch. That seems like internal mutt error, mostly due to a bug within mutt – you should never see errors like this. I suggest taking this to mutt developers, and/or trying newer/older mutt versions – I have never hit an error like this myself with mutt-1.5.21.

  1. October 22nd, 2012 at 22:53 | #1

7 × one =