Exchange is bad
My email
I work for a big corporation that uses exchange (not many identifying features there!), which has also killed (or just plain doesn’t have) it’s IMAP/SMTP/CalDav/CardDav integration. As such, I use davmail to solve the issue of “How can I access my email using emacs”. I’ve used emacs to manage my email for a while (one of possibly a dozen people who does so, no doubt) and it’s very fast, which I mostly want. It uses OrgMsg to make emails, so they usually look passable to folks reading them also. That’s pretty important.
But what about contacts
Outlook :(
Because most of my interfacing with this Exchange server was based on Outlook before I started using davmail, I got used to having a GAL which is in turn based on the Active Directory of the company where I work. This is a cool thing to have, because I sometimes have to phone people to talk about official legitimate business problems and I generally don’t have their phone number. Davmail does expose this AD through an LDAP proxy, but because it uses the webinterface (basically) it is limited to only 100 results when you ldapsearch with no way to extend it.
Incredibly legit use-case
I wanted an offline repository of things to query, for reasons I cannot properly articulate. Maybe some vague concern I would need a phone number in a crisis situation (one of those official legitimate business problems), or because I occasionally work during travel and my phone plan is both cheap (relatively, it’s still more than 4 coffees :( ) and slow (mostly in terms of ping, as it’s 4G and 3G if I’m very lucky). For these reasons, I wanted an export of this entire directory in a local tree that I could query using some sort of vcard manager. I picked khard, because it is what most people use and I had bigger fish to fry.
Ill-advised engineering
After messing with ldapsearch parameters that the internet told me would allow me to paginate and maybe get more results, I figured out the 100 results max was a limitation of davmail. As such, I decided to do the sort of reasonable but probably kind of stupid (more on that as the story develops) thing: I cloned the GAL to my address book. This is something folks do, for reasons I will now never understand. In a directory which is very large this will crash your Outlook, or at least freeze it. After 5 hours I decided it was probably done and I killed it. I then had a directory with a lot of the contacts, and I could sync it to my local machine using vdirsyncer. This is probably a great tool for a more normal setup, but because of the volume of vcards vdirsyncer would freeze (for as long as my attention span allowed at least, maybe 15m before I killed it). When I debugged it I was fairly certain it was sending a huge request to davmail, so perhaps the integration broke or something. Who knows.
The csv method
To solve this issue, I ended up exporting a csv from excel of all of my contacts and turning that into a bunch of vcards using a little rust application. It uses vobject and csv to read a csv and turn it into vcards, respectively. Helpfully, I found an rfc for the vcard format that helped me select a uuid scheme. For this I used uuid, another nice rust library that let me generate them based on the SHA1 hash of the email address (that is always unique in this directory). I wrote these files to a directory and added this directory to khard. That solved my lookup issues.
Future plans
Because I killed my Outlook-based GAL export halfway through the process of exporting it to my local address book, I am left over with some remaining contacts that do not yet exist in my directory but do in fact exist in real life. For this I can in fact use ldapsearch:
ldapsearch -LLL \
-w $(gpg --no-tty -q -d password.gpg) \
-H ldap://localhost:1389 \
-D <user@domain.tld> \
-b ou=people \
"mail=<query>*"
I format those source blocks as bash, but I port them in-place from the nushell code that I use (for which I could find no good highlighting package, sadly). So it may not be entirely correct all the time. This time probably yes, in the future maybe no.
This type of querying also lets me enumerate changes by iterating over the entire alphabet (e.g. mail=a*, mail=b*, etc), increasing the resolution if too many results come out (e.g. mail=aa*, etc). It occurred to me too late that if I had set it up this way, it would have saved me a little work because I wouldn’t need the csv export at all (If it can diff the entire tree it can create it from scratch. Or at least in this case it can). Sadly I am not quite that smart and I have to still write this. Maybe when the ldapsearch results parser is done I will publish this as a little application, as that will be useful to other people also.
Calendar
The calendar is the last thing I was interested in having locally, for that I do definitely use vdirsyncer along with khal. That works well enough for now, and I think will never annoy me enough for me to replace it. At some point I’ll integrate it with a little noise that I will have a meeting soon, which is currently the only thing Outlook is good for. But at least I don’t have to look at or use the interface anymore :)
Concluding
I spend a few hours total debugging and programming this over a few days. The rust ecosystem is pretty mature now, so all the components were there to be used immediately with no issues. Actually chugging through that whole csv takes less than a second and there are a lot of entries, so on that front I am also happy.
Because I am trying to learn rust with practical applications, I may try replacing khard/khal with rust variants that have an index to be updated based on what is on disk. That will make the search significantly faster. Currently I have to use the f flag in khal to ensure it searches through files directly, but it says this is not optimal (I have not spent the time to figure out whether or not it actually is for my use-case).