Friday, September 9, 2011

coming soon: memory-mapped db for OpenLDAP

short version:



I just ran across a site with news of a new database backend for OpenLDAP that's designed to be completely mapped to memory and is supposed to be faster, more memory efficient, and much easier to configure:

Memory-mapped Database for OpenLDAP


long version:



I'm a huge fan of OpenLDAP. not only is it the fastest and most scalable implementation of LDAP (see at the bottom for sources), best of all it's open-source. configuring it for optimal performance, however, is easier said than done. you have to configure indexes, the database cache size, the IDL cache size and of course the good ole entry cache size. Howard Chu, the current chief architect of OpenLDAP, describes the process pretty well:

"...it requires careful tuning to get good results and the tuning aspects can be quite complex. Data comes through three separate layers of caches before it may be used, and each cache layer has a significant footprint. Balancing the three layers against each other can be a difficult juggling act."

the good news: that quote comes from a site I just ran across where Chu announces a new database backend that's designed to be completely mapped to memory, known as "back-mdb". in Chu's description of back-mdb, he uses words like "extremely fast," "memory efficiency," and my favorite, "trivial configuration." I can't wait! (yes, I'm a giant nerd)

Chu said back-mdb will be ready sometime this month at the earliest, but whenever it comes, it will be worth waiting for. here's the link if you want to check out the details:

Memory-mapped Database for OpenLDAP

and here are those sources I promised you, ripped entirely from my wiki:

3 comments:

  1. The code has been available in git for a while now. It will also be in the upcoming OpenLDAP 2.4.27 release. You can check out my paper and presentation slides at the LDAPCon site http://www.daasi.de/ldapcon2011/index.php?site=program

    ReplyDelete
  2. thanks, Howard! you guys have made a really great piece of software with OpenLDAP. my only request: more benchmarks! the ones I listed are pretty old :)

    ReplyDelete
  3. More benchmarks are on the way. We have a new server arriving soon, will be re-testing everything to establish new baselines. the new box has 64 cores. I am expecting to find some bottlenecks in our code at first, and aim to eliminate them.

    ReplyDelete