Skip to content

Oracle Internet Directory and 2 Billion Entries?

LinuxOpenLDAP I've just finished reading The 2 billion entry directory tree - The scalability of Oracle Internet Directory at very large Directory Information Tree sizes, and thought I'd blog some initial thoughts (I'm sure our partners Symas will have something to say later).

The following is from the 2 Billion User Benchmark (Oracle Internet Directory 10.1.4.0.1) - Technical Whitepaper (PDF) on the Oracle Internet Directory homepage.

So here's the Network kit they used:



Some pretty accessible kit there! ;-)
Summary of Bulk Load Timings for the first Billion entries
- Offline Activity = 35 hours 47 mins
- Bulk Load “load” phase = 2 hours 28 mins
- Database Index creation = 16 hours

Summary of Bulk Load Timings for the second Billion entries
- Offline Activity = 33 hours 01 mins
- Bulk Load “load” phase = 2 hours 32 mins
- Database Index creation = 19 hours 41 mins


Quite a while. I wonder how long Active Directory would take...years.


4.2.5 Tuning and Optimization
- The Test Clients
SLAMD clients were used to generate LDAP[?] operations load in most of the test. A total of 8 nodes were available for the tests and for any given test we utilized all of them.
- OID LDAP Instance Configuration
-- ‘orclserverprocs’ was set to 72, this is the number of OID LDAP Server processes used in the OID LDAP instance.
-- ‘orclmaxcc’ was set to 1, this influences the number of threads and database connections used by OID LDAP Server processes.
- OID Root DSE
-- In the root DSE entry, the attribute ‘orclmatchdnenabled’ was set to 0 (zero) to disable detailed matching DN information
- OID DSA Configuration
-- In the entry “cn=dsaconfig,cn=configsets,cn=oracle internet directory”, the following attributes were updated –
'orclskiprefinsql' was set to 1, this enables optimized SQL generation and can be used when LDAP referral entries are not present in the DIT as was the case in this benchmark. Presence of LDAP referral entries is not common.
'Orclmaxconnincache' was set to 300000000.
- Password Policies and Verifier Profiles
-- The Password Policies on OID were disabled.
- OID Change Log Generation
-- OID Change log was disabled.


Also quite a few things disabled.


4.4 Conclusion
The following conclusions were drawn from the benchmark –
• OID installation and configuration is straightforward and does not require undue time or effort.
• The Bulk Load is a very fast operation in OID and 2Billion users LDIF data was loaded in under 118 hours.
• It was observed that, for the chosen test scenarios, OID could easily scale up to 101,000 searches/sec, 37,000 authentications/sec, 80,500 binds/sec, 14,000 modifications/sec and 84,000 comparisons/sec. OID sustained 99,500 searches/sec while serving 16,000 concurrent clients, beyond which, even though the system resources were not fully utilized, the load generation mechanism becomes the bottleneck.
• Given our experience with OID, we feel it is suitable for high-end mission critical applications requiring LDAP services, owing to its performance in the set of experiments conducted.


Now the above is the interesting part, let me explain. For the kit shown above in their image, I would expect much, much better results than above. OpenLDAP has been benchmarked doing pretty close to some of these results on only simple servers with average amounts of RAM

At a rough guess if the same equipment above was used in the above OpenLDAP tests, OpenLDAP would easily be around 5 times faster.

Lastly, in the article it implies that "open source" is not "high end"...hmmm, who owns BerkeleyDB again? and says:

BENEFITS
- Extreme Performance—eliminates client-server communication and SQL processing delays for applications with static access patterns
- High Reliability and Availability—full transactional semantics ensures data integrity and recovery from failures, and replication provides high availability and scalability
- Zero Administration—applications perform database administration, eliminating the need for a DBA and allowing continuous, unattended operation
- Low TCO—extreme performance reduces hardware costs and being embedded eliminates DBA costs



Gavin.

Trackbacks

No Trackbacks

Comments

Display comments as Linear | Threaded

Anonymous on :

*Is there a openldap benchmark papers with billion entries ?

Gavin Henry on :

*See http://www.symas.com/blog/?p=186#more-186

Add Comment

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
BBCode format allowed
Pavatar, Gravatar, Favatar, MyBlogLog, Pavatar author images supported.
Form options
tweetbackcheck