Search code examples
artificial-intelligenceclipsexpert-systemknowledge-management

CLIPS programming fact and rule base sizes


What is the scalability of Clips..that is what is the maximum limit on the number of rules and facts in CLIPS ? I am using CLIPS 6.24 but unable to upload one million facts!!


Solution

  • The implementation of the rete algorithm in CLIPS shares common patterns between rules, so pattern matching performance will scale reasonably as you add rules. The worst case performance will occur when no patterns are shared between rules.

    A large number of improvements were also made in version 6.3 to improve performance for large numbers of facts:

             CLIPS (V6.24 06/15/06)
    CLIPS> (timer (loop-for-count (?i 100000) (assert (data ?i))))
    3.78414106369019
    CLIPS> (reset)
    CLIPS> (timer (loop-for-count (?i 1000000) (assert (data ?i))))
    885.884355068207
    CLIPS> 
    
             CLIPS (6.30 3/17/15)
    CLIPS> (timer (loop-for-count (?i 100000) (assert (data ?i))))
    0.136654
    CLIPS> (reset)
    CLIPS> (timer (loop-for-count (?i 1000000) (assert (data ?i))))
    6.046085
    CLIPS>
    

    I haven't spent much time optimizing the loading of large rule sets, so it's certainly possible that this or other related development functionality doesn't scale well. I worked on an expert system using a commercial product that had a few thousand simple rules. Performance and compilation time were acceptable and probably would have scaled well, but synchronization between the desktop client and rule repository was painfully slow and buggy. I can't imagine trying to scale that application without improvements to that component of the product.

    There's a large number of factors that affect scalability beyond the number of fact and rules, notably the design of the rules, their complexity, and the relationship between the facts and the rule patterns. Your best bet when evaluating a tool is to write a simple program that generates a large number of rules similar to the ones you envision writing and associated facts that will match those rules and use those to test scalability.

    The largest CLIPS system that I've written had around 600 complex rules, processed tens of thousands of facts, and had acceptable performance with CLIPS 6.24. I don't recall anyone mentioning significantly larger systems, but practically I don't think you can scale to a million facts with 6.24 unless you're generous with what's acceptable performance.

    With 6.3, I think it's plausible that you can build a system with at least several thousand rules and a few hundred thousand facts, but again your best bet is to programmatically generate rules and facts and test the performance.