Search code examples
databaseconnectionsaveartificial-intelligenceassociative

Save Informations as a Data Net


My aim is to write an intelligent ChatBot. He should save known informations likely to the human brain.

That is why I am looking for a filetype wich stores data as a net of connected keywords. What filetype or database system could reach this?

Further Informations:

The information input will be wikipedia, google search, and facts teached by a human during a conversation.

I could give specific informations about my requirements and wishes but I don't know if there exists even any approach to this. Maybe there are more useful specifications as my thoughts.

Just one example: the connections should have weights. Requesting an information net should increase the weights of the used connections.

What I expect is that the ChatBot could get real associations (or ideas) using the data net.


Solution

  • As an extension to my above comments:

    A graph is definitely the way you want to go in terms of data representation...it maps perfectly to your problem description.

    What you seem to be asking is how you can [persistently] store this information on disk (rather than memory). That completely depends on what constraints you need. There is a "Graph Database" which is more geared to storing graphs than say relational or hierarchical databases, and would be perform far better than say pushing your adjacency matrix or list to a flat file. Here's the wikipedia entry:

    http://en.wikipedia.org/wiki/Graph_database

    Now, there is the issue of what happens when you have so many nodes and edges that you can't load them all into memory at once, and unfortunately if you have nodes that are connected to every other node, that can be a problem (because you won't be able to load the complete/valid graph. I can't answer that right now, but I'm sure there are paradigms to address this problem. I will update my answer after some digging.

    Edit-You'll probably have to consult someone who knows more about graph databases. It's possible that there are ways to load chunks of the graph from the database without loading the whole thing. If that's what your issue is, you may want to reform a question about working with large graphs stored on graph databases and post it again, tagged with graphs,databases,algorithms, stuff like that, and just post it again in a more specific manner.