Search code examples
pythonnltkstanford-nlp

Use lexical PCFG for generating meaningful phrase


How can i make use of lexical PCFG for generating grammar and thus sentense with more sense. I can generate phrases from CFG grammar using NLTK library, but it most phrases doesnt make sense, although its grammatically correct.

 s=("""
    S -> PRP RB VP
    VP -> VBP NP
    NP -> JJ NNS
    RB -> 'forcefully'|'strongly'|'rerely'
    PRP -> 'we'|'you'|'he'
    VBP -> 'actuarize'|'support'|'condemn'
    JJ -> 'black|fair'
    NNS -> 'markets'
    """)
#load into the grammar

grammar=CFG.fromstring(s)
for sentence in generate(grammar,depth=10):
    print(' '.join(sentence))

Solution

  • A syntactic grammar will generate grammatical sentences, but it makes no guarantees that the sentences make sense. Really, there's no way to make sentences that make semantic sense -- this would require the computer to understand the meaning of what it's saying on a deeper level than currently possible. You can try to combine your CFG with an n-gram language model, which should create more locally coherent sentences, but still not necessarily globally coherent.