Search code examples
google-searchschema.orgmicrodatajson-ldgooglebot

Quora's HTML doesn't show schema.org, but google shows them in the question/answer section, how?

I'm making a scraper to read question / answer data for students that supports RDFa, Json LD, and Microdata, but Quora confuses me. I need to understand how it's read so that I can read it in my HTML question / answer scraper for situations like this.

In a google search, I see a QA block, but if I go to the URL https://www.quora.com/What-happens-when-sodium-chloride-and-water-is-heated-to-dry I don't see any evidence of JSON LD, RDFa or Microdata. How is google reading quora's question / answer information?

Possible reasons I can think of:

  • They only show that data to search engine user-agents. So perhaps I should change the user-agent to a scraper when requesting the page.
  • Google figured it out on its own. This means I need to create some NLP solution to get the information.
  • Key words that identify the page as question / answer.
  • Google does something special for big Q/A sites like quora (but stack overflow has schema.org, so I don't think this is true).

PS: Even google doesn't show support for other formats: https://developers.google.com/search/docs/advanced/structured-data/qapage

enter image description here

Solution

  • It's shown only to search engine user agents, use Googlebot.

    @nikrant25 showed the schema does indeed exist: https://search.google.com/test/rich-results/result/r%2Fq-and-a?id=3aNOu3qg7TnhPNz-_xKuuQ . So I decided to do a scrape with Googlebot as the useragent and the schema showed up.