Search code examples
laravelweb-crawlermarkdowngoogle-crawlers

Will my pages be crawled by google as they are Markdown files asynchronously interpreted and injected into the DOM?

I am planning to start a blog so I created my own laravel website. My posts are markdown files with .md extension. When a user visits a post eg. example.com/how-to-create-a-webiste then my markdown file is fetched and parsed to generate html content and displayed on the view called post.

So actually I do not have any html files except for post.blade.php. So will this affect the crawler from crawling my website as I do not have html pages rather all my pages are markdown files?

Solution

  • The answer is NO, Google or any other SE crawler will read the compiled version of your HTML, not your markdown files.

    Google offers a tool to simulate crawling and even index manually your page, you will need to sign up in Search Console, check it out here.

    You have to be careful with dynamic content tho, if the time it takes to inject it in the DOM is too high the robot may leave before the content shows up.

    There are experiments, to test this theory, one that suits your situations is Asynchronous injection:

    Experiment

    After a time out of 1000 milliseconds, the test writes a string into a DIV element.

    Test content

    For the test to be succesful, the following content should get indexed.

    Asynchronously injected content can be found in Google search, and this is proof: ngwzjcrnub

    Result

    Google definitely indexes this content.

    Experiment screenshot

    Source of the experiment

    Hope this helps you.