Search code examples
reactjsaws-lambdasingle-page-applicationamazon-cloudfrontgoogle-crawlers

React JS - Google crawlers not able to crawl the pages


What is the best way I can make a web app built in react JS work for Google Crawlers for SEO purposes. I want the whole page to be able to crawl.

Basically the issue is as it is rendered on the client end, crawlers can't see anything except the root div.

Application is hosted on S3 Bucket and being server using the cloudfront. I have read about Lambda@Edge but not sure if that can work.

Help!


Solution

  • How to solve this depends heavily on your requirements and specifics. If your page does not change often, you could automatically create a static version of it and host it on S3.

    A Lambda@Edge could check the user agent and if it detects a bot, redirect the request to the static version of your page. Other requests are processed as before.

    One option to create such a static version of your page is to use Puppeteer. You can use Puppeteer with AWS Lambda. So you could build a Lambda function that gets an URL and then puts the HTML in the S3 bucket. You then write a Step Function that does that for all your pages, triggered daily through EventBridge etc.

    Another option is to call the Step Function once a new version of the project is deployed etc.

    There are a lot of options how to solve this. But all those options will probably involve some kind of crawler that will "render" the page into a static HTML file.