@justhodl
They are probably blocking the user agent or serving it a limited version.
The simplest way: ask it to write a script to fetch web content using a custom google bot user agent and remove javascript. This will save content to a html or text file that you can link to or upload directly to the ai.