I am going non-linear on crawling sites and documents to help me learn and code. I am going to submit my tool stack to the hackathon, and realized all the info and guides on preparing the submittal (i.e. api requirements, etc) is in the automator website. What I am now looking for is either a sitemap or git repo (Thanks Cole for showing us how to crawl both of those) to cleanly pull in that info for my coding assistant.
1 Like
Haha I love this! The Live Agent Studio doesn’t have a sitemap currently, but you can still use Crawl4AI to visit studio.ottomator.ai, grab all links from the home page, and then recursively scrape those.
Oddly enough it appears crawl4ai does not have one either, but I am checking. Really pushing here to submit my tool by the deadline. Beware scope creep!
Thanks!
1 Like
Their documentation page does have a sitemap!