Import Website (Crawler)

The website crawler on our platform is designed to scan websites and extract content. It works by analyzing a specified URL and collecting relevant information based on defined criteria, such as specific strings or sections of a webpage. The extracted content is then compiled into files, which can be added to your assistant’s knowledge base. This enrichment helps your assistant provide more accurate and intelligent responses. In essence, the more relevant content the crawler gathers, the smarter and more efficient your assistant becomes.

How to use website crawler

To use a website crawler on our platform, follow these steps:

• Open: https://voicevision.chat/knowledge/webupload

• Display Name: Enter a name for your crawler. This name will appear on the dashboard for easy identification.

• Crawling URL: Provide the URL where the crawler will begin its operation.

• URL Match: Specify a string that the crawler should always match while crawling. This ensures relevance in the crawling process.

• Selector: Define a selector to extract content from specific parts of the website. Use the browser's developer tools (F12) to test your query selector, like document.querySelector("[id='root']").

• Number of pages to crawl : Specify the number of pages the crawler should visit. This helps control the scope of the crawling process.