Your First Crawl

Let’s crawl your first webpage! Start by opening up a webpage that you'd like to crawl, and note the URL for later.

Logging in

To start crawling with hosted Browsertrix, you'll need a Browsertrix account. Sign up for an account and log in.

Self-hosting

If you'd like to try Browsertrix before signing up, or you have specialized hosting requirements, you can host Browsertrix yourself. Set up Browsertrix on your system and log in as your admin user.

Starting the crawl

Once you've logged in you should see your org overview. If you land somewhere else, navigate to Overview.

1. Tap the Create New... shortcut and select Crawl Workflow.
2. Choose Page List. We'll get into the details of the options later, but this is a good starting point for a simple crawl.
3. Enter the URL of the webpage that you noted earlier in Page URL(s).
4. Tap Review & Save.
5. Tap Save Workflow.
6. You should now see your new crawl workflow. Give the crawler a few moments to warm up, and then watch as it archives the webpage!

---

Next steps

After running your first crawl, check out the following to learn more about Browsertrix's features:

• A detailed list of crawl workflow setup options.
• Adding exclusions to limit your crawl's scope and evading crawler traps by editing exclusion rules while crawling.
• Best practices for crawling with browser profiles to capture content only available when logged in to a website.
• Managing archived items, including uploading previously archived content.
• Organizing and combining archived items with collections for sharing and export.
• Invite collaborators to your org.