Quickstart
Deploy your first web crawler in under 5 minutes
This quickstart assumes that you already have a Crawlspace account.
Sign up if you haven’t already — it’s free to create a new account.
Install the CLI
Install @crawlspace/cli globally to use the CLI from any directory.
Log in to the CLI
Enter the same email address you used when signing up. A one-time passcode will be sent to this address — paste it into the CLI when prompted to verify your identity.
Create a new crawler
The crsp new command generates a template inside the directory of your choosing.
Edit the generated files to customize your crawler’s behavior.
Read Build a web crawler to learn the basic syntax.
Test your crawler
Test out your crawler locally with the crsp dev command.
This emulates your crawler’s behavior locally.
Certain features like JavaScript rendering are unavailable in this mode.
Keep in mind that the speed will be much slower compared to running in production.
Deploy your crawler
That’s it! Note that you must have an active subscription before your crawler can start.
Was this page helpful?