> ## Documentation Index
> Fetch the complete documentation index at: https://docs.crawlspace.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction

> Crawlspace - The centralized web crawling platform

With Crawlspace, you can:

* Affordably crawl **tens of millions** of web pages per month
* Scrape web pages with **LLM extraction** or query selectors
* Store data in **SQLite**, a **vector database**, and/or **S3-compatible bucket**
* Respectfully adhere to rate-limiting and robots.txt out of the box

## Get up to speed

Set up your local environment, and learn about web crawling philosophies.

<CardGroup cols={2}>
  <Card title="Quickstart" icon="stopwatch" href="/welcome/quickstart">
    Deploy your first web crawler to Crawlspace in under 5 minutes
  </Card>
</CardGroup>

## Build a web crawler

Deploy web crawlers as easily as you deploy websites.

<CardGroup cols={2}>
  <Card title="Overview" icon="globe" href="/build/overview">
    Learn the fundamentals of building a web crawler on Crawlspace
  </Card>

  <Card title="Handle responses" icon="code" href="/build/responses">
    Use LLMs and query selectors to parse page responses
  </Card>

  <Card title="Store data" icon="database" href="/build/storage">
    Save data in a SQLite database, a vector database, or a bucket
  </Card>

  <Card title="Configure your options" icon="gear" href="/build/configuration">
    Tweak your crawler with configuration settings
  </Card>
</CardGroup>

## Learn by example

Read some sample code to learn as you go.

<CardGroup cols={2}>
  <Card title="Sample code" icon="graduation-cap" href="/learn/examples">
    Spark inspiration for your next web crawling project
  </Card>
</CardGroup>