> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/joey727/Phisherman/llms.txt
> Use this file to discover all available pages before exploring further.

# Redis setup

> How to provision an Upstash Redis instance for Phisherman.

Phisherman requires Redis for several critical subsystems. Without a working Redis connection, the server will not be able to serve feed data, cache scan results, or enforce rate limits.

## Why Redis is required

Redis is used for:

* **Feed data storage** — URLHaus, OpenPhish, PhishTank, and PhishStats feeds are loaded into Redis Sets on startup and refreshed on a schedule.
* **Scan result cache** — Completed scan results are stored in a Redis Hash (keyed by SHA-256 of the URL) with a 5-minute TTL to avoid re-running all checkers for repeat requests.
* **WHOIS cache** — WHOIS lookup results are cached in a Redis Hash for 24 hours to avoid repeated external lookups.
* **DNS cache** — DNS resolution results are cached in a Redis Hash for 1 hour.
* **Google Safe Browsing cache** — API responses are cached in a Redis Hash for 1 hour (15 minutes for error responses).
* **Rate limiting** — Each client IP has a Redis String key that tracks request count within a 15-minute window.

## Provisioning Upstash Redis

Phisherman uses the [`@upstash/redis`](https://github.com/upstash/upstash-redis) client, which communicates over HTTP REST — no persistent TCP connection is needed.

<Steps>
  <Step title="Create an Upstash account">
    Go to [upstash.com](https://upstash.com) and sign up for a free account.
  </Step>

  <Step title="Create a Redis database">
    In the Upstash console, click **Create Database**. Choose a region close to where you will deploy Phisherman to minimize round-trip latency. Select **Regional** (not Global) unless you are running a multi-region deployment.
  </Step>

  <Step title="Copy your credentials">
    After the database is created, open the **REST API** tab in the Upstash dashboard. Copy the **REST URL** and **REST Token** values shown there.
  </Step>

  <Step title="Set environment variables">
    Add the credentials to your `.env` file:

    ```bash theme={null}
    UPSTASH_REDIS_REST_URL=https://<your-instance>.upstash.io
    UPSTASH_REDIS_REST_TOKEN=<your-token>
    ```

    Phisherman initialises the Redis client in `src/utils/redis.ts` using these two variables:

    ```typescript theme={null}
    const redis = new Redis({
      url: process.env.UPSTASH_REDIS_REST_URL!,
      token: process.env.UPSTASH_REDIS_REST_TOKEN!,
    });
    ```
  </Step>
</Steps>

## Redis key inventory

The table below lists every key Phisherman writes to Redis.

| Key                   | Type   | Description                                                  | Refresh / TTL                               |
| --------------------- | ------ | ------------------------------------------------------------ | ------------------------------------------- |
| `urlhaus_blacklist`   | Set    | URLHaus active malware URLs                                  | Refreshed every 5 min                       |
| `openphish_urls`      | Set    | OpenPhish full URLs                                          | Refreshed every 15 min                      |
| `openphish_hosts`     | Set    | OpenPhish hostnames                                          | Refreshed every 15 min                      |
| `phishtank_urls`      | Set    | PhishTank verified phishing URLs                             | Refreshed every 1 hr                        |
| `phishstats_urls`     | Set    | PhishStats URLs (last 20,000 entries)                        | Refreshed every 90 min                      |
| `phishstats_hosts`    | Set    | PhishStats hostnames                                         | Refreshed every 90 min                      |
| `scan_results`        | Hash   | Cached scan results, SHA-256 keyed                           | 5-minute TTL per entry                      |
| `scan_results_expiry` | ZSET   | Expiry tracking for `scan_results`                           | Cleaned up on each `CacheManager` cycle     |
| `whois_data`          | Hash   | WHOIS lookup results per registered domain                   | 24-hour TTL per entry                       |
| `whois_expiry`        | ZSET   | Expiry tracking for `whois_data`                             | Cleaned up on each `CacheManager` cycle     |
| `gsb_cache_hash`      | Hash   | Google Safe Browsing results per URL                         | 1-hour TTL (15-min TTL for error responses) |
| `gsb_cache_expiry`    | ZSET   | Expiry tracking for `gsb_cache_hash`                         | Cleaned up on each `CacheManager` cycle     |
| `gwr_cache_hash`      | Hash   | Google Web Risk results per URL (checker currently disabled) | 1-hour TTL (15-min TTL for errors)          |
| `gwr_cache_expiry`    | ZSET   | Expiry tracking for `gwr_cache_hash`                         | Cleaned up on each `CacheManager` cycle     |
| `dns_cache_hash`      | Hash   | DNS resolution results per hostname                          | 1-hour TTL per entry                        |
| `dns_cache_expiry`    | ZSET   | Expiry tracking for `dns_cache_hash`                         | Cleaned up on each `CacheManager` cycle     |
| `ratelimit:<ip>`      | String | Request count for a client IP address                        | 15-minute window (900-second expiry)        |

<Note>
  The Upstash free tier provides 10,000 commands per day and 256 MB of storage. This is sufficient for personal projects and low-traffic deployments. For production workloads with high scan volume, monitor your command usage in the Upstash dashboard.
</Note>

<Tip>
  Choose a Redis region that is geographically close to your API server. Every scan involves multiple Redis reads (feed lookups, cache checks, rate limit increment), so even a few hundred milliseconds of cross-region latency will noticeably increase response times.
</Tip>