> ## Documentation Index
> Fetch the complete documentation index at: https://r2r-patch-fix-ingestion.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Services

# R2R Service Configuration Troubleshooting Guide: Postgres, Hatchet

This guide addresses common configuration problems for Postgres, and Hatchet services in R2R deployments.

## Postgres Configuration Issues

### 1. Connection Failures

**Symptom:** R2R cannot connect to Postgres database.

**Possible Causes and Solutions:**

a) Incorrect connection string:

* Verify the `DATABASE_URL` environment variable.
* Ensure it follows the format: `postgres://user:password@host:port/dbname`

b) Network issues:

* Check if Postgres is running and accessible from the R2R container.
* Verify network settings in Docker Compose file.

c) Authentication problems:

* Confirm that the username and password in the connection string are correct.
* Check Postgres logs for failed authentication attempts.

### 2. pgvector Extension Missing

**Symptom:** Vector operations fail or R2R reports missing pgvector functionality.

**Solution:**

* Ensure you're using the `pgvector/pgvector` image instead of the standard Postgres image.
* Verify that the pgvector extension is created in your database:
  ```sql
  CREATE EXTENSION IF NOT EXISTS vector;
  ```

### 3. Insufficient Resources

**Symptom:** Postgres performance issues or crashes under load.

**Solution:**

* Adjust resource limits in Docker Compose:
  ```yaml
  postgres:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 4G
  ```
* Tune Postgres configuration parameters like `shared_buffers`, `effective_cache_size`, etc.

## Hatchet Configuration Issues

### 1. RabbitMQ Connection Problems

**Symptom:** Hatchet cannot connect to RabbitMQ.

**Solution:**

* Verify RabbitMQ connection string:
  ```yaml
  environment:
    - SERVER_TASKQUEUE_RABBITMQ_URL=amqp://user:password@hatchet-rabbitmq:5672/
  ```
* Ensure RabbitMQ service is healthy and accessible.

### 2. Hatchet API Key Issues

**Symptom:** R2R cannot authenticate with Hatchet.

**Solution:**

* Check the Hatchet API key generation process:
  ```yaml
  setup-token:
    command: >
      sh -c "
        TOKEN=$(/hatchet/hatchet-admin token create --config /hatchet/config --tenant-id your-tenant-id)
        echo $TOKEN > /hatchet_api_key/api_key.txt
      "
  ```
* Verify that R2R is correctly reading the API key:
  ```yaml
  r2r:
    environment:
      - HATCHET_CLIENT_TOKEN=${HATCHET_CLIENT_TOKEN}
    command: >
      sh -c '
        export HATCHET_CLIENT_TOKEN=$(cat /hatchet_api_key/api_key.txt)
        exec your_r2r_command
      '
  ```

### 3. Hatchet Engine Health Check Failures

**Symptom:** Hatchet Engine service fails health checks.

**Solution:**

* Verify Hatchet Engine configuration:
  ```yaml
  hatchet-engine:
    environment:
      - SERVER_GRPC_BROADCAST_ADDRESS=host.docker.internal:7077
      - SERVER_GRPC_BIND_ADDRESS=0.0.0.0
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:7077/health"]
      interval: 10s
      timeout: 5s
      retries: 5
  ```
* Check Hatchet Engine logs for startup errors.
* Ensure all required environment variables are set correctly.

## General Troubleshooting Tips

1. **Check Logs:** Always start by examining the logs of the problematic service:
   ```
   docker-compose logs postgres
   docker-compose logs hatchet-engine
   ```

2. **Verify Network Connectivity:** Ensure services can communicate:
   ```
   docker-compose exec r2r ping postgres
   docker-compose exec r2r ping hatchet-engine
   ```

3. **Check Resource Usage:** Monitor CPU and memory usage:
   ```
   docker stats
   ```

4. **Recreate Containers:** Sometimes, recreating containers can resolve issues:
   ```
   docker-compose up -d --force-recreate <service-name>
   ```

5. **Verify Volumes:** Ensure data persistence by checking volume mounts:
   ```
   docker volume ls
   docker volume inspect <volume-name>
   ```

6. **Environment Variables:** Double-check all environment variables in your `.env` file and `docker-compose.yml`.

By following this guide, you should be able to diagnose and resolve most configuration issues related to Postgres, and Hatchet in your R2R deployment. If problems persist, consider seeking help from the R2R community or support channels.