sslh-multiplex-lab/README.md

# SSLH Multiplex Lab Infrastructure

A cross-platform Go CLI tool for provisioning and managing SSLH multiplex lab environments on Hetzner VPS with Namecheap DNS integration.

## Overview

This tool automates the setup of a lab environment that demonstrates protocol multiplexing using SSLH. It provisions a Hetzner VPS, configures SSLH to multiplex multiple services on port 443, sets up WireGuard VPN, manages DNS records via Namecheap, and provides a Docker-based client environment for testing.

### Disclaimer:
- Tested on macOS Silicon, it should work on any platform supported by the Go compiler but please submit a pull request if you run into any issues.

## Features

- **Automated VPS Provisioning**: Creates Hetzner Cloud servers with cloud-init configuration
- **SSLH Configuration**: Automatically generates SSLH config with SNI-based routing for 20+ services
- **DNS Management**: Creates and manages DNS records via Namecheap API
- **SSH Key Management**: Generates per-deployment SSH key pairs with secure random passphrases
- **WireGuard Integration**: Sets up WireGuard server and generates multi-platform client profiles
- **Service Configuration**: Installs and configures multiple services (SSH, HTTPS, SMB, LDAP, databases, etc.)
- **Client Docker Container**: Provides a restricted network environment (443-only) for testing
- **Comprehensive Cleanup**: Teardown removes all resources including VPS and DNS records

## Installation

### Prerequisites

- Go 1.21 or later
- Docker and Docker Compose (for client container)
- **Hetzner Cloud API token**: A read/write API token from the Hetzner Cloud Console
  - Create one at: https://console.hetzner.cloud/
  - Navigate to: Security → API Tokens → Generate API Token
  - Required permissions: Read & Write (to create, manage, and delete servers)
- **Namecheap API credentials** (API key, username, and whitelisted IP)
  - **Note**: The tool automatically switches your domain to Namecheap DNS servers if needed
  - Your domain must use Namecheap's DNS servers (BasicDNS, PremiumDNS, or FreeDNS) to manage DNS records via API
  - If your domain uses custom nameservers, the tool will automatically switch it to Namecheap DNS during setup

### Build

```bash
go build -o sslh-lab ./cmd/sslh-lab
```

## Configuration

Configuration can be provided via:

1. **CLI flags** (highest priority)
2. **Environment variables** (prefixed with `SSLH_`)
3. **Config file** at `~/.sslh-lab/config.yaml`

### Required Configuration

- `hetzner-key`: Hetzner Cloud API token (read/write permissions required)
  - Get from: https://console.hetzner.cloud/ → Security → API Tokens
  - Must have permissions to create, read, update, and delete servers
- `namecheap-key`: Namecheap API key
  - Get from: Namecheap account → Profile → Tools → Business & Dev Tools → Namecheap API Access
  - Enable API access and generate an API key
- `namecheap-user`: Namecheap API username (your Namecheap account username)
- `domain`: Domain name for DNS records
  - The tool will automatically switch the domain to Namecheap DNS servers if it's using custom nameservers
  - Must be a domain registered with Namecheap and present in your account
- `letsencrypt-email`: Email address for Let's Encrypt certificate registration (optional)
  - If provided, automatically provisions a TLS certificate for the domain and all SNI-required subdomains
  - Uses DNS-01 challenge (requires Namecheap DNS control)
  - Certificate is automatically revoked on teardown (if > 7 days until expiry)
  - Reuses existing valid certificates (> 30 days remaining) to avoid rate limits
  - Follows Let's Encrypt best practices: respects rate limits, avoids unnecessary requests

### Optional Configuration

- `region`: Hetzner region (default: `nbg1`)
- `server-type`: Hetzner server type (default: `cpx22`)
- `letsencrypt-email`: Email address for Let's Encrypt certificate registration
  - Automatically provisions TLS certificates for the domain and SNI-required subdomains
  - Uses DNS-01 challenge via Namecheap API
  - Certificates are stored in the deployment directory
  - Automatically revoked on teardown (if > 7 days until expiry)
  - Reuses existing valid certificates (> 30 days remaining) to respect rate limits
- `additional-services`: Include additional services beyond default (SSH, HTTPS, SMB)

### Example Config File

```yaml
hetzner_key: "your-hetzner-cloud-api-token"  # Read/Write API token from Hetzner Cloud Console
namecheap_key: "your-namecheap-api-key"
namecheap_user: "your-namecheap-username"
domain: "example.com"
region: "nbg1"
letsencrypt_email: "your-email@example.com"  # Optional: enables automatic TLS certificate provisioning
server_type: "cpx22"
```

## Usage

All commands support shorthand aliases for convenience:

- `setup` → `s`
- `status` → `st`
- `teardown` → `td`, `down`
- `client` → `c`, `cli`
- `ssh-info` → `ssh`, `si`
- `verify-passphrase` → `verify`, `vp`

### Setup

Provision and configure the lab environment:

```bash
# Full command
sslh-lab setup \
  --hetzner-key YOUR_HETZNER_CLOUD_API_TOKEN \
  --namecheap-key YOUR_NAMECHEAP_KEY \
  --namecheap-user YOUR_NAMECHEAP_USER \
  --domain example.com

# Shorthand
sslh-lab s \
  --hetzner-key YOUR_HETZNER_CLOUD_API_TOKEN \
  --namecheap-key YOUR_NAMECHEAP_KEY \
  --namecheap-user YOUR_NAMECHEAP_USER \
  --domain example.com
```

**Note**: The Hetzner API token must have read/write permissions to create and manage servers. Generate it from the [Hetzner Cloud Console](https://console.hetzner.cloud/) under Security → API Tokens.

This will:
1. Generate an SSH key pair with a random passphrase
2. Display the passphrase (save it securely - it won't be shown again)
3. Create a Hetzner VPS
4. Configure SSLH and services
5. Create DNS records for all services
6. Wait for DNS propagation
7. Create a low-privileged `testuser` account with password authentication enabled
   - Password is randomly generated and displayed during setup
   - This account is intended for demonstrating outbound connectivity through SSLH

**Note**: If services don't work immediately after setup, you may need to run the fix script on the VPS. See the [Troubleshooting](#troubleshooting) section below.

### Status

Check the current deployment status:

```bash
# Full command
sslh-lab status

# Shorthand
sslh-lab st
```

### SSH Info

Display SSH connection information:

```bash
# Full command
sslh-lab ssh-info

# Shorthand
sslh-lab ssh
# or
sslh-lab si
```

### Client Container

Launch the Docker client container for testing:

```bash
# Full command
sslh-lab client

# Shorthand
sslh-lab c
# or
sslh-lab cli

# Automatically connect to container shell
sslh-lab client --connect
# or
sslh-lab c -C
```

The container has restricted network access (only TCP 443 and UDP 53 outbound) and includes:
- Nginx HTTP server on port 8888 (for lateral movement demonstrations)
- SSH client for connecting through SSLH
- WireGuard tools for VPN connectivity
- SMB client for file sharing
- Network diagnostic tools

The HTTP server serves an admin panel and secrets file, accessible via reverse SSH tunnel for lateral movement demonstrations.

### Teardown

Destroy the lab environment and cleanup resources:

```bash
# Using config file (recommended)
sslh-lab teardown [--remove-keys]
# or shorthand
sslh-lab td [--remove-keys]

# With API keys via flags (if not in config file)
sslh-lab teardown --hetzner-key <key> --namecheap-key <key> --namecheap-user <user> [--remove-keys]
```

**Note:** If you have a config file at `~/.sslh-lab/config.yaml`, you don't need to provide API keys via flags. The command will automatically use values from the config file. API keys are optional - if not provided, only local files will be cleaned up (server and DNS records will remain).

The `--remove-keys` flag will also remove SSH keys from local storage.

### Verify Passphrase

Verify if a passphrase is correct for an SSH private key:

```bash
# Full command
sslh-lab verify-passphrase <key-path> <passphrase>

# Shorthand
sslh-lab verify <key-path> <passphrase>
# or
sslh-lab vp <key-path> <passphrase>
```

## Supported Services

### Default Services (Always Included)

The tool always configures these services:
1. **SSH** (22) - Protocol detection via builtin probe
2. **HTTPS** (443) - TLS routing to nginx on port 8444
3. **SMB** (445) - Regex pattern matching for SMB/CIFS protocol

### Additional Services (Optional)

Use the `--additional-services` flag to include:
4. **LDAP** (389) - Regex pattern matching
5. **LDAPS** (636) - SNI-based TLS routing
6. **RDP** (3389) - Regex pattern matching
7. **MySQL** (3306) - Regex pattern matching
8. **PostgreSQL** (5432) - Regex pattern matching

**Note**: The tool supports many more services (Redis, MongoDB, VNC, FTP, email protocols, etc.) via the `GetStandardServices()` function, but these are not included by default. The default and additional services are selected for common lab scenarios.

## Troubleshooting

If services don't work immediately after setup, or if you encounter issues with SSLH, nginx, or service configuration, you can run diagnostic and fix scripts on the VPS.

### Fix Deployment Issues

If SSLH or nginx aren't working correctly after initial setup, SSH into the VPS and run the fix script:

```bash
# SSH into the VPS (use ssh-info command to get connection details)
ssh -i ~/.sslh-lab/deployments/<deployment-id>/id_ed25519 <user>@<server-ip>

# Copy the fix script from your local project to the VPS
scp -i ~/.sslh-lab/deployments/<deployment-id>/id_ed25519 scripts/fix_deployment.sh <user>@<server-ip>:/tmp/

chmod +x /tmp/fix_deployment.sh
sudo /tmp/fix_deployment.sh
```

The fix script will:
- Ensure demo pages and nginx configuration files exist
- Fix SSLH systemd service configuration
- Restart nginx and SSLH services
- Verify services are listening on correct ports
- Test connectivity

### Diagnostic Scripts

Several diagnostic scripts are available in the `scripts/` directory:

- **`fix_deployment.sh`** - Fixes common deployment issues (nginx, SSLH, certificates)
- **`diagnose_deployment.sh`** - Comprehensive diagnostics for deployment issues
- **`verify_deployment.sh`** - Verifies all services and configurations
- **`diagnose_container.sh`** - Diagnostics for the Docker client container

To use these scripts, copy them to the VPS and run with appropriate permissions:

```bash
# Copy script to VPS
scp -i ~/.sslh-lab/deployments/<deployment-id>/id_ed25519 scripts/diagnose_deployment.sh <user>@<server-ip>:/tmp/

# SSH and run
ssh -i ~/.sslh-lab/deployments/<deployment-id>/id_ed25519 <user>@<server-ip>
chmod +x /tmp/diagnose_deployment.sh
sudo /tmp/diagnose_deployment.sh
```

### Common Issues

1. **SSLH not listening on port 443**
   - Run `fix_deployment.sh` to fix systemd configuration
   - Check SSLH config: `sudo cat /etc/sslh.cfg`
   - Check SSLH status: `sudo systemctl status sslh`

2. **Nginx not serving content**
   - Run `fix_deployment.sh` to recreate nginx configs
   - Verify nginx is listening: `ss -tlnp | grep 8444`
   - Check nginx config: `sudo nginx -t`

3. **DNS not resolving**
   - Wait a few minutes for DNS propagation (can take up to 48 hours)
   - Verify DNS records: `nslookup ssh.yourdomain.com`
   - Check Namecheap DNS settings in your account

4. **Services not accessible through SSLH**
   - Verify SSLH is running: `sudo systemctl status sslh`
   - Check SSLH config matches your services
   - Ensure backend services are listening on localhost

## Security Considerations

- SSH key pairs are generated per deployment with cryptographically secure random passphrases
- SSH private keys are encrypted with passphrases and stored in deployment-specific directories
- Passphrases are displayed only once during setup (never logged or stored)
- All API keys are stored securely (env vars or config file, never in code)
- Services are configured to listen on localhost only (for SSLH forwarding)
- Fail2ban is configured for SSH protection
- Firewall rules restrict services appropriately
- A `testuser` account is created with password authentication enabled for demonstration purposes
  - This account has no sudo privileges and is intended for testing outbound connectivity
  - The password is randomly generated and displayed during setup
  - The account is configured for both SSH and SMB access
  - Use this account to demonstrate connectivity through SSLH (port 443)

## Project Structure

```
sslh-multiplex-lab/
├── cmd/sslh-lab/          # CLI application
├── internal/
│   ├── config/            # Configuration management
│   ├── providers/         # Cloud provider clients (Hetzner, Namecheap)
│   ├── ssh/               # SSH key generation
│   ├── sslh/              # SSLH configuration generator
│   ├── wireguard/         # WireGuard server/client config
│   ├── services/          # Service definitions and installers
│   ├── templates/         # Cloud-init template generator
│   └── docker/            # Docker client container
└── pkg/utils/             # Utility functions (DNS, retry, validation)
```

## License

See LICENSE file for details.

## Contributing

Contributions are welcome! Please ensure all code follows Go best practices and includes appropriate error handling and tests.