Version: v1.0.33

🚀 Spot Deployer

Version: v1.0.35

Deploy AWS Spot Instances with Bacalhau compute nodes in seconds

🎯 Quick Deploy: Get started in 30 seconds

curl -sSL https://tada.wang/install.sh | bash -s -- create

📋 Table of Contents

Quick Start
Installation Guide
Commands Reference
Configuration
AWS Authentication
Bacalhau Integration
Architecture
Troubleshooting
Advanced Usage

⚡ Quick Start

Deploy AWS Spot instances running Bacalhau compute nodes with these simple commands:

1Setup

curl -sSL https://tada.wang/install.sh | bash -s -- setup

Creates configuration files

2Deploy

curl -sSL https://tada.wang/install.sh | bash -s -- create

Launches spot instances

✨ No Docker Required! The deployment tool runs directly via uvx (Python package runner) - no Docker installation needed on your machine!

📦 Installation Guide

Prerequisites

Python 3.8+ (uv/uvx installed automatically if needed)
AWS Account with EC2, VPC, and Security Group permissions
AWS Credentials configured (see Authentication section)
SSH Key Pair for instance access

Step-by-Step Installation

1Initial Setup

curl -sSL https://tada.wang/install.sh | bash -s -- setup

This creates a config.yaml file in your current directory with sensible defaults.

2Configure Your Settings

nano config.yaml

Edit the configuration file with your SSH key paths and preferences.

3Deploy Instances

curl -sSL https://tada.wang/install.sh | bash -s -- create

Launches your spot instances across multiple regions.

🔧 Commands Reference

Core Commands

setup - Initial configuration setup
create - Create and deploy spot instances
list - List running instances with status
destroy - Destroy all spot instances
help - Show detailed help message

Command Examples

# List all running instances
curl -sSL https://tada.wang/install.sh | bash -s -- list

# Destroy all instances
curl -sSL https://tada.wang/install.sh | bash -s -- destroy

# Get help
curl -sSL https://tada.wang/install.sh | bash -s -- help

⚙️ Configuration

The config.yaml file controls all aspects of your deployment:

Sample Configuration

aws:
  total_instances: 6
  username: ubuntu
  public_ssh_key_path: ~/.ssh/id_ed25519.pub
  private_ssh_key_path: ~/.ssh/id_ed25519
  instance_storage_gb: 20
  tags:
    Project: MyProject
    Environment: Development

regions:
  - us-west-2:
      machine_type: t3.medium
      image: auto
  - us-east-1:
      machine_type: t3.medium
      image: auto

Configuration Options

Multiple AWS regions - Deploy across regions for redundancy
Custom instance types - Choose optimal instance sizes
Dedicated VPCs - Isolated networking per deployment
Resource tagging - Organize and track costs
SSH key management - Secure instance access
Storage configuration - Customize disk space

🔐 AWS Authentication

The tool automatically detects your AWS credentials in this order:

Detection Methods

Environment Variables
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
EC2 Instance Role
When running on EC2
AWS SSO
Active SSO sessions
AWS Config Files
~/.aws/credentials, ~/.aws/config

Setup Examples

# AWS SSO (Recommended)
aws sso login

# Environment Variables
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret

# AWS CLI
aws configure

            💡 Pro Tip: The installer shows exactly which credentials it's using before deployment!
        

🌊 Bacalhau Integration

Spot Deployer automatically configures Bacalhau compute nodes on each instance. To connect to your orchestrator:

Orchestrator Credentials

Create these files in your project directory before deployment:

mkdir -p files
echo "nats://your-orchestrator.example.com:4222" > files/orchestrator_endpoint
echo "your-secret-token-here" > files/orchestrator_token

What Gets Deployed

Bacalhau compute nodes with Docker-in-Docker support
Sensor data generators for testing workloads
Automated node registration with your orchestrator
Health monitoring and automatic restarts

After Deployment

# Check your nodes
bacalhau node list

# Test with a job
bacalhau docker run ubuntu echo "Hello from spot instance!"

🏗️ Architecture

On Your Machine

✅ No Docker required - runs via uvx
🔄 Automatic dependency management
🚀 Direct execution from GitHub
🔒 Uses your AWS credentials securely
📊 Beautiful Rich terminal UI

On EC2 Instances

🐳 Bacalhau: Docker container with DIND support
📈 Sensor Generator: Test data generation
⚙️ Docker: Automatically installed via cloud-init
🔧 SystemD services: Automatic startup and monitoring
📝 Deterministic node identity: US city-based naming

🔍 Troubleshooting

Common Issues

❌ "No AWS credentials found"

Solution: Configure AWS authentication first

# Try AWS SSO
aws sso login

# Or set environment variables
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret

# Then retry
curl -sSL https://tada.wang/install.sh | bash -s -- create

❌ "SSH key not found"

Solution: Create SSH keys or update config paths

# Generate new SSH keys
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -N ''

# Or update config.yaml with correct paths
nano config.yaml

❌ "InsufficientInstanceCapacity"

Solution: Spot instances unavailable, try different regions/types

# Edit config.yaml to add more regions or change instance types
regions:
  - us-west-2:
      machine_type: t3.small  # Try smaller instances
  - us-east-1:
      machine_type: t3.small
  - eu-west-1:
      machine_type: t3.small

🚀 Advanced Usage

Direct uvx Usage

Skip the installer and run directly:

# Run directly from GitHub
uvx --from git+https://github.com/bacalhau-project/aws-spot-deployer spot-deployer create

# Install globally
uvx install --from git+https://github.com/bacalhau-project/aws-spot-deployer spot-deployer
spot-deployer list

Local Development

# Clone and use locally
git clone https://github.com/bacalhau-project/aws-spot-deployer.git
cd aws-spot-deployer

# Use local wrapper (detects AWS credentials automatically)
./spot create

# Or run directly with uv
uv run python -m spot_deployer create

Version Pinning

# Use specific version
curl -sSL https://tada.wang/install.sh | bash -s -- create --version v1.2.0

# Use latest development version
curl -sSL https://tada.wang/install.sh | bash -s -- create --version main

Dry Run Mode

# See what would be deployed without actually doing it
curl -sSL https://tada.wang/install.sh | bash -s -- create --dry-run

⚠️ Security Note: Never commit AWS credentials, private SSH keys, or Bacalhau tokens to version control!

📚 Additional Resources

Related Projects

🌊 Bacalhau - Compute over data
⚡ uv - Python package manager
☁️ AWS EC2 Spot - Cost-effective compute