Persistent Workspace Setup

Configure your GPU instance to persist your home directory, packages, and configurations across restarts.

Overview

By default, GPU instances use ephemeral storage - any files, packages, or configurations in your home directory are lost when the instance restarts. This guide shows you how to set up a persistent workspace so your environment survives restarts.

Prerequisites

• A running GPU instance with persistent storage attached
• SSH or terminal access to your instance

Quick Setup

Run this one-time setup script to configure workspace persistence:

#!/bin/bash
# Persistent Workspace Setup Script

# Set your persistent storage mount (adjust to your volume name)
STORAGE_PATH="/mnt/your-volume"

# Create workspace directories on persistent storage
mkdir -p $STORAGE_PATH/workspace/{home,pip,conda,cache}

# Create a setup script that runs on login
cat > $STORAGE_PATH/workspace/init.sh << 'EOF'
#!/bin/bash
# Link home directories to persistent storage
STORAGE_PATH="/mnt/your-volume"

# Create symlinks for common directories
ln -sfn $STORAGE_PATH/workspace/pip ~/.local
ln -sfn $STORAGE_PATH/workspace/cache ~/.cache

# Set environment variables
export PIP_CACHE_DIR="$STORAGE_PATH/workspace/pip"
export HF_HOME="$STORAGE_PATH/workspace/cache/huggingface"
export TRANSFORMERS_CACHE="$HF_HOME"

# Source any custom bashrc
if [ -f "$STORAGE_PATH/workspace/home/.bashrc_custom" ]; then
  source $STORAGE_PATH/workspace/home/.bashrc_custom
fi

echo "Workspace loaded from persistent storage"
EOF

chmod +x $STORAGE_PATH/workspace/init.sh

# Add to bashrc for automatic loading
echo 'source /mnt/your-volume/workspace/init.sh' >> ~/.bashrc

echo "Setup complete! Your workspace will now persist across restarts."

What Gets Persisted

Recommended Workflow

Store Projects on Persistent Storage

# Clone repos directly to persistent storage
cd /mnt/your-volume/workspace
git clone https://github.com/your/project.git

# Create a symlink in home for convenience
ln -s /mnt/your-volume/workspace/project ~/project

Install Packages with Persistence

# Install packages (will use persisted pip cache)
pip install torch transformers

# Or use a requirements file from persistent storage
pip install -r /mnt/your-volume/workspace/project/requirements.txt

Store Environment Variables

# Add custom environment variables
cat >> /mnt/your-volume/workspace/home/.bashrc_custom << 'EOF'
export MY_API_KEY="your-key-here"
export PROJECT_ROOT="/mnt/your-volume/workspace/project"
alias project="cd $PROJECT_ROOT"
EOF

After Restart

When your instance restarts:

1. Persistent storage is automatically remounted at /mnt/your-volume
2. Your .bashrc sources the init script (you need to re-run the setup once on fresh instances)
3. Symlinks restore access to your pip packages and caches
4. Your projects and data are immediately available

Pro Tips

Use a Setup Script Repository

# Store your setup script on GitHub
# Then on any new instance:
curl -sSL https://raw.githubusercontent.com/you/dotfiles/main/setup.sh | bash

Docker/Container Workflows

If you're using Docker:

# Mount persistent storage into containers
docker run -v /mnt/your-volume:/data -it your-image

Conda Environments

# Store conda environments on persistent storage
mkdir -p /mnt/your-volume/workspace/conda/envs
conda config --prepend envs_dirs /mnt/your-volume/workspace/conda/envs

# Create environment (persisted automatically)
conda create -n myenv python=3.10

Troubleshooting

Packages not found after restart

Ensure your init script is being sourced:

# Check if init script exists
cat /mnt/your-volume/workspace/init.sh

# Manually source it
source /mnt/your-volume/workspace/init.sh

# Verify symlinks
ls -la ~/.local

Permission errors

# Fix ownership
sudo chown -R $USER:$USER /mnt/your-volume/workspace

Need Help?

Contact us at support@packet.ai