...
This commit is contained in:
parent
41b445fdba
commit
2eb6f45631
159
tools/readme.md
159
tools/readme.md
@ -0,0 +1,159 @@
|
||||
# ITenv Tools - Tool Documentation
|
||||
|
||||
This directory contains various utility scripts for server management, disk operations, and git repository management. Each tool is designed for specific infrastructure tasks.
|
||||
|
||||
## Tools Overview
|
||||
|
||||
### 1. erase.sh
|
||||
**Purpose**: Secure disk erasure utility for SSD drives
|
||||
|
||||
**What it does**:
|
||||
- Detects all SSD drives on the system (excludes traditional HDDs)
|
||||
- Provides interactive menu to erase specific disks or all detected SSDs
|
||||
- Handles RAID array detection and cleanup before erasure
|
||||
- Removes RAID superblocks from partitions
|
||||
- Securely wipes the first 1GB and last 1MB of selected disks
|
||||
- Destroys partition tables and filesystem headers
|
||||
|
||||
**Safety Features**:
|
||||
- Multiple confirmation prompts before destructive operations
|
||||
- Root privilege verification
|
||||
- Automatic unmounting of active filesystems
|
||||
- RAID array stopping before disk erasure
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
sudo ./erase.sh
|
||||
```
|
||||
|
||||
**⚠️ WARNING**: This tool permanently destroys all data on selected disks. Use with extreme caution!
|
||||
|
||||
### 2. git_checkout.sh
|
||||
**Purpose**: Git repository management for itenv projects
|
||||
|
||||
**What it does**:
|
||||
- Verifies SSH agent has loaded keys for git authentication
|
||||
- Checks and configures git user.name and user.email if not set
|
||||
- Creates the standard directory structure: `/root/code/git.threefold.info/ourworld_web`
|
||||
- Clones or updates two repositories:
|
||||
- `itenv_web2` - Main web application repository
|
||||
- `itenv_tools` - Tools and utilities repository
|
||||
|
||||
**Prerequisites**:
|
||||
- SSH key must be loaded in ssh-agent
|
||||
- Git must be installed
|
||||
- Access to git.threefold.info repositories
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
./git_checkout.sh
|
||||
```
|
||||
|
||||
### 3. git_push.sh
|
||||
**Purpose**: Quick git commit and push utility
|
||||
|
||||
**What it does**:
|
||||
- Navigates to the parent directory of the tools folder
|
||||
- Stages all changes (`git add . -A`)
|
||||
- Creates a commit with message "init"
|
||||
- Pushes changes to the remote repository
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
./git_push.sh
|
||||
```
|
||||
|
||||
**Note**: This is a simple automation script for quick commits. For production use, consider more descriptive commit messages.
|
||||
|
||||
### 4. ubuntu_install.sh
|
||||
**Purpose**: Automated Ubuntu 24.04 installation for Hetzner dedicated servers
|
||||
|
||||
**What it does**:
|
||||
- Detects available NVMe drives (requires minimum 2 drives)
|
||||
- Creates Hetzner installimage configuration for Ubuntu 24.04
|
||||
- Sets up btrfs filesystem with RAID 1 across two drives
|
||||
- Configures btrfs subvolumes for better organization:
|
||||
- `@` (root filesystem)
|
||||
- `@home` (/home)
|
||||
- `@var` (/var)
|
||||
- `@var/log` (/var/log)
|
||||
- `@tmp` (/tmp)
|
||||
- `@opt` (/opt)
|
||||
- `@srv` (/srv)
|
||||
- `@snapshots` (/.snapshots)
|
||||
- Creates post-installation scripts for:
|
||||
- btrfs optimization and maintenance
|
||||
- Automatic snapshot management with snapper
|
||||
- RAID monitoring utilities
|
||||
|
||||
**Features**:
|
||||
- RAID 1 for data redundancy
|
||||
- btrfs compression (zstd:3) for space efficiency
|
||||
- Automated weekly balance and scrub operations
|
||||
- Snapshot management with configurable retention
|
||||
|
||||
**Prerequisites**:
|
||||
- Must be run from Hetzner rescue system
|
||||
- Requires at least 2 NVMe drives
|
||||
- installimage must be available
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
./ubuntu_install.sh
|
||||
```
|
||||
|
||||
**⚠️ WARNING**: This script will completely wipe the selected drives and install a fresh Ubuntu system.
|
||||
|
||||
### 5. example_autoconfig/autoconfig
|
||||
**Purpose**: Example configuration file for Hetzner installimage
|
||||
|
||||
**What it contains**:
|
||||
- Sample configuration for Ubuntu 24.04 installation
|
||||
- RAID 1 setup with two Samsung NVMe drives
|
||||
- Standard partition layout:
|
||||
- 4GB swap partition
|
||||
- 1024MB /boot partition (ext3)
|
||||
- Remaining space for root filesystem (ext4)
|
||||
- Network and hostname configuration examples
|
||||
- Comments explaining all configuration options
|
||||
|
||||
**Usage**:
|
||||
This file serves as a template and reference for creating custom installimage configurations. Copy and modify as needed for specific server setups.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
itenv_tools/tools/
|
||||
├── readme.md # This documentation file
|
||||
├── erase.sh # Disk erasure utility
|
||||
├── git_checkout.sh # Git repository management
|
||||
├── git_push.sh # Quick git commit/push
|
||||
├── ubuntu_install.sh # Ubuntu installation script
|
||||
└── example_autoconfig/ # Example configurations
|
||||
└── autoconfig # Sample installimage config
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **erase.sh**: Requires root privileges and permanently destroys data
|
||||
- **git_checkout.sh**: Requires SSH key access to private repositories
|
||||
- **ubuntu_install.sh**: Must be run in Hetzner rescue environment
|
||||
- All scripts include error handling and safety checks where appropriate
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### System Requirements
|
||||
- Linux environment (tested on Ubuntu/Debian)
|
||||
- Bash shell
|
||||
- Root access for disk operations
|
||||
- Git for repository management
|
||||
- SSH access for private repositories
|
||||
|
||||
### For Hetzner Operations
|
||||
- Hetzner rescue system access
|
||||
- installimage utility
|
||||
- Minimum 2 NVMe drives for RAID setup
|
||||
|
||||
## Support
|
||||
|
||||
These tools are designed for infrastructure automation and server management. Ensure you understand the implications of each script before execution, especially those involving disk operations or system installation.
|
||||
@ -147,109 +147,108 @@ log "Created installimage configuration at /autosetup"
|
||||
# Create autosetup directory for additional scripts
|
||||
mkdir -p /autosetup_scripts
|
||||
|
||||
# Create post-installation script for btrfs optimization
|
||||
cat > /autosetup_scripts/post_install.sh << 'EOF'
|
||||
#!/bin/bash
|
||||
# # Create post-installation script for btrfs optimization
|
||||
# cat > /autosetup_scripts/post_install.sh << 'EOF'
|
||||
# #!/bin/bash
|
||||
|
||||
# Post-installation script for btrfs optimization
|
||||
# # Post-installation script for btrfs optimization
|
||||
|
||||
log() {
|
||||
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
|
||||
}
|
||||
# log() {
|
||||
# echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
|
||||
# }
|
||||
|
||||
log "Starting post-installation btrfs optimization..."
|
||||
# log "Starting post-installation btrfs optimization..."
|
||||
|
||||
# Install btrfs-progs if not already installed
|
||||
if ! command -v btrfs &> /dev/null; then
|
||||
log "Installing btrfs-progs..."
|
||||
apt-get update
|
||||
apt-get install -y btrfs-progs
|
||||
fi
|
||||
# # Install btrfs-progs if not already installed
|
||||
# if ! command -v btrfs &> /dev/null; then
|
||||
# log "Installing btrfs-progs..."
|
||||
# apt-get update
|
||||
# apt-get install -y btrfs-progs
|
||||
# fi
|
||||
|
||||
# Set btrfs mount options for better performance and features
|
||||
log "Configuring btrfs mount options..."
|
||||
# # Set btrfs mount options for better performance and features
|
||||
# log "Configuring btrfs mount options..."
|
||||
|
||||
# Backup original fstab
|
||||
cp /etc/fstab /etc/fstab.backup
|
||||
# # Backup original fstab
|
||||
# cp /etc/fstab /etc/fstab.backup
|
||||
|
||||
# Update fstab with optimized btrfs mount options
|
||||
sed -i 's/btrfs\s\+defaults/btrfs defaults,noatime,compress=zstd:3,space_cache=v2,autodefrag/' /etc/fstab
|
||||
# # Update fstab with optimized btrfs mount options
|
||||
# sed -i 's/btrfs\s\+defaults/btrfs defaults,noatime,compress=zstd:3,space_cache=v2,autodefrag/' /etc/fstab
|
||||
|
||||
# Create btrfs maintenance scripts
|
||||
mkdir -p /etc/cron.weekly
|
||||
# # Create btrfs maintenance scripts
|
||||
# mkdir -p /etc/cron.weekly
|
||||
|
||||
# Weekly balance script
|
||||
cat > /etc/cron.weekly/btrfs-balance << 'BALANCE_EOF'
|
||||
#!/bin/bash
|
||||
# Weekly btrfs balance to optimize space usage
|
||||
/usr/bin/btrfs balance start -dusage=50 -musage=50 / 2>/dev/null || true
|
||||
BALANCE_EOF
|
||||
# # Weekly balance script
|
||||
# cat > /etc/cron.weekly/btrfs-balance << 'BALANCE_EOF'
|
||||
# #!/bin/bash
|
||||
# # Weekly btrfs balance to optimize space usage
|
||||
# /usr/bin/btrfs balance start -dusage=50 -musage=50 / 2>/dev/null || true
|
||||
# BALANCE_EOF
|
||||
|
||||
chmod +x /etc/cron.weekly/btrfs-balance
|
||||
# chmod +x /etc/cron.weekly/btrfs-balance
|
||||
|
||||
# Weekly scrub script for data integrity
|
||||
cat > /etc/cron.weekly/btrfs-scrub << 'SCRUB_EOF'
|
||||
#!/bin/bash
|
||||
# Weekly btrfs scrub for data integrity check
|
||||
/usr/bin/btrfs scrub start / 2>/dev/null || true
|
||||
SCRUB_EOF
|
||||
# # Weekly scrub script for data integrity
|
||||
# cat > /etc/cron.weekly/btrfs-scrub << 'SCRUB_EOF'
|
||||
# #!/bin/bash
|
||||
# # Weekly btrfs scrub for data integrity check
|
||||
# /usr/bin/btrfs scrub start / 2>/dev/null || true
|
||||
# SCRUB_EOF
|
||||
|
||||
chmod +x /etc/cron.weekly/btrfs-scrub
|
||||
# chmod +x /etc/cron.weekly/btrfs-scrub
|
||||
|
||||
# Install and configure snapper for automatic snapshots
|
||||
log "Installing and configuring snapper for automatic snapshots..."
|
||||
apt-get install -y snapper
|
||||
# # Install and configure snapper for automatic snapshots
|
||||
# log "Installing and configuring snapper for automatic snapshots..."
|
||||
# apt-get install -y snapper
|
||||
|
||||
# Create snapper config for root
|
||||
snapper -c root create-config /
|
||||
# # Create snapper config for root
|
||||
# snapper -c root create-config /
|
||||
|
||||
# Configure snapper for reasonable snapshot retention
|
||||
snapper -c root set-config TIMELINE_CREATE=yes
|
||||
snapper -c root set-config TIMELINE_CLEANUP=yes
|
||||
snapper -c root set-config NUMBER_CLEANUP=yes
|
||||
snapper -c root set-config NUMBER_MIN_AGE=1800
|
||||
snapper -c root set-config NUMBER_LIMIT=50
|
||||
snapper -c root set-config NUMBER_LIMIT_IMPORTANT=10
|
||||
# # Configure snapper for reasonable snapshot retention
|
||||
# snapper -c root set-config TIMELINE_CREATE=yes
|
||||
# snapper -c root set-config TIMELINE_CLEANUP=yes
|
||||
# snapper -c root set-config NUMBER_CLEANUP=yes
|
||||
# snapper -c root set-config NUMBER_MIN_AGE=1800
|
||||
# snapper -c root set-config NUMBER_LIMIT=50
|
||||
# snapper -c root set-config NUMBER_LIMIT_IMPORTANT=10
|
||||
|
||||
# Enable snapper timer
|
||||
systemctl enable snapper-timeline.timer
|
||||
systemctl enable snapper-cleanup.timer
|
||||
# # Enable snapper timer
|
||||
# systemctl enable snapper-timeline.timer
|
||||
# systemctl enable snapper-cleanup.timer
|
||||
|
||||
log "Post-installation btrfs optimization completed"
|
||||
EOF
|
||||
# log "Post-installation btrfs optimization completed"
|
||||
# EOF
|
||||
|
||||
chmod +x /autosetup_scripts/post_install.sh
|
||||
# chmod +x /autosetup_scripts/post_install.sh
|
||||
|
||||
log "Created post-installation script at /autosetup_scripts/post_install.sh"
|
||||
# log "Created post-installation script at /autosetup_scripts/post_install.sh"
|
||||
|
||||
# Create a script to monitor RAID status
|
||||
cat > /autosetup_scripts/raid_monitor.sh << 'EOF'
|
||||
#!/bin/bash
|
||||
# # Create a script to monitor RAID status
|
||||
# cat > /autosetup_scripts/raid_monitor.sh << 'EOF'
|
||||
# #!/bin/bash
|
||||
|
||||
# RAID monitoring script for btrfs
|
||||
# # RAID monitoring script for btrfs
|
||||
|
||||
check_btrfs_raid() {
|
||||
echo "=== Btrfs RAID Status ==="
|
||||
btrfs filesystem show
|
||||
echo
|
||||
echo "=== Btrfs Device Stats ==="
|
||||
btrfs device stats /
|
||||
echo
|
||||
echo "=== Btrfs Filesystem Usage ==="
|
||||
btrfs filesystem usage /
|
||||
}
|
||||
# check_btrfs_raid() {
|
||||
# echo "=== Btrfs RAID Status ==="
|
||||
# btrfs filesystem show
|
||||
# echo
|
||||
# echo "=== Btrfs Device Stats ==="
|
||||
# btrfs device stats /
|
||||
# echo
|
||||
# echo "=== Btrfs Filesystem Usage ==="
|
||||
# btrfs filesystem usage /
|
||||
# }
|
||||
|
||||
# Check if btrfs tools are available
|
||||
if command -v btrfs &> /dev/null; then
|
||||
check_btrfs_raid
|
||||
else
|
||||
echo "btrfs-progs not installed. Install with: apt-get install btrfs-progs"
|
||||
fi
|
||||
EOF
|
||||
# # Check if btrfs tools are available
|
||||
# if command -v btrfs &> /dev/null; then
|
||||
# check_btrfs_raid
|
||||
# else
|
||||
# echo "btrfs-progs not installed. Install with: apt-get install btrfs-progs"
|
||||
# fi
|
||||
# EOF
|
||||
|
||||
chmod +x /autosetup_scripts/raid_monitor.sh
|
||||
|
||||
log "Created RAID monitoring script at /autosetup_scripts/raid_monitor.sh"
|
||||
# chmod +x /autosetup_scripts/raid_monitor.sh
|
||||
# log "Created RAID monitoring script at /autosetup_scripts/raid_monitor.sh"
|
||||
|
||||
# Verify configuration
|
||||
log "Verifying configuration..."
|
||||
@ -284,4 +283,4 @@ else
|
||||
fi
|
||||
|
||||
log "Installation process completed."
|
||||
EOF
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user