ourworld_web/itenv_tools
20
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

...

Browse Source
This commit is contained in:
despiegk 2025-06-15 17:37:19 +02:00
parent 41b445fdba
commit 2eb6f45631
2 changed files with 238 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

159
tools/readme.md
View File

@ -0,0 +1,159 @@
# ITenv Tools - Tool Documentation
This directory contains various utility scripts for server management, disk operations, and git repository management. Each tool is designed for specific infrastructure tasks.
## Tools Overview
### 1. erase.sh
**Purpose**: Secure disk erasure utility for SSD drives
**What it does**:
- Detects all SSD drives on the system (excludes traditional HDDs)
- Provides interactive menu to erase specific disks or all detected SSDs
- Handles RAID array detection and cleanup before erasure
- Removes RAID superblocks from partitions
- Securely wipes the first 1GB and last 1MB of selected disks
- Destroys partition tables and filesystem headers
**Safety Features**:
- Multiple confirmation prompts before destructive operations
- Root privilege verification
- Automatic unmounting of active filesystems
- RAID array stopping before disk erasure
**Usage**:
```bash
sudo ./erase.sh
```
**⚠️ WARNING**: This tool permanently destroys all data on selected disks. Use with extreme caution!
### 2. git_checkout.sh
**Purpose**: Git repository management for itenv projects
**What it does**:
- Verifies SSH agent has loaded keys for git authentication
- Checks and configures git user.name and user.email if not set
- Creates the standard directory structure: `/root/code/git.threefold.info/ourworld_web`
- Clones or updates two repositories:
- `itenv_web2` - Main web application repository
- `itenv_tools` - Tools and utilities repository
**Prerequisites**:
- SSH key must be loaded in ssh-agent
- Git must be installed
- Access to git.threefold.info repositories
**Usage**:
```bash
./git_checkout.sh
```
### 3. git_push.sh
**Purpose**: Quick git commit and push utility
**What it does**:
- Navigates to the parent directory of the tools folder
- Stages all changes (`git add . -A`)
- Creates a commit with message "init"
- Pushes changes to the remote repository
**Usage**:
```bash
./git_push.sh
```
**Note**: This is a simple automation script for quick commits. For production use, consider more descriptive commit messages.
### 4. ubuntu_install.sh
**Purpose**: Automated Ubuntu 24.04 installation for Hetzner dedicated servers
**What it does**:
- Detects available NVMe drives (requires minimum 2 drives)
- Creates Hetzner installimage configuration for Ubuntu 24.04
- Sets up btrfs filesystem with RAID 1 across two drives
- Configures btrfs subvolumes for better organization:
- `@` (root filesystem)
- `@home` (/home)
- `@var` (/var)
- `@var/log` (/var/log)
- `@tmp` (/tmp)
- `@opt` (/opt)
- `@srv` (/srv)
- `@snapshots` (/.snapshots)
- Creates post-installation scripts for:
- btrfs optimization and maintenance
- Automatic snapshot management with snapper
- RAID monitoring utilities
**Features**:
- RAID 1 for data redundancy
- btrfs compression (zstd:3) for space efficiency
- Automated weekly balance and scrub operations
- Snapshot management with configurable retention
**Prerequisites**:
- Must be run from Hetzner rescue system
- Requires at least 2 NVMe drives
- installimage must be available
**Usage**:
```bash
./ubuntu_install.sh
```
**⚠️ WARNING**: This script will completely wipe the selected drives and install a fresh Ubuntu system.
### 5. example_autoconfig/autoconfig
**Purpose**: Example configuration file for Hetzner installimage
**What it contains**:
- Sample configuration for Ubuntu 24.04 installation
- RAID 1 setup with two Samsung NVMe drives
- Standard partition layout:
- 4GB swap partition
- 1024MB /boot partition (ext3)
- Remaining space for root filesystem (ext4)
- Network and hostname configuration examples
- Comments explaining all configuration options
**Usage**:
This file serves as a template and reference for creating custom installimage configurations. Copy and modify as needed for specific server setups.
## Directory Structure
```
itenv_tools/tools/
├── readme.md # This documentation file
├── erase.sh # Disk erasure utility
├── git_checkout.sh # Git repository management
├── git_push.sh # Quick git commit/push
├── ubuntu_install.sh # Ubuntu installation script
└── example_autoconfig/ # Example configurations
└── autoconfig # Sample installimage config
```
## Security Considerations
- **erase.sh**: Requires root privileges and permanently destroys data
- **git_checkout.sh**: Requires SSH key access to private repositories
- **ubuntu_install.sh**: Must be run in Hetzner rescue environment
- All scripts include error handling and safety checks where appropriate
## Prerequisites
### System Requirements
- Linux environment (tested on Ubuntu/Debian)
- Bash shell
- Root access for disk operations
- Git for repository management
- SSH access for private repositories
### For Hetzner Operations
- Hetzner rescue system access
- installimage utility
- Minimum 2 NVMe drives for RAID setup
## Support
These tools are designed for infrastructure automation and server management. Ensure you understand the implications of each script before execution, especially those involving disk operations or system installation.

159
tools/ubuntu_install.sh
View File

@ -147,109 +147,108 @@ log "Created installimage configuration at /autosetup"
# Create autosetup directory for additional scripts # Create autosetup directory for additional scripts
mkdir -p /autosetup_scripts mkdir -p /autosetup_scripts
# Create post-installation script for btrfs optimization # # Create post-installation script for btrfs optimization
cat > /autosetup_scripts/post_install.sh << 'EOF' # cat > /autosetup_scripts/post_install.sh << 'EOF'
#!/bin/bash # #!/bin/bash
# Post-installation script for btrfs optimization # # Post-installation script for btrfs optimization
log() { # log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1" # echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
} # }
log "Starting post-installation btrfs optimization..." # log "Starting post-installation btrfs optimization..."
# Install btrfs-progs if not already installed # # Install btrfs-progs if not already installed
if ! command -v btrfs &> /dev/null; then # if ! command -v btrfs &> /dev/null; then
log "Installing btrfs-progs..." # log "Installing btrfs-progs..."
apt-get update # apt-get update
apt-get install -y btrfs-progs # apt-get install -y btrfs-progs
fi # fi
# Set btrfs mount options for better performance and features # # Set btrfs mount options for better performance and features
log "Configuring btrfs mount options..." # log "Configuring btrfs mount options..."
# Backup original fstab # # Backup original fstab
cp /etc/fstab /etc/fstab.backup # cp /etc/fstab /etc/fstab.backup
# Update fstab with optimized btrfs mount options # # Update fstab with optimized btrfs mount options
sed -i 's/btrfs\s\+defaults/btrfs defaults,noatime,compress=zstd:3,space_cache=v2,autodefrag/' /etc/fstab # sed -i 's/btrfs\s\+defaults/btrfs defaults,noatime,compress=zstd:3,space_cache=v2,autodefrag/' /etc/fstab
# Create btrfs maintenance scripts # # Create btrfs maintenance scripts
mkdir -p /etc/cron.weekly # mkdir -p /etc/cron.weekly
# Weekly balance script # # Weekly balance script
cat > /etc/cron.weekly/btrfs-balance << 'BALANCE_EOF' # cat > /etc/cron.weekly/btrfs-balance << 'BALANCE_EOF'
#!/bin/bash # #!/bin/bash
# Weekly btrfs balance to optimize space usage # # Weekly btrfs balance to optimize space usage
/usr/bin/btrfs balance start -dusage=50 -musage=50 / 2>/dev/null || true # /usr/bin/btrfs balance start -dusage=50 -musage=50 / 2>/dev/null || true
BALANCE_EOF # BALANCE_EOF
chmod +x /etc/cron.weekly/btrfs-balance # chmod +x /etc/cron.weekly/btrfs-balance
# Weekly scrub script for data integrity # # Weekly scrub script for data integrity
cat > /etc/cron.weekly/btrfs-scrub << 'SCRUB_EOF' # cat > /etc/cron.weekly/btrfs-scrub << 'SCRUB_EOF'
#!/bin/bash # #!/bin/bash
# Weekly btrfs scrub for data integrity check # # Weekly btrfs scrub for data integrity check
/usr/bin/btrfs scrub start / 2>/dev/null || true # /usr/bin/btrfs scrub start / 2>/dev/null || true
SCRUB_EOF # SCRUB_EOF
chmod +x /etc/cron.weekly/btrfs-scrub # chmod +x /etc/cron.weekly/btrfs-scrub
# Install and configure snapper for automatic snapshots # # Install and configure snapper for automatic snapshots
log "Installing and configuring snapper for automatic snapshots..." # log "Installing and configuring snapper for automatic snapshots..."
apt-get install -y snapper # apt-get install -y snapper
# Create snapper config for root # # Create snapper config for root
snapper -c root create-config / # snapper -c root create-config /
# Configure snapper for reasonable snapshot retention # # Configure snapper for reasonable snapshot retention
snapper -c root set-config TIMELINE_CREATE=yes # snapper -c root set-config TIMELINE_CREATE=yes
snapper -c root set-config TIMELINE_CLEANUP=yes # snapper -c root set-config TIMELINE_CLEANUP=yes
snapper -c root set-config NUMBER_CLEANUP=yes # snapper -c root set-config NUMBER_CLEANUP=yes
snapper -c root set-config NUMBER_MIN_AGE=1800 # snapper -c root set-config NUMBER_MIN_AGE=1800
snapper -c root set-config NUMBER_LIMIT=50 # snapper -c root set-config NUMBER_LIMIT=50
snapper -c root set-config NUMBER_LIMIT_IMPORTANT=10 # snapper -c root set-config NUMBER_LIMIT_IMPORTANT=10
# Enable snapper timer # # Enable snapper timer
systemctl enable snapper-timeline.timer # systemctl enable snapper-timeline.timer
systemctl enable snapper-cleanup.timer # systemctl enable snapper-cleanup.timer
log "Post-installation btrfs optimization completed" # log "Post-installation btrfs optimization completed"
EOF # EOF
chmod +x /autosetup_scripts/post_install.sh # chmod +x /autosetup_scripts/post_install.sh
log "Created post-installation script at /autosetup_scripts/post_install.sh" # log "Created post-installation script at /autosetup_scripts/post_install.sh"
# Create a script to monitor RAID status # # Create a script to monitor RAID status
cat > /autosetup_scripts/raid_monitor.sh << 'EOF' # cat > /autosetup_scripts/raid_monitor.sh << 'EOF'
#!/bin/bash # #!/bin/bash
# RAID monitoring script for btrfs # # RAID monitoring script for btrfs
check_btrfs_raid() { # check_btrfs_raid() {
echo "=== Btrfs RAID Status ===" # echo "=== Btrfs RAID Status ==="
btrfs filesystem show # btrfs filesystem show
echo # echo
echo "=== Btrfs Device Stats ===" # echo "=== Btrfs Device Stats ==="
btrfs device stats / # btrfs device stats /
echo # echo
echo "=== Btrfs Filesystem Usage ===" # echo "=== Btrfs Filesystem Usage ==="
btrfs filesystem usage / # btrfs filesystem usage /
} # }
# Check if btrfs tools are available # # Check if btrfs tools are available
if command -v btrfs &> /dev/null; then # if command -v btrfs &> /dev/null; then
check_btrfs_raid # check_btrfs_raid
else # else
echo "btrfs-progs not installed. Install with: apt-get install btrfs-progs" # echo "btrfs-progs not installed. Install with: apt-get install btrfs-progs"
fi # fi
EOF # EOF
chmod +x /autosetup_scripts/raid_monitor.sh # chmod +x /autosetup_scripts/raid_monitor.sh
# log "Created RAID monitoring script at /autosetup_scripts/raid_monitor.sh"
log "Created RAID monitoring script at /autosetup_scripts/raid_monitor.sh"
# Verify configuration # Verify configuration
log "Verifying configuration..." log "Verifying configuration..."
@ -284,4 +283,4 @@ else
fi fi
log "Installation process completed." log "Installation process completed."
EOF