Ubuntu裸金属服务器核心服务资源保障与资源超限用户溯源方案咨询

Hey there, let’s break down practical solutions for your two key problems—since virtualization isn’t an option right now, all these fixes are tailored for bare-metal Ubuntu systems and are easy to roll out without major infrastructure changes.

1. 保障SSH及核心管理服务的资源优先级与预留 #

To make sure critical services like SSH and your custom management tools never get starved of resources, you’ve got a few solid, actionable options:

• Systemd Resource Controls (Recommended)
  Ubuntu uses systemd by default, so you can directly set resource guarantees and limits for your services. For example, to lock in resources for SSH:

  1. Create an override config file for sshd:

     sudo systemctl edit sshd.service
     

  2. Add these lines (adjust values based on your server’s specs):

     [Service]
     # Reserve at least 512MB of memory for SSHD to ensure it runs even under load
     MemoryMin=512M
     # Allow SSHD to use up to 20% of CPU (prevents it from hogging resources, but guarantees access to this share when needed)
     CPUQuota=20%
     # Set higher process priority (lower nice value = higher priority)
     Nice=-10
     

  3. Save and exit, then reload systemd and restart the service:

     sudo systemctl daemon-reload
     sudo systemctl restart sshd
     

  Repeat this process for your custom management services by editing their respective .service files.

• Cgroups Manual Configuration
  For more granular control, use cgroups to create a dedicated "critical-services" group:

  1. Install cgroup tools:

     sudo apt install cgroup-tools
     

  2. Create a cgroup for critical processes:

     sudo cgcreate -g cpu,memory:/critical-services
     

  3. Set CPU shares (higher value = more priority; default is 1024):

     sudo cgset -r cpu.shares=2048 critical-services
     

  4. Move running SSHD processes into the group:

     sudo cgclassify -g cpu,memory:/critical-services $(pidof sshd)
     

  To make this persistent across reboots, add the cgclassify command to a startup script or integrate it with systemd’s cgroup settings.

2. 识别并记录资源超限用户 #

Tracking users who hog resources will help you target training effectively. Here are practical tools and scripts to set this up:

• Use psacct for Detailed User Resource Tracking
  psacct (or acct) logs every user’s process activity, making it easy to spot frequent resource hogs:

  1. Install the package:

     sudo apt install psacct
     

  2. Start and enable the service:

     sudo systemctl start acct
     sudo systemctl enable acct
     

  3. Use these commands to analyze usage:
     • sa: Shows a summary of CPU/memory usage per user
     • lastcomm: Lists recent commands executed by users, including resource stats
     • ac: Shows total connect time per user (useful for identifying consistently active users)

• Custom Resource Monitoring Script
  For real-time tracking and persistent logs, create a simple bash script that captures top resource consumers:

  1. Create a script file (e.g., /usr/local/bin/resource-monitor.sh):

     #!/bin/bash
     LOG_FILE="/var/log/resource-hogs.log"
     TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S")
     
     echo "=== Resource Hog Report - $TIMESTAMP ===" >> "$LOG_FILE"
     echo "Top 10 CPU Consumers (User, PID, %CPU, Command):" >> "$LOG_FILE"
     ps aux --sort=-%cpu | awk '{print $1, $2, $3, $11}' | head -11 >> "$LOG_FILE"
     echo -e "\nTop 10 Memory Consumers (User, PID, %MEM, Command):" >> "$LOG_FILE"
     ps aux --sort=-%mem | awk '{print $1, $2, $4, $11}' | head -11 >> "$LOG_FILE"
     echo -e "\n==========================================\n" >> "$LOG_FILE"
     

  2. Make it executable:

     sudo chmod +x /usr/local/bin/resource-monitor.sh
     

  3. Schedule it to run every 5 minutes with cron:

     sudo crontab -e
     

     Add this line:

     */5 * * * * /usr/local/bin/resource-monitor.sh
     

  You can then review /var/log/resource-hogs.log regularly to identify repeat offenders.

• Sysstat for Historical Resource Trends
  Sysstat provides system activity reports that help you correlate resource spikes with user activity:

  1. Install sysstat:

     sudo apt install sysstat
     

  2. Enable it by editing /etc/default/sysstat and setting ENABLED="true"
  3. Restart the service:

     sudo systemctl restart sysstat
     

  4. Use sar -u to view CPU usage history, sar -r for memory usage, and sadf to export data for deeper analysis.

备注：内容来源于stack exchange，提问作者zhang