Linux服务器多用户处理器资源分配与队列管理方案咨询

Hey Amir, great question—this is exactly the kind of setup we use in academic computing labs all the time, so let's break this down in simple, actionable steps since you're new to Linux.

First off, you don't want to manage this manually (trust me, it'll get messy fast). The right tool for the job is a cluster workload scheduler, and the most popular (and beginner-friendly) option these days is Slurm Workload Manager. It's open-source, well-documented, and built to handle multi-server, multi-user environments like yours.

Step 1: Install Slurm on all your servers #

Slurm needs a "control node" (one server that manages the cluster) and "compute nodes" (your three 48-core servers). Here's a quick overview for Debian/Ubuntu (adjust commands for RHEL/CentOS if needed):

• On the control node:

  sudo apt update && sudo apt install slurm-wlm slurm-wlm-basic-plugins
  

• On each compute node:

  sudo apt update && sudo apt install slurm-wlm-basic-plugins
  

You'll need to edit the Slurm config file (/etc/slurm/slurm.conf) to define your cluster: set the control node name, list your compute nodes, and specify total CPUs per node (48 for each). There are simple online config generators that can help you create this file if you don't want to write it from scratch.

Step 2: Set up user resource quotas #

Slurm lets you enforce per-user CPU limits using its accounting system. First, enable accounting (install slurm-wlm-torque if you haven't already), then use the sacctmgr command to set quotas:

• Create an account for a user (e.g., user bob):

  sudo sacctmgr add user bob account bob_account
  

• Set a CPU limit for that user (e.g., max 8 CPUs at any time):

  sudo sacctmgr modify user bob set GrpCPUs=8
  

Now, whenever bob submits jobs that would push him over 8 CPUs, the extra jobs will automatically go into a queue until resources free up.

Step 3: How users submit jobs #

Users don't run commands directly on the nodes—they submit jobs to Slurm using sbatch (for batch jobs) or srun (for interactive jobs):

• Example batch job script (my_job.sh):

  #!/bin/bash
  #SBATCH --cpus-per-task=4  # Request 4 CPUs for this job
  #SBATCH --job-name=my_simulation
  
  # Your actual computational command here
  python my_model_script.py
  

• Submit the job:

  sbatch my_job.sh
  

If the user has available CPU quota, the job starts immediately. If not, it'll sit in the queue with a PD (pending) status until other jobs finish and free up resources.

Step 4: Check queue and resource status #

As an admin or user, you can easily monitor the cluster:

• View all jobs (queued and running):

  squeue
  

• View node status (how many CPUs are free/used on each server):

  sinfo
  

• View past job usage (to track who's using what resources):

  sacct
  

What if you don't want a full cluster scheduler? #

If you're only testing on one node first, you could use cgroups (Linux control groups) to limit CPU per user, but it's way more manual and doesn't handle queuing automatically. Slurm is worth the setup time because it scales to your three servers and takes care of all the hard parts for you.

Don't worry if the setup feels a bit overwhelming at first—Slurm has great beginner guides, and once it's running, it'll make managing your cluster so much easier. Let me know if you get stuck on any specific step!

备注：内容来源于stack exchange，提问作者amir