如何用WaitGroup与Goroutines优化Go多线程？性能调优求助

Let's break down your problems and fix them step by step—no copy-paste dumps, just clear reasoning and actionable changes that will boost your iteration rate and eliminate those annoying bugs.

First: Why Your Iteration Rate Drops So Much #

Your current code is not actually running in parallel—that's the biggest performance killer. In the main loop, you start a goroutine then immediately call wg.Wait(), which blocks until that single goroutine finishes. This means you're running one task at a time, with all the overhead of creating/destroying a goroutine every iteration. No wonder your rate plummets!

Second: The WaitGroup Reuse & Race Condition Mess #

You're using a global WaitGroup for all tasks—main loop, printKeys, and nested goroutines. This is a recipe for disaster: multiple goroutines are modifying the same WG counter at the same time, leading to unexpected behavior like reuse before Wait() completes. Plus, your global variables (found, pages_queried) are being modified without synchronization, causing race conditions that mess up your counts.

Third: Print Overlaps #

When multiple goroutines call fmt.Printf at the same time, the console output gets interleaved because printing isn't atomic. Adding goroutines to run printKeys makes this worse—no coordination means overlapping text.

Step-by-Step Fixes #

1. Switch to a Worker Pool for True Parallelism #

Instead of creating a goroutine per iteration, use a fixed pool of workers (one per CPU core, to maximize hardware usage) that pull tasks from a channel. This eliminates goroutine creation overhead and lets you utilize all your Cloud container's cores.

2. Ditch Global WaitGroups & Use Atomic Operations #

• Use local WaitGroups for independent task groups (like checking keys in printKeys).
• Use sync/atomic for global counters (found, pages_queried) to avoid race conditions.

3. Synchronize Console Output #

Add a global mutex to wrap all console printing operations—this ensures only one goroutine writes to stdout at a time, eliminating overlaps. Use ANSI control codes to redraw the status line in place instead of spamming new lines.

Refactored Code with Explanations #

First, update your imports and global variables to include atomic operations and a print mutex:

import (
    "fmt"
    "math/big"
    "os"
    "os/exec"
    "runtime"
    "sync"
    "sync/atomic"
    "time"
)

// Use atomic integers for counters to avoid race conditions
var found int64
var pagesQueried int64
var foundAddresses int64

var startTime = time.Now()
var bignum = new(big.Int)
var set = make(map[string]bool)
var addresses = []string{"6ab42gyr", "lo08n4g6"}
var printMutex sync.Mutex // Syncs all console output to prevent overlaps

Main Function: Worker Pool Setup #

This creates a pool of workers, feeds them tasks, and waits for all to complete:

func main() {
    bignum.SetString("1000000000000000000000000000", 10)
    pick := os.Args[1]
    kpp := 128

    switch pick {
    case "btc":
        i := new(big.Int)
        var ok bool
        i, ok = i.SetString(os.Args[2], 10)
        if !ok {
            fmt.Println("Invalid starting number—please use a valid integer string")
            return
        }

        // Clear the console once at startup
        cmd := exec.Command("clear")
        cmd.Stdout = os.Stdout
        cmd.Run()

        // Use all CPU cores for workers
        workerCount := runtime.NumCPU()
        jobs := make(chan *big.Int, workerCount*2) // Buffered channel to avoid blocking
        var wg sync.WaitGroup

        // Start workers
        for w := 0; w < workerCount; w++ {
            wg.Add(1)
            go worker(&wg, jobs, kpp)
        }

        // Send tasks to workers
        for i.Cmp(bignum) < 0 {
            // Copy the big.Int value to avoid race conditions in the main loop
            job := new(big.Int).Set(i)
            jobs <- job
            i.Add(i, big.NewInt(1))
        }

        close(jobs) // Tell workers no more tasks are coming
        wg.Wait()   // Wait for all workers to finish
    }
}

Worker Function: Handles Task Execution #

Each worker pulls tasks from the channel, processes them, and updates counters safely:

func worker(wg *sync.WaitGroup, jobs <-chan *big.Int, kpp int) {
    defer wg.Done()
    for job := range jobs {
        printKeys(job.String(), kpp)
        atomic.AddInt64(&pagesQueried, 1) // Safe increment of global counter
        printInfo(job)                    // Thread-safe status update
    }
}

Thread-Safe Status Printing #

This uses the mutex to ensure clean, non-overlapping status updates:

func printInfo(i *big.Int) {
    elapsed := time.Now().Sub(startTime)
    durationSec := int(elapsed.Seconds())
    if durationSec == 0 {
        return // Avoid division by zero early on
    }

    // Safely read atomic counters
    queried := atomic.LoadInt64(&pagesQueried)
    foundVal := atomic.LoadInt64(&found)

    printMutex.Lock()
    defer printMutex.Unlock()

    // ANSI codes: move to home position, clear the line, then print
    fmt.Printf("\033[H\033[2K")
    fmt.Printf("Started at %s | Found: %d | Elapsed: %s | Queried: %d pages | Current: %s | Rate: %d/s",
        startTime.Format(time.RFC3339), foundVal, elapsed.String(), queried, i.String(), int(queried)/durationSec)
}

Fixed printKeys: Local WaitGroup & Safe Printing #

No more global WG, and all address-found printing is synchronized:

func printKeys(pageNumber string, keysPerPage int) {
    keys := generateKeys(pageNumber, keysPerPage)
    keysLen := len(keys)
    addressesLen := len(addresses)

    var wg sync.WaitGroup // Local WG for this batch of key checks
    for i := 0; i < keysLen; i++ {
        wg.Add(1)
        // Capture the current index's key explicitly to avoid loop variable reuse bugs
        go func(idx int) {
            defer wg.Done()
            key := keys[idx]
            for ii := 0; ii < addressesLen; ii++ {
                if set[key.compressed] || set[key.uncompressed] {
                    printMutex.Lock()
                    defer printMutex.Unlock()
                    fmt.Printf("\nFound an address: %s!\n", addresses[ii])
                    fmt.Printf("%+v\n", key)
                    atomic.AddInt64(&found, 1)
                    atomic.AddInt64(&foundAddresses, 1)
                }
            }
        }(i)
    }
    wg.Wait() // Wait for all key checks in this batch to finish
    atomic.StoreInt64(&foundAddresses, 0)
}

Key Performance & Bug Fix Takeaways #

• True Parallelism: The worker pool uses all your CPU cores, eliminating the single-threaded bottleneck that killed your iteration rate.
• No Race Conditions: Atomic operations and local WaitGroups ensure shared state is accessed safely.
• Clean Output: The print mutex stops overlapping text, and ANSI codes keep your status line tidy.
• Lower Overhead: Fixed workers avoid the cost of creating/destroying goroutines for every iteration.

You should see a massive jump in your iteration rate now—easily hitting your target if your container's resources are being utilized properly.

内容的提问来源于stack exchange，提问作者Devin Rowan