Laravel的create()方法是否可能重复插入相同值？

Hey Vince, let's figure out why those duplicate entries are sneaking in even though you're checking for existing primary keys first. This is a classic issue with concurrent operations, so let's break down the root causes and straightforward fixes:

1. Race Conditions from Non-Atomic Checks #

The biggest culprit here is almost always race conditions. When your crawler runs in multiple processes, threads, or queue jobs, two separate instances can do the "check if exists" step at the exact same time—both see the key doesn't exist, then both run create() before either finishes inserting.

Fix: Use Laravel's Atomic Methods #

Replace your manual check + create() with firstOrCreate() or updateOrCreate(). These methods handle the check and insert in a single atomic database operation, which eliminates race conditions:

// Instead of:
if (!YourModel::where('source_id', $externalPk)->exists()) {
    YourModel::create(['source_id' => $externalPk, ...]);
}

// Do this:
YourModel::firstOrCreate(
    ['source_id' => $externalPk], // The unique identifier to check
    ['column1' => $data['val1'], 'column2' => $data['val2']] // Data to insert if not found
);

This works because Laravel uses database-level syntax like INSERT ... ON DUPLICATE KEY UPDATE (for MySQL) under the hood, which locks the row/table during the operation.

2. Missing Unique Database Constraint #

Even with firstOrCreate(), duplicates can happen if your source_id field doesn't have a unique index on the database table. Without this constraint, the database won't enforce uniqueness, and the atomic method can't do its job properly.

Fix: Add a Unique Index #

Via Migration:

// In your migration file
public function up()
{
    Schema::table('your_table_name', function (Blueprint $table) {
        $table->string('source_id')->unique()->change();
        // Or if the field already exists:
        // $table->unique('source_id');
    });
}

Via Raw SQL:

ALTER TABLE your_table_name ADD UNIQUE INDEX idx_unique_source_id (source_id);

3. Uncontrolled Crawler Concurrency #

If your crawler is processing multiple jobs in parallel (e.g., using Laravel Horizon or queue workers), you might need to add a lock to prevent multiple jobs from handling the same source_id at once.

Fix: Use Cache-Based Locks #

Add a temporary lock around your insertion logic to ensure only one process handles a given source_id at a time:

use Illuminate\Support\Facades\Cache;

$lockKey = "crawler:lock:source_id:{$externalPk}";

// Try to acquire a lock that lasts 10 minutes (prevents deadlocks if a job fails)
if (Cache::lock($lockKey, 600)->get()) {
    try {
        // Run your atomic insertion here
        YourModel::firstOrCreate(['source_id' => $externalPk], $data);
    } finally {
        // Always release the lock, even if an error occurs
        Cache::lock($lockKey)->release();
    }
}

4. Hidden Logic Flaws #

Double-check for these easy-to-miss issues:

• Case sensitivity: If your source_id is a string, make sure your database's collation doesn't treat "ABC" and "abc" as different (or vice versa) if that's not intended.
• Incorrect field checks: Ensure you're checking the exact same source_id value that you're inserting—sometimes crawlers might accidentally modify the value between check and insert.
• Transaction isolation levels: Rarely, but possible—if you're wrapping the check/insert in a custom transaction, make sure your isolation level isn't causing phantom reads. Stick to the default unless you have a specific reason to change it.

Quick Recap #

1. Add a unique index to your source_id field (non-negotiable).
2. Replace manual check+create with firstOrCreate() for atomic operations.
3. Add concurrency locks if you're running multiple crawler processes.

That should stamp out those duplicate entries for good!

内容的提问来源于stack exchange，提问作者Vince Carter