Laravel批量调用eBay API超时求助(无需修改服务器配置)
Hey there, let's break down why your current chunk approach isn't solving the timeout issues and fix it properly without touching server configs.
First, let's spot the issues in your current code
- Variable name collision in chunk callback: You're passing
$keywordCollectionboth as the callback parameter and in theuse()clause. This overwrites the variable unnecessarily and can cause unexpected behavior. - Synchronous API calls: Even with chunking, each
apirequestruns synchronously. If each API call takes just 1 second, 100 calls take 100 seconds—way over Nginx's typical 60-second timeout window, leading to 504 errors. - Hard exits on exceptions: Using
exitin your catch blocks terminates the entire script immediately, so remaining chunks won't process at all. - Potential eBay API rate limiting: Hammering the API with 100 requests in a row might trigger rate limits, leading to 405 or other errors from eBay's end.
Solution 1: Use Laravel Queues (Recommended)
Queues are perfect for this scenario—they offload API work to background processes, so your HTTP request finishes instantly, and jobs run asynchronously without timing out.
Step 1: Create a Job
Generate a job to handle a single keyword's API request:
php artisan make:job FetchEbayKeywordPrices
Update the job class (adjust namespaces to match your project):
<?php namespace App\Jobs; use Illuminate\Bus\Queueable; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Foundation\Bus\Dispatchable; use Illuminate\Queue\InteractsWithQueue; use Illuminate\Queue\SerializesModels; use App\Models\Keywords; use App\Models\Items; use Config; // Replace with your actual eBay SDK namespaces use YourEbaySdkNamespace\Types; use YourEbaySdkNamespace\Enums; class FetchEbayKeywordPrices implements ShouldQueue { use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; protected $keyword; protected $defaultStartDate; protected $ignoredItems; public function __construct($keyword, $defaultStartDate, $ignoredItems) { $this->keyword = $keyword; $this->defaultStartDate = $defaultStartDate; $this->ignoredItems = $ignoredItems; } public function handle() { try { $this->apirequest($this->keyword['id'], $this->keyword, $this->defaultStartDate, $this->ignoredItems); } catch (\Exception $e) { \Log::error("Failed to process keyword {$this->keyword['id']}: " . $e->getMessage()); $this->fail($e); // Mark job as failed for retry if needed } } private function apirequest($id, $keyword_result, $defaultStartDate, $ignoredItems) { try { $request = new Types\FindItemsAdvancedRequest(); $request->paginationInput = new Types\PaginationInput(); $request->paginationInput->entriesPerPage = Config::get('constants.paginationoptions.entries_per_page'); // Build condition filters $itemFilterForConditions = new Types\ItemFilter(); $itemFilterForConditions->name = 'Condition'; $itemFilterForConditions->value = ['1000', '1500', '3000']; $itemFilterduplicate = new Types\ItemFilter(); $itemFilterduplicate->name = 'HideDuplicateItems'; $itemFilterduplicate->value[] = 'true'; $itemFilterStartDateFrom = new Types\ItemFilter(); $itemFilterStartDateFrom->name = 'StartTimeFrom'; $itemFilterStartDateFrom->value[] = date('Y-m-d\TH:i:s\Z', mktime(0, 0, 0, date('m'), date('d') - $defaultStartDate['defaultStartDate'], date('y'))); $itemFilter = [$itemFilterForConditions, $itemFilterduplicate, $itemFilterStartDateFrom]; // Add price filter if applicable if (!empty($keyword_result['price']) && ($defaultStartDate['exclusionPercentage'] > 0)) { $percentageAmount = $keyword_result['price'] * ($defaultStartDate['exclusionPercentage'] / 100); $percentageAmount = floor($percentageAmount).substr($percentageAmount-floor($percentageAmount),1,3); $itemFilterPriced = new Types\ItemFilter(); $itemFilterPriced->name = 'MinPrice'; $itemFilterPriced->value[] = $percentageAmount; $itemFilter[] = $itemFilterPriced; } $request->itemFilter = $itemFilter; // Build search keyword $searchKeyword = (!empty($keyword_result['exclusions'])) ? $keyword_result['keyword'] . " -(" . trim($keyword_result['exclusions']) . ")" : $keyword_result['keyword']; $request->keywords = $searchKeyword; $request->sortOrder = 'PricePlusShippingLowest'; $request->descriptionSearch = ($defaultStartDate['advancedExclusions'] == 1) ? true : false; $service = $this->getEbayService(); // Replace with your service initialization logic from index() $this->apiresponse($request, $service, $id, $keyword_result['keyword'], $ignoredItems); } catch (\Exception $e) { throw $e; // Pass exception to job handler } } private function apiresponse($request, $service, $id, $keyword, $ignoredItems) { try { $response = $service->findItemsAdvanced($request); $newItems = []; $newOtherItems = []; $usedItems = []; if ($response->ack == "Success") { $results = $response->searchResult->item ?? []; foreach ($results as $item) { // Calculate item price if ($item->listingInfo->buyItNowAvailable == '1') { $itemPrice = $item->listingInfo->buyItNowPrice->value; } elseif (($item->listingInfo->listingType == 'Auction') && ($item->sellingStatus->currentPrice->value == 0)) { $itemResponse = $this->getItemDetails($item->itemId); $itemPrice = isset($itemResponse) ? $itemResponse : 0; } else { $itemPrice = $item->sellingStatus->currentPrice->value; } // Skip ignored items if (!in_array($item->itemId, $ignoredItems)) { switch ($item->condition->conditionId) { case '1000': $newItems[] = [ 'keywordId' => $id, 'conditionId' => (string) $item->condition->conditionId, 'itemId' => $item->itemId, 'title' => $item->title, 'price' => $itemPrice, 'itemURL' => $item->viewItemURL, 'galleryURL' => $item->galleryURL ?? '', ]; break; case '1500': $newOtherItems[] = [ 'keywordId' => $id, 'conditionId' => (string) $item->condition->conditionId, 'itemId' => $item->itemId, 'title' => $item->title, 'price' => $itemPrice, 'itemURL' => $item->viewItemURL, 'galleryURL' => $item->galleryURL ?? '', ]; break; case '3000': $usedItems[] = [ 'keywordId' => $id, 'conditionId' => (string) $item->condition->conditionId, 'itemId' => $item->itemId, 'title' => $item->title, 'price' => $itemPrice, 'itemURL' => $item->viewItemURL, 'galleryURL' => $item->galleryURL ?? '', ]; break; } } } // Limit to top 20 items per condition $newItems = array_slice($newItems, 0, 20); $newOtherItems = array_slice($newOtherItems, 0, 20); $usedItems = array_slice($usedItems, 0, 20); $insertItems = array_merge($newItems, $newOtherItems, $usedItems); // Update database Items::where('keywordId', $id)->delete(); Items::insert($insertItems); } elseif ($response->ack == "Failure") { $errors = ""; foreach ($response->errorMessage->error as $error) { $errors .= ($error->severity === Enums\ErrorSeverity::C_ERROR ? '<b>Error</b> ' : '<b>Warning</b> ') . $error->message . "\n\n"; } throw new \Exception($errors); } $this->setapilog($request, $response, $keyword); } catch (\Exception $e) { throw $e; } } // Add your getItemDetails, getEbayService, setapilog methods here }
Step 2: Update Your Route Handler
Dispatch jobs instead of processing synchronously:
public function updateAllKeywords(Request $request) { $defaultStartDate = // Your existing defaultStartDate logic $ignoredItems = // Your existing ignoredItems logic // Dispatch jobs in chunks to avoid memory overload Keywords::chunk(100, function ($keywords) use ($defaultStartDate, $ignoredItems) { foreach ($keywords as $keyword) { FetchEbayKeywordPrices::dispatch($keyword->toArray(), $defaultStartDate, $ignoredItems); } }); return response()->json(['message' => 'Keyword update jobs have been queued.']); }
Step 3: Run the Queue Worker
Start the worker to process jobs in the background:
php artisan queue:work
For production, use a process manager like Supervisor to keep the worker running continuously.
Solution 2: Optimize Synchronous Chunking (If Queues Aren't an Option)
If queues aren't feasible, tweak your existing code to reduce timeouts:
- Fix variable collision and error handling:
public function updateAllKeywords(Request $request) { set_time_limit(0); // Remove script timeout (or set to a high value like 3600) $defaultStartDate = // Your existing logic $ignoredItems = // Your existing logic Keywords::chunk(100, function ($chunk) use($request, $defaultStartDate, $ignoredItems) { foreach ($chunk as $keyword_result) { try { $this->apirequest($request, $keyword_result->id, $keyword_result->toArray(), $defaultStartDate, $ignoredItems); usleep(500000); // Add 0.5s delay to avoid eBay rate limits flush(); ob_flush(); } catch (\Exception $e) { \Log::error("Failed to process keyword {$keyword_result->id}: " . $e->getMessage()); // Continue processing other keywords instead of exiting } } }); return response()->json(['message' => 'Batch processing complete.']); }
- Remove hard exits in API methods:
Replace allecho json_encode(...); exit;with logging:
// Replace this: echo json_encode(array('ack' => 'Failure', 'msg' => $e->getMessage())); exit; // With this: \Log::error($e->getMessage()); throw $e; // Let the parent catch block handle it
Key Notes
- eBay Rate Limits: Always add small delays between requests to avoid hitting eBay's API rate limits.
- Memory Management: Use
chunk()orcursor()to keep memory usage low when processing thousands of records. - Error Resilience: Never terminate the script on a single failure—log errors and continue processing other records.
内容的提问来源于stack exchange,提问作者kinjal jethva

