php 151 lines · 3 tabs

Chunked CSV Product Import With Laravel Batched Queue Jobs

Shared by codesnips Aug 2026
3 tabs
<?php

namespace App\Http\Controllers;

use App\Jobs\ImportProductChunk;
use App\Models\Import;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\LazyCollection;

class ProductImportController extends Controller
{
    public function store(Request $request)
    {
        $request->validate([
            'file' => ['required', 'file', 'mimes:csv,txt', 'max:20480'],
        ]);

        $path = $request->file('file')->store('imports');
        $import = Import::create([
            'user_id' => $request->user()->id,
            'path' => $path,
            'status' => 'pending',
        ]);

        $rows = LazyCollection::make(function () use ($import) {
            $file = new \SplFileObject(storage_path('app/' . $import->path));
            $file->setFlags(\SplFileObject::READ_CSV | \SplFileObject::SKIP_EMPTY);
            foreach ($file as $line) {
                if ($line !== [null]) {
                    yield $line;
                }
            }
        });

        $header = $rows->first();

        $jobs = $rows->skip(1)->chunk(500)->map(function ($chunk) use ($import, $header) {
            return new ImportProductChunk($import->id, $header, $chunk->values()->all());
        });

        $batch = Bus::batch($jobs->all())
            ->name("product-import:{$import->id}")
            ->allowFailures()
            ->then(fn () => $import->markCompleted())
            ->catch(fn ($batch, $e) => $import->markFailed($e->getMessage()))
            ->finally(fn () => $import->refresh()->finalize())
            ->dispatch();

        $import->update(['batch_id' => $batch->id, 'status' => 'processing']);

        return response()->json(['import_id' => $import->id, 'batch_id' => $batch->id]);
    }
}
3 files · php Explain with highlit

Importing a large product CSV in a single HTTP request is a recipe for timeouts and memory blowups. This snippet shows the Laravel-idiomatic alternative: parse the file lazily, split it into fixed-size chunks, and dispatch each chunk as its own queued job inside a Bus::batch, so the work is durable, resumable, and observable.

In ProductImportController, the uploaded file is stored on disk (never held entirely in memory) and an Import record is created to track state. LazyCollection::make streams the file line by line with SplFileObject, and ->chunk(500) groups rows so each ImportProductChunk job handles a bounded slice. Wrapping the jobs in Bus::batch(...) gives a single batch id, progress counters, and lifecycle hooks: then marks the import completed, catch records the first failure, and finally always finalizes state regardless of outcome. allowFailures() lets good chunks succeed even when one row batch dies. The controller returns the batch->id so a frontend can poll progress.

ImportProductChunk is the unit of work. It carries the raw rows and the header, and its handle method maps each row to an associative array keyed by header. The crucial detail is Product::upsert(...): a single bulk statement that inserts new SKUs and updates existing ones by the unique sku column, making the job idempotent. If the queue retries the job after a transient failure, re-running the same rows produces the same result rather than duplicate products. WithoutOverlapping keyed by import id prevents two chunks of the same import from racing on the same rows, and $tries/backoff give bounded retries with delay.

The Import model in Import model exposes a progress accessor and small state transitions (markCompleted, markFailed) so controllers and views read intent, not raw columns. The trade-offs: chunk size balances per-job overhead against memory and retry granularity, and upserts require a real unique index on sku. This pattern fits any large-file ingestion — orders, contacts, inventory — where partial progress, retries, and back-pressure matter more than raw speed.


Related snips

Share this code

Here's the card — post it anywhere.

Chunked CSV Product Import With Laravel Batched Queue Jobs — share card
Link copied