class ExportWriter
def initialize(export)
@export = export
end
def append(rows)
return if rows.empty?
records = rows.map do |row|
{
export_id: @export.id,
row_key: row_key_for(row),
payload: serialize(row),
created_at: Time.current,
updated_at: Time.current
}
end
ExportRow.transaction do
ExportRow.upsert_all(
records,
unique_by: %i[export_id row_key]
)
@export.update!(
cursor: rows.last.id,
rows_written: @export.rows_written + rows.size
)
end
end
private
def row_key_for(row)
"#{row.class.name}:#{row.id}"
end
def serialize(row)
row.slice(:id, :name, :email, :created_at)
end
end
class ExportsController < ApplicationController
before_action :authenticate_user!
def create
export = current_user.exports.create!(
status: :processing,
cursor: 0,
rows_written: 0
)
ExportChunkJob.perform_later(export.id, cursor: 0)
respond_to do |format|
format.json { render json: export, status: :accepted }
format.html { redirect_to export_path(export) }
end
end
def show
export = current_user.exports.find(params[:id])
respond_to do |format|
format.json { render json: export.slice(:id, :status, :rows_written, :cursor) }
format.html
end
end
end
class ExportChunkJob < ApplicationJob
queue_as :exports
PAGE_SIZE = 500
after_perform do |job|
next unless job.instance_variable_defined?(:@next_cursor)
next_cursor = job.instance_variable_get(:@next_cursor)
ExportChunkJob.perform_later(job.arguments.first, cursor: next_cursor)
end
def perform(export_id, cursor:)
export = Export.find(export_id)
return if export.completed?
rows = export.source_scope
.where("id > ?", cursor)
.order(:id)
.limit(PAGE_SIZE)
.to_a
writer = ExportWriter.new(export)
writer.append(rows)
if continue?(rows)
@next_cursor = rows.last.id
else
finalize!(export)
end
end
private
def continue?(rows)
rows.size == PAGE_SIZE
end
def finalize!(export)
return if export.completed?
export.update!(status: :completed, finished_at: Time.current)
ExportMailer.with(export: export).ready.deliver_later
end
end
This snippet shows how a long-running export is split into a chain of dependent background jobs, where each job schedules its own successor from an ActiveJob callback rather than looping inside a single monster job. The pattern keeps individual jobs short (so GoodJob can retry them cheaply, respect timeouts, and interleave with other work) while still processing an unbounded dataset to completion.
In ExportChunkJob, perform processes exactly one page of rows starting from cursor, writes them into the export via ExportWriter, and then decides whether more work remains. Crucially it does not enqueue the next job directly in perform; instead it stores the next cursor in @next_cursor and lets an after_perform callback enqueue the continuation. Doing the enqueue in after_perform means the follow-up is only scheduled once the current unit of work has fully succeeded — if perform raises, no successor is queued and GoodJob retries the same chunk, preserving at-least-once semantics without duplicating the chain.
The continue? guard and finalize! split the terminal case from the recursive case, so the last job flips the export to completed and notifies the user exactly once. Idempotency matters because retries can re-run a chunk: ExportWriter#append in ExportWriter upserts on a deterministic row_key, so replaying the same cursor never produces duplicate output rows. The writer also advances export.cursor transactionally, which lets a resumed chain pick up exactly where it left off.
ExportsController kicks off the chain from create by enqueuing the first ExportChunkJob with cursor: 0, and exposes show for polling progress. Because the state lives in the Export row, the controller and jobs share a single source of truth.
The trade-off of this approach is latency: a chain of N chunks incurs N enqueue round-trips and is slower end-to-end than one tight loop. The payoff is resilience and fairness — no single job monopolizes a worker, memory stays flat per chunk, and a crash mid-export costs only one chunk of redo. A common pitfall is enqueuing the successor inside perform before the transaction commits; scheduling from after_perform avoids that race. This continuation style is worth reaching for whenever a task is naturally paginated and must survive restarts.
Related snips
class CommentsController < ApplicationController
before_action :set_post
def create
@comment = @post.comments.build(comment_params)System test: asserting Turbo Stream responses
class Post < ApplicationRecord
belongs_to :author, class_name: 'User'
has_many :comments, dependent: :destroy
scope :published, -> { where.not(published_at: nil).where('published_at <= ?', Time.current) }
scope :draft, -> { where(published_at: nil) }ActiveRecord scopes for reusable query logic
export interface RetryOptions {
retries: number;
baseMs: number;
maxMs: number;
signal?: AbortSignal;
onRetry?: (attempt: number, delay: number, err: unknown) => void;Exponential backoff with jitter for retries
module Api
module V1
class UsersController < BaseController
def show
user = User.includes(:profile).find(params[:id])ETags for conditional requests and caching
class PostsController < ApplicationController
def index
@posts = Post.includes(:author)
.order(created_at: :desc)
.page(params[:page])
.per(10)Turbo Frames: infinite scroll with lazy-loading frame
package dbutil
import (
"context"
"github.com/jackc/pgconn"Retry Postgres serialization failures with bounded attempts
Share this code
Here's the card — post it anywhere.
