class Order < ApplicationRecord
belongs_to :customer
scope :for_export, -> {
select(:id, :reference, :total_cents, :currency, :created_at, :customer_id)
.where.not(exported_at: nil)
}
def display_total
format('%.2f %s', total_cents / 100.0, currency)
end
end
require 'csv'
class ExportOrdersJob < ApplicationJob
queue_as :exports
BATCH_SIZE = 5_000
def perform(export_id)
export = Export.find(export_id)
path = Rails.root.join('tmp', "orders-#{export_id}.csv")
CSV.open(path, 'w') do |csv|
csv << %w[id reference total currency created_at]
Order.for_export.find_each(batch_size: BATCH_SIZE) do |order|
csv << serialize_row(order)
end
end
export.update!(status: 'ready', file_path: path.to_s)
end
private
def serialize_row(order)
[
order.id,
order.reference,
order.display_total,
order.currency,
order.created_at.iso8601
]
end
end
class ExportsController < ApplicationController
def create
export = Export.create!(status: 'pending', kind: 'orders')
ExportOrdersJob.perform_later(export.id)
render json: { id: export.id, status: export.status }, status: :accepted
end
def show
export = Export.find(params[:id])
if export.status == 'ready'
send_file export.file_path, type: 'text/csv', filename: "orders-#{export.id}.csv"
else
render json: { id: export.id, status: export.status }, status: :ok
end
end
end
When a Rails app needs to export or process hundreds of thousands of rows, the naive Model.all.each loads every record into memory at once, which can exhaust the heap and take down a worker. The find_each API solves this by pulling records in batches (default 1000) and yielding them one at a time, so only a slice of the table lives in memory during iteration. Pairing that with a narrow select further shrinks each object, since ActiveRecord otherwise hydrates every column — including large text blobs — into every instance.
In Order model, the for_export scope encodes the two decisions that matter for memory: select limits the query to just the columns the export actually needs, and where.not(exported_at: nil) filters at the database rather than in Ruby. Selecting a subset of columns means the returned objects are partial; touching an unselected attribute like notes would raise ActiveModel::MissingAttributeError, which is a deliberate guardrail rather than a bug.
The heavy lifting happens in ExportOrdersJob, which streams straight to a file. It opens a CSV, writes the header once, then calls for_export.find_each with an explicit batch_size. Because find_each orders by primary key and pages using id > last_id under the hood, it avoids the OFFSET performance cliff that plagues LIMIT/OFFSET pagination on large tables. A larger batch_size means fewer round trips but more memory per batch, so it is tuned to the row width.
One subtlety: find_each ignores any explicit order clause because it must sort by primary key to page correctly, so ordering is applied later or not at all here. The serialize_row helper reads only the selected columns, keeping the object graph flat.
In ExportsController, the request thread does almost nothing — it enqueues the job and returns 202 Accepted, delegating the slow, memory-sensitive work to a background worker. This is the pattern to reach for whenever a full-table scan would otherwise be attempted inline: batch the reads, select only what is needed, and never build one giant array in memory.
Related snips
class CommentsController < ApplicationController
before_action :set_post
def create
@comment = @post.comments.build(comment_params)System test: asserting Turbo Stream responses
class Post < ApplicationRecord
belongs_to :author, class_name: 'User'
has_many :comments, dependent: :destroy
scope :published, -> { where.not(published_at: nil).where('published_at <= ?', Time.current) }
scope :draft, -> { where(published_at: nil) }ActiveRecord scopes for reusable query logic
module Api
module V1
class UsersController < BaseController
def show
user = User.includes(:profile).find(params[:id])ETags for conditional requests and caching
class PostsController < ApplicationController
def index
@posts = Post.includes(:author)
.order(created_at: :desc)
.page(params[:page])
.per(10)Turbo Frames: infinite scroll with lazy-loading frame
package dbutil
import (
"context"
"github.com/jackc/pgconn"Retry Postgres serialization failures with bounded attempts
require "csv"
class PeopleCsvStream
include Enumerable
HEADERS = %w[id full_name email signed_up_at plan].freezeResilient CSV Export as a Streamed Response
Share this code
Here's the card — post it anywhere.
