module RetryableTransaction
module_function
RETRIABLE = [ActiveRecord::Deadlocked, ActiveRecord::LockWaitTimeout].freeze
def run(max_retries: 3, base_delay: 0.05, &block)
attempts = 0
begin
attempts += 1
ActiveRecord::Base.transaction(requires_new: true, &block)
rescue *RETRIABLE => e
raise e unless attempts <= max_retries
Rails.logger.warn(
"[RetryableTransaction] #{e.class} on attempt #{attempts}/#{max_retries + 1}, retrying"
)
sleep(backoff_for(attempts, base_delay))
retry
end
end
def backoff_for(attempt, base_delay)
exponential = base_delay * (2**(attempt - 1))
jitter = rand * base_delay
exponential + jitter
end
end
class AccountTransfer
Result = Struct.new(:from, :to, :amount, keyword_init: true)
def initialize(from_id:, to_id:, amount_cents:)
@from_id = from_id
@to_id = to_id
@amount_cents = amount_cents
end
def call
raise ArgumentError, "amount must be positive" unless @amount_cents.positive?
RetryableTransaction.run do
first, second = with_advisory_ordering
first.lock!
second.lock!
from = accounts.fetch(@from_id)
to = accounts.fetch(@to_id)
raise InsufficientFunds, from.id if from.balance_cents < @amount_cents
from.update!(balance_cents: from.balance_cents - @amount_cents)
to.update!(balance_cents: to.balance_cents + @amount_cents)
Result.new(from: from, to: to, amount: @amount_cents)
end
end
private
# Acquire locks in a stable id order to avoid ordering-based deadlocks.
def with_advisory_ordering
accounts.values.sort_by(&:id)
end
def accounts
@accounts ||= Account.where(id: [@from_id, @to_id]).index_by(&:id)
end
class InsufficientFunds < StandardError; end
end
class ProcessTransferJob < ApplicationJob
queue_as :payments
# Transient lock contention is handled in-process by RetryableTransaction.
# Only escalate persistent failures back to the queue.
retry_on ActiveRecord::ConnectionNotEstablished, wait: :polynomially_longer, attempts: 5
discard_on AccountTransfer::InsufficientFunds
def perform(transfer_id)
request = TransferRequest.pending.find(transfer_id)
result = AccountTransfer.new(
from_id: request.from_account_id,
to_id: request.to_account_id,
amount_cents: request.amount_cents
).call
request.update!(status: :settled, settled_at: Time.current)
LedgerMailer.transfer_receipt(result).deliver_later
end
end
Concurrent writes that touch the same rows in different orders eventually collide. When two transactions each hold a lock the other needs, the database picks a victim and aborts it with a deadlock error rather than hanging forever. These aborts are transient and safe to retry, but only if the entire transaction is replayed from scratch — retrying a half-applied unit of work would corrupt data. This snippet packages that discipline into a reusable helper.
In RetryableTransaction, the run method wraps a block in ActiveRecord::Base.transaction and rescues the specific transient errors: ActiveRecord::Deadlocked and ActiveRecord::LockWaitTimeout. Both are subclasses of StatementInvalid that Rails raises for the underlying Postgres/MySQL error codes, so matching on them is more precise than rescuing generic exceptions. The retryable? check ensures unrelated failures propagate immediately. On a retriable error the method sleeps for a randomized exponential backoff via backoff_for, which spreads out competing retries so two victims do not simply re-collide in lockstep. After max_retries attempts it re-raises so the caller — or a job runner — can fail loudly.
The key correctness point is that the transaction boundary lives inside the retry loop. Each attempt opens a fresh transaction, so any partial writes from a failed attempt are rolled back before the block runs again. The block must therefore be idempotent and free of external side effects like emails, since it can execute multiple times.
In AccountTransfer service, call uses RetryableTransaction.run to move money between accounts. Rows are locked with lock! in a consistent order (lowest id first) — ordering lock acquisition is the real fix for deadlocks, while the retry wrapper handles the residual races that ordering cannot fully eliminate. with_advisory_ordering sorts the accounts so every caller grabs locks the same way.
In ProcessTransferJob, the ActiveJob defines retry_on for genuinely persistent problems but delegates transient lock contention to the wrapper, keeping job-level retries cheap and rare. This layering — order locks, retry deadlocks in-process, escalate the rest to the queue — is the standard approach for reliable financial writes under load.
Related snips
class CommentsController < ApplicationController
before_action :set_post
def create
@comment = @post.comments.build(comment_params)System test: asserting Turbo Stream responses
class Post < ApplicationRecord
belongs_to :author, class_name: 'User'
has_many :comments, dependent: :destroy
scope :published, -> { where.not(published_at: nil).where('published_at <= ?', Time.current) }
scope :draft, -> { where(published_at: nil) }ActiveRecord scopes for reusable query logic
export interface RetryOptions {
retries: number;
baseMs: number;
maxMs: number;
signal?: AbortSignal;
onRetry?: (attempt: number, delay: number, err: unknown) => void;Exponential backoff with jitter for retries
module Api
module V1
class UsersController < BaseController
def show
user = User.includes(:profile).find(params[:id])ETags for conditional requests and caching
class PostsController < ApplicationController
def index
@posts = Post.includes(:author)
.order(created_at: :desc)
.page(params[:page])
.per(10)Turbo Frames: infinite scroll with lazy-loading frame
package dbutil
import (
"context"
"github.com/jackc/pgconn"Retry Postgres serialization failures with bounded attempts
Share this code
Here's the card — post it anywhere.
