codesnips

batching

sql 
-- Step 1: add the column nullable, no default.
-- Catalog-only change in Postgres 11+, returns instantly.
ALTER TABLE orders
  ADD COLUMN currency text;

-- Optional: keep the lock attempt bounded so a long-running

SQL migration safety: add column nullable, backfill, then constrain

postgres migrations reliability
by codesnips 3 tabs
python 
from alembic import op
import sqlalchemy as sa

revision = "20240612_add_status"
down_revision = "20240515_create_orders"
branch_labels = None

Zero-Downtime NOT NULL Column Backfill with Alembic and SQLAlchemy

alembic sqlalchemy migrations
by codesnips 2 tabs
python 
import hashlib
from datetime import datetime, timezone

from sqlalchemy import BigInteger, Column, DateTime, String, UniqueConstraint
from sqlalchemy.orm import declarative_base

Efficient Bulk Insert in SQLAlchemy with a Reusable Chunking Helper

sqlalchemy postgres bulk-insert
by codesnips 3 tabs
rust 
use sqlx::PgPool;
use std::time::Duration;
use tokio::sync::mpsc;

#[derive(Debug, Clone)]
pub struct MetricPoint {

Batching Database Writes in Rust with a Size- and Interval-Triggered Flush Buffer

rust tokio sqlx
by codesnips 3 tabs
javascript 
'use strict';

const { Transform } = require('stream');

class LineSplitter extends Transform {
  constructor(options = {}) {

Split a Large Log File Into Date-Based Chunks With a Node.js Transform Stream

nodejs streams backpressure
by codesnips 3 tabs
go 
package watch

import (
	"sync"
	"time"
)

Debounce Filesystem Change Events Before Triggering a Rebuild in Go

fsnotify debounce filewatcher
by codesnips 3 tabs
ruby 
namespace :cleanup do
  desc "Enqueue a job to purge expired sessions"
  task expired_sessions: :environment do
    job = ExpiredSessionCleanupJob.perform_later
    Rails.logger.info("[cleanup:expired_sessions] enqueued job #{job.job_id}")
  end

Recurring Cleanup with a Rake Task and an Idempotent Active Job in Rails

rails background-jobs active-job
by codesnips 4 tabs
go 
package feed

import "time"

type Post struct {
	ID        int64

Batch-Load Related Rows with a Single IN Query to Avoid N+1

go sql postgres
by codesnips 3 tabs
javascript 
class BatchLoader {
  constructor(batchFn, { cacheKeyFn = (k) => k } = {}) {
    this.batchFn = batchFn;
    this.cacheKeyFn = cacheKeyFn;
    this.cache = new Map();
    this.queue = [];

Coalescing Concurrent Reads with a DataLoader-Style Batch Loader in Node

dataloader batching graphql
by codesnips 3 tabs
javascript 
const fs = require('fs');
const readline = require('readline');

async function* streamCsvRows(filePath) {
  const stream = fs.createReadStream(filePath, { encoding: 'utf8' });
  const rl = readline.createInterface({ input: stream, crlfDelay: Infinity });

Stream and Import a Large CSV File Line-by-Line with Node.js readline

nodejs streams readline
by codesnips 3 tabs