codesnips

deduplication

javascript 
export class QueryCache {
  constructor() {
    this.entries = new Map();
  }

  getEntry(key) {

Building a Minimal useQuery Hook with a Shared Cache Provider in React

react hooks caching
by codesnips 4 tabs
typescript 
type Loader<T> = () => Promise<T>;

interface Entry<T> {
  value: T;
  expiresAt: number;
}

TTL Cache With In-Flight Request Deduplication for Async Calls

caching async deduplication
by codesnips 4 tabs
rust 
use std::collections::HashSet;
use std::hash::Hash;

pub struct DedupByKey<I, K, F> {
    inner: I,
    key_fn: F,

Order-Preserving Stream Deduplication by Key in Rust with a HashSet Guard

rust streams deduplication
by codesnips 3 tabs
typescript 
import { Queue } from "bullmq";
import { createHash } from "crypto";

export const connection = { host: "127.0.0.1", port: 6379 };

export interface ChargePayload {

BullMQ job idempotency via dedupe id

node redis background-jobs
by codesnips 3 tabs
python 
import datetime as dt

from sqlalchemy import Column, DateTime, Integer, String, UniqueConstraint
from sqlalchemy.orm import declarative_base

Base = declarative_base()

Idempotent Webhook Ingestion With a Postgres Dedupe Store in FastAPI

fastapi webhooks idempotency
by codesnips 3 tabs
rust 
use std::collections::HashMap;
use std::path::PathBuf;

use walkdir::WalkDir;

pub fn group_by_size(root: &str) -> HashMap<u64, Vec<PathBuf>> {

Detect Duplicate Files with Parallel Blake3 Checksums in Rust

rust checksums blake3
by codesnips 4 tabs
javascript 
'use strict';

function dedupeAsync(producer, keyFn) {
  const pending = new Map();
  const resolveKey = keyFn || ((...args) => JSON.stringify(args));

Deduplicate Concurrent Async Calls With a Promise Cache in Node.js

nodejs async promises
by codesnips 3 tabs
ruby 
class EmailDedupService
  DEFAULT_TTL = 5.minutes.to_i

  def self.claim(fingerprint, ttl: DEFAULT_TTL)
    key = "email_dedup:#{fingerprint}"
    # Atomic set-if-absent with expiry; returns true only for the first caller.

Throttle Duplicate Emails in Rails with a Mailer Interceptor and Redis Dedup Service

rails actionmailer redis
by codesnips 3 tabs
rust 
use crate::hashing::double_hash;

pub struct BloomFilter {
    bits: Vec<u64>,
    m: usize, // number of bits
    k: u32,   // number of hash probes

Bloom Filter for Deduplicating Seen URLs in a Web Crawler

bloom-filter probabilistic hashing
by codesnips 3 tabs
go 
package pricecache

import (
	"context"
	"sync"

Collapse Concurrent Identical Requests in Go with singleflight

go singleflight concurrency
by codesnips 3 tabs
php 
<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

Idempotent Stripe Webhook Processing With a Processed-Events Table in Laravel

laravel webhooks idempotency
by codesnips 3 tabs
go 
package mailer

import (
	"crypto/sha256"
	"encoding/hex"
	"strings"

Idempotent Email Sending with a Redis Fingerprint Set in Go

redis idempotency email
by codesnips 3 tabs