import logging
import re
_BASE_FIELDS = set(logging.makeLogRecord({}).__dict__.keys())
_PATTERNS = [
(re.compile(r"(?i)bearer\s+[a-z0-9._\-]+"), "Bearer [REDACTED]"),
(re.compile(r"[a-z0-9._%+\-]+@[a-z0-9.\-]+\.[a-z]{2,}", re.I), "[EMAIL]"),
(re.compile(r"\b\d{13,19}\b"), "[CARD]"),
]
class SensitiveDataFilter(logging.Filter):
MASK = "[REDACTED]"
def __init__(self, sensitive_keys=None):
super().__init__()
keys = sensitive_keys or ["password", "token", "secret", "authorization", "api_key"]
self.sensitive_keys = {k.lower() for k in keys}
def filter(self, record):
if isinstance(record.args, dict):
record.args = self._redact_value(record.args)
elif isinstance(record.args, tuple):
record.args = tuple(self._redact_value(a) for a in record.args)
record.msg = self._mask_patterns(record.msg)
self._scrub_record(record)
return True
def _scrub_record(self, record):
for name in list(record.__dict__.keys()):
if name in _BASE_FIELDS:
continue
if name.lower() in self.sensitive_keys:
record.__dict__[name] = self.MASK
else:
record.__dict__[name] = self._redact_value(record.__dict__[name])
def _redact_value(self, value):
if isinstance(value, dict):
return {
k: (self.MASK if k.lower() in self.sensitive_keys else self._redact_value(v))
for k, v in value.items()
}
if isinstance(value, (list, tuple)):
return type(value)(self._redact_value(v) for v in value)
return self._mask_patterns(value)
def _mask_patterns(self, value):
if not isinstance(value, str):
return value
for pattern, replacement in _PATTERNS:
value = pattern.sub(replacement, value)
return value
import json
import logging
_RESERVED = set(logging.makeLogRecord({}).__dict__.keys())
class JSONFormatter(logging.Formatter):
def format(self, record):
payload = {
"timestamp": self.formatTime(record, "%Y-%m-%dT%H:%M:%S%z"),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
}
for key, value in record.__dict__.items():
if key not in _RESERVED and not key.startswith("_"):
payload[key] = value
if record.exc_info:
payload["exception"] = self.formatException(record.exc_info)
return json.dumps(payload, default=str, ensure_ascii=False)
import logging
import sys
from json_formatter import JSONFormatter
from redaction import SensitiveDataFilter
def configure_logging(level=logging.INFO, extra_keys=None):
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JSONFormatter())
# Attach on the handler so every record it writes is scrubbed.
handler.addFilter(SensitiveDataFilter(sensitive_keys=extra_keys))
root = logging.getLogger()
root.handlers.clear()
root.addHandler(handler)
root.setLevel(level)
return root
if __name__ == "__main__":
log = configure_logging()
log.info(
"user login attempt for %(email)s",
{"email": "jane@example.com"},
extra={"password": "hunter2", "request_id": "r-91", "card": "4111111111111111"},
)
This snippet shows how to strip secrets and PII out of structured log records before they ever reach a handler, using Python's standard logging machinery rather than a bolt-on wrapper. The core idea is that a logging.Filter sits in the pipeline between a logger and its handlers and can mutate each LogRecord in place, so redaction happens centrally and cannot be forgotten at individual call sites.
In redaction.py, SensitiveDataFilter implements filter(record), which always returns True so the record is still emitted — the return value controls whether a record passes, not whether it is modific. The filter walks two sources of leakage: free-form message arguments and structured extra fields attached to the record. Key-based redaction is driven by sensitive_keys (things like password, token, authorization), while _PATTERNS catches values that look secret regardless of their key — bearer tokens, emails, and long digit runs that resemble card numbers. _redact_value recurses through nested dicts and lists so a token buried inside a payload is still masked.
A subtle but important detail is _scrub_record: structured attributes live as arbitrary attributes on the record object (that is how logger.info(msg, extra={...}) works), so the filter compares each record attribute against a snapshot of the base LogRecord fields and only rewrites the custom ones. This avoids clobbering internal fields like levelname or created.
json_formatter.py provides JSONFormatter, which serializes the already-redacted record into a single JSON line, pulling in any non-standard attributes as top-level keys. Because redaction runs as a filter and formatting runs afterward, the two concerns stay cleanly separated.
logging_setup.py wires it together: configure_logging attaches SensitiveDataFilter to the handler, not just the logger, because filters on a logger do not propagate to child loggers, whereas a handler-level filter guards everything that handler writes. The trade-off worth noting is cost — regex scanning every record adds overhead, so sensitive_keys should stay focused and patterns anchored. This pattern is the right reach when logs flow to a third-party sink and compliance requires that raw credentials never leave the process.
Related snips
package api
import (
"net/http"
"runtime/debug"
)Expose build metadata for debugging deploys
payload = {
sub: user.id,
iss: 'https://auth.example.com',
aud: 'codesnips-api',
exp: 15.minutes.from_now.to_i,
iat: Time.now.to_i,JWT issuance and verification without common footguns
use tracing::{info, instrument};
#[instrument]
fn process_request(user_id: u64) {
info!(user_id, "Processing request");
// Work happens heretracing for structured logging and distributed tracing
#!/usr/bin/env bash
set -euo pipefail
export VAULT_ADDR="https://vault.internal:8200"
export VAULT_TOKEN="${VAULT_TOKEN:?missing VAULT_TOKEN}"Secrets management with environment isolation and Vault
import { randomBytes, createHash } from "crypto";
import jwt from "jsonwebtoken";
import { RefreshTokenStore } from "./store";
const ACCESS_SECRET = process.env.ACCESS_SECRET!;
const ACCESS_TTL = "15m";JWT access + refresh token rotation (conceptual)
package files
import (
"context"
"time"Presigned S3 upload URLs (AWS SDK v2)
Share this code
Here's the card — post it anywhere.
