File Parsing Bot Version 2, Now With Transcription
On another page,page, I detailed how I created a Python script that will take recorded audio clips, parse the metadata, and post it to a Discord server. It works fantastic, but I figured I could improve things - this time, with audio transcription. Here is what I did on my Windows 11 2025 Server to get this tweaked. If you havent already, set up the normal one first, as these instructions assume you've already done that.
Prereqs
- Windows 11 / Server (64-bit)
- Python 3.13 x64 installed to
C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe - Microsoft VC++ 2015-2022 Redistributable (x64)
- Discord Bot & Server
Directory Layout
- Working Folder:
C:\bots\sdr-discord-uploader - Script:
C:\bots\sdr-discord-uploader\sdrtrunk_discord_uploader.py - SDRTrunk Output Folder:
C:\Users\-YourUser-\OneDrive\Scanner Recordings\SDRTrunk- I use OneDrive as an archive
Python Dependencies (Installed to the Correct Interpreter)
Use the call operator & in PowerShell when a path has spaces!
# Upgrade pip
& "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" -m pip install --upgrade pip
# Core deps
& "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" -m pip install -U ^
discord.py watchdog mutagen python-dotenv
# Transcription deps (CPU)
& "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" -m pip install -U ^
faster-whisper ffmpeg-python
# Optional: faster HF downloads (harmless to skip)
# & "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" -m pip install "huggingface_hub[hf_xet]"Sanity Test - this should print versions, no errors):
& "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" -c "import ctranslate2, faster_whisper; print('ctranslate2', ctranslate2.__version__); print('faster-whisper OK')".env Configuration
Create your .env file, I put mine in the Working Folder from above
# REQUIRED
DISCORD_TOKEN=YOUR_BOT_TOKEN_HERE
DEFAULT_CHANNEL_ID=12345678901234567890
# SDRTrunk folder to watch
WATCH_FOLDER=C:\Users\-YourUser-\OneDrive\Scanner Recordings\SDRTrunk
# Category that holds system channels
AUDIO_CATEGORY_NAME=AUDIO
# Map SDRTrunk "System" -> exact channel name in your server
SYSTEM_TO_CHANNEL_OVERRIDES=AudioCh1->a01-channel1,SKYNET2->a02-skynet
# Threads
ENABLE_THREADS=true
THREAD_AUTO_ARCHIVE_MIN=10080
THREAD_TITLE_MAX=64
# Backlog (upload files created while bot was offline)
PROCESS_BACKLOG_ON_START=true
# Logging
LOG_DIR=C:\bots\sdr-discord-uploader
PAUSE_ON_EXIT=true
# Transcription
TRANSCRIBE=true
WHISPER_MODEL=small.en
WHISPER_DEVICE=cpu
WHISPER_COMPUTE_TYPE=int8
WHISPER_THREADS=4
WHISPER_LANG=en
WHISPER_VAD=true
WHISPER_BEAM_SIZE=5
WHISPER_TEMPERATURE=0
TRANSCRIPT_FIELD_MAX=4096
TRANSCRIPT_MAX_FOLLOWUPS=5
# (Optional) Put model cache in a permanent folder
# WHISPER_CACHE_DIR=C:\bots\hf-cache
# Glossary & corrections (seed with your terms)
STT_GLOSSARY=Words, Commonly, Used, Here
STT_CORRECTIONS=four pigs->Four Peaks; sky net->SKYNET2; eear->E R; ed->ED; walkman->Welcome In; he's water->You as well sir; 4-8-0-2->Go, 802; 4 8 0 2->Go, 802; 4802->802
# Optional pre-processing (denoise + band-limit + normalize)
# STT_PREPROCESS=trueThe Script
The bot script has quite a few changes from the previous version, here are the highlights:
-
Watches
WATCH_FOLDER(non-recursive). -
Parses SDRTrunk filename + ID3 to extract date/time/system/site/TG/source.
-
Resolves destination text channel by system (using
AUDIOcategory or explicit overrides). -
Creates/uses public thread per TG (e.g.,
tg-111 Security). -
Builds a Discord embed with metadata and inline transcription (overflow as follow-ups).
-
Rotating log at
LOG_DIR\_uploader.log(5 MB × 3 files). -
Backlog processing on startup.
# sdrtrunk_discord_uploader.py
# ------------------------------------------------------------
# SDRTrunk -> Discord (Text Channels + per-TG Threads)
# Robust logging, ID3 + filename parsing, inline transcription with accuracy tweaks.
# ------------------------------------------------------------
import asyncio
import os
import re
import sqlite3
import sys
import time
import logging
from logging.handlers import RotatingFileHandler
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional, Dict, Any, Tuple
# --- SAFE import for faster-whisper (won't crash if missing) ---
HAVE_WHISPER = False
_IMPORT_ERR = None
try:
from faster_whisper import WhisperModel # type: ignore
HAVE_WHISPER = True
except Exception as e:
WhisperModel = None # type: ignore
_IMPORT_ERR = e
import discord
from discord import Embed, File
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
from mutagen import File as MutagenFile
from dotenv import load_dotenv
# -------------------- Load env --------------------
load_dotenv()
DISCORD_TOKEN = os.getenv("DISCORD_TOKEN", "").strip()
DEFAULT_CHANNEL_ID = int(os.getenv("DEFAULT_CHANNEL_ID", "0"))
WATCH_FOLDER = Path(os.getenv("WATCH_FOLDER", r"C:\Users\-YourUser-\OneDrive\Scanner Recordings\SDRTrunk"))
AUDIO_CATEGORY_NAME = os.getenv("AUDIO_CATEGORY_NAME", "AUDIO").strip()
ENABLE_THREADS = (os.getenv("ENABLE_THREADS", "true").lower() == "true")
THREAD_AUTO_ARCHIVE_MIN = int(os.getenv("THREAD_AUTO_ARCHIVE_MIN", "10080"))
THREAD_TITLE_MAX = int(os.getenv("THREAD_TITLE_MAX", "64"))
PROCESS_BACKLOG_ON_START = (os.getenv("PROCESS_BACKLOG_ON_START", "true").lower() == "true")
PAUSE_ON_EXIT = (os.getenv("PAUSE_ON_EXIT", "true").lower() == "true")
ALLOWED_EXT = {".mp3", ".wav", ".m4a", ".flac", ".ogg"}
# DB lives in the watch folder
DB_PATH = WATCH_FOLDER / "_uploaded.sqlite"
OVERRIDES_RAW = os.getenv("SYSTEM_TO_CHANNEL_OVERRIDES", "").strip()
TEMP_PREFIXES = {"~", "."}
TEMP_SUFFIXES = {".tmp", ".part", ".partial", ".crdownload"}
STABILITY_INTERVAL_SEC = 1.0
STABILITY_PASSES = 3
# Transcription toggles
TRANSCRIBE = os.getenv("TRANSCRIBE", "true").lower() == "true"
TRANSCRIPT_MAX_FOLLOWUPS = int(os.getenv("TRANSCRIPT_MAX_FOLLOWUPS", "5")) # 0 = unlimited
# Whisper tuning
WHISPER_MODEL_NAME = os.getenv("WHISPER_MODEL", "base.en")
WHISPER_DEVICE = os.getenv("WHISPER_DEVICE", "cpu")
WHISPER_COMPUTE_TYPE = os.getenv("WHISPER_COMPUTE_TYPE", "int8" if WHISPER_DEVICE == "cpu" else "float16")
WHISPER_THREADS = int(os.getenv("WHISPER_THREADS", "4"))
WHISPER_LANG = os.getenv("WHISPER_LANG") or "en"
WHISPER_VAD = os.getenv("WHISPER_VAD", "true").lower() == "true"
WHISPER_BEAM_SIZE = int(os.getenv("WHISPER_BEAM_SIZE", "5"))
WHISPER_TEMPERATURE = float(os.getenv("WHISPER_TEMPERATURE", "0"))
STT_PREPROCESS = os.getenv("STT_PREPROCESS", "false").lower() == "true" # optional ffmpeg pass
# -------------------- Logging (robust) --------------------
def _setup_logging() -> logging.Logger:
logger = logging.getLogger("sdrtrunk_uploader")
logger.setLevel(logging.INFO)
fmt = logging.Formatter("%(asctime)s [%(levelname)s] %(message)s")
ch = logging.StreamHandler(stream=sys.stdout)
ch.setFormatter(fmt)
logger.addHandler(ch)
log_dir_env = os.getenv("LOG_DIR", str(WATCH_FOLDER))
try:
log_dir = Path(log_dir_env)
log_dir.mkdir(parents=True, exist_ok=True)
log_path = log_dir / "_uploader.log"
fh = RotatingFileHandler(log_path, maxBytes=5_000_000, backupCount=3, encoding="utf-8")
fh.setFormatter(fmt)
logger.addHandler(fh)
logger.info(f"File logging to: {log_path}")
except Exception as e:
logger.warning(f"File logging disabled (could not open log file): {e}")
return logger
logger = _setup_logging()
logger.info(f"Python exe: {sys.executable}")
logger.info(f"Python ver: {sys.version.splitlines()[0]}")
if TRANSCRIBE and not HAVE_WHISPER:
logger.warning(f"Transcription disabled: faster-whisper not available ({_IMPORT_ERR}). "
f"Install with: py -m pip install -U faster-whisper ffmpeg-python")
# Ensure watch folder exists early
try:
WATCH_FOLDER.mkdir(parents=True, exist_ok=True)
except Exception as e:
logger.warning(f"Could not create WATCH_FOLDER '{WATCH_FOLDER}': {e}")
# -------------------- Helpers --------------------
def pause_if_needed():
if PAUSE_ON_EXIT:
try:
input("\nPress Enter to exit...")
except Exception:
pass
def slugify(s: str) -> str:
s = s.strip().lower()
s = re.sub(r"[^a-z0-9]+", "-", s)
s = re.sub(r"-+", "-", s).strip("-")
return s
def parse_overrides(raw: str) -> Dict[str, str]:
out: Dict[str, str] = {}
if not raw:
return out
for part in raw.split(","):
if "->" in part:
sys_name, ch_name = part.split("->", 1)
out[sys_name.strip()] = ch_name.strip()
return out
SYSTEM_TO_CHANNEL_OVERRIDES = parse_overrides(OVERRIDES_RAW)
def ensure_db() -> sqlite3.Connection:
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(DB_PATH)
conn.execute("""
CREATE TABLE IF NOT EXISTS uploaded (
path TEXT PRIMARY KEY,
mtime REAL NOT NULL,
size INTEGER NOT NULL,
uploaded_at REAL
)
""")
conn.execute("""
CREATE TABLE IF NOT EXISTS threads (
thread_key TEXT PRIMARY KEY, -- "<channel_id>:tg:<tgid>"
thread_id INTEGER NOT NULL,
name TEXT,
created_at REAL NOT NULL
)
""")
conn.commit()
return conn
def already_uploaded(conn: sqlite3.Connection, path: Path) -> bool:
cur = conn.execute("SELECT 1 FROM uploaded WHERE path=?", (str(path),))
return cur.fetchone() is not None
def mark_uploaded(conn: sqlite3.Connection, path: Path):
try:
st = path.stat()
except FileNotFoundError:
return
conn.execute(
"INSERT OR REPLACE INTO uploaded (path, mtime, size, uploaded_at) VALUES (?, ?, ?, ?)",
(str(path), st.st_mtime, st.st_size, time.time())
)
conn.commit()
def get_cached_thread_id(conn: sqlite3.Connection, key: str) -> Optional[int]:
cur = conn.execute("SELECT thread_id FROM threads WHERE thread_key=?", (key,))
row = cur.fetchone()
return int(row[0]) if row else None
def cache_thread_id(conn: sqlite3.Connection, key: str, thread_id: int, name: str):
conn.execute(
"INSERT OR REPLACE INTO threads (thread_key, thread_id, name, created_at) VALUES (?, ?, ?, ?)",
(key, thread_id, name, time.time())
)
conn.commit()
def delete_thread_cache(conn: sqlite3.Connection, key: str):
conn.execute("DELETE FROM threads WHERE thread_key=?", (key,))
conn.commit()
def channel_key_for_tg(channel_id: int, tgid: str) -> str:
return f"{channel_id}:tg:{tgid}"
# -------------------- STT accuracy helpers --------------------
def _parse_corrections(raw: str) -> Dict[str, str]:
mapping: Dict[str, str] = {}
for pair in (raw or "").split(";"):
if "->" in pair:
src, dst = pair.split("->", 1)
src = src.strip()
dst = dst.strip()
if src and dst:
mapping[src.lower()] = dst
return mapping
def apply_corrections(text: str) -> str:
mapping = _parse_corrections(os.getenv("STT_CORRECTIONS", ""))
for src, dst in mapping.items():
text = re.sub(rf"\b{re.escape(src)}\b", dst, text, flags=re.IGNORECASE)
return text
def build_initial_prompt(system_name: str, parsed: Dict[str, Any], tags: Dict[str, str]) -> Optional[str]:
terms = set()
for v in [system_name, parsed.get("site"), tags.get("album"), tags.get("grouping")]:
if v:
terms.update(re.split(r"[\s_\-]+", str(v)))
title_fixed = (tags.get("title") or "")
if title_fixed:
terms.update(re.split(r"[\s_\-]+", title_fixed))
extra = [w.strip() for w in os.getenv("STT_GLOSSARY", "").split(",") if w.strip()]
terms.update(extra)
terms = [t for t in sorted({t for t in terms if t and t.isascii()}) if t.lower() not in {"scanner", "audio"}]
if not terms:
return None
return "Proper nouns and radio terms you may hear: " + ", ".join(terms) + "."
# Optional: ffmpeg pre-processing
def preprocess_audio(in_path: Path) -> Path:
"""Denoise + bandlimit + normalize -> 16k mono wav in temp folder."""
import tempfile
try:
from ffmpeg import input as ffmpeg_input # type: ignore
except Exception as e:
logger.warning(f"ffmpeg-python not available ({e}); skipping pre-process.")
return in_path
out = Path(tempfile.gettempdir()) / (in_path.stem + ".proc.wav")
(
ffmpeg_input(str(in_path))
.audio
.filter("highpass", f=200)
.filter("lowpass", f=3500)
.filter("dynaudnorm")
.output(str(out), ac=1, ar=16000, format="wav", loglevel="error")
.overwrite_output()
.run()
)
return out
# Whisper model singleton
_WHISPER: Optional["WhisperModel"] = None
def get_whisper_model() -> Optional["WhisperModel"]:
global _WHISPER
if not TRANSCRIBE or not HAVE_WHISPER:
return None
if _WHISPER is not None:
return _WHISPER
_WHISPER = WhisperModel(
WHISPER_MODEL_NAME,
device=WHISPER_DEVICE,
compute_type=WHISPER_COMPUTE_TYPE,
num_workers=WHISPER_THREADS
) # type: ignore
logger.info(f"Whisper model loaded: {WHISPER_MODEL_NAME} (device={WHISPER_DEVICE}, compute_type={WHISPER_COMPUTE_TYPE}, workers={WHISPER_THREADS})")
return _WHISPER
def transcribe_clip_sync(audio_path: Path, initial_prompt: Optional[str] = None) -> str:
"""Return a single-line transcript string (may be empty on failure)."""
try:
model = get_whisper_model()
if model is None:
return ""
# Optional pre-processing
src = audio_path
if STT_PREPROCESS:
try:
src = preprocess_audio(audio_path)
except Exception as e:
logger.warning(f"Preprocess failed for {audio_path.name}: {e}")
src = audio_path
segments, _info = model.transcribe( # type: ignore
str(src),
language=WHISPER_LANG,
vad_filter=WHISPER_VAD,
vad_parameters=dict(min_silence_duration_ms=200),
beam_size=WHISPER_BEAM_SIZE,
temperature=WHISPER_TEMPERATURE,
condition_on_previous_text=False,
initial_prompt=initial_prompt,
)
parts = [seg.text.strip() for seg in segments if getattr(seg, "text", "").strip()]
text = " ".join(parts).strip()
text = apply_corrections(text)
return text
except Exception as e:
logger.warning(f"Transcription failed for {audio_path.name}: {e}")
return ""
# -------------------- File parsing --------------------
def is_stable(path: Path) -> bool:
try:
prev = None
stable_count = 0
time.sleep(0.25)
for _ in range(60): # ~60s cap
size = path.stat().st_size
if prev is None:
prev = size
stable_count = 1
else:
if size == prev:
stable_count += 1
else:
prev = size
stable_count = 1
if stable_count >= STABILITY_PASSES:
return True
time.sleep(STABILITY_INTERVAL_SEC)
return False
except FileNotFoundError:
return False
def parse_filename(fname: str) -> Dict[str, Any]:
stem = Path(fname).stem
if "_TO_" not in stem or "_FROM_" not in stem or "_" not in stem:
return {}
left, source = stem.rsplit("_FROM_", 1)
source = source.strip()
left2, dest = left.rsplit("_TO_", 1)
dest = dest.strip()
if "_" not in left2:
return {}
date_part, rest = left2.split("_", 1)
time_part = rest[:6] if len(rest) >= 6 else ""
remainder = rest[6:].strip("_")
system = ""
site = ""
if remainder:
parts = remainder.split("_")
if parts:
system = parts[0]
site_tokens = parts[1:]
site = " ".join([p for p in site_tokens if p]).replace("-", " ").replace("__", " ").strip()
dt = None
if len(date_part) == 8 and len(time_part) == 6:
try:
dt = datetime.strptime(f"{date_part}{time_part}", "%Y%m%d%H%M%S")
except ValueError:
pass
return {
"date": date_part or "",
"time": time_part or "",
"datetime": dt,
"system": system,
"site": site,
"talkgroup": dest,
"source_radio": source,
}
def read_tags(path: Path) -> Dict[str, str]:
out: Dict[str, str] = {}
try:
mf = MutagenFile(path)
if not mf or not mf.tags:
return out
def pick(*keys: str) -> str:
for k in keys:
if k in mf.tags:
v = mf.tags.get(k)
if isinstance(v, list):
v = v[0] if v else ""
return str(v)
return ""
out["title"] = pick("TIT2", "title") # e.g., "111'Operations'"
out["artist"] = pick("TPE1", "artist") # "999 Dispatcher"
out["album"] = pick("TALB", "album") # "SN2"
out["genre"] = pick("TCON", "genre") # "Scanner Audio"
out["composer"]= pick("TCOM", "composer") # "sdrtrunk v0.6.1"
out["date"] = pick("TDRC", "date") # "2025-08-16 15:10:31"
out["grouping"]= pick("TIT1", "grouping") # "SKYNET2"
out["comment"] = pick("COMM::XXX", "COMM", "comment")
except Exception as e:
logger.warning(f"Mutagen read failed for {path}: {e}")
return out
def fix_title_alias(s: str) -> str:
s = (s or "").strip()
if not s: return s
m = re.match(r"^\s*(\d+)\s*['\"]\s*(.*?)\s*['\"]\s*$", s)
if m: return f"{m.group(1)} {m.group(2)}".strip()
m = re.match(r"^\s*(\d+)\s*[:\-_/]\s*(.+)$", s)
if m: return f"{m.group(1)} {m.group(2)}".strip()
return s
def extract_tgid_and_alias(title_fixed: str) -> Tuple[Optional[str], Optional[str]]:
s = (title_fixed or "").replace("TG ", "").strip()
m = re.match(r"^\s*(\d+)\s+(.*)$", s)
if m: return m.group(1), (m.group(2).strip() or None)
if re.fullmatch(r"\d+", s): return s, None
return None, (s or None)
# -------------------- Watchdog --------------------
class NewFileHandler(FileSystemEventHandler):
def __init__(self, loop: asyncio.AbstractEventLoop, queue: asyncio.Queue):
self.loop = loop
self.queue = queue
def on_created(self, event):
if event.is_directory:
return
path = Path(event.src_path)
if path.suffix.lower() not in ALLOWED_EXT:
return
if any(path.name.startswith(p) for p in TEMP_PREFIXES):
return
if any(path.name.endswith(suf) for suf in TEMP_SUFFIXES):
return
self.loop.call_soon_threadsafe(self.queue.put_nowait, path)
# -------------------- Discord helpers --------------------
async def get_main_guild_and_category(client: discord.Client):
guild = None
category = None
if DEFAULT_CHANNEL_ID:
try:
ch = await client.fetch_channel(DEFAULT_CHANNEL_ID)
if hasattr(ch, "guild") and ch.guild:
guild = ch.guild
except Exception:
pass
if guild is None and client.guilds:
guild = client.guilds[0]
if guild:
for cat in guild.categories:
if cat.name.strip().lower() == AUDIO_CATEGORY_NAME.lower():
category = cat
break
return guild, category
def match_text_channel_for_system(system: str, category: Optional[discord.CategoryChannel]) -> Optional[discord.TextChannel]:
if not category or not system:
return None
desired = slugify(system)
for ch in category.channels:
if isinstance(ch, discord.TextChannel):
name = ch.name.lower()
name_no_prefix = re.sub(r"^a\d{2}-", "", name) # a01-honorhealth -> honorhealth
if name_no_prefix == desired:
return ch
return None
async def resolve_destination_text_channel(client: discord.Client, system: str) -> Optional[discord.TextChannel]:
guild, category = await get_main_guild_and_category(client)
# overrides by name -> exact channel name in guild
override_name = SYSTEM_TO_CHANNEL_OVERRIDES.get(system)
if override_name and guild:
ch = discord.utils.get(guild.channels, name=override_name)
if isinstance(ch, discord.TextChannel):
return ch
# match in AUDIO category by slug
ch = match_text_channel_for_system(system, category)
if ch:
return ch
# fallback to DEFAULT_CHANNEL_ID
if DEFAULT_CHANNEL_ID:
try:
fch = await client.fetch_channel(DEFAULT_CHANNEL_ID)
if isinstance(fch, discord.TextChannel):
return fch
except Exception:
pass
return None
def build_thread_name(tgid: str, alias: Optional[str], max_len: int) -> str:
tgid_clean = "".join(ch for ch in (tgid or "") if ch.isdigit()) or (tgid or "?")
alias = (alias or "").strip().replace("\n", " ")
if alias and len(alias) > max_len:
alias = alias[:max_len - 1] + "…"
base = f"tg-{tgid_clean}"
return f"{base} {alias}".strip()
async def get_or_create_tg_thread(
client: discord.Client,
conn: sqlite3.Connection,
parent: discord.TextChannel,
tgid: str,
alias: Optional[str]
) -> discord.abc.Messageable:
key = channel_key_for_tg(parent.id, tgid)
cached_id = get_cached_thread_id(conn, key)
if cached_id:
try:
ch = await client.fetch_channel(cached_id)
if isinstance(ch, discord.Thread) and ch.parent_id == parent.id:
return ch
delete_thread_cache(conn, key)
except discord.NotFound:
delete_thread_cache(conn, key)
except Exception:
pass
name = build_thread_name(tgid, alias, THREAD_TITLE_MAX)
try:
thread = await parent.create_thread(
name=name,
auto_archive_duration=THREAD_AUTO_ARCHIVE_MIN,
type=discord.ChannelType.public_thread
)
cache_thread_id(conn, key, thread.id, name)
return thread
except (discord.Forbidden, discord.HTTPException, AttributeError) as e:
logger.error(f"Thread create failed for TG {tgid} in #{parent}: {e}")
try:
seed = await parent.send(f"Auto-creating thread for TG {tgid}…")
thread = await seed.create_thread(
name=name,
auto_archive_duration=THREAD_AUTO_ARCHIVE_MIN
)
try:
await seed.delete()
except Exception:
pass
cache_thread_id(conn, key, thread.id, name)
return thread
except Exception as ee:
logger.error(f"Seed thread create failed for TG {tgid}: {ee}")
return parent # fallback: post into parent channel
# -------------------- Embed builder --------------------
def build_embed(path: Path, parsed: Dict[str, Any], tags: Dict[str, str]) -> Embed:
title_raw = tags.get("title") or ""
title_fixed = fix_title_alias(title_raw)
tgid_from_title, alias_from_title = extract_tgid_and_alias(title_fixed)
if tgid_from_title:
title = f"{tgid_from_title} {alias_from_title or ''}".strip()
elif parsed.get("talkgroup"):
title = f"{parsed['talkgroup']} {(alias_from_title or '')}".strip()
else:
title = path.stem
embed = Embed(title=title, description="Scanner clip", timestamp=datetime.now(timezone.utc))
if parsed.get("system"):
embed.add_field(name="System", value=parsed["system"], inline=True)
if parsed.get("site"):
embed.add_field(name="Site", value=parsed["site"], inline=True)
if parsed.get("talkgroup"):
embed.add_field(name="Talkgroup", value=parsed["talkgroup"], inline=True)
if parsed.get("source_radio"):
embed.add_field(name="Source", value=parsed["source_radio"], inline=True)
if parsed.get("datetime"):
embed.add_field(name="Timestamp", value=parsed["datetime"].strftime("%Y-%m-%d %H:%M:%S"), inline=True)
md_lines = []
label_map = [
("TX ID", "artist"),
("Site", "album"),
("Genre", "genre"),
("Software", "composer"),
("System", "grouping"),
("Tag Date", "date"),
("Comment", "comment"),
]
for label, key in label_map:
if tags.get(key):
md_lines.append(f"**{label}:** {tags[key]}")
if md_lines:
embed.add_field(name="Metadata", value="\n".join(md_lines), inline=False)
embed.set_footer(text=path.name)
return embed
# -------------------- Worker --------------------
async def upload_worker(queue: asyncio.Queue, client: discord.Client, conn: sqlite3.Connection):
await client.wait_until_ready()
logger.info("Uploader worker ready.")
while True:
path: Path = await queue.get()
try:
await asyncio.sleep(0.25)
if not is_stable(path):
logger.debug(f"Skipping unstable file: {path}")
queue.task_done(); continue
if already_uploaded(conn, path):
logger.debug(f"Already uploaded, skipping: {path}")
queue.task_done(); continue
parsed = parse_filename(path.name)
tags = read_tags(path)
system_name = tags.get("grouping") or parsed.get("system") or ""
parent = await resolve_destination_text_channel(client, system_name)
if parent is None:
logger.error(f"No text channel for system '{system_name}'. Check AUDIO category or overrides. Skipping {path.name}")
queue.task_done(); continue
title_fixed = fix_title_alias(tags.get("title") or "")
tgid_from_title, alias_from_title = extract_tgid_and_alias(title_fixed)
tgid = tgid_from_title or parsed.get("talkgroup") or "unknown"
alias = alias_from_title
# --- destination + embed ---
destination = await get_or_create_tg_thread(client, conn, parent, tgid, alias)
embed = build_embed(path, parsed, tags)
# --- transcribe (off the event loop) ---
transcript_text = ""
if TRANSCRIBE:
prompt = build_initial_prompt(system_name, parsed, tags)
transcript_text = await asyncio.to_thread(transcribe_clip_sync, path, prompt)
# --- pack transcript into embed description (≈4096 cap) ---
msg = None
first = ""
if transcript_text:
base_desc = embed.description or "Scanner clip"
header = "\n\n**Transcription**\n"
BUDGET = max(0, 4000 - len(base_desc) - len(header)) # safety margin
first = transcript_text[:BUDGET]
embed.description = base_desc + header + first
# --- send audio + embed (one post) ---
try:
msg = await destination.send(embed=embed, file=File(str(path)))
logger.info(f"Posted: {path.name} -> #{parent.name} / {getattr(destination, 'name', 'channel')} (transcript={'yes' if transcript_text else 'no'})")
except discord.HTTPException as e:
logger.warning(f"Embed send failed ({e}); sending as text+file: {path.name}")
msg = await destination.send(content=(embed.title or path.stem), file=File(str(path)))
# --- follow-ups for overflow (optional) ---
if transcript_text:
rest = transcript_text[len(first):]
if rest:
CHUNK = 1900 # under message cap
i = 0
while rest and (TRANSCRIPT_MAX_FOLLOWUPS == 0 or i < TRANSCRIPT_MAX_FOLLOWUPS):
part, rest = rest[:CHUNK], rest[CHUNK:]
await destination.send(part, reference=msg)
i += 1
if rest:
await destination.send("…(transcript truncated; exceeded follow-up limit)", reference=msg)
mark_uploaded(conn, path)
await asyncio.sleep(0.3)
except Exception as e:
logger.exception(f"Failed to upload {path}: {e}")
finally:
queue.task_done()
# -------------------- Client --------------------
class SDRUploader(discord.Client):
def __init__(self, conn: sqlite3.Connection, **kwargs):
super().__init__(**kwargs)
self.conn = conn
self.queue: asyncio.Queue[Path] = asyncio.Queue()
self.observer: Optional[Observer] = None
async def setup_hook(self) -> None:
self.loop.create_task(upload_worker(self.queue, self, self.conn))
async def on_ready(self):
logger.info(f"Logged in as {self.user} ({self.user.id})")
handler = NewFileHandler(self.loop, self.queue)
self.observer = Observer()
self.observer.schedule(handler, str(WATCH_FOLDER), recursive=False)
self.observer.start()
logger.info(f"Watching: {WATCH_FOLDER}")
if PROCESS_BACKLOG_ON_START:
candidates = [p for p in WATCH_FOLDER.glob("*") if p.suffix.lower() in ALLOWED_EXT]
candidates.sort(key=lambda p: (p.stat().st_mtime, p.name))
enqueued = 0
for path in candidates:
if any(path.name.startswith(pfx) for pfx in TEMP_PREFIXES):
continue
if any(path.name.endswith(suf) for suf in TEMP_SUFFIXES):
continue
if not already_uploaded(self.conn, path):
await self.queue.put(path); enqueued += 1
logger.info(f"Backlog enqueued: {enqueued} file(s)")
async def close(self):
if self.observer:
self.observer.stop()
self.observer.join()
await super().close()
# -------------------- Entrypoint --------------------
def validate_env() -> Optional[str]:
if not DISCORD_TOKEN:
return "DISCORD_TOKEN missing. Put it in .env next to this script."
if not WATCH_FOLDER.exists():
return f"WATCH_FOLDER does not exist: {WATCH_FOLDER}"
return None
def main():
err = validate_env()
if err:
logger.error(err)
pause_if_needed()
return
try:
conn = ensure_db()
intents = discord.Intents.default()
client = SDRUploader(conn=conn, intents=intents)
client.run(DISCORD_TOKEN)
except Exception as e:
logger.exception(f"Fatal error: {e}")
pause_if_needed()
if __name__ == "__main__":
main()Test Run
From PowerShell:
cd "C:\bots\sdr-discord-uploader"
& "C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe" .\sdrtrunk_discord_uploader.pyWhat you should see in the console/log:
- File logging to: C:\bots\sdr-discord-uploader\_uploader.log
- Logged in as <YourBot>
- Watching: <path>
- Backlog enqueued: N file(s)
- The first time transcription runs: model download info
- For each post: Posted: <file> -> #channel / tg-... (transcript=yes|no)
Start On Boot
Create a BAT file in the working directory:
@echo off
setlocal
set "PY=C:\Users\-YourUser-\AppData\Local\Programs\Python\Python313\python.exe"
set "DIR=C:\bots\sdr-discord-uploader"
cd /d "%DIR%"
echo [%date% %time%] starting
"%PY%" -u "%DIR%\sdrtrunk_discord_uploader.py"
echo [%date% %time%] exited with code %errorlevel%
pauseCreate a shortcut of the .bat file and place it in the startup folder
shell:startup