Add home-tunnel proxy control panel (tools/)

Tkinter GUI to run a local authenticated HTTP proxy on the user's residential IP, expose
it via a raw TCP tunnel (ngrok / bore / manual), test the egress IP + YouTube reach, and
one-click set the Space's YT_PROXY secret. Includes a step-by-step README. (Forward
proxies need a TCP tunnel; HTTP reverse tunnels like Cloudflare/Tailscale Funnel won't
carry CONNECT.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

tools/README.md

@@ -0,0 +1,90 @@
+# Home Proxy Panel — give the Space your home IP
+
+The TutorialMaker Space can't fetch transcripts or stream URLs from its datacenter IP
+(YouTube blocks it). This tool runs a small **authenticated HTTP proxy on your computer**
+(your home/residential IP) and exposes it through a **raw TCP tunnel**, then sets it as the
+Space's **`YT_PROXY`** secret. The Space's YouTube requests then exit from *your* IP and
+stop getting blocked.
+
+> **Why a TCP tunnel (not Cloudflare/Tailscale):** a forward proxy speaks HTTP `CONNECT`,
+> which only survives a transparent **TCP** tunnel. HTTP *reverse* tunnels (Cloudflare
+> quick-tunnel, Tailscale Funnel) cannot carry it — use **ngrok TCP** or **bore**.
+
+---
+
+## 1. Install the Python deps
+
+```bash
+pip install -r tools/requirements.txt
+```
+
+## 2. Install a tunnel (pick ONE)
+
+**Option A — ngrok (more reliable, free account):**
+1. Sign up at <https://ngrok.com> (free).
+2. Install the ngrok agent and put `ngrok` on your PATH
+   (<https://ngrok.com/download>).
+3. Copy your authtoken from the ngrok dashboard → you'll paste it into the panel.
+
+**Option B — bore (no account, less reliable):**
+1. Download `bore.exe` from <https://github.com/ekzhang/bore/releases> and put it on your
+   PATH (or in this folder).
+2. That's it — it uses the shared public `bore.pub` server.
+
+## 3. Launch the panel
+
+```bash
+python tools/home_proxy_panel.py
+```
+
+## 4. Use it (in order)
+
+1. **Proxy port / user / password** are pre-filled (a random password is generated). Keep
+   the password — it protects your public proxy from being used by strangers.
+2. Pick the **Tunnel provider** (`ngrok`, `bore`, or `manual`). For ngrok, paste your
+   **authtoken**.
+3. Click **Start proxy** → it runs `proxy.py` on `127.0.0.1:<port>`.
+4. Click **Start tunnel** → the **Public endpoint** field fills in
+   (`x.tcp.ngrok.io:NNNN` or `bore.pub:NNNN`) and the **YT_PROXY URL** is computed.
+5. Click **Test proxy** → in the log, the **egress IP should be your home IP** and YouTube
+   should be **reachable**. If so, the proxy works.
+6. Click **Set YT_PROXY on Space** → it writes the `YT_PROXY` secret to your Space
+   (auto-detecting your HF token, or paste one). The Space restarts and picks it up.
+   - Or click **Copy YT_PROXY URL** and paste it into the Space's secret manually.
+
+## 5. Verify on the Space
+
+Run a topic on the Space. The **transcript** and **screenshot** steps should now succeed
+instead of failing with an SSL/TLS block.
+
+## Keep it running
+
+The proxy + tunnel must stay running on your machine while the Space is in use. Close the
+panel (or **Stop proxy/Stop tunnel**) when you're done — the `YT_PROXY` secret will then
+point at a dead endpoint until you start it again (the Space falls back to text-only / a
+clear error).
+
+---
+
+## Security notes
+
+- The proxy is **public** while tunneled, so it **requires the username/password** — keep
+  them non-trivial (use **New password** to rotate). Without the right credentials nobody
+  can use your proxy.
+- This routes YouTube traffic through your home connection and consumes your bandwidth.
+- Stop the tunnel when not needed; rotate the password periodically.
+
+## Manual endpoint (router port-forward / other tunnel)
+
+If you forward a router port (and you're **not** behind CGNAT) or use another raw-TCP
+tunnel, choose provider **`manual`** and enter the public `host:port`. The panel will
+build the `YT_PROXY` URL and can set the secret + test it.
+
+## Troubleshooting
+
+- **`proxy.py` not found** → `pip install proxy.py`.
+- **ngrok: no tcp tunnel** → ensure the authtoken is set; free TCP tunnels are supported
+  but only one at a time — close other ngrok agents.
+- **Test egress IP is *not* your home IP** → the tunnel/proxy chain is wrong; restart both.
+- **YouTube still blocked on the Space** → confirm the secret value matches the panel's
+  YT_PROXY URL exactly (including `user:pass@`), and that the panel is still running.

tools/home_proxy_panel.py

@@ -0,0 +1,365 @@
+"""Home-tunnel proxy control panel for the TutorialMaker Space.
+
+Runs a local authenticated HTTP forward proxy on your machine (your residential IP) and
+exposes it to the internet through a RAW TCP tunnel (ngrok / bore), so the Hugging Face
+Space can route youtube-transcript-api + yt-dlp through your home IP and dodge the
+datacenter-IP block. One-click sets the resulting URL as the Space's YT_PROXY secret.
+
+Why a TCP tunnel: a forward proxy speaks HTTP CONNECT, which only survives a transparent
+TCP tunnel. HTTP reverse tunnels (Cloudflare quick-tunnel, Tailscale Funnel) do NOT work
+for this — don't use them here.
+
+Run:  python tools/home_proxy_panel.py
+Deps: pip install proxy.py huggingface_hub requests   (ngrok or bore on PATH for tunneling)
+"""
+from __future__ import annotations
+
+import json
+import os
+import queue
+import re
+import secrets
+import subprocess
+import sys
+import threading
+
+CONFIG_PATH = os.path.join(os.path.expanduser("~"), ".tutorialmaker_proxy.json")
+DEFAULT_REPO = "vivekchakraverty/TutorialMaker"
+_NO_WINDOW = 0x08000000 if os.name == "nt" else 0
+
+
+# ----------------------------------------------------------------- pure helpers (tested)
+def build_proxy_url(host: str, port, user: str, pw: str) -> str:
+    """Compose an http proxy URL, url-encoding the credentials."""
+    from urllib.parse import quote
+    auth = ""
+    if user or pw:
+        auth = f"{quote(user, safe='')}:{quote(pw, safe='')}@"
+    return f"http://{auth}{host}:{port}"
+
+
+def parse_ngrok_tunnels(json_text: str) -> str | None:
+    """Return 'host:port' from ngrok's /api/tunnels JSON (first tcp tunnel)."""
+    try:
+        data = json.loads(json_text)
+    except json.JSONDecodeError:
+        return None
+    for t in data.get("tunnels", []):
+        pub = t.get("public_url", "")
+        if pub.startswith("tcp://"):
+            return pub[len("tcp://"):]
+    return None
+
+
+def parse_bore_endpoint(text: str) -> str | None:
+    """Return 'bore.pub:PORT' parsed from bore's log output."""
+    m = re.search(r"(bore\.pub:\d+)", text)
+    if m:
+        return m.group(1)
+    m = re.search(r"listening at\s+([\w.-]+:\d+)", text)
+    return m.group(1) if m else None
+
+
+def hf_candidate_tokens() -> list[str]:
+    """HF tokens to try: cached token, then any huggingface.co git-credential."""
+    out, seen = [], set()
+    env = dict(os.environ)
+    env.pop("HF_TOKEN", None)
+    env.pop("HUGGING_FACE_HUB_TOKEN", None)
+    try:
+        from huggingface_hub import get_token
+        # get_token reads HF_TOKEN env first; temporarily clear it for the cached one
+        saved = os.environ.pop("HF_TOKEN", None)
+        try:
+            t = get_token()
+        finally:
+            if saved is not None:
+                os.environ["HF_TOKEN"] = saved
+        if t and t not in seen:
+            seen.add(t); out.append(t)
+    except Exception:
+        pass
+    cred = os.path.join(os.path.expanduser("~"), ".git-credentials")
+    if os.path.exists(cred):
+        with open(cred, encoding="utf-8", errors="ignore") as fh:
+            for line in fh:
+                m = re.search(r"https://[^:]*:([^@]+)@huggingface\.co", line)
+                if m and m.group(1) not in seen:
+                    seen.add(m.group(1)); out.append(m.group(1))
+    return out
+
+
+def set_space_secret(repo: str, key: str, value: str, token: str | None = None) -> str:
+    """Set a Space secret; return the authenticated username. Raises on failure."""
+    from huggingface_hub import HfApi
+    tokens = [token] if token else hf_candidate_tokens()
+    last = None
+    for tok in tokens:
+        try:
+            api = HfApi(token=tok)
+            who = api.whoami()["name"]
+            api.add_space_secret(repo_id=repo, key=key, value=value)
+            return who
+        except Exception as exc:
+            last = exc
+    raise RuntimeError(f"Could not set secret: {last}")
+
+
+def test_through_proxy(proxy_url: str) -> dict:
+    """Route a couple of requests through proxy_url; report egress IP + YouTube reach."""
+    import requests
+    proxies = {"http": proxy_url, "https": proxy_url}
+    out = {}
+    try:
+        out["egress_ip"] = requests.get("https://api.ipify.org", proxies=proxies,
+                                        timeout=25).text.strip()
+    except Exception as exc:
+        out["egress_ip"] = f"FAILED: {type(exc).__name__}: {str(exc)[:120]}"
+    try:
+        r = requests.get("https://www.youtube.com/robots.txt", proxies=proxies, timeout=25)
+        out["youtube"] = f"reachable (HTTP {r.status_code})"
+    except Exception as exc:
+        out["youtube"] = f"FAILED: {type(exc).__name__}: {str(exc)[:120]}"
+    return out
+
+
+# ------------------------------------------------------------------- subprocess manager
+class Proc:
+    """A subprocess whose stdout/stderr lines are streamed into a queue."""
+
+    def __init__(self, log_q: "queue.Queue", tag: str):
+        self.p = None
+        self.log_q = log_q
+        self.tag = tag
+
+    def start(self, cmd: list[str]):
+        self.stop()
+        self.log_q.put(f"[{self.tag}] $ " + " ".join(cmd))
+        self.p = subprocess.Popen(
+            cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+            text=True, bufsize=1, creationflags=_NO_WINDOW)
+        threading.Thread(target=self._pump, daemon=True).start()
+
+    def _pump(self):
+        for line in self.p.stdout:
+            self.log_q.put(f"[{self.tag}] " + line.rstrip())
+
+    def running(self) -> bool:
+        return self.p is not None and self.p.poll() is None
+
+    def stop(self):
+        if self.running():
+            try:
+                self.p.terminate()
+            except Exception:
+                pass
+        self.p = None
+
+
+# ---------------------------------------------------------------------------- GUI shell
+def main():
+    import tkinter as tk
+    from tkinter import ttk, messagebox
+
+    cfg = {}
+    if os.path.exists(CONFIG_PATH):
+        try:
+            cfg = json.load(open(CONFIG_PATH, encoding="utf-8"))
+        except Exception:
+            cfg = {}
+
+    root = tk.Tk()
+    root.title("TutorialMaker — Home Proxy Panel")
+    root.geometry("760x620")
+    log_q: "queue.Queue" = queue.Queue()
+    proxy_proc = Proc(log_q, "proxy")
+    tunnel_proc = Proc(log_q, "tunnel")
+    state = {"endpoint": None}
+
+    def v(name, default=""):
+        return tk.StringVar(value=str(cfg.get(name, default)))
+
+    port = v("port", "8899")
+    user = v("user", "tm")
+    pw = v("pw", secrets.token_urlsafe(10))
+    provider = v("provider", "ngrok")
+    ngrok_token = v("ngrok_token", "")
+    manual_ep = v("manual_ep", "")
+    repo = v("repo", DEFAULT_REPO)
+    hf_token = v("hf_token", "")
+    endpoint_var = tk.StringVar(value="(not running)")
+    yturl_var = tk.StringVar(value="")
+    status_var = tk.StringVar(value="idle")
+
+    def log(msg):
+        log_q.put(msg)
+
+    def save_cfg():
+        json.dump({"port": port.get(), "user": user.get(), "pw": pw.get(),
+                   "provider": provider.get(), "ngrok_token": ngrok_token.get(),
+                   "manual_ep": manual_ep.get(), "repo": repo.get()},
+                  open(CONFIG_PATH, "w", encoding="utf-8"))
+        log(f"[cfg] saved to {CONFIG_PATH}")
+
+    def compute_yturl():
+        ep = state["endpoint"]
+        if not ep:
+            yturl_var.set("")
+            return None
+        host, _, p = ep.partition(":")
+        url = build_proxy_url(host, p, user.get(), pw.get())
+        yturl_var.set(url)
+        return url
+
+    # --- proxy ---
+    def start_proxy():
+        cmd = [sys.executable, "-m", "proxy", "--hostname", "127.0.0.1",
+               "--port", port.get(), "--basic-auth", f"{user.get()}:{pw.get()}"]
+        proxy_proc.start(cmd)
+        status_var.set("proxy starting…")
+
+    def stop_proxy():
+        proxy_proc.stop(); log("[proxy] stopped")
+
+    # --- tunnel ---
+    def start_tunnel():
+        prov = provider.get()
+        if prov == "manual":
+            ep = manual_ep.get().strip()
+            if not ep:
+                messagebox.showwarning("Manual", "Enter host:port for the manual endpoint.")
+                return
+            state["endpoint"] = ep
+            endpoint_var.set(ep); compute_yturl(); log(f"[tunnel] manual endpoint {ep}")
+            return
+        if prov == "ngrok":
+            if ngrok_token.get().strip():
+                subprocess.run(["ngrok", "config", "add-authtoken", ngrok_token.get().strip()],
+                               creationflags=_NO_WINDOW)
+            tunnel_proc.start(["ngrok", "tcp", port.get(), "--log", "stdout"])
+            root.after(2500, _grab_ngrok)
+        elif prov == "bore":
+            tunnel_proc.start(["bore", "local", port.get(), "--to", "bore.pub"])
+        status_var.set("tunnel starting…")
+
+    def _grab_ngrok():
+        try:
+            import requests
+            j = requests.get("http://127.0.0.1:4040/api/tunnels", timeout=5).text
+            ep = parse_ngrok_tunnels(j)
+            if ep:
+                state["endpoint"] = ep; endpoint_var.set(ep); compute_yturl()
+                log(f"[tunnel] ngrok endpoint {ep}")
+            else:
+                log("[tunnel] ngrok: no tcp tunnel yet, retrying…"); root.after(2000, _grab_ngrok)
+        except Exception as exc:
+            log(f"[tunnel] ngrok api: {exc}"); root.after(2000, _grab_ngrok)
+
+    def stop_tunnel():
+        tunnel_proc.stop(); state["endpoint"] = None
+        endpoint_var.set("(not running)"); yturl_var.set(""); log("[tunnel] stopped")
+
+    # --- actions ---
+    def copy_url():
+        url = compute_yturl()
+        if url:
+            root.clipboard_clear(); root.clipboard_append(url); log("[yturl] copied")
+
+    def do_test():
+        url = compute_yturl()
+        if not url:
+            messagebox.showinfo("Test", "Start the tunnel first."); return
+        status_var.set("testing…")
+        def work():
+            res = test_through_proxy(url)
+            log(f"[test] egress IP: {res['egress_ip']}")
+            log(f"[test] youtube: {res['youtube']}")
+            status_var.set("test done")
+        threading.Thread(target=work, daemon=True).start()
+
+    def do_set_secret():
+        url = compute_yturl()
+        if not url:
+            messagebox.showinfo("Secret", "Start the tunnel first."); return
+        status_var.set("setting YT_PROXY…")
+        def work():
+            try:
+                who = set_space_secret(repo.get().strip(), "YT_PROXY", url,
+                                       hf_token.get().strip() or None)
+                log(f"[hf] YT_PROXY set on {repo.get()} (as {who})")
+                status_var.set("YT_PROXY set ✓")
+            except Exception as exc:
+                log(f"[hf] FAILED: {exc}"); status_var.set("secret failed")
+        threading.Thread(target=work, daemon=True).start()
+
+    # --- layout ---
+    pad = {"padx": 6, "pady": 3}
+    frm = ttk.Frame(root); frm.pack(fill="x", **pad)
+    r = 0
+    def row(label, var, show=None, width=42):
+        nonlocal r
+        ttk.Label(frm, text=label).grid(row=r, column=0, sticky="w", **pad)
+        e = ttk.Entry(frm, textvariable=var, width=width, show=show)
+        e.grid(row=r, column=1, columnspan=3, sticky="w", **pad); r += 1
+        return e
+
+    row("Proxy port", port, width=10)
+    row("Proxy user", user, width=18)
+    row("Proxy password", pw, width=24)
+    ttk.Label(frm, text="Tunnel provider").grid(row=r, column=0, sticky="w", **pad)
+    ttk.Combobox(frm, textvariable=provider, values=["ngrok", "bore", "manual"],
+                 width=10, state="readonly").grid(row=r, column=1, sticky="w", **pad); r += 1
+    row("ngrok authtoken (ngrok only)", ngrok_token, show="*")
+    row("Manual endpoint host:port (manual only)", manual_ep, width=28)
+    row("HF Space repo", repo, width=34)
+    row("HF token (blank = auto-detect)", hf_token, show="*")
+
+    btns = ttk.Frame(root); btns.pack(fill="x", **pad)
+    for txt, fn in [("Start proxy", start_proxy), ("Stop proxy", stop_proxy),
+                    ("Start tunnel", start_tunnel), ("Stop tunnel", stop_tunnel)]:
+        ttk.Button(btns, text=txt, command=fn).pack(side="left", **pad)
+
+    btns2 = ttk.Frame(root); btns2.pack(fill="x", **pad)
+    for txt, fn in [("Test proxy", do_test), ("Set YT_PROXY on Space", do_set_secret),
+                    ("Copy YT_PROXY URL", copy_url), ("Save config", save_cfg),
+                    ("New password", lambda: pw.set(secrets.token_urlsafe(10)))]:
+        ttk.Button(btns2, text=txt, command=fn).pack(side="left", **pad)
+
+    info = ttk.Frame(root); info.pack(fill="x", **pad)
+    ttk.Label(info, text="Public endpoint:").grid(row=0, column=0, sticky="w", **pad)
+    ttk.Label(info, textvariable=endpoint_var, foreground="#0a7").grid(row=0, column=1, sticky="w", **pad)
+    ttk.Label(info, text="YT_PROXY URL:").grid(row=1, column=0, sticky="w", **pad)
+    ttk.Entry(info, textvariable=yturl_var, width=70, state="readonly").grid(row=1, column=1, sticky="w", **pad)
+    ttk.Label(info, text="Status:").grid(row=2, column=0, sticky="w", **pad)
+    ttk.Label(info, textvariable=status_var, foreground="#06c").grid(row=2, column=1, sticky="w", **pad)
+
+    txt = tk.Text(root, height=16, wrap="word", bg="#111", fg="#ddd")
+    txt.pack(fill="both", expand=True, **pad)
+
+    def poll():
+        try:
+            while True:
+                line = log_q.get_nowait()
+                txt.insert("end", line + "\n"); txt.see("end")
+        except queue.Empty:
+            pass
+        # refresh running status
+        s = []
+        s.append("proxy:on" if proxy_proc.running() else "proxy:off")
+        s.append("tunnel:on" if tunnel_proc.running() else "tunnel:off")
+        if status_var.get() in ("idle", "proxy starting…", "tunnel starting…"):
+            status_var.set(" ".join(s))
+        root.after(400, poll)
+
+    def on_close():
+        proxy_proc.stop(); tunnel_proc.stop(); root.destroy()
+
+    root.protocol("WM_DELETE_WINDOW", on_close)
+    log("Ready. 1) Start proxy  2) Start tunnel  3) Test proxy  4) Set YT_PROXY on Space.")
+    log("Tip: 'Test proxy' egress IP should be your HOME IP, and YouTube should be reachable.")
+    poll()
+    root.mainloop()
+
+
+if __name__ == "__main__":
+    main()

tools/requirements.txt

@@ -0,0 +1,3 @@
+proxy.py>=2.4
+huggingface_hub>=0.27
+requests>=2.31