mlboydaisuke commited on
Commit
4b2475b
Β·
verified Β·
1 Parent(s): f416e24

gen-cards: regenerate Use-it block

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -23,6 +23,45 @@ Part of the community Core AI model zoo: **https://github.com/john-rocky/coreai-
23
  (full card: [`zoo/glm-4.7-flash.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/glm-4.7-flash.md)).
24
 
25
  <!-- gen-cards:use-it begin id=glm-4.7-flash (managed by scripts/gen-cards β€” edit cards.json / QuickStart.swift, not this block) -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  <!-- gen-cards:use-it end -->
27
 
28
  ## The `gather_qmm` kernel β€” 20.3 β†’ 52.4 tok/s (2.6Γ—)
 
23
  (full card: [`zoo/glm-4.7-flash.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/glm-4.7-flash.md)).
24
 
25
  <!-- gen-cards:use-it begin id=glm-4.7-flash (managed by scripts/gen-cards β€” edit cards.json / QuickStart.swift, not this block) -->
26
+ ## Use it
27
+
28
+ ▢️ **Run it (source)** β€” the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ChatDemo)
29
+ (GUI + CLI, one app for every chat model in the catalog):
30
+
31
+ ```bash
32
+ git clone https://github.com/john-rocky/coreai-kit
33
+ open coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
34
+ # β†’ Run, then pick "GLM-4.7-Flash (MoE+MLA)" in the model picker
35
+
36
+ # agents / headless (macOS):
37
+ cd coreai-kit/Examples/ChatDemo
38
+ swift run chat-cli --model glm-4.7-flash --prompt "What can you do, offline?"
39
+ ```
40
+
41
+ πŸ’» **Build with it** β€” complete; the glue is kit API, copy-paste runs:
42
+
43
+ ```swift
44
+ import CoreAIKit
45
+
46
+ let chat = try await ChatSession(catalog: "glm-4.7-flash")
47
+ let reply = try await chat.respond(to: prompt)
48
+ // reply: the answer, generated fully on-device
49
+ ```
50
+
51
+ The take-home is [`Examples/ChatDemo/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/ChatDemo/Sources/QuickStart.swift)
52
+ β€” this exact code as one typed function, no UI; the CLI is an argument shell over it, and
53
+ the GUI drives the same `ChatSession` across turns for its transcript.
54
+ Multi-turn? Hold the `ChatSession` and call `respond(to:)` per turn β€” it keeps the
55
+ conversation history; `streamResponse(to:)` yields tokens as they decode.
56
+
57
+ **Integration checklist**
58
+
59
+ - SPM: `https://github.com/john-rocky/coreai-kit` β†’ product **CoreAIKit**
60
+ - Info.plist: none needed
61
+ - Entitlements: none needed (macOS)
62
+ - First run downloads the model β€” 30.0 GB (Mac) β€” then it loads from the
63
+ local cache (Application Support; progress via the `downloadProgress` callback)
64
+ - Measure in Release β€” Debug is ~3Γ— slower on per-token host work
65
  <!-- gen-cards:use-it end -->
66
 
67
  ## The `gather_qmm` kernel β€” 20.3 β†’ 52.4 tok/s (2.6Γ—)