Upload folder using huggingface_hub
Browse files- .gitattributes +8 -0
- README.md +10 -7
- data/COCO_val2014_000000519046.jpg +0 -0
- data/test/0.wav +3 -0
- data/test/1.wav +3 -0
- data/test/COCO_val2014_000000166124.jpg +0 -0
- data/test/COCO_val2014_000000532867.jpg +0 -0
- data/test/COCO_val2014_000000534121.jpg +0 -0
- data/test/COCO_val2014_000000534733.jpg +0 -0
- data/test/COCO_val2014_000000535183.jpg +0 -0
- data/test/COCO_val2014_000000535292.jpg +0 -0
- data/test/COCO_val2014_000000535306.jpg +0 -0
- data/test/COCO_val2014_000000535578.jpg +0 -0
- data/test/COCO_val2014_000000536615.jpg +0 -0
- data/test/COCO_val2014_000000536791.jpg +0 -0
- data/test/COCO_val2014_000000538092.jpg +0 -0
- data/test/COCO_val2014_000000538451.jpg +0 -0
- data/test/COCO_val2014_000000539079.jpg +0 -0
- data/test/COCO_val2014_000000539124.jpg +0 -0
- data/test/COCO_val2014_000000539355.jpg +0 -0
- data/test/COCO_val2014_000000539784.jpg +0 -0
- data/test/COCO_val2014_000000540806.jpg +0 -0
- data/test/COCO_val2014_000000541965.jpg +0 -0
- data/test/COCO_val2014_000000542509.jpg +0 -0
- data/test/Model_Provisions_comprehensive_commercial_insurance_vehicles.pdf +3 -0
- data/test/sanguo.mp4 +3 -0
- data/test/sp.mp4 +3 -0
- data/test/test.mp4 +3 -0
- data/test/test.txt +2 -0
- data/test/国药准字H37020386_布洛芬片.pdf +3 -0
- docs/images/Multimodal_RAG.mp4 +3 -0
- docs/images/image-9.png +2 -2
.gitattributes
CHANGED
|
@@ -40,3 +40,11 @@ docs/images/image-6.png filter=lfs diff=lfs merge=lfs -text
|
|
| 40 |
docs/images/image-7.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
docs/images/image-8.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
docs/images/image-9.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
docs/images/image-7.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
docs/images/image-8.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
docs/images/image-9.png filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
data/test/0.wav filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
data/test/1.wav filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
data/test/Model_Provisions_comprehensive_commercial_insurance_vehicles.pdf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
data/test/sanguo.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
data/test/sp.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
data/test/test.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
data/test/国药准字H37020386_布洛芬片.pdf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
docs/images/Multimodal_RAG.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,10 +1,6 @@
|
|
| 1 |
# Multimodal RAG 智能问答系统(基于AX650N)
|
| 2 |
|
| 3 |
-
<
|
| 4 |
-
|
| 5 |
-
**端到端多模态检索增强生成** | **跨模态语义理解** | **AX650N 边缘 AI 部署**
|
| 6 |
-
|
| 7 |
-
</div>
|
| 8 |
|
| 9 |
---
|
| 10 |
|
|
@@ -241,6 +237,11 @@ uvicorn src.api.main:app --host 0.0.0.0 --port 8000 --reload
|
|
| 241 |
|
| 242 |
## 案例演示
|
| 243 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 244 |
### 1. 上传原始文件
|
| 245 |
|
| 246 |
支持 PDF、TXT、JPG、PNG、MP3、WAV、MP4 等格式,拖拽或点击上传。
|
|
@@ -269,11 +270,13 @@ uvicorn src.api.main:app --host 0.0.0.0 --port 8000 --reload
|
|
| 269 |
|
| 270 |
LLM 基于检索到的多模态上下文(文本段落 + 图片 + 视频关键帧)生成回答,每个回答附带来源引用。
|
| 271 |
|
| 272 |
-
提问「**图中有几
|
|
|
|
|
|
|
| 273 |
|
| 274 |

|
| 275 |
|
| 276 |
-
提问「**主持人的衣服颜色**」,系统检索到包含主持人的相关视频片段,并解析视频内容进行回答:「**主持人的衣服颜色为蓝色**」。
|
| 277 |
|
| 278 |

|
| 279 |
|
|
|
|
| 1 |
# Multimodal RAG 智能问答系统(基于AX650N)
|
| 2 |
|
| 3 |
+
<center><b> **端到端多模态检索增强生成** | **跨模态语义理解** | **AX650N 边缘 AI 部署**</b></center>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
---
|
| 6 |
|
|
|
|
| 237 |
|
| 238 |
## 案例演示
|
| 239 |
|
| 240 |
+
### 演示视频
|
| 241 |
+
|
| 242 |
+
[观看演示视频](docs\images\Multimodal_RAG.mp4)
|
| 243 |
+
|
| 244 |
+
|
| 245 |
### 1. 上传原始文件
|
| 246 |
|
| 247 |
支持 PDF、TXT、JPG、PNG、MP3、WAV、MP4 等格式,拖拽或点击上传。
|
|
|
|
| 270 |
|
| 271 |
LLM 基于检索到的多模态上下文(文本段落 + 图片 + 视频关键帧)生成回答,每个回答附带来源引用。
|
| 272 |
|
| 273 |
+
提问「**图片中有几个面包**」,系统在知识库中正确检索到相关的图片,并解析图片内容进行正确回答:「**5个面包**」。
|
| 274 |
+
|
| 275 |
+
|
| 276 |
|
| 277 |

|
| 278 |
|
| 279 |
+
提问「**sp视频中主持人的衣服颜色**」,系统检索到包含主持人的相关视频片段,并解析视频内容进行回答:「**主持人的衣服颜色为蓝色**」。
|
| 280 |
|
| 281 |

|
| 282 |
|
data/COCO_val2014_000000519046.jpg
ADDED
|
data/test/0.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13f6d2960cd920b7a472488e6158c19bab03768cf5ec9720a6df08c1a5c3c3a9
|
| 3 |
+
size 1158992
|
data/test/1.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e3426c70ce0aaf721df53542bf484f2fec9e28c49a99b02a081966deefda160
|
| 3 |
+
size 2076716
|
data/test/COCO_val2014_000000166124.jpg
ADDED
|
data/test/COCO_val2014_000000532867.jpg
ADDED
|
data/test/COCO_val2014_000000534121.jpg
ADDED
|
data/test/COCO_val2014_000000534733.jpg
ADDED
|
data/test/COCO_val2014_000000535183.jpg
ADDED
|
data/test/COCO_val2014_000000535292.jpg
ADDED
|
data/test/COCO_val2014_000000535306.jpg
ADDED
|
data/test/COCO_val2014_000000535578.jpg
ADDED
|
data/test/COCO_val2014_000000536615.jpg
ADDED
|
data/test/COCO_val2014_000000536791.jpg
ADDED
|
data/test/COCO_val2014_000000538092.jpg
ADDED
|
data/test/COCO_val2014_000000538451.jpg
ADDED
|
data/test/COCO_val2014_000000539079.jpg
ADDED
|
data/test/COCO_val2014_000000539124.jpg
ADDED
|
data/test/COCO_val2014_000000539355.jpg
ADDED
|
data/test/COCO_val2014_000000539784.jpg
ADDED
|
data/test/COCO_val2014_000000540806.jpg
ADDED
|
data/test/COCO_val2014_000000541965.jpg
ADDED
|
data/test/COCO_val2014_000000542509.jpg
ADDED
|
data/test/Model_Provisions_comprehensive_commercial_insurance_vehicles.pdf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2376233fdf6098ca6235ee8216db5cfdb571a214c04fbb6d576df6e9ddcec8e3
|
| 3 |
+
size 468578
|
data/test/sanguo.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:41ee0dcee07bca57472a2b84bdafddd731858f245136cfdbbd935e7826e16263
|
| 3 |
+
size 9175593
|
data/test/sp.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f746aff662493c4402c2e593cf9590d1d123aeef85de1f65cc29bbb30173ff92
|
| 3 |
+
size 36698404
|
data/test/test.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3fc96b879394bf3bfd915dae6d01398709decded80b983f9eeb3154ae91a8fbc
|
| 3 |
+
size 1537928
|
data/test/test.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
长颈鹿(英文名:Giraffe)是长颈鹿科、长颈鹿属动物的通称。共有4个物种:北方长颈鹿(Giraffa camelopardalis)、南方长颈鹿(Giraffa Giraffa)、网纹长颈鹿(Giraffa reticulata)和马赛长颈鹿(Giraffa tippelskirche)。拉丁文名字的意思是“长着豹纹的骆驼”。它们是世界上现存最高的陆生动物。站立时由头至脚可达4.5-6.1米,从头到臀部的长度约为4.5-6米, [34]体重1180-1930千克,刚出生的幼仔为2米高,体重为50-55千克。皮毛颜色花纹有斑点和网纹型,头的额部宽,吻部较尖,耳大竖立,头顶有1对骨质短角,角外包覆皮肤和茸毛;颈特别长(约2米),颈背有1行鬃毛;体较短;四肢高而强健,前肢略长于后肢,蹄阔大;尾短小,尾端为黑色簇毛。牙齿为原始的低冠齿,不能以草为主食,只能以树叶为主食;舌较长,可以用于取食;具皮骨角,角上被有毛的皮肤覆盖。 [6] [24] [35]
|
| 2 |
+
栖息于非洲热带、亚热带稀树草原、灌丛、开放的合欢林地、干旱而开阔的稀树草原地带和树木稀少的半沙漠地带。采用自由跑步态移动,时速可达50千米。是植食性动物,以树叶及嫩芽为主食。长颈鹿为均衡脑部血压,会叉开腿俯下头喝水。在野外长颈鹿的寿命为27年左右,动物园里的能活超过29年。原产于非洲,主要分布在撒哈拉以南到南非的德兰瓦士东部、纳塔尔和博茨瓦纳北部。已经从西非大部分地区消失,除了尼日尔的一部分残存个体。长颈鹿已经被重新引入南非的狩猎行列。是南非的国兽
|
data/test/国药准字H37020386_布洛芬片.pdf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:22954cb51781e685a42d5dd1abac0bde98906af75a6097871e3a937bdeaa4cdf
|
| 3 |
+
size 125946
|
docs/images/Multimodal_RAG.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:92753f5eefa27c5abeae341a60f305904112e9e4af97d9ed0747ad696bf76d5a
|
| 3 |
+
size 4445206
|
docs/images/image-9.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|