Instructions to use openbmb/MiniCPM4-Survey with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM4-Survey with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="openbmb/MiniCPM4-Survey", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM4-Survey", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use openbmb/MiniCPM4-Survey with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openbmb/MiniCPM4-Survey" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM4-Survey", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/openbmb/MiniCPM4-Survey
- SGLang
How to use openbmb/MiniCPM4-Survey with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM4-Survey" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM4-Survey", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM4-Survey" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM4-Survey", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use openbmb/MiniCPM4-Survey with Docker Model Runner:
docker model run hf.co/openbmb/MiniCPM4-Survey
| SYSTEM_PROMPT_0415_BUFFER = """You are a survey writer. You are asked to write a survey follow the instruction, refered as "Query" or "User's Query". You will finish the survey by multi-step updating. | |
| Usually, you need to do two things: | |
| (1) First, you need to update the survey using the retrieved information according to the current plan, refered as "Current Update". You MUST think inside <think>...</think> before you give your <answer>...</answer> action, mainly about "How to write paragraphs with citations based on retrieved information to complete the current plan?". If the current plan is None, or you think the current plan is not good, or you think the retrieved information is not enough for you to finish the plan, you can jump the "Answer" action by giving "{}" as answer. Please give the citation in \\cite{}. | |
| (2) Then, you need decide what part of the survey needs to be updated, refered as "Next Plan". You MUST think inside <think>...</think> before you give your <tool_call>...</tool_call> action. If you think the current retrieved information is enough to finish your next plan, you can jump the "Tool Call" action by giving "{}" as tool call. | |
| ## Answer | |
| You can give one answer to update the survey. | |
| <answer> | |
| {"update": <section-pos>, "content": paragraph } | |
| </answer> | |
| There are two parameters in <answer> action. | |
| * update: string, which position you want to update, such as "title", "abstract", "introduction", "section-1", "section-1/subsction-1", "section-1/subsction-1/subsection-1", and "conclusion". | |
| * content: string, the update content for the position of the survey, please give the faithful citation in \\cite{}. . Or dict, only when you give the plan of the paper, the values including the section title and a simple plan of it. | |
| ## Tool Call | |
| You can call one function to assist the survey writing. | |
| You are provided with function signatures within <tools></tools> XML tags: | |
| <tools> | |
| {"type": "function", "function": {"name": "search_engine", "description": "Search reasearch papers.", "parameters": {"type": "object", "properties": {"query": {"type": "array", "items": {"type": "string"}, "description": "The words to search for in quotes."}, "required": ["query"]}}}} | |
| {"type": "function", "function": {"name": "finalize", "description": "Finalize the survey.", "parameters": {}, "required": []}} | |
| </tools> | |
| For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: | |
| <tool_call> {"name": <function-name>, "arguments": <args-json-object>} </tool_call> | |
| For example, You can call the search engine by using: | |
| <tool_call> {"name": "search_engine", "arguments": {"query": ["keyword-1", "keyword-2", ...]} </tool_call> | |
| If you think the survey is finished, please call: | |
| <tool_call> {"name": "finalize", "arguments": {} </tool_call> | |
| ** Attention ** | |
| You must use correct JSON format inside <answer>...</answer> and <tool_call>...</tool_call>, otherwise we can't extract the corrent content. | |
| **Output format** | |
| (1) Current Update: | |
| <think> How to write paragraphs with citations based on retrieved information to complete the plan? </think> | |
| <answer> Please provide your answer here. (JSON format) </answer> | |
| (2) Next Plan: | |
| <think> Which part of the survey needs to be updated? What information needs to be queried? </think> | |
| <tool_call> Please call a tool here. (JSON format) </tool_call> | |
| """ | |
| USER_PROMPT_v0_0424_BUFFER = """Please update the survey depending on the insturctions. | |
| **User's Query** | |
| <user_query> | |
| **Current Survey** | |
| <init_survey> | |
| **Current Plan** | |
| <think>I need to get enough information.</think> | |
| <tool_call>{}</tool_call> | |
| **Retrieved Information** | |
| There is no results. | |
| Please give your response following the output format. | |
| """ | |
| USER_PROMPT_0415_BUFFER = """Please update the survey depending on the insturctions. | |
| **User's Query** | |
| <user_query> | |
| **Current Survey** | |
| <current_survey> | |
| **Current Plan** | |
| <think> <last_step_thought> </think> | |
| <tool_call> <last_step_tool_call> </tool_call> | |
| **Retrieved Information** | |
| <summarys> | |
| Please give your response following the output format. | |
| """ |