本地部署大模型
·
Ollama 启动后,默认在 http://localhost:11434 提供 API 服务。
1、对话接口
curl http://localhost:11434/api/chat -d '{
"model":"qwen3:4b",
"messages":[
{"role":"user","content":"你好"}
]
}'
2、流式输出
curl http://localhost:11434/api/chat -d '{
"model":"qwen3:4b",
"messages":[
{"role":"user","content":"你好"}
],
"stream":true
}'
3、Python调用
import requests
response = requests.post(
"http://localhost:11434/api/chat",
json = {
"model":"qwen3:4b",
"messages":[
{"role":"user", "content":"你好"}
],
"stream":False
}
)
print(response.json()["message"]["content"])
4、openai接口
from openai import OpenAI
client=OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama"
)
response = client.chat.completions.create(
model="qwen2.5:0.5b",
messages=[
{"role":"system","content":"你是一个Python专家"},
{"role":"user","content":"解释一下装饰器"}
]
)
print(response.choices[0].message.content)
或流式输出
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama"
)
response = client.chat.completions.create(
model = "qwen2.5:0.5b",
messages = [
{"role":"user","content":"你好"}
],
stream = True
)
for chunk in response:
text = chunk.choices[0].delta.content
if text:
print(text,end="",flush=True)
更多推荐

所有评论(0)