390 lines
9.6 KiB
Markdown
390 lines
9.6 KiB
Markdown
|
<h1><center>AI大模型Ollama结合Open-webui</center></h1>
|
|||
|
|
|
|||
|
|
**作者：行癫（盗版必究）**
|
|||
|
|
|
|||
|
|
------
|
|||
|
|
|
|||
|
|
## 一：认识 Ollama
|
|||
|
|
|
|||
|
|
#### 1.什么是Ollama
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
Ollama是一个开源的 LLM（大型语言模型）服务工具，用于简化在本地运行大语言模型，降低使用大语言模型的门槛，使得大模型的开发者、研究人员和爱好者能够在本地环境快速实验、管理和部署最新大语言模型
|
|||
|
|
|
|||
|
|
#### 2.官方网址
|
|||
|
|
|
|||
|
|
官方地址：https://ollama.com/
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
Ollama目前支持以下大语言模型：https://ollama.com/library
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
Ollama下载地址：https://ollama.com/download/ollama-linux-amd64.tgz
|
|||
|
|
|
|||
|
|
#### 3.注意事项
|
|||
|
|
|
|||
|
|
qwen、qwq、Llama等都是大语言模型
|
|||
|
|
|
|||
|
|
Ollama是大语言模型（不限于`Llama`模型）便捷的管理和运维工具
|
|||
|
|
|
|||
|
|
## 二：安装部署Ollama
|
|||
|
|
|
|||
|
|
#### 1.官方脚本安装
|
|||
|
|
|
|||
|
|
注意：服务器需要可以访问github等外网
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
curl -fsSL https://ollama.com/install.sh | sh
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
#### 2.二进制安装
|
|||
|
|
|
|||
|
|
参考网址：https://github.com/ollama/ollama/blob/main/docs/linux.md
|
|||
|
|
|
|||
|
|
获取二进制安装包
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
解压及安装
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
sudo tar -C /usr -xzf ollama-linux-amd64.tgz
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
创建管理用户和组
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
|
|||
|
|
sudo usermod -a -G ollama $(whoami)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
配置启动管理文件`/etc/systemd/system/ollama.service`
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[Unit]
|
|||
|
|
Description=Ollama Service
|
|||
|
|
After=network-online.target
|
|||
|
|
|
|||
|
|
[Service]
|
|||
|
|
ExecStart=/usr/bin/ollama serve
|
|||
|
|
User=ollama
|
|||
|
|
Group=ollama
|
|||
|
|
Restart=always
|
|||
|
|
RestartSec=3
|
|||
|
|
Environment="PATH=$PATH"
|
|||
|
|
|
|||
|
|
[Install]
|
|||
|
|
WantedBy=default.target
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
启动
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
sudo systemctl daemon-reload
|
|||
|
|
sudo systemctl enable ollama
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
注意：如果手动启动直接执行以下命令
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
ollama serve
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
#### 3.容器安装
|
|||
|
|
|
|||
|
|
CPU Only：
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
GPU Install：
|
|||
|
|
|
|||
|
|
参考网址：https://github.com/ollama/ollama/blob/main/docs/docker.md
|
|||
|
|
|
|||
|
|
#### 4.Kubernetes安装
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
apiVersion: apps/v1
|
|||
|
|
kind: Deployment
|
|||
|
|
metadata:
|
|||
|
|
annotations: {}
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: web
|
|||
|
|
k8s.kuboard.cn/name: ollama
|
|||
|
|
name: ollama
|
|||
|
|
namespace: xingdian-ai
|
|||
|
|
spec:
|
|||
|
|
progressDeadlineSeconds: 600
|
|||
|
|
replicas: 1
|
|||
|
|
revisionHistoryLimit: 10
|
|||
|
|
selector:
|
|||
|
|
matchLabels:
|
|||
|
|
k8s.kuboard.cn/layer: web
|
|||
|
|
k8s.kuboard.cn/name: ollama
|
|||
|
|
strategy:
|
|||
|
|
rollingUpdate:
|
|||
|
|
maxSurge: 25%
|
|||
|
|
maxUnavailable: 25%
|
|||
|
|
type: RollingUpdate
|
|||
|
|
template:
|
|||
|
|
metadata:
|
|||
|
|
creationTimestamp: null
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: web
|
|||
|
|
k8s.kuboard.cn/name: ollama
|
|||
|
|
spec:
|
|||
|
|
containers:
|
|||
|
|
- image: >-
|
|||
|
|
swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/ollama/ollama:latest
|
|||
|
|
imagePullPolicy: IfNotPresent
|
|||
|
|
name: ollama
|
|||
|
|
ports:
|
|||
|
|
- containerPort: 11434
|
|||
|
|
protocol: TCP
|
|||
|
|
volumeMounts:
|
|||
|
|
- mountPath: /root/.ollama
|
|||
|
|
name: volume-28ipz
|
|||
|
|
restartPolicy: Always
|
|||
|
|
volumes:
|
|||
|
|
- name: volume-28ipz
|
|||
|
|
nfs:
|
|||
|
|
path: /data/ollama
|
|||
|
|
server: 10.9.12.250
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
apiVersion: v1
|
|||
|
|
kind: Service
|
|||
|
|
metadata:
|
|||
|
|
annotations: {}
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: web
|
|||
|
|
k8s.kuboard.cn/name: ollama
|
|||
|
|
name: ollama
|
|||
|
|
namespace: xingdian-ai
|
|||
|
|
spec:
|
|||
|
|
clusterIP: 10.109.25.34
|
|||
|
|
clusterIPs:
|
|||
|
|
- 10.109.25.34
|
|||
|
|
internalTrafficPolicy: Cluster
|
|||
|
|
ipFamilies:
|
|||
|
|
- IPv4
|
|||
|
|
ipFamilyPolicy: SingleStack
|
|||
|
|
ports:
|
|||
|
|
- name: 3scykr
|
|||
|
|
port: 11434
|
|||
|
|
protocol: TCP
|
|||
|
|
targetPort: 11434
|
|||
|
|
selector:
|
|||
|
|
k8s.kuboard.cn/layer: web
|
|||
|
|
k8s.kuboard.cn/name: ollama
|
|||
|
|
sessionAffinity: None
|
|||
|
|
type: ClusterIP
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
#### 5.基本使用
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[root@xingdian-ai ~]# ollama -h
|
|||
|
|
Large language model runner
|
|||
|
|
|
|||
|
|
Usage:
|
|||
|
|
ollama [flags]
|
|||
|
|
ollama [command]
|
|||
|
|
|
|||
|
|
Available Commands:
|
|||
|
|
serve Start ollama
|
|||
|
|
create Create a model from a Modelfile
|
|||
|
|
show Show information for a model
|
|||
|
|
run Run a model
|
|||
|
|
stop Stop a running model
|
|||
|
|
pull Pull a model from a registry
|
|||
|
|
push Push a model to a registry
|
|||
|
|
list List models
|
|||
|
|
ps List running models
|
|||
|
|
cp Copy a model
|
|||
|
|
rm Remove a model
|
|||
|
|
help Help about any command
|
|||
|
|
|
|||
|
|
Flags:
|
|||
|
|
-h, --help help for ollama
|
|||
|
|
-v, --version Show version information
|
|||
|
|
|
|||
|
|
Use "ollama [command] --help" for more information about a command.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
获取大语言模型
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[root@xingdian-ai ~]# ollama pull gemma2:2b
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
注意：gemma是谷歌的大语言模型
|
|||
|
|
|
|||
|
|
查看已有的发语言模型
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[root@xingdian-ai ~]# ollama list
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
删除大语言模型
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[root@xingdian-ai ~]# ollama list qwen2:7b
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
运行大语言模型
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
[root@xingdian-ai ~]# ollama run qwen:0.5b
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## 三：认识 Open-webui
|
|||
|
|
|
|||
|
|
#### 1.什么是Open-webui
|
|||
|
|
|
|||
|
|
Open-WebUI 常用于给用户提供一个图形界面，通过它，用户可以方便地与机器学习模型进行交互
|
|||
|
|
|
|||
|
|
#### 2.官方地址
|
|||
|
|
|
|||
|
|
官方地址：https://github.com/open-webui/open-webui
|
|||
|
|
|
|||
|
|
## 四：安装部署和使用
|
|||
|
|
|
|||
|
|
#### 1.容器安装方式
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://example.com -e HF_ENDPOINT=https://hf-mirror.com -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
国内镜像地址：swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/open-webui/open-webui:main
|
|||
|
|
|
|||
|
|
OLLAMA_BASE_URL：指定Ollama地址
|
|||
|
|
|
|||
|
|
由于国内网络无法直接访问 huggingface , 我们需要更改为国内能访问的域名 hf-mirror.com
|
|||
|
|
|
|||
|
|
```shell
|
|||
|
|
报错：
|
|||
|
|
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json.
|
|||
|
|
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
|
|||
|
|
No WEBUI_SECRET_KEY provided
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
解决方法： 使用镜像站 -e HF_ENDPOINT=https://hf-mirror.com
|
|||
|
|
|
|||
|
|
#### 2.kubernetes方式
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
---
|
|||
|
|
apiVersion: apps/v1
|
|||
|
|
kind: Deployment
|
|||
|
|
metadata:
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: svc
|
|||
|
|
k8s.kuboard.cn/name: ai
|
|||
|
|
name: ai
|
|||
|
|
namespace: xingdian-ai
|
|||
|
|
spec:
|
|||
|
|
progressDeadlineSeconds: 600
|
|||
|
|
replicas: 1
|
|||
|
|
revisionHistoryLimit: 10
|
|||
|
|
selector:
|
|||
|
|
matchLabels:
|
|||
|
|
k8s.kuboard.cn/layer: svc
|
|||
|
|
k8s.kuboard.cn/name: ai
|
|||
|
|
strategy:
|
|||
|
|
rollingUpdate:
|
|||
|
|
maxSurge: 25%
|
|||
|
|
maxUnavailable: 25%
|
|||
|
|
type: RollingUpdate
|
|||
|
|
template:
|
|||
|
|
metadata:
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: svc
|
|||
|
|
k8s.kuboard.cn/name: ai
|
|||
|
|
spec:
|
|||
|
|
containers:
|
|||
|
|
- env:
|
|||
|
|
- name: OLLAMA_BASE_URL
|
|||
|
|
value: 'http://10.9.12.10:11434'
|
|||
|
|
- name: HF_ENDPOINT
|
|||
|
|
value: 'https://hf-mirror.com'
|
|||
|
|
image: '10.9.12.201/ollama/open-webui:main'
|
|||
|
|
imagePullPolicy: Always
|
|||
|
|
name: ai
|
|||
|
|
ports:
|
|||
|
|
- containerPort: 8080
|
|||
|
|
protocol: TCP
|
|||
|
|
resources: {}
|
|||
|
|
terminationMessagePath: /dev/termination-log
|
|||
|
|
terminationMessagePolicy: File
|
|||
|
|
restartPolicy: Always
|
|||
|
|
terminationGracePeriodSeconds: 30
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
apiVersion: v1
|
|||
|
|
kind: Service
|
|||
|
|
metadata:
|
|||
|
|
annotations: {}
|
|||
|
|
labels:
|
|||
|
|
k8s.kuboard.cn/layer: svc
|
|||
|
|
k8s.kuboard.cn/name: ai
|
|||
|
|
name: ai
|
|||
|
|
namespace: xingdian-ai
|
|||
|
|
spec:
|
|||
|
|
clusterIP: 10.99.5.168
|
|||
|
|
clusterIPs:
|
|||
|
|
- 10.99.5.168
|
|||
|
|
internalTrafficPolicy: Cluster
|
|||
|
|
ipFamilies:
|
|||
|
|
- IPv4
|
|||
|
|
ipFamilyPolicy: SingleStack
|
|||
|
|
ports:
|
|||
|
|
- name: cnr2rn
|
|||
|
|
port: 8080
|
|||
|
|
protocol: TCP
|
|||
|
|
targetPort: 8080
|
|||
|
|
selector:
|
|||
|
|
k8s.kuboard.cn/layer: svc
|
|||
|
|
k8s.kuboard.cn/name: ai
|
|||
|
|
sessionAffinity: None
|
|||
|
|
type: ClusterIP
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
#### 3.使用kong ingress 进行引流
|
|||
|
|
|
|||
|
|
创建upstream svc地址为：xingdian-ai.ai.svc:8080
|
|||
|
|
|
|||
|
|
创建services
|
|||
|
|
|
|||
|
|
创建route
|
|||
|
|
|
|||
|
|
配置DNS解析
|
|||
|
|
|
|||
|
|
#### 4.浏览器访问
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
事先在ollama上获取大模型，我们一qwen:0.5b为例，其他的大模型占用资源较多
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|