背景

概念

Flowise

Flowise 是一个开源的用户界面可视化工具，它允许用户通过拖放的方式来构建自定义的大型语言模型（LLM）流程。
Flowise基于LangChain.js，是一个非常先进的图形用户界面，用于开发基于LLM的应用程序。Flowise还支持Docker和NodeJS，可以在本地或者在容器中运行。如果有需求修改相关代码，可以本地部署运行。

Langflow

Langflow 是一个用于创建和运行语言模型流程的网络应用程序。
Langflow是一个动态图，每个节点都是一个可执行单元¹。它的模块化和交互式设计促进了快速的实验和原型制作。Langflow还提供了一种无代码的AI生态系统，可以无缝地与您的团队熟悉和喜欢的工具和堆栈集成。

Localai

LocalAI是一个免费的、开源的OpenAI替代品。它是一个与OpenAI API规范兼容的REST API，允许你在本地或者在本地网络上使用消费级硬件运行大型语言模型（LLM）。它支持多个与ggml格式兼容的模型系列。

1.Flowise安装部署

git

https://github.com/FlowiseAI/Flowise

官网

FlowiseAI - Build LLMs Apps Easily

docker部署

[root@master docker]# git clone https://github.com/FlowiseAI/Flowise.git
[root@master langchain_wk]# ls
Flowise
[root@master langchain_wk]# cd Flowise/
[root@master Flowise]# ls
artillery-load-test.yml  babel.config.js     CODE_OF_CONDUCT-ZH.md  CONTRIBUTING-ZH.md  Dockerfile  LICENSE.md    packages   README-ZH.md
assets                   CODE_OF_CONDUCT.md  CONTRIBUTING.md        docker              images      package.json  README.md  turbo.json
[root@master Flowise]# clear
[root@master Flowise]# ls
artillery-load-test.yml  babel.config.js     CODE_OF_CONDUCT-ZH.md  CONTRIBUTING-ZH.md  Dockerfile  LICENSE.md    packages   README-ZH.md
assets                   CODE_OF_CONDUCT.md  CONTRIBUTING.md        docker              images      package.json  README.md  turbo.json
[root@master Flowise]# cd docker/
[root@master docker]# ls
docker-compose.yml  Dockerfile  README.md
[root@master docker]# ls -alh
总用量 24K
drwxr-xr-x 2 root root 4.0K 11月 21 13:21 .
drwxr-xr-x 9 root root 4.0K 11月 21 13:21 ..
-rw-r--r-- 1 root root  988 11月 21 13:21 docker-compose.yml
-rw-r--r-- 1 root root  441 11月 21 13:21 Dockerfile
-rw-r--r-- 1 root root  675 11月 21 13:21 .env.example
-rw-r--r-- 1 root root 1.3K 11月 21 13:21 README.md
[root@master docker]# cp .env.example .env
[root@master docker]# vim .env
[root@master docker]#

修改.env配置文件

PORT=3090
DATABASE_PATH=/root/.flowise
APIKEY_PATH=/root/.flowise
SECRETKEY_PATH=/root/.flowise
LOG_PATH=/root/.flowise/logs

# NUMBER_OF_PROXIES= 1

# DATABASE_TYPE=postgres
# DATABASE_PORT=""
# DATABASE_HOST=""
# DATABASE_NAME="flowise"
# DATABASE_USER=""
# DATABASE_PASSWORD=""

# FLOWISE_USERNAME=user
# FLOWISE_PASSWORD=1234
# FLOWISE_SECRETKEY_OVERWRITE=myencryptionkey
DEBUG=true
# LOG_LEVEL=debug (error | warn | info | verbose | debug)
# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash

# LANGCHAIN_TRACING_V2=true
# LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
# LANGCHAIN_API_KEY=your_api_key
# LANGCHAIN_PROJECT=your_project

docker-compose启动

[root@master docker]# docker-compose up -d
WARNING: The FLOWISE_USERNAME variable is not set. Defaulting to a blank string.
WARNING: The FLOWISE_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_TYPE variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_PORT variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_HOST variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_NAME variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_USER variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The FLOWISE_SECRETKEY_OVERWRITE variable is not set. Defaulting to a blank string.
WARNING: The LOG_LEVEL variable is not set. Defaulting to a blank string.
WARNING: Found orphan containers (supabase-rest, supabase-studio, supabase-edge-functions, supabase-auth, realtime-dev.supabase-realtime, supabase-imgproxy, supabase-storage, supabase-meta, supabase-db, supabase-kong) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Pulling flowise (flowiseai/flowise:)...
latest: Pulling from flowiseai/flowise
96526aa774ef: Pull complete
3130715204cf: Pull complete
b06de8ab1c4f: Pull complete
90ef3ffc5156: Pull complete
018fd88f5f14: Pull complete
e405bd0a6e44: Pull complete
6678d95b4ce0: Pull complete
f0b6d32b7e1f: Pull complete
754763d6e863: Pull complete
db86a2a89c8a: Pull complete
Digest: sha256:348211f573335adca3fdcfa1d62345eadee65e19cc3d83301f96b0359f31dc03
Status: Downloaded newer image for flowiseai/flowise:latest
Creating docker_flowise_1 ... done
[root@master docker]# docker-compose logs -f
Attaching to docker_flowise_1
flowise_1  | 2023-11-21 05:30:05 [INFO]: Starting Flowise...
flowise_1  | 2023-11-21 05:30:06 [INFO]: ⚡️ [server]: Flowise Server is listening at 3090
flowise_1  | 2023-11-21 05:30:06 [INFO]: 📦 [server]: Data Source has been initialized!

这里不多做flowise的教程，可以参考相关官方文档。

2.Localai 部署

注意：这里默认用的是cpu，localai主推用CPU，LocalAI具有Rust后端，内存效率高，体积小,并且后端模型部署gRpc,至于gpu部署，需要不同的nvidia cuda版本，可根据自己的cuda版本拉取相关镜像，然后重新build镜像；

build镜像会非常，最好使用代理或更换国内相关系统镜像源，rust源，go源，pip源；

# docker images | grep nvcr.io/nvidia/cuda
nvcr.io/nvidia/cuda                                                        11.2.1-cudnn8-devel-ubuntu20.04   b74328ac303f   5 months ago    7.72GB
nvcr.io/nvidia/cuda                                                        11.4.2-cudnn8-devel-ubi8          260b885046fb   5 months ago    9.76GB
nvcr.io/nvidia/cuda                                                        11.4.0-cudnn8-devel-centos7       32f9c23be6b7   5 months ago    9.69GB
nvcr.io/nvidia/cuda                                                        11.4.2-cudnn8-runtime-ubi8        16112a9b5038   5 months ago    3.89GB

这里用的11.2.1-cudnn8-devel-ubuntu20.04；

GPU的部署和编译相对比较麻烦，可参考官方文档。

GPU编译方法

make BUILD_TYPE=cublas build

启动方法

# ./local-ai --debug=true --f16=true --autoload-galleries=true

3:17PM INF Starting LocalAI using 4 threads, with models path: /build/models
3:17PM INF LocalAI version: v1.40.0-22-g991ecce (991ecce00462db955934ec1293954b7d61c58ad2)
3:17PM DBG Model: bert-embeddings (config: {PredictionOptions:{Model:bert-MiniLM-L6-v2q4_0.bin Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:bert-embeddings F16:false Threads:0 Debug:false Roles:map[] Embeddings:true Backend:bert-embeddings TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})
3:17PM DBG Model: thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin (config: {PredictionOptions:{Model:wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_K_M.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin F16:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})
3:17PM DBG Extracting backend assets files to /tmp/localai/backend_data

 ┌───────────────────────────────────────────────────┐
 │                   Fiber v2.50.0                   │
 │               http://127.0.0.1:8080               │
 │       (bound on host 0.0.0.0 and port 8080)       │
 │                                                   │
 │ Handlers ............ 74  Processes ........... 1 │
 │ Prefork ....... Disabled  PID ............. 10297 │
 └───────────────────────────────────────────────────┘

MODEL GALLERY 包含了huggingface的项目，如果下载模型，可能回被墙，至于如何解决，请自行解决。

下载模型

[192.168.12.82]:62589 200 - POST /models/apply
3:41PM DBG Checking "bert-MiniLM-L6-v2q4_0.bin" exists and matches SHA
3:41PM DBG File "bert-MiniLM-L6-v2q4_0.bin" already exists and matches the SHA. Skipping download
3:41PM DBG Written config file /build/models/bert-embeddings.yaml

编码测试

3.Flowise文档知识库召回应用

转载请注明出处！