在国内大家对数据隐私看的很重,这点从私有云和混合云占据半壁江山就能看得出来了,想要大模型应用场景落地是离不开本地部署大模型的。这篇文章简单介绍下怎么用java调用本地大模型。
01
—
前置知识
在前面的文章中已经介绍了怎么在本地通过ollama运行本地大模型了,再此就不再赘述。假设你已经在本地启动ollama服务了。
02
—
spring AI调用ollama
工程沿用上一篇文章中的demo工程,首先在application.yml中增加ollama的地址配置和模型名称
spring:
# spring ai模块
:
ollama:
http://127.0.0.1:11434 :
chat:
model: qwen1-7b
controller代码如下
package com.example.demo.controller;
import com.example.demo.service.OllamaService;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;
@RestController
@RequestMapping("/ollamaApi")
public class OllamaController {
@Autowired
private OllamaService ollamaService;
@RequestMapping("/chat")
public String chat(){
return ollamaService.chat();
}
@RequestMapping("/stream")
public SseEmitter stream(HttpServletResponse response){
return ollamaService.stream(response);
}
}
service代码如下
package com.example.demo.service.impl;
import cn.hutool.core.collection.CollUtil;
import cn.hutool.core.map.MapUtil;
import com.example.demo.service.OllamaService;
import jakarta.servlet.http.HttpServletResponse;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.ai.chat.Generation;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.prompt.SystemPromptTemplate;
import org.springframework.ai.ollama.OllamaChatClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;
import java.util.List;
public class OllamaServiceImpl implements OllamaService {
private OllamaChatClient ollamaChatClient;
public String chat() {
String systemPrompt = "{prompt}";
SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemPrompt);
String userPrompt = "中国有哪些名胜古迹?";
Message userMessage = new UserMessage(userPrompt);
Message systemMessage = systemPromptTemplate.createMessage(MapUtil.of("prompt", "you are a helpful AI assistant"));
Prompt prompt = new Prompt(List.of(userMessage, systemMessage));
List<Generation> response = ollamaChatClient.call(prompt).getResults();
String result = "";
for (Generation generation : response){
String content = generation.getOutput().getContent();
result += content;
}
System.out.println(result);
return result;
}
public SseEmitter stream(HttpServletResponse response) {
response.setContentType("text/event-stream");
response.setCharacterEncoding("UTF-8");
SseEmitter emitter = new SseEmitter();
String systemPrompt = "{prompt}";
SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemPrompt);
String userPrompt = "中国有哪些名胜古迹?";
Message userMessage = new UserMessage(userPrompt);
Message systemMessage = systemPromptTemplate.createMessage(MapUtil.of("prompt", "you are a helpful AI assistant"));
Prompt prompt = new Prompt(List.of(userMessage, systemMessage));
ollamaChatClient.stream(prompt).subscribe(x -> {
try {
log.info("response: {}",x);
List<Generation> generations = x.getResults();
if(CollUtil.isNotEmpty(generations)){
for(Generation generation:generations){
AssistantMessage assistantMessage = generation.getOutput();
String content = assistantMessage.getContent();
if(StringUtils.isNotEmpty(content)){
emitter.send(content);
}else{
if(StringUtils.equals(content,"null"))
emitter.complete(); // Complete the SSE connection
}
}
}
} catch (Exception e) {
emitter.complete();
log.error("流式返回结果异常",e);
}
});
return emitter;
}
}
03
—
完整代码
完整代码后台回复springai免费获取,获取到项目代码后找到DemoApplication类,右键运行,服务端口为8080。启动后使用工具调用接口即可运行
http://localhost:8080/ollamaApi/chat
http://localhost:8080/ollamaApi/stream
原文始发于微信公众号(AI与网安):【大模型入门4】通过spring AI实现java调用本地大模型
免责声明:文章中涉及的程序(方法)可能带有攻击性,仅供安全研究与教学之用,读者将其信息做其他用途,由读者承担全部法律及连带责任,本站不承担任何法律及连带责任;如有问题可邮件联系(建议使用企业邮箱或有效邮箱,避免邮件被拦截,联系方式见首页),望知悉。
- 左青龙
- 微信扫一扫
-
- 右白虎
- 微信扫一扫
-
评论