在国内大家对数据隐私看的很重，这点从私有云和混合云占据半壁江山就能看得出来了，想要大模型应用场景落地是离不开本地部署大模型的。这篇文章简单介绍下怎么用java调用本地大模型。
01
—
前置知识
在前面的文章中已经介绍了怎么在本地通过ollama运行本地大模型了，再此就不再赘述。假设你已经在本地启动ollama服务了。
02
—
spring AI调用ollama
工程沿用上一篇文章中的demo工程，首先在application.yml中增加ollama的地址配置和模型名称
spring:# spring ai模块:ollama:: http://127.0.0.1:11434chat:model: qwen1-7b
controller代码如下
package com.example.demo.controller;import com.example.demo.service.OllamaService;import jakarta.servlet.http.HttpServletResponse;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;@RestController@RequestMapping("/ollamaApi")public class OllamaController {@Autowiredprivate OllamaService ollamaService;@RequestMapping("/chat")public String chat(){return ollamaService.chat();}@RequestMapping("/stream")public SseEmitter stream(HttpServletResponse response){return ollamaService.stream(response);}}
service代码如下
package com.example.demo.service.impl;import cn.hutool.core.collection.CollUtil;import cn.hutool.core.map.MapUtil;import com.example.demo.service.OllamaService;import jakarta.servlet.http.HttpServletResponse;import lombok.extern.slf4j.Slf4j;import org.apache.commons.lang3.StringUtils;import org.springframework.ai.chat.Generation;import org.springframework.ai.chat.messages.AssistantMessage;import org.springframework.ai.chat.messages.Message;import org.springframework.ai.chat.messages.UserMessage;import org.springframework.ai.chat.prompt.Prompt;import org.springframework.ai.chat.prompt.SystemPromptTemplate;import org.springframework.ai.ollama.OllamaChatClient;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;import java.util.List;public class OllamaServiceImpl implements OllamaService {private OllamaChatClient ollamaChatClient;public String chat() {String systemPrompt = "{prompt}";SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemPrompt);String userPrompt = "中国有哪些名胜古迹？";Message userMessage = new UserMessage(userPrompt);Message systemMessage = systemPromptTemplate.createMessage(MapUtil.of("prompt", "you are a helpful AI assistant"));Prompt prompt = new Prompt(List.of(userMessage, systemMessage));List<Generation> response = ollamaChatClient.call(prompt).getResults();String result = "";for (Generation generation : response){String content = generation.getOutput().getContent();result += content;}System.out.println(result);return result;}public SseEmitter stream(HttpServletResponse response) {response.setContentType("text/event-stream");response.setCharacterEncoding("UTF-8");SseEmitter emitter = new SseEmitter();String systemPrompt = "{prompt}";SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemPrompt);String userPrompt = "中国有哪些名胜古迹？";Message userMessage = new UserMessage(userPrompt);Message systemMessage = systemPromptTemplate.createMessage(MapUtil.of("prompt", "you are a helpful AI assistant"));Prompt prompt = new Prompt(List.of(userMessage, systemMessage));ollamaChatClient.stream(prompt).subscribe(x -> {try {log.info("response: {}",x);List<Generation> generations = x.getResults();if(CollUtil.isNotEmpty(generations)){for(Generation generation:generations){AssistantMessage assistantMessage = generation.getOutput();String content = assistantMessage.getContent();if(StringUtils.isNotEmpty(content)){emitter.send(content);}else{if(StringUtils.equals(content,"null"))emitter.complete(); // Complete the SSE connection}}}} catch (Exception e) {emitter.complete();log.error("流式返回结果异常",e);}});return emitter;}}
03
—
完整代码
完整代码后台回复springai免费获取，获取到项目代码后找到DemoApplication类，右键运行，服务端口为8080。启动后使用工具调用接口即可运行
http://localhost:8080/ollamaApi/chathttp://localhost:8080/ollamaApi/stream
原文始发于微信公众号（AI与网安）：【大模型入门4】通过spring AI实现java调用本地大模型
- 左青龙
- 微信扫一扫
-
- 右白虎
- 微信扫一扫
-
评论