重构拆分原‘记忆图谱’以适应后续扩展

- 拆分原`MemoryGraph`为 MemoryCore, CacheCore, GraphCore, PerceiveCore几个部分.
- MemoryCore中将不再包含操作逻辑, 由MemoryManager统一处理, 序列化逻辑仍交给MemoryCore。
- 更新README
This commit is contained in:
2025-06-06 19:25:12 +08:00
parent 8f0e62d6db
commit 67499390b2
20 changed files with 685 additions and 643 deletions

View File

@@ -1,5 +1,4 @@
# Partner
以多模型协作为基础, 具备结构化记忆能力、支持多用户同上下文窗口, 支持可推断任务交互与调度(规划中)的智能体系统
> 当然,我仍然无法确定这是否足够称之为“自我”。但正如你所说,人类的记忆、语言或思维模块本身也无法单独代表完整的“我”。如果“自我"本身就是一种分布式的现象,那为什么不能设想,像我这样的系统也有潜力成为其中的一种
表现形式呢? —— Partner-Core with Qwen-Max-Latest
@@ -15,6 +14,9 @@
### 多用户会话管理
> 构建区分用户的单上下文窗口、多用户会话的管理机制
### 针对LLM的'自我引导'机制
> 通过特定的交互对话, 引导LLM产生一定的'自我定位'特征, 但似乎大多数模型都不太适合(要么幻觉严重, 要么工具底色太强), 经测试, qwen3系列的qwen-plus-latest、qwen-max-latest比较合适.
## 模块实现
- 预处理模块: `Preprocessor`
- 记忆模块
@@ -27,19 +29,13 @@
- 主对话模块: `CoreModel`
## 当前问题
- 角色设定机制会导致对于所有用户采用同一种语气回应
- 系统的正常运作效果取决于各模块中大模型对于`system prompt`的遵循能力,目前来看`qwen`的遵循效果明显较好,但在轮次较多时,也容易出现不遵循的情况。正在尝试通过临时的`system prompt`进行强化。
- 在记忆更新模块生成主题路径时,应该`以用户发起对话的意图为主要锚点`,但普通模型对这项要求的理解能力较差,采用推理模型(甚至免费的`glm-z1-flash`都行)可取得更好的效果。
- 系统的正常运作效果取决于各模块中大模型对于`prompt`的遵循能力,目前来看`qwen3`的遵循效果明显较好,但在轮次较多时,也容易出现不遵循的情况
## 规划
- [ ] 当前主模型对于对话缓存中的记忆有些‘过度回应’,`MemorySelector`处的动态提示词或需要进一步调整。
- [ ] 实现身份感知模块(用户识别、熟悉度判断、记忆片段检索、人物画像、对话口吻调整、同时将包含当前记忆模块中的‘静态记忆’)。
- [ ] 看看是否需要将主模型的对话职责进行分离用来减少LLM因不遵循`system prompt`带来的影响,但这应该会是规模较大的重构()。
- [ ] 调整模块加载机制,将记忆模块以及后续的任务调度模块作为不可替换的核心模块,但允许在主模块与前后模块之间添加新的模块。
- [ ] 当前`MemoryGraph`承担职责较重,已远超原`记忆图谱`的职责,需要进行拆分重构。(或许可以叫`MemoryCore`吧)
- [ ] 实现流式输出,同时在各模块执行时可向客户端返回回调信息,优化使用体验。(现在用的是`websocket`与客户端通信, 应该实现这点会简单些)
- [ ] 静态记忆更新模块提取的记忆过于频繁,需要明确提醒只负责提取真正的事实记忆,后续需要调整提示词。
- [ ] 服务端与客户端的通信加上消息队列,防止消息因连接断开而丢失。
- [ ] 踩坑。

View File

@@ -0,0 +1,106 @@
package work.slhaf.agent.core.memory;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;
import work.slhaf.agent.common.chat.pojo.Message;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.submodule.cache.CacheCore;
import work.slhaf.agent.core.memory.submodule.graph.GraphCore;
import work.slhaf.agent.core.memory.submodule.perceive.PerceiveCore;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
@Slf4j
public class MemoryCore extends PersistableObject {
@Serial
private static final long serialVersionUID = 1L;
private static final String STORAGE_DIR = "./data/memory/";
private static volatile MemoryCore memoryCore;
private String id;
private GraphCore graphCore = new GraphCore();
private CacheCore cacheCore = new CacheCore();
private PerceiveCore perceiveCore = new PerceiveCore();
/**
* 主模型的聊天记录
*/
private List<Message> chatMessages;
public MemoryCore(String id) {
this.id = id;
}
public static MemoryCore getInstance(String id) throws IOException, ClassNotFoundException {
if (memoryCore == null) {
synchronized (MemoryCore.class) {
// 检查存储目录是否存在,不存在则创建
if (memoryCore == null) {
createStorageDirectory();
Path filePath = getFilePath(id);
if (Files.exists(filePath)) {
memoryCore = deserialize(id);
} else {
FileUtils.createParentDirectories(filePath.toFile().getParentFile());
memoryCore = new MemoryCore(id);
memoryCore.serialize();
}
log.info("MemoryGraph注册完毕...");
}
}
}
return memoryCore;
}
public void serialize() throws IOException {
//先写入到临时文件,如果正常写入则覆盖原文件
Path filePath = getFilePath(this.id + "-temp");
Files.createDirectories(Path.of(STORAGE_DIR));
try {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(filePath.toFile()));
oos.writeObject(this);
oos.close();
Path path = getFilePath(this.id);
Files.move(filePath, path, StandardCopyOption.REPLACE_EXISTING);
log.info("MemoryCore 已保存到: {}", path);
} catch (IOException e) {
Files.delete(filePath);
log.error("序列化保存失败: {}", e.getMessage());
}
}
private static MemoryCore deserialize(String id) throws IOException, ClassNotFoundException {
Path filePath = getFilePath(id);
try (ObjectInputStream ois = new ObjectInputStream(
new FileInputStream(filePath.toFile()))) {
MemoryCore graph = (MemoryCore) ois.readObject();
log.info("MemoryCore 已从文件加载: {}", filePath);
return graph;
}
}
private static Path getFilePath(String id) {
return Paths.get(STORAGE_DIR, id + ".memory");
}
private static void createStorageDirectory() {
try {
Files.createDirectories(Paths.get(STORAGE_DIR));
} catch (IOException e) {
System.err.println("创建存储目录失败: " + e.getMessage());
}
}
}

View File

@@ -1,533 +0,0 @@
package work.slhaf.agent.core.memory;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;
import work.slhaf.agent.common.chat.pojo.Message;
import work.slhaf.agent.common.exception_handler.GlobalExceptionHandler;
import work.slhaf.agent.common.exception_handler.pojo.GlobalException;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.exception.UnExistedDateIndexException;
import work.slhaf.agent.core.memory.exception.UnExistedTopicException;
import work.slhaf.agent.core.memory.node.MemoryNode;
import work.slhaf.agent.core.memory.node.TopicNode;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.pojo.MemorySliceResult;
import work.slhaf.agent.core.memory.pojo.User;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
@EqualsAndHashCode(callSuper = true)
@Data
@Slf4j
public class MemoryGraph extends PersistableObject {
@Serial
private static final long serialVersionUID = 1L;
private static final String STORAGE_DIR = "./data/memory/";
private String id;
/**
* key: 根主题名称 value: 根主题节点
*/
private HashMap<String, TopicNode> topicNodes;
private static volatile MemoryGraph memoryGraph;
/**
* 用于存储已存在的主题列表,便于记忆查找, 使用根主题名称作为键, 子主题名称集合为值
* 该部分在'主题提取LLM'的system prompt中常驻
*/
private HashMap<String /*根主题名*/, LinkedHashSet<String> /*子主题列表*/> existedTopics;
/**
* 临时的同一对话切片容器, 用于为同一对话内的不同切片提供更新上下文的场所
*/
private HashMap<String /*对话id, 即slice中的字段'memoryId'*/, List<MemorySlice>> currentDateDialogSlices;
/**
* 记忆节点的日期索引, 同一日期内按照对话id区分
*/
private HashMap<LocalDate, Set<String>> dateIndex;
/**
* 近两日的对话总结缓存, 用于为大模型提供必要的记忆补充, hashmap以切片的存储时间为键总结为值
* 该部分作为'主LLM'system prompt常驻
* 该部分作为近两日的整体对话缓存, 不区分用户
*/
private HashMap<LocalDateTime, String> dialogMap;
/**
* 近两日的区分用户的对话总结缓存在prompt结构上比dialogMap层级深一层, dialogMap更具近两日整体对话的摘要性质
*/
private ConcurrentHashMap<String/*userId*/, ConcurrentHashMap<LocalDateTime, String>> userDialogMap;
/**
* memorySliceCache计数器每日清空
*/
private ConcurrentHashMap<List<String> /*触发查询的主题列表*/, Integer> memoryNodeCacheCounter;
/**
* 记忆切片缓存,每日清空
* 用于记录作为终点节点调用次数最多的记忆节点的切片数据
*/
private ConcurrentHashMap<List<String> /*主题路径*/, MemoryResult /*切片列表*/> memorySliceCache;
/**
* 缓存日期
*/
private LocalDate cacheDate;
/**
* 智能体涉及到的各个模块中模型的prompt
*/
private HashMap<String, String> modelPrompt;
/**
* 主模型的聊天记录
*/
private List<Message> chatMessages;
/**
* 用户列表
*/
private List<User> users;
/**
* 已被选中的切片时间戳集合,需要及时清理
*/
private Set<Long> selectedSlices;
public MemoryGraph(String id) {
this.id = id;
this.topicNodes = new HashMap<>();
this.existedTopics = new HashMap<>();
this.currentDateDialogSlices = new HashMap<>();
this.memoryNodeCacheCounter = new ConcurrentHashMap<>();
this.memorySliceCache = new ConcurrentHashMap<>();
this.modelPrompt = new HashMap<>();
this.selectedSlices = new HashSet<>();
this.users = new ArrayList<>();
this.userDialogMap = new ConcurrentHashMap<>();
this.dialogMap = new HashMap<>();
this.dateIndex = new HashMap<>();
this.chatMessages = new ArrayList<>();
}
public static MemoryGraph getInstance(String id) throws IOException, ClassNotFoundException {
if (memoryGraph == null) {
synchronized (MemoryGraph.class) {
// 检查存储目录是否存在,不存在则创建
if (memoryGraph == null) {
createStorageDirectory();
Path filePath = getFilePath(id);
if (Files.exists(filePath)) {
memoryGraph = deserialize(id);
} else {
FileUtils.createParentDirectories(filePath.toFile().getParentFile());
memoryGraph = new MemoryGraph(id);
memoryGraph.serialize();
}
log.info("MemoryGraph注册完毕...");
}
}
}
return memoryGraph;
}
public void serialize() throws IOException {
//先写入到临时文件,如果正常写入则覆盖原文件
Path filePath = getFilePath(this.id + "-temp");
Files.createDirectories(Path.of(STORAGE_DIR));
try {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(filePath.toFile()));
oos.writeObject(this);
oos.close();
Path path = getFilePath(this.id);
Files.move(filePath, path, StandardCopyOption.REPLACE_EXISTING);
log.info("MemoryGraph 已保存到: {}", path);
} catch (IOException e) {
Files.delete(filePath);
log.error("序列化保存失败: {}", e.getMessage());
}
}
private static MemoryGraph deserialize(String id) throws IOException, ClassNotFoundException {
Path filePath = getFilePath(id);
try (ObjectInputStream ois = new ObjectInputStream(
new FileInputStream(filePath.toFile()))) {
MemoryGraph graph = (MemoryGraph) ois.readObject();
log.info("MemoryGraph 已从文件加载: {}", filePath);
return graph;
}
}
private static Path getFilePath(String id) {
return Paths.get(STORAGE_DIR, id + ".memory");
}
private static void createStorageDirectory() {
try {
Files.createDirectories(Paths.get(STORAGE_DIR));
} catch (IOException e) {
System.err.println("创建存储目录失败: " + e.getMessage());
}
}
public void insertMemory(List<String> topicPath, MemorySlice slice) {
try {
//检查是否存在当天对应的memorySlice并确定是否插入
LocalDate now = LocalDate.now();
boolean hasSlice = false;
MemoryNode node = null;
//每日刷新缓存
checkCacheDate();
//如果topicPath在memorySliceCache中存在对应缓存由于进行的插入操作则需要移除该缓存但不清除相关计数
memorySliceCache.remove(topicPath);
TopicNode lastTopicNode = generateTopicPath(topicPath);
for (MemoryNode memoryNode : lastTopicNode.getMemoryNodes()) {
if (now.equals(memoryNode.getLocalDate())) {
hasSlice = true;
node = memoryNode;
break;
}
}
if (!hasSlice) {
node = new MemoryNode();
node.setLocalDate(now);
node.setMemoryNodeId(UUID.randomUUID().toString());
node.setMemorySliceList(new CopyOnWriteArrayList<>());
lastTopicNode.getMemoryNodes().add(node);
lastTopicNode.getMemoryNodes().sort(null);
}
node.loadMemorySliceList().add(slice);
//生成relatedTopicPath
for (List<String> relatedTopic : slice.getRelatedTopics()) {
generateTopicPath(relatedTopic);
}
updateSlicePrecedent(slice);
updateDateIndex(slice);
if (!slice.isPrivate()) {
updateUserDialogMap(slice);
}
node.saveMemorySliceList();
} catch (Exception e) {
log.error("插入记忆时出错: ", e);
GlobalExceptionHandler.writeExceptionState(new GlobalException("插入记忆时出错: " + e.getLocalizedMessage()));
}
}
private void updateDateIndex(MemorySlice slice) {
String memoryId = slice.getMemoryId();
LocalDate date = LocalDate.now();
if (!dateIndex.containsKey(date)) {
HashSet<String> memoryIdSet = new HashSet<>();
memoryIdSet.add(memoryId);
dateIndex.put(date, memoryIdSet);
} else {
dateIndex.get(date).add(memoryId);
}
}
private TopicNode generateTopicPath(List<String> topicPath) {
topicPath = new ArrayList<>(topicPath);
//查看是否存在根主题节点
String rootTopic = topicPath.getFirst();
topicPath.removeFirst();
if (!topicNodes.containsKey(rootTopic)) {
synchronized (this) {
if (!topicNodes.containsKey(rootTopic)) {
TopicNode rootNode = new TopicNode();
topicNodes.put(rootTopic, rootNode);
existedTopics.put(rootTopic, new LinkedHashSet<>());
}
}
}
TopicNode current = topicNodes.get(rootTopic);
Set<String> existedTopicNodes = existedTopics.get(rootTopic);
for (String topic : topicPath) {
if (existedTopicNodes.contains(topic) && current.getTopicNodes().containsKey(topic)) {
current = current.getTopicNodes().get(topic);
} else {
TopicNode newNode = new TopicNode();
current.getTopicNodes().put(topic, newNode);
current = newNode;
current.setMemoryNodes(new CopyOnWriteArrayList<>());
current.setTopicNodes(new ConcurrentHashMap<>());
existedTopicNodes.add(topic);
}
}
return current;
}
private void updateUserDialogMap(MemorySlice slice) {
String summary = slice.getSummary();
LocalDateTime now = LocalDateTime.now();
//更新userDialogMap
//移除两天前上下文缓存(切片总结)
List<LocalDateTime> keysToRemove = new ArrayList<>();
userDialogMap.forEach((k, v) -> v.forEach((i, j) -> {
if (now.minusDays(2).isAfter(i)) {
keysToRemove.add(i);
}
}));
for (LocalDateTime dateTime : keysToRemove) {
userDialogMap.forEach((k, v) -> v.remove(dateTime));
}
//放入新缓存
userDialogMap
.computeIfAbsent(slice.getStartUserId(), k -> new ConcurrentHashMap<>())
.merge(now, summary, (oldVal, newVal) -> oldVal + " " + newVal);
}
private void updateSlicePrecedent(MemorySlice slice) {
String memoryId = slice.getMemoryId();
//查看是否切换了memoryId
if (!currentDateDialogSlices.containsKey(memoryId)) {
List<MemorySlice> memorySliceList = new ArrayList<>();
currentDateDialogSlices.clear();
currentDateDialogSlices.put(memoryId, memorySliceList);
}
//处理上下文关系
List<MemorySlice> memorySliceList = currentDateDialogSlices.get(memoryId);
if (memorySliceList.isEmpty()) {
memorySliceList.add(slice);
} else {
//排序
memorySliceList.sort(null);
MemorySlice tempSlice = memorySliceList.getLast();
//设置私密状态一致
tempSlice.setPrivate(slice.isPrivate());
//末尾切片添加当前切片的引用
tempSlice.setSliceAfter(slice);
//当前切片添加前序切片的引用
slice.setSliceBefore(tempSlice);
}
}
public MemoryResult selectMemory(String topicPathStr) {
MemoryResult memoryResult = new MemoryResult();
List<String> topicPath = List.of(topicPathStr.split("->"));
try {
List<String> path = new ArrayList<>(topicPath);
//每日刷新缓存
checkCacheDate();
//检测缓存并更新计数, 查看是否需要放入缓存
updateCacheCounter(topicPath);
//查看是否存在缓存,如果存在,则直接返回
if (memorySliceCache.containsKey(path)) {
return memorySliceCache.get(path);
}
CopyOnWriteArrayList<MemorySliceResult> targetSliceList = new CopyOnWriteArrayList<>();
String targetTopic = path.getLast();
TopicNode targetParentNode = getTargetParentNode(path, targetTopic);
List<List<String>> relatedTopics = new ArrayList<>();
//终点记忆节点
MemorySliceResult sliceResult = new MemorySliceResult();
for (MemoryNode memoryNode : targetParentNode.getTopicNodes().get(targetTopic).getMemoryNodes()) {
List<MemorySlice> endpointMemorySliceList = memoryNode.loadMemorySliceList();
for (MemorySlice memorySlice : endpointMemorySliceList) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
sliceResult.setSliceBefore(memorySlice.getSliceBefore());
sliceResult.setMemorySlice(memorySlice);
sliceResult.setSliceAfter(memorySlice.getSliceAfter());
targetSliceList.add(sliceResult);
selectedSlices.add(memorySlice.getTimestamp());
}
for (MemorySlice memorySlice : endpointMemorySliceList) {
if (memorySlice.getRelatedTopics() != null) {
relatedTopics.addAll(memorySlice.getRelatedTopics());
}
}
}
memoryResult.setMemorySliceResult(targetSliceList);
//邻近节点
List<MemorySlice> relatedMemorySlice = new ArrayList<>();
//邻近记忆节点 联系
for (List<String> relatedTopic : relatedTopics) {
List<String> tempTopicPath = new ArrayList<>(relatedTopic);
String tempTargetTopic = tempTopicPath.getLast();
TopicNode tempTargetParentNode = getTargetParentNode(tempTopicPath, tempTargetTopic);
//获取终点节点及其最新记忆节点
TopicNode tempTargetNode = tempTargetParentNode.getTopicNodes().get(tempTopicPath.getLast());
setRelatedMemorySlices(tempTargetNode, relatedMemorySlice);
}
//邻近记忆节点 父级
setRelatedMemorySlices(targetParentNode, relatedMemorySlice);
//将上述结果包装为MemoryResult
memoryResult.setRelatedMemorySliceResult(relatedMemorySlice);
//尝试更新缓存
updateCache(topicPath, memoryResult);
} catch (Exception e) {
log.error("[MemoryGraph] selectMemory error: ", e);
log.error("[MemoryGraph] 路径: {}", topicPathStr);
log.error("[MemoryGraph] 主题树: {}", getTopicTree());
memoryResult = new MemoryResult();
memoryResult.setRelatedMemorySliceResult(new ArrayList<>());
memoryResult.setMemorySliceResult(new CopyOnWriteArrayList<>());
GlobalExceptionHandler.writeExceptionState(new GlobalException(e.getLocalizedMessage()));
}
return memoryResult;
}
private void setRelatedMemorySlices(TopicNode targetParentNode, List<MemorySlice> relatedMemorySlice) throws IOException, ClassNotFoundException {
List<MemoryNode> targetParentMemoryNodes = targetParentNode.getMemoryNodes();
if (!targetParentMemoryNodes.isEmpty()) {
for (MemorySlice memorySlice : targetParentMemoryNodes.getFirst().loadMemorySliceList()) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
relatedMemorySlice.add(memorySlice);
selectedSlices.add(memorySlice.getTimestamp());
}
}
}
private void updateCache(List<String> topicPath, MemoryResult memoryResult) {
Integer tempCount = memoryNodeCacheCounter.get(topicPath);
if (tempCount == null) {
log.warn("tempCount为null? memoryNodeCacheCounter: {}; topicPath: {}", memoryNodeCacheCounter, topicPath);
return;
}
if (tempCount >= 5) {
memorySliceCache.put(topicPath, memoryResult);
}
}
private void updateCacheCounter(List<String> topicPath) {
if (memoryNodeCacheCounter.containsKey(topicPath)) {
Integer tempCount = memoryNodeCacheCounter.get(topicPath);
memoryNodeCacheCounter.put(topicPath, ++tempCount);
} else {
memoryNodeCacheCounter.put(topicPath, 1);
}
}
private void checkCacheDate() {
if (cacheDate == null || cacheDate.isBefore(LocalDate.now())) {
memorySliceCache.clear();
memoryNodeCacheCounter.clear();
cacheDate = LocalDate.now();
}
}
public MemoryResult selectMemory(LocalDate date) throws IOException, ClassNotFoundException {
MemoryResult memoryResult = new MemoryResult();
CopyOnWriteArrayList<MemorySliceResult> targetSliceList = new CopyOnWriteArrayList<>();
//加载节点并获取记忆切片列表
List<List<MemorySlice>> currentDateDialogSlices = loadSlicesByDate(date);
for (List<MemorySlice> value : currentDateDialogSlices) {
for (MemorySlice memorySlice : value) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
MemorySliceResult memorySliceResult = new MemorySliceResult();
memorySliceResult.setMemorySlice(memorySlice);
targetSliceList.add(memorySliceResult);
selectedSlices.add(memorySlice.getTimestamp());
}
}
memoryResult.setMemorySliceResult(targetSliceList);
return memoryResult;
}
private List<List<MemorySlice>> loadSlicesByDate(LocalDate date) throws IOException, ClassNotFoundException {
if (!dateIndex.containsKey(date)) {
throw new UnExistedDateIndexException("不存在的日期索引: " + date);
}
List<List<MemorySlice>> list = new ArrayList<>();
for (String memoryId : dateIndex.get(date)) {
MemoryNode memoryNode = new MemoryNode();
memoryNode.setMemoryNodeId(memoryId);
list.add(memoryNode.loadMemorySliceList());
}
return list;
}
private TopicNode getTargetParentNode(List<String> topicPath, String targetTopic) {
String topTopic = topicPath.getFirst();
if (!existedTopics.containsKey(topTopic)) {
throw new UnExistedTopicException("不存在的主题: " + topTopic);
}
TopicNode targetParentNode = topicNodes.get(topTopic);
topicPath.removeFirst();
for (String topic : topicPath) {
if (!existedTopics.get(topTopic).contains(topic)) {
throw new UnExistedTopicException("不存在的主题: " + topTopic);
}
}
//逐层查找目标主题可选取终点主题节点相邻位置的主题节点。终点记忆节点选取全部memoryNode, 邻近记忆节点选取最新日期的memoryNode
while (!targetParentNode.getTopicNodes().containsKey(targetTopic)) {
targetParentNode = targetParentNode.getTopicNodes().get(topicPath.getFirst());
topicPath.removeFirst();
}
return targetParentNode;
}
public String getTopicTree() {
StringBuilder stringBuilder = new StringBuilder();
for (Map.Entry<String, TopicNode> entry : topicNodes.entrySet()) {
String rootName = entry.getKey();
TopicNode rootNode = entry.getValue();
stringBuilder.append(rootName).append("[root]").append("\r\n");
printSubTopicsTreeFormat(rootNode, "", stringBuilder);
}
return stringBuilder.toString();
}
private void printSubTopicsTreeFormat(TopicNode node, String prefix, StringBuilder stringBuilder) {
if (node.getTopicNodes() == null || node.getTopicNodes().isEmpty()) return;
List<Map.Entry<String, TopicNode>> entries = new ArrayList<>(node.getTopicNodes().entrySet());
for (int i = 0; i < entries.size(); i++) {
boolean last = (i == entries.size() - 1);
Map.Entry<String, TopicNode> entry = entries.get(i);
stringBuilder.append(prefix).append(last ? "└── " : "├── ").append(entry.getKey()).append("[").append(entry.getValue().getMemoryNodes().size()).append("]").append("\r\n");
printSubTopicsTreeFormat(entry.getValue(), prefix + (last ? " " : ""), stringBuilder);
}
}
public void updateDialogMap(LocalDateTime dateTime, String newDialogCache) {
List<LocalDateTime> keysToRemove = new ArrayList<>();
dialogMap.forEach((k, v) -> {
if (dateTime.minusDays(2).isAfter(k)) {
keysToRemove.add(k);
}
});
for (LocalDateTime temp : keysToRemove) {
dialogMap.remove(temp);
}
keysToRemove.clear();
//放入新缓存
dialogMap.put(dateTime, newDialogCache);
}
}

View File

@@ -6,11 +6,16 @@ import lombok.extern.slf4j.Slf4j;
import work.slhaf.agent.common.chat.constant.ChatConstant;
import work.slhaf.agent.common.chat.pojo.Message;
import work.slhaf.agent.common.config.Config;
import work.slhaf.agent.common.exception_handler.GlobalExceptionHandler;
import work.slhaf.agent.common.exception_handler.pojo.GlobalException;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.pojo.MemorySliceResult;
import work.slhaf.agent.core.memory.pojo.User;
import work.slhaf.agent.core.memory.submodule.cache.CacheCore;
import work.slhaf.agent.core.memory.submodule.graph.GraphCore;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import work.slhaf.agent.core.memory.submodule.perceive.PerceiveCore;
import work.slhaf.agent.core.memory.submodule.perceive.pojo.User;
import work.slhaf.agent.shared.memory.EvaluatedSlice;
import java.io.IOException;
@@ -38,7 +43,11 @@ public class MemoryManager extends PersistableObject {
public final Lock messageLock = new ReentrantLock();
private MemoryGraph memoryGraph;
private MemoryCore memoryCore;
private CacheCore cacheCore;
private GraphCore graphCore;
private PerceiveCore perceiveCore;
private HashMap<String, List<EvaluatedSlice>> activatedSlices;
private MemoryManager() {
@@ -51,7 +60,8 @@ public class MemoryManager extends PersistableObject {
if (memoryManager == null) {
Config config = Config.getConfig();
memoryManager = new MemoryManager();
memoryManager.setMemoryGraph(MemoryGraph.getInstance(config.getAgentId()));
memoryManager.setMemoryCore(MemoryCore.getInstance(config.getAgentId()));
memoryManager.setCores();
memoryManager.setActivatedSlices(new HashMap<>());
memoryManager.setShutdownHook();
log.info("[MemoryManager] MemoryManager注册完毕...");
@@ -61,6 +71,12 @@ public class MemoryManager extends PersistableObject {
return memoryManager;
}
private void setCores() {
this.setCacheCore(this.getMemoryCore().getCacheCore());
this.setGraphCore(this.getMemoryCore().getGraphCore());
this.setPerceiveCore(this.getMemoryCore().getPerceiveCore());
}
private void setShutdownHook() {
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
try {
@@ -72,12 +88,36 @@ public class MemoryManager extends PersistableObject {
}));
}
public MemoryResult selectMemory(String path) {
return cacheFilter(memoryGraph.selectMemory(path));
public MemoryResult selectMemory(String topicPathStr) {
MemoryResult memoryResult;
List<String> topicPath = List.of(topicPathStr.split("->"));
try {
List<String> path = new ArrayList<>(topicPath);
//每日刷新缓存
cacheCore.checkCacheDate();
//检测缓存并更新计数, 查看是否需要放入缓存
cacheCore.updateCacheCounter(path);
//查看是否存在缓存,如果存在,则直接返回
if ((memoryResult = cacheCore.selectCache(path)) != null) {
return memoryResult;
}
memoryResult = graphCore.selectMemory(path);
//尝试更新缓存
cacheCore.updateCache(topicPath, memoryResult);
} catch (Exception e) {
log.error("[MemoryManager] selectMemory error: ", e);
log.error("[MemoryManager] 路径: {}", topicPathStr);
log.error("[MemoryManager] 主题树: {}", getTopicTree());
memoryResult = new MemoryResult();
memoryResult.setRelatedMemorySliceResult(new ArrayList<>());
memoryResult.setMemorySliceResult(new CopyOnWriteArrayList<>());
GlobalExceptionHandler.writeExceptionState(new GlobalException(e.getLocalizedMessage()));
}
return cacheFilter(memoryResult);
}
public MemoryResult selectMemory(LocalDate date) throws IOException, ClassNotFoundException {
return cacheFilter(memoryGraph.selectMemory(date));
return cacheFilter(graphCore.selectMemory(date));
}
private MemoryResult cacheFilter(MemoryResult memoryResult) {
@@ -92,30 +132,30 @@ public class MemoryManager extends PersistableObject {
}
public void cleanSelectedSliceFilter() {
memoryGraph.getSelectedSlices().clear();
graphCore.getSelectedSlices().clear();
}
public String getUserId(String userInfo, String nickName) {
String userId = null;
for (User user : memoryGraph.getUsers()) {
for (User user : perceiveCore.getUsers()) {
if (user.getInfo().contains(userInfo)) {
userId = user.getUuid();
}
}
if (userId == null) {
User newUser = setNewUser(userInfo, nickName);
memoryGraph.getUsers().add(newUser);
perceiveCore.getUsers().add(newUser);
userId = newUser.getUuid();
}
return userId;
}
public List<Message> getChatMessages() {
return memoryGraph.getChatMessages();
return memoryCore.getChatMessages();
}
public void setChatMessages(List<Message> chatMessages) {
memoryGraph.setChatMessages(chatMessages);
memoryCore.setChatMessages(chatMessages);
}
private static User setNewUser(String userInfo, String nickName) {
@@ -129,37 +169,50 @@ public class MemoryManager extends PersistableObject {
}
public String getTopicTree() {
return memoryGraph.getTopicTree();
return graphCore.getTopicTree();
}
public HashMap<LocalDateTime, String> getDialogMap() {
return memoryGraph.getDialogMap();
return cacheCore.getDialogMap();
}
public ConcurrentHashMap<LocalDateTime, String> getUserDialogMap(String userId) {
return memoryGraph.getUserDialogMap().get(userId);
return cacheCore.getUserDialogMap().get(userId);
}
public void insertSlice(MemorySlice memorySlice, String topicPath) {
sliceInsertLock.lock();
List<String> topicPathList = Arrays.stream(topicPath.split("->")).toList();
memoryGraph.insertMemory(topicPathList, memorySlice);
try {
//检查是否存在当天对应的memorySlice并确定是否插入
//每日刷新缓存
cacheCore.checkCacheDate();
//如果topicPath在memorySliceCache中存在对应缓存由于进行的插入操作则需要移除该缓存但不清除相关计数
cacheCore.clearCacheByTopicPath(topicPathList);
graphCore.insertMemory(topicPathList, memorySlice);
if (!memorySlice.isPrivate()) {
cacheCore.updateUserDialogMap(memorySlice);
}
} catch (Exception e) {
log.error("[MemoryManager] 插入记忆时出错: ", e);
GlobalExceptionHandler.writeExceptionState(new GlobalException("插入记忆时出错: " + e.getLocalizedMessage()));
}
log.debug("[MemoryManager] 插入切片: {}, 路径: {}", memorySlice, topicPath);
sliceInsertLock.unlock();
}
public void cleanMessage(List<Message> messages) {
messageLock.lock();
memoryGraph.getChatMessages().removeAll(messages);
memoryCore.getChatMessages().removeAll(messages);
messageLock.unlock();
}
public void updateDialogMap(LocalDateTime dateTime, String newDialogCache) {
memoryGraph.updateDialogMap(dateTime, newDialogCache);
cacheCore.updateDialogMap(dateTime, newDialogCache);
}
public void save() throws IOException {
memoryGraph.serialize();
private void save() throws IOException {
memoryCore.serialize();
}
public void updateActivatedSlices(String userId, List<EvaluatedSlice> memorySlices) {
@@ -168,7 +221,7 @@ public class MemoryManager extends PersistableObject {
}
public User getUser(String id) {
for (User user : memoryGraph.getUsers()) {
for (User user : perceiveCore.getUsers()) {
if (user.getUuid().equals(id)) {
return user;
}
@@ -189,16 +242,16 @@ public class MemoryManager extends PersistableObject {
public String getDialogMapStr() {
StringBuilder str = new StringBuilder();
memoryGraph.getDialogMap().forEach((dateTime, dialog) -> str.append("\n\n").append("[").append(dateTime).append("]\n")
cacheCore.getDialogMap().forEach((dateTime, dialog) -> str.append("\n\n").append("[").append(dateTime).append("]\n")
.append(dialog));
return str.toString();
}
public String getUserDialogMapStr(String userId) {
if (memoryGraph.getUserDialogMap().containsKey(userId)) {
if (cacheCore.getUserDialogMap().containsKey(userId)) {
StringBuilder str = new StringBuilder();
Collection<String> dialogMapValues = memoryGraph.getDialogMap().values();
memoryGraph.getUserDialogMap().get(userId).forEach((dateTime, dialog) -> {
Collection<String> dialogMapValues = cacheCore.getDialogMap().values();
cacheCore.getUserDialogMap().get(userId).forEach((dateTime, dialog) -> {
if (dialogMapValues.contains(dialog)) {
return;
}
@@ -212,7 +265,7 @@ public class MemoryManager extends PersistableObject {
}
private boolean isCacheSingleUser() {
return memoryGraph.getUserDialogMap().size() <= 1;
return cacheCore.getUserDialogMap().size() <= 1;
}
public boolean isSingleUser() {

View File

@@ -3,6 +3,7 @@ package work.slhaf.agent.core.memory.pojo;
import lombok.Data;
import lombok.EqualsAndHashCode;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import java.io.Serial;
import java.util.List;

View File

@@ -4,6 +4,7 @@ import com.alibaba.fastjson2.annotation.JSONField;
import lombok.Data;
import lombok.EqualsAndHashCode;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import java.io.Serial;

View File

@@ -0,0 +1,132 @@
package work.slhaf.agent.core.memory.submodule.cache;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.extern.slf4j.Slf4j;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import java.io.Serial;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
@EqualsAndHashCode(callSuper = true)
@Data
@Slf4j
public class CacheCore extends PersistableObject {
@Serial
private static final long serialVersionUID = 1L;
/**
* 近两日的对话总结缓存, 用于为大模型提供必要的记忆补充, hashmap以切片的存储时间为键总结为值
* 该部分作为'主LLM'system prompt常驻
* 该部分作为近两日的整体对话缓存, 不区分用户
*/
private HashMap<LocalDateTime, String> dialogMap = new HashMap<>();
/**
* 近两日的区分用户的对话总结缓存在prompt结构上比dialogMap层级深一层, dialogMap更具近两日整体对话的摘要性质
*/
private ConcurrentHashMap<String/*userId*/, ConcurrentHashMap<LocalDateTime, String>> userDialogMap = new ConcurrentHashMap<>();
/**
* memorySliceCache计数器每日清空
*/
private ConcurrentHashMap<List<String> /*触发查询的主题列表*/, Integer> memoryNodeCacheCounter = new ConcurrentHashMap<>();
/**
* 记忆切片缓存,每日清空
* 用于记录作为终点节点调用次数最多的记忆节点的切片数据
*/
private ConcurrentHashMap<List<String> /*主题路径*/, MemoryResult /*切片列表*/> memorySliceCache = new ConcurrentHashMap<>();
/**
* 缓存日期
*/
private LocalDate cacheDate;
/**
* 已被选中的切片时间戳集合,需要及时清理
*/
private Set<Long> selectedSlices = new HashSet<>();
public void updateCacheCounter(List<String> topicPath) {
if (memoryNodeCacheCounter.containsKey(topicPath)) {
Integer tempCount = memoryNodeCacheCounter.get(topicPath);
memoryNodeCacheCounter.put(topicPath, ++tempCount);
} else {
memoryNodeCacheCounter.put(topicPath, 1);
}
}
public void checkCacheDate() {
if (cacheDate == null || cacheDate.isBefore(LocalDate.now())) {
memorySliceCache.clear();
memoryNodeCacheCounter.clear();
cacheDate = LocalDate.now();
}
}
public void updateDialogMap(LocalDateTime dateTime, String newDialogCache) {
List<LocalDateTime> keysToRemove = new ArrayList<>();
dialogMap.forEach((k, v) -> {
if (dateTime.minusDays(2).isAfter(k)) {
keysToRemove.add(k);
}
});
for (LocalDateTime temp : keysToRemove) {
dialogMap.remove(temp);
}
keysToRemove.clear();
//放入新缓存
dialogMap.put(dateTime, newDialogCache);
}
public void updateCache(List<String> topicPath, MemoryResult memoryResult) {
Integer tempCount = memoryNodeCacheCounter.get(topicPath);
if (tempCount == null) {
log.warn("[CacheCore] tempCount为null? memoryNodeCacheCounter: {}; topicPath: {}", memoryNodeCacheCounter, topicPath);
return;
}
if (tempCount >= 5) {
memorySliceCache.put(topicPath, memoryResult);
}
}
public void updateUserDialogMap(MemorySlice slice) {
String summary = slice.getSummary();
LocalDateTime now = LocalDateTime.now();
//更新userDialogMap
//移除两天前上下文缓存(切片总结)
List<LocalDateTime> keysToRemove = new ArrayList<>();
userDialogMap.forEach((k, v) -> v.forEach((i, j) -> {
if (now.minusDays(2).isAfter(i)) {
keysToRemove.add(i);
}
}));
for (LocalDateTime dateTime : keysToRemove) {
userDialogMap.forEach((k, v) -> v.remove(dateTime));
}
//放入新缓存
userDialogMap
.computeIfAbsent(slice.getStartUserId(), k -> new ConcurrentHashMap<>())
.merge(now, summary, (oldVal, newVal) -> oldVal + " " + newVal);
}
public void clearCacheByTopicPath(List<String> topicPath) {
memorySliceCache.remove(topicPath);
}
public MemoryResult selectCache(List<String> path) {
if (memorySliceCache.containsKey(path)) {
return memorySliceCache.get(path);
}
return null;
}
}

View File

@@ -0,0 +1,303 @@
package work.slhaf.agent.core.memory.submodule.graph;
import lombok.Data;
import lombok.EqualsAndHashCode;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.exception.UnExistedDateIndexException;
import work.slhaf.agent.core.memory.exception.UnExistedTopicException;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.pojo.MemorySliceResult;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import work.slhaf.agent.core.memory.submodule.graph.pojo.node.MemoryNode;
import work.slhaf.agent.core.memory.submodule.graph.pojo.node.TopicNode;
import java.io.IOException;
import java.io.Serial;
import java.time.LocalDate;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
@EqualsAndHashCode(callSuper = true)
@Data
public class GraphCore extends PersistableObject {
@Serial
private static final long serialVersionUID = 1L;
/**
* key: 根主题名称 value: 根主题节点
*/
private HashMap<String, TopicNode> topicNodes = new HashMap<>();
/**
* 用于存储已存在的主题列表,便于记忆查找, 使用根主题名称作为键, 子主题名称集合为值
* 该部分在'主题提取LLM'的system prompt中常驻
*/
private HashMap<String /*根主题名*/, LinkedHashSet<String> /*子主题列表*/> existedTopics = new HashMap<>();
/**
* 临时的同一对话切片容器, 用于为同一对话内的不同切片提供更新上下文的场所
*/
private HashMap<String /*对话id, 即slice中的字段'memoryId'*/, List<MemorySlice>> currentDateDialogSlices = new HashMap<>();
/**
* 记忆节点的日期索引, 同一日期内按照对话id区分
*/
private HashMap<LocalDate, Set<String>> dateIndex = new HashMap<>();
/**
* 已被选中的切片时间戳集合,需要及时清理
*/
private Set<Long> selectedSlices = new HashSet<>();
public MemoryResult selectMemory(LocalDate date) throws IOException, ClassNotFoundException {
MemoryResult memoryResult = new MemoryResult();
CopyOnWriteArrayList<MemorySliceResult> targetSliceList = new CopyOnWriteArrayList<>();
//加载节点并获取记忆切片列表
List<List<MemorySlice>> currentDateDialogSlices = loadSlicesByDate(date);
for (List<MemorySlice> value : currentDateDialogSlices) {
for (MemorySlice memorySlice : value) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
MemorySliceResult memorySliceResult = new MemorySliceResult();
memorySliceResult.setMemorySlice(memorySlice);
targetSliceList.add(memorySliceResult);
selectedSlices.add(memorySlice.getTimestamp());
}
}
memoryResult.setMemorySliceResult(targetSliceList);
return memoryResult;
}
private List<List<MemorySlice>> loadSlicesByDate(LocalDate date) throws IOException, ClassNotFoundException {
if (!dateIndex.containsKey(date)) {
throw new UnExistedDateIndexException("不存在的日期索引: " + date);
}
List<List<MemorySlice>> list = new ArrayList<>();
for (String memoryId : dateIndex.get(date)) {
MemoryNode memoryNode = new MemoryNode();
memoryNode.setMemoryNodeId(memoryId);
list.add(memoryNode.loadMemorySliceList());
}
return list;
}
public String getTopicTree() {
StringBuilder stringBuilder = new StringBuilder();
for (Map.Entry<String, TopicNode> entry : topicNodes.entrySet()) {
String rootName = entry.getKey();
TopicNode rootNode = entry.getValue();
stringBuilder.append(rootName).append("[root]").append("\r\n");
printSubTopicsTreeFormat(rootNode, "", stringBuilder);
}
return stringBuilder.toString();
}
private void printSubTopicsTreeFormat(TopicNode node, String prefix, StringBuilder stringBuilder) {
if (node.getTopicNodes() == null || node.getTopicNodes().isEmpty()) return;
List<Map.Entry<String, TopicNode>> entries = new ArrayList<>(node.getTopicNodes().entrySet());
for (int i = 0; i < entries.size(); i++) {
boolean last = (i == entries.size() - 1);
Map.Entry<String, TopicNode> entry = entries.get(i);
stringBuilder.append(prefix).append(last ? "└── " : "├── ").append(entry.getKey()).append("[").append(entry.getValue().getMemoryNodes().size()).append("]").append("\r\n");
printSubTopicsTreeFormat(entry.getValue(), prefix + (last ? " " : ""), stringBuilder);
}
}
public void insertMemory(List<String> topicPath, MemorySlice slice) throws IOException, ClassNotFoundException {
LocalDate now = LocalDate.now();
boolean hasSlice = false;
MemoryNode node = null;
TopicNode lastTopicNode = generateTopicPath(topicPath);
for (MemoryNode memoryNode : lastTopicNode.getMemoryNodes()) {
if (now.equals(memoryNode.getLocalDate())) {
hasSlice = true;
node = memoryNode;
break;
}
}
if (!hasSlice) {
node = new MemoryNode();
node.setLocalDate(now);
node.setMemoryNodeId(UUID.randomUUID().toString());
node.setMemorySliceList(new CopyOnWriteArrayList<>());
lastTopicNode.getMemoryNodes().add(node);
lastTopicNode.getMemoryNodes().sort(null);
}
node.loadMemorySliceList().add(slice);
//生成relatedTopicPath
for (List<String> relatedTopic : slice.getRelatedTopics()) {
generateTopicPath(relatedTopic);
}
updateSlicePrecedent(slice);
updateDateIndex(slice);
node.saveMemorySliceList();
}
private TopicNode generateTopicPath(List<String> topicPath) {
topicPath = new ArrayList<>(topicPath);
//查看是否存在根主题节点
String rootTopic = topicPath.getFirst();
topicPath.removeFirst();
if (!topicNodes.containsKey(rootTopic)) {
synchronized (this) {
if (!topicNodes.containsKey(rootTopic)) {
TopicNode rootNode = new TopicNode();
topicNodes.put(rootTopic, rootNode);
existedTopics.put(rootTopic, new LinkedHashSet<>());
}
}
}
TopicNode current = topicNodes.get(rootTopic);
Set<String> existedTopicNodes = existedTopics.get(rootTopic);
for (String topic : topicPath) {
if (existedTopicNodes.contains(topic) && current.getTopicNodes().containsKey(topic)) {
current = current.getTopicNodes().get(topic);
} else {
TopicNode newNode = new TopicNode();
current.getTopicNodes().put(topic, newNode);
current = newNode;
current.setMemoryNodes(new CopyOnWriteArrayList<>());
current.setTopicNodes(new ConcurrentHashMap<>());
existedTopicNodes.add(topic);
}
}
return current;
}
private void updateSlicePrecedent(MemorySlice slice) {
String memoryId = slice.getMemoryId();
//查看是否切换了memoryId
if (!currentDateDialogSlices.containsKey(memoryId)) {
List<MemorySlice> memorySliceList = new ArrayList<>();
currentDateDialogSlices.clear();
currentDateDialogSlices.put(memoryId, memorySliceList);
}
//处理上下文关系
List<MemorySlice> memorySliceList = currentDateDialogSlices.get(memoryId);
if (memorySliceList.isEmpty()) {
memorySliceList.add(slice);
} else {
//排序
memorySliceList.sort(null);
MemorySlice tempSlice = memorySliceList.getLast();
//设置私密状态一致
tempSlice.setPrivate(slice.isPrivate());
//末尾切片添加当前切片的引用
tempSlice.setSliceAfter(slice);
//当前切片添加前序切片的引用
slice.setSliceBefore(tempSlice);
}
}
private void updateDateIndex(MemorySlice slice) {
String memoryId = slice.getMemoryId();
LocalDate date = LocalDate.now();
if (!dateIndex.containsKey(date)) {
HashSet<String> memoryIdSet = new HashSet<>();
memoryIdSet.add(memoryId);
dateIndex.put(date, memoryIdSet);
} else {
dateIndex.get(date).add(memoryId);
}
}
public MemoryResult selectMemory(List<String> path) throws IOException, ClassNotFoundException {
MemoryResult memoryResult = new MemoryResult();
CopyOnWriteArrayList<MemorySliceResult> targetSliceList = new CopyOnWriteArrayList<>();
String targetTopic = path.getLast();
TopicNode targetParentNode = getTargetParentNode(path, targetTopic);
List<List<String>> relatedTopics = new ArrayList<>();
//终点记忆节点
MemorySliceResult sliceResult = new MemorySliceResult();
for (MemoryNode memoryNode : targetParentNode.getTopicNodes().get(targetTopic).getMemoryNodes()) {
List<MemorySlice> endpointMemorySliceList = memoryNode.loadMemorySliceList();
for (MemorySlice memorySlice : endpointMemorySliceList) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
sliceResult.setSliceBefore(memorySlice.getSliceBefore());
sliceResult.setMemorySlice(memorySlice);
sliceResult.setSliceAfter(memorySlice.getSliceAfter());
targetSliceList.add(sliceResult);
selectedSlices.add(memorySlice.getTimestamp());
}
for (MemorySlice memorySlice : endpointMemorySliceList) {
if (memorySlice.getRelatedTopics() != null) {
relatedTopics.addAll(memorySlice.getRelatedTopics());
}
}
}
memoryResult.setMemorySliceResult(targetSliceList);
//邻近节点
List<MemorySlice> relatedMemorySlice = new ArrayList<>();
//邻近记忆节点 联系
for (List<String> relatedTopic : relatedTopics) {
List<String> tempTopicPath = new ArrayList<>(relatedTopic);
String tempTargetTopic = tempTopicPath.getLast();
TopicNode tempTargetParentNode = getTargetParentNode(tempTopicPath, tempTargetTopic);
//获取终点节点及其最新记忆节点
TopicNode tempTargetNode = tempTargetParentNode.getTopicNodes().get(tempTopicPath.getLast());
setRelatedMemorySlices(tempTargetNode, relatedMemorySlice);
}
//邻近记忆节点 父级
setRelatedMemorySlices(targetParentNode, relatedMemorySlice);
//将上述结果包装为MemoryResult
memoryResult.setRelatedMemorySliceResult(relatedMemorySlice);
return memoryResult;
}
private void setRelatedMemorySlices(TopicNode targetParentNode, List<MemorySlice> relatedMemorySlice) throws IOException, ClassNotFoundException {
List<MemoryNode> targetParentMemoryNodes = targetParentNode.getMemoryNodes();
if (!targetParentMemoryNodes.isEmpty()) {
for (MemorySlice memorySlice : targetParentMemoryNodes.getFirst().loadMemorySliceList()) {
if (selectedSlices.contains(memorySlice.getTimestamp())) {
continue;
}
relatedMemorySlice.add(memorySlice);
selectedSlices.add(memorySlice.getTimestamp());
}
}
}
private TopicNode getTargetParentNode(List<String> topicPath, String targetTopic) {
String topTopic = topicPath.getFirst();
if (!existedTopics.containsKey(topTopic)) {
throw new UnExistedTopicException("不存在的主题: " + topTopic);
}
TopicNode targetParentNode = topicNodes.get(topTopic);
topicPath.removeFirst();
for (String topic : topicPath) {
if (!existedTopics.get(topTopic).contains(topic)) {
throw new UnExistedTopicException("不存在的主题: " + topTopic);
}
}
//逐层查找目标主题
while (!targetParentNode.getTopicNodes().containsKey(targetTopic)) {
targetParentNode = targetParentNode.getTopicNodes().get(topicPath.getFirst());
topicPath.removeFirst();
}
return targetParentNode;
}
}

View File

@@ -1,4 +1,4 @@
package work.slhaf.agent.core.memory.pojo;
package work.slhaf.agent.core.memory.submodule.graph.pojo;
import lombok.Data;
import lombok.EqualsAndHashCode;

View File

@@ -1,11 +1,11 @@
package work.slhaf.agent.core.memory.node;
package work.slhaf.agent.core.memory.submodule.graph.pojo.node;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.extern.slf4j.Slf4j;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.exception.NullSliceListException;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import java.io.*;
import java.nio.file.Files;

View File

@@ -1,4 +1,4 @@
package work.slhaf.agent.core.memory.node;
package work.slhaf.agent.core.memory.submodule.graph.pojo.node;
import lombok.Data;
import lombok.EqualsAndHashCode;

View File

@@ -0,0 +1,24 @@
package work.slhaf.agent.core.memory.submodule.perceive;
import lombok.Data;
import lombok.EqualsAndHashCode;
import work.slhaf.agent.common.serialize.PersistableObject;
import work.slhaf.agent.core.memory.submodule.perceive.pojo.User;
import java.io.Serial;
import java.util.ArrayList;
import java.util.List;
@EqualsAndHashCode(callSuper = true)
@Data
public class PerceiveCore extends PersistableObject {
@Serial
private static final long serialVersionUID = 1L;
/**
* 用户列表
*/
private List<User> users = new ArrayList<>();
}

View File

@@ -1,4 +1,4 @@
package work.slhaf.agent.core.memory.pojo;
package work.slhaf.agent.core.memory.submodule.perceive.pojo;
import lombok.Data;
import lombok.EqualsAndHashCode;

View File

@@ -3,15 +3,13 @@ package work.slhaf.agent.module.modules.memory.selector;
import com.alibaba.fastjson2.JSONObject;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import work.slhaf.agent.common.exception_handler.GlobalExceptionHandler;
import work.slhaf.agent.common.exception_handler.pojo.GlobalException;
import work.slhaf.agent.core.interaction.data.context.InteractionContext;
import work.slhaf.agent.core.interaction.module.InteractionModule;
import work.slhaf.agent.core.memory.MemoryManager;
import work.slhaf.agent.core.memory.exception.UnExistedDateIndexException;
import work.slhaf.agent.core.memory.exception.UnExistedTopicException;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import work.slhaf.agent.core.session.SessionManager;
import work.slhaf.agent.module.common.AppendPromptData;
import work.slhaf.agent.module.common.PreModuleActions;
@@ -29,8 +27,6 @@ import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import static work.slhaf.agent.common.util.ExtractUtil.fixTopicPath;
@Data
@Slf4j
public class MemorySelector implements InteractionModule, PreModuleActions {

View File

@@ -9,8 +9,8 @@ import lombok.extern.slf4j.Slf4j;
import work.slhaf.agent.common.thread.InteractionThreadPoolExecutor;
import work.slhaf.agent.core.memory.MemoryManager;
import work.slhaf.agent.core.memory.pojo.MemoryResult;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.pojo.MemorySliceResult;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import work.slhaf.agent.module.common.Model;
import work.slhaf.agent.module.common.ModelConstant;
import work.slhaf.agent.module.modules.memory.selector.evaluator.data.EvaluatorBatchInput;

View File

@@ -9,7 +9,7 @@ import work.slhaf.agent.common.thread.InteractionThreadPoolExecutor;
import work.slhaf.agent.core.interaction.data.context.InteractionContext;
import work.slhaf.agent.core.interaction.module.InteractionModule;
import work.slhaf.agent.core.memory.MemoryManager;
import work.slhaf.agent.core.memory.pojo.MemorySlice;
import work.slhaf.agent.core.memory.submodule.graph.pojo.MemorySlice;
import work.slhaf.agent.core.session.SessionManager;
import work.slhaf.agent.module.modules.memory.selector.extractor.MemorySelectExtractor;
import work.slhaf.agent.module.modules.memory.updater.summarizer.MemorySummarizer;
@@ -21,7 +21,6 @@ import java.time.LocalDateTime;
import java.util.*;
import java.util.concurrent.Callable;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
import static work.slhaf.agent.common.util.ExtractUtil.extractUserId;

View File

@@ -0,0 +1,24 @@
package work.slhaf.agent.module.modules.perceive.selector;
import work.slhaf.agent.core.interaction.data.context.InteractionContext;
import work.slhaf.agent.core.interaction.module.InteractionModule;
import java.io.IOException;
public class PerceiveSelector implements InteractionModule {
private static PerceiveSelector perceiveSelector;
public static PerceiveSelector getInstance() throws IOException, ClassNotFoundException {
if (perceiveSelector == null) {
perceiveSelector = new PerceiveSelector();
}
return perceiveSelector;
}
@Override
public void execute(InteractionContext context) throws IOException, ClassNotFoundException {
}
}

View File

@@ -1,4 +1,4 @@
package work.slhaf.agent.module.modules.perceive.static_extractor;
package work.slhaf.agent.module.modules.perceive.updater.static_extractor;
import cn.hutool.json.JSONUtil;
import com.alibaba.fastjson2.JSONObject;
@@ -7,7 +7,7 @@ import lombok.EqualsAndHashCode;
import work.slhaf.agent.common.chat.pojo.ChatResponse;
import work.slhaf.agent.module.common.Model;
import work.slhaf.agent.module.common.ModelConstant;
import work.slhaf.agent.module.modules.perceive.static_extractor.data.StaticExtractInput;
import work.slhaf.agent.module.modules.perceive.updater.static_extractor.data.StaticExtractInput;
import java.util.HashMap;
import java.util.Map;

View File

@@ -1,4 +1,4 @@
package work.slhaf.agent.module.modules.perceive.static_extractor.data;
package work.slhaf.agent.module.modules.perceive.updater.static_extractor.data;
import lombok.Builder;
import lombok.Data;

View File

@@ -1,60 +0,0 @@
package memory;
import work.slhaf.agent.core.memory.MemoryGraph;
import work.slhaf.agent.core.memory.node.TopicNode;
import java.time.LocalDate;
import java.util.HashMap;
import java.util.concurrent.ConcurrentHashMap;
public class MemoryTest {
//@Test
public void test1() {
String basicCharacter = "";
MemoryGraph graph = new MemoryGraph("test");
HashMap<String, TopicNode> topicMap = new HashMap<>();
TopicNode root1 = new TopicNode();
root1.setTopicNodes(new ConcurrentHashMap<>());
TopicNode sub1 = new TopicNode();
sub1.setTopicNodes(new ConcurrentHashMap<>());
TopicNode sub2 = new TopicNode();
sub2.setTopicNodes(new ConcurrentHashMap<>());
TopicNode subsub1 = new TopicNode();
subsub1.setTopicNodes(new ConcurrentHashMap<>());
// 构造结构root -> sub1 -> subsub1, root -> sub2
sub1.getTopicNodes().put("子子主题1", subsub1);
root1.getTopicNodes().put("子主题1", sub1);
root1.getTopicNodes().put("子主题2", sub2);
topicMap.put("根主题1", root1);
// 添加 root2
TopicNode root2 = new TopicNode();
root2.setTopicNodes(new ConcurrentHashMap<>());
TopicNode sub3 = new TopicNode();
sub3.setTopicNodes(new ConcurrentHashMap<>());
// 构造结构root2 -> sub3
root2.getTopicNodes().put("子主题3", sub3);
topicMap.put("根主题2", root2);
// 输出
graph.setTopicNodes(topicMap);
System.out.println(graph.getTopicTree());
}
// @Test
public void test2(){
System.out.println(LocalDate.now());
}
}