How to gracefully implement "user confirmation before tool execution"? ChatResponse cannot be passed across requests

[EkkoWH]

I am currently using Spring AI to build a front-end and back-end integrated AI Agent system. One typical use case I need to implement is:
The user needs to confirm a tool call via the UI before it is executed by the backend.

I have already learned and used internalToolExecutionEnabled = false to prevent Spring from executing the tool call automatically, so that the tool execution can be manually triggered. However, I encountered the following difficulty in implementation:

ToolCallingManager#executeToolCalls(Prompt prompt, ChatResponse chatResponse)
requires a ChatResponse object to execute tools, which is difficult to persist across multiple interactions.

Specific Issue:
User initiates a conversation → receives a ChatResponse containing tool calls → tools are not executed automatically

The frontend renders the tool call information and waits for the user to click “Confirm Execution”

On the second request when the user confirms → it becomes very difficult to retrieve the original ChatResponse because:

Spring AI’s memory only stores Message objects and cannot reconstruct the full ChatResponse

If I cache the ChatResponse for future confirmation, it results in complicated state management, such as needing to persist “pending tool call state” when the user refreshes the page, closes it, or delays the confirmation

My Questions:
Is it recommended to store the ChatResponse in a database? This seems too heavy-weight since ChatResponse includes a lot of contextual information.

Is there a better way to enable the front-end and back-end to coordinate using a “lightweight reference” to restore the previous ToolCall context?

Is Spring AI planning to provide a “lightweight identifier-based caching or restoration mechanism” for ToolCalls? Something like structuring ToolCall (with parameters) into an encodable format, so that the follow-up request only needs to carry an ID or snippet?

Looking forward to suggestions from the community or maintainers. Thank you!

[luyi404]

Same Question

[davidbellem]

Also running into the same issue. Has anybody found a solution to this?

[Kaiser000]

maybe put chatresponse.result.output to database,this is the message of toolCall

[davidbellem]

Had a look at the code, and you don't need to store a lot from ChatResponse.

I got it to work by creating the ChatResponse in the following way:

AssistantMessage assistantMessage = new AssistantMessage("", Map.of(), List.of(new AssistantMessage.ToolCall(tool.getToolId(), tool.getToolType(), tool.getToolName(), tool.getToolParameters())));
var chatResponse = new ChatResponse(List.of(new Generation(assistantMessage)));

Here tool is an entity I saved in the database. You only need to save the id, type, name and parameters. These you store from the original chatResponse.

In total it looks something like this:
Save the values after this snippet:

if (chatResponse.hasToolCalls()) {
            Optional<Generation> toolCallGeneration = chatResponse.getResults()
                    .stream()
                    .filter(g -> !CollectionUtils.isEmpty(g.getOutput().getToolCalls()))
                    .findFirst();

            if (toolCallGeneration.isEmpty()) {
                throw new IllegalStateException("No tool call requested by the chat model");
            }

            AssistantMessage assistantMessage = toolCallGeneration.get().getOutput();

            AssistantMessage.ToolCall toolCall = assistantMessage.getToolCalls().getFirst();

In the ToolCall Record you find everything you need.

Later, continue

Prompt promptWithMemory = new Prompt(chatMemory.get(getConversationId(tool)), chatOptions);
AssistantMessage assistantMessage = new AssistantMessage("", Map.of(), List.of(new AssistantMessage.ToolCall(tool.getToolId(), tool.getToolType(), tool.getToolName(), tool.getToolParameters())));

var chatResponse = new ChatResponse(List.of(new Generation(assistantMessage)));
        
ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(promptWithMemory,
       chatReponse);
chatMemory.add(conversationId, toolExecutionResult.conversationHistory()
        .get(toolExecutionResult.conversationHistory().size() - 1));
promptWithMemory = new Prompt(chatMemory.get(conversationId), chatOptions);
var chatResponse = chatModel.call(promptWithMemory);
chatMemory.add(conversationId, chatResponse.getResult().getOutput());

[EkkoWH]

I believe this is a great approach. Thank you for your valuable insights. I look forward to any future ideas you may have.

[davidbellem]

Just a word of warning: Currently the JdbcChatMemoryRepository implementation that comes with Spring AI does not support tool calls (#3342). Some already suggested a fix here.

In case someone is struggling with this as well. This is what I had to do to make it work, maybe it will save someone some time.

Create a file called JdbcChatMemoryRepository.java in your project that contains the fix (based on the fix mentioned above):

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.memory.ChatMemoryRepository;
import org.springframework.ai.chat.memory.repository.jdbc.JdbcChatMemoryRepositoryDialect;
import org.springframework.ai.chat.messages.*;
import org.springframework.ai.model.ModelOptionsUtils;
import org.springframework.jdbc.core.BatchPreparedStatementSetter;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.RowMapper;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.lang.Nullable;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import org.springframework.util.Assert;

import javax.sql.DataSource;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicLong;

/**
 * An implementation of {@link ChatMemoryRepository} for JDBC.
 *
 * @author Jonathan Leijendekker
 * @author Thomas Vitale
 * @author Linar Abzaltdinov
 * @author Mark Pollack
 * @author Yanming Zhou
 * @author DoHoon Kim
 * @since 1.0.0
 */
public final class JdbcChatMemoryRepository implements ChatMemoryRepository {

	private final JdbcTemplate jdbcTemplate;

	private final TransactionTemplate transactionTemplate;

	private final JdbcChatMemoryRepositoryDialect dialect;

	private static final Logger logger = LoggerFactory.getLogger(JdbcChatMemoryRepository.class);

	private JdbcChatMemoryRepository(JdbcTemplate jdbcTemplate, JdbcChatMemoryRepositoryDialect dialect,
                                     PlatformTransactionManager txManager) {
		Assert.notNull(jdbcTemplate, "jdbcTemplate cannot be null");
		Assert.notNull(dialect, "dialect cannot be null");
		this.jdbcTemplate = jdbcTemplate;
		this.dialect = dialect;
		this.transactionTemplate = new TransactionTemplate(
				txManager != null ? txManager : new DataSourceTransactionManager(jdbcTemplate.getDataSource()));
	}

	@Override
	public List<String> findConversationIds() {
		return this.jdbcTemplate.queryForList(this.dialect.getSelectConversationIdsSql(), String.class);
	}

	@Override
	public List<Message> findByConversationId(String conversationId) {
		Assert.hasText(conversationId, "conversationId cannot be null or empty");
		return this.jdbcTemplate.query(this.dialect.getSelectMessagesSql(), new MessageRowMapper(), conversationId);
	}

	@Override
	public void saveAll(String conversationId, List<Message> messages) {
		Assert.hasText(conversationId, "conversationId cannot be null or empty");
		Assert.notNull(messages, "messages cannot be null");
		Assert.noNullElements(messages, "messages cannot contain null elements");

		this.transactionTemplate.execute(status -> {
			deleteByConversationId(conversationId);
			this.jdbcTemplate.batchUpdate(this.dialect.getInsertMessageSql(),
					new AddBatchPreparedStatement(conversationId, messages));
			return null;
		});
	}

	@Override
	public void deleteByConversationId(String conversationId) {
		Assert.hasText(conversationId, "conversationId cannot be null or empty");
		this.jdbcTemplate.update(this.dialect.getDeleteMessagesSql(), conversationId);
	}

	public static Builder builder() {
		return new Builder();
	}

	private record AddBatchPreparedStatement(String conversationId, List<Message> messages,
			AtomicLong instantSeq) implements BatchPreparedStatementSetter {

		private AddBatchPreparedStatement(String conversationId, List<Message> messages) {
			this(conversationId, messages, new AtomicLong(Instant.now().toEpochMilli()));
		}

		@Override
		public void setValues(PreparedStatement ps, int i) throws SQLException {
			var message = this.messages.get(i);

			ps.setString(1, this.conversationId);
			ps.setString(2, message.getText());
			ps.setString(3, message.getMessageType().name());

			// Handle tool_calls column
			String toolCallsJson = null;
			if (message instanceof ToolResponseMessage toolResponseMessage) {
				toolCallsJson = ModelOptionsUtils.toJsonString(toolResponseMessage.getResponses());
			} else if (message instanceof AssistantMessage assistantMessage) {
				toolCallsJson = ModelOptionsUtils.toJsonString(assistantMessage.getToolCalls());
			}
			ps.setString(4, toolCallsJson);

			ps.setTimestamp(5, new Timestamp(this.instantSeq.getAndIncrement()));
		}

		@Override
		public int getBatchSize() {
			return this.messages.size();
		}
	}

	private static class MessageRowMapper implements RowMapper<Message> {

		@Override
		@Nullable
		public Message mapRow(ResultSet rs, int i) throws SQLException {
			var content = rs.getString(1);
			var type = MessageType.valueOf(rs.getString(2));
			var toolCallsJson = rs.getString(3);

			return switch (type) {
				case USER -> new UserMessage(content);
				case ASSISTANT -> {
					List<AssistantMessage.ToolCall> toolCalls = List.of();
					if (toolCallsJson != null && !toolCallsJson.isBlank()) {
						try {
							toolCalls = ModelOptionsUtils.OBJECT_MAPPER.readValue(toolCallsJson,
									ModelOptionsUtils.OBJECT_MAPPER.getTypeFactory()
										.constructCollectionType(List.class, AssistantMessage.ToolCall.class));
						}
						catch (Exception e) {
							logger.warn("Failed to deserialize tool calls JSON: {}", toolCallsJson, e);
						}
					}
					yield new AssistantMessage(content, Map.of(), toolCalls);
				}
				case SYSTEM -> new SystemMessage(content);
				// The content is always stored empty for ToolResponseMessages.
				// If we want to capture the actual content, we need to extend
				// AddBatchPreparedStatement to support it.
				case TOOL -> {
					List<ToolResponseMessage.ToolResponse> toolCalls = List.of();
					if (toolCallsJson != null && !toolCallsJson.isBlank()) {
						try {
							toolCalls = ModelOptionsUtils.OBJECT_MAPPER.readValue(toolCallsJson,
									ModelOptionsUtils.OBJECT_MAPPER.getTypeFactory()
											.constructCollectionType(List.class, ToolResponseMessage.ToolResponse.class));
						} catch (Exception e) {
							logger.warn("Failed to deserialize tool calls JSON: {}", toolCallsJson, e);
						}
					}
					yield new ToolResponseMessage(toolCalls);
				}
			};
		}

	}

	public static final class Builder {

		private JdbcTemplate jdbcTemplate;

		private JdbcChatMemoryRepositoryDialect dialect;

		private DataSource dataSource;

		private PlatformTransactionManager platformTransactionManager;

		private static final Logger logger = LoggerFactory.getLogger(Builder.class);

		private Builder() {
		}

		public Builder jdbcTemplate(JdbcTemplate jdbcTemplate) {
			this.jdbcTemplate = jdbcTemplate;
			return this;
		}

		public Builder dialect(JdbcChatMemoryRepositoryDialect dialect) {
			this.dialect = dialect;
			return this;
		}

		public Builder dataSource(DataSource dataSource) {
			this.dataSource = dataSource;
			return this;
		}

		public Builder transactionManager(PlatformTransactionManager txManager) {
			this.platformTransactionManager = txManager;
			return this;
		}

		public JdbcChatMemoryRepository build() {
			DataSource effectiveDataSource = resolveDataSource();
			JdbcChatMemoryRepositoryDialect effectiveDialect = resolveDialect(effectiveDataSource);
			return new JdbcChatMemoryRepository(resolveJdbcTemplate(), effectiveDialect,
					this.platformTransactionManager);
		}

		private JdbcTemplate resolveJdbcTemplate() {
			if (this.jdbcTemplate != null) {
				return this.jdbcTemplate;
			}
			if (this.dataSource != null) {
				return new JdbcTemplate(this.dataSource);
			}
			throw new IllegalArgumentException("DataSource must be set (either via dataSource() or jdbcTemplate())");
		}

		private DataSource resolveDataSource() {
			if (this.dataSource != null) {
				return this.dataSource;
			}
			if (this.jdbcTemplate != null && this.jdbcTemplate.getDataSource() != null) {
				return this.jdbcTemplate.getDataSource();
			}
			throw new IllegalArgumentException("DataSource must be set (either via dataSource() or jdbcTemplate())");
		}

		private JdbcChatMemoryRepositoryDialect resolveDialect(DataSource dataSource) {
			if (this.dialect == null) {
				try {
					return JdbcChatMemoryRepositoryDialect.from(dataSource);
				}
				catch (Exception ex) {
					throw new IllegalStateException("Could not detect dialect from datasource", ex);
				}
			}
			else {
				warnIfDialectMismatch(dataSource, this.dialect);
				return this.dialect;
			}
		}

		/**
		 * Logs a warning if the explicitly set dialect differs from the dialect detected
		 * from the DataSource.
		 */
		private void warnIfDialectMismatch(DataSource dataSource, JdbcChatMemoryRepositoryDialect explicitDialect) {
			try {
				JdbcChatMemoryRepositoryDialect detected = JdbcChatMemoryRepositoryDialect.from(dataSource);
				if (!detected.getClass().equals(explicitDialect.getClass())) {
					logger.warn("Explicitly set dialect {} will be used instead of detected dialect {} from datasource",
							explicitDialect.getClass().getSimpleName(), detected.getClass().getSimpleName());
				}
			}
			catch (Exception ex) {
				logger.debug("Could not detect dialect from datasource", ex);
			}
		}

	}

}

Then create your own version of PostgresChatMemoryRepositoryDialect.java as well (or whatever dialect you are using)

import org.springframework.ai.chat.memory.repository.jdbc.JdbcChatMemoryRepositoryDialect;

/**
 * Dialect for Postgres.
 *
 * @author Mark Pollack
 * @author DoHoon Kim
 * @since 1.0.0
 */
public class PostgresChatMemoryRepositoryDialect implements JdbcChatMemoryRepositoryDialect {

	@Override
	public String getSelectMessagesSql() {
		return "SELECT content, type, tool_calls FROM SPRING_AI_CHAT_MEMORY WHERE conversation_id = ? ORDER BY \"timestamp\"";
	}

	@Override
	public String getInsertMessageSql() {
		return "INSERT INTO SPRING_AI_CHAT_MEMORY (conversation_id, content, type, tool_calls, \"timestamp\") VALUES (?, ?, ?, ?::jsonb, ?)";
	}

	@Override
	public String getSelectConversationIdsSql() {
		return "SELECT DISTINCT conversation_id FROM SPRING_AI_CHAT_MEMORY";
	}

	@Override
	public String getDeleteMessagesSql() {
		return "DELETE FROM SPRING_AI_CHAT_MEMORY WHERE conversation_id = ?";
	}

}

Make sure to adapt your DB schema creation to

CREATE TABLE IF NOT EXISTS SPRING_AI_CHAT_MEMORY
(
    conversation_id VARCHAR(36) NOT NULL,
    content         TEXT NULL,
    type            VARCHAR(10) NOT NULL CHECK (type IN ('USER', 'ASSISTANT', 'SYSTEM', 'TOOL')),
    tool_calls JSONB NULL,
    "timestamp"     TIMESTAMP   NOT NULL
);

CREATE INDEX IF NOT EXISTS SPRING_AI_CHAT_MEMORY_CONVERSATION_ID_TIMESTAMP_IDX
    ON SPRING_AI_CHAT_MEMORY (conversation_id, "timestamp");

Lastly, setup your Chatmemory with your own custom JdbcChatMemoryRepository and PostgresChatMemoryRepositoryDialect (make sure to import your own, not the spring AI one!)

ChatMemoryRepository chatMemoryRepository = JdbcChatMemoryRepository.builder()
        .jdbcTemplate(jdbcTemplate)
        .dialect(new PostgresChatMemoryRepositoryDialect())
        .build();
this.chatMemory = MessageWindowChatMemory.builder()
        .chatMemoryRepository(chatMemoryRepository)
        .maxMessages(20)
        .build();