Each chat message sent has a token value based on the total content of all the messages in the chat session. As each chat session logs messages, the token total for the chat session increases as the chat session is used.
To automatically manage chat session tokens, methods in the Chats method category have Trim Tokens? and Custom Token Limit parameters that you can use.
Trim Tokens?
Set this parameter to true to automatically trim chat sessions down to below the token limit for the model – 8192. When you trim a chat session, Cyclr trims the oldest message in the chat session first. This process repeats until the chat session is below the token limit, and then Cyclr sends the request with the trimmed chat session.
Warning: If you enable Trim Tokens? messages that Cyclr removes from chat sessions are permanently removed.