We've added new settings to help you get better results from AI chats while saving time and resources.

Important note: Only the "Maximum Tokens for AI Thinking Before Responding" feature is exclusively available for Sonnet 3.7. The other settings (Maximum Recent Messages to Retain and Maximum Tokens from Recent Messages) are available for all language models. For more information about Sonnet 3.7 features, please refer to the article "Experience Claude 3.7 Sonnet Today."

This guide explains these settings in simple terms and shows you how to use them effectively.

Three Key Settings Explained

1. Maximum Tokens for AI Thinking Before Responding

What it is: This controls how much "thinking space" the AI has before answering you.

In simple terms: Tokens are pieces of words the AI processes. More tokens = more thorough thinking but potentially slower responses.

Choose:

- Higher limit (up to 32,000): For complex questions requiring detailed answers

- Lower limit: For quicker, simpler responses

2. Maximum Recent Messages to Retain

What it is: This controls how many previous messages the AI remembers.

In simple terms: It's like telling the AI how far back in your conversation it should look.

Choose:

- Higher number: When your entire conversation history matters

- Lower number: When only recent messages are relevant

- Zero: To remember the entire conversation

3. Maximum Tokens from Recent Messages

What it is: This limits how much content from previous messages the AI remembers.

In simple terms: It determines how much detail the AI recalls from your earlier messages.

Choose:

- Higher limit: To keep detailed information from previous messages

- Lower limit:To focus only on key points from earlier messages

Understanding Tokens

Tokens are how AI processes language:

- Short English words like "and" or "the" = 1 token

- Longer words like "computer" = multiple tokens

- Non-English characters often use more tokens

Practical Examples

Example 1: Short Messages

User: Hello!

AI: Hello, how can I help you?

User: What's the weather today?

AI: I can't check the weather in real-time.

User: How does memory limitation work?

AI: Memory limitations control how much of our conversation I can access.

- If you set "Maximum Recent Messages" to 3: When responding to "How does memory limitation work?", the AI would see:

1. The current question ("How does memory limitation work?")

2. Its previous response ("I can't check the weather in real-time.")

3. The previous user message ("What's the weather today?")

But it would no longer remember the initial exchange ("Hello!" and "Hello, how can I help you?").

- If you set "Maximum Tokens" to 50: All messages would be remembered (because they're short)

Example 2: Long Message

User: Hello!
AI: Hello!
User: [Long 500-word technical problem]

- If you set "Maximum Recent Messages" to 3: All messages would be remembered

- If you set "Maximum Tokens" to 200: Most of the long message would be cut off

When to Use Each Setting

Use Message Count Limits When:

- You want to focus on the last few exchanges

- Older messages aren't important anymore

- You want predictable behavior

Use Token Count Limits When:

- Messages vary greatly in length

- You need to maximize information within limits

- You're trying to optimize resource usage

How to Activate These Settings

1. Open the installed KBs in the OpenKBS platform or install new KBs from the platform.

2. Navigate to the "Settings" section in the top menu.

3. Select the “Chat Setting” dropdown menu.

4. Adjust the sliders for:

- Thinking Tokens (Sonnet 3.7 only)

- Messages Window

- Tokens Window

5. Save your changes by clicking the "Save" button.

Real-World Applications

For Customer Support: Set the AI to remember the last 5 messages to focus on current issues without being distracted by older conversations.

For Personal Assistance: Limit tokens to remember only important details of tasks, like deadlines and key steps.

By adjusting these settings, you can customize how the AI interacts with you, making conversations more efficient and relevant to your needs.