Supported Models

ChatBotKit supports various models to create engaging conversational AI experiences. These include foundational OpenAI models such as O1, GPT4o, GPT4, and GPT3, along with models from Anthropic, Mistral, Groq, Facebook, DeepSeek, and others. Additionally, ChatBotKit uses several of its own models, including text-algo-005 and text-algo-004, for our in-house general assistant.

Below is a table that summarizes the different models. It includes their names, short descriptions, and context sizes (the maximum number of tokens).

Bring Your Own Model

ChatBotKit offers the unique option of bringing your own model and keys to the platform. This feature is designed for those who desire more control over their models and costs.

This could be beneficial, especially if you have particular budget constraints or specific cost strategies. In essence, with ChatBotKit, you're not just limited to using our pre-built models, but you can also introduce your custom-made models, providing you with more flexibility and control to meet your specific needs.

Here is an outline of the steps required to create your own custom model.

1. Navigate to the Bot Configuration Screen

   • From the main dashboard, click on the "Bots" section in the left-hand menu.
   • Select the bot you want to configure or create a new bot.

2. Choose the Model

   • Under the "Model" section, select "custom" from the dropdown menu as shown in the first screenshot.

   • Press the “Settings” button.

     

3. Model Configuration Window

   • Enter a name for your custom model in the "Name" field. For example, "gpt-3.5-turbo."

   • Choose the provider of your custom model from the "Provider" dropdown menu. In this case, select "OpenAI."

   • Provide the necessary credentials for accessing the custom model. Click on the credentials field and enter the required information.

   • Define the maximum number of tokens the chatbot will use for each interaction in the "Max Tokens" field. The default value is 4096.

     

BYOK Caveats

When you opt to use your own key (BYOK) for model access, you assume full responsibility for the model's availability and operational limits. This shift occurs because you are no longer utilizing the default ChatBotKit service tiers, which may offer different capabilities and restrictions.

Customising Model Settings

To customize a model's settings, click on the settings icon next to the model name.

There are four main properties that can be customized: Max Tokens, Temperature, Interaction Max Messages, Region, Frequency Penalty, Presence Penalty, and Vision.

Max Tokens: This property determines the maximum number of tokens that the model can consume when generating a response. By default, this is set to the maximum context size for the model, but you can reduce it to limit the amount of resources used by the model. This can help save token cost but may also reduce the ability of the chatbot to keep up with the conversation.

Interaction Max Messages: The maximum number of messages to use per model interaction. Setting this value to low will make the model more deterministic. Increasing the value will result in more creativity. For Q&A-style conversation it is recommended to keep the value to 2.

Region: The region property allows you to specify the geographical region for the model. This can be particularly useful for services that have specific regional requirements or restrictions. However, it's important to note that the availability of certain models may vary depending on the region.

Frequency Penalty: This property determines how much the model penalizes the repetition of certain words or phrases in its responses. A higher frequency penalty value will result in responses that are more varied and less repetitive.

Presence Penalty: This property determines how much the model penalizes the use of certain words or phrases in its responses. A higher presence penalty value will result in responses that are less likely to contain specific words or phrases.

Vision: This property applies solely to vision models. It enables bots to utilize native vision capabilities as opposed to Skillset Vision Actions. While we generally recommend Skillset for cost-efficiency and control, there are situations where native vision capabilities may be preferred.

By customizing these properties, you can fine-tune the behavior of the model to better suit your specific use case and requirements. However, it's important to note that changing these properties can have a significant impact on the model's performance and accuracy, so it's recommended to experiment with different settings to find the best balance between performance and creativity.

FAQ

Can I get regional access to some models?

Yes. Some models such as Claude can be accessed within your own designated region. Please contact us for more information.

Can I bring my own model?

Our models are designed to scale no matter the circumstances. However, customers that wish to bring their own model can do so on some of our higher-tier plans such as Pro, Pro Plus and Team.

How is token usage calculated?

There are many factors that affect your monthly usage including but not limited to the model you are using, the number of datasets, skillsets and their types.