What Are SillyTavern and KoboldCPP?

Local AI roleplay has exploded in popularity during 2026, and for good reason. After spending months testing various setups, I’ve found that the SillyTavern and KoboldCPP combination offers the best balance of features, performance, and ease of use for beginners and advanced users alike.

This setup gives you complete privacy. Everything runs on your computer with no data sent to cloud services. You also get unlimited usage without API limits or monthly subscriptions, and freedom from content censorship found in commercial AI services.

In this guide, I’ll walk you through everything you need to know to set up your own local AI roleplay system. By the end, you’ll have a working installation and your first character ready to chat.

Prerequisites & System Requirements

Before diving in, let’s make sure your system can handle local AI inference. I’ve tested this on various hardware configurations, and here’s what I’ve learned about realistic requirements.

Minimum vs Recommended Specifications

Hardware Tier Recommendations

Operating System Requirements

This setup works on Windows 10/11 and most Linux distributions. macOS is supported but with limitations (mainly M1/M2 Macs have specific requirements). I primarily use Windows 11 for testing, and these instructions reflect that experience.

For Windows users, you’ll need the Visual C++ Redistributable. Most gaming PCs already have this, but if you see errors about missing DLLs, download it from Microsoft’s official website.

Installing KoboldCPP (The Backend)

KoboldCPP is the backend that actually runs the AI model. Think of it as the engine that powers everything. I’ve installed KoboldCPP dozens of times across different computers, and the process is straightforward once you know what to look for.

Step 1: Download KoboldCPP

1. Visit the official GitHub repository: Go to github.com/LostRuins/koboldcpp
2. Navigate to Releases: Click on “Releases” in the right sidebar or look for the latest release link
3. Download the correct version: Choose koboldcpp_novid_win64.zip for Windows or koboldcpp_linux.zip for Linux
4. Extract the files: Create a folder named “KoboldCPP” and extract the contents there

Step 2: Launch KoboldCPP for the First Time

Run koboldcpp.exe (Windows) or ./koboldcpp (Linux). You’ll see a launcher window with several options. Don’t worry about most settings yet.

For your first run, use these default settings:

• Threads: Auto (or leave at default)
• GPU Layers: Auto detect
• Context Size: 2048 (good starting point)
• Batch Size: 512

Step 3: Configure GPU Acceleration (If Available)

If you have an NVIDIA GPU, KoboldCPP should detect it automatically. Look for “CUDA” or “cuBLAS” in the backend selection. For AMD GPUs, you’ll need ROCm installed, which is more complex.

Step 4: Start the Server

Click “Launch” to start KoboldCPP. A command window will open showing server status. Look for the line that says something like:

Server listening on http://127.0.0.1:5001

This means KoboldCPP is running and ready to accept connections. Keep this window open in the background.

Installing SillyTavern (The Frontend)

SillyTavern is the user interface where you’ll create characters, manage chats, and interact with your AI models. I find it much more polished and feature-rich than alternatives.

Step 1: Download SillyTavern

1. Visit the official repository: Go to github.com/SillyTavern/SillyTavern
2. Download the latest release: Click “Code” then “Download ZIP” or use the direct download link from Releases
3. Extract to a folder: I recommend D:\AI\SillyTavern or similar location

Step 2: Launch SillyTavern

On Windows, simply double-click Start.bat. This batch file handles everything automatically. On Linux or Mac, run the appropriate shell script.

A command window will open showing initialization progress. Once complete, it will display something like:

SillyTavern is listening on: http://localhost:8000

Step 3: Open SillyTavern in Your Browser

Your default browser should open automatically to the SillyTavern interface. If not, navigate to http://localhost:8000 manually.

Connecting SillyTavern to KoboldCPP

Now comes the crucial part: connecting the two applications. I’ve seen many beginners get stuck here, but the process is simple once you understand it.

Step 1: Open Connection Settings

1. In SillyTavern, click the “Connect” button in the left sidebar (three vertical dots icon)
2. Select “KoboldCPP” from the dropdown under “API Source”
3. Enter the KoboldCPP URL: http://127.0.0.1:5001
4. Click “Connect”

Step 2: Verify the Connection

After a moment, the status should change from “Disconnected” to “Connected” with a green indicator. If you see an error, check that KoboldCPP is still running in the background.

Once connected, you’ll see additional options appear in SillyTavern’s interface, including the ability to load models and adjust generation settings.

Step 3: Test with a Simple Prompt

Before loading a full model, you can test the connection by sending a simple message. Create a temporary character or use the default one, then type “Hello” in the chat box.

If everything is working, you’ll see a loading indicator followed by a response. If you get an error, double-check your connection settings.

Downloading and Loading AI Models

You need an AI model to actually generate responses. KoboldCPP supports the GGUF format, which has become the standard for local LLMs.

Understanding GGUF Models

Where to Download Models

The primary source for GGUF models is Hugging Face. Here are some popular starting points:

• TheBloke: Largest collection of quantized models, always reliable
• Metharme: Excellent for roleplay and creative writing
• Pygmalion: Specifically trained for character roleplay
• Mythalion: Great balance of creativity and coherence

Recommended Models by Hardware

Loading Your First Model

1. Download a model: Click the download icon on Hugging Face for your chosen model
2. Place in KoboldCPP’s models folder: Usually found at the same level as the executable
3. In KoboldCPP, click “Load Model” and navigate to your downloaded file
4. Wait for loading: Large models take 1-3 minutes to load

Once loaded, the model is ready to generate responses. You’ll see the model name and some stats in the KoboldCPP window.

Creating Your First Character

This is where the fun begins. Creating compelling characters is both an art and a science. I’ve created hundreds of characters over the past year, and here’s what works best.

Anatomy of a Character Card

Every character card in SillyTavern contains several key components:

• Name: The character’s name
• Description: Physical appearance, background, and context
• Personality: Traits, behaviors, and mannerisms
• First Message: The opening greeting that starts conversations
• Example Dialogue: Sample conversations that demonstrate voice
• Scenario: Optional setting or context for the roleplay

Step-by-Step Character Creation

1. Open Character Management: Click the character icon (looks like a person) in the left sidebar
2. Click “Create New Character”: This opens the character editor
3. Fill in the Basics: Enter a name and brief description
4. Write the Personality: Be specific about traits and behaviors
5. Create the First Message: This sets the tone for all interactions
6. Add Example Dialogue: This is crucial for consistent characterization
7. Save and Start Chatting: Click save, then select your new character

Character Card Template

Here’s a template I use when creating new characters. Feel free to modify it:

Name: [Character Name]

Description: [Physical appearance, age, occupation, setting]

Personality: [3-5 key personality traits with brief explanations]

First Message: [Greeting that establishes the character and situation]

Scenario: [Optional: world context and current situation]

Example Dialogue:
<START>
{{user}}: [Example user input]
{{char}}: [How your character responds]
<START>
{{user}}: [Another example]
{{char}}: [Another characteristic response]

Importing Pre-made Characters

You don’t have to create everything from scratch. Sites like Chub.ai host thousands of community-created character cards. Simply download a PNG card file and drag it into SillyTavern to import.

Using SillyTavern Features

Once you have a character loaded and connected to KoboldCPP, you can start exploring SillyTavern’s features. I’ve spent countless hours testing these, and here are the highlights.

Basic Chat Controls

• Type in the input box: Press Enter or click Send to generate responses
• Regenerate: Click the regeneration icon if you don’t like the response
• Edit messages: Click the pencil icon to edit previous messages
• Continue: Generate more text from the last response

Generation Settings (Presets)

SillyTavern comes with presets that configure how the AI generates text. Key parameters include:

Lorebooks and World Info

For complex roleplays with established settings, lorebooks provide context about characters, locations, and items. I’ve used lorebooks for fantasy worlds with detailed magic systems and historical contexts.

To create a lorebook, go to the “World Info” tab and add entries. Each entry has triggers (keywords) that activate it when mentioned in conversation.

Saving and Exporting Chats

SillyTavern automatically saves your conversations. You can export chats as text files or JSON backups. I recommend creating regular backups, especially for long-running roleplays you care about.

Troubleshooting Common Issues

After helping dozens of users set up their systems, I’ve identified the most common problems and their solutions.

Connection Issues

Memory Errors

Slow Generation

Model Loading Failures

AMD GPU Issues

AMD GPUs require ROCm instead of CUDA. This setup is more complex and may not work on all cards. Check the KoboldCPP wiki for specific ROCm installation instructions. Some AMD GPUs work better with CPU inference.

Advanced Tips and Optimization

Once you have the basics working, here are some advanced techniques to improve your experience.

Parameter Tuning for Roleplay

For roleplay specifically, I recommend these starting settings:

• Temperature: 1.0-1.2 (balanced creativity)
• Top P: 0.9-0.95
• Top K: 40-100
• Repetition Penalty: 1.1-1.2
• Context Size: 2048-4096 (if RAM allows)

Context Management

Long conversations can degrade quality as the context fills. Use these strategies:

• Summarize periodically: Add summaries of past events to keep context fresh
• Use lorebooks: Offload persistent information to world info
• Split long sessions: Start new chats for major story arcs

Group Chat Setup

SillyTavern supports multiple characters in a single conversation. I’ve run group RPs with 5+ characters successfully. Enable this in the character menu and add multiple characters to the chat.

Automated Scripts and Extensions

For power users, SillyTavern supports JavaScript-based extensions that can automate tasks, modify outputs, and add custom behaviors. This requires programming knowledge but enables virtually unlimited customization.

Frequently Asked Questions

Next Steps and Resources

Congratulations on setting up your local AI roleplay system. After spending months with this setup, I can confidently say it’s worth the initial effort. The privacy, unlimited usage, and creative freedom are unmatched.

For further learning, I recommend joining the r/SillyTavern subreddit and r/LocalLLaMA community. These forums are invaluable for troubleshooting, model recommendations, and character sharing.

The official documentation at docs.sillytavern.app and the KoboldCPP wiki cover advanced features I couldn’t include in this starter guide.

Happy roleplaying, and may your characters come alive in ways you never expected.