Build a Self-Hosted AI Assistant with Ollama and Home Assistant
Introduction
Your smart home knows when you wake up, how much energy you use, and whether you left the garage door open. But here's the uncomfortable question: where does that data actually go? For most people using Alexa, Google Home, or Siri, the answer is someone else's cloud server. Every voice command, every temperature reading, every "what's on my calendar today?" gets processed remotely.
That model works fine—until it doesn't. Privacy advocates have long warned about the risks, but the trade-off seemed unavoidable. You needed cloud AI to get genuinely smart home automation.
That trade-off is dissolving. Open-source large language models (LLMs) have improved to the point where they can run on hardware you already own. Combined with Home Assistant—the open-source home automation platform that runs on a Raspberry Pi or an old PC—you can build a fully local AI assistant that understands natural language, controls your devices, and never sends a single packet to a third-party server.
This article walks you through the entire process: the technology behind it, the hardware you'll need, step-by-step setup, customization, and real-world examples. By the end, you'll have a private, self-hosted AI assistant that answers to you and no one else.
Understanding the Core Technologies
What is Ollama?
Ollama is an open-source tool that runs large language models locally on your own hardware. Instead of sending prompts to OpenAI's servers or Anthropic's API, Ollama downloads model weights directly to your machine and runs inference on your CPU or GPU. Launched in 2023, it has been downloaded over 10 million times since.
The tool provides a simple command-line interface, a RESTful API, and a library of over 300 open-source models. You type ollama run llama3, and within minutes you're chatting with a model running entirely on your hardware. No account creation. No API keys. No usage limits.
What is Home Assistant?
Home Assistant is a free, open-source home automation platform that runs locally on your network. It integrates with over 1,000 different devices and services—from Zigbee sensors and Z-Wave locks to Philips Hue lights and Nest thermostats. Unlike cloud-dependent hubs like SmartThings or Wink, Home Assistant continues to function even when your internet goes down.
Its core philosophy is local control and privacy. All your device data, automation history, and configuration files stay on your own hardware.
How Ollama and Home Assistant Work Together
Home Assistant has a built-in "conversation" integration that lets you interact with your home using natural language. By default, it uses basic pattern matching—you say "turn off the lights," and it looks for keywords. With Ollama as the conversation agent, Home Assistant sends your natural language commands to the local LLM, which interprets them and triggers the appropriate device actions.
The integration works through Ollama's API endpoint (default port 11434). Home Assistant sends a prompt, the LLM processes it, and returns either a text response or a function call that executes a specific automation.
Why Go Self-Hosted? The Benefits and Trade-offs
Privacy and Data Ownership
When you use Alexa or Google Home, every command is recorded, transcribed, and stored on company servers. With a local setup, all conversation logs, device states, and sensor data remain on your private network. This matters if you're using your assistant for sensitive tasks like checking bank balances, managing security cameras, or controlling door locks.
Latency and Offline Operation
Cloud AI requires a round trip to a remote server. Even with fast internet, you're looking at 1–2 seconds of latency per request. A local LLM on a modern GPU can respond in under 500 milliseconds. More importantly, a local assistant works when your internet goes down—an advantage if you rely on voice commands for critical functions like locking doors or turning off appliances.
Cost Savings vs. Cloud AI
Cloud AI services charge per token. A heavy smart home user making hundreds of requests daily could spend $20–50 per month. Ollama is completely free. The only cost is electricity and hardware depreciation.
Limitations of Local LLMs
Let's be honest: local models aren't as capable as GPT-4 or Claude 3.5. They struggle with complex reasoning, have smaller knowledge bases, and can hallucinate more frequently. For home automation—which is mostly short, structured commands—this limitation rarely matters. But if you're expecting your local assistant to draft legal documents or solve advanced math problems, you'll be disappointed.
Key Takeaway: Self-hosted AI gives you privacy, speed, and zero per-request costs. The trade-off is raw intelligence. For smart home tasks, that's a fair exchange.
Hardware Requirements and Model Selection
Minimum Hardware for Running Ollama
The hardware floor depends on which model you choose. For the smallest usable models (like Phi-3 Mini or Gemma 2B), you need:
- 8GB RAM minimum
- An x86-64 processor (ARM works with limitations)
- 4GB free storage for model weights
CPU vs. GPU: Performance Considerations
Running on CPU alone is possible but slow. A 7B parameter model on a modern CPU generates roughly 5–10 tokens per second. That's fine for text responses but sluggish for voice interactions.
A GPU changes everything. An NVIDIA RTX 3060 (12GB VRAM) runs Llama 3 8B at 40–60 tokens per second. An RTX 4090 can handle larger models like Llama 3 70B at usable speeds.
If you're using a Raspberry Pi or a mini-PC without a discrete GPU, stick to models with 3B parameters or fewer. They'll be less capable but still functional for basic commands.
Popular Models and Their Requirements
| Model | Parameters | Minimum RAM | Best For |
|---|---|---|---|
| Llama 3 | 8B | 8GB | General purpose, good instruction following |
| Mistral | 7B | 8GB | Fast inference, solid reasoning |
| Gemma | 2B | 4GB | Low-powered devices like Raspberry Pi |
| Phi-3 Mini | 3.8B | 4GB | Compact, surprisingly capable |
| Llama 3 | 70B | 32GB | Complex queries, needs serious hardware |
Key Takeaway: Match the model to your hardware. An 8B model on 8GB RAM works but will be slow. A 2B model on a Raspberry Pi is responsive but limited. There's no free lunch.
Step-by-Step Setup Guide
Installing Ollama on Your Server or PC
On Linux or macOS, run:
curl -fsSL https://ollama.com/install.sh | sh
Windows users download the installer from ollama.com. Once installed, verify with:
ollama --version
Downloading and Running a Model
ollama run llama3
This downloads the model (about 4.7GB) and starts an interactive session. To keep Ollama running as a background service, use:
ollama serve
The API will be available at http://localhost:11434.
Installing Home Assistant (If Not Already)
The easiest method is Home Assistant OS, which you flash to an SD card for a Raspberry Pi, or install as a virtual machine. For existing servers, the Docker installation is straightforward:
docker run -d --name homeassistant --restart=unless-stopped \
-v /path/to/config:/config \
-v /etc/localtime:/etc/localtime:ro \
-p 8123:8123 \
ghcr.io/home-assistant/home-assistant:stable
Connecting Home Assistant to Ollama
Open your Home Assistant instance (usually http://your-server:8123). Navigate to Settings → Devices & Services → Add Integration. Search for "Ollama." Enter the URL of your Ollama server (e.g., http://192.168.1.50:11434), select your model, and save.
Using HACS for the Ollama Integration
The native integration is solid, but the HACS (Home Assistant Community Store) version adds features like function calling support and better error handling. To install:
- Install HACS if you haven't already (instructions at hacs.xyz)
- Go to HACS → Integrations → Search "Ollama"
- Install and restart Home Assistant
Configuring the Conversation Agent
In Settings → Voice Assistant, create a new assistant. Set the conversation agent to "Ollama." You can now test it in the Home Assistant dashboard by typing "turn off the living room lights."
Customizing Your AI Assistant
Setting System Prompts and Personality
The system prompt tells the LLM how to behave. In the Ollama integration settings, you can add something like:
"You are Jarvis, a helpful home assistant. Respond concisely. When the user gives a command, acknowledge it and explain what you're doing. If you can't perform a task, say so directly."
You can make it formal, sarcastic, or completely utilitarian. The prompt shapes everything.
Using Function Calling to Control Devices
The HACS Ollama integration supports function calling. This lets the LLM invoke Home Assistant services directly. For example, if a user says "I'm cold," the LLM can call the climate service to raise the thermostat by 2 degrees—without a pre-built automation.
Creating Custom Automations with Ollama
You can also use Ollama inside Home Assistant automations. Create a script that sends sensor data to the LLM and asks for a decision:
automation:
- alias: "Smart Energy Decision"
trigger:
platform: time_pattern
hours: "/1"
action:
- service: ollama.generate
data:
prompt: "Current energy price is {{ states('sensor.energy_price') }}. Should I charge the EV now? Reply yes or no."
Integrating with Voice: Whisper and Piper
For a fully voice-controlled assistant, pair Ollama with:
- Whisper (local speech-to-text)—converts your spoken words into text
- Piper (local text-to-speech)—reads the AI's responses aloud
Both integrate natively with Home Assistant and run entirely on your hardware.
Key Takeaway: The system prompt is the personality dial. Function calling is the action dial. Together, they turn a generic LLM into a purpose-built home assistant.
Real-World Use Cases and Examples
Morning Routine Automation
A Raspberry Pi 5 with 8GB RAM runs Llama 3 8B. When you say "Good morning," the AI:
- Turns on the coffee maker (via a smart plug)
- Adjusts the thermostat to 72°F
- Reads the day's first calendar event
- Tells you the weather forecast
All responses come through local text-to-speech. Zero cloud involvement.
Energy Management and Smart Thermostats
A user with an RTX 3060 runs Mistral 7B. They've set up an automation that queries the LLM hourly with current energy prices, outdoor temperature, and home occupancy. The AI decides whether to pre-heat the house before peak pricing hours and sends a notification explaining its reasoning.
Security and Notifications
When a motion sensor triggers at 2 AM, Home Assistant sends the event to Ollama: "Motion detected at the back door at 2:03 AM. The user is asleep. What should I do?" The LLM responds with instructions: activate exterior lights, send an alert to the user's phone, and start recording from the security camera.
Handling Complex Queries with Larger Models
A user with an RTX 4090 runs Llama 3 70B. They can ask questions like, "What's the most energy-efficient way to heat my home today, given that electricity costs $0.32/kWh from 4-9 PM?" The model processes real-time data and delivers a nuanced, context-aware recommendation.
Troubleshooting and Optimization
Common Setup Issues and Fixes
"Connection refused" error: The Ollama server isn't reachable. Check that it's running (ollama serve), the port is open, and your firewall allows traffic on port 11434.
Slow responses: Your model is too large for your hardware. Switch to a smaller model or reduce context length in the integration settings.
Model doesn't understand device names: Your system prompt needs to include a list of your devices and their entity IDs. Add them explicitly.
Improving Inference Speed
- Use GPU acceleration if available (Ollama uses CUDA or ROCm automatically)
- Reduce the context window (max tokens) in the integration settings
- Use a quantized model (e.g., Llama 3 8B Q4 instead of Q8)—slightly lower quality but 2x faster
Monitoring Resource Usage
Ollama exposes metrics at http://localhost:11434/api/ps. For real-time monitoring, use htop or a tool like Grafana with the Prometheus exporter.
Scaling to Multiple Users or Devices
Ollama handles concurrent requests, but each request uses memory. If multiple users are querying simultaneously, you'll need more RAM or a GPU with larger VRAM. For heavy use, consider running Ollama on a dedicated server rather than the same machine as Home Assistant.
Security and Privacy Considerations
Keeping Data Local
The core benefit of this setup is data locality. But verify that your Home Assistant instance isn't accidentally sending data elsewhere. Disable any cloud integrations you don't need, and check your network traffic periodically.
Network Security Best Practices
- Run Home Assistant and Ollama on a VLAN separate from your main network
- Use HTTPS for remote access (Home Assistant's built-in remote access supports this)
- Don't expose Ollama's API directly to the internet—put it behind a reverse proxy with authentication if remote access is necessary
Managing Access and Permissions
Home Assistant has multi-user support. Create separate user accounts for family members with appropriate permissions. The conversation agent uses the permissions of the user who initiates the conversation, so a child's account won't be able to unlock doors.
Key Takeaway: Local AI removes the cloud risk but introduces your own security responsibilities. A properly configured self-hosted setup is more private than any cloud service—if you configure it correctly.
Future Outlook and Community Resources
The Evolution of Local AI Models
Open-source models are improving rapidly. Llama 3.1, Mistral's latest releases, and Qwen 2.5 all show that local models are closing the gap with cloud offerings. The community is also developing smaller, more efficient architectures that run on edge devices.
Upcoming Features in Home Assistant and Ollama
Home Assistant's voice assistant pipeline is actively under development, with better wake-word detection and more natural conversation flows. Ollama continues to add model support and performance optimizations, including experimental multimodal models that can process images.
Community Forums and Further Reading
- Home Assistant Community Forum—active discussions on AI integrations
- Ollama GitHub—source code, issues, and feature requests
- r/selfhosted—broader self-hosting community with frequent AI assistant threads
- HACS—for the latest community integrations
Conclusion
Building a self-hosted AI assistant with Ollama and Home Assistant is not a weekend project for everyone. It requires hardware, configuration, and a willingness to troubleshoot. But the payoff is real: a private, fast, and cost-effective assistant that answers to you.
The right choice depends on your priorities. If you're comfortable with cloud services and don't mind the privacy trade-offs, existing solutions work fine. If you value data ownership and want an assistant that works even when the internet doesn't, this setup is worth the effort.
Start small. Install Ollama on a machine you already have. Download a small model. Connect it to Home Assistant. Add a voice pipeline. You can always upgrade hardware and models later.
FAQ
What hardware do I need to run Ollama with Home Assistant?
At minimum, 8GB of RAM and a decent CPU. A Raspberry Pi 5 with 8GB is the entry point. For comfortable performance, a mini-PC with 16GB RAM or a desktop with a dedicated GPU (NVIDIA RTX 3060 or better) is recommended.
Is it possible to use a local AI assistant without an internet connection?
Yes. Once the model is downloaded and Home Assistant is configured, everything runs locally. The only exception is integrations that require cloud services (e.g., weather APIs, some device brands).
How do I install the Ollama integration in Home Assistant?
Go to Settings → Devices & Services → Add Integration → search for "Ollama." Enter your Ollama server's URL and select your model. For the enhanced version with function calling, install via HACS.
Can the local AI control all my smart home devices?
Any device that Home Assistant can control, the AI can control—provided you've set up the integration correctly and the device entities are exposed to the conversation agent.
What are the limitations of using a local LLM compared to cloud AI like ChatGPT?
Local models are smaller, so they have less general knowledge and weaker reasoning. They can also hallucinate more. For structured home automation commands, they're more than adequate. For open-ended conversation, they'll disappoint.
How do I choose the right model for my hardware?
Match model size to available RAM. A 7B model needs roughly 8GB of RAM. An 8B model needs 8–12GB. A 70B model needs 32GB+. If in doubt, start with a smaller model and upgrade if you need more capability.
Can I customize the personality of my AI assistant?
Absolutely. The system prompt in the Ollama integration settings defines the assistant's personality, tone, and behavioral constraints. You can make it formal, playful, minimalist, or anything in between.
Is my data safe when using a local AI assistant?
Yes, if configured correctly. Data stays on your network. Just ensure your network is secure, don't expose Ollama's API to the internet, and use strong passwords for Home Assistant.
What if I don't have a dedicated GPU?
You can run Ollama on CPU. Expect 5–10 tokens per second with a 7B model—usable but not snappy. Stick to 2B–4B models for better responsiveness.
Can I use Ollama with other platforms besides Home Assistant?
Yes. Ollama's API is RESTful and works with any application that can make HTTP requests. You can connect it to Node-RED, custom scripts, web apps, or even use it as a backend for your own chatbot.
Ready to take control of your smart home AI? Start by exploring the Ollama model library and installing Home Assistant today. Join the community forums to share your setup and learn from others.