Making Local AI and Obsidian Actually Work Together: From Gemini Scribe to Antigravity and Gemma
I like testing technology to see if it actually solves problems in my day-to-day life.
Lately, that testing has centered on two things: my personal notes in Obsidian and running local AI models.
I’ve written before about how the "P" in PKM stands for Personal, not Perfect. My Obsidian vault isn’t an academic showcase. It’s my working brain: where I track daily thoughts, store highlights from books and articles, manage ADHD focus, and draft content.
Like a lot of people, I was drawn to the idea of running open-source AI models right on my own computer. The appeal is straightforward:
Privacy: Your private journals, therapy reflections, and half-formed ideas stay on your drive instead of getting beamed to somebody else's cloud.
Cost: You aren't paying a monthly subscription or watching API meter counters tick up.
Offline: It works whether you have internet or not.
So I started testing Google's Gemma models locally using Ollama. But my first setup—shoving AI directly into Obsidian with the Gemini Scribe plugin—quickly ran into practical roadblocks.
Earlier today, I took a step back, cleaned out the clutter, and rebuilt the workflow from the ground up using Google Antigravity.
Here is what failed, what changed, and how you can run a fast, private, local AI workflow alongside Obsidian without bogging down your notes.
1. The First Attempt: Shoving AI Inside Obsidian
When you decide you want AI in your notes, the most obvious path is to find an Obsidian community plugin.
I set up Ollama on my machine to run local Gemma, installed the Gemini Scribe plugin in Obsidian, and configured it to talk to my local model.
At first, it felt pretty neat. I could highlight a chunk of text in a note, press a shortcut, and have the model summarize it or suggest ideas right inside my editor pane.
[In-Editor Plugin Setup]
┌──────────────────────────────────────────────┐
│ Obsidian │
│ ┌────────────────────┐ ┌──────────────────┐ │
│ │ My Note │ │ Gemini Scribe │ │
│ │ (Markdown) │ │ (Chat / Sidebar) │ │
│ └────────────────────┘ └──────────────────┘ │
└────────────┬─────────────────────┬───────────┘
▼ ▼
Vault Files Local OllamaWhere It Fell Apart
The excitement wore off quickly once I tried living with it.
The Mobile Wall
I rely heavily on Obsidian Sync across my desktop, tablet, and phone. I really wanted Obsidian to work smoothly on mobile, and I genuinely wanted the power of AI there too.
In practice, community plugins with chat panes and heavy background listeners are rough on mobile. On a smartphone screen, you want fast, frictionless capture. You don’t have room for a sidebar widget squishing your text, and you don't want plugins draining your battery or slowing app launch times.
In fact, I’m literally reviewing this post on my phone right now. If a setup doesn't feel snappy and natural on mobile, I'm just not going to stick with it.
Plugin Bloat and the YAGNI Rule
In software development, there’s a principle called YAGNI (You Aren't Gonna Need It). It basically means: don't add stuff until you actually need it.
Every plugin added to Obsidian makes the app a little heavier, increases startup time, and adds visual noise. With my ADHD, visual noise and small delays are friction points that can derail my entire train of thought.
The Sandbox Limit
An Obsidian plugin lives inside Obsidian's window. It can only touch what Obsidian allows it to touch. It can't easily run Python scripts, inspect files across other directories, run system diagnostics, or do multi-step batch jobs without slowing down your editing experience.
I was trying to turn my note-taking app into an AI command center, and it was making both the app and the AI worse.
Earlier today, I decided to simplify. I archived Gemini Scribe, cleared out over ten community plugins that were piling up, and moved them into an archive folder.
Obsidian went back to being what it does best: a fast, clean, plain-text notebook.
2. Decoupling the AI: Enter Google Antigravity
The fix wasn't giving up on local AI—it was changing where the AI lives.
Instead of forcing Obsidian to host the AI, why not let an external tool do the heavy lifting and just have it read and write plain Markdown files directly to the vault?
I’ve been using Google Antigravity at work and have really liked the workflow and how easy it is to develop custom skills for it. Antigravity runs in the developer environment, understands files, runs commands, and uses tools cleanly.
Even better, Google recently added native support for local models to the Antigravity SDK. That means you can point Antigravity straight at your local Ollama setup (or run LiteRT locally) using LocalOpenAIAgentConfig.
[Decoupled Setup]
┌───────────────────────────────────────┐
│ Obsidian (Desktop & Mobile) │
│ • Fast, clean, plain Markdown │
│ • Zero heavy AI plugins │
│ • Syncs instantly on phone & tablet │
└──────────────────┬────────────────────┘
│
▼ (Reads & writes .md)
[Your Vault Files on Disk]
(D:\Flow Vault Pro\)
▲
│ (Edits & creates notes)
┌──────────────────┴────────────────────┐
│ Google Antigravity │
│ • Plans the work & checks vault links │
│ • Dispatches drafting to Ollama │
└──────────────────┬────────────────────┘
│
▼
[Local Gemma 4 via Ollama]
• $0.00 cost
• Runs 100% on your machineNow, Obsidian stays completely lightweight. It doesn't even know AI is involved. It just sees new notes appear and existing notes get updated.
3. The "Plan Here, Execute Locally" Pattern
Running local models on your own hardware comes with real trade-offs.
If you ask a smaller local model to plan a complex project, search the web, check five files, and write an essay all in one go, it can lose track of things or run slowly.
On the other hand, if you send everything to big cloud models, you run up API bills and defeat the purpose of keeping your personal notes private.
So we set up a simple division of labor called the Architect-Builder pattern:
[Cloud Planner: Gemini Flash] (< 5% of tokens)
• Scopes out what needs to be done
• Reads vault folder structure
• Creates a clear outline and prompt
│
▼ (Passes the plan)
[Local Worker: Gemma 4 via Ollama] (~95% of tokens)
• Writes the long draft or summary
• Cleans up repetitive note formatting
• 100% offline & $0.00 token costHow It Works in Practice
The Planner (Cloud Gemini Flash):
Spends just a tiny handful of tokens (costing fractions of a cent, or free tier).
Looks at my idea, checks where notes live in my vault, and writes a solid outline with clear instructions.
The Worker (Local Gemma 4 via Ollama):
Takes that outline and does the bulk writing locally on my machine.
It can churn out thousands of words, expand rough bullet points, or clean up text without a single byte of my draft leaving my computer.
The Setup
To connect Antigravity to local Gemma, we set up a small Python environment with a direct configuration:
from google.antigravity import Agent, LocalOpenAIAgentConfig
# Point Antigravity to the local Ollama instance
config = LocalOpenAIAgentConfig(
model="gemma4:latest",
base_url="http://127.0.0.1:11434/v1",
system_instructions=(
"You are an assistant for a personal knowledge vault. "
"Draft clear, practical Markdown notes that connect ideas "
"using clean wikilinks."
),
)
agent = Agent(config=config)Antigravity hands the task to Gemma, Gemma writes the content locally, and Antigravity saves the resulting Markdown file directly into the vault (like Outputs/Blog Posts/ or Zettelkasten Brain/Main Notes/).
4. What This Looks Like in My Daily Notes
Because Antigravity works with my actual vault files on disk, it respects the exact rules I already use:
Mobile Daily Notes: Quick thoughts get appended cleanly under
## ⚡ Quick Capturein my daily note (Flows/Daily Notes/YYYY-MM-DD.md) using single-column formatting that looks great on my phone.Permanent Concept Notes: Synthesized ideas get placed in
Zettelkasten Brain/Main Notes/with genuine Luhmann numbers and[[wikilinks]].Clean File Names: No weird characters (
:,?,*) that cause sync issues between Windows, Android, and iOS.Privacy for People Notes: Contact notes in
Tools/People/stay untouched and private.
Obsidian stays fast, clean, and enjoyable to write in. When I need heavy AI assistance, I don't touch a clunky sidebar—I run the task through Antigravity.
5. The Real Trade-Offs (No Hype)
I want to be completely honest about what works and what doesn't, so you can decide if this makes sense for you:
When Local Gemma Shines:
Expanding rough outlines into full first drafts.
Summarizing personal reading highlights or journal entries you don't want online.
Batch cleanup, tagging, or formatting across dozens of notes.
When Cloud is Worth Using:
When you need live web searches and fresh sources.
When analyzing images, diagrams, or handwritten notes.
When you need high-level planning across complex multi-step projects.
The Takeaway
If you've been frustrated trying to make AI work inside Obsidian—especially on mobile—you aren't alone.
Trying to cram an AI chat interface into your note-taking app often ruins the very thing that makes Obsidian great: speed, simplicity, and a clean writing space.
By separating the two:
Obsidian stays a fast, distraction-free markdown canvas that syncs cleanly to your phone.
Antigravity acts as the orchestrator that knows your vault structure.
Local Gemma handles the heavy typing privately on your machine for free.
You don't need to turn your text editor into an operating system. Let Obsidian be your notebook, and let local models do the heavy lifting on your machine where they belong.
How Are You Handling AI in Your Notes?
Have you tried running local models with your vault? Are you sticking with plugins, or do you prefer keeping AI outside the app?
Let me know what your setup looks like!
