Personal Knowledge Management at Scale - Analyzing 8,000 Notes and 64,000 Links
A deep dive into the technical implementation and analysis of my Personal Knowledge Management system containing 8,000 notes and 64,000 links, exploring its architecture, key principles, automation and impact.
A deep dive into the technical implementation and analysis of my Personal Knowledge Management system containing 8,000 notes and 64,000 links, exploring its architecture, key principles, automation and impact.
I've been managing my entire knowledge base in Obsidian for the past 3+ years. Today, I'd like to share insights from analyzing nearly 8,000 notes containing 2.8M words (equivalent to ~32 books!), interconnected by over 64,000 internal links. My goal is also to explain why my system scales.
Introduction
The journey to build this system started with a simple goal: create a reliable "second brain" that could scale with my own needs. Like many developers, I initially underestimated the technical challenges of managing large amounts of interconnected knowledge. What began as a collection of Markdown files evolved into a sophisticated system that now powers my content creation, research, and business decisions.
The thing is that I always wanted to have a single source of truth for all my content and ideas. But it took me a long while to get there. My system has grown a lot in the last few years, and I now rely on it every single day, for all sorts of things.
My Knowledge Base at a Glance
All statistics are current as of February 2025.
More up to date ones can be found here:
I also sometimes update the HTML version:
Core Stats
- Total Notes: 8,053
- Total Words: 2.93M
- Internal Links: 64,890
- External Links: 11,953
- Reading Time: 195.4 hours
- Books Equivalent: 32.6
- 256 published articles with 439,325 words
- Knowing that I still have many to migrate to Obsidian (not urgent per se)
Link density varies significantly by note type:
- MOCs: 90.1 links per note
- Published articles: 6.3 links per note
- Book notes: 5.4 links per note
- Regular notes: 2.9 links per note
What makes these numbers interesting is not their size, but what they reveal. The high number of internal links (~8 per note) shows that it's not so much about collecting, but about connecting. Each new note adds value not just through its content, but through its relationships with existing information.
Size does matter too in a sense, because it shows that my system scales really well. The only drawback at this point is the global graph view of Obsidian, which has become unusable (too slow to load). But that's about it. In any case, the global graph is more of a toy. The local graph is much more interesting in general.
Organization
Instead of adopting a deep and complex hierarchy, I've kept the structure flat and maintainable, and rely a lot more on tags and metadata for organizing information:
- 92 total folders
- 2,141 unique tags
- Average of 3.7 tags per note
- 7,940 total tasks tracked
Key sections include:
- Permanent notes (1,458 notes, 413K words)
- Literature notes (2,227 notes, 169K words)
- MOCs (118 notes)
- People (635 notes, 81K words)
- Quotes (1,741 notes, 169K words)
I'm very fond of quotes, but I also enjoy connecting ideas with the people behind them. That's why I have so many person notes.
Data Storage
The vault comprises:
- 17.91 MB of Markdown files
- 708.71 MB of attachments
- 1,159 public attachments
- 150 private attachments
- 193 Git commits tracking changes (content, configuration, etc)
Automation and Templates
I've developed:
- 27 core templates for consistent note creation
- Automation for filing notes based on tags
- Automation for backing up everything
- Automated stats generation
Periodic Notes
- 1,204 daily notes averaging 209 words
- 178 weekly notes averaging 657 words
- 37 monthly notes averaging 2,047 words
- 4 yearly notes averaging 17,230 words
System Architecture
I use Obsidian as my Tool for Thought (TfT) and Single Source of Truth (SSOT). Everything I explore, capture, process, think and plan is in there. I use Obsidian for capturing the ideas of others, my own, as well as document systems/processes, organize my work, plan and track progress on my projects, and much more.
My knowledge management system in Obsidian is built on three fundamental pillars designed to make knowledge capture, processing, and creation as frictionless as possible.
First, structure:
- The PARA method for simplicity, cross-platform compatibility, and flexibility
- The Johnny Decimal system for systematic numbering, logical ordering, and maintaining lean organization
This combination provides both flexibility and structure. PARA (Projects, Areas, Resources, Archives) (imagined by Tiago Forte) offers a simple, and logical way to organize information that works across all tools and platforms. On the other hand, the Johnny Decimal system adds a systematic numbering approach and a few rules that keep folders lean and organized. For instance, all my Areas are numbered from 30-39, with sub-areas like "33 Permanent Notes" containing further categorized sections. Note that I'm not trying to make it all perfect, and I don't hesitate to violate "rules" when I feel the need to. The most important point about my structure is that it has been very stable for a long while. I don't waste time trying to make it perfect. As long as I don't feel too much friction, it's fine.
Second, I have templates and automation rules in place to remove tedious/time-consuming maintenance work and help me remain consistent:
- Templates for all content types to ensure consistency: each main type of note has a dedicated template (e.g., periodic notes, meeting notes, literature notes, quotes, etc)
- Automated filing to eliminate friction during content capture: I use plugins such as the Auto note mover plugin for Obsidian to automatically take care of the filing based on tags
- Dedicated Resources folder for all attachments (1,309 files, ~700MB): I don't want to have to think about where to put attachments. They all go into a single "Attachments" folder. To find those back, I simply rely on links in the relevant notes
The key to maintaining a large knowledge base such as mine is automation. Every repetitive task that can be automated should be. My system uses templates not just for consistency, but as triggers for automation. When I create a new note, the template automatically adds the correct tags, metadata, and then the note is automatically moved to its proper location. This removes the cognitive overhead of "where should I put this?" and lets me focus on the content itself.
Third, and most importantly, there's an actual system in place. I'm not capturing mindlessly. I know about my goals, my priorities, and how I want information to turn into knowledge I can leverage. All the information I capture comes into a funnel, no matter the source, and ends up in my Tool for Thought. All the rest is considered transient
Core Design Principles
Friction Reduction & Scaling
- Use as few tools as possible to cover the whole PKM process and reduce integration needs
- Notes automatically move to appropriate folders based on tags
- Templates for consistency (handle metadata and initial structure)
- Automated periodic note creation and organization
- Automated rollover of tasks from one day to the next
- Kanban boards for projects and different time horizons
- Projects based on metadata
Future-Proofing
- Focus on the system, not on the tools. Parts should be replaceable
- All notes stored as Markdown files
- YAML front matter for metadata
- No vendor lock-in
- Ensure the content "works" on its own
- Git for version control (193 commits)
- Tons of automated backups (local & remote)
- Ensures easy AI integration
Organization
- Everything ends up in my Tool for Thought. All the rest is transient
- Combined PARA method with Johnny Decimal system and Zettelkasten
- Flat structure where possible
- Strategic use of subfolders only when necessary
- Clearly isolate by type of content rather than by topic
The architecture decisions weren't obvious from the start. Early versions of the system used a deeper folder hierarchy that seemed logical but proved unwieldy as the system grew. The current flat structure with strategic subfolders emerged through iterative refinement, driven by both performance needs and usage patterns. In the future, I intend to further reduce the number of folders, and to rely on more live queries to create specific dashboards for important content types.
One crucial lesson was that the best technical architecture is often the simplest one that can handle the complexity of your use case. While it was tempting to build complex automation and organization schemes, the combination of plain text files, simple folder structures, and automated tag-based organization has proven remarkably robust and maintainable.
The Knowledge Funnel
I have a systematic process for going from content I discover, curate and decide to consume to a set of connected notes in my system, and eventually to published articles. I've documented this high-level process here: Personal Knowledge Management Process.
The main steps/phases of that process are: Explore → Curate → Consume → Capture → Distill → Organize → Connect → Develop → Create → Share
This "pipeline" represents the complete lifecycle of knowledge throughout my system. Each stage has its own purpose and tooling. For example, exploration might start with my RSS reader, or social media. For curation, I currently use Readwise. While consuming, I use Readwise Reader, but intend to switch to the new Obsidian Web Clipper in combination with the Obsidian Web Viewer. My goal is to reduce the number of tools that I use and depend on, further simplifying the system.
Sebastien Dubois
Every idea I capture is added to my daily notes, and I extract those into separate notes during my Periodic reviews, adding metadata, tags and links as I do so. Finally, the information rolls into my weekly notes, then monthly and yearly ones. I've described this knowledge ingestion process in the following articles:
Sebastien Dubois
Sebastien Dubois
During weekly reviews, I extract information from daily notes into dedicated notes and replace the original content with links. This process repeats at monthly and yearly levels, creating increasingly dense connection networks:
- Daily notes: 1,186 notes (avg. 11.5 links)
- Weekly notes: 178 notes (avg. 45.7 links)
- Monthly notes: 37 notes (avg. 161.6 links)
- Yearly notes: 4 notes (avg. 1,443.8 links)
This progressive summarization and linking strategy explains why higher-level periodic notes have significantly more connections – they serve as indexes and connection hubs for all the knowledge captured below them.
Last but not least I have a solid backup solution in place for my notes, which is critical because this system is central in my life and work. I described it here:
Sebastien Dubois
Content Organization
In my system, I isolate different content types. This enables me to implement different workflows for different types of content.
I also separate content based on its source/authors:
- Literature notes: Third-party content, processed inputs
- Permanent notes: My own content, refined knowledge
The distinction between literature notes and permanent notes, which comes from the Zettelkasten method is crucial for me as an author and content creator. Literature notes contain processed insights from external sources, as much as possible with clear attribution. Permanent notes are where I develop my own ideas, building on but distinct from the literature notes. This separation ensures I never confuse others' ideas with my own and helps maintain intellectual integrity.
I further separate note types based on the main following tags:
#articles/published(251 notes)#book_notes(156 notes)#mocs(116 notes)#meeting_notes(16 notes)#my_quotes(238 notes)#people(630 notes)#quotes(1,737 notes)#daily_notes(1,186 notes)#weekly_notes(178 notes)#monthly_notes(37 notes)#yearly_notes(4 notes)#private_notes(for sensitive content)
Tags provide a flexible layer of organization on top of the folder structure. They allow notes to belong to multiple categories without duplicating files. More importantly, they serve as triggers for automation. My system watches for specific tags and automatically files notes accordingly. This tagging system also enables powerful queries and filtering, making it easy to find and combine information in different ways. Note that I use hundreds of additional tags for describing the content. And yes, an improvement could be to use "type" prefix for those tags (e.g., type/quote), but I'm not interested in changing right now.
The Resources folder deserves special mention. Rather than manually organizing attachments (images, PDFs, etc.) into topic-specific folders, everything goes into a single Resources folder with a unique name (and usually prefixed by the current date as YYYY-mm-dd - ). This significantly reduces the friction of adding attachments to notes and eliminates the cognitive overhead of deciding where each attachment should go.
The organization challenge became apparent around the 1,000-note mark. Manual filing was not only time-consuming but also inconsistent. The key insight was that tags are more flexible than folders for organization. While a note can only exist in one folder, it can have multiple tags. This allows for more natural organization patterns and enables automation based on clear rules rather than subjective decisions.
System Documentation
As I started working on my Knowledge Management course, I realized that I could further improve the documentation of my own system. For that purpose, I created a free template that you can find here: PKM System Handbook Template.
I built it based on my own documentation (i.e., My PKM System Handbook), and ensuring it aligned with my vision of the Personal Knowledge Management Process.
If you're curious about this, then check out my upcoming course:
Key Learnings
- Use friction as a guide: No need to tweak the system all the time if friction is low. Focus on using the system instead of trying to make it perfect
- Documentation Helps Improve: When I turned my system into a product, I had to document it. This enabled me to improve it a lot, because it highlighted all the caveats and parts that needed some design
- Plain text is future-proof: Markdown is where it's at. Data privacy is not the only concern. Also, there are so many ways to leverage the information (e.g., recently with the Model Context Protocol (MCP)) compared to what proprietary platforms enable
- Manual organization breaks down past 1,000 notes: And actually even before that
- Atomic Notes Scale Better: Small, focused notes enable better linking and reuse
- Tags > Links > Hierarchy: A flat structure with dense connections provides better navigation than deep hierarchies. Combine tags, links and Maps of Content to increase the link density
- Maps of Content as Hubs: Maps of Content serve as powerful entry points to leverage the accumulated knowledge. I currently use MoCs more and more using complex queries, especially with AI
- Templates Rule: Use templates to maximize consistency and ease automation
- Automation is Critical for Scale: Templates help maintain consistency
- Use a single Source of Truth for your knowledge: Centralize as much as you can instead of maintaining silos. The Tool for Thought is where knowledge lives on. All the rest should be transient
- Git is Your Friend: Assuming you have plain text files, Git is a great companion for your Knowledge base. I regularly save file hunks to track my changes more precisely
- You Need a Serious Backup Plan: Catastrophes will happen. Be ready
Why I Publish My Notes
Out of my nearly 8,000 notes, 6,478 are public. This isn't by accident – I believe in the power of shared knowledge. Personal Knowledge Management remains an underutilized tool that can transform how people learn, think, and make decisions.
By publishing my notes, I aim to:
- Demonstrate the real-world impact of systematic note-taking
- Help others understand PKM's potential for personal development, career growth, and creativity
- Share practical examples of knowledge management in action
- Contribute to the broader conversation about learning and knowledge work
Making PKM Accessible
After two years of refining my system, I packaged my experience into the Obsidian Starter Kit. It's not just a template – it's my entire PKM "infrastructure":
- My complete folder structure
- My carefully selected and configured plugins
- My battle-tested templates and workflows
- My automation scripts and processes
Nearly 1,000 users have adopted this system, validating its effectiveness and scalability. The kit helps people skip the months of trial and error I went through, providing a proven foundation for their PKM journey.
And as I mentioned before in the article, also make sure to check out my upcoming Knowledge Management course
Does it still matter in 2025+?
You might wonder if all of this stuff is still meaningful now that AI is changing the world. What's the point of accumulating knowledge in a Tool for Thought if you can ask Deepseek, GPT4o, Claude, Llama & al for all the information in the world.
Well to me, the answer is a resounding YES. Let me explain why.
LLMs know it "all", but don't know much about what YOU know, think, and want. And unless Neuralink makes a huge breakthrough and can start reading your mind, that won't change for a long while. Every time you start a discussion with AI, you're starting over. Yes, you may create a project, update documents, etc, but at the end of the day, you're just facing the main character from Memento. But if you accumulate your own knowledge and ideas in a Tool for Thought, then you're creating content that you'll be able to leverage with AI. And then it becomes much more interesting. In fact, you can already do this using things such as the Model Context Protocol (MCP), and various tools. Using those, you can share your own ideas with LLMs, and start the discussions from there. I'm doing this a lot right now, and I see the value, compared to just starting from scratch every time.
Beyond the leverage it creates when combined with LLMs, having your own (serious) knowledge base if a great way to create a place where you can think deeply. A fourth place. It's an idea I discussed in a previous article. The gist of it is that you need a space where you can organize your ideas, explore those further, store the results of your experiments, capture important achievements, things you're grateful for, what inspires you, etc. It not only creates a space where you can think, but also a place you can use to look at the past more objectively. And to me, that's invaluable.
Sebastien Dubois
Conclusion
What started as a personal project has evolved into something much more interesting: a laboratory for me to understand the world around me, to think, to develop my own ideas, and to share insights with others.
Building a knowledge management system that scales requires careful consideration. But few of us have the drive, the time, or the need to spend so much time building one. Many try, and fall in different traps, such as hopping from tool to tool all the time, making things too complex, relying too much on folders, not paying attention to vendor lock-in, etc. That's why I'm convinced that sharing my lessons learned is always useful to someone out there. And that's also why I'm working hard to release my damn Knowledge Management course that's been in the making for too long ;-)
That's it for today ✨
About Sébastien
I am Sébastien Dubois. You can follow me on X 🐦 and on BlueSky 🦋.
I am an author, founder, and coach. I write books and articles about Knowledge Work, Personal Knowledge Management, Note-taking, Lifelong Learning, Personal Organization, and Zen Productivity. I also craft lovely digital products . You can learn more about my projects here.
If you want to follow my work, then become a member.
Ready to get to the next level?
To embark on your Knowledge Management journey, consider investing in resources that will equip you with the tools and strategies you need. Check out the Obsidian Starter Kit and the accompanying video course. It will give you a rock-solid starting point for your note-taking and Knowledge Management efforts.
If you want to take a more holistic approach, then the Knowledge Worker Kit is for you. It covers PKM, but expands into productivity, personal organization, project/task management, and more:
If you are in a hurry, then do not hesitate to book a coaching session with me:
