Blog

Personal Knowledge Management at Scale - Analyzing 8,000 Notes and 64,000 Links

A deep dive into the technical implementation and analysis of my Personal Knowledge Management system containing 8,000 notes and 64,000 links, exploring its architecture, key principles, automation and impact.

Sebastien Dubois

05 Feb 2025 — 16 min read

Visualization of my PKM system's knowledge graph: 7,957 notes connected by 64,274 links

I've been managing my entire knowledge base in Obsidian for the past 3+ years. Today, I'd like to share insights from analyzing nearly 8,000 notes containing 2.8M words (equivalent to ~32 books!), interconnected by over 64,000 internal links. My goal is also to explain why my system scales.

Introduction

The journey to build this system started with a simple goal: create a reliable "second brain" that could scale with my own needs. Like many developers, I initially underestimated the technical challenges of managing large amounts of interconnected knowledge. What began as a collection of Markdown files evolved into a sophisticated system that now powers my content creation, research, and business decisions.

The thing is that I always wanted to have a single source of truth for all my content and ideas. But it took me a long while to get there. My system has grown a lot in the last few years, and I now rely on it every single day, for all sorts of things.

Let me listen to this 🎙️

I have used AI to generate a podcast version of this article. You can find it here:

My Knowledge Base at a Glance

All statistics are current as of February 2025.

More up to date ones can be found here:

Stats - DeveloPassion

DeveloPassion

I also sometimes update the HTML version:

Knowledge Base Stats

Core Stats

Total Notes: 8,053
Total Words: 2.93M
Internal Links: 64,890
External Links: 11,953
Reading Time: 195.4 hours
Books Equivalent: 32.6
256 published articles with 439,325 words
- Knowing that I still have many to migrate to Obsidian (not urgent per se)

Link density varies significantly by note type:

MOCs: 90.1 links per note
Published articles: 6.3 links per note
Book notes: 5.4 links per note
Regular notes: 2.9 links per note

What makes these numbers interesting is not their size, but what they reveal. The high number of internal links (~8 per note) shows that it's not so much about collecting, but about connecting. Each new note adds value not just through its content, but through its relationships with existing information.

Size does matter too in a sense, because it shows that my system scales really well. The only drawback at this point is the global graph view of Obsidian, which has become unusable (too slow to load). But that's about it. In any case, the global graph is more of a toy. The local graph is much more interesting in general.

Organization

Instead of adopting a deep and complex hierarchy, I've kept the structure flat and maintainable, and rely a lot more on tags and metadata for organizing information:

92 total folders
2,141 unique tags
Average of 3.7 tags per note
7,940 total tasks tracked

Key sections include:

Permanent notes (1,458 notes, 413K words)
Literature notes (2,227 notes, 169K words)
MOCs (118 notes)
People (635 notes, 81K words)
Quotes (1,741 notes, 169K words)

I'm very fond of quotes, but I also enjoy connecting ideas with the people behind them. That's why I have so many person notes.

Data Storage

The vault comprises:

17.91 MB of Markdown files
708.71 MB of attachments
1,159 public attachments
150 private attachments
193 Git commits tracking changes (content, configuration, etc)

Automation and Templates

I've developed:

27 core templates for consistent note creation
Automation for filing notes based on tags
Automation for backing up everything
Automated stats generation

Periodic Notes

1,204 daily notes averaging 209 words
178 weekly notes averaging 657 words
37 monthly notes averaging 2,047 words
4 yearly notes averaging 17,230 words

System Architecture

I use Obsidian as my Tool for Thought (TfT) and Single Source of Truth (SSOT). Everything I explore, capture, process, think and plan is in there. I use Obsidian for capturing the ideas of others, my own, as well as document systems/processes, organize my work, plan and track progress on my projects, and much more.

My knowledge management system in Obsidian is built on three fundamental pillars designed to make knowledge capture, processing, and creation as frictionless as possible.

First, structure:

The PARA method for simplicity, cross-platform compatibility, and flexibility
The Johnny Decimal system for systematic numbering, logical ordering, and maintaining lean organization

This combination provides both flexibility and structure. PARA (Projects, Areas, Resources, Archives) (imagined by Tiago Forte) offers a simple, and logical way to organize information that works across all tools and platforms. On the other hand, the Johnny Decimal system adds a systematic numbering approach and a few rules that keep folders lean and organized. For instance, all my Areas are numbered from 30-39, with sub-areas like "33 Permanent Notes" containing further categorized sections. Note that I'm not trying to make it all perfect, and I don't hesitate to violate "rules" when I feel the need to. The most important point about my structure is that it has been very stable for a long while. I don't waste time trying to make it perfect. As long as I don't feel too much friction, it's fine.

Second, I have templates and automation rules in place to remove tedious/time-consuming maintenance work and help me remain consistent:

Templates for all content types to ensure consistency: each main type of note has a dedicated template (e.g., periodic notes, meeting notes, literature notes, quotes, etc)
Automated filing to eliminate friction during content capture: I use plugins such as the Auto note mover plugin for Obsidian to automatically take care of the filing based on tags
Dedicated Resources folder for all attachments (1,309 files, ~700MB): I don't want to have to think about where to put attachments. They all go into a single "Attachments" folder. To find those back, I simply rely on links in the relevant notes

The key to maintaining a large knowledge base such as mine is automation. Every repetitive task that can be automated should be. My system uses templates not just for consistency, but as triggers for automation. When I create a new note, the template automatically adds the correct tags, metadata, and then the note is automatically moved to its proper location. This removes the cognitive overhead of "where should I put this?" and lets me focus on the content itself.

Third, and most importantly, there's an actual system in place. I'm not capturing mindlessly. I know about my goals, my priorities, and how I want information to turn into knowledge I can leverage. All the information I capture comes into a funnel, no matter the source, and ends up in my Tool for Thought. All the rest is considered transient

Core Design Principles

Friction Reduction & Scaling

Use as few tools as possible to cover the whole PKM process and reduce integration needs
Notes automatically move to appropriate folders based on tags
Templates for consistency (handle metadata and initial structure)
Automated periodic note creation and organization
Automated rollover of tasks from one day to the next
Kanban boards for projects and different time horizons
Projects based on metadata

Future-Proofing

Focus on the system, not on the tools. Parts should be replaceable
All notes stored as Markdown files
YAML front matter for metadata
No vendor lock-in
- Ensure the content "works" on its own
Git for version control (193 commits)
Tons of automated backups (local & remote)
Ensures easy AI integration

Organization

Everything ends up in my Tool for Thought. All the rest is transient
Combined PARA method with Johnny Decimal system and Zettelkasten
Flat structure where possible
Strategic use of subfolders only when necessary
Clearly isolate by type of content rather than by topic

The architecture decisions weren't obvious from the start. Early versions of the system used a deeper folder hierarchy that seemed logical but proved unwieldy as the system grew. The current flat structure with strategic subfolders emerged through iterative refinement, driven by both performance needs and usage patterns. In the future, I intend to further reduce the number of folders, and to rely on more live queries to create specific dashboards for important content types.

One crucial lesson was that the best technical architecture is often the simplest one that can handle the complexity of your use case. While it was tempting to build complex automation and organization schemes, the combination of plain text files, simple folder structures, and automated tag-based organization has proven remarkably robust and maintainable.

The Knowledge Funnel

I have a systematic process for going from content I discover, curate and decide to consume to a set of connected notes in my system, and eventually to published articles. I've documented this high-level process here: Personal Knowledge Management Process.

The main steps/phases of that process are: Explore → Curate → Consume → Capture → Distill → Organize → Connect → Develop → Create → Share

This "pipeline" represents the complete lifecycle of knowledge throughout my system. Each stage has its own purpose and tooling. For example, exploration might start with my RSS reader, or social media. For curation, I currently use Readwise. While consuming, I use Readwise Reader, but intend to switch to the new Obsidian Web Clipper in combination with the Obsidian Web Viewer. My goal is to reduce the number of tools that I use and depend on, further simplifying the system.

Every idea I capture is added to my daily notes, and I extract those into separate notes during my Periodic reviews, adding metadata, tags and links as I do so. Finally, the information rolls into my weekly notes, then monthly and yearly ones. I've described this knowledge ingestion process in the following articles:

During weekly reviews, I extract information from daily notes into dedicated notes and replace the original content with links. This process repeats at monthly and yearly levels, creating increasingly dense connection networks:

Daily notes: 1,186 notes (avg. 11.5 links)
Weekly notes: 178 notes (avg. 45.7 links)
Monthly notes: 37 notes (avg. 161.6 links)
Yearly notes: 4 notes (avg. 1,443.8 links)

This progressive summarization and linking strategy explains why higher-level periodic notes have significantly more connections – they serve as indexes and connection hubs for all the knowledge captured below them.

Last but not least I have a solid backup solution in place for my notes, which is critical because this system is central in my life and work. I described it here:

Content Organization

In my system, I isolate different content types. This enables me to implement different workflows for different types of content.

I also separate content based on its source/authors:

Literature notes: Third-party content, processed inputs
Permanent notes: My own content, refined knowledge

The distinction between literature notes and permanent notes, which comes from the Zettelkasten method is crucial for me as an author and content creator. Literature notes contain processed insights from external sources, as much as possible with clear attribution. Permanent notes are where I develop my own ideas, building on but distinct from the literature notes. This separation ensures I never confuse others' ideas with my own and helps maintain intellectual integrity.

I further separate note types based on the main following tags:

#articles/published (251 notes)
#book_notes (156 notes)
#mocs (116 notes)
#meeting_notes (16 notes)
#my_quotes (238 notes)
#people (630 notes)
#quotes (1,737 notes)
#daily_notes (1,186 notes)
#weekly_notes (178 notes)
#monthly_notes (37 notes)
#yearly_notes (4 notes)
#private_notes (for sensitive content)

Tags provide a flexible layer of organization on top of the folder structure. They allow notes to belong to multiple categories without duplicating files. More importantly, they serve as triggers for automation. My system watches for specific tags and automatically files notes accordingly. This tagging system also enables powerful queries and filtering, making it easy to find and combine information in different ways. Note that I use hundreds of additional tags for describing the content. And yes, an improvement could be to use "type" prefix for those tags (e.g., type/quote), but I'm not interested in changing right now.

The Resources folder deserves special mention. Rather than manually organizing attachments (images, PDFs, etc.) into topic-specific folders, everything goes into a single Resources folder with a unique name (and usually prefixed by the current date as YYYY-mm-dd - ). This significantly reduces the friction of adding attachments to notes and eliminates the cognitive overhead of deciding where each attachment should go.

The organization challenge became apparent around the 1,000-note mark. Manual filing was not only time-consuming but also inconsistent. The key insight was that tags are more flexible than folders for organization. While a note can only exist in one folder, it can have multiple tags. This allows for more natural organization patterns and enables automation based on clear rules rather than subjective decisions.

System Documentation

As I started working on my Knowledge Management course, I realized that I could further improve the documentation of my own system. For that purpose, I created a free template that you can find here: PKM System Handbook Template.
I built it based on my own documentation (i.e., My PKM System Handbook), and ensuring it aligned with my vision of the Personal Knowledge Management Process.

If you're curious about this, then check out my upcoming course:

Key Learnings

Use friction as a guide: No need to tweak the system all the time if friction is low. Focus on using the system instead of trying to make it perfect
Documentation Helps Improve: When I turned my system into a product, I had to document it. This enabled me to improve it a lot, because it highlighted all the caveats and parts that needed some design
Plain text is future-proof: Markdown is where it's at. Data privacy is not the only concern. Also, there are so many ways to leverage the information (e.g., recently with the Model Context Protocol (MCP)) compared to what proprietary platforms enable
Manual organization breaks down past 1,000 notes: And actually even before that
Atomic Notes Scale Better: Small, focused notes enable better linking and reuse
Tags > Links > Hierarchy: A flat structure with dense connections provides better navigation than deep hierarchies. Combine tags, links and Maps of Content to increase the link density
Maps of Content as Hubs: Maps of Content serve as powerful entry points to leverage the accumulated knowledge. I currently use MoCs more and more using complex queries, especially with AI
Templates Rule: Use templates to maximize consistency and ease automation
Automation is Critical for Scale: Templates help maintain consistency
Use a single Source of Truth for your knowledge: Centralize as much as you can instead of maintaining silos. The Tool for Thought is where knowledge lives on. All the rest should be transient
Git is Your Friend: Assuming you have plain text files, Git is a great companion for your Knowledge base. I regularly save file hunks to track my changes more precisely
You Need a Serious Backup Plan: Catastrophes will happen. Be ready

Why I Publish My Notes

Out of my nearly 8,000 notes, 6,478 are public. This isn't by accident – I believe in the power of shared knowledge. Personal Knowledge Management remains an underutilized tool that can transform how people learn, think, and make decisions.

By publishing my notes, I aim to:

Demonstrate the real-world impact of systematic note-taking
Help others understand PKM's potential for personal development, career growth, and creativity
Share practical examples of knowledge management in action
Contribute to the broader conversation about learning and knowledge work

Making PKM Accessible

After two years of refining my system, I packaged my experience into the Obsidian Starter Kit. It's not just a template – it's my entire PKM "infrastructure":

My complete folder structure
My carefully selected and configured plugins
My battle-tested templates and workflows
My automation scripts and processes

Nearly 1,000 users have adopted this system, validating its effectiveness and scalability. The kit helps people skip the months of trial and error I went through, providing a proven foundation for their PKM journey.

And as I mentioned before in the article, also make sure to check out my upcoming Knowledge Management course

Does it still matter in 2025+?

You might wonder if all of this stuff is still meaningful now that AI is changing the world. What's the point of accumulating knowledge in a Tool for Thought if you can ask Deepseek, GPT4o, Claude, Llama & al for all the information in the world.
Well to me, the answer is a resounding YES. Let me explain why.

LLMs know it "all", but don't know much about what YOU know, think, and want. And unless Neuralink makes a huge breakthrough and can start reading your mind, that won't change for a long while. Every time you start a discussion with AI, you're starting over. Yes, you may create a project, update documents, etc, but at the end of the day, you're just facing the main character from Memento. But if you accumulate your own knowledge and ideas in a Tool for Thought, then you're creating content that you'll be able to leverage with AI. And then it becomes much more interesting. In fact, you can already do this using things such as the Model Context Protocol (MCP), and various tools. Using those, you can share your own ideas with LLMs, and start the discussions from there. I'm doing this a lot right now, and I see the value, compared to just starting from scratch every time.

Beyond the leverage it creates when combined with LLMs, having your own (serious) knowledge base if a great way to create a place where you can think deeply. A fourth place. It's an idea I discussed in a previous article. The gist of it is that you need a space where you can organize your ideas, explore those further, store the results of your experiments, capture important achievements, things you're grateful for, what inspires you, etc. It not only creates a space where you can think, but also a place you can use to look at the past more objectively. And to me, that's invaluable.

Conclusion

What started as a personal project has evolved into something much more interesting: a laboratory for me to understand the world around me, to think, to develop my own ideas, and to share insights with others.

Building a knowledge management system that scales requires careful consideration. But few of us have the drive, the time, or the need to spend so much time building one. Many try, and fall in different traps, such as hopping from tool to tool all the time, making things too complex, relying too much on folders, not paying attention to vendor lock-in, etc. That's why I'm convinced that sharing my lessons learned is always useful to someone out there. And that's also why I'm working hard to release my damn Knowledge Management course that's been in the making for too long ;-)

That's it for today ✨

Bonus: Stats Generation Scripts

Here's how I generate the stats for my Obsidian vault.

First of all, I use the following template using the Templater plugin for Obsidian. It's a template with custom code that analyses the vault and generates the raw stats. The script also writes the stats to two different files:

Stats.md, which I publish on my notes Website
stats.json, which I then process with another program

You can find it here: https://gist.github.com/dsebastien/99c5fe34a954e2f65a89022e5221328e

Then, I run a JavaScript script using node: node ./statsToHtml.js. That script generates the HTML version that I then publish on GitHub.

You can find that script here: https://gist.github.com/dsebastien/026ae53e891eab3d28c9f00e1ff9b358

The HTML file generation could also be done by the Templater template, but I don't care much about improving this right now.

About Sébastien

I am Sébastien Dubois. You can follow me on X 🐦 and on BlueSky 🦋.

I am an author, founder, and coach. I write books and articles about Knowledge Work, Personal Knowledge Management, Note-taking, Lifelong Learning, Personal Organization, and Zen Productivity. I also craft lovely digital products . You can learn more about my projects here.

If you want to follow my work, then become a member.

Ready to get to the next level?

To embark on your Knowledge Management journey, consider investing in resources that will equip you with the tools and strategies you need. Check out the Obsidian Starter Kit and the accompanying video course. It will give you a rock-solid starting point for your note-taking and Knowledge Management efforts.

If you want to take a more holistic approach, then the Knowledge Worker Kit is for you. It covers PKM, but expands into productivity, personal organization, project/task management, and more:

If you are in a hurry, then do not hesitate to book a coaching session with me:

Personal Knowledge Management at Scale - Analyzing 8,000 Notes and 64,000 Links

Sebastien Dubois

Introduction

Let me listen to this 🎙️