In the vast ecosystem of coding assistants and IDE plugins, one open source project gets steady attention - Aider. Designed by Paul Gauthier, Aider is not just another tool; it's a forward-thinking fusion of advanced AI and programming utility.
A Dynamic Coding Assistant
At its core, Aider functions as a coding companion, aiming to enhance the experience akin to pair programming. But instead of a human partner, you're collaborating with GPT-4, one of the most sophisticated AI models available today.
How Does Aider Work?
Aider's modus operandi is unique. Instead of a complex GUI or integrated in an IDE, it operates as a chat-bot on the command-line interface. The program reads an existing repository, then crafts and suggests changes based on user prompts. From adding new features, programming additional functions, to even refactoring, Aider wears many hats.
One might wonder about the feasibility of integrating AI-generated code into an existing project. Aider solves this by utilizing c-tags to create a 'map' of the software. This map allows GPT-4 to maintain a holistic view of the project without requiring the entire source code in its immediate context. As a result, the AI model can understand, analyze, and enhance substantial portions of the project seamlessly.
Learning from Mistakes
Errors are part and parcel of coding. Unfortunately this even holds true when you are using an AI coding assistent. But Aider also helps with correcting errors. Aider has the capability to use error messages from program runs to derive and suggest corrective actions. It's akin to a feedback loop, but with coding.
In Conclusion
The software development world is ever-evolving, with AI playing an increasingly significant role. Aider stands as a testament to this evolution, seamlessly marrying coding with the capabilities of GPT-4. As developers look for smarter tools to augment their skills, Aider might just be the next big thing in their toolkit.
A "context window" refers to the maximum length or number of tokens from the prior textual input that a language model, such as a Transformer, can consider and process. It determines the model's ability to handle long sequences and is constrained by factors such as the model's architecture, time, and memory complexity.
Why is it important?
The size of the context window is important for many reasons. Think about comparing several large contracts to one another.
In code generation the size of the context window is a critical factor since it influences the model's ability to understand complex code structures, maintain consistency, perform transformations, integrate with existing code, and more.
Why can't we just make bigger context windows?
Attention based LLM have quadratic time and memory complexity with respect to the length of the input. This increasing the context window size means that the computations required grow rapidly, leading to longer training and inference times.
Also the memory consumption grows rapidly. This limits the the feasibility of using very large context windows on typical GPUs as well as high end hardware.
What can we do about this?
We can enhance the model via fine tuning. This way we include part of the context as parametric knowledge in the LLM.
Another approach is Retrieval-Augmented Generation (RAG) where we add source knowledge to the input / context in a "just-in-time" manner [1].
Other approaches aim at using the context window as efficiently as possible. See the aider-project for example.
What trends do we see?
In a recent paper "LongNet: Scaling Transformers to 1,000,000,000 Tokens" the authors suggested a new way to handle long contexts. This seems to make long context windows computationally feasible [2].
However in another paper "Lost in the Middle: How Language Models Use Long Contexts" it is shown that longer context windows lead to poor quality in the results that we get back from LLMs [3].
So we have to see how this plays out in the meantime fine tuning and RAG will be our most generally usable approaches.
[2]: Jiayu Ding and others, ‘LongNet: Scaling Transformers to 1,000,000,000 Tokens’ (arXiv, 2023) <http://arxiv.org/abs/2307.02486> [accessed 2 August 2023].
[3]: Nelson F. Liu and others, ‘Lost in the Middle: How Language Models Use Long Contexts’ (arXiv, 2023) <http://arxiv.org/abs/2307.03172> [accessed 2 August 2023].
Retrieval-Augmented Generation (RAG) is a powerful method in Natural Language Processing (NLP) that leverages the capabilities of pre-trained language models and information retrieval systems to generate responses. It's especially beneficial when a language model needs to access information beyond its training data.
RAG operates by initiating with a user's question or query, followed by the retrieval of relevant documents or passages using an information retrieval system. This could be based on keyword matching, semantic search, or other techniques. The retrieved documents serve as context for the language model, which then generates a fluent, coherent, and factually correct response.
Here are the steps for RAG:
1. **Embedding Creation**: First, each document in the database is transformed into a semantic embedding using an embedding model. This model could be a pre-trained language model, a transformer model, or any other model capable of creating meaningful embeddings.
2. **Embedding Storage**: These embeddings are then stored in a database or an index. This allows for efficient retrieval of documents based on their embeddings.
3. **Query Embedding**: When a user asks a question, the question is also transformed into a semantic embedding using the same embedding model.
4. **Document Retrieval**: The system then retrieves the documents whose embeddings are most similar to the question's embedding. This is typically done using a method like cosine similarity or nearest neighbor search.
5. **Answer Generation**: The retrieved documents are then provided as context to a language model, which generates a response.
Parametric versus source memory
The effectiveness of RAG lies in its combination of parametric and source memory. Parametric memory refers to the static knowledge encoded in the model's parameters (weights and biases) during its training phase. On the other hand, source memory pertains to the dynamic, external knowledge that the model accesses during inference, such as the retrieved documents in response to a user's question. This combination allows RAG to generate responses that are not only fluent and coherent but also factually accurate and up-to-date.
Semantic embedding
A critical component of RAG is semantic embedding, a technique that transforms words, phrases, sentences, or documents into vectors of real numbers, capturing their semantic content. In RAG, semantic embeddings are crucial in the document retrieval step. Each document in the database is transformed into a semantic embedding using an embedding model and stored in a database or index for efficient retrieval. When a user asks a question, it's also transformed into a semantic embedding, and documents with similar embeddings are retrieved.
The advantage of semantic embeddings is their ability to retrieve documents based on semantic content rather than just keyword matching. This allows the system to retrieve relevant documents even if they don't contain the exact words from the user's question, making RAG a more powerful and flexible approach compared to traditional information retrieval methods.
Is the Zettelkasten only useful at the frontiers of knowledge?
Niklas Luhmann developed the Zettelkasten (ZK) to find new approaches in sociology. The ZK methodology is primarily discussed in academic research or as a method of creative writing.
Quite obviously, then, the Zettelkasten helps work on the "frontiers of knowledge." In other words: Zettelkasten at the frontiers of knowledge
But can the slip box also be used when not doing academic work? How can the note box help me if I don't have the goal of writing blog articles?
The Zettelkasten is a method to learn complex subjects. It doesn't matter for your progress, whether you are researching something wholly new or whether you want to familiarize yourself with an already well-known area. In both cases, it is opening up a conceptual world and relating the various sub-concepts to one another. Here we can use the Zettelkasten to our benefit.
One can explore a new subject area through the concepts described in one's own words as notes and their careful linking. It also leads to further questions and ideas by connecting them with other topics.
The note box is therefore not only suitable for scientists. It may help students to learn a subject area. But also managers, musicians, artists, athletes, software developers, and many other professions have to deal - more than ever - with complex terms and concepts. The Zettelkasten can help all of them.
A Zettelkasten is a structured collection of notes. The Zettelkasten is a tool to approach complex ideas and concepts. It serves to support and structure one’s thinking about essential topics. Thus the note box can help to formulate and publish own thoughts.
Zettel is the German word for a small piece of paper, and Kasten means box. So a Zettelkasten is a box full of small pieces of paper. Therefore a Zettelkasten is often referred to as a slip box.
The idea of a slip box goes back to Niklas Luhmann (1927-1998) [1]. An exceptionally productive sociologist measured by his publications. Söhnke Ahrens explained Luhmann’s approach in his book “Das Zettelkasten-Prinzip” [2] and contributed significantly to popularizing the concept. The book was translated into English under “How to take smart notes.“ Through this title, one can find numerous explanations of the methodology on the Internet.
Every single note, every single piece of paper meets the following criteria:
The note is permanent. I.e., after we place it in the note box, we should not change it.
We should write the notes independently. Especially we should not copy-paste from other texts. This step of reformulation makes the note part of one's thinking.
The notes should be "atomic." So there should be one idea per note.
We sort these notes one after the other so that they are arranged as trains of thought or along the lines of reasoning. It is vital that we can change that order easily. Therefore we must formulate notes without context. That is, each note must make sense on its own. Even after years, we must understand the text without referring to other notes. In this sense, notes are "context-free" and thus usable in isolation from each other. In particular, we should add citations to each note [3].
We link the notes together. Navigating the various "stems of thoughts" in our Zettelkasten allows us to see new relationships between the notes. Reviewing the Zettelkasten may lead to unique insight and ideas [3].
Only accept Notes in your Zettelkasten on Topics you deeply care about in years to come. We should think about each note on how to write it and link it. Notes you take while reading or notes on tasks or projects are not part of the note box.
When we follow the rules above, the note box enables the association between ideas and thoughts that one has encountered at very different times and in very different contexts. The Zettelkasten becomes a creativity tool.
Matt Giaro found a very apt quote from Mark Twain here [3]:
"There is no such thing as a new idea. It is impossible. We simply take a lot of old ideas and put them into a sort of mental kaleidoscope. We give them a turn and they make new and curious combinations. We keep on turning and making new combinations indefinitely; but they are the same old pieces of colored glass that have been in use through all the ages."
-- Mark Twain
How to set up the Zettelkasten?
The Zettelkasten is not based on individual physical pieces of paper. We store each note as a Markdown file containing images, formulas, or code snippets. Thereby we build a digital form of the Zettelkasten.
We store the Markdown files in a simple folder with the following directory structure:
The notes are contained in the "mycelium" folder. Here the order of the notes is shown by a naming scheme of these files. We store images in the folder "mycelium/images."
The folder "input" contains unfinished notes that do not yet have the necessary quality to be sorted into the note box (i.e., the folder "mycelium").
So technically, the Zettelkasten is nothing more than a collection of markdown files. Thus, the Zettelkasten is independent of applications or cloud offerings. Cross-device use can be achieved, for example, by creating folders on Dropbox. But we can also decide not to use the cloud. For Backup and versioning, we can use a versioning system like Git.
We use a strict naming scheme for the Markdown files to form the structure of the Zettelkasten
A typical name for a Markdown file could be, for example:
It sounds like a tedious way to name files but stay with me. We have some scripts to help you here.
The name consists of 3 components:
01_09 denotes the location of the note in the Zettelkasten. In this case, the note is located in subject area 01 at location 09. So the file "01_08_How_to_integrate_notes_into_the_note_box_e0d27e3ad.md" contains a thought that leads to the considered note. The files 01_10_ ... contain subsequent thoughts. A file 01_09_01_note_making_from_fleeting_notes_eb0a34faa.md contains a detailed thought about the note, which we could continue as a sub-discourse in the files 01_09_02_... In this way, when viewed in a simple file manager, the files are logically ordered correctly.
"From_note_taking_to_note_making_to_communicating_with_your_note_box" denotes the name of the note. So that you can immediately see what the note is about without having to open the file, the file name should be as descriptive as possible. The name must always start with a letter.
1b6058a3a is a unique identifier of the note. This identifier is necessary to automatically correct references between notes after restructuring the note box.
This way, you get a good overview of the content and structure of the note box already when you view it in a file manager. No particular application is needed!
The structure of the Zettelkasten can be seen in a file manager
Structure of a single note
A note is a straightforward markdown file that starts with a heading. It can contain images, formulas, and code snippets.
Here is an example:
As we will see, some scripts will help you handle the files in the Zettelkasten and adjust the links between notes if you change the structure in the Zettelkasten. I open sourced these Scripts under https://github.com/rreben/tools4zettelkasten
Working with the Zettelkasten
Adding new notes to the Zettelkasten
I save my notes from books or articles I've read in Evernote or Zotero. I then go through these notes afterward and write down essential ideas as isolated, context-free text in individual Markdown files in the input/ folder. Also, when I have an idea or insight during my workday, I jot down those ideas as individual Markdown files in the input/ folder.
At this step, I don't care about the file name yet. I name the file whatever is quickest.
I go through these markdown files and revise the text to be able to read and understand the note in isolation. I add source notes and formulate a clear, descriptive heading.
I then run a script using the command:
python -m tools4notebox stage
This command renames the files. The filenames now have the form: 0_0_How_to_integrate_notes_into_the_note_box_e0d27e3ad.md. Here, the speaking part of the file name How_to_integrate_notes_into_the_box is derived from the heading of the file. The script generates a unique identifier and prepends the 0_0.
Now I look at which place in the Zettelkasten the file fits. Let's assume that in the note box (folder mycelium/), there are the following files:
Then the file fits between the 12 and 13 in the 01 strand of the Zettelkasten. I will then rename the file to 01_12a_How_to_integrate_notes_into_the_note_box_e0d27e3ad.md and move the file to the mycelium/ folder.
Let us have a look at this example: Another file: 01_12b_Working_with_alphanumeric_Ordering_en036a6bd.md has been integrated. With the files 01_12a_1_Integration_of_fleeting_notes_97a19382a.md and 01_12a_2_Integration_of_literature_notes_ec83f31a2.md a sub stem of thought has been created.
Instead of now having alphanumeric abbreviations like 5_16a_1n_... in my slip box. The classification abbreviations can also be canonicalized with a script.
This way, the structure always remains easy to read. Links between notes are automatically adjusted by the script so that the links still work after renaming.
With this mechanism, we can easily adjust the structure of the Zettelkasten.
Analyze the Zettelkasten
There are many discussions about the best structure of the note box. We group notes into strands of thoughts or lines of reasoning. This way, they are organized thematically about the alphanumeric sorting. We can easily find the notes we are looking for.
From note taking to note making to communicating with your zettelkasten
Niklas Luhmann focused his Zettelkasten on his work as a scientist. Writing and publishing were an essential part of his work, and therefore the note box is very much focused on the writing process.
I work in IT. Writing and publishing have more of a secondary role in my work. But understanding ever-changing IT architectures, development tools, infrastructures, and the concepts and objectives behind each is crucial to me. For me, arriving at the proper assessments here is primarily a learning process. For me, the slip box is a central learning tool for penetrating complex concepts and developing new solutions.
In doing so, the process of decomposing, i.e., breaking down a construct into individual components, is very important to me. I keep the original construct by ordering or hierarchy of the elements in my box of notes. I then either trace the individual elements back to what is already known to me or identify it as something originally new.
Anne-Laure Le Cunff has presented in an excellent article [1] the difference between "note-taking" and "note-making." It is essential for the understanding of complicated contexts to summarize the contents of a foreign text in one's own words.
In books and longer texts in general, key ideas are examined several times from different points of view and in other contexts. Then the notes mustn't be just simple (linear) summaries of the foreign text. Instead, each note should only contain one thought. Thus a text is "decomposed" into individual notes. These notes are then put into context and linked to other existing notes. We should continue to pay attention to good referencing to do a correct citation if this is necessary.
Eva Keiffenheim describes the process through three levels of generated notes: Fleeting Notes, Literature Notes, and Permanent Notes. [2] In any case, there has to be a very careful quality filtering of all notes that we add to the note box. Otherwise, there will be a flood of notes. These notes might contain a lot of information but are not accessible due to their poor quality.
After note-taking, it must be possible to communicate with the Zettelkasten. One has to look at the notes (relevant in the respective work context) again and again. Here I am currently experimenting with special notes with guiding questions. I try to answer these questions in "trains of thought" (linear arrangements of notes).
It is necessary to build rather local structures in the Zettelkasten with relatively few far-reaching, cross-topic links. Eva Thomas has elaborated this in a remarkable article. [3]
We need to enable cliques in a knowledge graph in a mathematical sense. I want to form these structures in my Zettelkasten, which has led me to work with Tools4Zettelkasten on my solutions for building a simple plain-text-oriented system.
Hierarchy vs. emergent structures through links between notes
The tool Obsidian makes it very easy to create links between notes. The graphical mode in Obsidian also displays only those links that the user in the text explicitly created.
Niklas Luhmann also linked his notes to each other. But that wasn't the only way he brought structure to his note box. Luhmann organized the notes and devised a system especially for this. He could insert further notes between two notes, even whole subhierarchies of notes.
It was important to him to jump between topics and thus find surprising associations or thematic links. But it was at least as vital for him to put his notes in good order. In this way, he built up arguments, documented "trains of thought," and critically reviewed them repeatedly.
In German, "Train of Thoughts" is also called "Gedankengang," which means "walk of thoughts." These trains of thought are not just arbitrary paths through a network of notes but well-considered and repeatedly revised steps to test and present an idea.
This way of dealing with trains of thought is fundamental to me. That's why I'm working with the "Tools4Zettelkasten" on a system that enables simple networking and structure building via sequence and hierarchy of notes.
Filtering input correctly for the Zettelkasten (slip box)
Only valuable notes are allowed in the slip box. These are insights, i.e. thoughts independently formulated by oneself. [1]
Collection of information, references, potentially once useful may not be included in the slip box! Otherwise the slip box will quickly become confusing, full of what is only temporarily valid [1].
The note box must not become a universal record or general project file. These things belong either in a chronolgically kept journal or in project files.
On the other hand concepts, procedures, or key insights don't belong in the project file, but in an overarching repository (Anki or slip box).
A collection of information becomes more complex and cumbersome the larger it gets. The (well-managed) box of notes is different. It becomes more useful as it contains more ideas [1].
This positive dynamic arises on the one hand because there are simply more and more useful things in the note box, but above all also through the links that grow superlinearly with the content and thus promote creativity and new insights [1].
[1]: Sönke Ahrens, Das Zettelkasten-Prinzip: erfolgreich wissenschaftlich Schreiben und Studieren mit effektiven Notizen (Norderstedt: Books on Demand, 2017).
Mycelium of Knowledge (MoK) is about methods and tools. We will develop some tools open source and publish them on Github. MoK should be as open as possible. Therefore it must be low tech, i.e. that we do **not** build an integrated solution. The tools should only do simple things that you could do by hand (with some effort). We don't want to build a central framework with plugins but rather a loose collection of tools that are underpinned with ideas for the personal methodology (or vice versa). This is the only way to achieve true portability.
operation on-premise or in the cloud
also without server (e.g. file based from dropbox)
with a database or without
for just one user or a group
Mobile or on PC / Mac
Synched over several devices or based on the local file system
You should always have paper and pencil with you to be able to make input for the note box Zettelkasten at any time.
You read something. What is interesting? How would You put it in your own words?
You have an idea. How can You capture it? (even in two years You should be able to recognize the idea from the note).
You have important insights. How can You formulate them in a comprehensible way?
All this is written on short notes. These notes are then incorporated in the Zettelkasten. The entirety of all thes interlinked notes provide a valuable basis for your work in the future.
Each input should be classified or linked from the keyword register, so that nothing gets lost in the box (dangling references).
[1] Sönke Ahrens, Das Zettelkasten-Prinzip: erfolgreich wissenschaftlich Schreiben und Studieren mit effektiven Notizen (Norderstedt: Books on Demand, 2017).
