Another week, another newsletter! I hope that you all had a great one 🤩
Hello there! This week has been really fun.
My kids are at home for the last week of vacation, so there wasn't much time for me to work on the Knowledge Management course, but I've used the little time I had to write, and explore cutting-edge AI image generation models. And woah, I'm super impressed, and I want to tell you more about it. Put your seatbelt on!
Alright, let's gooooo 🚀
The lab 🧪
Since my work time was very limited, I've decided to do some R&D work. A while ago, I heard about FLUX.1, the new AI image generation model of Black Forest Labs, and wanted to give it a try. Black Forest Labs is a group composed of people who were behind Stable Diffusion, a popular open-source model. I knew it was worth checking out, and I wasn't disappointed.
FLUX.1 is able to generate very realistic faces, and hands. It's also able to generate text much better than previous text-to-image image generation models. It is currently (August 2024) the number 1 model on Imgen Arena.
And I can tell you that The Verge is spot on with their article, we're not ready for this. The images that FLUX.1 is able to generate are hyper realistic, and with some effort, it's going to fool everyone. Fake images are now going to be the default in this world. And fake videos will also be the default real soon... It's both awesome for creativity, and scary for the future.
As of August 2024, there are three main variants of FLUX.1:
flux-pro: the state-of-the-art model
flux-dev: a 12B parameter version of FLUX.1
flux-schnell: a fast variant tailored for local development and personal use
At first, I tried those base models to generate various images, such as this one:
Prompt: obsidian rock in the forest with the word "Obsidian" engraved. Canon pro photo, 50mm, bokeh, flare, vignetting. Image generated using AI
I wanted to see how well it was able to generate text, and was amazed, because it's MUCH more powerful than all the previous models I've tried:
Image generated using AI
The ability to generate images with text is one that I was really after. It's an enabler to create powerful visuals for my articles.
I also tried generating images with different styles:
Image generated using AIImage generated using AIImage generated using AI
Each and every time, FLUX.1 generated amazing results. It rocks. Even using 'schnell', the fastest variant.
I started using Replicate.com, an online platform that enables running machine learning models in the cloud. The models run on their infrastructure, and they provide an API and SDKs to interact with the platform. Using Replicate.com, you can use various AI models, including those hosted on HuggingFace.
It's possible to use the models either through their Web user interface, or using the API/SDKs:
Using AI models through the user interface of Replicate.com
Here's an example of how to use a model through (JS) code:
import Replicate from "replicate";
const replicate = new Replicate({auth: process.env.REPLICATE_API_TOKEN});
const model = "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478";
const input = {prompt: "a 19th century portrait of a raccoon gentleman wearing a suit"};
const output = await replicate.run(model, { input });
console.log(output);
Quite straightforward if you know how to code!
Then, I read a few articles, and learned more about Low Rank Adapters (LoRAs). I discovered how to train custom versions of FLUX.1 using my own photos, and I realized this was the next level.
After a bit of trial and error, my first model was created, and I could use it from Replicate.com (and anywhere else actually):
A FLUX.1 LoRA I trained with my own face
And now, I can easily generate images such as these:
Fake dSebastien. Image generated using AI
This image was generated using the following prompt: realistic photography extreme close up of a man called dsebastien with white painted floral profoundly sad contorted pouty face with a angry scowl of a frown defined with black makeup looking directly forward with an extremely sad contorted frown white painted faces
Fake dSebastien. Image generated using AIFake dSebastien. Image generated using AIFake dSebastien. Image generated using AIFake dSebastien. Image generated using AIFake dSebastien. Image generated using AIFake dSebastien. Image generated using AI
As you can see, while I know it's not me, it's actually pretty accurate.
Later, I'll try to train custom models locally, to see how hard it is. I don't think it should be too complicated.
I've also experimented with creating visuals for LinkedIn and Twitter. I was not disappointed:
Image generated using AIImage generated using AIImage generated using AIImage generated using AI
I feel like this can severely impact people who design for a living, to say the least. Hopefully, many of them, including the ones I know, tend to adopt those tools to boost their productivity...
I then trained custom models with my kids, and we had some family-time fun:
My son in an action movie. Image generated using AIMy son in an action movie. Image generated using AIMy daughter on a Disney movie poster. Image generated using AI
I'm still amazed at how powerful this model is! Imagination is the only limit, or so it seems. ❤️
What matters to generate powerful images is writing clear, and precise descriptions. And since that's just text, LLMs such as ChatGPT and Claude.ai are perfect tools to improve the prompts.
One resource I found really useful is MidLibrary, a collection of visual styles, artists, and visual features:
Right now, I'm also exploring the possibilities to combine multiple LoRAs, to be able to generate images leveraging different models in a single generated image. This should enable me to generate images combining different styles, different people, things, etc:
Aside from that, I've also built an Obsidian plugin to integrate image generation capabilities using Replicate.com. Now, I can easily generate images directly from the Obsidian editor:
Demo of the Obsidian replicate plugin I've built
The plugin works great, and I will soon submit a PR to the Obsidian team, to turn it into an official community plugin 🚀. For now, the source code is available if you want to install it already:
The recent launch on ProductHunt continues to bring more attention to the Obsidian Starter Kit, but sales don't quite follow. I feel like the current price may be judged too high, which I find disappointing given how much effort I've already put into building it, and the overwhelmingly positive reviews/testimonials. I'm thinking about launching a survey, asking the opinion of all past customers, to try and find a better price point...
I only sold 21 digital products in the past month, and the trend is far from reassuring:
My sales on Gumroad.com in the past 30 days
Clearly, my sales engine is rusty. Something is definitely wrong. The problem is that I don't know what and why. It may be because of the way I mention products in articles, because I don't reach new people who could be interested in my products, because of the prices, etc. I really have no clue, and it's tough for me to find out.
If you haven't bought anything from me, then I'd really be curious to understand "why". Don't hesitate to reply to tell me, I'm all ears.
This newsletter is also on a standstill. We're still at $112/month, with 19 paid subscribers out of 1724:
DeveloPassion's newsletter stats
I continue to hope that more of you will decide to become paid subscribers, and actually support the work that I do, making it more reliable as a source of income for me as a creator.
Subscribe now ❤️:
So, it's one more month where I just cannot pay myself, and struggle to keep the business alive. It's scary, but I'm not willing to give it all up just yet. I'm five years in, and I really have this feeling that I could be a few steps away from success. Who knows!
New articles
After seeing the same questions over and over on the Obsidian forums and on Reddit, I decided to write a small blog post to share my recommendations for newcomers:
Many people seem to procrastinate, hopping from one tool to the next, looking for the silver bullet. It's an anti-pattern, and I want to help them avoid that.
New published notes
This week, I took some time to describe my communications plan. You might want to check that out, if you're curious about how to stay in the know:
As I mentioned before, I've also started collecting notes about AI and AI image generation. I intend to expand that part of my knowledge base a lot more in the future:
By the way, consider the fact that Zettelkasten, and Knowledge Management in general, are not only for writers and content creators! Above all, those are THINKING tools.
dsebastien
Sebastien Dubois
Spencer Greenberg
Stowe Boyd
Steph Ango
