Using AI to Generate a Children's Book

June 16, 2023 ·Updated January 17, 2024

“The Amazing Animal Alphabet Adventure” book with a little boy’s hand grabbing it off the table.

I’ve done my fair share of traveling with little ones under the age of two and one of the things that I ALWAYS like to bring with me are books. When I first started taking books with me I could only ever take one or two because children’s board books are so large and bulky. This was fine for around town, but when my husband and I decided to take a 5-week trip to Europe with a one-year-old, I knew that only one or two books wasn’t going to cut it. I also didn’t want to take a stack of board books with us either. So, I improvised by taking photos of our existing books and printing them out on 5 1/2-inch square books. I was able to make fourteen books to take with us on our trip, and they were perfect! They were lightweight and small enough that they were easy to pull in and out of the diaper bag on planes, boats, in cars, and at restaraunts, but not so small that they got lost or were hard to read.

At the end of the trip they held up so well and were so portable that I decided to always keep all fourteen in my diaper bag. Even a year later they’ve held up better than the board books, and my little one still loves them.

But, there’s one problem with all of this… copyright. Even though I bought these books, I’m not supposed to make copies of them. Now, I’m not reselling the books in any way and they have only been for personal use - never commercial. I’ve never given any out as gifts or for others to even borrow. That being said, you should definitely not go and take photos of your own books and make copies. This is a bummer because there’s not a lot of good options out there for travel-sized kid’s books, and I’ve had lot’s of people ask about where I got mine. So, I decided to make my own perfect travel-friendly children’s books, starting with “The Amazing Animal Alphabet Adventure” now available on my Etsy shop! Of course, I’m not a children’s book writer or an illustrator, but I do have a degree in computer science, so I played around with two different AI generators to help me out.

Process of Writing a Children’s Book With Only AI

First, I needed a story. My initial thought was to take old fairytales, like the three little pigs or the ugly duckling, and use an image generator called Stable Diffusion to create the illustrations. I quickly realized that this wasn’t viable since image generators, as they are right now, have difficulty creating the same character across multiple images. For example, having an AI generate an image of an ugly duckling with its mom and then generate another image of the same duckling running through a farm gate isn’t really possible. This makes illustrating a story very difficult. In the images below you can see how each image it created did show a part of the story but with no continuity in style.

The three little pigs images generated using Stable Diffusion:

The ugly duckling images generated using Stable Diffusion:

After a couple of weeks of brainstorming, I decided that creating a simple alphabet book would be much easier on the image generator since I wouldn’t need continuity between characters. That’s when I hit another snag - coming up with nouns for all the letters of the alphabet. So, why not use AI to generate that too? I opened up ChatGPT, a text generator, and asked it to write a children’s alphabet book with an animal theme. It did a great job, not only coming up with animals, but also writing short poems to go along with each letter and animal.

ChatGPT output A is for Alligator. In the marshy swamps, lurking so low, An alligator waits, ready to go. With sharp teeth and a powerful jaw, It’s a reptilian wonder, full of awe.

Then I took the animals it came up with and started crafting a prompt for the image generator that would generate a similar style but different animal illustrations for each letter. This took several hours of trial and error before I was able to get an illustration style I liked. Then it was just a matter of plugging in each animal and let the AI generate the illustrations. I found that it had a hard time with some animals more than others. For example, dolphins, whales and narwhals did not come out with complete bodies or would have multiple fins.


6 different AI generated illustration of what should be dolphins.


6 different AI generated illustration of what should be whales.


6 different AI generated illustration of what should be narwhals.

Despite this, I was able to find combinations of animals and prompts that generated good results.

Screenshot of Stable Diffusion output showing consistent animal output results.

How Do AI Image Generators Work?

Simply put, an AI image generator, like Stable Diffusion, takes text as an input and tries its best to create an image based on the input given. To make a good AI image generator you train a model (a program that has lots of knobs that can be turned up or down) on a large set of images that already have text descriptions. With these images you can use a “training” technique that involves giving the model one of the text descriptions, and then rating how close the generated image is to the actual image. The model then changes some fo the knobs, and tries to guess again. It does this over and over again until it gets an image as close to the original as possible.

For example, I’d show the model a picture of a dog, and it outputs a cat. This would get a low rating since it didn’t output exactly what I wanted it to display. As I keep showing it images of dogs and eventually the knobs turn enough to where when I ask it for a dog, it knows how to show me a dog. If you do this on enough images, and given enough knobs, the image generator can output almost any image you describe.

This isn’t the only way to train AI models, and there’s quite a bit more going on under the hood. In fact, the reason AI is getting so good is because there are many very intelligent people at research institutions and companies working on making the AI models better at learning. But this simple explanation gives you a general idea of what is going on. If you’d like a visual explanation, take a look at this video clip by Stuff Made Here. In his video, Shane is talking about how to use AI to create natural looking handwriting, and gives a great visual on how the AI does this.

Text generators work the similarly, but the output is text instead of images.


After many, many hours of tinkering with and refining my text prompts, and some quick design work to put all the text and images together, I can finally say that I am happy with the outcome! A whopping 64 full color pages that include 26 AI generated images and text for all the letters, compiled into a small travel sized book. You can get your own copy on my Etsy shop!

Get a copy of “The Amazing Animal Alphabet Adventure” on my Etsy shop!