Intro

AI is taking over the world. The technologies are rocking our world daily: ChatGPT and Stable Diffusion.
ChatGPT can answer any question in the world using its GPT3.5 model.
Stable Diffusion is a text-to-image model that can convert any text into a picture.
Combining both technologies is the future of AI.
I will show you how to create an entire website branding by combining the two in this article.

I am really excited about this one 🤩
Please click on the “Like” button so I know you like it and you want more.

Novu – the first open-source notification infrastructure

Just a quick background about us. Novu is the first open-source notification infrastructure. We basically help to manage all the product notifications. It can be In-App (the bell icon like you have in Facebook – Websockets), Emails, SMSs and so on.

I would be super happy if you could give us a star! And let me also know in the comments ❤️
https://github.com/novuhq/novu

What is Stable Diffusion?

Stable Diffusion is a text-to-image latent diffusion model that enables users to generate images within a few seconds based on a given text input. It is also used in processes such as in-painting, out-painting, and generating image-to-image translations.

ChatGPT is also an AI language model trained by OpenAI to generate text and interact with users in a human-like conversational manner. Users can submit requests and get information or answers to questions from a wide range of topics such as history, science, mathematics, and current events in just a few seconds.

At the end of this article, you’ll learn how to create images from text using the Stable Diffusion WebUI and send messages to ChatGPT from a Node.js application.

Installing and Running the Stable Diffusion Web UI

You can install Stable Diffusion Web UI on Windows, Linux, and Apple Silicon. Here, I’ll guide you through the installation on Apple Silicon.

Prerequisites: Ensure you have Homebrew installed on your computer. It is a package that enables you to install various applications not provided by Apple.

• Open a new terminal window and run the command below to install the WebUI dependencies.

MAC:

1brew install cmake protobuf rust python@3.10 git wget

Debian-based:

1sudo apt install wget git python3 python3-venv

Red Hat-based

1sudo dnf install wget git python3

Arch-based:

1sudo pacman -S wget git python3

• Clone the web UI repository by running the below:

1git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

• We need to download the Stable Diffusion model (large file). Navigate to stable-diffusion-webui/models/Stable-diffusion

1cd stable-diffusion-webui/models/Stable-diffusion

Download the model – For our use case we will use Stable Diffusion 1.5 but feel free to download any other version you want.
To download use:

1wget https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt

rename the model from v1-5-pruned-emaonly.ckpt to model.ckpt
mv v1-5-pruned-emaonly.ckpt model.ckpt

• Navigate into the stable-diffusion-webui folder and run the web UI project to create a virtual environment and install its dependencies.

1cd stable-diffusion-webui
2./webui.sh

• Visit the local URL – http://127.0.0.1:7860 to interact with the Web UI via its user interface.

• To relaunch the web UI process later, navigate into the stable-diffusion-webui folder on your terminal and run the command ./webui.sh. If you want to use the Web UI API, add the flag api flag to the command ./webui.sh --api

Building the web application

In this section, I’ll guide you through creating the project environment for the web application. We’ll use React.js for the front end and Node.js for the backend server.

Create the project folder for the web application by running the code below:

1mkdir stable-diffusion-app
2cd stable-diffusion-app
3mkdir client server

Setting up the React application

Navigate into the client folder via your terminal and create a new React.js project.

1cd client
2npx create-react-app ./

Delete the redundant files, such as the logo and the test files from the React app, and update the App.js file to display “Hello World” as below.

1function App() {
2    return (
3        <div>
4            <p>Hello World!</p>
5        </div>
6    );
7}
8export default App;

Navigate into the src/index.css file and copy the code below. It contains all the CSS required for styling this project.

1@import url("https://fonts.googleapis.com/css2?family=Space+Grotesk:wght@300;400;500;600;700&display=swap");
2* {
3    box-sizing: border-box;
4    margin: 0;
5    padding: 0;
6    font-family: "Space Grotesk", sans-serif;
7}
8.app,
9.loading,
10.result__container > div {
11    display: flex;
12    align-items: center;
13    justify-content: center;
14}
15.app {
16    width: 100%;
17    margin: 50px auto;
18    flex-direction: column;
19}
20.app > h1 {
21    margin-bottom: 30px;
22    color: #2b3467;
23}
24form {
25    display: flex;
26    flex-direction: column;
27    width: 80%;
28}
29textarea {
30    padding: 20px;
31    border: 1px solid #ddd;
32    outline: none;
33    border-radius: 5px;
34    margin: 5px 0px;
35    box-shadow: 0 2px 8px 0 rgba(99, 99, 99, 0.2);
36}
37button {
38    margin-top: 15px;
39    display: inline-block;
40    width: 200px;
41    padding: 20px;
42    cursor: pointer;
43    font-weight: bold;
44    border: none;
45    border-radius: 5px;
46    outline: none;
47    font-size: 18px;
48    background-color: #f0dbdb;
49}
50.loading {
51    width: 100%;
52    height: 100vh;
53    background-color: #fefcf3;
54}
55.result__container {
56    display: flex;
57    align-items: center;
58    flex-wrap: wrap;
59    margin-top: 30px;
60}
61.result__container > div {
62    margin: 5px;
63    flex-direction: column;
64}
65.image {
66    width: 400px;
67    height: 300px;
68    margin-bottom: 15px;
69}

Update the App.js file to display an input field that allows you to enter the proposed website description.

1import React, { useState } from "react";
2
3const App = () => {
4    const [description, setDescription] = useState("");
5
6    const handleSubmit = (e) => {
7        e.preventDefault();
8        console.log({ description });
9        setDescription("");
10    };
11
12    return (
13        <div className='app'>
14            <h1>Website Idea Generator</h1>
15            <form method='POST' onSubmit={handleSubmit}>
16                <label htmlFor='description'>Enter the description</label>
17                <textarea
18                    name='description'
19                    rows={6}
20                    value={description}
21                    onChange={(e) => setDescription(e.target.value)}
22                />
23                <button>GENERATE</button>
24            </form>
25        </div>
26    );
27};
28
29export default App;

Setting up the Node.js server

Navigate into the server folder within your terminal and create a package.json file.

1cd server & npm init -y

Install Express, Nodemon, and the CORS library.

1npm install express cors nodemon

ExpressJS is a fast, minimalist framework that provides several features for building web applications in Node.js, CORS is a Node.js package that allows communication between different domains, and Nodemon is a Node.js tool that automatically restarts the server after detecting file changes.

Create an index.js file – the entry point to the web server.

1touch index.js

Set up a Node.js server using ExpressJS. The code snippet below returns a JSON object when you visit the http://localhost:4000/api in your browser.

1//👇🏻index.js
2const express = require("express");
3const cors = require("cors");
4const app = express();
5const PORT = 4000;
6
7app.use(express.urlencoded({ extended: true }));
8app.use(express.json());
9app.use(cors());
10
11app.get("/api", (req, res) => {
12    res.json({
13        message: "Hello world",
14    });
15});
16
17app.listen(PORT, () => {
18    console.log(`Server listening on ${PORT}`);
19});

Install Axios, the unofficial ChatGPT API library, and Puppeteer. The ChatGPT API uses Puppeteer as an optional peer dependency to automate bypassing the Cloudflare protections.

1npm install chatgpt puppeteer axios

To use the ChatGPT API within the server/index.js file, you need to configure the file to use both the require and import keywords for importing libraries.
Therefore, update the server/package.json file to contain the type keyword.

1{ "type": "module" }

Add the code snippet below at the top of the server/index.js file.

1import { createRequire } from "module";
2const require = createRequire(import.meta.url);
3//...other code statements

Once you have completed the last two steps, you can now use ChatGPT within the index.js file.

Configure Nodemon by adding the start command to the list of scripts in the package.json file. The code snippet below starts the server using Nodemon.

1//In server/package.json
2
3"scripts": {
4    "test": "echo \\"Error: no test specified\\" && exit 1",
5    "start": "nodemon index.js"
6  },

Congratulations! You can now start the server by using the command below.

1npm start

How to communicate with ChatGPT in a Node.js application

In this section, you’ll learn how to communicate with ChatGPT from the Node.js server via the unofficial ChatGPT library.

The library uses a fully automated browser-based solution that enables us to scrape our way in – meaning it performs a full browser automation that allows us to log in to the OpenAI website, solve the captcha, and send messages via the OpenAI cookies.

You’ve installed the library in the previous section; next, import the ChatGPT API library into the index.js file as done below. The ChatGPT API uses Puppeteer as an optional peer dependency to automate bypassing the Cloudflare protections.

1import { ChatGPTAPIBrowser } from "chatgpt";

We need ChatGPT to generate a domain name for any description specified by the user and to create a prompt for the Stable Diffusion API.

Therefore, create a route on the server that accepts the description from the React app.

1app.post("/api", async (req, res) => {
2    const { prompt } = req.body;
3    console.log(prompt);
4});

Create a function within the React app that sends the description to the /api endpoint on the server once the user submits the form

1async function sendDescription() {
2    try {
3        const request = await fetch("http://localhost:4000/api", {
4            method: "POST",
5            body: JSON.stringify({
6                prompt: description,
7            }),
8            headers: {
9                Accept: "application/json",
10                "Content-Type": "application/json",
11            },
12        });
13        const res = await request.json();
14        console.log(res);
15    } catch (err) {
16        console.error(err);
17    }
18}

Add a loading state within the React app that holds the request status and call the async function once the form is submitted.

1const [description, setDescription] = useState("");
2const [loading, setLoading] = useState(false);
3
4const handleSubmit = (e) => {
5    e.preventDefault();
6    //👇🏻 calls the async function
7    sendDescription();
8    setDescription("");
9    setLoading(true);
10};

Update the /api endpoint to send the description to ChatGPT and generate a domain name and prompt for Stable Diffusion.

1async function chatgptFunction(content) {
2    try {
3        const api = new ChatGPTAPIBrowser({
4            email: "YOUR_CHATGPT_EMAIL_ADDRESS",
5            password: "YOUR_CHATGPT_PASSWORD",
6        });
7        await api.initSession();
8
9        //👇🏻 sends the instruction for the domain name to ChatGPT
10        const getDomainName = await api.sendMessage(
11            `Can you generate a domain name for a website about: ${content}`
12        );
13        let domainName = getDomainName.response;
14
15        //👇🏻 sends the instruction for the prompt to ChatGPT
16        const generatePrompt = await api.sendMessage(
17            `I have a website for ${content}, and I want to generate a logo for it, can you generate a prompt for dall-e for me? make it long like 50 words, you don't need to tell me why you generated the prompt`
18        );
19        const diffusionPrompt = generatePrompt.response;
20
21        console.log({ domainName, diffusionPrompt });
22    } catch (err) {
23        console.error(err);
24    }
25}
26
27app.post("/api", async (req, res) => {
28    const { prompt } = req.body;
29    const result = await chatgptFunction(prompt);
30    res.json({ message: "Retrieved successfully!" });
31});

The code snippet above uses Puppeteer to launch the browser and enable you to sign in to your ChatGPT account automatically. After authentication, ChatGPT processes the requests and returns the response.

In the upcoming section, I’ll guide you on how to send the generated prompt to the Stable Diffusion API.

How to interact with the Stable Diffusion API

To interact with the Stable Diffusion API, relaunch the web UI process by running the command below:

1cd stable-diffusion-webui
2./webui.sh --api

You can view the available API endpoints at http://127.0.0.1:7860/docs We’ll make use of the /sdapi/v1/txt2img endpoint for the text-to-image conversion.

Make a POST request to the /sdapi/v1/txt2img endpoint with the generated prompt as the payload.

1async function chatgptFunction(content) {
2    try {
3        const api = new ChatGPTAPIBrowser({
4            email: "asaoludavid234@yahoo.com",
5            password: "davidasaolu",
6        });
7        await api.initSession();
8
9        const getDomainName = await api.sendMessage(
10            `Can you generate a domain name for a website about: ${content}`
11        );
12        let domainName = getDomainName.response;
13
14        const generatePrompt = await api.sendMessage(
15            `I have a website for ${content}, and I want to generate a logo for it, can you generate a prompt for dall-e for me? make it long like 50 words, you don't need to tell me why you generated the prompt`
16        );
17        const diffusionPrompt = generatePrompt.response;
18
19        //👇🏻 Makes a POST request via Axios with the prompt as the payload
20        const request = await axios.post("http://127.0.0.1:7860/sdapi/v1/txt2img", {
21            prompt: diffusionPrompt,
22        });
23        //👇🏻 returns the generated logo and the domain name
24        let logoImage = await request.data.images;
25        return { logoImage, domainName };
26    } catch (err) {
27        console.error(err);
28    }
29}

From the code snippet above, the /sdapi/v1/txt2img endpoint accepts a required parameter called prompt – the text description of the image to be generated.

Update the /api endpoint on the Node.js server to save the result and send it to the React.js application.

1//👇🏻 holds the results
2const database = [];
3
4app.post("/api", async (req, res) => {
5    const { prompt } = req.body;
6    const result = await chatgptFunction(prompt);
7    //👇🏻 saves the result to the database array
8    database.push(result);
9    //👇🏻 return the result as a response
10    res.json({ message: "Retrieved successfully!", result: database });
11});

Displaying the results with the React app

Update the sendDescription function to receive the response from the server.

1//👇🏻 React state that holds the result
2const [result, setResult] = useState([]);
3
4async function sendDescription() {
5    try {
6        const request = await fetch("http://localhost:4000/api", {
7            method: "POST",
8            body: JSON.stringify({
9                prompt: description,
10            }),
11            headers: {
12                Accept: "application/json",
13                "Content-Type": "application/json",
14            },
15        });
16        const res = await request.json();
17        if (res.message) {
18            //👇🏻 update the loading and result states
19            setLoading(false);
20            setResult(res.result);
21        }
22    } catch (err) {
23        console.error(err);
24    }
25}

Create a Loading component that is shown to the user whenever the request is pending.

1import React from "react";
2
3const Loading = () => {
4    return (
5        <div className='loading'>
6            <h1>Loading, please wait...</h1>
7        </div>
8    );
9};
10
11export default Loading;

Add the code snippet below to display the component when the request is pending.

1if (loading) return <Loading />;

Update the component to render the results generated as done below.

1return (
2    <div className='app'>
3        <h1>Website Idea Generator</h1>
4        <form method='POST' onSubmit={handleSubmit}>
5            <label htmlFor='description'>Enter the description</label>
6            <textarea
7                name='description'
8                rows={6}
9                value={description}
10                onChange={(e) => setDescription(e.target.value)}
11            />
12            <button>GENERATE</button>
13        </form>
14
15        <div className='result__container'>
16            {result.length > 0 &&
17                result.map((item, index) => (
18                    <div key={index}>
19                        <img
20                            src={`data:image/png;base64,${item.logoImage}`}
21                            alt={item.domainName}
22                            className='image'
23                        />
24                        <p>Domain: {item.domainName}</p>
25                    </div>
26                ))}
27        </div>
28    </div>
29);

The code snippet above displays the logo and domain name generated for various requests. Congratulations!🎉 You’ve completed the project for this tutorial.

Here is a sample of the results gotten for cocktail websites:

Conclusion

So far, you’ve learnt:

• what Stable Diffusion is,
• how to install and set up Stable Diffusion on your computer
• how to send messages to ChatGPT from a Node.js application, and
• how to generate images from text via the Stable Diffusion API.

This tutorial walks you through an example of an application you can build using Stable Diffusion and ChatGPT. These AI technologies can be used to create powerful applications useful in various fields.

The source code for this tutorial is available here:

https://github.com/novuhq/blog/tree/main/webapp-with-stable-diffusion-and-chatgpt

Thank you for reading!

Help me out!

If you feel like this article helped you, I would be super happy if you could give us a star! And let me also know in the comments ❤️

https://github.com/novuhq/novu