Why did I create this article?

For a long time, I tried to create a way to do onboarding for members to go through some web page and fill in their details. I searched for many open-source libraries that can do it and found nothing. So I have decided to implement it myself.

How are we going to do it?

For this article, I will use Puppeteer and ReactJS.
Puppeteer is a Node.js library that automates several browser actions such as form submission, crawling single-page applications, UI testing, and in particular, generating screenshot and PDF versions of web pages.

We will open a webpage with Puppeteer, send to the client (React) a screenshot of every frame and reflect actions to Puppeteer by clicking on the image. To begin with, let’s set up the project environment.

Novu – the first open-source notification infrastructure

Just a quick background about us. Novu is the first open-source notification infrastructure. We basically help to manage all the product notifications. It can be In-App (the bell icon like you have in the Dev Community – Websockets), Emails, SMSs and so on.

I would be super happy if you could give us a star! It will help me to make more articles every week 🚀
https://github.com/novuhq/novu

How to create a real-time connection with Socket.io & React.js

Here, we’ll set up the project environment for the screen-sharing app. You’ll also learn how to add Socket.io to a React and Node.js application and connect both development servers for real-time communication via Socket.io.

Create the project folder containing two sub-folders named client and server.

1mkdir screen-sharing-app
2cd screen-sharing-app
3mkdir client server

Navigate into the client folder via your terminal and create a new React.js project.

1cd client
2npx create-react-app ./

Install Socket.io client API and React Router. React Router is a JavaScript library that enables us to navigate between pages in a React application.

1npm install socket.io-client react-router-dom

Delete the redundant files such as the logo and the test files from the React app, and update the App.js file to display Hello World as below.

1function App() {
2    return (
3        <div>
4            <p>Hello World!</p>
5        </div>
6    );
7}
8export default App;

Navigate into the server folder and create a package.json file.

1cd server & npm init -y

Install Express.js, CORS, Nodemon, and Socket.io Server API.

Express.js is a fast, minimalist framework that provides several features for building web applications in Node.js. CORS is a Node.js package that allows communication between different domains.

Nodemon is a Node.js tool that automatically restarts the server after detecting file changes, and Socket.io allows us to configure a real-time connection on the server.

1npm install express cors nodemon socket.io

Create an index.js file – the entry point to the web server.

1touch index.js

Set up a simple Node.js server using Express.js. The code snippet below returns a JSON object when you visit the http://localhost:4000/api in your browser.

1//index.js
2const express = require("express");
3const app = express();
4const PORT = 4000;
5
6app.use(express.urlencoded({ extended: true }));
7app.use(express.json());
8
9app.get("/api", (req, res) => {
10    res.json({
11        message: "Hello world",
12    });
13});
14
15app.listen(PORT, () => {
16    console.log(`Server listening on ${PORT}`);
17});

Import the HTTP and the CORS library to allow data transfer between the client and the server domains.

1const express = require("express");
2const app = express();
3const PORT = 4000;
4
5app.use(express.urlencoded({ extended: true }));
6app.use(express.json());
7
8//New imports
9const http = require("http").Server(app);
10const cors = require("cors");
11
12app.use(cors());
13
14app.get("/api", (req, res) => {
15    res.json({
16        message: "Hello world",
17    });
18});
19
20http.listen(PORT, () => {
21    console.log(`Server listening on ${PORT}`);
22});

Next, add Socket.io to the project to create a real-time connection. Before the app.get() block, copy the code below. Next, add Socket.io to the project to create a real-time connection. Before the app.get() block, copy the code below.

1//New imports
2.....
3const socketIO = require('socket.io')(http, {
4    cors: {
5        origin: "http://localhost:3000"
6    }
7});
8
9//Add this before the app.get() block
10socketIO.on('connection', (socket) => {
11    console.log(`⚡: ${socket.id} user just connected!`);
12    socket.on('disconnect', () => {
13      console.log('🔥: A user disconnected');
14    });
15});

From the code snippet above, the socket.io("connection") function establishes a connection with the React app, then creates a unique ID for each socket and logs the ID to the console whenever a user visits the web page.

When you refresh or close the web page, the socket fires the disconnect event showing that a user has disconnected from the socket.

Configure Nodemon by adding the start command to the list of scripts in the package.json file. The code snippet below starts the server using Nodemon.

1//In server/package.json
2
3"scripts": {
4    "test": "echo \"Error: no test specified\" && exit 1",
5    "start": "nodemon index.js"
6  },

You can now run the server with Nodemon by using the command below.

1npm start

Building the user interface

Here, we’ll create a simple user interface to demonstrate the interactive screen-sharing feature.

Navigate into client/src and create a components folder containing Home.js and a sub-component named Modal.js.

1cd client/src
2mkdir components
3touch Home.js Modal.js

Update the App.js file to render the newly created Home component.

1import React from "react";
2import { BrowserRouter, Route, Routes } from "react-router-dom";
3import Home from "./components/Home";
4
5const App = () => {
6    return (
7        <BrowserRouter>
8            <Routes>
9                <Route path='/' element={<Home />} />
10            </Routes>
11        </BrowserRouter>
12    );
13};
14
15export default App;

Navigate into the src/index.css file and copy the code below. It contains all the CSS required for styling this project.

1@import url("https://fonts.googleapis.com/css2?family=Space+Grotesk:wght@300;400;500;600;700&display=swap");
2
3body {
4    margin: 0;
5    padding: 0;
6    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "Roboto",
7        "Oxygen", "Ubuntu", "Cantarell", "Fira Sans", "Droid Sans",
8        "Helvetica Neue", sans-serif;
9    -webkit-font-smoothing: antialiased;
10    -moz-osx-font-smoothing: grayscale;
11}
12* {
13    font-family: "Space Grotesk", sans-serif;
14    box-sizing: border-box;
15}
16.home__container {
17    display: flex;
18    min-height: 55vh;
19    width: 100%;
20    flex-direction: column;
21    align-items: center;
22    justify-content: center;
23}
24.home__container h2 {
25    margin-bottom: 30px;
26}
27.createChannelBtn {
28    padding: 15px;
29    width: 200px;
30    cursor: pointer;
31    font-size: 16px;
32    background-color: #277bc0;
33    color: #fff;
34    border: none;
35    outline: none;
36    margin-right: 15px;
37    margin-top: 30px;
38}
39.createChannelBtn:hover {
40    background-color: #fff;
41    border: 1px solid #277bc0;
42    color: #277bc0;
43}
44.form {
45    width: 100%;
46    display: flex;
47    align-items: center;
48    justify-content: center;
49    flex-direction: column;
50    margin-bottom: 30px;
51}
52.form__input {
53    width: 70%;
54    padding: 10px 15px;
55    margin: 10px 0;
56}
57.popup {
58    width: 80%;
59    height: 500px;
60    background: black;
61    border-radius: 20px;
62    padding: 20px;
63    overflow: auto;
64}
65.popup-ref {
66    background: white;
67    width: 100%;
68    height: 100%;
69    position: relative;
70}
71.popup-ref img {
72    top: 0;
73    position: sticky;
74    width: 100%;
75}
76@media screen and (max-width: 768px) {
77    .login__form {
78        width: 100%;
79    }
80}

Copy the code below into the Home.js. It renders a form input for the URL, a submit button, and the Modal component.

1import React, { useCallback, useState } from "react";
2import Modal from "./Modal";
3
4const Home = () => {
5    const [url, setURL] = useState("");
6    const [show, setShow] = useState(false);
7    const handleCreateChannel = useCallback(() => {
8        setShow(true);
9    }, []);
10
11    return (
12        <div>
13            <div className='home__container'>
14                <h2>URL</h2>
15                <form className='form'>
16                    <label>Provide a URL</label>
17                    <input
18                        type='url'
19                        name='url'
20                        id='url'
21                        className='form__input'
22                        required
23                        value={url}
24                        onChange={(e) => setURL(e.target.value)}
25                    />
26                </form>
27                {show && <Modal url={url} />}
28                <button className='createChannelBtn' onClick={handleCreateChannel}>
29                    BROWSE
30                </button>
31            </div>
32        </div>
33    );
34};
35
36export default Home;

Add an image representing the screencast to the Modal.js file and import the Socket.io library.

1import { useState } from "react";
2import socketIO from "socket.io-client";
3const socket = socketIO.connect("http://localhost:4000");
4
5const Modal = ({ url }) => {
6    const [image, setImage] = useState("");
7    return (
8        <div className='popup'>
9            <div className='popup-ref'>{image && <img src={image} alt='' />}</div>
10        </div>
11    );
12};
13
14export default Modal;

Start the React.js server.

1npm start

Check the terminal where the server is running; the ID of the React.js client should appear on the terminal.

Congratulations 🥂 , We can now start communicating with the Socket.io server from the app UI.

Taking screenshots with Puppeteer and Chrome DevTools Protocol

In this section, you’ll learn how to take automatic screenshots of web pages using Puppeteer and the Chrome DevTools Protocol. Unlike the regular screenshot function provided by Puppeteer, Chrome’s API creates very fast screenshots that won’t slow down Puppeteer and your runtime because it is asynchronous.

Navigate into the server folder and install Puppeteer.

1cd server
2npm install puppeteer

Update the Modal.js file to send the URL for the web page provided by the user to the Node.js server.

1import { useState, useEffect } from "react";
2import socketIO from "socket.io-client";
3const socket = socketIO.connect("http://localhost:4000");
4
5const Modal = ({ url }) => {
6    const [image, setImage] = useState("");
7
8    useEffect(() => {
9        socket.emit("browse", {
10            url,
11        });
12    }, [url]);
13
14    return (
15        <div className='popup'>
16            <div className='popup-ref'>{image && <img src={image} alt='' />}</div>
17        </div>
18    );
19};
20
21export default Modal;

Create a listener for the browse event on the backend server.

1socketIO.on("connection", (socket) => {
2    console.log(`⚡: ${socket.id} user just connected!`);
3
4    socket.on("browse", async ({ url }) => {
5        console.log("Here is the URL >>>> ", url);
6    });
7
8    socket.on("disconnect", () => {
9        socket.disconnect();
10        console.log("🔥: A user disconnected");
11    });
12});

Since we’ve been able to collect the URL from the React app, let’s create screenshots using Puppeteer and Chrome DevTools Protocol.

Create a screen.shooter.js file and copy the code below:

1const { join } = require("path");
2
3const fs = require("fs").promises;
4const emptyFunction = async () => {};
5const defaultAfterWritingNewFile = async (filename) =>
6    console.log(`${filename} was written`);
7
8class PuppeteerMassScreenshots {
9    /*
10    page - represents the web page
11    socket - Socket.io
12    options - Chrome DevTools configurations
13    */
14    async init(page, socket, options = {}) {
15        const runOptions = {
16            //👇🏻 Their values must be asynchronous codes
17            beforeWritingImageFile: emptyFunction,
18            afterWritingImageFile: defaultAfterWritingNewFile,
19            beforeAck: emptyFunction,
20            afterAck: emptyFunction,
21            ...options,
22        };
23        this.socket = socket;
24        this.page = page;
25
26        //👇🏻 CDPSession instance is used to talk raw Chrome Devtools Protocol
27        this.client = await this.page.target().createCDPSession();
28        this.canScreenshot = true;
29
30        //👇🏻 The frameObject parameter contains the compressed image data 
31    //   requested by the Page.startScreencast.
32        this.client.on("Page.screencastFrame", async (frameObject) => {
33            if (this.canScreenshot) {
34                await runOptions.beforeWritingImageFile();
35                const filename = await this.writeImageFilename(frameObject.data);
36                await runOptions.afterWritingImageFile(filename);
37
38                try {
39                    await runOptions.beforeAck();
40                    /*👇🏻 acknowledges that a screencast frame  (image) has been received by the frontend.
41                    The sessionId - represents the frame number
42                    */
43                    await this.client.send("Page.screencastFrameAck", {
44                        sessionId: frameObject.sessionId,
45                    });
46                    await runOptions.afterAck();
47                } catch (e) {
48                    this.canScreenshot = false;
49                }
50            }
51        });
52    }
53
54    async writeImageFilename(data) {
55        const fullHeight = await this.page.evaluate(() => {
56            return Math.max(
57                document.body.scrollHeight,
58                document.documentElement.scrollHeight,
59                document.body.offsetHeight,
60                document.documentElement.offsetHeight,
61                document.body.clientHeight,
62                document.documentElement.clientHeight
63            );
64        });
65        //Sends an event containing the image and its full height
66        return this.socket.emit("image", { img: data, fullHeight });
67    }
68    /*
69    The startOptions specify the properties of the screencast
70    👉🏻 format - the file type (Allowed fomats: 'jpeg' or 'png')
71    👉🏻 quality - sets the image quality (default is 100)
72    👉🏻 everyNthFrame - specifies the number of frames to ignore before taking the next screenshots. (The more frames we ignore, the less screenshots we will have)
73    */
74    async start(options = {}) {
75        const startOptions = {
76            format: "jpeg",
77            quality: 10,
78            everyNthFrame: 1,
79            ...options,
80        };
81        try {
82            await this.client?.send("Page.startScreencast", startOptions);
83        } catch (err) {}
84    }
85
86    /* 
87    Learn more here 👇🏻: 
88    https://github.com/shaynet10/puppeteer-mass-screenshots/blob/main/index.js
89    */
90    async stop() {
91        try {
92            await this.client?.send("Page.stopScreencast");
93        } catch (err) {}
94    }
95}
96
97module.exports = PuppeteerMassScreenshots;

• From the code snippet above:
  • The runOptions object contains four values. beforeWritingImageFile and afterWritingImageFile must contain asynchronous functions that run before and after sending the images to the client.
  • beforeAck and afterAck represent the acknowledgment sent to the browser as asynchronous code showing that images were received.
  • The writeImageFilename function calculates the full height of the screencast and sends it together with the screencast image to the React app.

Create an instance of the PuppeteerMassScreenshots and update the server/index.js file to take the screenshots.

1//👇🏻 Add the following imports
2const puppeteer = require("puppeteer");
3const PuppeteerMassScreenshots = require("./screen.shooter");
4
5socketIO.on("connection", (socket) => {
6    console.log(`⚡: ${socket.id} user just connected!`);
7
8    socket.on("browse", async ({ url }) => {
9        const browser = await puppeteer.launch({
10            headless: true,
11        });
12        //👇🏻 creates an incognito browser context
13        const context = await browser.createIncognitoBrowserContext();
14        //👇🏻 creates a new page in a pristine context.
15        const page = await context.newPage();
16        await page.setViewport({
17            width: 1255,
18            height: 800,
19        });
20        //👇🏻 Fetches the web page
21        await page.goto(url);
22        //👇🏻 Instance of PuppeteerMassScreenshots takes the screenshots
23        const screenshots = new PuppeteerMassScreenshots();
24        await screenshots.init(page, socket);
25        await screenshots.start();
26    });
27
28    socket.on("disconnect", () => {
29        socket.disconnect();
30        console.log("🔥: A user disconnected");
31    });
32});

Update the Modal.js file to listen for the screencast images from the server.

1import { useState, useEffect } from "react";
2import socketIO from "socket.io-client";
3const socket = socketIO.connect("http://localhost:4000");
4
5const Modal = ({ url }) => {
6    const [image, setImage] = useState("");
7    const [fullHeight, setFullHeight] = useState("");
8
9    useEffect(() => {
10        socket.emit("browse", {
11            url,
12        });
13
14        /*
15        👇🏻 Listens for the images and full height 
16             from the PuppeteerMassScreenshots.
17           The image is also converted to a readable file.
18        */
19        socket.on("image", ({ img, fullHeight }) => {
20            setImage("data:image/jpeg;base64," + img);
21            setFullHeight(fullHeight);
22        });
23    }, [url]);
24
25    return (
26        <div className='popup'>
27            <div className='popup-ref' style={{ height: fullHeight }}>
28                {image && <img src={image} alt='' />}
29            </div>
30        </div>
31    );
32};
33
34export default Modal;

Congratulations!💃🏻 We’ve been able to display the screenshots in the React app. In the following section, I’ll guide you on making the screencast images interactive.

Making the screenshots interactive

Here, you’ll learn how to make the screencasts fully interactive such that it behaves like a browser window and responds to the mouse scroll and move events.

Reacting to the cursor’s click and move events.

Copy the code below into the Modal component.

1const mouseMove = useCallback((event) => {
2    const position = event.currentTarget.getBoundingClientRect();
3    const widthChange = 1255 / position.width;
4    const heightChange = 800 / position.height;
5
6    socket.emit("mouseMove", {
7        x: widthChange * (event.pageX - position.left),
8        y:
9            heightChange *
10            (event.pageY - position.top - document.documentElement.scrollTop),
11    });
12}, []);
13
14const mouseClick = useCallback((event) => {
15    const position = event.currentTarget.getBoundingClientRect();
16    const widthChange = 1255 / position.width;
17    const heightChange = 800 / position.height;
18    socket.emit("mouseClick", {
19        x: widthChange * (event.pageX - position.left),
20        y:
21            heightChange *
22            (event.pageY - position.top - document.documentElement.scrollTop),
23    });
24}, []);

• From the code snippet above:
  • [event.currentTarget.getBoundingClient()](https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect) returns an object containing information about the size and position of the screencasts relative to the viewport.
  • event.pageX – returns the position of the mouse pointer; relative to the left edge of the document.
  • Then, calculate the cursor’s position and send it to the backend via the mouseClick and mouseMove events.

Create a listener to both events on the backend.

1socket.on("browse", async ({ url }) => {
2    const browser = await puppeteer.launch({
3        headless: true,
4    });
5    const context = await browser.createIncognitoBrowserContext();
6    const page = await context.newPage();
7    await page.setViewport({
8        width: 1255,
9        height: 800,
10    });
11    await page.goto(url);
12    const screenshots = new PuppeteerMassScreenshots();
13    await screenshots.init(page, socket);
14    await screenshots.start();
15
16    socket.on("mouseMove", async ({ x, y }) => {
17        try {
18            //sets the cursor the position with Puppeteer
19            await page.mouse.move(x, y);
20            /*
21            👇🏻 This function runs within the page's context, 
22               calculates the element position from the view point 
23               and returns the CSS style for the element.
24            */
25            const cur = await page.evaluate(
26                (p) => {
27                    const elementFromPoint = document.elementFromPoint(p.x, p.y);
28                    return window
29                        .getComputedStyle(elementFromPoint, null)
30                        .getPropertyValue("cursor");
31                },
32                { x, y }
33            );
34
35            //👇🏻 sends the CSS styling to the frontend
36            socket.emit("cursor", cur);
37        } catch (err) {}
38    });
39
40    //👇🏻 Listens for the exact position the user clicked
41    //   and set the move to that position.
42    socket.on("mouseClick", async ({ x, y }) => {
43        try {
44            await page.mouse.click(x, y);
45        } catch (err) {}
46    });
47});

Listen to the cursor event and add the CSS styles to the screenshot container.

1import { useCallback, useEffect, useRef, useState } from "react";
2import socketIO from "socket.io-client";
3const socket = socketIO.connect("http://localhost:4000");
4
5const Modal = ({ url }) => {
6    const ref = useRef(null);
7    const [image, setImage] = useState("");
8    const [cursor, setCursor] = useState("");
9    const [fullHeight, setFullHeight] = useState("");
10
11    useEffect(() => {
12        //...other functions
13
14        //👇🏻 Listens to the cursor event
15        socket.on("cursor", (cur) => {
16            setCursor(cur);
17        });
18    }, [url]);
19
20    //...other event emitters
21
22    return (
23        <div className='popup'>
24            <div
25                ref={ref}
26                className='popup-ref'
27                style={{ cursor, height: fullHeight }} //👈🏼 cursor is added
28            >
29                {image && (
30                    <img
31                        src={image}
32                        onMouseMove={mouseMove}
33                        onClick={mouseClick}
34                        alt=''
35                    />
36                )}
37            </div>
38        </div>
39    );
40};
41
42export default Modal;

Responding to scroll events

Here, I’ll guide you through making the screencast scrollable to view all the web page’s content.

Create an onScroll function that measures the distance from the top of the viewport to the screencast container and sends it to the backend.

1const Modal = ({ url }) => {
2    //...other functions
3
4    const mouseScroll = useCallback((event) => {
5        const position = event.currentTarget.scrollTop;
6        socket.emit("scroll", {
7            position,
8        });
9    }, []);
10
11    return (
12        <div className='popup' onScroll={mouseScroll}>
13            <div
14                ref={ref}
15                className='popup-ref'
16                style={{ cursor, height: fullHeight }}
17            >
18                {image && (
19                    <img
20                        src={image}
21                        onMouseMove={mouseMove}
22                        onClick={mouseClick}
23                        alt=''
24                    />
25                )}
26            </div>
27        </div>
28    );
29};

Create a listener for the event to scroll the page according to the document’s coordinates.

1socket.on("browse", async ({ url }) => {
2    //....other functions
3
4    socket.on("scroll", ({ position }) => {
5        //scrolls the page
6        page.evaluate((top) => {
7            window.scrollTo({ top });
8        }, position);
9    });
10});

Congratulations!💃🏻 We can now scroll through the screencast and interact with the web page’s content.

Conclusion

So far, you’ve learned how to set up a real-time connection with React.js and Socket.io, take screenshots of webpages with Puppeteer and Chrome DevTools Protocol, and make them interactive.

This article is a demo of what you can build with Puppeteer. You can also generate PDFs of pages, automate form submission, UI testing, test chrome extensions, and many more. Feel free to explore the documentation.

The source code for this tutorial is available here: https://github.com/novuhq/blog/tree/main/screen-sharing-with-puppeteer.

P.S I would be super happy if you could give us a star! It will help me to make more articles every week 🚀
https://github.com/novuhq/novu

Thank you for reading!