One of the improvements was the introduction of OpenID Connect (OIDC), which allows developers to automate package publishing through github actions, without having to setup any kind of api key or access token, and today we are having a look at how you can implement it for your packages as well.

Step 1: setup a node package (or skip if you already have one)

As an example, we’ll create a fresh package which just logs hello world to console

NOTE: this package is only meant as a demo for this blog. It is not recommended to create such packages for yourself and you should just setup OIDC on your existing packages :)

mkdir npm-oidc-demo && cd npm-oidc-demo
npm init -y
touch index.js

export function helloWorld() {
    console.log("Hello World");
}

Update your package.json to look like this:

{
  "name": "npm-oidc-demo",
  "version": "1.0.1",
  "description": "A simple hello world package",
  "main": "index.js",
  "type": "module",
  "exports": {
    ".": "./index.js"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/ankushKun/npm-oidc-demo"
  },
  "keywords": [
    "npm",
    "oidc"
  ],
  "author": "",
  "license": "ISC"
}

an already published package is required to use OIDC (might change in the future, who knows…)

Run npm login and follow the steps to link the npm cli with your npmjs account, when successful, run npm whoami and it should print your username.

and we are ready for publishing, just run npm publish

NOTE: If you ignored the previous note and still followed this guide to publish the demo package yourself, you might need to change your package name as I have already used “npm-oidc-demo”

$ npm whoami
ankushkun

$ npm publish
npm notice
npm notice 📦  npm-oidc-demo@1.0.0
npm notice Tarball Contents
npm notice 428B README.md
npm notice 64B index.js
npm notice 239B package.json
npm notice Tarball Details
npm notice name: npm-oidc-demo
npm notice version: 1.0.0
npm notice filename: npm-oidc-demo-1.0.0.tgz
npm notice package size: 602 B
npm notice unpacked size: 731 B
npm notice shasum: 338ccdfe6ad57442bef858098fc247680fb23c9e
npm notice integrity: sha512-XAKUT4avwrfcV[...]lEjW6hNUMJieQ==
npm notice total files: 3
npm notice
npm notice Publishing to https://registry.npmjs.org/ with tag latest and default access
Authenticate your account at:
https://www.npmjs.com/auth/cli/5f8fa35b-c07a-4af8-9851-831f191efc3e
Press ENTER to open in the browser...

+ npm-oidc-demo@1.0.0

The package is now available at https://www.npmjs.com/package/npm-oidc-demo

Step 2: Update OIDC details on package settings

Head over to the settings tab on our freshly published package and select GitHub Actions as the trusted publisher

I have already created a github repo at ankushKun/npm-oidc-demo, now we just need to create a workflow file. Put this inside .github/workflows/publish.yml

name: Publish to npm

on:
  release:
    types: [published]
  workflow_dispatch:

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: "20.x"
          registry-url: "https://registry.npmjs.org"

      - name: Update npm
        run: npm install -g npm@latest

      - name: Publish to npm
        run: npm publish

Push the yml file to the repo and update the OIDC details on the package settings. I have kept the environment name empty because I don’t have any explicit environment setup for actions.

Now when you create a new release on the github repo, a workflow will start running which will publish the package to npm with provenance.

And that’s how one can setup cicd to publish npm packages from github actions without having to create any access tokens themselves!

An API — Application Programming Interface, defines how different applications can talk to each other and exchange data through structures requests and responses

API Types?
1. Web APIs — the one we will be focusing on today
2. OS APIs — Windows APIs, Android APIs, etc
3. Library APIs — Like a python library
4. Hardware APIs — allow accessing hardware functions (embedded libraries?)

RESTful Web APIs

A RESTful Web API is one that follows the “REST” (Representational State Transfer) principles, a software architecture style that ensures scalability, simplicity, and stateless communication between servers and clients.

Common REST HTTP Methods
There are 4 common methods/requests to call REST APIs
1. GET — to get some data from the server
2. POST — to send/create some data from the server and get a response
3. PUT — to update existing data on a server
4. DELETE — to remove some data form a server

When browsing the web, GET request is used to fetch the client side content that gets rendered by the browser

API response codes
Whenever an API is called, it will always return a status code (a number) to signify the result of the request. Categorised into 1XX (informational response), 2XX (success codes), 3XX (redirects), 4XX (client errors), 5XX (server errors)

200 — OK, successful request
301 — Moved Permanently
400 — Bad Request, invalid inputs
401 — Unauthorised
404 — Not Found
500 — Internal Server Error

Lets write an API endpoint ourselves

Make sure you have NodeJS + NPM already installed and follow these steps

mkdir api-test
cd api-test
npm init -y
npm i express
touch index.js

This will create a folder for our express API backend and create a JS file to write code in

Edit the index.js file and add this code

const express = require('express');
const app = express();

app.get('/', (req, res) => {
    res.send('Hello World');
});

app.listen(3000, () => {
    console.log('Server is running on port 3000');
});

We have just initialised an express backend which is serving the “Hello World” string at root (/) endpoint. i.e. on sending a GET request to /, Hello World will be returned

Run this file with node index.js and open http://localhost:3000/ on any browser. It should show Hello World

Try changing the string in res.send('this one') to a simple HTML like <h1>Hi there</h1> and restart the express server (ctrl+c and node index.js again), check if something changes on the served page

But this is just for GET requests, we cannot make POST requests to this endpoint

As you can see it gives an error if we try to make a POST request to the GET endpoint we created

Lets define a POST endpoint now

// add this after defining defining the app
app.use(express.json());

// POST endpoint
app.post('/', (req, res) => {
    const name = req.body.name
    res.send(`Hello ${name}!`);
});

restart the app and let’s try sending a post request to the server

You can use any HTTP client like Thunder Client on VSCode or Postman. I am going to use cURL coz simple

curl http://localhost:3000/ -d '{ "name": "Ankush" }'

this should print the response as Hello Ankush!

BOOM! You just created a GET and POST request! A majority of web APIs are comprised of just GET and POST requests

Time to write some frontend

<html>
  <body>
    <input type="text" id="name" name="name" placeholder="Enter your name">
    <button onclick="handleSubmit()">Submit</button>
    <script>
      async function handleSubmit() {
        const name = document.getElementById('name').value;
        const response = await fetch('/', {
          method: 'POST',
          headers: {'Content-Type': 'application/json'},
          body: JSON.stringify({ name })
        });
        const data = await response.text();
        document.body.innerHTML = data;
      }
    </script>
  </body>
</html>

We will serve this HTML through the GET endpoint we created earlier

app.get('/', (req, res) => {
    res.send(`
        <html>
            <body>
                <input type="text" id="name" name="name" placeholder="Enter your name">
                <button onclick="handleSubmit()">Submit</button>
                <script>
                    async function handleSubmit() {
                        const name = document.getElementById('name').value;
                        const response = await fetch('/', {
                            method: 'POST',
                            headers: {
                                'Content-Type': 'application/json'
                            },
                            body: JSON.stringify({ name })
                        });
                        const data = await response.text();
                        document.body.innerHTML = data;
                    }
                </script>
            </body>
        </html>
    `);
});

Restart the express app and refresh localhost:3000, and try entering your name

Now if you open developer tools and go into the network tab, you can see the POST request being sent and the different data associated with it

Congrats you just sent a POST request with some data, received some data back in response and showed it on the frontend!

Final Code:

const express = require('express');
const app = express();

app.use(express.json());

// To test this endpoint using curl, you can use the following command:
// curl http://localhost:3000/ 
app.get('/', (req, res) => {
    res.send(`
        <html>
            <body>
                <input type="text" id="name" name="name" placeholder="Enter your name">
                <button onclick="handleSubmit()">Submit</button>
                <script>
                    async function handleSubmit() {
                        const name = document.getElementById('name').value;
                        const response = await fetch('/', {
                            method: 'POST',
                            headers: {
                                'Content-Type': 'application/json'
                            },
                            body: JSON.stringify({ name })
                        });
                        const data = await response.text();
                        document.body.innerHTML = data;
                    }
                </script>
            </body>
        </html>
    `);
});

// To test this endpoint using curl, you can use the following command:
// curl http://localhost:3000/ -d '{ "name": "Ankush" }'
app.post('/', (req, res) => {
    const name = req.body.name;
    res.send(`Hello ${name}!`);
});

app.listen(3000, () => {
    console.log('Server is running on port http://localhost:3000');
});

What’s this about?

I had recently attended a workshop by Intel on their oneAPI programming model which was conducted by Abhishek Nandy at IIT Roorkee during the Cognizance 2023. I attended this workshop with my friend, Krish Agrawal who is also the co-author of this blog. The workshop was held for 2 days in which Abhishek Sir on the first day told about this new model by Intel and how to get started with its implementation with the help of JupyterLab. Before ending the day, we were told that there is a hackathon where he will take ideas by all the students in the form of teams as to how we use Intel oneAPI in real-life projects.

SYCL (/sɪkl/) is a programming model for high-performance computing that allows developers to write code for heterogeneous systems that use accelerators such as GPUs, FPGAs, and other specialised processing units. It is an open standard developed by the Khronos Group, an industry consortium that also develops other widely used graphics and compute APIs such as OpenGL and Vulkan. SYCL code can be executed on a variety of devices, including CPUs, GPUs, and FPGAs, without modification.

Fractals are fascinating geometric shapes that repeat themselves infinitely on different scales. They are created through a process of repeating a simple mathematical equation or algorithm to form increasingly intricate patterns. These fractal shapes are naturally occurring and can be observed in many aspects of nature, such as the patterns of a fern leaf, the coastlines of continents, and even the formation of clouds. One of the most well-known and captivating fractals is the Mandelbrot fractal. This type of fractal is generated through an iterative process using complex numbers, resulting in a shape that is renowned for its intricate detail and infinite complexity. It’s a true marvel of mathematics!

Now that we know what our project revolves around, let’s see its implementation in JupyterLab.

Getting started on DevCloud

DevCloud is sandbox that intel provides to learn and try out their oneAPI ecosystem. We will be using the devcloud jupyter lab to run our SYCL fractal generation project. It is way easier and quicker to run SYCL projects on the sandbox than on your local machine.

So head over to https://devcloud.intel.com/oneapi/ and create an account. Navigate to the ‘Getting Started’ tab, scroll down to ‘Connect with JupyterLab’ and Launch a JupyterLab instance. It might take some time or get loaded instantly depending on the server load so have some patience.

Once launched, you will find a bunch of folders in the file explorer, including a ‘mandelbrot’ folder.

Open the Terminal from the landing page, cd into the mandelbrot directory and run the build script.

cd mandelbrot
chmod +x build.sh
./build.sh

This will compile and run the project and create a ‘mandelbrot.png’ file inside the ‘build’ folder. As you can see the image is 1024x1024 pixels and it took just 190 milliseconds to generate the image (serial time + parallel time). Now if we were to do the same in python or regular C++ without SYCL, it would take seconds to generate the same result. So that’s the power of SYCL.

If you checkout the ‘src/mandel.hpp’ file, you can find 4 variables row and column size, max iterations and repetitions. We can modify these values to generate Mandelbrot sets of different resolutions and qualities.

Try changing these values and see how much time it takes for the images of different configurations to generate.

Automate it

Let’s now write a bash script that automatically changes the values in the C++ file and builds the project.

Create a file ‘autogen.sh’ and inside it write the following lines.

#!/bin/bash

PASSES=(10 100 500 1000 5000)
SIZES=(1024 2048 4096 8192 16384)

EDIT_FILE="src/mandel.hpp"
BUILD_FILE="build.sh"

prev_size=100
prev_pass=100
let count=0

The variable ‘PASSES’ contains a list of iterations that will be used in each image of the image generated and sizes contains the list of resolutions for the images in pixels. ‘prev_size’ and ‘prev_pass’ is a placeholder that we will use later. Count is the number of images generated.

for SIZE in ${SIZES[@]}; do
    for PASS in ${PASSES[@]}; do
        echo ""
        echo "Generating fractal for $SIZE x $SIZE with $PASS passes"
        echo ""

        ...

    done
done

We create a nested loop for the values of SIZES and PASSES, so we can have all the combinations of sizes and passes to generate the images with. In the body of the loop we print a message to display to the user.

In order to edit the values of the 3 variables in the .hpp file we will use the sed command.

sed -ie "s/row_size=$prev_size/row_size=$SIZE/g" $EDIT_FILE
sed -ie "s/col_size=$prev_size/col_size=$SIZE/g" $EDIT_FILE
sed -ie "s/max_iterations=$prev_pass/max_iterations=$PASS/g" $EDIT_FILE
sed -ie "s/repetitions=$prev_pass/repetitions=$PASS/g" $EDIT_FILE

prev_size=$SIZE
prev_pass=$PASS
count=$((count+1))

This uses find and replace to replace the previous values of the variables with new values and updates the count in each iteration of the bash loop.

To run the build command we simply do

./$BUILD_FILE

echo ""
echo "Generated for $SIZE x $SIZE with $PASS passes"
echo ""

The project is built and image is saved in the build folder, so we will have to copy it from there into a location of our choice. Create a folder images in the mandelbrot folder and add the following copy command to the autogen.sh script

cp "./build/mandelbrot.png" "./images/$count.mandelbrot_${SIZE}_${PASS}.png"

This copies the image from build into images folder and also renames it to contain useful information such as size and number of passes and the count.

Run the autogen script and let the fractals be created!

chmod +x autogen.sh
./autogen.sh

Here is the full autogen script

#!/bin/bash

PASSES=(10 100 500 1000 5000)
SIZES=(1024 2048 4096 8192 16384)

EDIT_FILE="src/mandel.hpp"
BUILD_FILE="build.sh"

prev_size=100
prev_pass=100
let count=0

for SIZE in ${SIZES[@]}; do
    for PASS in ${PASSES[@]}; do
        echo ""
        echo "Generating fractal for $SIZE x $SIZE with $PASS passes"
        echo ""
        sed -ie "s/row_size=$prev_size/row_size=$SIZE/g" $EDIT_FILE
        sed -ie "s/col_size=$prev_size/col_size=$SIZE/g" $EDIT_FILE
        sed -ie "s/max_iterations=$prev_pass/max_iterations=$PASS/g" $EDIT_FILE
        sed -ie "s/repetitions=$prev_pass/repetitions=$PASS/g" $EDIT_FILE

        prev_size=$SIZE
        prev_pass=$PASS
        count=$((count+1))

        ./$BUILD_FILE

        echo ""
        echo "Generated for $SIZE x $SIZE with $PASS passes"
        echo ""

        cp "./build/mandelbrot.png" "./images/$count.mandelbrot_${SIZE}_${PASS}.png"
        
    done
done

More iterations means better quality of the image, you can zoom in and see the small details and tiny fractal repetitions, but to see these clearly the resolution must be increased.

The winning moment

It was Day 2 and we both had absolutely nothing with us. We sat at the back thinking what we would present the crowd with. We had about 3 hours in our hands to build something. While brainstorming, we were randomly playing with the Mandelbrot directory already given by Intel in their Virtual Private Server. We ran the code with the proper command and was later told to rerun using qbus -I which then redirected usto another portal maybe through some kind of routing. We then decided to create a bash script to randomise four values in the original program i.e. row_size, column_size, max_iterations and repetitions. We were able to do this and now we had started generating images in a folder. Some images with higher resolution or more repetitions took more time than the rest. The longest time was taken by 16K resolution image with 1000 repetitions which took approximately 50 minutes (still wayy less than an average C++ or Python script generating the same iterations and resolution).

We presented our solution made on a presentation made on Keynote. We remarkably remember that we were the only team with the minimal interaction with Abhishek Sir and also the only ones to receive a round of applause from him. At the end of all prizes, 6 stood out of 13 and were congratulated on stage for the same. Then came the cash prizes for the Top 3. As soon as second was done, we thought the claps were consolatory of nature and we were already packing. But then we realised, we were indeed 1st. (Out of excitement, I slammed the top of Krish’s Macbook Pro 😂). In the end we got ₹12000 as a prize in the form of Amazon Vouchers.

If you scrolled till this point, you may as well checkout my social handles and leave a follow, Thanks for reading ;)