Creating custom URL shortener with Nodejs

April 10, 2018 0 Comments

Creating custom URL shortener with Nodejs

 

 

Source: Unsplash
goog.ly

But by implementing above solutions, we will be facing another issue. A string value, goo.gl will be appended along with the URL shortener.

So every URL will have the domain name goo.gl. In order to avoid this issue, we can implement a new shortening service.

Note:

  • Google is planning to end the support for goog.gl service in 2018.Read here
  • These services providing some additional features like analytics, dashboard, no. of clicks etc.

Here, I am using Nodejs for building the shortener service.

Architecture

url-shortener architecture

Pre-requisites

Have the following pre-installed:

  • Yarn — Package manager
  • Node- Runtime environment for running the javascript applications outside the browser.
  • MongoDb- Non-relational database
  • Nodemon- Monitor any changes in your Nodejs application
  • Nginx — Event based web server.

Create a main repository

mkdir URL-Shortner && cd URL-Shortner

Lets start from Backend

Here’s a list of the technologies I’m using for the backend

  • Express.js -Node.js web application framework
  • MongoDB — Document-oriented NoSQL database
  • Mongoose- MongoDB object modeling tool
  • Short-id- Short id generator
  • Valid-url- URI validation functions
  • Nginx- Nginx is event-based and asynchronous web server.

Let us start by creating a directory called server

In your terminal type the following:

mkdir server && cd server
yarn init -y

This first command will create our directory and move into it, then we initialize a package.json accepting defaults.

{
"name": "server",
"version": "1.0.0",
"main": "index.js",
"license": "MIT"
}

Add dependencies express,mongoose,shortid and valid-url
In your terminal type the following:

yarn add express mongoose shortid valid-url

Setting up Express

In the root directory (server) we create the entry file index.js

touch index.js

At this point you can open your favourite editor (mine is VS Code), then point the editor to the root of this project .

Open URL-Shortener/server/index.js and paste the following:

// Require express module
const express = require("express");
const app = express();
const PORT = 7000;
//Start server on Port 7000
app.listen(PORT, () => {
console.log(Server started on port, PORT);
});

Add following lines in the package.json file to start the server

"scripts": {
"start": "nodemon index.js "
}

Where nodemon is used for automatically restarting the server when we make the changes in the code.

package.json

{
"name": "server",
"version": "1.0.0",
"main": "index.js",
"license": "MIT",
"scripts": {
"start": "nodemon index.js "
},

"dependencies":
{
"express": "^4.16.3",
"mongoose": "^5.0.12",
"shortid": "^2.2.8",
"valid-url": "^1.0.9"
}
}

Run the application using the command

yarn run start

You will get a log message

Server started on port 7000

Now the express server is running in the port 7000

I am assuming that MongoDB is already installed on your local machine. Check MongoDB is running or not by using the following command.

mongod

It will start your mongod server if MongoDB is properly installed on your machine.

Now we will to connect the express app with MongoDB by using Mongoose.

The following is the code for connecting the express app with MongoDB

const mongoURI = "mongodb://localhost/url-shortner";
const mongoose = require("mongoose");
const connectOptions = {
keepAlive: true,
reconnectTries: Number.MAXVALUE
};
mongoose.Promise = global.Promise;
mongoose.connect(mongoURI, connectOptions, (err, db) => {
if (err) console.log(Error, er);
console.log(Connected to MongoDB);
});

url-shortener is the database name.

Complete code

index.js

const express = require("express");
const bodyParser = require("body-parser");
const mongoose = require("mongoose");
const mongoURI = "mongodb://localhost/url-shortner";
const connectOptions = {
keepAlive: true,
reconnectTries: Number.MAX
VALUE
};
//Connect to MongoDB
mongoose.Promise = global.Promise;
mongoose.connect(mongoURI, connectOptions, (err, db) =>
{
if (err) console.log(Error, err);
console.log(Connected to MongoDB);
});

const app = express();
app.use(bodyParser.json());
const PORT = 7000;
app.listen(PORT, () => {
console.log(Server started on port, PORT);
});

Now our basic server with db connection is ready.

Mongoose allows us to have access to the MongoDB commands.

Models represent documents which can be saved and retrieved from our database.

Models are defined using the Schema interface. The Schema allows you to define the fields stored in each document along with their validation requirements and default values.

Here we need one model UrlShorten with following fields,

  • originalUrl- URL used to perform the shortening service.
  • shortUrl- Unique short code generated by using short-id
  • urlCode- The Unique code created in shortUrl.
  • createdAt- Created date
  • updatedAt -updated date

Then open a new tab in terminal and create a schema file UrlShorten.js under models folder

mkdir models
touch models/UrlShorten.js

below is the schema definition for UrlShorten

Initialise the schema in the index.js file

require('./models/UrlShorten')

Create RESTful APIs

Basically we need 2 apis

  • Creating the shorten url from original url
  • Render original url by passing shorten url

Create a file urlshorten.js inside routes folder

mkdir routes
touch routes/urlshorten.js

urlshorten.js

const mongoose = require("mongoose");
const UrlShorten = mongoose.model("UrlShorten");
module.exports = app => {
//GET API for redirecting to Original URL
app.get("/api/item/:code", async (req, res) => {

});
//POST API for creating short url from Original URL
app.post("/api/item", async (req, res) => {
  });
};

Complete code urlshorten.js

our main entry point is index.js.

We need to route this two APIs to corresponding files. Initialise the urlshorten.js in the index.js file

require("./routes/urlshorten")(app);

Finally we build two APIS

Test the shorten generation API by using postman

We got a short url http://localhost/By3rndysf for the url https://www.amazon.in/Apple-iPhone-X-Silver-256GB/dp/B071P37652/ref=sr11?s=electronics&ie=UTF8&qid=1522661136&sr=1-1&keywords=iphone

So successfully created the shorten service

Start front end

Note: I am not going into detail about front end and will be highlighting information required only for the shorten service

Go to main directory

cd URL-Shortner
mkdir client

Clone the project URL-Shortner.

copy all the contents from client folder and paste to newly createdclient folder.

Then open a new tab in terminal and install dependencies

cd client
yarn install
Complete project structure

Run the front end project

yarn run start

The project will start in the URL http://localhost:3000/. Then start to explore the URL shortener .

Start copying a link to the text box and click on submit button.You will get an error message some think like in the below image.

“response to the preflight request doesn’t pass access control check no ‘access-control-allow-origin’”
cors-error

This means that requests can only be made via AJAX to the same domain (or sub domain). Requests to an entirely different domain will fail.
There are several ways to fix this CORS issues.

  • Use a plugin for your browser
  • Enable CORS on server

Here I am using the second approach. Below is the code for enabling CORS on Express application


app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "");
res.header("Access-Control-Allow-Methods", "GET,PUT,POST,DELETE");
res.header(
"Access-Control-Allow-Headers",
"Content-type,Accept,x-access-token,X-Key"
);
if (req.method == "OPTIONS") {
res.status(200).end();
} else {
next();
}
});

Update index.js

Then click the submit button again, you will get a shortened URL in the form of

http://localhost/By3rndysf

So our shorten service ready.But when you click on that link.You will get the following error.

error

It means redirection is not working. The solution is map Short URL to Original URL. This is the core functionality of shortening service. We can use Nginx for that.

Configure Nginx

The actual magic is happening is on the Nginx server.First check Nginx is installed or not by using the command.

sudo systemctl status nginx

Open your nginx.confg file

sudo nano /etc/nginx/sites-enabled/default

Add a rewrite rules for the Original URL.

When we call http://localhost/[0–9a-z@]{5,15}$ [Shortened URL] then Nginx will call the API http://localhost:7000/api/item/$1.So express server render corresponding Original URL

Add bolded lines

server {
listen 80
index index.html
server_name _;
location ~
"^/[0-9a-z@]{5,15}$" {
rewrite ^/(.*)$
http://localhost:7000/api/item/$1 redirect;
}

}

Restart nginx server

sudo systemctl restart nginx

When you try to call

http://localhost/By3rndysf

The Nginx will take BJMVw_ksf code from the URL and try to call the API

http://localhost:7000/api/item/By3rndysf

So above API will return corresponding original URL .

The above API retruns a Web page

Note: When you try to host the web app on the server.The localhost will change to your IP address or domain name.

Demo


Tag cloud