I took a formal Mongo course by Stephen Grider on Udemy. I am very happy with the way Stephen teaches. It is slow. Very relevant and practical. I feel accomplished and satisfied after the course. I prepared a few interview questions on Mongo and Mongoose which are quite useful.
- How do you create a model with Mongoose?
- What is the difference between an embedded document and an associated document?
- How do you write a middleware which removes an associated document as well?
- What are update operators? Suggest how $in can be used?
- How does Mongo store geo-location?
Taking the Udemy course has given me the vocabulary that is required to talk about Mongo in interviews. Apart from the interview questions, I believe building a Mongo database will help me in assimilating the knowledge further. This blog post is about building an express API over a Mongo database. And the Mongo database that we will build in this post is a database of birds!
Overview of Bird database and API
Our bird database has a collection of birds, users and comments. The bird collection has information about birds: name, family, photo URL, etc. The user collection is pretty straight-forward. It has an email and name. Users can leave a comment on the bird page. These comments are available in the comments collection. There are two APIs to access the bird database: admin API and public API. Admin API allows to create, edit and delete bird details. The public API displays the bird details to all users, both anonymous as well as registered users. Registered users have the option to perform two operations. They can mark a bird as seen. (The app is for bird lovers. And they like to mark birds as seen). And sometimes, registered users want to leave a comment on the bird page. In this blog post, we will learn how to build Mongoose models as well as the Express API to query and update the data. There is a git repository for the bird api.
Create the models
There are three models: Bird, User and Comment. We will create the relevant schemas and worry about connecting them up a bit later.
Bird model
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const BirdSchema = new Schema({
name: String,
family: String,
appearance: String,
distribution: String,
photoUrl: String,
});
const Bird = mongoose.model('bird', BirdSchema);
module.exports = Bird;
User model
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const UserSchema = new Schema({
name: String,
email: String,
password: String,
role: String
});
const User = mongoose.model('user', UserSchema);
module.exports = User;
Comment model
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const CommentSchema = new Schema({
content: String,
datePosted: Date
});
const Comment = mongoose.model('comment', CommentSchema);
module.exports = Comment;
Associating the models
A bird has a list of comments.
comments: [{
type: Schema.Types.ObjectId,
ref: 'comment'
}]
User also has a list of comments. In addition, every user has a seen property. The seen property holds an array of birds which he has seen.
seen: [{
type: Schema.Types.ObjectId,
ref: 'bird'
}],
Comment is associated with both bird and user.
user: {
type: Schema.Types.ObjectId,
ref: 'user'
},
bird: {
type: Schema.Types.ObjectId,
ref: 'bird'
}
Super! We are done with model design. Now, it is time to access them via API.
Scaffolding Express API
Express is easy to set. Connect to mongo database. Use the body parser middleware. Set the routes for each of the CRUD operations. It is customary to wire-up the API with a controller. In this case, we have a bird controller. The controller accepts a request, works with the model and sends out a response.
const express = require('express');
const mongoose = require('mongoose');
const bodyParser = require('body-parser');
const BirdController = require('./controllers/birdController');
const PORT = 3000;
mongoose.Promise = global.Promise;
mongoose.connect('mongodb://localhost/birddb', { useMongoClient: true });
const app = express();
app.use(bodyParser.json());
app.post('/api/birds', BirdController.create);
app.put('/api/birds/:id', BirdController.update);
app.delete('/api/birds/:id', BirdController.remove);
app.get('/api/birds', BirdController.index);
app.listen(PORT, () => {
console.log(`listening at ${PORT}`);
});
Bird controller
A controller is a class with static methods.
Create method
The create method handles POST requests. It uses the create method of the model.
static create(req, res) {
Bird.create(req.body)
.then(bird => {
res.send(bird);
});
}
Update method
The update method handles PUT requests. It updates a model by ID.
static update(req, res) {
const { id } = req.params;
Bird.findByIdAndUpdate(id, req.body)
.then(() => Bird.findById(id))
.then(bird => {
res.send(bird);
});
}
Remove method
The remove method handles DELETE requests. It removes a model by ID.
static remove(req, res) {
const { id } = req.params;
Bird.findByIdAndRemove(id)
.then(() => res.send({ id }));
}
Index method
The index method handles GET requests. Query string provides pagination parameters. All the birds fetched are sent as response.
static index(req, res) {
const { offset, limit } = req.query;
Promise.all([
Bird.find({})
.skip(offset)
.limit(limit),
Bird.count()
]).then(([birds, count]) => {
res.send({
birds,
count
});
});
Testing the API
Postman is a good tool to test our API. The top part has the request body. The bottom part shows the response received from the API.
Authentication
Authentication is implemented using User model. There are three main API: signup, login and user.
app.post('/api/signup', UserController.signup);
app.post('/api/login', loginMiddleware, UserController.login);
app.get('/api/user', tokenMiddleware, UserController.index);
User controller implements the API. However, we define two middleware as shown in the above routes.
Signup
Signup accepts an email and password and provides a JWT token. It also performs a few validations. First, we check if email and password is not empty. Then, we ensure that the email is not in use.
static signup(req, res) {
const { email, password } = req.body;
if (!email || !password) {
res.status(422).send({ error: 'Email and password is required.' });
}
User.findOne({ email })
.then(user => {
if (user) {
return Promise.reject('Email is in use');
} else {
const user = new User({ email, password });
return user.save();
}
})
.then((user) => {
const token = jwt.encode({ sub: user.id }, process.env.JWT_SECRET);
res.send({ token });
})
.catch(error => {
res.status(422).send({ error });
});
}
The above code looks like quite a lot. But it performs the validations, create a new user and finally generate a token (JWT or JSON Web Token). To generate a JWT, we take the help of a helper package, jwt-simple. Within the token, we encrypt a JSON object which has the user id. For encrypting, we make use of a secret. This secret should not be revealed to anyone. But stored as an environment variable in a .env file.
Login
Login flow makes use of a middleware. It is easy to write a middleware which verifies if the email and password are right. We make it a bit harder by using a helper package named passport which is popular for authentication in Node environment. Instead of writing our own middleware, we expose the middleware provided by Passport. However, Passport requires configuration. And that is what makes our job a bit harder. I will drop in the code for our loginMiddleware and explain it below.
const passport = require('passport');
const { Strategy } = require('passport-local');
const User = require('../models/user');
const strategy = new Strategy({
usernameField: 'email'
}, function(email, password, done) {
User.findOne({ email })
.then(user => {
if (user) {
user.comparePassword(password, function(err, match) {
if (match) {
done(null, user);
} else {
done(null, false);
}
});
} else {
done(null, false);
}
});
});
passport.use(strategy);
const loginMiddleware = passport.authenticate('local', { session: false });
module.exports = loginMiddleware;
Passport makes use of strategies for authentication. For loginMiddleware, we use a local strategy available in passport-local package. The last line in the above code, passport.authenticate returns a function which is a middleware used for authentication.
We write a verification function for local strategy. The verification function checks if the user is available in the database. And if available, verifies if the password matches with the stored password. If you observe the above code closely, you should see the comparePassword method on the user model. This is a helper function. We need the helper function because we store password hashes in the database.
Encrypting passwords
We do not store passwords as plain text in the database. Instead, we hash them. To create password hashes, we take the help of bcrypt-nodejs package.
To encrypt passwords, we write a middleware for our mongo model. This middleware should not be confused with express middleware we talked about earlier.
UserSchema.pre('save', function(next) {
const salt = bcrypt.genSaltSync();
this.password = bcrypt.hashSync(this.password, salt);
next();
});
In our save middleware, we use bcrypt to generate a password hash. We store the password hash in the password property of the model. So, every time, we save a new user object, we substitute the password with the password hash and store it.
Remember, the comparePassword function in our loginMiddleware. We add this method to the user model as follows.
UserSchema.methods.comparePassword = function(password, cb) {
const match = bcrypt.compareSync(password, this.password);
cb(null, match);
}
The comparePassword compares the password provided with the password hash stored in the database.
User Controller
User controller has three static methods: signup, login and index. We have already seen signup method. There is no middleware placed in front of the signup method. However, we have placed a loginMiddleware in front of the login method. We have seen how the loginMiddleware works. But, we have not covered the login method of the User controller. It is shown below.
static login(req, res) {
// passport attaches the user to the request!
const { user } = req;
const token = jwt.encode({ sub: user.id }, process.env.JWT_SECRET);
res.send({ token });
}
Passport middleware attaches the user object to the request. In the login method, we retrieve that user object. We get the user id, create a JWT and send it as response. If the verification step in loginMiddleware fails, we get an Unauthorized response with a 401 status code. The unauthorized response is sent by the Passport middleware itself.
User controller has an index method. The index method returns the details of the current user. It is trivial.
static index(req, res) {
const { name, email } = req.user;
res.send({ name, email });
}
We pass the name and email of the current user in the response. But how do we identify the current user? To identify the current user, the client should pass the JWT or token in the request header. Our server also implements a token middleware with the help of Passport. The token middleware validates the token.
JWT Validation
JWT is passed in the request header for every request. The server validates the token with the help of middleware exposed by Passport. Passport has a strategy for JWT in a package named passport-jwt. We configure the strategy by providing a few options: how to retrieve the token and how to get the JSON object within the token.
const options = {
jwtFromRequest: ExtractJwt.fromHeader('authorization'),
secretOrKey: process.env.JWT_SECRET
};
JWT strategy also has a callback method to verify the payload within the token. We retrieve the user by id. If we get a valid user, attached it to the express request object.
const strategy = new Strategy(options, function(payload, done) {
const userId = payload.sub;
User.findById(userId)
.then(user => {
if (user) {
done(null, user);
} else {
done(null, false);
}
});
});
Finally, our tokenMiddleware exports the passport middleware exposed by the authenticate method.
passport.use(strategy);
const tokenMiddleware = passport.authenticate('jwt', { session: false });
module.exports = tokenMiddleware;
Every authenticated route should have the tokenMiddleware run before the route handler. The middleware ensures that we get an unauthorized response message with 401 status code if the token is not valid.
Role based authorization
Earlier, we defined bird routes. But, we did not stick any middleware in front of it. Creating or Updating birds should be performed only by an user with admin role. The additional role validation is performed by a custom middleware that we write.
app.post('/api/birds', tokenMiddleware, adminMiddleware, BirdController.create);
app.put('/api/birds/:id', tokenMiddleware, adminMiddleware, BirdController.update);
app.delete('/api/birds/:id', tokenMiddleware, adminMiddleware, BirdController.remove);
app.get('/api/birds', BirdController.index);
Each bird request passes through tokenMiddleware. If the token is valid, the request passes through the adminMiddleware. This middleware verifies if the current user is an admin. Our signup does not have any role properties. So, where are we creating this admin user?
For our simple API, we just have only one admin user. We seed the admin user as part of the server startup.
mongoose.Promise = global.Promise;
mongoose.connect(process.env.MONGO_URL, { useMongoClient: true });
mongoose.connection.once('open', () => {
// seed admin user.
User.count()
.then(count => {
if (count === 0) {
const user = new User({
name: process.env.ADMIN_NAME,
email: process.env.ADMIN_EMAIL,
password: process.env.ADMIN_PASSWORD,
role: 'admin'
});
return user.save();
}
});
});
At server startup, we check if there are any users in the database. If there are no users, we create an admin user. This admin user is the one which will create, update and delete birds. And we enforce this restriction using our admin middleware. The admin middleware is just a few lines of code.
module.exports = function(req, res, next) {
const { role } = req.user;
if (role === 'admin') {
next();
} else {
res.status(401).send('Unauthorized');
}
};
We get the user model from the request object. If the user role is admin, we are all good and call next method. If the role is not an admin, we send an unauthorized response.
I hope the blog post covers the essential aspects of scaffolding an Express api based on Mongo database. Unfortunately, the post has outgrown the length of a regular blog post. There are a few more things that the bird api has. We won’t cover these aspects in the blog post. They are available in the github repo. Please feel free to ask me questions in the comments section if any of this is not clear.