- 1. TypeScript Express tutorial #1. Middleware, routing, and controllers
- 2. TypeScript Express tutorial #2. MongoDB, models and environment variables
- 3. TypeScript Express tutorial #3. Error handling and validating incoming data
- 4. TypeScript Express tutorial #4. Registering users and authenticating with JWT
- 5. TypeScript Express tutorial #5. MongoDB relationships between documents
- 6. TypeScript Express tutorial #6. Basic data processing with MongoDB aggregation
- 7. TypeScript Express tutorial #7. Relational databases with Postgres and TypeORM
- 8. TypeScript Express tutorial #8. Types of relationships with Postgres and TypeORM
- 9. TypeScript Express tutorial #9. The basics of migrations using TypeORM and Postgres
- 10. TypeScript Express tutorial #10. Testing Express applications
- 11. TypeScript Express tutorial #11. Node.js Two-Factor Authentication
- 12. TypeScript Express tutorial #12. Creating a CI/CD pipeline with Travis and Heroku
- 13. TypeScript Express tutorial #13. Using Mongoose virtuals to populate documents
- 14. TypeScript Express tutorial #14. Code optimization with Mongoose Lean Queries
- 15. TypeScript Express tutorial #15. Using PUT vs PATCH in MongoDB with Mongoose
In this part of the Express TypeScript tutorial, we cover using a MongoDB database. To do that properly, we also learn the concept of environment variables. Since we follow the MVC pattern, we introduce the idea of a model. Just as in the previous part of the tutorial, all of the code from this part is in the express-typescript repository. Feel free to give it a star if you have not yet done so.
TypeScript Express tutorial #2: Express MongoDB
To use Express MongoDB in this article, we use a platform called mLab. It provides hosting for your database and has a free plan also, called the sandbox. After creating a new database, you can see its URI. In my case it is mongodb://<dbuser>:<dbpassword>@ds227594.mlab.com:27594/tutorial.
The URI includes the dbuser and dbpassword. These are not the credentials to your mLab account, but to the database itself. To create a user for your database, go to the tab named users.
In this Express MongoDB part of the tutorial, we use Mongoose. It is an Object Data Modeling (ODM) library that makes interacting with your MongoDB easier.
The question is how to provide credentials to our application. Even though for some the first thought might be to hardcode it into our code, it is not that good of an idea for various reasons. First of them is that you probably wouldn’t want to submit the credentials to your repository. The second reason is that you might wish to have different databases for each of your environment, for example on production and staging. The solution here is using environment variables.
Environment variables
The environment variables are values accessible in the environment in which the program runs. The operating system sets some of them up when it boots. The variables can be either global or just inside of one process. In Node.js, you have access to them through the process.env global object.
1 |
const MONGO_USER = process.env.MONGO_USER; |
The easiest way to set them is to do it in your NPM scripts:
1 2 3 |
"scripts": { "dev": "MONGO_USER=mwanago MONGO_PASSWORD=myPassword MONGO_PATH=@ds227594.mlab.com:27594/tutorial ts-node ./src/server.ts" } |
That does not seem like a good solution either. Another approach that you can take is to create a file named .env in which you store all your environment variables. The most popular way to handle a file like that is to use the dotenv package. First, create the .env file in the parent directory of your project.
.env
1 2 3 4 |
MONGO_USER=mwanago MONGO_PASSWORD=myPassword MONGO_PATH=@ds227594.mlab.com:27594/tutorial PORT=5000 |
As you can see, I’ve also created the PORT variable. It might be useful if you would want to change the port on which your application runs
At the beginning of your server.ts file (the one that is the entrypoint of your project) import dotenv/config. This import runs the config function for you and loads the .env file. The only thing left to establish the connection to our database is to run the mongoose.connection function.
src/server.ts
1 2 3 4 5 6 7 8 9 10 |
import * as mongoose from 'mongoose'; import 'dotenv/config'; const { MONGO_USER, MONGO_PASSWORD, MONGO_PATH, } = process.env; mongoose.connect(`mongodb://${MONGO_USER}:${MONGO_PASSWORD}${MONGO_PATH}`); |
It is a convention for environmental variables to be in uppercase.
Since you definitely shouldn’t commit your .env file, I recommend adding it to your .gitignore.
Validating environment variables
Forgetting to set one of your environment variables might cause your application to malfunction. To prevent it, we can verify them. To do that, I use the envalid package.
src/utils/validateEnv.ts
1 2 3 4 5 6 7 8 9 10 11 12 |
import { cleanEnv, str, } from 'envalid'; function validateEnv() { cleanEnv(process.env, { MONGO_PASSWORD: str(), MONGO_PATH: str(), MONGO_USER: str(), PORT: port(), }); } |
It throws an error if you forgot to provide one of the defined variables or if they are of a wrong type.
You can put the initialization of the database connection in the App class, and the validation of environment variables in the server.ts file.
src/server.ts
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import 'dotenv/config'; import App from './app'; import PostsController from './posts/posts.controller'; import validateEnv from './utils/validateEnv'; validateEnv(); const app = new App( [ new PostsController(), ], ); app.listen(); |
src/app.ts
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
import * as bodyParser from 'body-parser'; import * as express from 'express'; import * as mongoose from 'mongoose'; import Controller from './interfaces/controller.interface'; class App { public app: express.Application; constructor(controllers: Controller[]) { this.app = express(); this.connectToTheDatabase(); this.initializeMiddlewares(); this.initializeControllers(controllers); } public listen() { this.app.listen(process.env.PORT, () => { console.log(`App listening on the port ${process.env.PORT}`); }); } private initializeMiddlewares() { this.app.use(bodyParser.json()); } private initializeControllers(controllers: Controller[]) { controllers.forEach((controller) => { this.app.use('/', controller.router); }); } private connectToTheDatabase() { const { MONGO_USER, MONGO_PASSWORD, MONGO_PATH, } = process.env; mongoose.connect(`mongodb://${MONGO_USER}:${MONGO_PASSWORD}${MONGO_PATH}`); } } export default App; |
Express MongoDB with Mongoose
Once we have the connection established, let’s dig into the Express MongoDB and Mongoose more. MongoDB is a non-relational database that stores the data in JSON-like documents. Fields can vary from a document to document, and the data structure can change over time.
The MongoDB database holds documents in collections. They are analogous to tables in relational databases. If you want to save data into the database using Mongoose, the first thing to do is to create a schema. It defines the shape of the documents within a collection.
1 2 3 4 5 6 7 |
import * as mongoose from 'mongoose'; const postSchema = new mongoose.Schema({ author: String, content: String, title: String, }); |
Every key of our postSchema defines a property in our documents. There are multiple types that you can use. We explore them later, but if you want to check them out, look into the documentation.
The schema is just a definition of how should the document look like. To use it, you need to create a Model.
1 |
const PostModel = mongoose.model('Post', postSchema); |
The first argument is the name of the collection that your model is for. Please, keep in mind that it is supposed to be singular. Mongoose automatically converts it to a plural version, and therefore a collection named posts is created.
Models
In this tutorial, I decided to follow the Model-view-controller pattern. The Express MongoDB part of the course is a proper time to introduce the next part of the MVC pattern: the model. It couples well with the idea of the model from Mongoose. The model is responsible for managing the data of the application and represents its structure. Let’s create a distinct file for its definition.
src/posts/posts.model.ts
1 2 3 4 5 6 7 8 9 10 11 12 |
import * as mongoose from 'mongoose'; import Post from './post.interface'; const postSchema = new mongoose.Schema({ author: String, content: String, title: String, }); const postModel = mongoose.model<Post & mongoose.Document>('Post', postSchema); export default postModel; |
Thanks to using <Post & mongoose.Document>, TypeScript is now aware of all the fields you defined in the interface and knows that it can expect them to be available in the post model.
Our model is now ready to be used!
Saving a document
To save a document in the database, you first need to create it. To do it, we use the model that we prepared above. An instance of a model is called a document.
1 2 3 4 5 6 7 8 9 10 |
import postModel from './posts.model'; function createPost(request: express.Request, response: express.Response) { const postData: Post = request.body; const createdPost = new postModel(postData); createdPost.save() .then(savedPost => { response.send(savedPost); }) } |
The post that was created by using const createdPost = new postModel(postData) is not yet saved into the database, but it already has a property called _id . It is a unique id and is a combination of both a timestamp and a random string.
When you run createdPost.save()Â it is saved to the collection of posts. It returns a promise.
You can always look into the data in your database through the interface on mLab.
Retrieving all documents
There are quite a few ways to retrieve documents. The most basic one is with the usage of the find function.
1 2 3 4 5 6 7 8 |
import postModel from './posts.model'; function getAllPosts(request: express.Request, response: express.Response) { postModel.find() .then(posts => { response.send(posts); }) } |
The find method returns a Query object. You can call the then function on it which causes the query to be executed with a promise.
The ability to call the then method on the Query can be a bit misleading, but remember that postModel.find() does not return the promise itself. Please note that the find method does not cause the query to be executed, it happens after you call the then function. You can also do it by calling the exec function that returns a promise.
1 postModel.find().exec()
The find method can also accept an argument with conditions. For example, postModel.find({ author: 'Marcin' }) would return only my posts. The course will cover this aspect later.
HTTP GET http://localhost:5000/posts returns all the posts.
Retrieve a certain document
There might often be a situation in which you only want to get one document. A way to do that is to use its unique id. You can pass it using additional route parameters.
1 2 3 4 5 6 7 8 9 10 11 12 |
import postModel from './posts.model'; const router = express.Router(); this.router.get('posts/:id', getThePostById); function getPostById(request: express.Request, response: express.Response) { const id = request.params.id; postModel.findById(id) .then(post => { response.send(post); }) } |
The parameters are available to you in the request.params object. The function findById finds a single document based on the id provided. The findOne function is similar to findById. You can call postModel.findOne({ _id: id }) . It also accepts other conditions.
We respond with a newly created document by calling response.send(post).
HTTP GETÂ http://localhost:5000/posts/[id]Â returns the post with a given id, if found.
Replacing a document
Another common use of the API is when you intend to change an existing document. You can do so with the use of HTTP PATCH
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import postModel from './posts.model'; const router = express.Router(); this.router.patch('posts/:id', getThePostById); function modifyPost(request: express.Request, response: express.Response) { const id = request.params.id; const postData: Post = request.body; this.post.findByIdAndUpdate(id, postData, { new: true }) .then(post => { response.send(post); }) } |
The findByIdAndUpdate function updates a document with a given id using the provided data. If we pass the { new: true } into the options, in the callback we have the new, modified document instead of the old one. We then send it back with response.send(post).
Similar to findById, you can also use findOneAndUpdate instead of findByIdAndUpdate .
With HTTP PATCH you are supposed to only send the parameters you wish to change. If you send only some of the properties, the rest of them shouldn’t be modified. This is not the case with HTTP PUT that should replace the whole document with a new one. If you don’t provide the properties that you don’t wish to change, they should disappear.
HTTP PATCH http://localhost:5000/posts/[id]Â replaces properties of a post with a given id sent with the request body.
Removing a document
The way to remove an existing document is to use HTTP DELETE.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import postModel from './posts.model'; const router = express.Router(); this.router.delete('posts/:id', deleteAPost); function deletePost(request: express.Request, response: express.Response) { const id = request.params.id; this.post.findByIdAndDelete(id) .then(successResponse => { if(successResponse) { response.send(200); } else { response.send(404); } }) } |
To perform the action, we use findByIdAndDelete. If the removal succeeds, we send back status 200 meaning that the task completed. If it fails, in this simple example we assume that the post wasn’t found and send back the 404 status.
HTTP DELETE http://localhost:5000/posts/[id]Â deletes a post with a given id if found.
The controller
Summing up all of the model manipulations, the controller now looks like that:
src/posts/posts.controller.ts
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 |
import * as express from 'express'; import Controller from '../interfaces/controller.interface'; import Post from './post.interface'; import postModel from './posts.model'; class PostsController implements Controller { public path = '/posts'; public router = express.Router(); private post = postModel; constructor() { this.initializeRoutes(); } private initializeRoutes() { this.router.get(this.path, this.getAllPosts); this.router.get(`${this.path}/:id`, this.getPostById); this.router.patch(`${this.path}/:id`, this.modifyPost); this.router.delete(`${this.path}/:id`, this.deletePost); this.router.post(this.path, this.createPost); } private getAllPosts = (request: express.Request, response: express.Response) => { this.post.find() .then((posts) => { response.send(posts); }); } private getPostById = (request: express.Request, response: express.Response) => { const id = request.params.id; this.post.findById(id) .then((post) => { response.send(post); }); } private modifyPost = (request: express.Request, response: express.Response) => { const id = request.params.id; const postData: Post = request.body; this.post.findByIdAndUpdate(id, postData, { new: true }) .then((post) => { response.send(post); }); } private createPost = (request: express.Request, response: express.Response) => { const postData: Post = request.body; const createdPost = new this.post(postData); createdPost.save() .then((savedPost) => { response.send(savedPost); }); } private deletePost = (request: express.Request, response: express.Response) => { const id = request.params.id; this.post.findByIdAndDelete(id) .then((successResponse) => { if (successResponse) { response.send(200); } else { response.send(404); } }); } } export default PostsController; |
As you can see, I introduced the controller interface to make sure that all the controllers have certain properties.
src/interfaces/controller.interface.ts
1 2 3 4 5 6 7 8 |
import { Router } from 'express'; interface Controller { path: string; router: Router; } export default Controller; |
Summary
In this article, we’ve covered all the basics of using Express MongoDB. It also included using environment variables so that it can be done in a clean and maintainable manner. We’ve learned how to list, create, modify and delete documents. To do that we used HTTP methods: GET, POST, PATCH and DELETE. Tune in for upcoming parts of the tutorial, because we have important matters to cover such as the error handling.
Just a hint for those who hate callback hell:
This is not yet a callback hell, but using async/await is worth looking into anyway 🙂
to any one who are strunggling with mongodb connection error ‘failed to connect to server on first connect …’
if you are doing this with public wifi like cafe wifi, check if you add your public ip in mongodb.atlas – security – network access – ip whitelist
you can find out your public ip in website like https://www.whatismyip.com
but whiltelisting public ip might not be a good idea if your database is not for personal purpose
Could anybody help me with this error ?
Missing environment variables:
MONGO_PASSWORD: undefined
MONGO_PATH: undefined
MONGO_USER: undefined
PORT: undefined
Here is my code http://leteckaposta.cz/136673781
The envalid library was not ported to typescript (I test @types/envalid). How to maintain this feature without using envalid or how to make envalid compatible with typescript?
Excellent posts,
“When you run reatedPost.save()” -> “When you run createdPost.save()”,
There is a typo in article
In the “Validating environment variables” chapter you have accidentally repeat “if you”
And in the “Saving a document” chapter you lost letter ‘c’ in “createdPost.save()”
Thanks a lot, I fixed it.