API with NestJS #2. Setting up a PostgreSQL database with TypeORM

JavaScript NestJS TypeScript

This entry is part 2 of 121 in the API with NestJS

The next important thing when learning how to create an API is how to store the data. In this article, we look into how to do so with PostgreSQL and NestJS. To make the managing of a database more convenient, we use an Object-relational mapping (ORM) tool called TypeORM. To have an even better understanding, we also look into some SQL queries. By doing so, we can grasp what advantages ORM gives us.

You can find all of the below code in this repository.

Creating a PostgreSQL database

The most straightforward way of kickstarting our development with a Postgres database is to use docker.
Here, we use the same setup as in the TypesScript Express series.

The first thing to do is to install Docker and Docker Compose. Now, we need to create a docker-compose file and run it.

docker-compose.yml

The useful thing about the above configuration is that it also starts a pgAdmin console. It gives us the possibility to view the state of our database and interact with it.

To provide credentials used by our Docker containers, we need to create the docker.env file. You might want to skip committing it by adding it to your .gitignore.

docker.env

Once all of the above is set up, we need to start the containers:

Environment variables

A crucial thing to running our application is to set up environment variables. By using them to hold configuration data, we can make it easily configurable. Also, it is easier to keep sensitive data from being committed to a repository.

In the Express Typescript series, we use a library called dotenv to inject our variables. In NestJS, we have a   that we can use in our application. It uses dotenv under the hood.

app.module.ts

As soon as we create a .env file at the root of our application, NestJS injects them into a ConfigSerivice that we will use soon.

.env

Validating environment variables

It is an excellent idea to verify our environment variables before running the application. In the TypeScript Express series, we’ve used a library called envalid.

The   built into NestJS supports @hapi/joi that we can use to define a validation schema.

app.module.ts

Connecting a NestJS application with PostgreSQL

A first thing to do once we have our database running is to define a connection between our application and the database. To do so, we use  .

To keep our code clean, I suggest creating a database module.

database.module.ts

The synchronize flag above is very important. We will elaborate on it a lot later

An essential thing above is that we use the  and . The   method can access the environment variables thanks to providing the   and   arrays. We elaborate on these mechanisms in the upcoming parts of this series.

Now, we need to import our .

app.module.ts

Entities

The most crucial concept to grasp when using TypeORM is the entity. It is a class that maps to a database table. To create it, we use the   decorator.

post.entity.ts

A neat thing about TypeORM is that it integrates well with TypeScript because it is written in it. To define our columns, we can use various decorators.

@PrimaryGeneratedColumn()

primary key is a column used to identify a row uniquely in a table. Although we might use an existing column and make it primary, we usually create an id column. By choosing   from TypeORM, we create an integer primary column that has a value generated automatically.

@Column()

The   decorator marks a property as a column. When using it, we have two possible approaches.

The first approach is not to pass the column type explicitly. When we do it, TypeORM figures out the column using our TypeScript types. It is possible because NestJS uses reflect-metadata under the hood.

The second approach would be to pass the type of column explicitly, for example, by using  . The available column types differ between databases like MySQL and Postgres. You can look them up in the TypeORM documentation.

It is a proper moment to discuss different ways to store strings in Postgres. Relying on TypeORM to figure out the type of a string column results in the “character varying” type, also called varchar.

Varchar is very similar to a text type of a column but gives us a possibility to limit the length of a string. Both types are the same performance-wise.

SQL query

In pgAdmin, we can check a query equivalent to what TypeORM did for us under the hood.

There are a few interesting things to notice above

Using   results in having an int column. It defaults to the return value of a   function that returns unique ids. An alternative would be to use a serial type instead and would make the query shorter, but it works the same under the hood.

Our entity has varchar columns that use COLLATE. Collation is used to specify the sort order and character classification. To see our default collation, we can run this query:

en_US.utf8

The above value is defined in a query that was used to create our database. It is UTF8 and English by default.

Also, our   query puts a constraint on our ids so that they are always unique.

PK_be5fda3aac270b134ff9c21cdee is a name of the above constraint and was generated

Repositories

With repositories, we can manage a particular entity. A repository has multiple functions to interact with entities. To access it, we use the   again.

posts.module.ts

Now, in our  , we can inject the Posts repository.

Finding

With the  function, we can get multiple elements. If we don’t provide ith with any options, it returns all.

To get just one element we use the   function. By providing it with a number we indicate that we want an element with a particular id. If the result is undefined, it means that the element wasn’t found.

Creating

By using the  function, we can instantiate a new Post. We can use the  function afterward to populate the database with our new entity.

Modifying

To modify an existing element, we can use the  function. Afterward, we would use the   function to return the modified element.

A significant thing is that it accepts a partial entity, so it acts as a PATCH, not as a PUT. If you want to read more on PUT vs PATCH (although with MongoDB), check out TypeScript Express tutorial #15. Using PUT vs PATCH in MongoDB with Mongoose

Deleting

To delete an element with a given id, we can use the   function.

By checking out the documentation of the DELETE command, we can see that we have access to the count of removed elements. This data is available in the   property. If it equals zero, we can assume that the element does not exist.

Handling asynchronous errors

A beneficial thing about NestJS controllers is that they handle asynchronous errors very well.

If the   function throws an error, NestJS catches it automatically and parses it. When using pure Express, we would do this ourselves:

Summary

In this article, we’ve gone through the basics of connecting our NestJS application with a PostgreSQL database. Not only did we use TypeORM, but we’ve also looked into some SQL queries. NestJS and TypeORM have lots of features built-in and ready to use. In the upcoming parts of this series, we will look into them more, so stay tuned!

Series Navigation<< API with NestJS #1. Controllers, routing and the module structureAPI with NestJS #3. Authenticating users with bcrypt, Passport, JWT, and cookies >>
Subscribe
Notify of
guest
56 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Kuba
Kuba
4 years ago

I like the series but posts tend to be really chaotic and missing some steps/vital information. Like here – where does docker start pgadmin ?Or you define the post entity, and then check the table structure in pgadmin, but we’ve never actually run anything yet. And you can’t run the server at this step, since the rest of the app was not updated (post.service for instance).

sujana pathuri
sujana pathuri
3 years ago
Reply to  Marcin Wanago

Is it have inbuilt retry functionality to connect with the database in case of any error? if it has, may I know how it is working b/w node_modules @nestjs/typeorm, typeorm, pg?

Hossein
Hossein
4 years ago

Thanks for this great article. It helps much.
I have two questions.

1. There is any way that automates the creation of the DB?
It seems that we should write our query manually, and I think it’s not much interested that do the exact same thing when we want to deploy it on real servers or event Netlify.

2. How to have only one .env file in dockerized projects?
I’m working on a dockerized project that some variables are shared between them. I don’t want to have separated .env files for each project.

Thanks and best of luck.

Alex
Alex
4 years ago
Reply to  Hossein

2: There are env_file option for docker-compose. You can specify the same .env file for each service.

Hossein
Hossein
4 years ago
Reply to  Alex

Thanks.

Hossein
Hossein
4 years ago
Reply to  Marcin Wanago

Actually, it was a misunderstanding from me (:
Sorry for that.

Itunedy
Itunedy
3 years ago

Thank Wanago so much for this awesome series. I really love them and they help me a lot.

I have a question about your docker-compose please?

I see you have create services for “postgres”, “pgadmin”, that’s really good. But I don’t see that you create a service for “Nest server API”.

So if you run server normally with “yarn start:dev” for example, how is your server connect with docker container postgres ?

In my case, I need to create other service for server API in docker-compose (and run with Dockerfile). And I run them together with docker-compose up/build.

Thanks so much for your time.

Bhargav Gohil
Bhargav Gohil
2 years ago
Reply to  Itunedy

I wanted to ask, how you linked the postgres with web server in docker. and you how you given the docker env for postgres connection in images filesystem?

Dmitry
Dmitry
2 years ago
Reply to  Itunedy

ports:
– “5432:5432”

Means that the database will be able to handle connections from host machine (localhost:5432), so yes, node app on host machine will be able to connect to dockerized database. If you want to run the whole app in containers, just make sure you use the same network in docker-compose, and use container name as hostname for database (postgres:5432)

More info about docker networking available in docker documentation.

anh
anh
1 year ago
Reply to  Dmitry

i am using ubuntu, after running docker compose, how do i use PG admin ? where can i use PG Admin like local ?

Shivraj Nag
Shivraj Nag
1 year ago
Reply to  Marcin Wanago

Hi Marcin, what’s the credentials for the pgAdmin that is running on http://localhost:8080/ ? I tried using my local machine pgAdmin password and credentials from .env file of this project, but it throwing invalid credentials?

DpOriginals
DpOriginals
3 years ago

find this amazing blog in google. nice content, thank you!

Alex
Alex
3 years ago

Hello, cant run service and have a Error on this steps

” [ExceptionHandler] Nest can’t resolve dependencies of the PostsService (?). Please make sure that the argument PostRepository at index [0] is available in the AppModule context.”

it’s normally?

Danny P
2 years ago
Reply to  Alex

I expect this is no longer an issue for you being a year old comment but for the sake of others with similar issue:
Remember to add PostsService to the ‘exports’ array of your PostsModule. i.e.

Yiannis
3 years ago

Great series, just started applying the tutorials to a demo course, thanks Marcin! I only had a couple of issues in my local env. Had to change the following lines or I would get errors
database.module.ts

posts.service.ts constructor

Alex
Alex
3 years ago
Reply to  Yiannis

How you change that?

Yiannis
3 years ago
Reply to  Alex

Hey Alex, in the tutorial above there are the original lines that were posted. I commented with the changes I made.

Brahianpdev
2 years ago
Reply to  Yiannis

You save my! Thanks a lot <3

Aleksei
Aleksei
3 years ago

In pgAdmin, we can check a query equivalent to what TypeORM did for us under the hood.

Starting from this point – couldn’t find out where, when and how did the TypeORM worked for us. I mean – i suppose that it’s assumed that at that moment there should be a tables created but i can’t see none of them.

Have i missed something? Or the author meant that some actions should be taken by a reader himself?

Thanks.

Fengwei
Fengwei
3 years ago
Reply to  Marcin Wanago

Hi Wanago, When I login to pgAdmin, and expand the Service icon on the treeview to the left of the pgAdmin user interface, there is nothing there. Is there anything I have missed?

Mihai
Mihai
3 years ago
Reply to  Fengwei

@Fengwei did you manage to figure out what was wrong? I have the same issue…

Last edited 3 years ago by Mihai
Aidyn
Aidyn
2 years ago
Reply to  Mihai

Maybe I’m late to the party, but pretty sure that it will be helpful for other guys. So the issue is that you run pgadmin, but you should also create connection to server. I resolved this issue by adding this lines to docker-compose file,  
networks:
    – postgres
    environment:
      – POSTGRES_HOST_AUTH_METHOD=trust
    hostname: postgres,

docker-compose up , then go to localhost:8080, and you need to create the connection by clicking to register server, give the name of the server(whatever you want, for example nestjs), after that go to the connection tab, fill hostname/address with the hostname you provided to docker-compose file(postgres in our case),
so the form should look like this,
hostname/address: postgres
port:5432
maintenance database: postgres
username: admin,
password: admin,
role: ”,
service:”.
Click save, and you should see that connection is established and on left panel should appear server called nestjs

Navninder
2 years ago
Reply to  Aleksei

You will need to login to the PostgreSQL database using pgAdmin. TypeOrm will automatically create the tables if the synchornize option is set to true. I host my PostgreSQL on AWS RDS and I can view and query anything using pgAdmin. It is a great tool. It is not possible with Nest to view the raw tables. You may use querybuilder to get the raw entities but I have never used them.

Justin
Justin
3 years ago

Thanks for this great article. Got one question.
When I run npm ran start:dev, it typeorm would create table based on the entities, right?
I connected the pgadmin and didn’t see anything. Is it right?

Last edited 3 years ago by Justin
Justin
Justin
3 years ago
Reply to  Justin

I figured out there is a stupid mistake I made. Now, everything works. Thanks

Last edited 3 years ago by Justin
Achref
Achref
3 years ago
Reply to  Justin

hey Justin, I have the same issue. How did you resolve that please?

Abner Simões
3 years ago
Reply to  Justin

when connecting pgadmin, use the address “postgres” instead of “localhost”

Danny P
2 years ago
Reply to  Abner Simões

I set my HOST envvar to ‘host.docker.internal’ in order for TypeORM to successfully connect. Couldn’t get a connection through ‘localhost’ or ‘postgres’

Alessandro
Alessandro
3 years ago

Hi Marcin,
thanks a lot for this tutorial.
I have some problem when I run npm run start because I receive this error:
—————————————————————————————————

—————————————————————————————————  

I can say you that I’m able to connect to postgresql via CLI; here some log:
—————————————————————————————————

—————————————————————————————————

I have tryed also you branch part-2 but I receive the same error.
How can I fix it?

thanks in advance

Navninder
2 years ago
Reply to  Alessandro

You can fix it by allowing access to the database. You may have restricted it by using private domain wherever you are hosting your database. This happened to me as well when I used AWS RDS. I had to change the policies for allowing accessing to the databse with a public URL

Alessandro
Alessandro
3 years ago

Hi Marcin,
I discovered the problem: I had a previous local installation of Postgresql, so when typeOrm tryed to connect to Dockerzied Postgresql I had the error.
I uninstalled Postgresql on my local machine and all work.
Maybe at the beginning of this lesson, you could give an hint to remove any previous installation of Postgresql.
You can ignore my previous message.

Thank you.

Vladimir
Vladimir
3 years ago

Thanks a lot for this article? Do you have any experience with Sequelize ORM? I have some and It seems very inconvenient to use because of lack of query-builder and Typescript (written on Javascript, extra package required). Looking forward to use TypeORM. If you have Sequelize experience, could you please compare Sequelize and TypeORM?

Isaac
Isaac
3 years ago

Thanks for this article, but I am yet to see how one2one relationships are handled here.

Константин
Константин
3 years ago

Hi
How can we automatically generate and run migrations when application starts with the database module?
I try to do so but it didn’t work out (

Elves Brito
Elves Brito
2 years ago

I’m really enjoying it is a different approach than just following a video and playing it. Congratulations and I dare to suggest the creation of a course perhaps in this same footprint and perhaps with some deliberate mistakes to break your head a little. After all, that’s what we’re going to find on the market.

Oleh
Oleh
2 years ago

Why should we use synchronize: true flag for DatabaseModule? How can we use migrations?

Navninder
2 years ago
Reply to  Oleh

If set to false. You will not be able to create new columns in the entities. This is done to restrict users from modifying the entities when in production environment.

baked-potato
baked-potato
2 years ago

First of all, I’d like to say big Thank You to Marcin for producing such a cool tutorial!

Now I’ll share a few issues that I encountered with this particular post.

1) There is an issue with docker-compose config and macOS (Catalina or newer). It turns out that root path (“/”) is not writable so docker-compose is not able to mount “/data/postgres” or “/data/pgadmin” as a volume. So I replaced them with “/var/tmp/docker/<whatever-name>” and it worked fine. It looks like this:

2) Regarding the connection to database via pgAdmin, this comment was helpful.

3) There are typos in code examples. Please refer to this comment.

4) When you initially run your app and connect to database (via pgAdmin or somehow else), there is not going to be any “post” table. It will be created lazily when you push first data to it.

Finally if someone lacks instructions regarding database and pgAdmin setup, the workflow is as follows. First, run “docker-compose up” in the project directory and wait till everything is started. Then, build the app and run it. Now you should be good:)

dennis
dennis
2 years ago

I’m getting this problem
invalid length of startup packet.
what could be the solution

Omer
Omer
2 years ago

I really wished to follow your tutorials, since you cover a lot, but they are too messy.
You probably didn’t try to follow your own tutorial and see if stuff works / does it make any sense.

I’m wasting a lot of time on error fixing / looking up missing information and trying to make stuff work, and for now I’m just going to abandon it and find other tutorials that work and properly guide through this stuff.

Thanks for the effort, I’m sure you tried to put your heart into this – I hope to come back at another time and find it better.

Omer
Omer
2 years ago
Reply to  Marcin Wanago

Thanks, appreciated. I just joined

Navninder
2 years ago
Reply to  Omer

I am going to be creating a same tutorial just like Marcin’s. My tutorial will be focused on creating an email marketing application using the same technologies. I will try to improve as much as I can. When finished, I will post a comment here. You may give me a feedback?

CND
CND
2 years ago

good, but your example code makes me hard to understand. I need to visit your GitHub repository to understand it.

PostEntity is Post, you need to define the name more clear for class at post.entity.ts

Last edited 2 years ago by CND
phardi
phardi
2 years ago

Thanks for this great article. I have an issue about synchronize to database, cause the config typeorm ‘synchronize: true’ but the database not created after run the code with no error. I had try change to this code but still not synchronize
entities: [
__dirname + ‘/../**/*.entity{.ts,.js}’,
],

Ahmad Sharif
Ahmad Sharif
2 years ago

  const updatedPost = await this.postsRepository.findOne(id);

findOne(id)
no longer works like this the syntax probably updated or somthing else
now it is working like

findOne({
      where: { id },
    });

  thanks me later <3

kien
kien
1 year ago

Hello, everyone can help me this error
throw new Error(Config validation error: ${error.message});
Error: Config validation error: “POSTGRES_HOST” is required. 

Thank you very much!

kien
kien
1 year ago
Reply to  Marcin Wanago

This is my .env file:

John
John
1 year ago

Great

giserman001
giserman001
1 year ago

The tutorial was awesome and I learned a lot