30 Ormcurd Orm Tool Gorm Package Introduction and Practice

30 Ormcurd Orm Tool Gorm Package Introduction and Practice

30 ORMCURD ORM Tool GORM Package Introduction and Practice #

Hello, I am Kong Lingfei.

When developing projects with Go, we inevitably have to deal with databases. Every programming language has excellent ORM options to choose from, and Go is no exception. Some popular ones in the Go community are gorm, xorm, and gorose. Currently, GORM has the most stars on GitHub and is also the most widely used ORM in Go projects.

The IAM project also utilizes GORM. In this lecture, I will provide a detailed explanation of the basics of GORM and how iam-apiserver uses GORM to perform CRUD operations on data.

Introduction to GORM Basics #

GORM is an ORM package for the Go language, known for its powerful functionality and easy-to-use interface. Major companies like Tencent, Huawei, and Alibaba use GORM to build enterprise-level applications. GORM has many features, and here are some of the core features commonly used in development:

Full functionality: GORM provides interfaces for various database operations using ORM, satisfying various needs for database calls in our development.
Support for hook methods: These hook methods can be applied to Create, Save, Update, Delete, and Find methods.
Developer-friendly and easy to use.
Support for Auto Migration.
Support for association queries.
Support for multiple relational databases, such as MySQL, Postgres, SQLite, SQLServer, etc.

GORM has two versions, V1 and V2. Following the principle of using the new and not the old, the IAM project uses the latest V2 version.

Learning GORM through Examples #

Next, let’s take a quick look at an example that uses GORM to learn about GORM. The example code is located in the marmotedu/gopractise-demo/gorm/main.go file. Since the code is quite long, you can use the following commands to clone it to your local machine:

$ mkdir -p $GOPATH/src/github.com/marmotedu
$ cd $GOPATH/src/github.com/marmotedu
$ git clone https://github.com/marmotedu/gopractise-demo
$ cd gopractise-demo/gorm/

Assuming we have a MySQL database, with the connection address and port being 127.0.0.1:3306, the username being iam, and the password being iam1234. After creating the main.go file, execute the following command to run it:

$ go run main.go -H 127.0.0.1:3306 -u iam -p iam1234 -d test
2020/10/17 15:15:50 totalcount: 1
2020/10/17 15:15:50 	code: D42, price: 100
2020/10/17 15:15:51 totalcount: 1
2020/10/17 15:15:51 	code: D42, price: 200
2020/10/17 15:15:51 totalcount: 0

In the development of enterprise-level Go projects, the GORM library is mainly used to perform the following database operations:

Connecting to and closing the database. When connecting to the database, you may need to set some parameters, such as maximum connection count, maximum idle connection count, maximum connection duration, etc.
Inserting table records. You can insert a single record or batch insert records.
Updating table records. You can update a single field or multiple fields.
Viewing table records. You can view a single record or a list of records that meet certain conditions.
Deleting table records. You can delete a single record or delete in batches. Deleting also supports permanent deletion and soft deletion.
In some small projects, the table structure auto migration feature of GORM may also be used.

GORM is powerful, and the example code above demonstrates a more common way of using it.

In the above code, first a GORM model (Models) is defined. The Models are standard Go structs used to represent a table structure in the database. We can add the TableName method to the Models to tell GORM which table the Models map to in the database. The definition of Models is as follows:

type Product struct {
    gorm.Model
    Code  string `gorm:"column:code"`
    Price uint   `gorm:"column:price"`
}

// TableName maps to mysql table name.
func (p *Product) TableName() string {
    return "product"
}

If no table name is specified, GORM uses the snake-cased and pluralized struct name as the table name. For example, if the struct name is DockerInstance, then the table name is dockerInstances.

In the subsequent code, Pflag is used to parse the command-line arguments, and the database address, username, password, and database name are specified as command-line arguments. Afterwards, these arguments are used to generate the configuration file required to establish a MySQL connection, and the gorm.Open function is called to establish a database connection:

var (
    host     = pflag.StringP("host", "H", "127.0.0.1:3306", "MySQL service host address")
    username = pflag.StringP("username", "u", "root", "Username for access to mysql service")
    password = pflag.StringP("password", "p", "root", "Password for access to mysql, should be used pair with password")
    database = pflag.StringP("database", "d", "test", "Database name to use")
    help     = pflag.BoolP("help", "h", false, "Print this help message")
)

func main() {
    // Parse command line flags
    pflag.CommandLine.SortFlags = false
    pflag.Usage = func() {
        pflag.PrintDefaults()
    }
    pflag.Parse()
    if *help {
        pflag.Usage()
        return
    }

    dsn := fmt.Sprintf(`%s:%s@tcp(%s)/%s?charset=utf8&parseTime=%t&loc=%s`,
        *username,
        *password,
        *host,
        *database,
        true,
        "Local")
    db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})

    if err != nil {
        panic("failed to connect database")
    }
}

After creating the database connection, the database instance db will be returned. You can then call methods from the db instance to perform CRUD operations on the database. The specific operations are as follows and can be divided into six steps:

The first step is to automatically migrate the table structure.

// 1. Auto migration for given models
db.AutoMigrate(&Product{})

I do not recommend automatically migrating table structures in production code. Modifying the structure of a production database is a high-risk operation. The addition or modification of fields in a production database requires rigorous evaluation before being implemented. This approach hides changes in the code, making it difficult for developers to know when changes to the table structures are made. If the component is started, it may automatically modify the production table structure, which could potentially cause major incidents in the production environment.

The AutoMigrate method in GORM only performs changes on newly added fields or indexes, which theoretically has no risk. In actual Go project development, many developers use the AutoMigrate method to automatically synchronize table structures. However, I prefer standardized and perceivable operations, so in my actual development, I manually modify the table structure. Of course, you can choose whichever method suits your needs.

The second step is to insert records into the table.

// 2. Insert the value into database
if err := db.Create(&Product{Code: "D42", Price: 100}).Error; err != nil {
    log.Fatalf("Create error: %v", err)
}
PrintProducts(db)

A record is created using the db.Create method. After inserting the record, the PrintProducts method is called to print all data records in the current table, to test whether the insertion was successful.

The third step is to retrieve records that match certain conditions.

// 3. Find first record that match given conditions
product := &Product{}
if err := db.Where("code= ?", "D42").First(&product).Error; err != nil {
    log.Fatalf("Get product error: %v", err)
}

The First method only returns the first record in the list that matches the specified conditions. You can use the First method to retrieve detailed information about a particular resource.

The fourth step is to update records in the table.

// 4. Update value in database, if the value doesn't have primary key, will insert it
product.Price = 200
if err := db.Save(product).Error; err != nil {
    log.Fatalf("Update product error: %v", err)
}
PrintProducts(db)

The Save method allows you to update any fields in the product variable that are inconsistent with the database. The specific steps are as follows: first, retrieve the detailed information of a certain resource. Then, use assignment statements such as product.Price = 200 to update some fields. Finally, call the Save method to update these fields in the database. You can consider these operations as a way to update the database.

The fifth step is to delete records from the table.

You can delete records from the table using the Delete method, as shown in the code below:

// 5. Delete value match given conditions
if err := db.Where("code = ?", "D42").Delete(&Product{}).Error; err != nil {
    log.Fatalf("Delete product error: %v", err)
}
PrintProducts(db)

Please note that because Product has a gorm.DeletedAt field, the above delete operation does not actually delete the record from the database table. Instead, it sets the deletedAt field of the product table in the database to the current time to simulate deletion.

The sixth step is to retrieve a list of records from the table.

products := make([]*Product, 0)
var count int64
d := db.Where("code like ?", "%D%").Offset(0).Limit(2).Order("id desc").Find(&products).Offset(-1).Limit(-1).Count(&count)
if d.Error != nil {
    log.Fatalf("List products error: %v", d.Error)
}

The PrintProducts function will print all the current records, and you can determine whether the database operations were successful based on the output.

Explanation of Common GORM Operations #

After reading the examples above, I believe you have a basic understanding of how to use GORM. Next, I will give you a detailed introduction to the database operations supported by GORM.

Model Definition #

GORM uses models to map a database table. By default, it uses ID as the primary key, the snake_case of the struct name as the table name, the snake_case of the field name as the column name, and CreatedAt, UpdatedAt, and DeletedAt fields to track creation, update, and delete times.

Using GORM’s default rules can reduce code complexity, but I prefer a more direct way of specifying field and table names. For example, consider the following model:

type Animal struct {
  AnimalID int64        // Column name `animal_id`
  Birthday time.Time    // Column name `birthday`
  Age      int64        // Column name `age`
}

The above model corresponds to a table named animals, with columns named animal_id, birthday, and age. We can rename the table and columns, and set AnimalID as the primary key, by using the following approach:

type Animal struct {
    AnimalID int64     `gorm:"column:animalID;primarykey"` // Set column name to `animalID`
    Birthday time.Time `gorm:"column:birthday"`            // Set column name to `birthday`
    Age      int64     `gorm:"column:age"`                 // Set column name to `age`
}

func (a *Animal) TableName() string {
    return "animal"
}

In the above code, the primary key is specified by the primaryKey tag, column names are specified using the column tag, and the table name is specified by adding a TableName method to the model.

A typical database table usually contains 4 fields:

ID: Auto-increment field, also serves as the primary key.
CreatedAt: Records creation time.
UpdatedAt: Records update time.
DeletedAt: Records delete time (useful for soft deletes).

GORM also provides predefined models that include these 4 fields. When defining our own models, we can embed them within the struct. For example:

type Animal struct {
    gorm.Model
    AnimalID int64     `gorm:"column:animalID"` // Set column name to `animalID`
    Birthday time.Time `gorm:"column:birthday"` // Set column name to `birthday`
    Age      int64     `gorm:"column:age"`      // Set column name to `age`
}

Fields within the models can support many GORM tags, but if we don’t use the automatic table creation and migration features of GORM, many tags are unnecessary. In development, the most commonly used tag is column.

Connecting to the Database #

Before performing CRUD operations on the database, we first need to establish a connection. You can connect to a MySQL database using the following code:

import (
  "gorm.io/driver/mysql"
  "gorm.io/gorm"
)

func main() {
  // Refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name for details
  dsn := "user:pass@tcp(127.0.0.1:3306)/dbname?charset=utf8mb4&parseTime=True&loc=Local"
  db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
}

If you want GORM to handle time.Time types correctly, you need to include the parseTime parameter when connecting to the database. If you want to support full UTF-8 encoding, you can change charset=utf8 to charset=utf8mb4.

GORM supports connection pooling and maintains the connection pool using the database/sql package. The connection pool settings are as follows:

sqlDB, err := db.DB()
sqlDB.SetMaxIdleConns(100)              // Set the maximum number of idle connections for MySQL (recommended: 100)
sqlDB.SetMaxOpenConns(100)              // Set the maximum number of open connections for MySQL (recommended: 100)
sqlDB.SetConnMaxLifetime(time.Hour)     // Set the maximum idle connection lifetime for MySQL (recommended: 10s)

These settings can also be applied in large backend projects.

Creating Records #

We can use the db.Create method to create a record:

type User struct {
  gorm.Model
  Name         string
  Age          uint8
  Birthday     *time.Time
}
user := User{Name: "Jinzhu", Age: 18, Birthday: time.Now()}
result := db.Create(&user) // Create a record using the pointer to the data

The db.Create function will return the following 3 values:

user.ID: Returns the primary key value of the inserted data, which is assigned directly to the user variable.
result.Error: Returns an error.
result.RowsAffected: Returns the number of inserted records.

When dealing with large amounts of data to be inserted, batch insertion can be used to improve performance:

var users = []User{{Name: "jinzhu1"}, {Name: "jinzhu2"}, {Name: "jinzhu3"}}
DB.Create(&users)

for _, user := range users {
  user.ID // 1,2,3
}

Deleting Records #

We can use the Delete method to delete records:

// DELETE from users where id = 10 AND name = "jinzhu";
db.Where("name = ?", "jinzhu").Delete(&user)

GORM also supports deleting records based on the primary key, for example:

// DELETE FROM users WHERE id = 10;
db.Delete(&User{}, 10)

However, I prefer using the Where method for deletion because it has two advantages.

The first advantage is that the deletion method is more versatile. Using Where, you can not only delete records based on the primary key but also combine conditions for deletion.

The second advantage is that the deletion method is more explicit, which means it is easier to read. If you use db.Delete(&User{}, 10), you still need to confirm the primary key for the User model, and if you get the primary key wrong, you might introduce bugs.

Additionally, GORM supports batch deletion:

db.Where("name in (?)", []string{"jinzhu", "colin"}).Delete(&User{})

GORM supports two types of deletion: soft delete and permanent delete. Let me explain each of them separately.

Soft Delete

Soft delete means that when executing Delete, the record is not actually deleted from the database. GORM sets the DeletedAt field to the current time and the record cannot be queried in the normal way. If the model includes a gorm.DeletedAt field, GORM performs a soft delete when executing a delete operation.

The following is an example of a soft delete:

// UPDATE users SET deleted_at="2013-10-29 10:23" WHERE age = 20;
db.Where("age = ?", 20).Delete(&User{})

// SELECT * FROM users WHERE age = 20 AND deleted_at IS NULL;
db.Where("age = 20").Find(&user)

As you can see, GORM does not actually delete the record from the database but only updates the deleted_at field. When querying, GORM adds the condition AND deleted_at IS NULL, so records with the deleted_at field set will not be queried. Soft deletion can be used to delete important data in a way that allows for recovery and makes it easier to troubleshoot later on.

We can find soft-deleted records using the following method:

// SELECT * FROM users WHERE age = 20;
db.Unscoped().Where("age = 20").Find(&users)

Permanent Delete

If you want to permanently delete a record, you can use Unscoped:

// DELETE FROM orders WHERE id=10;
db.Unscoped().Delete(&order)

Alternatively, you can remove `gorm.DeletedAt` from the model.

### Update Record

The most commonly used update method in GORM is as follows:

```go
db.First(&user)

user.Name = "jinzhu 2"
user.Age = 100
// UPDATE users SET name='jinzhu 2', age=100, birthday='2016-01-01', updated_at = '2013-11-17 21:34:10' WHERE id=111;
db.Save(&user)

The above method retains all fields, so when executing Save, you need to execute First first to retrieve the values of all columns for a specific record, and then set the values for the fields that need to be updated.

You can also specify updating a single column:

// UPDATE users SET age=200, updated_at='2013-11-17 21:34:10' WHERE name='colin';
db.Model(&User{}).Where("name = ?", "colin").Update("age", 200)

You can also specify updating multiple columns:

// UPDATE users SET name='hello', age=18, updated_at = '2013-11-17 21:34:10' WHERE name = 'colin';
db.Model(&user).Where("name", "colin").Updates(User{Name: "hello", Age: 18, Active: false})

Note that this method only updates non-zero value fields.

Querying Data #

GORM supports various query methods. Below, I will explain three common query methods used in development: retrieving a single record, retrieving all records that meet certain criteria, and selectively retrieving fields.

Retrieve a Single Record

Here is an example code for retrieving a single record:

// Get the first record (ascending by primary key)
// SELECT * FROM users ORDER BY id LIMIT 1;
db.First(&user)

// Get the last record (descending by primary key)
// SELECT * FROM users ORDER BY id DESC LIMIT 1;
db.Last(&user)
result := db.First(&user)
result.RowsAffected // Returns the number of records found
result.Error        // Returns error

// Check for ErrRecordNotFound error
errors.Is(result.Error, gorm.ErrRecordNotFound)

If the model type does not define a primary key, it will be sorted by the first field.

Retrieve All Records That Meet Criteria

Here is an example code for retrieving all records that meet certain criteria:

users := make([]*User, 0)

// SELECT * FROM users WHERE name <> 'jinzhu';
db.Where("name <> ?", "jinzhu").Find(&users)

Selectively Retrieve Fields

You can use the Select method to choose specific fields. We can define a smaller struct to receive the selected fields:

type APIUser struct {
  ID   uint
  Name string
}

// SELECT `id`, `name` FROM `users` LIMIT 10;
db.Model(&User{}).Limit(10).Find(&APIUser{})

In addition to the three commonly used basic query methods mentioned above, GORM also supports advanced queries. Let me introduce four of them.

Advanced Queries #

GORM supports many advanced query features. Here, I will primarily introduce 4 of them.

Specifying the Sort Order when Retrieving Records

Here is an example code for specifying the sort order when retrieving records:

// SELECT * FROM users ORDER BY age desc, name;
db.Order("age desc, name").Find(&users)

Limit & Offset

Offset specifies the starting point of the query, and Limit specifies the maximum number of records to return. When Offset and Limit values are both -1, it removes the Offset and Limit conditions. Also, the location of Limit and Offset affects the result.

// SELECT * FROM users OFFSET 5 LIMIT 10;
db.Limit(10).Offset(5).Find(&users)

Distinct

Distinct selects unique values from the database records.

db.Distinct("name", "age").Order("name, age desc").Find(&results)

Count

Count retrieves the number of matches.

var count int64
// SELECT count(1) FROM users WHERE name = 'jinzhu'; (count)
db.Model(&User{}).Where("name = ?", "jinzhu").Count(&count)

GORM also supports many other advanced query features, such as inline conditions, Not conditions, Or conditions, Group & Having, Joins, Group, FirstOrInit, FirstOrCreate, iteration, FindInBatches, and more. Since these advanced features are not used in the IAM project, I will not elaborate on them here. If you are interested, you can refer to the official GORM documentation.

Raw SQL #

GORM supports querying SQL and executing raw SQL queries. Here’s how you can perform raw SQL queries:

type Result struct {
  ID   int
  Name string
  Age  int
}

var result Result
db.Raw("SELECT id, name, age FROM users WHERE name = ?", 3).Scan(&result)

And here’s how you can execute raw SQL queries:

db.Exec("DROP TABLE users")
db.Exec("UPDATE orders SET shipped_at=? WHERE id IN ?", time.Now(), []int64{1,2,3})

GORM Hooks #

GORM supports hook functions, such as the following hook function that runs before creating a record:

func (u *User) BeforeCreate(tx *gorm.DB) (err error) {
  u.UUID = uuid.New()

  if u.Name == "admin" {
    return errors.New("invalid name")
  }

  return nil
}

The hooks supported by GORM are shown in the table below:

## CURD Operations in iam-apiserver

Next, I will introduce how iam-apiserver uses GORM to perform CURD operations on data.

**First,**
we need to configure various parameters for connecting to MySQL. iam-apiserver creates a variable of type [MySQLOptions](https://github.com/marmotedu/iam/blob/v1.0.4/internal/pkg/options/mysql_options.go#L17) with default values using the [NewMySQLOptions](https://github.com/marmotedu/iam/blob/v1.0.4/internal/pkg/options/mysql_options.go#L29) function. This variable is then passed to the [NewApp](https://github.com/marmotedu/iam/blob/v1.0.4/pkg/app/app.go#L157) function. In the App framework, the AddFlags method provided by MySQLOptions is called, which adds the command-line parameters provided by MySQLOptions to the Cobra command line.

**Then,**
in the [PrepareRun](https://github.com/marmotedu/iam/blob/v1.0.4/internal/apiserver/server.go#L81) function, the [GetMySQLFactoryOr](https://github.com/marmotedu/iam/blob/v1.0.4/internal/apiserver/store/mysql/mysql.go#L55) function is called to initialize and obtain an instance of the repository layer called [mysqlFactory](https://github.com/marmotedu/iam/blob/v1.0.4/internal/apiserver/store/mysql/mysql.go#L50). mysqlFactory implements the [store.Factory](https://github.com/marmotedu/iam/blob/v1.0.4/internal/apiserver/store/store.go#L12) interface:

```go
type Factory interface {
    Users() UserStore
    Secrets() SecretStore
    Policies() PolicyStore
    Close() error
}

The GetMySQLFactoryOr function uses the singleton pattern mentioned in Lesson 11 to ensure that there is only one instance of the repository layer in the iam-apiserver process, which reduces memory consumption and system performance overhead.

In the GetMySQLFactoryOr function, the MySQL instance is created using the New function provided by the github.com/marmotedu/iam/pkg/db package. The code for the New function is as follows:

func New(opts *Options) (*gorm.DB, error) {    
    dsn := fmt.Sprintf(`%s:%s@tcp(%s)/%s?charset=utf8&parseTime=%t&loc=%s`,    
        opts.Username,                                            
        opts.Password,            
        opts.Host,                
        opts.Database,    
        true,                    
        "Local")    
                                
    db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{    
        Logger: logger.New(opts.LogLevel),                                            
    })    
    if err != nil {                
        return nil, err        
    }    
        
    sqlDB, err := db.DB()    
    if err != nil {            
        return nil, err    
    }         
             
    // SetMaxOpenConns sets the maximum number of open connections to the database.
    sqlDB.SetMaxOpenConns(opts.MaxOpenConnections)

    // SetConnMaxLifetime sets the maximum amount of time a connection may be reused.
    sqlDB.SetConnMaxLifetime(opts.MaxConnectionLifeTime)

    // SetMaxIdleConns sets the maximum number of connections in the idle connection pool.
    sqlDB.SetMaxIdleConns(opts.MaxIdleConnections)

    return db, nil
}

In the above code, we first create an instance of type *gorm.DB and then perform the following settings on that instance:

SetMaxOpenConns sets the maximum number of open connections to the MySQL database (recommended value: 100).
SetConnMaxLifetime sets the maximum idle connection lifetime to MySQL (recommended value: 10s).
SetMaxIdleConns sets the maximum number of idle connections in the MySQL connection pool (recommended value: 100).

Finally, using the mysqlFactory variable of the repository layer, we call the methods provided by its db field to perform database CURD operations. For example, creating a secret, updating a secret, deleting a secret, getting the details of a secret, and querying a list of secrets. The specific code is as follows (code located in the secret.go file):

// Create creates a new secret.
func (s *secrets) Create(ctx context.Context, secret *v1.Secret, opts metav1.CreateOptions) error {
    return s.db.Create(&secret).Error
}

// Update updates an secret information by the secret identifier.
func (s *secrets) Update(ctx context.Context, secret *v1.Secret, opts metav1.UpdateOptions) error {
    return s.db.Save(secret).Error
}

// Delete deletes the secret by the secret identifier.
func (s *secrets) Delete(ctx context.Context, username, name string, opts metav1.DeleteOptions) error {
    if opts.Unscoped {
        s.db = s.db.Unscoped()
    }

    err := s.db.Where("username = ? and name = ?", username, name).Delete(&v1.Secret{}).Error
    if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
        return errors.WithCode(code.ErrDatabase, err.Error())
    }

    return nil
}

}

// Get returns a secret based on the secret identifier.
func (s *secrets) Get(ctx context.Context, username, name string, opts metav1.GetOptions) (*v1.Secret, error) {
    secret := &v1.Secret{}
    err := s.db.Where("username = ? and name= ?", username, name).First(&secret).Error
    if err != nil {
        if errors.Is(err, gorm.ErrRecordNotFound) {
            return nil, errors.WithCode(code.ErrSecretNotFound, err.Error())
        }

        return nil, errors.WithCode(code.ErrDatabase, err.Error())
    }

    return secret, nil
}

// List returns all secrets.
func (s *secrets) List(ctx context.Context, username string, opts metav1.ListOptions) (*v1.SecretList, error) {
    ret := &v1.SecretList{}
    ol := gormutil.Unpointer(opts.Offset, opts.Limit)

    if username != "" {
        s.db = s.db.Where("username = ?", username)
    }

    selector, _ := fields.ParseSelector(opts.FieldSelector)
    name, _ := selector.RequiresExactMatch("name")

    d := s.db.Where(" name like ?", "%"+name+"%").
        Offset(ol.Offset).
        Limit(ol.Limit).
        Order("id desc").
        Find(&ret.Items).
        Offset(-1).
        Limit(-1).
        Count(&ret.TotalCount)

    return ret, d.Error
}

In the above code, s.db is a field of type *gorm.DB.

The code performs the following operations:

Updates various fields of the database table using s.db.Save.
Permanently deletes a row from the table using s.db.Unscoped. For resources that support soft deletion, we can control whether to permanently delete the record using the opts.Unscoped option. true performs a permanent delete, false performs a soft delete, and the default is a soft delete.
Determines whether the error returned by GORM is of type “record not found” using errors.Is(err, gorm.ErrRecordNotFound).
Gets the value of the query condition “name” using the following two lines of code:

selector, _ := fields.ParseSelector(opts.FieldSelector)
name, _ := selector.RequiresExactMatch("name")

Our entire call chain is: controller -> service -> repository. You may wonder how we call the instance mysqlFactory in the repository layer.

This is because our controller instance contains the service layer instance. When creating the controller instance, we pass in the service layer instance:

type UserController struct {
    srv srvv1.Service
}

// NewUserController creates a user handler.
func NewUserController(store store.Factory) *UserController {
    return &UserController{
        srv: srvv1.NewService(store),
    }
}

The service layer instance contains the repository layer instance. When creating the service layer instance, we pass in the repository layer instance:

type service struct {
    store store.Factory
}

// NewService returns Service interface.
func NewService(store store.Factory) Service {
    return &service{
        store: store,
    }
}

Through this containment relationship, we can call the service layer instance in the controller layer, and in the service layer, we can call the repository layer instance. This way, we can perform CRUD operations on the database using the db field (*gorm.DB type) of the repository layer instance.

Summary #

In Go projects, we need to use an ORM for CRU(D) operations on databases. In the Go ecosystem, the most popular ORM is GORM, which is also used by the IAM project. GORM has many features, including model definition, database connection, record creation, record deletion, record update, and data querying. The common usage of these features is as follows:

package main

import (
	"fmt"
	"log"

	"github.com/spf13/pflag"
	"gorm.io/driver/mysql"
	"gorm.io/gorm"
)

type Product struct {
	gorm.Model
	Code  string `gorm:"column:code"`
	Price uint   `gorm:"column:price"`
}

// TableName maps to mysql table name.
func (p *Product) TableName() string {
	return "product"
}

var (
	host     = pflag.StringP("host", "H", "127.0.0.1:3306", "MySQL service host address")
	username = pflag.StringP("username", "u", "root", "Username for access to MySQL service")
	password = pflag.StringP("password", "p", "root", "Password for access to MySQL, should be used pair with username")
	database = pflag.StringP("database", "d", "test", "Database name to use")
	help     = pflag.BoolP("help", "h", false, "Print this help message")
)

func main() {
	// Parse command line flags
	pflag.CommandLine.SortFlags = false
	pflag.Usage = func() {
		pflag.PrintDefaults()
	}
	pflag.Parse()
	if *help {
		pflag.Usage()
		return
	}

	dsn := fmt.Sprintf(`%s:%s@tcp(%s)/%s?charset=utf8&parseTime=%t&loc=%s`,
		*username,
		*password,
		*host,
		*database,
		true,
		"Local")
	db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
	if err != nil {
		panic("failed to connect database")
	}

	// 1. Auto migration for given models
	db.AutoMigrate(&Product{})

	// 2. Insert the value into the database
	if err := db.Create(&Product{Code: "D42", Price: 100}).Error; err != nil {
		log.Fatalf("Create error: %v", err)
	}
	PrintProducts(db)

	// 3. Find first record that matches given conditions
	product := &Product{}
	if err := db.Where("code = ?", "D42").First(&product).Error; err != nil {
		log.Fatalf("Get product error: %v", err)
	}

	// 4. Update value in the database, if the value doesn't have a primary key, it will be inserted
	product.Price = 200
	if err := db.Save(product).Error; err != nil {
		log.Fatalf("Update product error: %v", err)
	}
	PrintProducts(db)

	// 5. Delete value that matches given conditions
	if err := db.Where("code = ?", "D42").Delete(&Product{}).Error; err != nil {
		log.Fatalf("Delete product error: %v", err)
	}
	PrintProducts(db)
}

// List products
func PrintProducts(db *gorm.DB) {
	products := make([]*Product, 0)
	var count int64
	d := db.Where("code LIKE ?", "%D%").Offset(0).Limit(2).Order("id desc").Find(&products).Offset(-1).Limit(-1).Count(&count)
	if d.Error != nil {
		log.Fatalf("List products error: %v", d.Error)
	}

	log.Printf("totalcount: %d", count)
	for _, product := range products {
		log.Printf("\tcode: %s, price: %d\n", product.Code, product.Price)
	}
}

In addition, GORM also supports native SQL queries and executions to satisfy more complex SQL scenarios.

In GORM, there is a very useful feature called Hooks. Hooks can be called before executing a certain CRU(D) operation. In a hook, you can add some useful functionalities, such as generating a unique ID. Currently, GORM supports BeforeXXX, AfterXXX, and AfterFind hooks, where XXX can be Save, Create, Delete, or Update.

Finally, I have also introduced the practical use of GORM in the IAM project. The specific usage is similar to the example code in the summary. You can refer to the document for more details.

After-class Exercises #

GORM supports the AutoMigrate feature. Consider whether you can use the AutoMigrate feature in your production environment, and why?
Refer to the GORM official documentation to learn how to implement transaction rollback functionality using GORM.

Feel free to leave a message in the comment section to discuss and exchange ideas. See you in the next lesson.