Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Return embedded structs as slices when joining and selecting a unique key #2643

Unanswered
Discussion options

Based on the discussion in this Discord thread which references this SO question about sqlc.embed() and joins.

In the following example, it would be nice if sqlc returned a struct like (note the Users slice)

type FlatDetails struct {
 Flat Flat
 Users []User
}

for this query (note the :one in the annotation):

-- name: GetFlatWithUsers :one
SELECT 
 sqlc.embed(f),
 sqlc.embed(u)
FROM
 flats f
JOIN 
 user_flats uf ON f.id = uf.flat_id
JOIN 
 users u ON uf.username = u.username
WHERE 
 f.id = $1;

With an annotation of :many on the above query, sqlc would return a slice of the above FlatDetails struct.

Relevant schema:

CREATE TABLE users (
 username VARCHAR(255) PRIMARY KEY,
 email VARCHAR(255) UNIQUE NOT NULL
);
CREATE TABLE flats (
 id SERIAL PRIMARY KEY,
 name VARCHAR(255) NOT NULL
);
CREATE TABLE user_flats (
 id SERIAL PRIMARY KEY,
 username VARCHAR(255) NOT NULL REFERENCES users(username),
 flat_id INTEGER NOT NULL REFERENCES flats(id)
);

I think this should be possible given that flats.id is a unique key. But I'm not sure if this is desirable default behavior. What do other people think?

You must be logged in to vote

Replies: 6 comments 4 replies

Comment options

I think it's a very good Idea sometimes we want to push from a join query with one-to-many relations all at once and in a structured way, many tools do that like Prisma, TypeORM

You must be logged in to vote
0 replies
Comment options

one-shot loading of relations is one of the few things where sqlc currently is more awkward to use than a ORM. Arguably that's good because relation pre/auto loading also causes lots of (perf & design) headaches with ORMs.

What I can't tell from your proposal is

  1. would this also work for :many to load multiple FlatDetails?
  2. would sqlc be able to detect the cardinality of each embed? e.g. what happens if f.id is not unique.
You must be logged in to vote
2 replies
Comment options

andrewmbenton Sep 20, 2023
Collaborator Author

Good questions.

  1. would this also work for :many to load multiple FlatDetails?

I think there's no issue with that, assuming sqlc can infer the many to one relationship which is the whole premise of this idea in the first place. I've edited my original post to clarify.

  1. would sqlc be able to detect the cardinality of each embed? e.g. what happens if f.id is not unique.

sqlc can see the column uniqueness constraints, so my assumption was that it could use the information to infer the shape of the output struct. If f.id is not unique then I'd expect sqlc to just do what it does now.

Comment options

makes sense!

Comment options

I think the above proposal is fantastic if the query does the joins you can argue that people would expect the relationship to be exposed there.

Currently, if you do something like PrimaryTable.*, sqlc.embed(Secondary) sqlc would generate a struct with Primary properties and a single element of Secondary. Which secondary came through? who knows (especially if there's no order by).

I'd even say that the generated models should include the relationships by default (meaning if I fetch a :one Primary it should not generate a new struct on the queries generated code but just use the model for Primary) Following the example of your tables it should return an instance of.

type FlatDetails struct {
 Id. String
 Flat Flat
 Users []User
}
You must be logged in to vote
0 replies
Comment options

Would it be possible to do something like this?

-- name: GetFlatWithUsers :one
SELECT
 sqlc.embed(f),
 sqlc.group(
 sqlc.embed(u)
 )
FROM
 flats f
JOIN 
 user_flats uf ON f.id = uf.flat_id
JOIN 
 users u ON uf.username = u.username
WHERE 
 f.id = $1;

Internally, this could perform the same query as if the fields in .group(...) were written in the SELECT ... list, but then those fields will be grouped by the remaining using a hashcode (yes, I'm a Java dev, sorry!).

For example, this would be the generated code for :one:

package sql
import (
	"context"
)
const getFlatWithUsers = `-- name: GetFlatWithUsers :one
SELECT
 f.id, f.name,
 u.username, u.email
FROM
 flats f
JOIN
 user_flats uf ON f.id = uf.flat_id
JOIN
 users u ON uf.username = u.username
WHERE
 f.id = 1ドル
`
type GetFlatWithUsersGroup struct {
	Flat Flat
	Users []User
}
func (data GetAllFlatsWithUsersGroup) HashCode() (string, error) {
	// Imagine we have a proper hashcode generation algorithm
	return fmt.Sprintf("%v", data), nil
}
func (q *Queries) GetFlatWithUsers(ctx context.Context, id int32) (GetFlatWithUsersGroup, error) {
	group := GetFlatWithUsersGroup{}
	rows, err := q.db.Query(ctx, getFlatWithUsers, id)
	if err != nil {
		return group, err
	}
	defer rows.Close()
	hashCode := map[string]bool{}
	for rows.Next() {
		var i GetFlatWithUsersGroup
		var j User
		if err := rows.Scan(
			&i.Flat.ID,
			&i.Flat.Name,
			&j.Username,
			&j.Email,
		); err != nil {
			return group, err
		}
		h, err := i.HashCode()
		if err != nil {
			return group, err
		}
		if len(hashCode) == 0 {
			group = i
			hashCode[h] = true
		}
		if _, ok := hashCode[h]; ok {
			group.Users = append(group.Users, j)
		}
	}
	if err := rows.Err(); err != nil {
		return group, err
	}
	return group, nil
}

And this can be the generated code for :many:

package sql
import (
	"context"
)
const getAllFlatsWithUsers = `-- name: GetAllFlatsWithUsersGroup :many
SELECT
 f.id, f.name,
 u.username, u.email
FROM
 flats f
JOIN
 user_flats uf ON f.id = uf.flat_id
JOIN
 users u ON uf.username = u.username
`
type GetAllFlatsWithUsersGroup struct {
	Flat Flat
	Users []User
}
func (data GetAllFlatsWithUsersGroup) HashCode() (string, error) {
	// Imagine we have a proper hashcode generation algorithm
	return fmt.Sprintf("%v", data), nil
}
func (q *Queries) GetAllFlatsWithUsers(ctx context.Context) ([]GetAllFlatsWithUsersGroup, error) {
	rows, err := q.db.Query(ctx, getAllFlatsWithUsers)
	if err != nil {
		return nil, err
	}
	defer rows.Close()
	items := map[string]GetAllFlatsWithUsersGroup{}
	for rows.Next() {
		var i GetAllFlatsWithUsersGroup
		var j User
		if err := rows.Scan(
			&i.Flat.ID,
			&i.Flat.Name,
			&j.Username,
			&j.Email,
		); err != nil {
			return nil, err
		}
		hashCode, err := i.HashCode()
		if err != nil {
			return nil, err
		}
		if item, ok := items[hashCode]; ok {
			item.Users = append(item.Users, j)
		} else {
			items[hashCode] = i
			i.Users = []User{j}
		}
	}
	if err := rows.Err(); err != nil {
		return nil, err
	}
	groups := make([]GetAllFlatsWithUsersGroup, 0, len(items))
	for _, item := range items {
		groups = append(groups, item)
	}
	return groups, nil
}

There is one limitation: you can use only one sqlc.group() statement per query (in a similar fashion as a GROUP BY statement in sql).

You must be logged in to vote
0 replies
Comment options

@andrewmbenton Can this be escalated? It is a pretty useful and important functionality

You must be logged in to vote
2 replies
Comment options

+1

Comment options

+1

Comment options

Is there any workaround for the moment? Is this being pursued currently?

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

AltStyle によって変換されたページ (->オリジナル) /