Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add support for locale and encoding, fix #406 #416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mscherer wants to merge 2 commits into sclorg:master
base: master
Choose a base branch
Loading
from mscherer:add_locale_support

Conversation

@mscherer
Copy link
Contributor

@mscherer mscherer commented Jul 22, 2021
edited by github-actions bot
Loading

No description provided.

Copy link
Collaborator

Can one of the admins verify this patch?

2 similar comments
Copy link
Collaborator

Can one of the admins verify this patch?

Copy link
Collaborator

Can one of the admins verify this patch?

@mscherer mscherer marked this pull request as draft July 22, 2021 19:24
@mscherer mscherer changed the title (削除) Draft: Add support for locale and encoding, fix #406 (削除ここまで) (追記) Add support for locale and encoding, fix #406 (追記ここまで) Jul 22, 2021
Copy link
Contributor Author

Tagged as draft, since the tests are missing.

@mscherer mscherer marked this pull request as ready for review July 26, 2021 14:29
Copy link
Member

hhorak commented Sep 15, 2021

Thanks for this contribution. It makes sense to me to have such options.

Copy link
Member

hhorak commented Sep 15, 2021

[test]

Copy link
Member

hhorak commented Sep 15, 2021

What I'm thinking about is whether this shouldn't be set earlier, for the whole DB dir, as https://www.postgresql.org/docs/13/multibyte.html says: "The default character set is selected while initializing your PostgreSQL database cluster using initdb".

Copy link
Member

hhorak commented Sep 15, 2021

@fila43 what do you think?

Copy link
Contributor Author

oups, seems I pushed the wrong branch and removed my own code...

Copy link
Contributor Author

12:06:00 + run_locales_test
12:06:00 + local name=pg-test-locales
12:06:00 + DOCKER_ARGS='-e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C'
12:06:00 + create_container pg-test-locales
12:06:00 + local name=pg-test-locales
12:06:00 + shift
12:06:00 + local 'cargs=-e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C'
12:06:00 ++ echo '-e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C'
12:06:00 ++ tr '\n' ' '
12:06:00 + cargs='-e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C '
12:06:00 + cidfile=/tmp/tmp.oNB18ocbyupostgresql_test_cidfiles/pg-test-locales
12:06:00 + eval 'docker run -e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C --cidfile $cidfile -d $IMAGE_NAME "$@"'
12:06:00 ++ docker run -e POSTGRESQL_ENCODING=ISO_8859_6 -e POSTGRESQL_LOCALE=C --cidfile /tmp/tmp.oNB18ocbyupostgresql_test_cidfiles/pg-test-locales -d f31/postgresql:0
12:06:00 7cb14cec8112f6789df61a5cd5a1f1240f0077946da4bc497f421de838334773
12:06:01 ++ cat /tmp/tmp.oNB18ocbyupostgresql_test_cidfiles/pg-test-locales
12:06:01 Created container 7cb14cec8112f6789df61a5cd5a1f1240f0077946da4bc497f421de838334773
12:06:01 + echo 'Created container 7cb14cec8112f6789df61a5cd5a1f1240f0077946da4bc497f421de838334773'
12:06:01 + wait_ready pg-test-locales
12:06:01 ++ get_cid pg-test-locales
12:06:01 ++ local id=pg-test-locales
12:06:01 ++ shift
12:06:01 +++ cat /tmp/tmp.oNB18ocbyupostgresql_test_cidfiles/pg-test-locales
12:06:01 ++ echo 7cb14cec8112f6789df61a5cd5a1f1240f0077946da4bc497f421de838334773
12:06:01 + docker exec 7cb14cec8112f6789df61a5cd5a1f1240f0077946da4bc497f421de838334773 /usr/libexec/check-container
12:06:01 rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:1122: getting init process 7285 start time caused "read /proc/7285/stat: no such process"
12:06:01

I am not sure exactly what to do with that error message

@mscherer mscherer force-pushed the add_locale_support branch 2 times, most recently from 541e601 to a2cf268 Compare September 15, 2021 16:22
Copy link
Contributor Author

So using LANG, one can influence the default encoding and collate. However, this doesn't work that well.

I tried to create a database with LANG=C.UTF8, and it set the server encoding correctly, but not LC_COLLATE.

So I guess we need initdb + the createdb options.

Copy link
Contributor Author

And also, postgresql crash if I set LC_COLLATE to UTF8, so I suspect that using LANG and similar hacks is out of question, as this seems quite fragile and obscure.

Copy link
Contributor Author

initdb will derive the encoding from the locales (here ). And this code answer a hardcoded SQL_ASCII answer when the locale is C which mean that I can't use LANG only to get what I need for Synapse (encoding = UTF8, locale=C).

Copy link
Member

hhorak commented Sep 17, 2021

12:06:01 rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:1122: getting init process 7285 start time caused "read /proc/7285/stat: no such process"
I am not sure exactly what to do with that error message

I read this as the container's process ended before the exec was done, which might be what you refer to with "postgresql crash if I set LC_COLLATE".

Copy link
Contributor Author

So I pushed a fix, but as the CI is manually triggered, if someone could trigger for me, it would help (as I have no idea how to do it locally)

Copy link
Member

hhorak commented Sep 22, 2021

Thanks.
[test]

@mscherer mscherer force-pushed the add_locale_support branch 2 times, most recently from 87a666b to 690c02c Compare October 19, 2021 20:32
Copy link
Contributor Author

mscherer commented Oct 19, 2021
edited
Loading

So I fixed my tests (and my code), now it fail on:

ERROR: File /help.1 does not include 'POSTGRESQL\_ADMIN\_PASSWORD'.

It also fail on the same error for the master branch, so I guess that's not my code causing that.

Copy link
Contributor Author

The issue reported in my last comment is tracked on #421

Copy link
Member

phracek commented Sep 14, 2022

[test-all]

Copy link
Member

phracek commented Sep 14, 2022
edited
Loading

@mscherer Can you please rebased it against master and also run these two commands:

$ make clean-versions
$ make generate-all

And commit all changes?

Copy link
Contributor Author

Done

Copy link
Member

phracek commented Mar 15, 2023

[test-all]

Copy link
Member

phracek commented Mar 15, 2023

@fila43 @hhorak Please folks, go through this pull request and let me know if everything is fine, you are experts in PostgreSQL then /me.

Copy link
Member

@phracek phracek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I went through your code and did not hit any issue. Let's wait on the tests. Thanks for adding also test for it. Tests are more then welcome.

Copy link
Contributor Author

mscherer commented Oct 5, 2023

I rebased (before rediffing the other PR)

Copy link

Any progress on this would be highly appreciated.

@mscherer mscherer deleted the add_locale_support branch October 2, 2024 16:29
Copy link
Contributor Author

mscherer commented Oct 2, 2024

mhh seems I got it wrong, I erased the branch instead of updating it :(

Copy link

github-actions bot commented Nov 7, 2024
edited
Loading

Pull Request validation

Failed

🔴 Review - Missing review from a member (1 required)

Success

🟢 CI - All checks have passed

Signed-off-by: Michael Scherer <misc@redhat.com>
Signed-off-by: Michael Scherer <misc@redhat.com>
Copy link
Contributor Author

I rediffed the patch again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@phracek phracek phracek left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /