-
Notifications
You must be signed in to change notification settings - Fork 501
GitLab Object Storage Configuration #1302
Open
Labels
Description
GitLab has object storage enabled with proxy mode disabled. Currently, when uploading large files via the client using Git LFS, the returned URL address is the GitLab service address instead of the object storage URL.
GitLab configuration is as follows:
gitlab_rails['lfs_enabled'] = true gitlab_rails['lfs_object_store_enabled'] = true gitlab_rails['lfs_object_store_proxy_download'] = false gitlab_rails['lfs_object_store_remote_directory'] = "gitlab-lfs" gitlab_rails['lfs_object_store_connection'] = { 'provider' => 'AWS', 'region' => 'huhehaote1', 'aws_access_key_id' => 'x', 'aws_secret_access_key' => 'x', 'endpoint' => 'https://eos-huhehaote-1.cmecloud.cn', 'force_path_style' => true } ### GitLab uploads gitlab_rails['uploads_object_store_enabled'] = true gitlab_rails['uploads_object_store_proxy_download'] = false gitlab_rails['uploads_object_store_remote_directory'] = "gitlab-uploads" gitlab_rails['uploads_object_store_connection'] = { 'provider' => 'AWS', 'region' => 'huhehaote1', 'aws_access_key_id' => 'x', 'aws_secret_access_key' => 'x', 'endpoint' => 'https://eos-huhehaote-1.cmecloud.cn', 'force_path_style' => true }
Configuration validation shows it is effective:
root@gitlab-bf8dfc847-dvq5p:/# gitlab-rails console -------------------------------------------------------------------------------- Ruby: ruby 3.1.4p223 (2023年03月30日 revision 957bb7cb81) [x86_64-linux] GitLab: 16.9.1 (0ef8ba69a8f) FOSS GitLab Shell: 14.33.0 PostgreSQL: 14.10 ------------------------------------------------------------[ booted in 17.43s ] Loading production environment (Rails 7.0.8) irb(main):001:1* lfs_connection = Aws::S3::Client.new( irb(main):002:1* access_key_id: 'xxx', irb(main):003:1* secret_access_key: 'xx', irb(main):004:1* region: 'huhehaote1', irb(main):005:1* endpoint: 'https://eos-huhehaote-1.cmecloud.cn', irb(main):006:0> force_path_style: true ) => #<Aws::S3::Client> irb(main):007:0> s3_resource = Aws::S3::Resource.new(client: lfs_connection) => #<Aws::S3::Resource:0x00007efdfb2b8b58 @client=#<Aws::S3::Client>> irb(main):008:0> bucket = s3_resource.bucket('gitlab-lfs') irb(main):009:0> puts bucket.exists? true => nil irb(main):010:0> # Upload file => nil irb(main):011:0> bucket.object('test.txt').put(body: 'Test content') irb(main):012:0* # Download file irb(main):013:0> obj = bucket.object('test.txt') irb(main):014:0> puts obj.get.body.read Test content => nil
Error logs are as follows:
16:08:07.672334 trace git-lfs: tq: enqueue retry #8 after 10.00s for "d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e" (size: 1914151) 16:08:17.672600 trace git-lfs: tq: sending batch of size 1 16:08:17.672753 trace git-lfs: api: batch 1 files 16:08:17.672787 trace git-lfs: HTTP: POST http://root:glpat-Lp8wFs6yMgWLs47CyGzb@192.168.10.11:30004/root/sss.git/info/lfs/objects/batch ... 16:08:17.719046 trace git-lfs: HTTP: {"objects":[{"oid":"d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e","size":1914151,"actions":{"upload":{"href":"http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151","header":{"Authorization":"Basic cm9vdDpleUowZVhBaU9pSktWMVFpTENKaGJHY2lPaUpJVXpJMU5pSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pY205dmRDSjlMQ0pxZEdraU9pSTVZamd3TURSbE55MDFaRGxtTFRRMFlUUXRPV1JoWXkwMk5ESXlaalJsWlRWaU9HRWlMQ0pwWVhRaU9qRTNOalF3TlRnd09UY3NJb ... 16:08:17.720041 trace git-lfs: HTTP: PUT http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151 ... 16:08:17.746184 trace git-lfs: tq: refusing to retry "d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e", too many retries (8) LFS: Put "http://gitlab-bf8dfc847-dvq5p/root/sss.git/gitlab-lfs/objects/d38637ef4e60ce027b574b4142cd639cce739ade469da65fa6d948f5216c841e/1914151": dial tcp: lookup gitlab-bf8dfc847-dvq5p on 114.114.114.114:53: no such host Error: failed to push some refs to 'http://192.168.10.11:30004/root/sss.git' [root@RK1 sss]#
What are the potential solutions to this issue?