Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

How to handle multiple commits with multithreading #1182

Answered by Byron
Be5yond asked this question in Q&A
Discussion options

Hi,
I am using GitPython to create a application that one of the feature is presenting good format git commit log.
something like this.

Num |Commit |Author |Date |Stats |Description

41 |725b5dc |wyy |2021年02月19日 15:30:16 |+7 -2 |[HEAD][️main][️origin/main][️origin/HEAD]⚙️ log命令返回增加
40 |837ce08 |wyy |2021年02月09日 15:23:33 |+43 -20 |⚙️ log 增加翻页功能
39 |8a72746 |wyy |2021年02月09日 15:17:34 |+3 -27 |📝 更新readme
38 |9d4dbf1 |wyy |2021年02月09日 11:25:02 |+38 -27 |[️test][️develop]⚙️ 合并switch命令到branch中
37 |5dabfb0 |wyy |2021年02月08日 18:27:53 |+10 -1 |⚙️ branch 增加remote分支信息

The problem is, it became really slow while precessing commits one by one.
HSplit([row(cmt) for cmt in commits]
So I try to replace it by using multithreading
with ThreadPoolExecutor(max_workers=5) as executor: futures = [executor.submit(row, cmt) for cmt in commits] wait(futures, return_when=ALL_COMPLETED)
And I got this Error

_Traceback (most recent call last):
 File "C:\Users\ushareit\AppData\Roaming\Python\Python39\Scripts\gf-script.py", line 33, in <module>
 sys.exit(load_entry_point('gitflo', 'console_scripts', 'gf')())
 File "C:\Python39\lib\site-packages\typer\main.py", line 214, in __call__
 return get_command(self)(*args, **kwargs)
 File "C:\Python39\lib\site-packages\click\core.py", line 829, in __call__
 return self.main(*args, **kwargs)
 File "C:\Python39\lib\site-packages\click\core.py", line 782, in main
 rv = self.invoke(ctx)
 File "C:\Python39\lib\site-packages\click\core.py", line 1259, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "C:\Python39\lib\site-packages\click\core.py", line 1066, in invoke
 return ctx.invoke(self.callback, **ctx.params)
 File "C:\Python39\lib\site-packages\click\core.py", line 610, in invoke
 return callback(*args, **kwargs)
 File "C:\Python39\lib\site-packages\typer\main.py", line 497, in wrapper
 return callback(**use_params) # type: ignore
 File "c:\users\ushareit\documents\program\gf\gf\main.py", line 143, in log
 log_dialog(size)
 File "c:\users\ushareit\documents\program\gf\gf\log.py", line 71, in log_dialog
 data = future.result()
 File "C:\Python39\lib\concurrent\futures\_base.py", line 433, in result
 return self.__get_result()
 File "C:\Python39\lib\concurrent\futures\_base.py", line 389, in __get_result
 raise self._exception
 File "C:\Python39\lib\concurrent\futures\thread.py", line 52, in run
 result = self.fn(*self.args, **self.kwargs)
 File "c:\users\ushareit\documents\program\gf\gf\log.py", line 32, in row
 message = cmt.message.split('\n')[0]
 File "C:\Python39\lib\site-packages\gitdb\util.py", line 253, in __getattr__
 self._set_cache_(attr)
 File "C:\Python39\lib\site-packages\git\objects\commit.py", line 143, in _set_cache_
 self._deserialize(BytesIO(stream.read()))
 File "C:\Python39\lib\site-packages\git\cmd.py", line 446, in read
 data = self._stream.read(size)
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
 File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
 File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
 File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
 File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file_

Any suggestions about solving this error would be greatly appreciated.
I am using gitpython version 3.1.11.
Thanks a lot.

You must be logged in to vote

GitPython is not thread safe, and each thread must own its own instance of a Repo while only performing read operations on them.

Even though I am closing this issue, please feel free to keep posting in the comments when needed.

Replies: 2 comments

Comment options

GitPython is not thread safe, and each thread must own its own instance of a Repo while only performing read operations on them.

Even though I am closing this issue, please feel free to keep posting in the comments when needed.

You must be logged in to vote
0 replies
Answer selected by Byron
Comment options

Ok, Thanks for your reply.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants
Converted from issue

This discussion was converted from issue #1128 on February 26, 2021 11:18.

AltStyle によって変換されたページ (->オリジナル) /