302 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
1
vote
1
answer
111
views
dill.load_session() causes `UnpicklingError: pickle data was truncated` even though dill.dump_session() pickled perfectly fine
I'm running a Jupyter notebook on a remote server through interactive nodes. I ran many cells of the notebook, and in order to save the state of the notebook, I ran
import dill
filename = <insert ...
0
votes
0
answers
110
views
multiprocess library barely works
I'm using the multiprocess library to accelerate a CPU-bound task (a method inside a user-defined class).
The function processes a page of a document, in my example a 500-page document takes around 20 ...
0
votes
2
answers
115
views
How to pickle a class instance with persistent methods in Python?
I want to serialize a class instance in python and keep methods persistent. I have tried with joblib and pickle and am really close with dill, but can't quite get it.
Here is the problem. Say I want ...
3
votes
0
answers
107
views
How to Store llm_client Object in Redis with FastAPI using Pickle or Dill?
I’m working on a FastAPI application where I need to store an Azure OpenAI llm_client object in Redis, along with some session data, and set an expiration time. I initially tried using pickle for ...
4
votes
1
answer
148
views
Why does recurse=True cause dill not to respect globals in functions?
If I pickle a function with dill that contains a global, somehow that global state isn't respected when the function is loaded again. I don't understand enough about dill to be anymore specific, but ...
quant's user avatar
- 23.6k
1
vote
1
answer
354
views
What could even be an error when loading a pickled file in Python?
The pickled file is voc_final.pkl, in the source code used in the paper GAMENet.
I tried to load it:
import dill
voc = dill.load(open("voc_final.pkl", 'rb'))
But got the error:
TypeError: ...
0
votes
1
answer
91
views
How to do inter-process communication with pathos/ppft?
I'm using the pathos framework to do tasks concurrently, in different processes. Under the hood, this is done with ppft, which is part of pathos. My current approach uses a pathos.multiprocessing....
1
vote
1
answer
252
views
Passing dynamically generated classes' instances as parameters for celery tasks
Goal
The objective is to pass an instance of a dynamically generated class (inheriting from a statically defined one) as a parameter to a task to be executed on a Celery worker.
Issue
I am ...
0
votes
0
answers
376
views
Import dill failed (but only in my process : No module named 'dill)
I'm facing a strange situation that I can't seem to understand: my main program is an application built with PySide6 that launches several QProcesses. I want to use dill.dumps and dill.loads to ...
0
votes
0
answers
61
views
python - ImportError: cannot import name '_is_imported_module' from 'dill._dill'
Installed datasets package into python virtual environment. When I try to import it, running,
from datasets import load_dataset, I get this error, "ImportError: cannot import name '...
3
votes
1
answer
71
views
isinstance() fails on an object contained in a list, after using dill.dump and dill.load
Is this expected behaviour (and if so, can someone explain why)?
This only happens when using dill, not pickle.
from pathlib import Path
import dill
class MyClass:
def __init__(self) -> None:
...
1
vote
2
answers
616
views
AttributeError: module 'dill' has no attribute 'PY3'
This is the error I encountered when trying to apply to the dataset, is there any way to fix it?
/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:2645: FutureWarning:...
0
votes
2
answers
278
views
Not pickable object using multiprocessing in SAP GUI with Python
there.
I have two daily extractions using Python in SAP GUI that consume a lot of time, and this must be done faster. So, i am trying to run in parallel two transactions for solve this issue.
I am ...
1
vote
1
answer
141
views
Can instances of classes defined in a module (i.e. not top level) be pickled somehow?
According to the documentation, this is not possible. But I am wondering if there is a workaround to this.
It seems like a very arbitrary limitation that would mess up with the overall code structure
...
0
votes
0
answers
50
views
Pickle the python module so that after loading it I can use it normally even if there are no .py files that correspond to that module anymore
Pickle can not dump modules at all, and dill python library can. But I was not able to figure out how I can dump it with dill so that it doesn't "ask" for having source python files that ...