homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: MemoryError While Executing Python Code
Type: resource usage Stage: test needed
Components: Versions: Python 2.6
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: brian.curtin, freakcycle, mark.dickinson, p_noblebose, pitrou
Priority: normal Keywords:

Created on 2010年02月10日 14:58 by p_noblebose, last changed 2022年04月11日 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
Aura_Test_Scripts_FCT_ACTUSER_EVENT_L.tsv p_noblebose, 2010年02月11日 08:13 Test cases document
Messages (8)
msg99164 - (view) Author: Prashanth noble (p_noblebose) Date: 2010年02月10日 14:58
we are executing two queries and storing the result sets and matching the data.
the table has got around 8 million records and while executing the same i am getting the Memory Issue.
TRACE Arguments: [ <DatabaseLibrary.resultset.Resultset instance at 0x01750828> | <DatabaseLibrary.resultset.Resultset instance at 0x01750E68> ] 
FAIL MemoryError 
DEBUG Traceback (most recent call last):
 File "c:\python26\lib\site-packages\DatabaseLibrary\__init__.py", line 52, in resultsets_should_match
 resultset1.matches_with(resultset2)
 File "c:\python26\lib\site-packages\DatabaseLibrary\resultset.py", line 37, in matches_with
 for row_a, row_b in zip(self, other): 
Please let me know how to resolve this issue
msg99197 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010年02月11日 04:14
Can you provide a test case which can reproduce the issue? Specifically one which doesn't require the use of your DatabaseLibrary package?
msg99201 - (view) Author: Prashanth noble (p_noblebose) Date: 2010年02月11日 08:13
please find the testcase i am using
msg99210 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010年02月11日 14:46
Prashanth, I'm not even sure what to do with that file.
Can you provide a Python script which demonstrates your issue? It should be something that myself or others could run on their computer, free of any third party packages. This will isolate the issue to Python and show us how to reproduce what you are seeing.
msg99213 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010年02月11日 15:10
If these 8 million records are loaded into memory, it can be normal to get a MemoryError. It depends on the complexity of the records (number of fields) and the way they are kept in memory (tuples? dicts?); and of course of the amount of available RAM in your system.
In any case your description of the problem is not sufficiently detailed for us to now whether it's a problem with Python, a problem with your code, or simply some mismatching expectations on your part.
msg109366 - (view) Author: Peter Wolf (freakcycle) Date: 2010年07月06日 04:44
I am new to this so apologies if I posting this in the wrong spot.
I am also getting a memory error on an 86MB python data file.It is one long list.As far as I can tell the data file is valid python code.My computer has 3.4 GB of ram.
When I type the following in a terminal 
> python mydatafile.py
On the following line I get 
MemoryError
That is all.
msg109375 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010年07月06日 09:32
Peter, this looks like a different problem; please could you open a new issue?
When/if you do, please could you also report:
 - what version of Python you're using
 - your system details (what OS? 32-bit or 64-bit?)
If you can give us a better idea of what's in your 86Mb file, that would also help: perhaps post the first few and last few lines. What's in this list? Integers? Floats? Strings?
It's not totally implausible that Python really *does* need gigabytes of memory to process a list in an 86Mb file.
msg109381 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010年07月06日 10:01
Closing the original issue: there's insufficient information to reproduce or analyze the problem.
History
Date User Action Args
2022年04月11日 14:56:57adminsetgithub: 52147
2010年07月06日 10:01:47mark.dickinsonsetstatus: open -> closed
resolution: works for me
messages: + msg109381
2010年07月06日 09:32:03mark.dickinsonsetnosy: + mark.dickinson
messages: + msg109375
2010年07月06日 04:44:36freakcyclesetnosy: + freakcycle
messages: + msg109366
components: - Windows
2010年02月11日 15:10:27pitrousetnosy: + pitrou
messages: + msg99213
2010年02月11日 14:46:32brian.curtinsetmessages: + msg99210
2010年02月11日 08:13:38p_noblebosesetfiles: + Aura_Test_Scripts_FCT_ACTUSER_EVENT_L.tsv

messages: + msg99201
2010年02月11日 04:14:53brian.curtinsetpriority: normal

nosy: + brian.curtin
messages: + msg99197

type: performance -> resource usage
stage: test needed
2010年02月10日 14:58:28p_noblebosecreate

AltStyle によって変換されたページ (->オリジナル) /