This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2007年07月19日 18:17 by taleinat, last changed 2022年04月11日 14:56 by admin. This issue is now closed.
| Messages (8) | |||
|---|---|---|---|
| msg32530 - (view) | Author: Tal Einat (taleinat) * (Python committer) | Date: 2007年07月19日 18:17 | |
This bug manifests only when running with a subprocess. Trying to display an instance of BeautifulSoup's NavigableString class, this is the result: RuntimeError: maximum recursion depth exceeded See http://mail.python.org/pipermail/idle-dev/2007-July/002600.html for details (including how to recreate the error). Diagnosis: The problem arises when trying to pickle such instances - pickle enters an endless loop and reaches the max recursion limit (eventually). This happens regardless of the protocol used. IDLE is probably trying to pickle them because their class, NavigableString, inherits from unicode, so isinstance(<NavigableString instance>, basestring) return True. Possibly related to SF bug #1581183: "pickle protocol 2 failure on int subclass" http://sourceforge.net/tracker/index.php?funchttp://sourceforge.net/tracker/index.php?func=detail&aid=1581183&group_id=5470&atid=105470=detail&aid=1512695&group_id=5470&atid=105470 Fix: IDLE should check for any exception when trying to pickle, not just pickle.PicklingError, to avoid such errors. If pickle doesn't work, for whatever reason, IDLE can still work around it with str() and repr(). I'll post a bug report for Pickle as well. |
|||
| msg84318 - (view) | Author: Guilherme Polo (gpolo) * (Python committer) | Date: 2009年03月28日 18:06 | |
I can't seem to reproduce this here. I've tried both python 2.4.5 and 2.5.2 using beautifulsoup 3.0.7. I also used the sample html attached in that email as well part of the code that is supposed to cause the problem, and I can also run pickle on the body's contents without getting a RuntimeError. Any chance you can retry it ? And if you do reproduce it then include the instructions here instead of pointing to a mail list, please. |
|||
| msg84412 - (view) | Author: Tal Einat (taleinat) * (Python committer) | Date: 2009年03月29日 15:59 | |
To recreate use BeautifulSoup 3.0.4 and run the following:
>>> from BeautifulSoup import BeautifulSoup
>>> soup = BeautifulSoup("<html>aa</html")
>>> x = soup.find('html').contents[0]
>>> x
u'aa'
>>> print x
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
print x
RuntimeError: maximum recursion depth exceeded
This is caused by a bug in BeautifulSoup which was fixed in version
3.0.5. The bug manifests when trying to pickle an instance of the
NavigableString class.
In the above scenario, IDLE has the subprocess pickle the object and
send it to the parent process. Since the problem is with the pickling,
turning the object into a string in the subprocess (instead of sending
it as-is to the parent process) avoids generating the error:
>>> print str(x)
aa
>>> print repr(x)
u'aa'
To verify that pickle is the culprit:
>>> import pickle
>>> pickle.dumps(x)
(very long traceback...)
RuntimeError: maximum recursion depth exceeded
Like I said in my first post, IMO IDLE should check for any exception
(not just pickle.PicklingError) when trying to pickle an object for
sending to the parent process. If pickle doesn't work, for whatever
reason, IDLE can still try to work around it with str() and/or repr().
(I tried this with Python 2.5 but I've tested this in the past with 2.6
as well. I haven't tried it with 3.0 or 2.7 yet.)
|
|||
| msg84425 - (view) | Author: Guilherme Polo (gpolo) * (Python committer) | Date: 2009年03月29日 19:49 | |
> IMO IDLE should check for any exception > (not just pickle.PicklingError) when trying to pickle an object for > sending to the parent process. If pickle doesn't work, for whatever > reason, IDLE can still try to work around it with str() and/or repr(). > Do you have some specific suggestion on how to do so ? As I'm seeing it rpc.SocketIO.putmessage would return a custom exception when a PicklingError doesn't happen, which then PyShell.ModifiedInterpreter.runcode could handle and decide to run the code object locally. Sounds unpleasant. |
|||
| msg84436 - (view) | Author: Tal Einat (taleinat) * (Python committer) | Date: 2009年03月29日 22:01 | |
Sending a code object back to the parent process and having it deal with the situation sounds very unpleasant indeed! I think a completely different type of solution may be possible. In general, I can't think of any reason for IDLE to pickle "user objects" from the subprocess and send them to the parent process; it should merely send back the output (as strings), with special cases for exceptions and such. By "user objects" I mean objects "inside the interpreter", as opposed to those used by IDLE itself. I'll have to unwind the spaghetti in rpc.py, run.py and PyShell.py a bit more to propose a specific set of changes; I hope to get to that tomorrow. |
|||
| msg116705 - (view) | Author: Mark Lawrence (BreamoreBoy) * | Date: 2010年09月17日 18:31 | |
Can we close this given "This is caused by a bug in BeautifulSoup which was fixed in version 3.0.5." from msg84412? |
|||
| msg116715 - (view) | Author: Ned Deily (ned.deily) * (Python committer) | Date: 2010年09月17日 19:32 | |
While BeautifulSoup may have been fixed, the issue here still points to an underlying problem in IDLE being vulnerable to pickling errors. The given test case still fails in 2.7 (I didn't try to construct a test for Python 3). |
|||
| msg165126 - (view) | Author: Roger Serwy (roger.serwy) * (Python committer) | Date: 2012年07月09日 21:12 | |
I think that issue13532 fixes this issue, as only strings are now pickled from the subprocess to the IDLE front-end for writing to stdout and stderr. This should address the pickling vulnerability. I downloaded BeautifulSoup 3.0.4 and did not see this problem when using IDLE 2.7.1, and then against the latest dev version from the repo. I am closing this issue as being out of date. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:56:25 | admin | set | github: 45221 |
| 2012年07月09日 21:12:50 | roger.serwy | set | status: open -> closed nosy: + roger.serwy messages: + msg165126 resolution: out of date |
| 2012年07月09日 04:43:32 | terry.reedy | set | nosy:
+ terry.reedy |
| 2010年09月17日 19:32:29 | ned.deily | set | status: pending -> open title: IDLE + BeautifulSoup = Error -> Unexpected "maximum recursion depth exceeded" in IDLE shell with objects that cannot be pickled nosy: + kbk, ned.deily versions: + Python 2.7 messages: + msg116715 resolution: not a bug -> (no value) |
| 2010年09月17日 18:31:19 | BreamoreBoy | set | status: open -> pending nosy: + BreamoreBoy messages: + msg116705 resolution: not a bug |
| 2009年03月29日 22:01:33 | taleinat | set | messages: + msg84436 |
| 2009年03月29日 19:49:27 | gpolo | set | messages: + msg84425 |
| 2009年03月29日 15:59:21 | taleinat | set | messages: + msg84412 |
| 2009年03月28日 18:06:41 | gpolo | set | nosy:
+ gpolo messages: + msg84318 |
| 2007年07月19日 18:17:37 | taleinat | create | |