This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2014年12月30日 10:52 by serhiy.storchaka, last changed 2022年04月11日 14:58 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| ipaddress_pickle.patch | serhiy.storchaka, 2014年12月30日 10:52 | review | ||
| ipaddress_pickle_2.patch | serhiy.storchaka, 2015年01月16日 10:08 | Pickle addresses as ints | review | |
| ipaddress_pickle_3.patch | serhiy.storchaka, 2015年01月16日 10:08 | + optimization | review | |
| Messages (10) | |||
|---|---|---|---|
| msg233201 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年12月30日 10:52 | |
Currently ipaddress classes support pickling, but the pickling is not efficient and is implementation depened. Proposed patch makes pickling more compact and implementation agnostic. |
|||
| msg234118 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2015年01月16日 08:49 | |
Patch looks good to me. For further efficiency, addresses could be pickled as ints (but beware of interfaces and networks). |
|||
| msg234121 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2015年01月16日 10:06 | |
Yes, pickling (and especially unpickling) ints is more efficient, but the code will more complex. Interfaces should be pickled as strings for backward compatibility, and interfaces are subclasses of addresses.
Here are microbenchmarks:
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"
Results for unpatched module:
1000 loops, best of 3: 1.56 msec per loop
1000 loops, best of 3: 1.62 msec per loop
1000 loops, best of 3: 1.08 msec per loop
1000 loops, best of 3: 1.09 msec per loop
With ipaddress_pickle.patch:
100 loops, best of 3: 3.43 msec per loop
100 loops, best of 3: 10.6 msec per loop
100 loops, best of 3: 7.76 msec per loop
100 loops, best of 3: 8.58 msec per loop
With ipaddress_pickle_2.patch:
1000 loops, best of 3: 1.11 msec per loop
1000 loops, best of 3: 1.16 msec per loop
1000 loops, best of 3: 1.88 msec per loop
100 loops, best of 3: 2.05 msec per loop
With ipaddress_pickle_3.patch:
1000 loops, best of 3: 1.12 msec per loop
1000 loops, best of 3: 1.15 msec per loop
1000 loops, best of 3: 1.13 msec per loop
1000 loops, best of 3: 1.15 msec per loop
|
|||
| msg234123 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2015年01月16日 10:20 | |
ipaddress_pickle_3.patch breaks one test (testMissingAddressVersion). Is this test needed? |
|||
| msg234124 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2015年01月16日 10:43 | |
I don't understand what the test is for. I think it's safe it's remove it. |
|||
| msg234264 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2015年01月18日 18:27 | |
Then I'll remove it. Could you please make a review of optimized patch? |
|||
| msg234270 - (view) | Author: Antoine Pitrou (pitrou) * (Python committer) | Date: 2015年01月18日 20:13 | |
The patch looks fine to me. |
|||
| msg234274 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2015年01月18日 20:39 | |
New changeset 781b54f7bccc by Serhiy Storchaka in branch 'default': Issue #23133: Pickling of ipaddress objects now produces more compact and https://hg.python.org/cpython/rev/781b54f7bccc |
|||
| msg234276 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2015年01月18日 20:53 | |
Thank you Antoine.
And here is comparison of pickle size.
Unpatched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))
2971
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))
4071
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))
19341
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))
22741
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))
10614
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))
22741
Patched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))
1531
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))
2631
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))
2963
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))
3256
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))
2938
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))
3209
|
|||
| msg234277 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2015年01月18日 20:57 | |
New changeset 712ac77b772b by Serhiy Storchaka in branch 'default': Fixed tests for issue #23133 (pickling of IPv4Network was not tested). https://hg.python.org/cpython/rev/712ac77b772b |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:58:11 | admin | set | github: 67322 |
| 2015年01月18日 20:57:32 | python-dev | set | messages: + msg234277 |
| 2015年01月18日 20:53:56 | serhiy.storchaka | set | status: open -> closed resolution: fixed messages: + msg234276 stage: patch review -> resolved |
| 2015年01月18日 20:39:56 | python-dev | set | nosy:
+ python-dev messages: + msg234274 |
| 2015年01月18日 20:13:29 | pitrou | set | messages: + msg234270 |
| 2015年01月18日 18:27:23 | serhiy.storchaka | set | messages: + msg234264 |
| 2015年01月16日 14:49:59 | serhiy.storchaka | set | assignee: serhiy.storchaka |
| 2015年01月16日 10:43:24 | pitrou | set | messages: + msg234124 |
| 2015年01月16日 10:20:34 | serhiy.storchaka | set | messages: + msg234123 |
| 2015年01月16日 10:08:51 | serhiy.storchaka | set | files: + ipaddress_pickle_3.patch |
| 2015年01月16日 10:08:25 | serhiy.storchaka | set | files: + ipaddress_pickle_2.patch |
| 2015年01月16日 10:08:02 | serhiy.storchaka | set | files: - ipaddress_pickle_3.patch |
| 2015年01月16日 10:07:53 | serhiy.storchaka | set | files: - ipaddress_lightweight_2.patch |
| 2015年01月16日 10:07:43 | serhiy.storchaka | set | files: + ipaddress_pickle_3.patch |
| 2015年01月16日 10:06:50 | serhiy.storchaka | set | files:
+ ipaddress_lightweight_2.patch messages: + msg234121 |
| 2015年01月16日 08:49:12 | pitrou | set | nosy:
+ pitrou messages: + msg234118 |
| 2014年12月30日 10:56:39 | serhiy.storchaka | link | issue23103 dependencies |
| 2014年12月30日 10:52:49 | serhiy.storchaka | create | |