matplotlib-devel Mailing List for matplotlib

Eric Firing wrote:
> Jeff Whitaker wrote:
>> Ted Drain wrote:
>>> Could someone point me at a discussion/article that explains the 
>>> need for namespace packages? I'm sure there is some good reason for 
>>> it but on the surface it seems very confusing. I've always thought 
>>> that the purpose of the __init__ file is to define the public 
>>> interface for a package. So when you say:
>>>
>>> import foo
>>>
>>> You get foo defined in the way it should be. I'm not sure how doing :
>>>
>>> import foo.api as foo
>>>
>>> is an improvement. Whether the api is defined in __init__.py or 
>>> api.py doesn't seem to matter (though I'm sure this is where I'm not 
>>> understanding things...). I've googled and found references to 
>>> needing to install and distribute sub-packages separately but that 
>>> doesn't really seem to explain why __init__ can't be used in the 
>>> sub-package. Is this primarily a limitation in the distribution and 
>>> setup tools?
>>>
>>> Can someone shed some light on this for me?
>>>
>>> Ted
>>
>> Ted: I was wrong in my previous email - only the __init__.py in the 
>> top-level toolkits directory (now called mpl_toolkits) needs to be 
>> empty. So in the case of basemap, this means that the import has 
>> changed from
>>
>> from matplotlib.toolkits.basemap import Basemap
>>
>> to
>>
>> from mpl_toolkits.basemap import Basemap.
>>
>> We don't actually need to stuff things into an api.py file.
>
> But the reason you had to move to an independent mpl_toolkits package 
> (which is a namespace package) is because mpl itself can't be a 
> namespace package, because it has an __init__.py stuffed full of 
> goodies. Right? Not that having an independent mpl_toolkits is a bad 
> thing; it may make sense anyway. I think it simplifies the directory 
> structure of basemap (which seemed needlessly deeply nested), doesn't it?
Eric: Yes, you're correct on both counts.
>
> Also, the namespace package constraint is inherent in distributing 
> subpackages as eggs, correct? Without eggs there would be no such 
> constraint. Subpackages would physically land in subdirectories of 
> the main package upon installation.
Right again. 
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Jeff Whitaker wrote:
> Ted Drain wrote:
>> Could someone point me at a discussion/article that explains the need 
>> for namespace packages? I'm sure there is some good reason for it but 
>> on the surface it seems very confusing. I've always thought that the 
>> purpose of the __init__ file is to define the public interface for a 
>> package. So when you say:
>>
>> import foo
>>
>> You get foo defined in the way it should be. I'm not sure how doing :
>>
>> import foo.api as foo
>>
>> is an improvement. Whether the api is defined in __init__.py or 
>> api.py doesn't seem to matter (though I'm sure this is where I'm not 
>> understanding things...). I've googled and found references to 
>> needing to install and distribute sub-packages separately but that 
>> doesn't really seem to explain why __init__ can't be used in the 
>> sub-package. Is this primarily a limitation in the distribution and 
>> setup tools?
>>
>> Can someone shed some light on this for me?
>>
>> Ted
> 
> Ted: I was wrong in my previous email - only the __init__.py in the 
> top-level toolkits directory (now called mpl_toolkits) needs to be 
> empty. So in the case of basemap, this means that the import has 
> changed from
> 
> from matplotlib.toolkits.basemap import Basemap
> 
> to
> 
> from mpl_toolkits.basemap import Basemap.
> 
> We don't actually need to stuff things into an api.py file.
But the reason you had to move to an independent mpl_toolkits package 
(which is a namespace package) is because mpl itself can't be a 
namespace package, because it has an __init__.py stuffed full of 
goodies. Right? Not that having an independent mpl_toolkits is a bad 
thing; it may make sense anyway. I think it simplifies the directory 
structure of basemap (which seemed needlessly deeply nested), doesn't it?
Also, the namespace package constraint is inherent in distributing 
subpackages as eggs, correct? Without eggs there would be no such 
constraint. Subpackages would physically land in subdirectories of the 
main package upon installation.
Eric

Ted Drain wrote:
> Could someone point me at a discussion/article that explains the need 
> for namespace packages? I'm sure there is some good reason for it but 
> on the surface it seems very confusing. I've always thought that the 
> purpose of the __init__ file is to define the public interface for a 
> package. So when you say:
>
> import foo
>
> You get foo defined in the way it should be. I'm not sure how doing :
>
> import foo.api as foo
>
> is an improvement. Whether the api is defined in __init__.py or 
> api.py doesn't seem to matter (though I'm sure this is where I'm not 
> understanding things...). I've googled and found references to 
> needing to install and distribute sub-packages separately but that 
> doesn't really seem to explain why __init__ can't be used in the 
> sub-package. Is this primarily a limitation in the distribution and 
> setup tools?
>
> Can someone shed some light on this for me?
>
> Ted
Ted: I was wrong in my previous email - only the __init__.py in the 
top-level toolkits directory (now called mpl_toolkits) needs to be 
empty. So in the case of basemap, this means that the import has 
changed from
from matplotlib.toolkits.basemap import Basemap
to
from mpl_toolkits.basemap import Basemap.
We don't actually need to stuff things into an api.py file.
-Jeff
>
> At 08:11 AM 1/10/2008, Jeff Whitaker wrote:
>> Andrew Straw wrote:
>> > Great -- hopefully that saved you some API re-arrangement pain. No
>> > problem on shuffling mpl_sizer around -- please go ahead do it if you
>> > have time.
>> >
>> > -Andrew
>> >
>> > Jeff Whitaker wrote:
>> >> Andrew: Thanks, you've convinced me. Is it OK with you if I go
>> >> ahead and make those changes to mplsizer at the same time I do 
>> basemap?
>> >>
>> >> -Jeff
>> >>
>> >
>> Andrew: OK, the change to mpl_toolkits (which is now a proper namespace
>> package) is all done.
>>
>> -Jeff
>>
>> -- 
>> Jeffrey S. Whitaker Phone : (303)497-6313
>> Meteorologist FAX : (303)497-6449
>> NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
>> 325 Broadway Office : Skaggs Research Cntr 1D-124
>> Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
>>
>>
>> -------------------------------------------------------------------------
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace 
>>
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel 
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> 
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Could someone point me at a discussion/article that explains the need 
for namespace packages? I'm sure there is some good reason for it 
but on the surface it seems very confusing. I've always thought that 
the purpose of the __init__ file is to define the public interface 
for a package. So when you say:
import foo
You get foo defined in the way it should be. I'm not sure how doing :
import foo.api as foo
is an improvement. Whether the api is defined in __init__.py or 
api.py doesn't seem to matter (though I'm sure this is where I'm not 
understanding things...). I've googled and found references to 
needing to install and distribute sub-packages separately but that 
doesn't really seem to explain why __init__ can't be used in the 
sub-package. Is this primarily a limitation in the distribution and 
setup tools?
Can someone shed some light on this for me?
Ted
At 08:11 AM 1/10/2008, Jeff Whitaker wrote:
>Andrew Straw wrote:
> > Great -- hopefully that saved you some API re-arrangement pain. No
> > problem on shuffling mpl_sizer around -- please go ahead do it if you
> > have time.
> >
> > -Andrew
> >
> > Jeff Whitaker wrote:
> >> Andrew: Thanks, you've convinced me. Is it OK with you if I go
> >> ahead and make those changes to mplsizer at the same time I do basemap?
> >>
> >> -Jeff
> >>
> >
>Andrew: OK, the change to mpl_toolkits (which is now a proper namespace
>package) is all done.
>
>-Jeff
>
>--
>Jeffrey S. Whitaker Phone : (303)497-6313
>Meteorologist FAX : (303)497-6449
>NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
>325 Broadway Office : Skaggs Research Cntr 1D-124
>Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
>
>
>-------------------------------------------------------------------------
>Check out the new SourceForge.net Marketplace.
>It's the best place to buy or sell services for
>just about anything Open Source.
>http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
>_______________________________________________
>Matplotlib-devel mailing list
>Mat...@li...
>https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Andrew Straw wrote:
> Great -- hopefully that saved you some API re-arrangement pain. No 
> problem on shuffling mpl_sizer around -- please go ahead do it if you 
> have time.
>
> -Andrew
>
> Jeff Whitaker wrote:
>> Andrew: Thanks, you've convinced me. Is it OK with you if I go 
>> ahead and make those changes to mplsizer at the same time I do basemap?
>>
>> -Jeff
>>
>
Andrew: OK, the change to mpl_toolkits (which is now a proper namespace 
package) is all done.
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

There was a question on the matplotlib-users list about symbols in 
scatter, that made me realize that the new "marker = (5,0)" ... syntax, 
that I contributed some time ago, were not documented. So I tried to 
write a doc string.
Can anyone please check this and add the patch? Btw: shouldn't the 
"verts" keyword be removed, since marker=(verts,0) is equivalent ?
Manuel

Great -- hopefully that saved you some API re-arrangement pain. No 
problem on shuffling mpl_sizer around -- please go ahead do it if you 
have time.
-Andrew
Jeff Whitaker wrote:
> Andrew: Thanks, you've convinced me. Is it OK with you if I go ahead 
> and make those changes to mplsizer at the same time I do basemap?
>
> -Jeff
>

Andrew Straw wrote:
> Jeff Whitaker wrote:
> 
>> Darren Dale wrote:
>> 
>>> On Wednesday 09 January 2008 7:01:14 pm Jeff Whitaker wrote:
>>> 
>>> 
>>>> Andrew Straw wrote:
>>>> 
>>>> 
>>>>> As the author of the only other known MPL toolkit (at least in the MPL
>>>>> tree), I'm happy with the idea of using a namespace package for
>>>>> mpl_toolkits. I understand your proposal to mean that each toolkit
>>>>> would
>>>>> have a directory structure:
>>>>>
>>>>> setup.py
>>>>> lib/
>>>>> mpl_toolkits/
>>>>> __init__.py (empty)
>>>>> basemap/
>>>>> __init__.py
>>>>> other_files.py
>>>>>
>>>>> This is OK with me, but I question is whether it's necessary to
>>>>> have the
>>>>> "lib" directory -- it seems entirely redundant. I'm fine with either
>>>>> way, though.
>>>>>
>>>>> -Andrew
>>>>> 
>>>>> 
>>>> Andrew: Yes, that's it, except that all the __init__.py files must be
>>>> empty, not just the first one.
>>>> 
>>>> 
>>> Just to clarify, all __init__.pys must be empty? I have no experience
>>> with setuptools, but the way I read the documentation, it sounded
>>> like only the __init__.py in the namespace package needed to be
>>> empty, like Andrew showed.
>>>
>>> Hopefully not muddying the waters,
>>> Darren
>>> 
>>> 
>> Darren: I was assuming they both needed to be namespace packages
>> (mpl_tookits and mpl_toolkits.basemap).
>> I just went back and re-read it, and now I'm just not sure ...
>> The waters are muddy. 
>> 
> I dealt with this recently in my as-yet-to-be-really-announced set of
> packages for realtime image analysis: motmot, at
> http://code.astraw.com/projects/motmot
>
> I'm reasonably certain that Darren is correct -- __init__.py only needs
> to be empty if the package is a namespace package. In other words, if
> you wanted to have additional packages in the mpl_toolkits.basemap
> namespace, mpl_toolkits/basemap/__init__.py would have to be empty. If
> you don't see anything else in that namespace, you can keep __init__.py
> containing whatever it does.
>
> Note that for 2nd level namespace packages to work at all, you need the
> most recent setuptools (0.6c7), as a bug was recently discovered by the
> Enthought Tool Suite folks regarding this feature.
> 
Andrew: Thanks, you've convinced me. Is it OK with you if I go ahead 
and make those changes to mplsizer at the same time I do basemap?
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449
325 Broadway Boulder, CO, USA 80305-3328

Jeff Whitaker wrote:
> Darren Dale wrote:
>> On Wednesday 09 January 2008 7:01:14 pm Jeff Whitaker wrote:
>> 
>>> Andrew Straw wrote:
>>> 
>>>> As the author of the only other known MPL toolkit (at least in the MPL
>>>> tree), I'm happy with the idea of using a namespace package for
>>>> mpl_toolkits. I understand your proposal to mean that each toolkit
>>>> would
>>>> have a directory structure:
>>>>
>>>> setup.py
>>>> lib/
>>>> mpl_toolkits/
>>>> __init__.py (empty)
>>>> basemap/
>>>> __init__.py
>>>> other_files.py
>>>>
>>>> This is OK with me, but I question is whether it's necessary to
>>>> have the
>>>> "lib" directory -- it seems entirely redundant. I'm fine with either
>>>> way, though.
>>>>
>>>> -Andrew
>>>> 
>>> Andrew: Yes, that's it, except that all the __init__.py files must be
>>> empty, not just the first one.
>>> 
>>
>> Just to clarify, all __init__.pys must be empty? I have no experience
>> with setuptools, but the way I read the documentation, it sounded
>> like only the __init__.py in the namespace package needed to be
>> empty, like Andrew showed.
>>
>> Hopefully not muddying the waters,
>> Darren
>> 
> Darren: I was assuming they both needed to be namespace packages
> (mpl_tookits and mpl_toolkits.basemap).
> I just went back and re-read it, and now I'm just not sure ...
> The waters are muddy. 
I dealt with this recently in my as-yet-to-be-really-announced set of
packages for realtime image analysis: motmot, at
http://code.astraw.com/projects/motmot
I'm reasonably certain that Darren is correct -- __init__.py only needs
to be empty if the package is a namespace package. In other words, if
you wanted to have additional packages in the mpl_toolkits.basemap
namespace, mpl_toolkits/basemap/__init__.py would have to be empty. If
you don't see anything else in that namespace, you can keep __init__.py
containing whatever it does.
Note that for 2nd level namespace packages to work at all, you need the
most recent setuptools (0.6c7), as a bug was recently discovered by the
Enthought Tool Suite folks regarding this feature.

Darren Dale wrote:
> On Wednesday 09 January 2008 7:01:14 pm Jeff Whitaker wrote:
> 
>> Andrew Straw wrote:
>> 
>>> As the author of the only other known MPL toolkit (at least in the MPL
>>> tree), I'm happy with the idea of using a namespace package for
>>> mpl_toolkits. I understand your proposal to mean that each toolkit would
>>> have a directory structure:
>>>
>>> setup.py
>>> lib/
>>> mpl_toolkits/
>>> __init__.py (empty)
>>> basemap/
>>> __init__.py
>>> other_files.py
>>>
>>> This is OK with me, but I question is whether it's necessary to have the
>>> "lib" directory -- it seems entirely redundant. I'm fine with either
>>> way, though.
>>>
>>> -Andrew
>>> 
>> Andrew: Yes, that's it, except that all the __init__.py files must be
>> empty, not just the first one.
>> 
>
> Just to clarify, all __init__.pys must be empty? I have no experience with 
> setuptools, but the way I read the documentation, it sounded like only the 
> __init__.py in the namespace package needed to be empty, like Andrew showed.
>
> Hopefully not muddying the waters,
> Darren
> 
Darren: I was assuming they both needed to be namespace packages 
(mpl_tookits and mpl_toolkits.basemap).
I just went back and re-read it, and now I'm just not sure ... 
The waters are muddy.
-Jeff
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449
325 Broadway Boulder, CO, USA 80305-3328

On Wednesday 09 January 2008 7:01:14 pm Jeff Whitaker wrote:
> Andrew Straw wrote:
> > As the author of the only other known MPL toolkit (at least in the MPL
> > tree), I'm happy with the idea of using a namespace package for
> > mpl_toolkits. I understand your proposal to mean that each toolkit would
> > have a directory structure:
> >
> > setup.py
> > lib/
> > mpl_toolkits/
> > __init__.py (empty)
> > basemap/
> > __init__.py
> > other_files.py
> >
> > This is OK with me, but I question is whether it's necessary to have the
> > "lib" directory -- it seems entirely redundant. I'm fine with either
> > way, though.
> >
> > -Andrew
>
> Andrew: Yes, that's it, except that all the __init__.py files must be
> empty, not just the first one.
Just to clarify, all __init__.pys must be empty? I have no experience with 
setuptools, but the way I read the documentation, it sounded like only the 
__init__.py in the namespace package needed to be empty, like Andrew showed.
Hopefully not muddying the waters,
Darren

Andrew Straw wrote:
> As the author of the only other known MPL toolkit (at least in the MPL
> tree), I'm happy with the idea of using a namespace package for
> mpl_toolkits. I understand your proposal to mean that each toolkit would
> have a directory structure:
>
> setup.py
> lib/
> mpl_toolkits/
> __init__.py (empty)
> basemap/
> __init__.py
> other_files.py
>
> This is OK with me, but I question is whether it's necessary to have the
> "lib" directory -- it seems entirely redundant. I'm fine with either
> way, though.
>
> -Andrew
Andrew: Yes, that's it, except that all the __init__.py files must be 
empty, not just the first one. Additionally, there would be a file 
'api.py' in the same directory as 'other_files.py' that would have all 
the stuff that previously was imported directly into __init__.py. The 
rationale for this convention is discussed at 
http://neuroimaging.scipy.org/neuroimaging/ni/ticket/86, and is being 
used now in several projects.
 The lib directory is not necessary, you would just change the
package_dirs = {'':'lib'}
line in setup.py accordingly.
-Jeff
> 
>
> Jeff Whitaker wrote:
> 
>> Now that the transforms branch has merged with the trunk, I'd like to 
>> resurrect namespace packages so that toolkits will work again when 
>> matplotlib is installed as an egg. As was discussed in a previous 
>> thread, all __init__.py files in the toolkit hierarchy must be empty 
>> (aside declare_namespace statement). Since lib/matplotlib/__init__.py 
>> contains a lot of important stuff, I think the path of least resistance 
>> is to move the toolkits out of lib/matplotlib and into a separate 
>> directory lib/mpl_toolkits. The semantics of importing a toolkit would 
>> have to change from
>>
>> import matplotlib.toolkits.toolkit
>>
>> to
>>
>> import mpl_toolkit.toolkit.
>>
>> Of course, all the toolkit __init__.py files would need to be emptied. 
>> In the case of basemap, this would be changing imports from
>>
>> from matplotlib.toolkits.basemap import Basemap
>>
>> to something like
>>
>> from mpl_toolkits.basemap.api import Basemap
>>
>> All the stuff now imported directly into __init__.py would go in api.py.
>>
>> I've tried this in my local tree and it seems to work fine. Does this 
>> sound reasonable? If there's general agreement, I can make the 
>> necessary mods in matplotlib trunk and the mplsizer and basemap toolkits.
>>
>> -Jeff
>>
>> 
>> 
>
> 
-- 
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jef...@no...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg