[openstack-dev] [oslo] Option registration convention?
Mark McLoughlin
markmc at redhat.com
Thu Jun 13 06:09:53 UTC 2013
On Thu, 2013年06月13日 at 13:35 +0800, Zhongyue Luo wrote:
> Hi, all
>>> I'm facing problems with sample config file generation on some
> projects.
>>> The doc web team is working on finding a way to get all options for
> all projects (or as needed) to create more up-to-date and accurate doc
> config tables. They are planning to make the process as generic as
> possible to work with the source for any project so as to generate all
> config options, so that we can then have the options list converted to
> docbook format and update the docs site.
>>> Currently, listing of all register options can be done in Nova and
> Cinder using the tools below
> https://github.com/openstack/nova/blob/master/tools/conf/generate_sample.sh
>> https://github.com/openstack/cinder/blob/master/tools/conf/generate_sample.sh
>>>> So our plan is to move the "generate_sample.sh" to oslo with an
> additional docbook format ouput module which will enable:
> ./generate_sample.sh /opt/stack/nova
> ./generate_sample.sh /opt/stack/quantum
>> ./generate_sample.sh /opt/stack/keystone
>>>> The module which traverses the target file handed by
> "generate_sample.sh" is
> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/config/generator.py
>>>> The "generate_sample.sh" looks for files containing "Opt(" which means
> that file has a high chance of defining and registering an option
> inside. For instance,
>>> resource_tracker_opts = [
>>> cfg.IntOpt('reserved_host_disk_mb', default=0,
>>> help='Amount of disk in MB to reserve for the host'),
>>> cfg.IntOpt('reserved_host_memory_mb', default=512,
>>> help='Amount of memory in MB to reserve for the host'),
>>> cfg.StrOpt('compute_stats_class',
>>> default='nova.compute.stats.Stats',
>>> help='Class that will manage stats for the local
> compute host')
>>> ]
>>> CONF = cfg.CONF
>>> CONF.register_opts(resource_tracker_opts)
>>>>> So if a file is passed to "generator.py", a module object is created
> by importing the target file and all attributes in that module
> object which are oslo.config.cfg.Opt sub-classes or a list of
> oslo.config.cfg.Opt sub-classes are printed out in the sample config
> file.
>>> Therefore the assumption was that an option will be defined and
> registered in the same module.
>>> However, some projects have options defined and registered on
> different modules. For instance,
> https://github.com/openstack/keystone/blob/master/keystone/common/config.py
>>> Here, the options are defined but instead of directly registering
> them, it provides a function which registers the options. And the
> actual registration happens in,
> https://github.com/openstack/keystone/blob/master/keystone/config.py#L24
>>>> Another pattern which is hard to detect the registered options is,
> https://github.com/openstack/glance/blob/master/glance/cmd/scrubber.py#L54
>>>> This line is not only registering a option inside a function but the
> option is also declared inside cfg.register_opt() which is makes it
> even harder to detect the option.
>>> How can we solve this problem of not being able to detect options?
Well, first thing I'd say is that I wouldn't worry about handling every
possible way of declaring cfg config options.
We can have a convention for how to register config options and a tool
that supports projects using that convention.
I think we have that now, but keystone chooses not to follow that
convention ... e.g.
https://review.openstack.org/4547
I'm pretty sure part of my argument for those keystone changes was that
the sample config file generator wouldn't work with keystone as it
stands now.
So, IMHO - any project which doesn't follow the convention should just
come up with its own custom approach to automating config documentation.
> I have two suggestions.
>>> First method is to change how generate.py works.
> Rake all modules in project which imports oslo.config.cfg and create a
> module object of it to inspect the cfg.CONF object inside.
> We can cache all the options which were already seen and print out
> newly discovered options.
Ok, so we'd actually look in cfg.CONF for registered options rather than
just looking at *declared* options.
I don't think we should rely on everything being visible through
cfg.CONF. There can be good reasons not to use the global object. Also,
the cfgfilter thing in oslo-incubator would mean that not all options
are visible through cfg.CONF.
> The downside of this solution is that we have to call a private
> function in ConfigOpts called _all_opt_infos() to look at all options
> registered.
> https://github.com/openstack/oslo.config/blob/master/oslo/config/cfg.py#L1822
>> We would have to create a public api version of it.
I think it would be useful to have an API which would return the Opt
object associated with a given config option name:
def lookup(self, name, group=None):
return self._get_opt_info(opt_name, group)['opt']
and then we can just do:
def print_help(conf, group=None):
d = conf if group is None else conf[group]
for key in d.keys():
if isinstance(conf.GroupAttr, d[key]):
print_help(conf, group=key)
else:
print key, conf.lookup(key, group).help
> The second option is to make it a convention that all options should
> be declared in global view and also registered in the same module
> where they were defined.
Hmm, why would that help?
Doesn't the current tool just require that you declare options at
top-level module scope i.e.
my_opt = cfg.StrOpt(...)
my_opts = [cfg.StrOpt(...), cfg.IntOpt(...)
but the problem with keystone is it is dynamically creating the opt
objects at runtime.
> However it would be difficult to write a hacking.py routine to check
> this and there will be significant amount of code we have to change.
>>> So some questions,
> 1) Is it worth documenting all the options in a project?
Yes.
> 2) Should there be a convention of declaring and registering options?
There is a convention, but keystone doesn't follow it.
Also, I think the important thing for the sample generator should be the
convention for declaring options, not registering options.
> Or just have generate.py handle things on its own.
> 3) Any objections of adding a public version of _all_opt_infos()?
No objection to something like the lookup() above, but we already have a
way of iterating over registered options and groups.
Cheers,
Mark.
More information about the OpenStack-dev
mailing list