python data structure suggestion?

Question

I have a complex pricing data, like a tree structure

Example will be like computers price (monitor price, motherboard price etc.) and in monitors category I have more sub-category, and under those sub-categories I have more categories (monitor which is 27 inch, which made by dell, which is curved)

I need to frequently read these pricing information (read only) like thousands of times.

I want to use class to store these information. Because I don't know if I can do it in dictionaries. Anyone have a suggestion?

• zenlc2000: Use a SQLite database or a better fit might be TinyDB tinydb.readthedocs.io/en/latest
• Moinuddin Quadri: Use any relational database if possible. If not, you need to follow the OOP principles. Create a class as Monitor with properties as price, manufacturer, display_size, monitor_type, etc. Or more classes for sub-categories. See where you can inherit properties
• Loïc: I'd use big data for that.
• Nickpick: My vote goes for monbodb, you can store you dictionaries and retrieve them easily, it is also scalable among different servers
• Daniel Yang: @nickpick my pricing data is very limited (below 50 entries), however it need to be accessed frequently (thousands of times), do you think mongoDB is overkilled?

Answer 1

Mongodb is definitely a good possibility but in your case with only 50 entries and only to read it's probably an overkill, especially as you will need to get familiar with how to do the queries.

A quicker way is most likely via pandas: use a nested dictionary, best create an input JSON file or string (as in the example below) and then read it in do a pandas dataframe.

You can then normalize it in the way you want it and do the calculations necessary in pandas, which you can learn much quicker:

Here an example of how it could look like: http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.io.json.json_normalize.html

>>> data = [{'state': 'Florida',
... 'shortname': 'FL',
... 'info': {
... 'governor': 'Rick Scott'
... },
... 'counties': [{'name': 'Dade', 'population': 12345},
... {'name': 'Broward', 'population': 40000},
... {'name': 'Palm Beach', 'population': 60000}]},
... {'state': 'Ohio',
... 'shortname': 'OH',
... 'info': {
... 'governor': 'John Kasich'
... },
... 'counties': [{'name': 'Summit', 'population': 1234},
... {'name': 'Cuyahoga', 'population': 1337}]}]
>>> from pandas.io.json import json_normalize
>>> result = json_normalize(data, 'counties', ['state', 'shortname',
... ['info', 'governor']])
>>> result
 name population info.governor state shortname
0 Dade 12345 Rick Scott Florida FL
1 Broward 40000 Rick Scott Florida FL
2 Palm Beach 60000 Rick Scott Florida FL
3 Summit 1234 John Kasich Ohio OH
4 Cuyahoga 1337 John Kasich Ohio OH

For the above dataframe you can easily get the sum of the population for all entries with shortname=='FL' as follows:

sum_of_fl_population = result[result['shortname']=='FL'].population.sum()
Out[11]: 112345

Have a look at this link to get an introduction how to handle the pandas dataframes. It's probably the best way to solve your problem. http://pandas.pydata.org/pandas-docs/stable/10min.html

• Daniel Yang: Thanks a lot! This is very useful. I'm going to give it a try first.
• Nickpick: Great, please accept the answer once you think it's what you need

Answer 2

I think a good way of storing this data could be to have a Computer Object, which could have member variables such Motherboard, Monitor etc.

You could then have a single dictionary which contained a unique id mapped to a Computer object. So you could have something along the lines of

 computers.get('12345').getMotherBoard().getMake()
 computers.get('45678').getMonitor().getScreenResolution()
 computers.get('54342').getRam().getPrice()

• Daniel Yang: how about going for computer class, and then monitor sub-class and so on? is that going to work fine?
• chatton: with inheritance, there is an is-a relationship. So if class B inherits from class A, you should be able to say B is an A. So eg. take a parent class Dog, a sensible subclass could be a breed of dog, say a Lab. You can say a Lab is a Dog. In this case, if Monitor inherits from Computer, we should be able to say that a Monitory is a Computer. Here we're more thinking about a has a relationship. so a Computer has-a monitor, and has-a motherboard. So here I don't think it makes sense for Monitor to inherit from Computer.

Answer 3

Go for big data.

Put ElasticSearch, MongoDB, CouchDB or many others (I guess) in your stack and store every product as a document, flattening your base.

Create an index per type of document (one for computer screens, one for motherboards, ...), basically one index per type of object that share the same properties and you are ready to go.

For elasticSearch for instance about parenting : https://www.elastic.co/guide/en/elasticsearch/guide/2.x/parent-child.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/parent-child-mapping.html

I believe this is faster than SQL, and maybe easier to use. There is one downside though, not many frameworks allow you to use such "new" databases. Though for the better known there are mapper with the ORMs

• Daniel Yang: Actually I think go for all those DBs is kind of a overkill for me. I only have under 50 pricing info (which need to be accessed frequently of course). I'm think about get this problem solved within python