Store and output hard-coded relationships among hosts

Question

The following code has begun to smell, but I have not yet decided with what to replace it, other than, obviously, a database.

I made a very unsatisfactory workaround for my attempt to make relationships among tuple members as I assigned them.

How can I modify a piece of code like this that contains some hostnames or a few secrets, and hierarchical relationships among them, to better follow good coding practices for Python 3.5+?

#!/usr/bin/env python3.5
'''
This file contains basic data about hosts for clustered ReDiS, as deployed.
Example Usage:
>>> import socket
>>> import redislist
>>> import redislist.b01red
>>> for redisHost in b01red:
... print (redisHost.hostname)
... print (socket.getaddrinfo(redisHost.hostname, redisHost.port))
...
'''
from collections import namedtuple
RedisContainer = namedtuple('RedisContainer', 'hostname addr port slaveof')
b01red = ['', '', '', ''] # These values are placeholders to prevent
 # premature assignment errors.
b01red = [RedisContainer(hostname='b01rd00009',addr='10.253.128.1',port='7381', slaveof='master'), \
 # Slaves are b01rd00000, b01grd00009, b01grd00000.
 RedisContainer(hostname='b01rd00000',addr='10.253.128.2',port='7381', slaveof=b01red[0] ),
 # 10.253.128.2 (b01rd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01rd00001',addr='10.253.128.3',port='7381', slaveof='master'), \
 # Slaves are b01rd00002, b01grd00001, b01grd00002.
 RedisContainer(hostname='b01rd00002',addr='10.253.128.4',port='7381', slaveof=b01red[2] )]
 # 10.253.128.4 (b01rd00002) slaveof 10.253.128.3 (b01rd00001).
# Reassign to account for new values for relative variables.
b01red[0] = RedisContainer(hostname='b01rd00009',addr='10.253.128.1',port='7381', slaveof='master')
 # Slaves are b01rd00000, b01grd00009, b01grd00000.
b01red[1] = RedisContainer(hostname='b01rd00000',addr='10.253.128.2',port='7381', slaveof=b01red[0])
 # 10.253.128.2 (b01rd00000) slaveof 10.253.128.1 (b01rd00009).
b01red[2] = RedisContainer(hostname='b01rd00001',addr='10.253.128.3',port='7381', slaveof='master')
 # Slaves are b01rd00002, b01grd00001, b01grd00002.
b01red[3] = RedisContainer(hostname='b01rd00002',addr='10.253.128.4',port='7381', slaveof=b01red[2])
 # 10.253.128.4 (b01rd00002) slaveof 10.253.128.3 (b01rd00001).
b01gred = ['', '', '', '', '', '', ''] # These values are placeholders to prevent
 # premature assignment errors.
b01gred = [RedisContainer(hostname='b01grd00009',addr='10.150.55.1',port='7381', slaveof=b01red[0]), \
 # 10.150.55.1 (b01grd00009) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00000',addr='10.150.55.2',port='7381', slaveof=b01red[0]), \
 # 10.150.55.2 (b01grd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00001',addr='10.150.55.3',port='7381', slaveof=b01red[2]), \
 # 10.150.55.3 (b01grd00001) slaveof 10.253.128.3 (b01rd00001).
 RedisContainer(hostname='b01grd00002',addr='10.150.55.4',port='7381', slaveof=b01red[2])]
 # 10.150.55.4 (b01grd00002) slaveof 10.253.128.3 (b01rd00001).
b01gred[0] = RedisContainer(hostname='b01grd00009',addr='10.150.55.1',port='7381', slaveof=b01red[0])
 # 10.150.55.1 (b01grd00009) slaveof 10.253.128.1 (b01rd00009).
b01gred[1] = RedisContainer(hostname='b01grd00000',addr='10.150.55.2',port='7381', slaveof=b01red[0])
 # 10.150.55.2 (b01grd00000) slaveof 10.253.128.1 (b01rd00009).
b01gred[2] = RedisContainer(hostname='b01grd00001',addr='10.150.55.3',port='7381', slaveof=b01red[2])
 # 10.150.55.3 (b01grd00001) slaveof 10.253.128.3 (b01rd00001).
b01gred[3] = RedisContainer(hostname='b01grd00002',addr='10.150.55.4',port='7381', slaveof=b01red[2])
 # 10.150.55.4 (b01grd00002) slaveof 10.253.128.3 (b01rd00001).
b03red = ['', '', '', '', '', '', '']
b03red = [RedisContainer(hostname='b03rd00009',addr='10.253.128.1',port='7381', slaveof=''), \
 # Slaves are b03rd00010, gb03rd00009, gb03rd00010
 RedisContainer(hostname='b03rd00010',addr='10.253.128.2',port='7381', slaveof=''), \
 # 10.253.128.2 (b03rd00010) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='b03rd00011',addr='10.253.128.51',port='7381', slaveof=''), \
 # Slaves are b03rd00012, gb03rd00011, gb03rd00012
 RedisContainer(hostname='b03rd00012',addr='10.253.128.84',port='7381', slaveof=''), \
 # 10.253.128.84 (b03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='b03rd00013',addr='10.253.128.17',port='7381', slaveof=''), \
 # Slaves are b03rd00014, gb03rd00013, gb03rd00014
 RedisContainer(hostname='b03rd00014',addr='10.253.128.222',port='7381', slaveof=b03red[4])]
b03red[0] = RedisContainer(hostname='b03rd00009',addr='10.253.128.1',port='7381', slaveof='master')
b03red[1] = RedisContainer(hostname='b03rd00010',addr='10.253.128.2',port='7381', slaveof=b03red[0])
b03red[2] = RedisContainer(hostname='b03rd00011',addr='10.253.128.51',port='7381', slaveof='master')
b03red[3] = RedisContainer(hostname='b03rd00012',addr='10.253.128.84',port='7381', slaveof=b03red[2])
b03red[4] = RedisContainer(hostname='b03rd00013',addr='10.253.128.17',port='7381', slaveof='master')
b03red[5] = RedisContainer(hostname='b03rd00014',addr='10.253.128.222',port='7381', slaveof=b03red[4])
gb03red = ['','','','','','','']
gb03red = [RedisContainer(hostname='gb03rd00009',addr='10.150.55.1',port='7381', slaveof=b03red[0]), \
 # 10.150.55.1 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00010',addr='10.150.55.2',port='7381', slaveof=b03red[0]), \
 # 10.150.55.2 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00011',addr='10.150.55.3',port='7381', slaveof=b03red[2]), \
 # 10.150.55.3 (gb03rd00011) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00012',addr='10.150.55.4',port='7381', slaveof=b03red[2]), \
 # 10.150.55.4 (gb03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00013',addr='10.150.55.5',port='7381', slaveof=b03red[4]), \
 # 10.150.55.5 (gb03rd00013) slaveof 10.253.128.17 (b03rd00013)
 RedisContainer(hostname='gb03rd00014',addr='10.150.55.6',port='7381', slaveof=b03red[4]) ]
 # 10.150.55.6 (gb03rd00014) slaveof 10.253.128.17 (b03rd00013)
# Reassign to account for new values.
gb03red[0] = RedisContainer(hostname='gb03rd00009',addr='10.150.55.1',port='7381', slaveof=b03red[0])
gb03red[1] = RedisContainer(hostname='gb03rd00010',addr='10.150.55.2',port='7381', slaveof=b03red[0])
gb03red[2] = RedisContainer(hostname='gb03rd00011',addr='10.150.55.3',port='7381', slaveof=b03red[2])
gb03red[3] = RedisContainer(hostname='gb03rd00012',addr='10.150.55.4',port='7381', slaveof=b03red[2])
gb03red[4] = RedisContainer(hostname='gb03rd00013',addr='10.150.55.5',port='7381', slaveof=b03red[4])
gb03red[5] = RedisContainer(hostname='gb03rd00014',addr='10.150.55.6',port='7381', slaveof=b03red[4])
clusters = [b01red, b01gred, b03red, gb03red]
def list_hosts():
 for a_cluster in clusters:
 for redisHost in a_cluster:
 if type(redisHost.slaveof) == RedisContainer:
 print (redisHost.hostname, redisHost.addr, "Redis Master IP Address = ", redisHost.slaveof.addr)
 else:
 print (redisHost.hostname, redisHost.addr, "Redis Master IP Address = ", redisHost.slaveof)

The following shows some of the output when this runs: \

$ python3.5
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 5 2015, 21:12:44)
Type "help", "copyright", "credits" or "license" for more information.
>>> import redislist
>>> redislist.list_hosts()
b01rd00009 10.253.128.1 Redis Master IP Address = master
b01rd00000 10.253.128.2 Redis Master IP Address = 10.253.128.1
b01rd00001 10.253.128.3 Redis Master IP Address = master
b01rd00002 10.253.128.4 Redis Master IP Address = 10.253.128.3
b01grd00009 10.150.55.1 Redis Master IP Address = 10.253.128.1
b01grd00000 10.150.55.2 Redis Master IP Address = 10.253.128.1
b01grd00001 10.150.55.3 Redis Master IP Address = 10.253.128.3
b01grd00002 10.150.55.4 Redis Master IP Address = 10.253.128.3
b03rd00009 10.253.128.1 Redis Master IP Address = master
b03rd00010 10.253.128.2 Redis Master IP Address = 10.253.128.1
b03rd00011 10.253.128.51 Redis Master IP Address = master
b03rd00012 10.253.128.84 Redis Master IP Address = 10.253.128.51
b03rd00013 10.253.128.17 Redis Master IP Address = master
b03rd00014 10.253.128.222 Redis Master IP Address = 10.253.128.17
gb03rd00009 10.150.55.1 Redis Master IP Address = 10.253.128.1
gb03rd00010 10.150.55.2 Redis Master IP Address = 10.253.128.1
gb03rd00011 10.150.55.3 Redis Master IP Address = 10.253.128.51
gb03rd00012 10.150.55.4 Redis Master IP Address = 10.253.128.51
gb03rd00013 10.150.55.5 Redis Master IP Address = 10.253.128.17
gb03rd00014 10.150.55.6 Redis Master IP Address = 10.253.128.17

I don't have much experience with named tuples, here, and I'm especially interested in better ways to take advantage of them. Also, let me know if this is annoyingly long, and I will reduce its size.

Currently, the best alternative to my oddly-named tuples seems to be the "Ordered Dictionary", described in this page.

Summary:

If I really must use a .py to "store" some data (like hostnames or secrets) where some parts depend on other parts, as exemplified in the above code, what best practices for coding in Python 3.5 will improve this code?

Answer 1

You’re right, defining the same list three time is not a good idea.

For starter, you could use a list as the slaveof parameter and separate the definitions of machines from the topology:

#!/usr/bin/env python3.5
'''
This file contains basic data about hosts for clustered ReDiS, as deployed.
Example Usage:
>>> import socket
>>> import redislist
>>> import redislist.b01red
>>> for redisHost in b01red:
... print (redisHost.hostname)
... print (socket.getaddrinfo(redisHost.hostname, redisHost.port))
...
'''
from collections import namedtuple
RedisContainer = namedtuple('RedisContainer', 'hostname addr port slaveof')
b01red = [
 RedisContainer(hostname='b01rd00009',addr='10.253.128.1',port='7381', slaveof=[]),
 # Slaves are b01rd00000, b01grd00009, b01grd00000.
 RedisContainer(hostname='b01rd00000',addr='10.253.128.2',port='7381', slaveof=[]),
 # 10.253.128.2 (b01rd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01rd00001',addr='10.253.128.3',port='7381', slaveof=[]),
 # Slaves are b01rd00002, b01grd00001, b01grd00002.
 RedisContainer(hostname='b01rd00002',addr='10.253.128.4',port='7381', slaveof=[]),
 # 10.253.128.4 (b01rd00002) slaveof 10.253.128.3 (b01rd00001).
]
b01gred = [
 RedisContainer(hostname='b01grd00009',addr='10.150.55.1',port='7381', slaveof=[]),
 # 10.150.55.1 (b01grd00009) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00000',addr='10.150.55.2',port='7381', slaveof=[]),
 # 10.150.55.2 (b01grd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00001',addr='10.150.55.3',port='7381', slaveof=[]),
 # 10.150.55.3 (b01grd00001) slaveof 10.253.128.3 (b01rd00001).
 RedisContainer(hostname='b01grd00002',addr='10.150.55.4',port='7381', slaveof=[]),
 # 10.150.55.4 (b01grd00002) slaveof 10.253.128.3 (b01rd00001).
]
b03red = [
 RedisContainer(hostname='b03rd00009',addr='10.253.128.1',port='7381', slaveof=[]),
 # Slaves are b03rd00010, gb03rd00009, gb03rd00010
 RedisContainer(hostname='b03rd00010',addr='10.253.128.2',port='7381', slaveof=[]),
 # 10.253.128.2 (b03rd00010) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='b03rd00011',addr='10.253.128.51',port='7381', slaveof=[]),
 # Slaves are b03rd00012, gb03rd00011, gb03rd00012
 RedisContainer(hostname='b03rd00012',addr='10.253.128.84',port='7381', slaveof=[]),
 # 10.253.128.84 (b03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='b03rd00013',addr='10.253.128.17',port='7381', slaveof=[]),
 # Slaves are b03rd00014, gb03rd00013, gb03rd00014
 RedisContainer(hostname='b03rd00014',addr='10.253.128.222',port='7381', slaveof=[]),
]
gb03red = [
 RedisContainer(hostname='gb03rd00009',addr='10.150.55.1',port='7381', slaveof=[]),
 # 10.150.55.1 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00010',addr='10.150.55.2',port='7381', slaveof=[]),
 # 10.150.55.2 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00011',addr='10.150.55.3',port='7381', slaveof=[]),
 # 10.150.55.3 (gb03rd00011) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00012',addr='10.150.55.4',port='7381', slaveof=[]),
 # 10.150.55.4 (gb03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00013',addr='10.150.55.5',port='7381', slaveof=[]),
 # 10.150.55.5 (gb03rd00013) slaveof 10.253.128.17 (b03rd00013)
 RedisContainer(hostname='gb03rd00014',addr='10.150.55.6',port='7381', slaveof=[]),
 # 10.150.55.6 (gb03rd00014) slaveof 10.253.128.17 (b03rd00013)
]
# Topology
def _set_slave_of(master, slave):
 slave.slaveof.append(master)
_set_slave_of(b01red[0], b01red[1])
_set_slave_of(b01red[2], b01red[3])
_set_slave_of(b01red[0], b01gred[0])
_set_slave_of(b01red[0], b01gred[1])
_set_slave_of(b01red[2], b01gred[2])
_set_slave_of(b01red[2], b01gred[3])
_set_slave_of(b03red[0], b03red[1])
_set_slave_of(b03red[2], b03red[3])
_set_slave_of(b03red[4], b03red[5])
_set_slave_of(b03red[0], gb03red[0])
_set_slave_of(b03red[0], gb03red[1])
_set_slave_of(b03red[2], gb03red[2])
_set_slave_of(b03red[2], gb03red[3])
_set_slave_of(b03red[4], gb03red[4])
_set_slave_of(b03red[4], gb03red[5])
clusters = [b01red, b01gred, b03red, gb03red]
def list_hosts():
 for a_cluster in clusters:
 for redisHost in a_cluster:
 if not redisHost.slaveof:
 print (redisHost.hostname, redisHost.addr, "Redis Master Machine")
 else:
 for master in redisHost.slaveof:
 print (redisHost.hostname, redisHost.addr, "Redis Master IP Address = ", master.addr)

But it adds the boilerplate slaveof=[] for each object, plus the handling of this parameter in the listing function is less ideal.

So, since you need to modify dynamically your objects, why use a namedtuple instead of using your own class? If you really need to be able to access the attributes as in a tuple, you can implement the __iter__ method so you can do my_tuple = tuple(my_redis_object):

#!/usr/bin/env python3.5
'''
This file contains basic data about hosts for clustered ReDiS, as deployed.
Example Usage:
>>> import socket
>>> import redislist
>>> import redislist.b01red
>>> for redisHost in b01red:
... print (redisHost.hostname)
... print (socket.getaddrinfo(redisHost.hostname, redisHost.port))
...
'''
class RedisContainer:
 def __init__(self, hostname, addr, port='7381', master=None):
 self.hostname = hostname
 self.addr = addr
 self.port = port
 self.slaveof = master
 def __iter__(self):
 yield self.hostname
 yield self.addr
 yield self.port
 yield self.slaveof
b01red = [
 RedisContainer(hostname='b01rd00009',addr='10.253.128.1'),
 # Slaves are b01rd00000, b01grd00009, b01grd00000.
 RedisContainer(hostname='b01rd00000',addr='10.253.128.2'),
 # 10.253.128.2 (b01rd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01rd00001',addr='10.253.128.3'),
 # Slaves are b01rd00002, b01grd00001, b01grd00002.
 RedisContainer(hostname='b01rd00002',addr='10.253.128.4'),
 # 10.253.128.4 (b01rd00002) slaveof 10.253.128.3 (b01rd00001).
]
b01gred = [
 RedisContainer(hostname='b01grd00009',addr='10.150.55.1'),
 # 10.150.55.1 (b01grd00009) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00000',addr='10.150.55.2'),
 # 10.150.55.2 (b01grd00000) slaveof 10.253.128.1 (b01rd00009).
 RedisContainer(hostname='b01grd00001',addr='10.150.55.3'),
 # 10.150.55.3 (b01grd00001) slaveof 10.253.128.3 (b01rd00001).
 RedisContainer(hostname='b01grd00002',addr='10.150.55.4'),
 # 10.150.55.4 (b01grd00002) slaveof 10.253.128.3 (b01rd00001).
]
b03red = [
 RedisContainer(hostname='b03rd00009',addr='10.253.128.1'),
 # Slaves are b03rd00010, gb03rd00009, gb03rd00010
 RedisContainer(hostname='b03rd00010',addr='10.253.128.2'),
 # 10.253.128.2 (b03rd00010) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='b03rd00011',addr='10.253.128.51'),
 # Slaves are b03rd00012, gb03rd00011, gb03rd00012
 RedisContainer(hostname='b03rd00012',addr='10.253.128.84'),
 # 10.253.128.84 (b03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='b03rd00013',addr='10.253.128.17'),
 # Slaves are b03rd00014, gb03rd00013, gb03rd00014
 RedisContainer(hostname='b03rd00014',addr='10.253.128.222'),
]
gb03red = [
 RedisContainer(hostname='gb03rd00009',addr='10.150.55.1'),
 # 10.150.55.1 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00010',addr='10.150.55.2'),
 # 10.150.55.2 (gb03rd00009) slaveof 10.253.128.1 (b03rd00009).
 RedisContainer(hostname='gb03rd00011',addr='10.150.55.3'),
 # 10.150.55.3 (gb03rd00011) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00012',addr='10.150.55.4'),
 # 10.150.55.4 (gb03rd00012) slaveof 10.253.128.51 (b03rd00011).
 RedisContainer(hostname='gb03rd00013',addr='10.150.55.5'),
 # 10.150.55.5 (gb03rd00013) slaveof 10.253.128.17 (b03rd00013)
 RedisContainer(hostname='gb03rd00014',addr='10.150.55.6'),
 # 10.150.55.6 (gb03rd00014) slaveof 10.253.128.17 (b03rd00013)
]
# Topology
b01red[1].slaveof = b01red[0]
b01red[3].slaveof = b01red[2]
b01gred[0].slaveof = b01red[0]
b01gred[1].slaveof = b01red[0]
b01gred[2].slaveof = b01red[2]
b01gred[3].slaveof = b01red[2]
b03red[1].slaveof = b03red[0]
b03red[3].slaveof = b03red[2]
b03red[5].slaveof = b03red[4]
gb03red[0].slaveof = b03red[0]
gb03red[1].slaveof = b03red[0]
gb03red[2].slaveof = b03red[2]
gb03red[3].slaveof = b03red[2]
gb03red[4].slaveof = b03red[4]
gb03red[5].slaveof = b03red[4]
clusters = [b01red, b01gred, b03red, gb03red]
def list_hosts():
 for a_cluster in clusters:
 for redisHost in a_cluster:
 if not redisHost.slaveof:
 print (redisHost.hostname, redisHost.addr, "Redis Master Machine")
 else:
 print (redisHost.hostname, redisHost.addr, "Redis Master IP Address = ", redisHost.slaveof.addr)

This also let you define default values if they change very few times (such as the port).

---

As far as style is concerned:

• You should use spaces after your comas;
• you should not use spaces between a function name and its parameters list;
• you don't need \ at the end of a line when defining lists, the line continuation is implied until ] is reached;
• Python recommends to use snake_case instead of camelCase for variables and functions names; TitleCase for class names is good.

• Nathan Basanese: Any changing of replication relationships happens between runs. This just shows the initial, known good set of ReDiS cluster masters and slaves. Because of this, I prefer the named tuple solution. I want to avoid classes if I can, since we, as humans, are bad at tracking "state changes". Makes sense to avoid classes if possible, right? As far as the actual, current ReDiS cluster masters and slaves, we use github.com/Grokzen/redis-py-cluster to ask the ReDiS cluster itself, directly. What do you think?
• Nathan Basanese: This may not relate to the actual code, but based on what I've shown here, if I did end up making a sort of "Finite State Machine" with values for sharding, slave-priority, failover, authentication, and more, could it go in this module?
• Nathan Basanese: As it happens, because I intend to use this to tell which slave goes to which master "at a glance", I may have to replace the first answer with something like a dictionary, with hostname as the key. That way I can use a hostname string as the value for slaveof, and still have it work as a key for accessing that master. Lists of named tuples just don't seem like the right solution, here, after all, I guess.
• 301_Moved_Permanently: @NathanBasanese Just because you use classes to store state doesn't mean that you have to change said state. And since you need slight modifications of that state right after creating the objects, then a class is better suited than a tuple. I considered using a dictionary at some point, but not knowing your use case didn't help. So I stayed close of what you had.
• 301_Moved_Permanently: @NathanBasanese Yep, that's exactly what the OrderedDict is used for. There is no access by index, but you are guaranteed to iterate over the elements in the same order you inserted them.