JSON packet transmitter

Question

I'm not familiar with how data transmission works, but I gave my best shot trying to code a server and client-side JSON packet transmitter via Python. Essentially client.py generates random JSON packets and sends them to server.py which binds a port using a socket and listens for a JSON packet, which then stores the JSON packet as a file to prove the transmission was successful.

One problem I am aware of is that within my client.py, the sock class is kept being created and closed through each cycle. Though, if I write it like

 while True:
 json_packet = generate_json_packet()
 send_json_packet(sock, json_packet)
 time.sleep(1)

Python emits the following: ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine after the first successful received packet.

Server.py

# Server-Side that receives json packets from client over the network using port 5000. Then it saves the json packet to a file with the filename of current time.
import socket
import json
import random
import string
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 5000)
print('starting up on {} port {}'.format(*server_address))
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
# Function to receive json packets from client
def receive_json(conn):
 data = conn.recv(1024)
 data = data.decode('utf-8')
 data = json.loads(data)
 return data
# Saves json packets to file and name it with id
def save_json(data):
 filename = ''.join(random.choice(
 string.digits) for _ in range(3))
 filename = filename + '.json'
 with open(filename, 'w') as f:
 json.dump(data, f)
if __name__ == '__main__':
 while True:
 # Wait for a connection
 print('waiting for a connection')
 connection, client_address = sock.accept()
 try:
 print('connection from', client_address)
 # Receive the data in json format
 data = receive_json(connection)
 print('received {!r}'.format(data))
 # Save the json packet to a file
 filename = save_json(data)
 print('saved to', filename)
 finally:
 # Clean up the connection
 connection.close()

# Sends random json packets to server over port 5000
import socket
import json
import random
import time
def generate_json_packet():
 # Generate random json packet with hashed data bits
 return {
 "id": random.randint(1, 100),
 "timestamp": time.time(),
 "data": hash(str(random.randint(1, 100)))
 }
# Send json packet to server
def send_json_packet(sock, json_packet):
 sock.send(json.dumps(json_packet).encode())
ip = "127.0.0.1"
port = "5000"
if __name__ == "__main__":
 while True:
 # Create socket
 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 # Connect to server
 sock.connect((ip, int(port)))
 # Generate random json packet
 json_packet = generate_json_packet()
 # Send json packet to server
 send_json_packet(sock, json_packet)
 time.sleep(1)
 sock.close()

• Toby Speight: Your edit invalidated an answer, which breaks the question-and-answer nature of this site. See What should I do when someone answers my question?.
• Peter Mortensen: JSON, not json or Json.

Answer 1

Your implementation is fatally flawed, and this is mostly not your fault: sockets are non-intuitive. First, when you say

receives json packets

that's not in the slightest what you're doing. You're sending JSON messages that happen to usually fit in one packet, and this coincidence is unreliable and will break under many, many common conditions. This trips up nearly every beginner socket programmer. You need to almost forget about packets entirely, assume that your message may be segmented into an arbitrary number of packets returned to the listener in a series of arbitrarily-sized byte buffers. This in turn means that you cannot rely on packet boundaries to be message boundaries. You need to define your own message boundary. One easy message boundary is a newline, and Python makes it easy to use a StreamRequestHandler that listens for a newline regardless of how many packets were received. So, to stabilise your implementation,

• Call them JSON messages, not JSON packets
• Use sendall, not send
• Don't use socket at all on the server side; instead use StreamRequestHandler and TCPServer
• Use rfile.readline() to guarantee a stable message boundary

Other assorted points:

• file with the filename of current time is a lie
• The server should not need to go through a JSON round-trip, and can just write the bytes out to a file - if you don't care about validating the data
• You should be using with context management to guarantee socket closure

Suggested

Server:

"""
Server-Side that receives json packets from client over the network using port
5000. Then it saves the json packet to a file with the filename of current time.
"""
import random
import string
from socketserver import StreamRequestHandler, TCPServer
def save_json(data: bytes) -> None:
 """Saves json packets to file and name it with id"""
 filename = ''.join(
 random.choice(string.digits)
 for _ in range(3)
 ) + '.json'
 with open(filename, 'wb') as f:
 f.write(data)
 print('saved to', filename)
class DumpHandler(StreamRequestHandler):
 def handle(self) -> None:
 """receive json packets from client"""
 print('connection from {}:{}'.format(*self.client_address))
 try:
 while True:
 data = self.rfile.readline()
 if not data:
 break
 print('received', data.decode().rstrip())
 save_json(data)
 finally:
 print('disconnected from {}:{}'.format(*self.client_address))
def main() -> None:
 server_address = ('localhost', 5000)
 print('starting up on {}:{}'.format(*server_address))
 with TCPServer(server_address, DumpHandler) as server:
 print('waiting for a connection')
 server.serve_forever()
if __name__ == '__main__':
 try:
 main()
 except KeyboardInterrupt:
 pass

Client:

"""Sends random json packets to server over port 5000"""
import socket
import json
from random import randint
from time import time, sleep
from typing import Dict, Any
IP = "127.0.0.1"
PORT = 5000
def generate_json_message() -> Dict[str, Any]:
 # """Generate random json packet with hashed data bits"""
 return {
 "id": randint(1, 100),
 "timestamp": time(),
 "data": hash(str(randint(1, 100)))
 }
def send_json_message(
 sock: socket.socket,
 json_message: Dict[str, Any],
) -> None:
 """Send json packet to server"""
 message = (json.dumps(json_message) + '\n').encode()
 sock.sendall(message)
 print(f'{len(message)} bytes sent')
def main() -> None:
 with socket.socket() as sock:
 sock.connect((IP, PORT))
 while True:
 json_message = generate_json_message()
 send_json_message(sock, json_message)
 sleep(1)
if __name__ == "__main__":
 try:
 main()
 except KeyboardInterrupt:
 pass

• Miguel Alorda: Why should the program ignore keyboard interruptions?
• Reinderien: @MiguelAlorda Because that's the typical way to cancel a program - Ctrl+C. There's no value in showing the stack trace.
• Joe: Thank you @Reinderien! I do have some convention question that I want to ask you: why have you enunciated -> Dict[str, Any] or -> None for specific functions? I also see you skipping parsing raw data to json and directly saved it into a file. Does this work on every filetype?
• Reinderien: @Joe Those are type hints compliant with python.org/dev/peps/pep-0484. They are very useful to tell yourself and other programmers the types of your variables, method parameters and returns; and to verify the correctness of your program using tools like mypy.
• Reinderien: @Joe Yes, skipping a parse & re-serialize is possible for all data formats so long as they can enter and exit in bytes format and so long as you don't care to validate the data.