Quartz 5

05 Socket Programming

15 min read

Socket Programming

Socket Programming

Level Google Classroom Demo: file:///C:/Users/Andrea/Downloads/Socket%20Demo.html

Sockets → used to send data from 1 program to the other and vice versa, when 2 Python programs are running at the same time

  • Bidirectional: can carry data (rep. by bytes) in both directions

Internet socket

  • Can delivers data between any 2 programs, even programs running on diff. computers, as long as the 2 computers can access each other over the network
  • The 2 programs set up a socket connection
  • Data transmitted through an Internet socket may pass through multiple devices before reaching its destination
  • Any of these devices can steal or modify the data ⇒ encrypt data first
  • Uses the same TCP / IP suite that is used to transmit data over the Internet

Networks can become congested ⇒ data sent over the Internet sockets may not be transmitted instantaneously (e.g. program may receive only first half of the message, the rest arrive later)

  • To avoid working with incomplete data, define a protocol so the start and end of messages can be detected unambiguously

IP addresses and ports

  • Each end of a socket is associated with a running program and is uniquely identified by a combined IP address and port number
  • IP address: identifies which device that end of the socket is attached to
  • IPv4 addresses: have 32 bits; usually presented as 4 denary numbers separated by dots
  • Each denary can range from 0-255 (inclusive) and corresponds to 1 byte (8 bits) of the IP address
  • 127.0.0.1: local computer
  • 0.0.0.0: all IP addresses for local compute
  • IPv6 addresses
  • Have 128 bits (usually 8 groups of 4 hexadecimal digits separated by colons; leading zeros & up to 1 consecutive sequence of zero-only groups may be omitted)
  • Port number: identifies which program on that device is using the socket
  • Distinguishes between attached sockets
  • Keeps track of which port numbers (0-65, 535) are still available for use by new sockets
  • Only first 1024 port numbers are reserved for specific programs
  • [cmd prompt] netstat -n ⇒ list sockets that are currently open on your computer (IP address & port number for each of its ends)
  • [cmd prompt] netstat -no ⇒ reveal process ID (PID) associated with each socket
    Then open Task Manager and match each PID to the name of a running program

Creating a socket connection

  • Requires 2 programs – one server and one client
  • Server’s IP address and port number for accepting connections must be known ahead of time by the client
  • Server creates a passive socket, binds it to the pre-chosen port number and listens for an incoming connection ⇒ is not connected, merely waits for an incoming connection
  • Server’s IP address and port number is known to the client
  • Client then initiates a connection request using server’s IP address and port number
  • If no server is listening on the chosen port, the connection will be refused
  • If connection request reaches an IP address and port number that a server is listening on, the server accepts and creates a new socket for the requesting client using a dynamically assigned port number (client’s port number)
  • The passive socket goes back to listening for new connections;
    Client and server can now exchange data using the newly-created socket
  • The newly-created socket is symmetrical: data sent on one end is received on the other end and vice versa ⇒ can transfer data in both directions (symmetric info sharing)
  • Once a socket is established, it can send data both from client to server and from server to client

Unicode and encodings

  • Sockets can only send and receive data in the form of raw bytes ⇒ must encode data into a sequence of 8-bit characters
  • ‘string’.encode() / b’string**’.encode()** and bytes.decode()
  • A Python str is actually treated as a sequence of numbers ⇒ Unicode code points
  • UTF-8: represent code points using bytes in a space-efficient & consistent way
  • To enter a sequence of bytes directly in code, use a bytes literal that starts with b, then a sequence of bytes (in the form of ASCII characters), enclosed using ‘ or ‘’
  • Note: most escape codes that work for str also work for byte literals
b’Raw bytes’.decode().strip() # convert bytes to str using UTF-8 encoding
‘Unicode str’.encode().strip() # convert str to bytes using UTF-8 encoding
# .strip() to remove /n when sending bytes
  • 中 can be written as the str literal ‘\u4e2d’ in Python
    ⇒ Uses an escape code that produces a character by specifying its Unicode code point
  • len(‘\u4e2d’) ⇒ 1
  • len(‘\u4e2d’.encode()) ⇒ 3
  • The Unicode code point for 中 is represented using 3 bytes in UTF-8

Using the socket module

  • Import socket module & create socket objects ⇒ create and manage sockets in Python
MethodsDescription
bind((host, port))Binds socket object to the given address tuple (host, port)
Host = server’s IPv4 address
Port = server’s port number (pre-chosen)
listen()Enables socket to listen for incoming connections from clients
accept()Waits for an incoming connection, and returns a tuple containing a new socket object for the connection and an address tuple (host, port)
Host = client’s IPv4 address
Port = client’s port number (dynamically assigned)
connect((host, port))Initiates a connection to the given address tuple (host, port)
Host = server’s IPv4 address
Port = server’s port number (pre-chosen)
recv(max_bytes)Receives and returns up to the given number of bytes from the socket
sendall(bytes)Sends the given bytes to the socket

E.g. basic_server.py: basic server program that listens for a client on port 12345, accepts a connection request, sends b’Hello from server\n’ to client through socket, then closes socket

import socket
my_socket = socket.socket()
my_socket.bind((‘127.0.0.1’, 12345)) # or use any large port number
my_socket.listen()
new_socket, addr = my_socket.accept() # returns a new socket and nested address tuple
print(‘Connected to: ’ + str(addr))
new_socket.sendall(b’Hello from server\n’) # new_socket is used to send and receive data
new_socket.close()
my_socket.close()

Server should appear stuck shortly after it is started, because socket.accept() method is blocking (waiting for some event to occur) the program and prevents it from continuing until a connection request is received

To create a client that can connect to this server, start a 2nd copy of Python:

Move any windows from the 1st copy of Python to one side so the 2 copies of Python are clearly separated. Create a new Python program using the 2nd copy of Python.

E.g. basic_client.py: ask for server’s IP address and port number, request for a connection, receive and print at most 1024 bytes from server, then close socket

import socket
my_socket = socket.socket()
address = input(“Enter IPv4 address of server: ”)
port = int(input(“Enter port number of server: ”))
my_socket.connect((address, port))
print(my_socket.recv(1024)) # 1024 or any small power of 2 (i.e. recv up to 1024 bytes)
my_socket.close()
  • Run using 2nd copy of Python & ensure server you started previously is still running
  • Else close client, restart server, and reopen client using 2nd shell window
  • Each program should affect a different shell window when run (e.g. pressing F5)
  • Client should prompt you for the address & port number of server
  • Use the special IPv4 address 127.0.0.1 (local machine), enter 12345 as port number
  • Client should successfully connect to server and print out the bytes received
  • Server program should become unstuck and end normally

Flaw: when basic server program is used to send longer sequences of bytes, only part of the data may be successfully transmitted even if we inc. max no. of bytes for socket.recv()

  • E.g. If sequence of bytes sent is long enough that it needs to be sent as multiple packets
  • Simulate this by breaking sequence into 2 pieces & calling socket.sendall() twice
  • To simulate a busy network that may delay transport of the 2nd packet, import time module and call time.sleep() before sending 2nd piece
# basic_server_split.py
import socket
import time
my_socket = socket.socket()
my_socket.bind((‘127.0.0.1’, 12345))
my_socket.listen()
new_socket, addr = my_socket.accept()
new_socket.sendall(b’Hello fr’)
time.sleep(0.1)
new_socket.sendall(b’om server\n’)
new_socket.close()
my_socket.close()
  • Run this server, then run client ⇒ both programs run simultaneously on same machine
  • Client should receive only 1st piece of data
  • If client closed socket, server may produce error when trying to send 2nd piece of data
  • To be certain that any received data is complete (all bytes sent over at one go), agree beforehand on a protocol / set of rules for how communication should take place
  • E.g. agree beforehand that any data transmitted will always end with \n (newline character) and that the data itself will never contain \n ⇒ detect end of transmission / message easily by just searching for \n
  • New client calls socket.recv() continuously and appends the received bytes to a variable data until \n character encountered
# basic_client_protocol.py: successfully receives and prints all data sent by server up to and including the \n character
import socket
my_socket = socket.socket()
address = input(“Enter IPv4 address of server: ”)
port = int(input(“Enter port number of server: ”))
my_socket.connect((address, port))
data = b’’
while b’\n’ not in data:
data += my_socket.recv(1024)
print(data)
my_socket.close()
  • Server program exists immediately after it finished working with a client

Iterative and Concurrent Servers

  • Server program runs continuously (deals with clients in an infinite loop) ⇒ always listening and available for multiple clients to send connection requests
# basic_server_iterative.py
import socket
my_socket = socket.socket()
my_socket.bind((‘128.0.0.1’, 12345))
my_socket.listen()
while True:
new_socket, addr = my_socket.accept()
new_socket.sendall(b’Hello from server\n’)
new_socket.close()
  • Server’s passive socket keeps a queue of connection requests that have been received
  • A request is removed from this queue each time socket.accept() is called to create a connection
  • If queue empty, socket.accept() will block the program until a connection request is received
  • socket.accept() is called each time the infinite loop repeats ⇒ program can handle multiple clients by processing them 1 at a time ⇒ iterative server
  • Iterative servers are easy to write but limited: can only handle one client at a time
  • OR can write a server that starts a thread that runs simultaneously with the main program each time a client tries to connect ⇒ makes program more complicated but will let it handle multiple clients at the same time ⇒ concurrent server

Writing a Chat Program

  • 2 users (server + client) take turns sending single lines of text to each other
  • Each message is restricted to a single line ⇒ \n will never be part of a message
  • Can use protocol of using \n to detect end of message
# chat_server.py
import socket
listen_socket = socket.socket()
listen_socket.bind((‘127.0.0.1’, 6789))
listen_socket.listen()
chat_socket, addr = listen_socket.accept()
while True:
data = input(‘INPUT SERVER: ’).encode()
chat_socket.sendall(data + b’\n’)
if data == b’quit’:
break
print(“waiting for client…”)
data = b’’
while b’\n’ not in data:
data += chat_socket.recv(1024)
print(‘client wrote: ’ + data.decode())
if data == b’quit\n’:
break
# if use quit:
listen_socket.close()
chat_socket.close()
Alt: use while loop and Boolean indicator to exit the program with quit message
# chat_server.py
import socket
listen_socket = socket.socket()
listen_socket.bind((‘127.0.0.1’, 6789))
listen_socket.listen()
chat_socket, addr = listen_socket.accept()
leave = False
while not leave:
data = input(‘INPUT SERVER: ’)
chat_socket.sendall(data**.encode()** + b’\n’)
if data == ’quit’:
leave = True
else:
print(“waiting for client…”)
data = b’’
while b’\n’ not in data:
data += chat_socket.recv(1024)
print(‘client wrote: ’ + data.decode())
if data == b’quit\n’:
leave = True
# if use quit:
listen_socket.close()
chat_socket.close()
# chat_client.py
import socket
chat_socket = socket.socket()
address = input(“Enter IPv4 address of server: ”)
port = int(input(“Enter port number of server: ”))
chat_socket.connect((address, port))
while True:
print(“waiting for server…”)
data = b’’
while b’\n’ not in data:
data += chat_socket.recv(1024)
print(“server wrote: ” + data.decode())
if data == b’quit\n’:
break
data = input(“INPUT CLIENT: ”).encode()
chat_socket.sendall(data + b’\n’)
if data == b’quit’:
break
# if use quit:
chat_socket.close()
# Alt: use while loop and Boolean indicator to exit the program with quit message
import socket
chat_socket = socket.socket()
addr = input("Enter IPv4 address of server: ")
port = int(input("Enter port number of server: "))
chat_socket.connect((addr, port))
leave = False
while not leave:
print("waiting for server...")
data = b''
while b'\n' not in data:
data += chat_socket.recv(1024)
print("server wrote: ", data.decode()
if data == b'quit\n':
leave = True
else:
data = input("INPUT CLIENT: ")
chat_socket.sendall(data.encode() + b’\n’)
if data == 'quit':
leave = True
chat_socket.close()
  • Run server & client using 2 diff copies of Python, on the same machine
  • Can use 127.0.0.1 as server’s IPv4 address and 6789 as port number
  • Exit both programs once message “quit’ is sent by any user
  • Make sure all sockets are closed properly before exiting

Writing a Turn-Based Game

  • Both server and protocol designs may be based on an existing standard or developed by someone else
# tictactoe.py
N = 3 # size of grid
width = len(str(N**2)) # width for each cell
players = (‘O’, ‘X’) # player symbols
class TicTacToe:
def __init__(self):
self.board = [ ]
for i in range(N):
self.board.append([None]*N)
def render_row(self, row_index):
start = row_index * N + 1 # 1st no. of the row?
row = self.board[row_index].copy()
for column_index in range(N):
if row[column_index] is None:
cell = str(start + column_index)
else:
cell = players[row[column_index]]
if len(cell) < width:
cell += ‘ ’ * (width - len(cell))
row[column_index] = ‘ ’ + cell + ‘ ’
return ‘|’.join(row) + ‘\n’
def render_board(self):
rows = [ ]
for row_index in range(N):
rows.append(self.render_row(row_index))
divider = ‘-’ * ((width+3) * N - 1) + ‘\n’
return divider.join(rows)
def make_move(self, player_index, cell_index):
cell_index -= 1
self.board[cell_index // N][cell_index % N] = player_index
def is_valid_move(self, cell_index):
if cell_index < 1 or cell_index > N ** 2:
return False
cell_index -= 1
return self.board[cell_index // N][cell_index % N] is None
def is_full(self):
for row_index in range(N):
for column_index in range(N):
if self.board[row_index][column_index] is None:
return False
return True
def get_winner(self):
# check diagonals
if self.board[0][0] is not None: # left to right diagonal
found = True
for i in range(N):
if self.board[0][0] != self.board[i][i]:
found = False
break
if found:
return self.board[0][0]
if self.board[0][N-1] is not None: # right to left diagonal
found = True
for i in range(N):
if self.board[0][N-1] != self.board[i][N - i - 1]:
found = False
break
if found:
return self.board[0][N-1]
# check rows and columns
for i in range(N):
if self.board[i][0] is not None:
found = True
for j in range(N):
if self.board[i][0] != self.board[i][j]:
found = False
break
if found:
return self.board[i][0]
if self.board[0][i] is not None:
found = True
for j in range(N):
if self.board[0][i] != self.board[j][i]:
found = False
break
if found:
return self.board[0][i]
# no matching lines were found, so no winner
return None
chatgpt’s alt to make board:
class TicTacToe:
def __init__(self, N):
self.N = N
self.board = [[None] * N for _ in range(N)] # nested
self.players = ('O', 'X')
self.width = len(str(N * N)) # dynamic width
def render_board(self):
rows = []
for i in range(self.N):
row = []
for j in range(self.N):
cell = self.board[i][j]
if cell is None:
value = str(i * self.N + j + 1)
else:
value = self.players[cell]
row.append(value.rjust(self.width)) # right justify
rows.append(” | “.join(row)) # + separator
divider = "\n" + "-" * ((self.width + 3) * self.N - 3) + "\n"
# = N*width ( ⇒ cells) + (N-1)*3 ( ⇒ separators)
return divider.join(rows)
def make_move(self, player, position):
position -= 1
i, j = divmod(position, self.N)
if self.board[i][j] is not None:
return False # invalid move
self.board[i][j] = player
return True
…
def get_winner(self):
N = self.N
board = self.board
# Rows
for i in range(N):
first = board[i][0]
if first is not None and all(board[i][j] == first for j in range(N)):
return first
# Columns
for j in range(N):
first = board[0][j]
if first is not None and all(board[i][j] == first for i in range(N)):
return first
# Main diagonal
first = board[0][0]
if first is not None and all(board[i][i] == first for i in range(N)):
return first
# Anti-diagonal
first = board[0][N-1]
if first is not None and all(board[i][N-i-1] == first for i in range(N)):
return first
return None
methodsdescription
render_row(row_index)returns a string representation of the specified row
e.g. 1 | 2 | 3
render_board()returns a string representation of the entire board
e.g. 1 | 2 | 3
-----------
4 | 5 | 6
-----------
7 | 8 | 9
-----------
make_move(player_index, cell_index)modifies board such that the specified cell is marked with the symbol for the specified player
is_valid_move(cell_index)returns whether the specified cell is currently blank
is_full()returns whether the entire board has been filled up
get_winner()returns winning player for the current board or None if there is no winner
# game_server.py
# for game_client.py, swap “client” and “server”, and move last step to the front
⇒ symmetrical game: same rules for both players
import socket
import tictactoe
listen_socket = socket.socket() # use game_socket for game_client.py
listen_socket.bind((‘127.0.0.1’, 3456)) # use game_socket.connect
listen_socket.listen() # not needed for game_client.py
game_socket, addr = listen_socket.accept() # not needed for game_client.py
game = tictactoe.TicTacToe()
# infinite loop
while True:
# display current tic-tac-toe board
print(game.render_board())
# check if current player won, end game with opponent winning
if game.get_winner() is not None:
print(“opponent wins!”)
print()
break
# check if board is full, end game with a stalemate if so
if game.is_full():
print(“stalemate”)
print()
break
# prompt for move from server player, update game board if valid move
# server layer always starts first ⇒ start by receiving & processing server
move = -1
while move != 0 and not game.is_valid_move(move):
move = int(input(“server moves ” + “(0 to quit): ”))
print()
if move == 0:
game_socket.sendall(b’END\n’)
print(‘you quit, opponent wins!’)
print()
break
game.make_move(0, move)
game_socket.sendall(b’MOVE’ + str(move).encode() + b’\n’)
# display current tic-tac-toe board again
print(game.render_board())
# check if server player won, end game with player winning if so
if game.get_winner() is not None:
print(“you win!”)
print()
break
# check if board is full, end game with a stalemate if so
if game.is_full():
print(“stalemate”)
print()
break
# move to front for game_client.py
# receive move from client player (opponent) via socket ⇒ last step
received = b’’
while b’\n’ not in received:
# end game with player winning
received += game_socket.recv(1024)
if received.startswith(b’MOVE’):
# update board accordingly
move = int(received[4:])
print(‘client moves: ’ + str(move))
print()
game.make_move(1, move)
elif received.startswith(b’END’):
print(‘opponent quits, you win!’)
print()
break
game_socket.close()
listen_socket.close() # no needed for game_client.py

Asymmetrical game: 2 players behave differently from each other

E.g. guess-the-number (server generates random no. 1-100, client tries to guess within 5 tries)

# guess_client.py
import socket
s = socket.socket()
s.connect((‘127.0.0.1’, 9999))
data = b’’
while True:
while b’\n’ not in data:
data += s.recv(1024)
received = data[:data.find(b’\n’)]
data = data[len(received) + 1:] # +1 since “\n” = 1 chr; this removes the received + \n
if received == b’LOW’:
print(“your guess is too low”)
elif received == b’HIGH’:
print(“your guess is too high”)
elif received == b’GUESS’:
guess = int(input(“enter guess (1-100): ”))
s.sendall(str(guess).encode() + b’\n’)
elif received == b’WIN’:
print(“you win!”)
break
elif received == b’GAMEOVER’:
print(“you ran out of tries! game over.”)
break
s.close()
# guess_server.py
import socket
from random import randint
my_socket = socket.socket()
my_socket.bind((‘127.0.0.1’, 9999))
my_socket.listen()
game_socket, addr = my_socket.accept()
answer = randint(1,100)
print(answer) # to check when testing
count = 1
game_socket.sendall(b’GUESS\n’)
# or use guessed = False
while count < 5: # or use for i in range(5)
guess = b''
while b'\n' not in guess:
guess += game_socket.recv(1024)
# guess = int(guess.decode())
guess = int(data[:data.find(b'\n')].decode())
if guess == answer:
game_socket.sendall(b’WIN\n’) # guessed = True
break
elif guess > answer:
game_socket.sendall(b’HIGH\n’)
game_socket.sendall(b’GUESS\n’) # or only send “if guessed: “
elif guess < answer:
game_socket.sendall(b’LOW\n’)
game_socket.sendall(b’GUESS\n’) # or only send “if guessed: “
count += 1
if count == 5:
game_socket.sendall(b’GAMEOVER\n’) # “else” clause for “if guessed: ”
game_socket.close()
my_socket.close()

socket module summary

methodsdescription
bind((host, port))binds socket object to the given address tuple (host, port)
host = IPv4 address
port = port number
listen()enables socket to listen for incoming connections from clients
accept()waits for an incoming connection and returns a tuple containing a new socket object for the connection and an address tuple (host, port)
host = the IPv4 address of the connected client
port = its port number
connect((host, port))initiates a connection to the given address tuple (host, port)
host = IPv4 address of server
port = its port number
recv(max_bytes)receives and returns up to the given no. of bytes from the socket
sendall(bytes)sends the given bytes to the socket

All notebooks, including extra practice, in jupyter notebook folder!

Comments from the Word document

Graph View

Backlinks