• notice
  • Congratulations on the launch of the Sought Tech site

Python network programming socket and socketserver

1. Socket socket programming based on TCP protocol


1. Socket workflow

Let's start with the server side. The server first initializes the Socket, then binds it to the port, listens to the port, calls accept to block, and waits for the client to connect. At this time, if a client initializes a Socket, and then connects to the server (connect), if the connection is successful, then the connection between the client and the server is established. The client sends a data request, the server receives the request and processes the request, then sends the response data to the client, the client reads the data, and finally closes the connection, and an interaction ends, using the following Python code:

import socket
# socket_family can be AF_UNIX or AF_INET. socket_type can be SOCK_STREAM or SOCK_DGRAM. protocol is generally not filled, the default value is 0
socket.socket(socket_family, socket_type, protocol=0)
# get tcp/ip socket
tcpSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# get udp/ip socket
udpSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)


1. Server socket function

  • s.bind(): bind (host, port number) to the socket

  • s.listen(): start TCP listening

  • s.accept(): passively accept connections from TCP clients, (blocking) waiting for connections to arrive


2. Client socket function

  • s.connect(): Actively initialize the TCP server connection

  • s.connect_ex(): An extended version of the connect() function that returns an error code on error instead of throwing an exception


3. Socket functions for public use

  • s.recv(): Receive TCP data

  • s.send(): Send TCP data (when the amount of data to be sent is greater than the remaining space in the own-end buffer, the data is lost and will not be sent out)

  • s.sendall(): Send complete TCP data (the essence is to call send in a loop. When the amount of data to be sent is greater than the remaining space in the buffer at its own end, the data is not lost, and send is called in a loop until it is sent out)

  • s.recvfrom(): Receive UDP data

  • s.sendto(): send UDP data

  • s.getpeername(): the address of the remote end connected to the current socket

  • s.getsockname(): the address of the current socket

  • s.getsockopt(): returns the parameters of the specified socket

  • s.setsockopt(): Set the parameters of the specified socket

  • s.close(): close the socket


4. Lock-oriented socket method

  • s.setblocking(): Set the blocking and non-blocking mode of the socket

  • s.settimeout(): Set the timeout for blocking socket operations

  • s.gettimeout(): Get the timeout for blocking socket operations


5. Functions of file-oriented sockets

  • s.fileno(): the file descriptor of the socket

  • s.makefile(): create a file associated with the socket


2. Socket programming based on TCP protocol

You can netstat -an | findstr 8080check the socket status by



1. Server

import socket
# 1. Buy a mobile phone
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # tcp is called streaming protocol, udp is called datagram protocol SOCK_DGRAM
# print(phone)
# 2. Insert/bind mobile phone card
# phone.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
phone.bind(("127.0.0.1", 8080))
# 3. Boot
phone.listen(5) # Semi-connection pool, limiting the number of requests
# 4. Wait for the phone to connect
print("start....")
while True: # join loop
conn, client_addr = phone.accept() # (two-way connection established by three-way handshake, (client's ip, port))
# print(conn)
print("A connection has been established successfully", client_addr)
# 5. Communication: Send and receive messages
while True: # communication loop
try:
print("The server is receiving data...")
data = conn.recv(1024) # The maximum number of bytes received, no data will be waiting to be received in place, that is, the amount of data sent by the sender must be > 0bytes
# print("===>")
if len(data) == 0: break # Only when the client disconnects unilaterally, the server will receive empty data
print("Data from client", data)
conn.send(data.upper())
except ConnectionResetError:
break
# 6. Hang up the phone connection
 conn.close()
# 7. Shut down
phone.close()
#start....
# A connection has been established successfully ("127.0.0.1", 4065)
# The server is receiving data...
# data from client b"xad"
# The server is receiving data...



2. Client

import socket
# 1. Buy a mobile phone
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# print(phone)
# 2. Dial the phone
phone.connect(("127.0.0.1", 8080)) # Specify the server ip and port
# 3. Communication: send and receive messages
while True: # communication loop
msg = input(">>: ").strip() # msg=""
if len(msg) == 0: continue
phone.send(msg.encode("utf-8"))
# print("has send----->")
data = phone.recv(1024)
# print("has recv----->")
print(data)
# 4. Close
phone.close()
# >>: ah
# b"a"
# >>: ah ah
# b"xb0xa1xb0xa1"
# >>:


3. Address occupation problem

This is because your server still has the time_wait state of four waved hands occupying the address (if you do not understand, please in-depth study 1.tcp three-way handshake, four waved hands 2.syn flood attack 3. In the case of high server concurrency, there will be a large number of optimization method of the time_wait state)


1. Method 1: Add a socket configuration and reuse ip and port

phone=socket(AF_INET,SOCK_STREAM)
phone.setsockopt(SOL_SOCKET,SO_REUSEADDR,1) #That's it, add it before bind
phone.bind(("127.0.0.1",8080))


2. Method 2: By adjusting the linux kernel parameters

It is found that there are a large number of connections in the TIME_WAIT state in the system, which can be solved by adjusting the Linux kernel parameters.
vi /etc/sysctl.conf
Edit the file and add the following:
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_fin_timeout=30
Then execute /sbin/sysctl -p to make the parameters take effect.
net.ipv4.tcp_syncookies = 1 means enable SYN Cookies. When the SYN waiting queue overflows, enable cookies to deal with it, which can prevent a small number of SYN attacks. The default value is 0, which means it is closed;
net.ipv4.tcp_tw_reuse = 1 means to enable reuse. Allow TIME-WAIT sockets to be reused for new TCP connections, the default is 0, which means close;
net.ipv4.tcp_tw_recycle = 1 means to enable fast recycling of TIME-WAIT sockets in TCP connections, the default is 0, which means close.
net.ipv4.tcp_fin_timeout Modify the default TIMEOUT time of the system


4. Simulate ssh remote command execution

The server executes the command through subprocess, and then returns the result of the command.

Server:

from socket import *
import subprocess
server = socket(AF_INET, SOCK_STREAM)
server.bind(("127.0.0.1", 8000))
server.listen(5)
print("start...")
while True:
conn, client_addr = server.accept()
while True:
print("from client:", client_addr)
cmd = conn.recv(1024)
if len(cmd) == 0: break
print("cmd:", cmd)
obj = subprocess.Popen(cmd.decode("utf8"), # cmd command entered
shell=True, # run through the shell
stderr=subprocess.PIPE, # pipe error output for printing
stdout=subprocess.PIPE) # pipe the correct output for printing

stdout = obj.stdout.read() # print the correct output
stderr = obj.stderr.read() # print error output

conn.send(stdout)
conn.send(stderr)
conn.close()
server.close()

client

import socketclient = socket.socket(socket.AF_INET, socket.SOCK_STREAM)client.connect(("127.0.0.1", 8000))while True:data = input("please enter your data")client.send(data.encode("utf8"))data = client.recv(1024)print("from server:", data)client.close()

Enter dirthe command. Since the server sends less than 1024 bytes, the client can accept it.

Enter tasklistthe command, because the server sends more than 1024 bytes, the client only accepts part of the data, and when you enter dirthe command again, the client will receive dirthe result of the command, but will print the remaining unsent last time. Data, this is the sticky package problem.


5. Sticky bag


1. The sender needs to wait for the buffer to be full before sending out, resulting in sticky packets

The time interval for sending data is very short, and the amount of data is small, and they are combined together to generate sticky packets.

Server

# _*_coding:utf-8_*_from socket import *ip_port = ("127.0.0.1", 8080)TCP_socket_server = socket(AF_INET, SOCK_STREAM)TCP_socket_server.bind(ip_port)TCP_socket_server.listen(5)conn, addr = TCP_socket_server.accept()data1 = conn.recv(10)data2 = conn.recv(10)print("----->", data1.decode("utf-8"))print("----->", data2.decode("utf-8"))conn.close()

client

# _*_coding:utf-8_*_import socketBUFSIZE = 1024ip_port = ("127.0.0.1", 8080)s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)res = s.connect_ex(ip_port)s.send("hello".encode("utf-8"))s.send("world".encode("utf-8"))# The server receives b"helloworld" together


2. The receiver does not receive the packets in the buffer in time, resulting in multiple packet reception

The client sends a piece of data, but the server only receives a small part. When the server receives the data next time, it still takes the data left over from the last time from the buffer to generate sticky packets.

Server

# _*_coding:utf-8_*_from socket import *ip_port = ("127.0.0.1", 8080)TCP_socket_server = socket(AF_INET, SOCK_STREAM)TCP_socket_server.bind(ip_port)TCP_socket_server.listen(5)conn, addr = TCP_socket_server.accept()data1 = conn.recv(2) # did not receive a completedata2 = conn.recv(10) # When receiving the next time, the old data will be taken first, and then the new one will be takenprint("----->", data1.decode("utf-8"))print("----->", data2.decode("utf-8"))conn.close()

client

# _*_coding:utf-8_*_import socketBUFSIZE = 1024ip_port = ("127.0.0.1", 8080)s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)res = s.connect_ex(ip_port)s.send("hello feng".encode("utf-8"))


6. Solve the sticky package problem


1. The total size of the byte stream sent first (low version)

The root of the problem is that the receiver does not know the length of the byte stream that the sender will transmit, so the solution to the sticky packet is how to let the sender let the receiver send the total size of the byte stream it will send before sending the data. The end knows, and then the receiving end receives all the data in an infinite loop.

Why low: The running speed of the program is much faster than the network transmission speed, so before sending a segment of bytes, use send to send the length of the byte stream, which will amplify the performance loss caused by network delay.

Server:

import socket, subprocessserver = socket.socket (socket.AF_INET, socket.SOCK_STREAM)server.bind(("127.0.0.1", 8000))server.listen(5)while True:conn, addr = server.accept()print("start...")while True:cmd = conn.recv(1024)print("cmd:", cmd)obj = subprocess.Popen(cmd.decode("utf8"),shell=True,stderr=subprocess.PIPE,stdout=subprocess.PIPE)stdout = obj.stdout.read()if stdout:ret = stdoutelse:stderr = obj.stderr.read()ret = stderrret_len = len (ret)
 conn.send(str(ret_len).encode("utf8"))data = conn.recv(1024).decode("utf8")if data == "recv_ready":conn.sendall(ret)conn.close()server.close()

Client:

import socketclient = socket.socket(socket.AF_INET, socket.SOCK_STREAM)client.connect(("127.0.0.1", 8000))while True:msg = input("please enter your cmd you want>>>").strip()if len(msg) == 0: continueclient.send(msg.encode("utf8"))length = int(client.recv(1024))client.send("recv_ready".encode("utf8"))send_size = 0recv_size = 0data = b""while recv_size < length:data = client.recv(1024)recv_size += len(data)print(data.decode("utf8"))


2. Custom fixed-length header (struct module)

struct module analysis

import structimport json# "i" is the formattry:obj = struct.pack("i", 1222222222223)except Exception as e:print(e)obj = struct.pack("i", 1222)print (obj, len (obj))# "i" format requires -2147483648 <= number <= 2147483647# b"xc6x04x00x00" 4res = struct.unpack("i", obj)print(res[0])# 1222

The core of solving the sticky packet problem is to add a custom fixed-length header to the byte stream, and the header contains the length of the byte stream, and then send it to the peer at one time. When the peer receives it, it first takes out the fixed-length header from the cache. , and then take the real data.

1. Use the struct module to create a header:

import jsonimport structheader_dic = {"filename": "a.txt","total_size":111111111111111111111111111111111222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222223131232,"hash": "asdf123123x123213x"}header_json = json.dumps(header_dic)header_bytes = header_json.encode("utf-8")print(len(header_bytes))# 223# "i" is the formatobj = struct.pack("i", len(header_bytes))print (obj, len (obj))# b"xdfx00x00x00" 4res = struct.unpack("i", obj)print(res[0])# 223

2. Server:

from socket import *import subprocessimport structimport jsonserver = socket(AF_INET, SOCK_STREAM)server.bind(("127.0.0.1", 8000))server.listen(5)print("start...")while True:conn, client_addr = server.accept()print(conn, client_addr)while True:cmd = conn.recv(1024)obj = subprocess.Popen(cmd.decode("utf8"),shell=True,stderr=subprocess.PIPE,stdout=subprocess.PIPE)stderr = obj.stderr.read()stdout = obj.stdout.read()# make headerheader_dict = {"filename": "a.txt","total_size": len(stdout) + len(stderr),"hash": "xasf123213123"}header_json = json.dumps(header_dict)header_bytes = header_json.encode("utf8")# 1. First package the length of the header len(header_bytes) into 4 bytes, and then sendconn.send(struct.pack("i", len(header_bytes)))# 2. Send header
 conn.send(header_bytes)# 3. Send real data
 conn.send(stdout)conn.send(stderr)conn.close()server.close()

3. Client:

from socket import *import jsonimport structclient = socket(AF_INET, SOCK_STREAM)client.connect(("127.0.0.1", 8000))while True:cmd = input("please enter your cmd you want>>>")if len(cmd) == 0: continueclient.send(cmd.encode("utf8"))# 1. Receive 4 bytes first, these 4 bytes contain the length of the headerheader_len = struct.unpack("i", client.recv(4))[0]# 2. Receive the header againheader_bytes = client.recv(header_len)# 3. Parse what you want from the headerheader_json = header_bytes.decode("utf8")header_dict = json.loads(header_json)total_size = header_dict["total_size"]# 4. Re-receive real datarecv_size = 0res = b""while recv_size < total_size:data = client.recv(1024)res += datarecv_size += len(data)print(res.decode("utf8"))client.close()


2. Socket socket programming based on UDP protocol

  • UDP is unconnected, whichever side is started first will not report an error, and multiple clients can communicate with the server at the same time

  • The UDP protocol is a datagram protocol. When it is empty, it will also bring its own header. Therefore, if the client input is empty, the server can also receive it.

  • The UPD protocol is generally not used to transmit large data.

  • UPD socket has no sticky packet problem, but cannot replace TCP socket, because UPD protocol has a defect: if data is lost in the way of data transmission, the data will be lost, while TCP protocol will not have this defect, So generally UPD socket users don't care about data sending, such as qq chat.


Simple example of UDP socket


1. Server

import socketserver = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Datagram protocol-"UDPserver.bind(("127.0.0.1", 8080))while True:data, client_addr = server.recvfrom(1024)print("===>", data, client_addr)server.sendto(data.upper(), client_addr)server.close()


2. Client

import socketclient = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Datagram protocol-"UDPwhile True:msg = input(">>: ").strip() # msg=""client.sendto(msg.encode("utf-8"), ("127.0.0.1", 8080))data, server_addr = client.recvfrom(1024)print(data)client.close()


3. Implement concurrent socket programming based on socketserver


1. Based on TCP protocol

The key to tcp-based sockets is two loops, a link loop and a communication loop

The socketserver module is divided into two categories: the server class (to solve the link problem) and the request class (to solve the communication problem).


1. server class


2. Request class

Based on tcp socketserver in our own defined class.

  • self.server is the socket object

  • self.request is a link

  • self.client_address is the client address


3. Server

import socketserverclass MyHandler(socketserver.BaseRequestHandler):def handle(self):# communication loopwhile True:# print(self.client_address)# print(self.request) #self.request=conntry:data = self.request.recv(1024)if len(data) == 0: breakself.request.send(data.upper())except ConnectionResetError:breakif __name__ == "__main__":s = socketserver.ThreadingTCPServer(("127.0.0.1", 8080), MyHandler, bind_and_activate=True)s.serve_forever() # represents the connection loop# Loop to establish a connection, each time a connection is established, a thread (waiter) is started + an object is generated by calling the Myhanlder class, and the handle method under the object is called to communicate with the newly established connection.


4. Client

import socketphone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)phone.connect(("127.0.0.1", 8080)) # Specify the server ip and portwhile True:# msg=input(">>: ").strip() #msg=""msg = "client33333" # msg=""if len(msg) == 0: continuephone.send(msg.encode("utf-8"))data = phone.recv(1024)print(data)phone.close()


2. Based on UDP protocol

udp-based socketserver in our own defined class

  • self.request is a tuple (the first element is the data sent by the client, and the second part is the udp socket object of the server), such as (b'adsf', )

  • self.client_address is the client address

1. Server

import socketserverclass MyHandler(socketserver.BaseRequestHandler):def handle(self):# communication loopprint(self.client_address)print(self.request)data = self.request[0]print("Customer message", data)self.request[1].sendto(data.upper(), self.client_address)if __name__ == "__main__":s = socketserver.ThreadingUDPServer(("127.0.0.1", 8080), MyHandler)s.serve_forever()

2. Client

import socketclient = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Datagram protocol-"udpwhile True:# msg=input(">>: ").strip() #msg=""msg = "client1111"client.sendto(msg.encode("utf-8"), ("127.0.0.1", 8080))data, server_addr = client.recvfrom(1024)print(data)client.close()


4. Python Internet Module

Some important modules of Python network programming are listed below:

protocolFunctional useThe port numberPython modules
HTTPweb access80httplib, urllib, xmlrpclib
NNTPRead and post news articles, commonly known as "posts"119nntplib
FTPfile transfer20ftplib, urllib
SMTPsend email25smtplib
POP3incoming mail110poplib
IMAP4get mail143imaplib
TelnetCommand Linetwenty threetelnetlib
GopherInformation lookup70gopherlib, urllib


Tags

Technical otaku

Sought technology together

Related Topic

1 Comments

author

buy atorvastatin 20mg sale & lt;a href="https://lipiws.top/"& gt;order atorvastatin 20mg pills& lt;/a& gt; buy lipitor 20mg sale

Rqlckn

2024-03-09

Leave a Reply

+