Python Socket is receiving inconsistent messages from Server


Question


So I am very new to networking and I was using the Python Socket library to connect to a server that is transmitting a stream of location data.

Here is the code used.

import socket

BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((gump.gatech.edu, 756))

try:
    while (1):
        data = s.recv(BUFFER_SIZE).decode('utf-8')
        print(data)
except KeyboardInterrupt:
    s.close()

The issue is that the data arrives in inconsistent forms.

Most of the times it arrives in the correct form like this:

2016-01-21 22:40:07,441,-84.404153,33.778685,5,3

Yet other times it can arrive split up into two lines like so:

2016-01-21

22:40:07,404,-84.396004,33.778085,0,0

The interesting thing is that when I establish a raw connection to the server using Putty I only get the correct form and never the split. So I imagine that there must be something happening that is splitting the message. Or something Putty is doing to always assemble it correctly.

What I need is for the variable data to contain the proper line always. Any idea how to accomplish this?


Answer 1:


It is best to think of a socket as a continuous stream of data, that may arrive in dribs and drabs, or a flood.

In particular, it is the receivers job to break the data up into the "records" that it should consist of, the socket does not magically know how to do this for you. Here the records are lines, so you must read the data and split into lines yourself.

You cannot guarantee that a single recv will be a single full line. It could be:

  • just part of a line;
  • or several lines;
  • or, most probably, several lines and another part line.

Try something like: (untested)

# we'll use this to collate partial data
data = ""

while 1:
    # receive the next batch of data
    data += s.recv(BUFFER_SIZE).decode('utf-8')

    # split the data into lines
    lines = data.splitlines(keepends=True)

    # the last of these may be a part line
    full_lines, last_line = lines[:-1], lines[-1]

    # print (or do something else!) with the full lines
    for l in full_lines:
        print(l, end="")

    # was the last line received a full line, or just half a line?
    if last_line.endswith("\n"):
        # print it (or do something else!)
        print(last_line, end="")

        # and reset our partial data to nothing
        data = ""
    else:
        # reset our partial data to this part line
        data = last_line



Answer 2:


The easiest way to fix your code is to print the received data without adding a new line, which the print statement (Python 2) and the print() function (Python 3) do by default. Like this:

Python 2:

print data,

Python 3:

print(data, end='')

Now print will not add its own new line character to the end of each printed value and only the new lines present in the received data will be printed. The result is that each line is printed without being split based on the amount of data received by each `socket.recv(). For example:

from __future__ import print_function
import socket

s = socket.socket()
s.connect(('gump.gatech.edu', 756))

while True:
    data = s.recv(3).decode('utf8')
    if not data:
        break    # socket closed, all data read
    print(data, end='')

Here I have used a very small buffer size of 3 which helps to highlight the problem.

Note that this only fixes the problem from the POV of printing the data. If you wanted to process the data line-by-line then you would need to do your own buffering of the incoming data, and process the line when you receive a new line or the socket is closed.




Answer 3:


Edit: socket.recv() is blocking and like the others said, you wont get an exact line each time you call the method. So as a result, the socket is waiting for data, gets what it can get and then returns. When you print this, because of pythons default end argument, you may get more newlines than you expected. So to get the raw stuff from your server, use this:

import socket 
BUFFER_SIZE = 1024 
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('gump.gatech.edu', 756)) 
try: 
    while (1):   
        data=s.recv(BUFFER_SIZE).decode('utf-8')
        if not data: break
        print(data, end="") 
except KeyboardInterrupt: 
    s.close()


来源:https://stackoverflow.com/questions/34935857/python-socket-is-receiving-inconsistent-messages-from-server


码神部落- 版权声明 1、本主题所有言论和图片纯属会员个人意见,与码神部落立场无关。
2、本站所有主题由该帖子作者发表,该帖子作者爷依然萧洒码神部落享有帖子相关版权。
3、码神部落管理员和版主有权不事先通知发贴者而删除本文。
4、其他单位或个人使用、转载或引用本文时必须同时征得该帖子作者爷依然萧洒码神部落的同意。
5、帖子作者须承担一切因本文发表而直接或间接导致的民事或刑事法律责任。
6、本帖部分内容转载自其它媒体,但并不代表本站赞同其观点和对其真实性负责。
7、如本帖侵犯到任何版权问题,请立即告知本站,本站将及时予与删除并致以最深的歉意。

最新回复 (0)
    • 码神部落
      2
        立即登录 立即注册 GitHub登录
返回
发新帖
作者最近主题: