overview
in numpy, have array of bools. array retrieved image; two-dimensional , contains 1024 columns , 768 rows. want push data on ethernet cable. there multiple ways this, purposes speed extremely crucial, , therefore memory crucial.
since there 1024 x 768 = 786432 elements (pixels) in each array, , each element either true or false, theoretically possible pack array 98,304 uncompressed bytes or 96 kilobytes.
786432 bits / 8 bits per byte = 98304 bytes 98304 bytes / 1024 bytes per kilobyte = 96 kilobytes this requires flattening array
[ [true, false, true, ..., true] [false, true, true, ..., true] ... [true, true, false, ..., false] ] # flatten array [true, false, true, ..., false] which can theoretically represented bits of bytes, since 786,432 bits fits evenly 98,304 bytes; each array should able represented 98,304 eight-bit chars.
the question
how can send 1024-by-768 bool numpy arrays on ethernet? i'm looking bitstring python library, i'm not sure how pipe numpy arrays bitstring class.
additional information / questions
to specific, i'm sending these arrays raspberry pi 2 regular raspberry pi.
- is
socket,sock_streamfastest way go this? - given rpis computing power, faster compress , decompress arrays? if so, compression must lossless.
- i've looked serializing numpy arrays rather using
bitstringstuff, pickled objects large send onsock_stream. doing wrongsocketstuff?
my code / solution [solved]
client
import socket scipy.misc import imread import numpy ip = '127.0.0.1' port = 7071 address = (ip, port) sock = socket.socket(socket.af_inet, socket.sock_stream) image = imread('input.png')[:,:,[2]] image[image < 170] = 0 image[image != 0] = 1 image = numpy.reshape(image, (-1, 1)) image = numpy.packbits(image) data = image.tostring() sock.connect(address) in range(0, 93804, 1024): sock.send(data[i:i+1024]) sock.shutdown(socket.shut_wr) sock.close() server
import socket scipy.misc import imsave import numpy ip = '127.0.0.1' port = 7071 address = (ip, port) sock = socket.socket(socket.af_inet, socket.sock_stream) sock.bind(address) sock.listen(1) while true: c, addr = sock.accept() data = '' package = c.recv(1024) while package: data += package package = c.recv(1024) image = numpy.fromstring(data, dtype=numpy.uint8) image = numpy.unpackbits(image) image = numpy.reshape(image, (-1, 768)) imsave('output.png', image) c.close() sock.close() as can see, ended each array on tcp/sock_stream via series of 1024-byte packets.
you can use np.packbits pack contents of np.bool array np.uint8 array 1/8th of size, such each 'packed' boolean element uses single bit. original array can recovered using np.unpackbits.
import numpy np x = array([0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1], dtype=np.bool) print(x.itemsize, x.nbytes) # (1, 16) xp = np.packbits(x) print(xp) # [ 24 139] print(xp.itemsize, xp.nbytes) # (1, 2) print(np.unpackbits(xp)) # [0 0 0 1 1 0 0 0 1 0 0 0 1 0 1 1] the obvious way go here serialize packed array raw string of bytes, pipe through udp socket, deserialize , unpack on other side. numpy's native serialization (.tostring() , np.fromstring()) faster using pickle or cpickle.
if want play around compression, 1 option use native zlib module compress string of bytes before passing through pipe, decompress on other side. whether see benefit depend on how compressible input arrays are, on hardware that's doing compression/decompression.
