Video Feed via TCP Socket

Question

I managed to send video feed over tcp socket using code and decode of string64. My problem is that the video feed is kinda laggy, and it's probably because of this time between coding and decoding.

Is there a simple way to improve this?

C#

private void enviar_data()
 {
 string data = "";
 string data2 = "";
 string string_master_frame, string_slave_frame, string_arena_frame;
 string conetado, bateria, tensao, altitude, roll, pitch, yaw, velx, vely, velz, estado, atual, desejado;
 string conetado2, bateria2, tensao2, altitude2, roll2, pitch2, yaw2, velx2, vely2, velz2, estado2, atual2, desejado2;
 conetado = master.sconetado;
 bateria = master.sbateria;
 tensao = master.stensao;
 altitude = master.saltitude;
 roll = master.sroll;
 pitch = master.spitch;
 yaw = master.syaw;
 velx = master.svelx;
 vely = master.svely;
 velz = master.svelz;
 estado = master.sestado;
 atual = master.satual;
 desejado = master.sdesejado;
 conetado2 = slave.sconetado;
 bateria2 = slave.sbateria;
 tensao2 = slave.stensao;
 altitude2 = slave.saltitude;
 roll2 = slave.sroll;
 pitch2 = slave.spitch;
 yaw2 = slave.syaw;
 velx2 = slave.svelx;
 vely2 = slave.svely;
 velz2 = slave.svelz;
 estado2 = slave.sestado;
 atual2 = slave.satual;
 desejado2 = slave.sdesejado;
 ImageConverter converter = new ImageConverter();
 byte[] sendBytes = (byte[])converter.ConvertTo( master.picturebox_master.Image, typeof(byte[])); 
 string_master_frame = Convert.ToBase64String(sendBytes);
 byte[] sendBytes2 = (byte[])converter.ConvertTo(slave.picturebox_master.Image, typeof(byte[]));
 string_slave_frame = Convert.ToBase64String(sendBytes2);
 Bitmap arena = new Bitmap(chart1.Width, chart1.Height);
 chart1.DrawToBitmap(arena, new Rectangle(0,0,chart1.Width,chart1.Height));
 byte[] sendBytes3 = (byte[])converter.ConvertTo(arena, typeof(byte[]));
 string_arena_frame = Convert.ToBase64String(sendBytes3);
 data = conetado + "\n" + bateria + "\n" + tensao + "\n" + altitude + "\n" + roll + "\n" + pitch + "\n" + yaw + "\n" + velx + "\n" + vely + "\n" + velz + "\n" + estado + "\n" + atual + "\n" + desejado + "\n" +
 conetado2 + "\n" + bateria2 + "\n" + tensao2 + "\n" + altitude2 + "\n" + roll2 + "\n" + pitch2 + "\n" + yaw2 + "\n" + velx2 + "\n" + vely2 + "\n" + velz2 + "\n" + estado2 + "\n" + atual2 + "\n" + desejado2 + "\n";
 data2 = conetado + "\n" + bateria + "\n" + tensao + "\n" + altitude + "\n" + roll + "\n" + pitch + "\n" + yaw + "\n" + velx + "\n" + vely + "\n" + velz + "\n" + estado + "\n" + atual + "\n" + desejado + "\n" +
 conetado2 + "\n" + bateria2 + "\n" + tensao2 + "\n" + altitude2 + "\n" + roll2 + "\n" + pitch2 + "\n" + yaw2 + "\n" + velx2 + "\n" + vely2 + "\n" + velz2 + "\n" + estado2 + "\n" + atual2 + "\n" + desejado2 + "\n" + string_master_frame +
 "\n" + string_slave_frame + "\n" + string_arena_frame + "\n\n";
 //data = data.Substring(0, data.Length - 2);
 tcpServer1.Send(data);
 }
 private void timer1_Tick(object sender, EventArgs e)
 {
 enviar_data();
 }

Timer1 is 250ms atm. It was the best I could find. If I decrease, it lags even more.

Android Client

protected Void doInBackground(Void... arg0) {
 Socket socket = null;
 try {
 socket = new Socket(dstAddress, dstPort);
 //Scanner r = new Scanner(new InputStreamReader(socket.getInputStream()));
 BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
 BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
 //PrintWriter bw = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())),true);
 //PrintWriter bw = new PrintWriter(socket.getOutputStream(),true);
 while (true) {
 String line;
 int i = 0;
 while((line = br.readLine()) != null) {
 if (i < valores.length) {
 valores[i] = line;
 Log.e("MSG", valores[i]);
 i++;
 } else break;
 }
 publishProgress(valores[0], valores[1], valores[2], valores[3], valores[4], valores[5], valores[6], valores[7], valores[8], valores[9], valores[10], valores[11], valores[12],
 valores[13], valores[14], valores[15], valores[16], valores[17], valores[18], valores[19], valores[20], valores[21], valores[22], valores[23], valores[24], valores[25], valores[26],
 valores[27],valores[28]);
 }
 } catch (UnknownHostException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 } catch (IOException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 } finally {
 if (socket != null) {
 try {
 socket.close();
 } catch (IOException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 }
 }
 return null;
}
@Override
protected void onProgressUpdate(String... values) {
 super.onProgressUpdate(values);
 conetado.setText(values[0]);
 bateria.setText(values[1]);
 tensao.setText(values[2]);
 altitude.setText(values[3]);
 roll.setText(values[4]);
 pitch.setText(values[5]);
 yaw.setText(values[6]);
 velx.setText(values[7]);
 vely.setText(values[8]);
 velz.setText(values[9]);
 estado.setText(values[10]);
 atual.setText(values[11]);
 desejado.setText(values[12]);
 conetado2.setText(values[13]);
 bateria2.setText(values[14]);
 tensao2.setText(values[15]);
 altitude2.setText(values[16]);
 roll2.setText(values[17]);
 pitch2.setText(values[18]);
 yaw2.setText(values[19]);
 velx2.setText(values[20]);
 vely2.setText(values[21]);
 velz2.setText(values[22]);
 estado2.setText(values[23]);
 atual2.setText(values[24]);
 desejado2.setText(values[25]);
 Bitmap master_bitmap,slave_bitmap,arena_bitmap;
 byte[] decodedString = Base64.decode(values[26], Base64.NO_WRAP);
 master_bitmap = BitmapFactory.decodeByteArray(decodedString, 0, decodedString.length);
 master_frame.setImageBitmap(master_bitmap);
 byte[] decodedString2 = Base64.decode(values[27], Base64.NO_WRAP);
 slave_bitmap = BitmapFactory.decodeByteArray(decodedString2, 0, decodedString2.length);
 slave_frame.setImageBitmap(slave_bitmap);
 byte[] decodedString3 = Base64.decode(values[28], Base64.NO_WRAP);
 arena_bitmap = BitmapFactory.decodeByteArray(decodedString3, 0, decodedString3.length);
 arena_frame.setImageBitmap(arena_bitmap);
}
}

• Mathieu Guindon: Welcome to CR! Please edit your post to fix the code formatting; a code block requires 4 leading spaces. The easiest way to do it right is to indent the entire code 4 spaces in the IDE, select all, and then paste into your post - you'll want to preserve your original indentation, since indentation is one of the things reviewers can comment on.
• Bruno Costa: Go with @svick suggestion but you also should prefer implement Serializable objects instead of sending them as a string.

Answer 1

You should not send video as a string. Encoding to and decoding from base64 adds unnecessary overhead to the whole process, both in terms of time and in terms of bandwidth. Instead, you should send the video as binary data.

It's hard to say if doing this will fix your performance issues, but I think there is a good chance it will.