Converting javascript Integer to byte array and back

Question

function intFromBytes( x ){
 var val = 0;
 for (var i = 0; i < x.length; ++i) { 
 val += x[i]; 
 if (i < x.length-1) {
 val = val << 8;
 }
 }
 return val;
}
function getInt64Bytes( x ){
 var bytes = [];
 var i = 8;
 do {
 bytes[--i] = x & (255);
 x = x>>8;
 } while ( i )
 return bytes;
}

I am trying to convert a javascript number to a byte array and then back to a number. However the above functions produce incorrect output with a very large number.

var array = getInt64Bytes(23423423); 
var value = intFromBytes(array);
console.log(value); //Prints 23423423 - correct
var array = getInt64Bytes(45035996273704); 
var value = intFromBytes(array);
console.log(value); //Prints -1030792152 - incorrect

It is my understanding that javascript floats are 53 bits so it shouldn't be overflowing? alert(Math.pow(2,53)) works fine.

• Crozin: Bit shifts are always performed on signed, 32-bits integers.
• user75832: That makes sense, make it an answer and i'll accept. Thanks
• Melvin Sovereign: Since this question ranks high in google, it's probably worth pointing out that there are now array buffers: stackoverflow.com/questions/15761790/…

Answer 1

Using the hint provided by Susanoh13, here are the two functions that allow conversion of number from/to ByteArray:

longToByteArray = function(/*long*/long) {
 // we want to represent the input as a 8-bytes array
 var byteArray = [0, 0, 0, 0, 0, 0, 0, 0];
 for ( var index = 0; index < byteArray.length; index ++ ) {
 var byte = long & 0xff;
 byteArray [ index ] = byte;
 long = (long - byte) / 256 ;
 }
 return byteArray;
};
byteArrayToLong = function(/*byte[]*/byteArray) {
 var value = 0;
 for ( var i = byteArray.length - 1; i >= 0; i--) {
 value = (value * 256) + byteArray[i];
 }
 return value;
};

• DrArt: Sometimes in byteArrayToLong() byteArray[i] is treated as a string (browser dependent) and the resultant value is calculated wrongly. I have solved by *1: value = (value * 256) + byteArray[i] * 1
• Drew Noakes: You should specify whether this considers the bytes as containing signed or unsigned integers.
• ygoe: This doesn't work for negative numbers. Instead, large positive numbers are decoded.
• yeerk: DON'T USE THIS CODE. This code would only be valid if Javascript numbers were 64 bit unsigned integers. However they are 64 bit floating point numbers, which only store up to 52 bits as an exact integer. This means that while this code works for many bytes (ex, [0,1,2,3,4,5,6,7]), it will fail when all of the bits are actually used. Some failing cases are(where converting to a number and back again give a different result): ([255,255,255,255,255,255,255,255] => [0, 0, 0, 0, 0, 0, 0, 255]), ([1,2,3,4,5,6,7,8] => [0, 2, 3, 4, 5, 6, 7, 8]), ([1,0,0,0,0,0,0,1] => [0, 0, 0, 0, 0, 0, 0, 1])
• Seth Falco: This solution works for my use-case. However, it's worth noting that this assumes the input array is sorted from the least significant byte to the most significant byte. (i.e. [1, 128] === 32769) If your input bytes are from most to least instead, you can increment from 0 in the loop instead of decrementing to 0.

Answer 2

In JavaScript bit shifts (>>, <<) are always performed on signed, 32-bits integers. This leads to range overflow for large numbers.

• Timo Kähkönen: But if long (64 bit signed integer) is needed, what is the solution?
• Crozin: JS doesn't support 64 bits-long integers. See this: stackoverflow.com/questions/9643626/… and stackoverflow.com/questions/5353388/javascript-parsing-int64
• user4584267: @TimoKähkönen x<<y === x*(2**y) The right-hand solution can be used for 64-bit integers.
• Derk Jan Speelman: JavaScript (ESNext) supports 64-bit numbers now. Either signed and unsigned.
• timotheecour: @DerkJanSpeelman any link? unless you're referring to BigInt, but this isn't what I'd call a 64-bit number

Answer 3

try (** is power operator, << and >>> are bit-shift operators) - intFromBytes works only for arrays generated from positive integers

function getInt64Bytes(x) {
 let y= Math.floor(x/2**32);
 return [y,(y<<8),(y<<16),(y<<24), x,(x<<8),(x<<16),(x<<24)].map(z=> z>>>24)
}
function intFromBytes(byteArr) {
 return byteArr.reduce((a,c,i)=> a+c*2**(56-i*8),0)
}

function getInt64Bytes(x) {
 let y= Math.floor(x/2**32);
 return [y,(y<<8),(y<<16),(y<<24), x,(x<<8),(x<<16),(x<<24)].map(z=> z>>>24)
}
function intFromBytes(byteArr) {
 return byteArr.reduce((a,c,i)=> a+c*2**(56-i*8),0)
}
// TEST
let n = 40*2**40 + 245*2**32 + 194*2**24 + 143*2**16 + 92*2**8 + 40;
let b = getInt64Bytes(n);
let i = intFromBytes(b);
console.log(`number : ${n}`);
console.log(`int to bytes: [${b}]`);
console.log(`bytes to int: ${i}`);

• snickle: The value for y in getInt64Bytes needs to be wrapped in Math.floor or else it breaks for negative integers. For example, getInt64Bytes(-Number.MAX_SAFE_INTEGER) returns [255, 224, 0, 1, 0, 0, 0, 1] when it should return [255, 224, 0, 0, 0, 0, 0, 1].
• Kamil Kiełczewski: @snickle this procedure works only for positive integers - the Math.floor not fix the problem in intFromBytes (e.g. when we put minus in let n= -... in snippet) - currently I don't have time to fix this problem - but fell free create your answer which develop my answer and fix this problem.

Answer 4

In 2022, you should use Buffer to deal with bytes, and BigInt to deal with big integers.

So your code may look like this:

import { Buffer } from "node:buffer";
/**
 * @param {BigInt} x
 * @returns {Buffer}
 */
function getInt64Bytes(x) {
 const bytes = Buffer.alloc(8);
 bytes.writeBigInt64LE(x);
 return bytes;
}
/**
 * @param {Buffer} x 
 * @returns {BigInt}
 */
function intFromBytes(x) {
 return x.readBigInt64LE();
}
var array = getInt64Bytes(BigInt(23423423));
var value = intFromBytes(array);
console.log(value); // 23423423n
var array = getInt64Bytes(BigInt(45035996273704));
var value = intFromBytes(array);
console.log(value); // 45035996273704n

• Pablo LION: little endian is heresy🤣
• Anton Strogonoff: Buffer is not ES standard, Node-only.

Answer 5

Brainfuck-style Lodash version. Just 4 lulz! don't use it!

const uintToArray = (uint, size) => _.chunk(_.padStart(uint, size*2, 0).split(''), 2).map((a)=>parseInt(a[0]+a[1]))

Answer 6

Doing a bit shift is the same as multiplying by 2^(# of bits+1), so instead of shifting the bits val = val<<8, you can just do val = val*256. See if that works.

Answer 7

<html>
<head>
 <meta charset="utf-8">
 <title>Uint32_To_Byte_Array</title>
 <script>
 function body_Add(Msg)
 {
 document.body.innerHTML = document.body.innerHTML + Msg;
 }
 class Byte 
 {
 constructor(Value) 
 {
 this.Number = new Uint8Array(1);
 this.Number[0] = Value;
 }
 get Get() 
 {
 return this.Number[0];
 }
 set Set(newValue) 
 {
 this.Number[0] = newValue;
 }
 };
 class Uint32
 {
 constructor(Value) 
 {
 this.Number = new Uint32Array(1);
 this.Number[0] = Value;
 }
 get Get() 
 {
 return this.Number[0];
 }
 set Set(newValue) 
 {
 this.Number[0] = newValue;
 }
 };
 var Conversion =
 {
 Uint32_To_Byte_Array: function(Source_Num)
 {
 var Uint32_Num = new Uint32(Source_Num);
 var Byte_Num = new Byte(0);
 var Byte_Arr = new Uint8Array(4);
 for (var i = 0; i < 4; i++)
 {
 if (Source_Num > 255)
 {
 Uint32_Num.Set = Source_Num / 256;
 Byte_Num.Set = Source_Num - Uint32_Num.Get * 256;
 }
 else
 {
 Byte_Num.Set = Uint32_Num.Get;
 Uint32_Num.Set = 0;
 }
 Byte_Arr[i] = Byte_Num.Get;
 Source_Num = Uint32_Num.Get;
 }
 return(Byte_Arr);
 },
 Byte_Array_To_Uint32: function(Source_Byte_Array, Start_Position)
 {
 var Uint32_Num = new Uint32(0);
 var Multiplier = 1;
 for (let i = 0; i < 4; i++)
 {
 Uint32_Num.Set = Uint32_Num.Get + Source_Byte_Array[Start_Position + i] * Multiplier;
 Multiplier = Multiplier * 256;
 }
 return (Uint32_Num.Get);
 }
 };
 function Load_Page()
 {
 var Numbers = [0,1,257,4294967295];
 Numbers.forEach(Convert);
 function Convert(Item, Index)
 {
 var Uint32_Number = Item;
 var Byte_Array = Conversion.Uint32_To_Byte_Array(Uint32_Number);
 var Uint32_Number_Restored = Conversion.Byte_Array_To_Uint32(Byte_Array, 0);
 body_Add("Conversion: Source number: " + Uint32_Number.toString() + ", Byte array: " + Byte_Array.toString() + ", Restored number: " + Uint32_Number_Restored.toString() + "<br>");
 };
 };
 </script>
</head>
<body onload="Load_Page()"> 
</body>

• Valery Rode: Result:Conversion: Source number: 0, Byte array: 0,0,0,0, Restored number: 0; Conversion: Source number: 1, Byte array: 1,0,0,0, Restored number: 1; Conversion: Source number: 257, Byte array: 1,1,0,0, Restored number: 257; Conversion: Source number: 4294967295, Byte array: 255,255,255,255, Restored number: 4294967295
• Jure: This is exactly what I need! Just changing a bit :) Thanks!! :)

Answer 8

If you happen to be on Node.js, Buffer is the correct way to handle any byte array/stream in Javascript/Typescript:

https://nodejs.org/api/buffer.html

Although the docs are more comprehensive, Stack Overflow recommends code snippets here in case that link 404s, so here are a couple of the most important code examples in that doc:

// Creates a Buffer containing the UTF-8-encoded bytes for the string 'tést':
// [0x74, 0xc3, 0xa9, 0x73, 0x74] (in hexadecimal notation)
// [116, 195, 169, 115, 116] (in decimal notation)
const buf6 = Buffer.from('tést');
// Creates a Buffer containing the bytes [1, 2, 3].
const buf4 = Buffer.from([1, 2, 3]);

• Alberto Torre: Yes, to use buffer is the correct way to handle any byte array when is already a buffer or byte array. But the main question is: How do you convert an INTEGER to a byte array using Buffer? Buffer.from(INTEGER) is not allowed. TypeError [ERR_INVALID_ARG_TYPE]: The "value" argument must not be of type number. Received type number at Function.from (buffer.js:215:11)

Answer 9

Just so that someone coming here after node version 16.5 knows that the buffer API from node.js now provides a way to write/read various sizes of numbers to byte array and vice-versa using the

writeUIntXXX

apis. These cover integers, long, double, float, and in Big and Small Endian formats. So I guess we don't need to spin off our own solutions anymore.

Answer 10

Modern solution for modern browsers

// Number or BigInt to Uint8Array
NumberToUint8Array = number => {
 var array = [], bigint = BigInt(number)
 for (let i = 0; i < Math.ceil(Math.floor(Math.log2(new Number(number)) + 1) / 8); i++)
 array.unshift(new Number((bigint >> BigInt(8 * i)) & 255n))
 return new Uint8Array(array)
}
// Uint8Array to BigInt
Uint8ArrayToBigInt = Uint8Array => [...Uint8Array].reverse().reduce((prev, curr, index) => BigInt(prev) | (BigInt(curr) << BigInt(index * 8)))
// example
NumberToUint8Array(12312321312312309876543234567890n)
// returns: [155, 103, 65, 124, 45, 109, 221, 194, 90, 249, 152, 238, 210]
Uint8ArrayToBigInt([155, 103, 65, 124, 45, 109, 221, 194, 90, 249, 152, 238, 210])
// returns: 12312321312312309876543234567890n

Answer 11

This will work,

 let buf;
 if (num < 128)
 {
 buf = Buffer.from([num]);
 }
 else if (num < 256)
 {
 buf = Buffer.from([129, num]);
 }
 else if (num < 65536)
 {
 buf = Buffer.from([130, 256, num % 256]);
 }
 else if (num < 65536)
 {
 buf = Buffer.from([130, num / 256, num % 256]);
 }
 else if (num < 16777216)
 {
 buf = Buffer.from([131, num / 65536, num / 256, num % 256]);
 }
 console.log(buf.toString('hex'));

• Pablo LION: Could you please explain why or add some reference about the magic numbers 129,130,131? Also, there are two branches of num<65536 which I believe is a typo.