Strange Aritmetic overflow

Strange Aritmetic overflow

• gm64

  I declared that tB(Offset + 3) = &H3F.

  &H3F << 8 = &H3F00 so says Windows Calculator

  tB(Offset + 3) << 8 Causes an error.

  tB is a byte array.

  Please post only if you can help me. Thank you.

  
  
  

• s_ruchit

  I have a theory troy. We dont know what the datatype of TB is. For the last year people have been explaining to you that bytes have eight bits.

  If tb() is a byte array

  (tB(Offset + 3) << 8)

  Will never ever work and will inevitably generate an over flow. But we can't know because you didn't specify TB's datatype.

  
  
  

• Ross B.

  With vb if you do:
  
  Dim a as Byte = &H3F
  Dim b as Byte = a << 8
  
  Then the result will be &H3F, because ... quote MSDN:
  
  "To prevent shifting by more bits than the result can hold, Visual Basic masks the value of amount with a size mask corresponding to the data type of pattern. The binary AND of these values is used for the shift amount."
  
  For a byte it uses 7. 8 AND 7 = 0, so in effect a << 8 is the same as a << 0 for a byte (in vb.net).
  
  So:
  tB(Offset + 3) << 8 ' evaluates to a Byte.
  
  will just return a Byte equal to tb(Offset + 3), you need to cast the Byte to an Int first:
  
  CInt(tB(Offset + 3)) << 8 ' evaluates to an Int
  
  The size mask for an integer is 31, 8 AND 31 = 8 so it really will shift it 8 bits.
  
  That has nothing to do with the overflow, the bit shift operator can't cause that.
  
  but &HFF + (&H3F << 8) does work
  
  - here all the types are Integers already.
  
  From this I'm guessing that the expression above, put into context in your code is something like:
  
  someByte = (tB(Offset + 3) << 8 ) + &HFF
  
  So...
  The expression in the brackets evaluates to a byte, and you try to add the maximum value of a byte to that, which you can't do as a byte can't store more than &HFF
  
  someInt = Cint(tB(offset + 3) << 8) + &HFF
  
  will work.
  
  

• pappascd

  Thank you for the explanation.
  
  

• Thegamingchoice

  "&H3F << 8 = &H3F00 so says Windows Calculator

  If TB is a byte array each element has 8 bits. That's all and they've had eight bits for the last sixty years.

  Windows calculator doesn't display in bytes.    Your result &H3F00 has 16 bits and a byte has eight. The largest value a byte can display is &HFF . That's 1,2,3,4,5,6,7,8 bits. 16 bits will not fit into eight bits because 16 bits is eight bits more than a byte.

  I hope that was helpful.

  
  
  
• Strange Aritmetic overflow