Datatypes: Bitoperations With Floating Point Numbers
Hello,
from my point of view, I'm having a really tiny problem, but I am not able to solve it by myself. Is it really that hard?
My final goal is to be able to split 32bit floating point variables (as Sinlgle) binary into 4 seperate bytes (which is necessary to work with a given transmission protocol). Therefor I am trying to read and write single bits in a floating point variable (e.g. Single).
I already succeeded with 32bit variables of the type Long. So my problem would already be solved, if you can help me to write the 32 bits of the 32bit Single variable bitwise into the 32 bits of a 32bit Long variable.
Code:
dim s as Single
'dim s as Long
Dim bit(0 To 30) As Boolean
s = 12.34
's = 12
For j = 0 To 30
bit(j) = CBool(s And 2 ^ j)
Next j
This code works fine for the commented line, when the variable s is of type Long. When s is of type Single, VB automaticly casts the 12.34 to an Integervalue of 12.
So how can I split (and recombine) the 32bit Singlevalue into Bytes (8bit)?
Thanks in advance for your effort, Parity
View Complete Forum Thread with Replies
See Related Forum Messages: Follow the Links Below to View Complete Thread
Floating Point Numbers
I have a binary file containing vertex data. I need to read this information from the file as a 32 bit float for each x, y, and z position. No matter what I declare the variables I'm storing this information into, the program constantly returns the values as a Double.
Is there a way to do this in VB?
All help is greatly appreciated,
SoDA
Floating Point Numbers
Admittedly this is a more general than just a vb-specific question: could someone give me a good address for a tutorial on floating point numbers, i.e. how single and double precision values are coded? I don't ask you guys to post an explanation as I guess it might take a good deal of writing.
Thanks in advance.
Floating Point Numbers
Anyone know where I can get some information on Floating Point Numbers?
Like: What are they?
What do you use them for?
Thanks
Floating Point Numbers
How can I get the user to enter a floating point number? If I use a text box then the val function, I believe errors occur.
Floating Point Numbers
Hello,
I would like to return the floating point remainder of a calculation and use it for further calculation
For instance 60000/256 = 234.375 I would like to strip the 375 and add it to a variable any help will be very much appreciated, and thanks in advance fo any replies.
Formatted Output Floating Point Numbers
Does anyone know how to output a floating point numbers in scientific notations ?
Such as: 1 will be displayed as 1.0E0
0.1 will be displayed as 1.0E-1
and so on.
Thanks!
Limiting Displayed Floating Point Numbers
I want all my displayed data to be limited to 4 decimal places. I was using the format function. However I have come across a problem
I have a small number e.g. 6.07222318649292E-07 If I limit this with the format function, I get 0., which is clearly wrong.
What I would like is 6.0722E-07.
Here is how I am using the format function
VB Code:
my_text = format(my_number,"0.##")
Any ideas?
Cheers
Fast Sorting For Floating Point Numbers - Radix Or Merge?
Hi Everybody
I am not so sure if this is the right place for it because I presumed that sorting applies to all - databases, Excel etc etc. So sorry if this is in the wrong place.
I understand that generally (with some restrictions) MSD Radix sort is perhaps the fastest (O(N)) sorting algorithm for Strings. Whilst LSD Radix sort can be effectively used for Intergers and Longs, for Floats it becomes a bit ugly.
Given the complexity involved with using Radix sort for floating point numbers, can it safely taken for granted that one would ordinarily be better off using Merge sort for numeric data and Radix sort for alphanumeric data?
The reason for this query is that I am currently trying to modify the generalised code provided by Daving Ring for Radix sort specialised for Strings on vdevx website to suit my requirements within Excel using VBA (Excel) and whilst I was on it, I might be able do another version of the same (or similar) for numeric data as well.
If someone feels that it might be worth giving it a go, can anyone suggest where can I get a base code in VB which I can go about altering to suit my needs?
Best regards
Deepak Agarwal
Beyond Floating Point
I have a 'program' written in Excel that uses cell formulas to come up with an answer. In itself, it isn't that exciting, the thing is that there are about 16000 rows and 50 some odd columns. Needless to say it takes a while to calculate. I have converted this to Visual Basic, however I get different answers, generally a couple of percent off. I think this may be because Excel stores 30 digits wheras VB only has 15. Is there any way I can store 30 digits in VB. If not, what language would you suggest I use that could handle this. Thanx.
Grover
String To Floating Point
Is there an easy way to convert a string = "3,5" to the floating point value 3,5 (or 3.5 for those outside of Norway?
Floating Point Variable To Hex Value
I am trying to show the hex values of a floating point number but having a problem.
My code:
Dim x As Single
x = CSng(Text1)
Label1 = Hex$(x)
Label2 = CSng("&h" & Label1)
If I enter 1.1 then label1 shows 1 and label2 shows 1
if I enter -1.1 then label1 shows FFFFFFFF and label2 shows 1
I searched the site and ran into the copymemory api and tried it but it crashed the program. I guess it couldn't move it to a label.
Is there a way to get the full hex value of a floating point variable?
Thanks in advance,
16-bit Floating Point Variables
Hi, does anyone know of a way to use 16-bit floating point variables/arrays in Visual Basic 6.0? I'm processing great amounts of data, so 32-bit floats are not an option (to memory intensive).
Perhaps there's a way to use integer variables/arrays to store it, but process them differently...
Floating Point Issues
I'm trying to work with a file that includes floating point information in 32 bit form. I need to read that information in and display it as a decimal number, allow the user to change that decimal number to another decimal number and then put it back into the file in 32bit form.
Getting it in and out of the file isn't a problem - I have it in a byte array , and once I've got it back into a byte array I don't have the slightest problem putting it back into the file.
So lets say that I have a 32 bit floating point number in a byte array as follows:
41 9E 66 66
Now I happen to know that converted to decimal that is 19.8; but I've no idea how to program a routine to make the change.
And equally, if my user enters "3" into a text box I have no idea how to turn that into the byte array 40 40 00 00.
Any help much appreciated,
Rob
Function To Cut The Floating Point Value
Hi, I am looking for the function that cuts the point in numbers.
EXAMPLE:
a = func("5.7")
a becomes 5
See what I mean, just want to cut the point in a real/floating point value.
I tried using an Integer but that just rounds it up (I do not want to do this)
3.9 becomes 4 (<-- NOT RIGHT)
3.9 becomes 3 (<-- RIGHT)
Thanks
Check Floating Point
I want to make an error checking to check whether the value in the textbox is a floating point or not. Please help me...........
Bytes To Floating Point
I want to convert 4 bytes in row to a single precision number. These are read from a file by means of Get commands but it's in big endian order so I must reverse them.
Specifically, I'd like to know if there is a similar function to ntohs and ntohl for single and double precision floating point numbers. Or is there any command to read a file backwards?
Floating Point Error
Hi VB Programmers,
Here is my short code
Code:
Dim x as Double
Dim y as Double
Dim z as Double
x = 10.0005
y = 10.0004
z = x - y
MsgBox z
When I execute it, it show unexpected 9.9999999999759E-05.
How can I get the expected result such 0.0001?
Thank You.
8 Byte Floating Point Variable
Hello everyone im taking a class were we are doing some programming and basically our 1st program is a simple interest calculation program, but he said we need to have double precision 8 byte floating point variable. I have no idea how to do this in visual basic 6.0 can someone tell me how to set this up so that all my numbers are in this format, thanks so much. Im really new at VB
To Transform IEEE-P 754 80-bit Floating Point
Hallo experts!
I import the 80bits as string to VB. Is there a routine/function to transform the values to VB type "double". (of course by loss of precision) and vis versa.
Thanks and regards
Hans
Floating Point Scientific Notation
Hi,
I'm designing a function shich returns a floating point value "single". how can I forca Visualbasic to print it normally without converting to the scientific notation.
MNT
Long 2 32bit Floating Point
I'm useing this bit of code to get values stored as 32bit floating point numbers in a file:
Private Sub Command1_Click()
Dim vals(3) As Long
Dim sngOut As Single
Dim lngIn As Long
vals(0) = Val("&H" & Text1.Text)
vals(1) = (Val("&H" & Text2.Text) * 256)
vals(2) = (Val("&H" & Text3.Text) * 65536)
vals(3) = (Val("&H" & Text4.Text) * 16777216)
Label1.Caption = vals(0) + vals(1) + vals(2) + vals(3)
lngIn = Label1.Caption
sngOut = 0
CopyMemory sngOut, lngIn, 4
Label2.Caption = sngOut
End Sub
It works fine for some values e.g the hex values 00,00,00,3F should retern 0.5 and they do, but some overflow the Long can anyone think of a way round this?
Floating Point Type Overflows
I'm writing a function that computes taxes, and when I input a number over 33,000 it gives me an overflow error. Obviously, it is because of the byte size of the datatype, but how do I allocate more memory for my double?
-Steve
Invalid Floating Point Operation
Hello,
I am experiencing a hard to catch error in my VB6 application. After running for several hours, it stops working correctly (but does not crash), also a messagebox pops up, titled VB6, with in it the text: "Invalid floating point operation".
I have no clue what might be causing this error, and it does not seem to be reproducable. I just happens, after some hours of running. No other application is active, the PC, running Win 98, is as clean as it can be.
Maybe anyone recognizes this kind of problem and have a solution?
Regards,
Bas
Floating Point Over A Serial Port
I would like to send single precision floating point numbers over the RS232 buss. There should be a way to do this directly by sending 4 bytesper number. Does anyone have any examples of this. I've seen the LSetinstruction used to move data between different data types on a byte bybyte basis if the types are the same size. This implies that a floatingpoint single precision number could be converted into 4 characters andthese characters then could be sent over the serial port. There areexamples athttp://support.microsoft.com/support/kb/articles/ Q235/8/56.ASP?LN=EN-U&SD=gn&FR=0&qry=floati ng%20point&rnk=12&src=DHCS_MSPSS_gn_SRCHSPR= VBB but I can't seem to get it to work in my applications.
Precision Of The Floating-point Types?
I need to find out the precision of the floating-point data types viz. Single, Double, and Decimal i.e. the maximum number of decimal digits ( i.e. base 10 digits ) that may be specified in thier literals. As this is not documented I thought I'd keep on enterring ever larger numbers in the Immediate Window until I got an overflow error and then count the number of digits. However to my frustration, as soon as I exceed 15 9' i.e. 9999999999999999, it gets converted to 1E+16.
How can I find out the precision and scale of these three data types?
8-byte Floating Point Number
Hi All,
If I have an 8-byte floating point number in hex how do I convert it to a
double?
0 0 0 0 0 98 C1 3F this should be about .13746
Thanks
Dan...
Single Floating Point Conversion
I'm trying to convert a 4-byte Hex value which supposed to be represented as a single floating point value under the IEEE 754. The following is what I did, but the result is not correct.
Dim test as Single
test = &HC2934000
test is eqaul to -1.030537E+09, but it suppose to be 1.150390625E+06 which is -73.625
Any help is greatly appreciated. Thanks!
Floating Point In MSHFlexGrid Control
Hi you all,
How do i set the floating point precision of a numeric field in the recordset of a MSHFlexGrid control. I am activating a query that rounds the value by 2 and still on the MSHFlexGrid control it shows as 3.4599999999999999 for example.
How do i fix it ?
Convert Hexadecimal To Floating Point
Call me an idiot, but for the life in me i cannot figure out how to do a simple conversion......
I need to convert
FB7E0E41 to 8.9060
32 bit IEEE Hex to decimal
can ANYONE Help ME!!!!!
Thanks
Nick Stone
Changing A Floating Point Number To An Integer
I have an long equation that returns a value as a long floating point number my question is there a way to convert this to an integer value so that it can be compared to an integer number that is input by the user.
This is the equation.
Text7.Text = 90 - (Atn((32.2 * 180) / ((Text1.Text / 5280) * 3600) ^ 2) * (180 / 3.14159265359))
it usually returns a number like 1.45838568387579 I know converting it into an integer will lose accuracy but that doesnt matter.
Convert Bytes To IEEE-754 Floating Point
I have 4 ordered bytes values (0-255) which I receive from a device's output. These 4 bytes represent a 32bit floating point value in IEEE-754 format. I need a VB6 function which converts the 4 bytes into this value. Thanks in advance.
Simple Way To Generate A Floating Point Error
Here is something that might leave you with bald patches on your head at the end of the day. A simple decimal, less its integer portion, results in a floating point error in the 11th decimal place.
A = 2436116.81
B = Int(A)
C = A - B
Result:
C = 0.810000000055879
I mean seriously, I can do this in my head. My pocket calculator can do it too. Why can't Visual Basic 6.0 do it right?
How To Distinguish If The Numerical Value Is Integer Or Floating Point?
Hi, I'm a newbie of VB, I'm working on my VB assignment at this moment and encounter a problem.
In a part of my assignment, I need to make a Textbox and type a numerical value in. And I need to know if this number is Integer or floating point value.
Any VB build-In function can do that? or I should check if the string value contain "."?
Thanks for helping
Floating-point Inexact Result Error
I am running vb6 with sp5. I am making a simple text editor. The application runs perfect in the ide. however when I compile, run, end exit the program I get this error. I looked it up on MSDN and discovered this err was suposed to be fixed in sp4
I am using a coolbar controll (microsoft windows common controlls-3 6.0(sp5)
any ideas of how I can fix this?
Floating-point Inexact Result Error...
When compiled, I get the following:
Unhandled Exception at 0x7c81eb33 in <myprogram.exe> 0xC000008F, floating-point inexact result...
Is there anyway to track this down? It doesn't happen in the VB6 IDE at all, just from the compiled exe...
Data Types: Double And Floating Point
Hey guys,I was wondering is there any other vairable type to hold a fractional number other then Double?And what does "floating point" mean?Also what is "fixed point" then?
Thanks in advance,
Misha
Floating Point Error Raised After END Statement.
I have an application which, in certain circumstances, unloads all forms and is then terminated with an END statement. However on some (but not all occasions) when this code is run on machines with a Win32 debugger installed I get the following error message.
---------------------------
Messenger:Messenger.exe - Application Error
---------------------------
The exception Floating-point inexact result.
(0xc000008f) occurred in the application at location 0x77e87e08.
Click on OK to terminate the program
Click on CANCEL to debug the program
---------------------------
OK Cancel
---------------------------
Does anyone have any ideas as to how I may resolve this?
VBVM60 Error / Floating Point Exception
Hi All,
I've made an small application in which i'm using hook, unhook and WndProc. When unhooking at the exit (through exe), it shows an error in VBVM60 and if NT machine then error is Floating point exception. I'm using SetWindowLong for unhooking.
Thanks.
Byte Array To Floating Point Number
I have a binary array which I wish to convert to a 4-byte floating point number.
I have searched but can't find a way to do this. Any help would be appreciated.
Using VB6. Thanks.
Help - Floating Point Inexact Result - Not Coolbar.
I have a very simple form (simple now that I've stripped everything else out of it) that consists of a timer and a winsock control. The winsock listens for incoming connections and the timer fires to see how long ago the last connection was. There are no other controls. I had a tray icon that popped up a menu to end the program, but now (for debugging) you click the form to end it.
When the program ends, I get :
The exception Floating-point inexact result.
(0xc000008f) occurred in the application at location 0x7c59ba9d.
I have disabled the timer and removed all code from the Load event of the form so that the winsock is not active. Essentially there's nothing going on, but I always get this error.
In the development environment, issuing an End command either in code or from the immediate window hangs VB and I have to kill it form task manager. I have no float, single, or double data types in the program and no math.
I have used the winsock control for years and have never seen this. Any ideas?
Thanks,
Bob
Edited by - rlisak on 2/5/2004 8:17:23 AM
Floating Point Problem In SQL UPDATE String
I'm using the following string to update some records in a MSAccess table.
Code:"UPDATE DatiAnnuali SET saldatt = " & dati(dovesto, 2) & " WHERE disegno like '" & currentdisegno & "' AND anno = " & I
It works just fine until one of the variables contains a value with decimals. What follows is how the Local Variables window resolves the string when it stops for the syntax error.
"UPDATE DatiAnnuali SET saldatt = 512,5 WHERE disegno like '830847' AND anno = 2005"
By trial and error I realised it does not like the comma in "512,5". That number is coming from dati(x,y) which is a Single (Precision) array, and its value populates a Single Precision field in the db table.
How do I overcome this???
Thank you!
Cam
Validation Between Integer And Floating Point(single)
hi,
does anone know how to validate/check between a integer(without the '.') and Single(floating point with '.') data type? eg. i have a text box for user to enter a value, how to check what type of value(integer or single) does the user enter without check it manually as we treat it as string?
Thanks for help.
Inserting A Floating-point Number Into An Msaccess Database?
hi!
can someone tell me how to insert an floating-point number into an msaccess database via ado? i need the syntax of the sql statement.
in msaccess it works using the following statement:
INSERT INTO table (field) VALUES ('12,33')
but using the same statement in vb with ado only the number 12 will be inserted, the positions right of the comma are 0.
|
TRACKER
