Re: [tlug] issues with format of double (or IEEE754)

[Jim <jep200404@example.com>]

On Wed, 26 Jul 2006 18:05:48 +0200 Michal Hajek <hajek1@example.com> wrote:

> Hello :)
> 
>  
> > > on an i386 mashine...

> Oh, I see the point now... it is not an actual 386 procesor. 

OK. 

> Sorry for the confusion. My mystake.  

OK. Precision is good in technical writing. 

> > > char result_normal_plus[17]="00000000000000000";
> > 
> > If that is global, keep in mind that result_normal_plus will 
> > initialized only _once_. 
>  
> yes, thank you for pointing that out. But I consider the inicialization
> as a side issue for the time being :) 

It is, but such wierdness raises my suspicion 
that you don't know what you are doing. 

You might have had a good reason for doing what you did, 
but when I see several wierdnesses, 
I start thinking very broadly about programming errors. 

> as for the number 17, it is just ad-hoc number. 
> I choose it only to be long enough. 

OK. 

By the way, 17 is a very special number. 
Did you know that it is the only random prime number? :-)

> Since later I put "end of string" mark at position 10, ...

... you needed something longer than just 10 bytes. I understood that much. 
I was curious about the remaining 6 bytes. 

> ... I did not bother to use the other 7 bits in any way. 

bits or bytes? (It's a rhetorical question. Don't answer it. Just be careful)

> I thought I can change it to lower number later, when everything works
> well. 

Yup. 

> > I am confused about which data is binary and which data is ASCII. 
> > I get the hazy impression that ibrd() does binary, 
> > so seeing result_normal_plus[ibcnt]='\0' looks odd. 
> 
> The function ibrd is implemented (by people writing gpib code for linux)
> as:
> int ibrd(int ud, void *buffer, long num_bytes); 

I know, I read the link you cited enough to see the function prototype. 
It has a strong binary feel to it, not a string feel to it. 

> The problem is, that first 16bit read from the nanovoltmeter are two
> asci characters (#0 header) which I do not want. Being a poor programer
> myself, I could not find any other solution than just take everything
> and store it in a string. 

The concept of a string in C has a very strong convention of being 
NUL terminated. Because you have binary data (which might have a 0 byte 
which could prematurely terminate data), 
you can not handle the overall data as a NUL terminated string. 
That's part of why ibrd() is passed and returns byte counts. 

result_normal_plus[ibcnt]='\0'; likely causes no problem, 
but is a strong indication that you don't fully understand the 
difference between handling arbitrary binary data and NUL terminated 
strings in C. 

Although the wierdnesses I see do not reveal any definite bugs to me, 
they indicate low understaning of the data and C, so this raises the 
probability that you have a programming bug somewhere. 

As far as accessing the first two bytes as characters, 
here are a couple of valid ways to handle them: 

1. You can deal with individual characters as just individual characters
   (that are not in a string). I.e., you can directly access the first 
   two characters as result_normal_plus[0] and result_normal_plus[1]. 

2. Access them as a NUL-terminated string. The NUL terminator would likely 
   clobber the following binary data payload, so either the two character 
   header or the following binary data would have to be copied elsewhere. 
   I prefer copying the header to a NUL terminated string: 

      #define HEADER_LEN (2)
      char header[HEADER_LEN+1];

      header[0]=result_normal_plus[0];
      header[1]=result_normal_plus[1];
      header[2]='\0';

   or

      #define HEADER_LEN (2)
      char header[HEADER_LEN+1];

      {
         int i;

         for (i=0;i<HEADER_LEN;i++)
             header[i]=result_normal_plus[i];
         header[i]='\0';
      }

> Than 

then 

> point another variable (a pointer to
> double) to the right place in the string (kind of saying "from this
> point on take the following 8 characters as a double floating point
> number").

I fully understood: 

   double *presult;
   presult=result_normal_plus+2;

It's a valid way to do stuff. The concern is that the exact format 
from the meter is different than the floating point format used by 
your C program. Since reading on Wikipedia that the '387 fully 
supported IEEE 754, this is only a small concern. 

> > Parse it by hand. Calculate what you believe the numbers to be. 
> 
> Out of despair, I have already started to work on that :) 

Good. 

> But the results are not so easy to obtain... :) 

I fully understand. I have done so before standardization when I 
was trying to understand undocumented floating point formats. 

> > (BTW, where does ibcnt come from?)
> 
> ibcnt is set by ibrd() to the number of bytes read. 

OK. I see now that I RTFM more. 

> So I use the information to cut the string in the right place. 

You have ibcnt binary bytes. 
You do _not_ have a nul terminated string. 
Stop thinking in of the mixed ASCII and binary data returned by ibrd() 
in terms of a nul terminated string. 

> Sometimes it could be 0, if something goes wrong with the devices on
> gpib bus or the bus itself. If all works ok, than the ibcnt is same as
> the number you give to ibrd as last parameter. 

Yup. 

Since you are dealing with binary data, 
drop the result_normal_plus[ibcnt]='\0';. 

> > First, make it work right, then make it work correct. 
> ...
> > Did reading in ASCII, then converting with atof() work correctly? 
> 
> yes, it works well. No problem. 

Then use it. 

Even if you had trouble with it, it would be easier to fix 
because it is already in a human readable format. 
Also, the maintainance would be easier, because atof() 
does not rely on IEEE754 being the native format and 
so buys you portability. Portability is a good thing. 

BTW, when you get ASCII data, you probably _should_ deal with 
the ASCII data as nul-terminated strings. 
(Clobbering a comma delimiter with '\0' is probably OK)
In other words, when dealing with binary data, 
result_normal_plus[ibcnt]='\0' is very wierd. 
When dealing with a nul-terminated string, 
result_normal_plus[ibcnt]='\0' is required. 

> > How much time does it take to do it _correctly_? 
> well, I intend to run a measurement in a loop. One such loop has about
> four atof() and whole together takes about 0.8 seconds to finish. 

0.8 seconds? That is a very long time, even for a 16MHz '386. 
atof() is NOT your speed problem. Just make sure the string 
you give to atof() is nul terminated. 

> I do not know how much time I can save with avoindig atof(). 

The speed of atof() is not your problem. 

page 15-5 of the Keithley manual is not clear about 
which end the speed degradation could be from. 
Although speed is mentioned along with BASIC, 
there could be speed issues inside the meter itself. 
Even so, I still think it likely that ASCII format 
is not responsible for your speed problems. 

> From profiler I
> cannot source the information, since atof() is buried inside another
> function. But if I run the program in gdb, than my personal subjective
> impression is, that atof() takes considerably more time than for example
> ibrd(). 

ibrd() is likely a "blocking" call. 
It likely spends most of its time asleep waiting for input. 

> I do not know any other way how to find the information :( 

Since you know how to use a profiler, 
you can put atof() inside an encapsulating function, 
just so that you can measure the time of the encapsulating function. 

I.e., in your program call debug_atof() instead of atof(), where: 

   double debug_atof(const char *nptr)
   {
      return atof(nptr);
   }

then run your profiler on debug_atof(). 

> > Is the ASCII format and atof(), the fastest way that works _correctly_?
> > Avoid premature optimization. 
> 
> Well, the program is rather simple. 

Good. Simplicity is a very practical high virtue. 

> Maybe this is a wrong path from the
> point of view of having work done. But I also want to know how things
> work. So my line of reasoning was based on these assumptions :
> 
> 1. the manual (which I believed) says the nanovoltmeter can return
> values in binary IEEE754 double precision format

OK. 

> 2. it should be possible to exploit this feature and avoid atof(). 

Yes. 

But where does your C compiler _guarantee_ that it likes IEEE 754? 

(Your C compiler probably _does_ grok IEEE 754, 
but where is that guaranteed?)

> When the program does not work like I expected, I tend to look for the
> errors in the program first and try to stick to assumptions as long as
> possible. 

Excellent! 

Considering the wierdnesses I've seen, it is likely that you have bugs. 

> Only [after] I understand why the assumptions are wrong, I am
> satisfied and can leave them aside happily :) 

Me too. 

> > Why do you care about speed? How much speed is enough? 
> 
> This is difficult to say. The nanovoltmeter can make a conversion very
> quickly, but for my purpose about 16.6ms would be ok. 

atof() even on a 16MHz '386 is not your speed problem. 

In one loop, there
> are 4 measurements. Together less than 0.07s.  For my math model, 7 loops
> per second would be great. So there is about 0.5s for all the remaining
> work on the computer side. But for a computer 0.5s could be a lot of
> time, right? :) 

0.5s is a very long time. 

> I do not know if I can reach this value, but every speed up will be
> good. I want to go to the possible limit of hw.  Maybe I will have to go
> realtime, but for the first approximation, I want to try without it. 

Your program can be made to be damn fast. 

> > just worry about correctness and making your program easy to understand. 
> 
> [A]gree[d]. At least easy to understand is definitely my goal too. 

If you don't understand your code, then it is almost guaranteed to have bugs, 
especially considering the wierdnesses that I've seen in what little 
you've shown. 

> Unfortunately, right now the timing of my work is a problem. I have
> about 24 hours to find a solution and than I am leaving for 7 day
> holiday.  

Make it work _right_, then make it work _fast_. 

Always code for clear understanding. 

(Unless you forgot a nul-terminator for the argument), 
atof() is not your speed problem. 

By the way, is there a speed penalty in the _meter_ for outputing 
in ASCII? If you don't try to parse or display them, how fast 
can you read numbers from the meter? If you can read fast, then 
your speed problem is in your full program. If you can not read 
fast, then maybe there is a speed problem in the meter.