drezzzz
20th January 2009, 18:26
Good day to all,
I have a problem with a string conversion :
I am doing :

sql = sql.parse("SELECT * FROM tiitm001")
sql.exec(sql)
rowcount=0
while ( true )
on case ( sql.fetch(sql) )
case eendfile:
break
case 0:
oki=seq.puts(strip$(tiitm001.item)&"|"&str$(tiitm001.copr), filpo)
| etc ...


The problem is that the output of tiitm001.copr is rounded ... I get "2230.305411" when it is suppose to be "2230.30541117".

Where is the round applied and how to avoid it ?

Thank you,
Fred

mark_h
20th January 2009, 18:37
Search on edit$ - that might be what you are looking for.

Hitesh Shah
20th January 2009, 18:50
Though value in debugger is rounded to 6 digits (as per domain) , internally its still what u expect and what's in db . U can check this by comparing the variable to these values in debugger .

zardoz
20th January 2009, 18:54
There is the limit in 6 decimals in fields in Baan. Thats the reason why there is the rounding. In the DB, the decimals could be more but you cannot view in the debugger, as Hitesh said.
I don't know if this is corrected, but also there was a limit for 4 decimals in some Baan IV tools domain definitions. I know this because I had a customer (during the conversion ITL Lira --> EUR) that wanted 6 decimals for cost price. This was done by me putting the value 6 directly in the tools table that contains the domain definition :)

drezzzz
20th January 2009, 21:58
The domain definition tells :
Double(15)(4) $$ in [-999999999.9999,999999999.9999]

The values from db dump is with 0 to 8 decimals. It really seems to come from str$() and edit$() or the query treatment ...? or sql.fetch() ?

I tried edit$() and didn't find a way to makes it work well. I tried :
- edit$(tiitm001.copr, " ") | which is 15 spaces as the domain def.
gives '################' seems like an overflow to me ...?
- edit$(tiitm001.copr, "999999VD99999999")
gives the rounded value but with added 0 ...

mark_h
20th January 2009, 22:46
Well I normally do not go over 2 places for a cost, but you might try sprintf$. Something like buff$ = sprintf$("%S %6.8F", tiitm001.item, tiitm001.copr). Not sure if I got the format correct since I do not have access to anything to check it right now.