Wrox Programmer Forums
Go Back   Wrox Programmer Forums > Microsoft Office > Excel VBA > Excel VBA
|
Excel VBA Discuss using VBA for Excel programming.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Excel VBA section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
 
Old January 5th, 2009, 02:02 PM
Friend of Wrox
 
Join Date: Jan 2006
Posts: 131
Thanks: 10
Thanked 0 Times in 0 Posts
Default Problem when subtracting variables

Hi,

I feel a bit embarrssed to ask this one. But nonetheless it has out-foxed me...

I have 2 variables (both 'double' data types), where one value is subtracted from the other.

When I perform the sum of 3.63 - 3.57 at run-time or in the immediate window I get an incorrect result of '6.00000000000001E-02', whereas I'm expecting '0.06'

I have tried wrappng the Val and CDbl functions around the variables
(For example: Val(dblA) - Val(dblB))
- but I'm still not getting the same result.

Note
If I choose the '+' or '*' or '/' operators, I get the appropriate result in each case.

Any ideas?

Thanks in advance,
__________________
Neal

A Northern Soul
 
Old January 5th, 2009, 04:06 PM
Friend of Wrox
 
Join Date: Feb 2007
Posts: 163
Thanks: 0
Thanked 2 Times in 2 Posts
Default

There are work around options for this problem like declaring the precision as displayed option or rounding the number normally. For more information on why this is a problem review this MS KB article:

http://support.microsoft.com/kb/214118/EN-US/
 
Old January 6th, 2009, 01:06 PM
Friend of Wrox
 
Join Date: Jan 2006
Posts: 131
Thanks: 10
Thanked 0 Times in 0 Posts
Default

Thanks for the help. Very interesting article.
__________________
Neal

A Northern Soul





Similar Threads
Thread Thread Starter Forum Replies Last Post
subtracting 1 day from the current date Jeff C# 2 March 12th, 2009 01:16 AM
Subtracting Quantities rsm42 ASP.NET 1.0 and 1.1 Basics 2 January 7th, 2007 10:44 AM
subtracting period columns in a matrix PhilVan Reporting Services 0 December 14th, 2006 09:57 AM
Subtracting a value from a previous value Wes Access 8 March 29th, 2006 12:17 PM
negative number returned when subtracting gavmc Classic ASP Basics 2 November 29th, 2005 06:34 AM





Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.