Convert string to double

[Satoshi Yoda]

Good day! :3

Some time ago I had an issue with converting NaNs:
https://www.cplusplus.com/forum/windows/279382
https://github.com/msys2/MINGW-packages/issues/9659

At that time my problem was solved when I moved from atof() to stod().

And now I noticed another problem on that account:
stod() is slower. And not just "slower", but A LOT slower, like ~x100 times slower.
I googled a little, found this:
https://stackoverflow.com/questions/11445700/why-is-string-to-number-conversion-so-slow-in-c
Looks like stod() tries to use some "regional standards" from OS. So, every time I want to convert string to double there is a "request" to Windows about "What are correct decimal separator?" Or something...

That situation leads to 2 major problems:
1) I want to parse text files with a lot of doubles in text form, and the speed is important.
2) Maybe there is some operation systems and/or configurations, on which my code won't work, because my text files will contain "wrong" decimal separators. This is just awesome =)

So, now I can not use stod() because of that. And I can not use atof() because it does not parse NaNs on some systems (my system for example).

The question: is there some way to convert string to double, that is not system-dependent? Preferably fast. Preferably without using external libraries. And preferably in modern C++ slyle.
Or should I invent my own converter? xD

[kigar64551]

Did you actually test that stod() is much slower than atof(), or just read about it being slow?

This could come out quite different between different compilers/platforms.

And did you run many conversions in a loop, to make sure that not just the first conversion is slower?

Also, what exactly do you mean with atof() doesn't handle NaNs? It does not correctly parse the specific string "NaN"? Well, that special case would be rather easy to handle yourself, if that is all, I think.

Finally, using the function scanf("%lf", &outvar) would be yet another option to give a try.

[EDIT]

Yet another option, if using third-party code is possible for you:
https://github.com/lemire/fast_double_parser

[Satoshi Yoda]

Of course I tested it. That is how I noticed that in a first place. Then I read about it.
Just out of a blue my program starts to load like 5 seconds instead of 200ms or about that. Yes, that is not first conversion, that is a loop(s).
And when I change just one line of code from stod() back to atof(): I am back to 200ms again.

About NaNs is described in old topic. In a nutshell: it parse "nan" as 0 on my system.

Thank you for fast_double_parser link, looks like I should give it a try ^_^

Also, scanf() is interesting to...

[kigar64551]

n a nutshell: it parse "nan" as 0 on my system.

If that really is the reason why you can't use plain atof(), you could try something like this:

inline double my_atof(const char *const str)
{
    return stricmp(str, "nan") ? atof(str) : NAN;
}

The above code doesn't skip whitespaces before the "nan" string. If that is desired (because number parsing functions generally ignore leading spaces), it would be easy to add that to the code too:

inline double my_atof(const char *str)
{
    while ((*str) && isspace(*str))
        ++str;
    return stricmp(str, "nan") ? atof(str) : NAN;
}

[seeplus]

The fastest way to convert is std::from_chars() introduced with C++17
https://en.cppreference.com/w/cpp/utility/from_chars

[lastchance]

@Satoshi Yoda,
Any conversion has overheads. But:
(1) Any conversion you do will pale into insignificance compared to the speed of reading from file.
(2) Why don't you simply read them in as doubles in the first place? Throw an exception if it fails to read.

[jonnin]

we had this conversation a little while back and from_chars won, as noted. It was at least as good as a hand-rolled solution, which is what I had been doing because I didn't know that.

2) do you control the file? depending on what it is, there are 2 more answers here.
-- if its not human readable, you can use pseudobinary. that is, cast your doubles to 64 bit ints and write those to the file. reverse this (read as 64 bit int, cast back to double). It grants full precision and only needs 16 characters for that (hex), and the conversion is much faster.
-- or, use a binary file? then its only 8 bytes, and no converting at all.

[Satoshi Yoda]

@kigar64551
Yes, this is exactly what I do now =) Just wondering, maybe there is a less messy solution.

@lastchance

Any conversion you do will pale into insignificance compared to the speed of reading from file.

That was exactly what I thought! Imagine how surprised I was when I discovered that parsing doubles with stod() eats more then 95% of the time (on my system in one specific example). Maybe it is because I test on high-end NVMe though, or thanks to ram-caching, I don't know. Anyway, with atof() I get about 250 MB/sec with small text files, which is totally ok. With stod() a lot less.

@seeplus @jonnin
Thanks! C++17 from_chars(), ok, looks good!

@jonnin
I do not exactly control the files. They are relatively small jsons with messy entries like that:
"values_min_max" : ["12.345 24.567", "12.345 nan"],
So, after parsing json I do need to parse these space-separated strings with doubles.
About binary files: that is exactly what I do =) Trying to convert these messy jsons to binary format. It is one-time operation (sort of), so I don't need a blazing fast optimization. But I just was confused with stod() performance, so want to know some solutions for future. And I don't want to wait 5 seconds every time I run a program during development now.

[seeplus]

Note that with std::from_chars(), leading white space is not ignored!