Read from ofstream output

[Sreiner]

Hello everyone,
I'm a student of civil engineering in the middle of creating a topology optimization algorithm using C++. My only experience with coding was Matlab and some Python so I'm in over my head a bit. I've managed to create the basics of my algorithm but I can't seem to get the reading files part right. I use multiple scripts that need to have files imported and exported. Mostly .txt arrays.
This is the code I used to "write" the array on a .txt file

{ofstream fout("f1.txt");
fout << f1 << endl;
}

Here is the .txt file that is created from the array..
https://drive.google.com/file/d/1jBmi-3RCqplwt3E2HBanO1LkIRFqYtgl/view?usp=sharing
Now I'm trying to read the array on a different script.
I thought about reading the first line which is the array size and then defining the array with it but I can't get it to work.
I had to do the same thing with matrices which proved way easier by just defining an matrix value and using ifstream. Thought about doing something like the following code just like I did with matrices but its fundamentally wrong cause I haven't defined the array size.

real[int] f1;
{ifstream fin("f1.txt");
fin >> f1;
}

How would you go on about tackling this problem?

[kigar64551]

I suggest storing the values in an std::vector<int>. Or std::vector<double>, if they are real numbers.

Try something like:

std::vector<double> numbers;
std::ifstream inputFile("data.txt");

// read all numbers from file
if (inputFile.good()) // <-- make sure file was opened successfully
{        
    double value;
    for(;;)
    {
        inputFile >> value; // <-- try to read next value
        if (!inputFile.good()) // <-- make sure value was read successfully
        {
            break; // <-- read error or end-of-file
        }
        numbers.push_back(value); // <-- append the value to the end of the vector
    }
}

// print all numbers we have read
for (auto iter = numbers.begin(); iter != numbers.end(); ++iter)
{
    std::cout << *iter << std::endl;
}

[jonnin]

array size must be known when you code it in pure c++. a few compilers support doing it on the fly, but that is a language extension that you can't rely on.

you should use vector:
#include<vector>
..
fin >> filesize; //assuming this is correct, the number of values in the file up front
std::vector<double> f1(filesize); //the (filesize) part sizes the vector to fit your data
for(auto &a : f1) //you can do this any way. this is a range based for loop over the vector.
{
fin >> a;
}

vectors can do much more than this, but for now, you can just use it like a size to fit array.
f1[0] = 42.0; //this works just like an array.

[lastchance]

#include <iostream>
#include <fstream>
#include <valarray>
#include <string>
using namespace std;

valarray<double> readFile( const string &filename )
{
   ifstream in( filename );
   int npts;
   in >> npts;
   valarray<double> V( npts );
   for ( double &e : V ) in >> e;
   return V;
}

int main()
{
   valarray<double> V = readFile( "f1.txt" );
   cout << "Points:  " << V.size() << '\n'
        << "Minimum: " << V.min()  << '\n'
        << "Maximum: " << V.max()  << '\n'
        << "Average: " << V.sum() / V.size()  << '\n';
}

Points:  193
Minimum: -0.24445
Maximum: 2.56377
Average: 0.558475

[seeplus]

@kigar64551 - that is un-necessarilly long code and also doesn't deal with the first number in the file being the number of elements to read.

Using a std::vector (as opposed to a std::valarray as per lastchance above) then perhaps:

#include <vector>
#include <iostream>
#include <fstream>
#include <string>

bool readFile(std::vector<double>& vd, const std::string& filename) {
	vd.clear();

	if (std::ifstream ifs { filename }) {
		size_t noelems {};

		ifs >> noelems;
		vd.reserve(noelems);

		for (double elem; ifs >> elem; vd.push_back(elem));
		return vd.size() == noelems;
	}

	return false;
}

int main() {
	std::vector<double> vd;

	if (readFile(vd, "f1.txt")) {
		std::cout << vd.size() << " elements read\n";

		for (const auto& d : vd)
			std::cout << d << ' ';

		std::cout << '\n';
	} else
		std::cout << "Problem reading the file\n";
}

[kigar64551]

This was only meant to illustrate the basic idea ;-)

Also, if you don't do error checking, then you can have "shorter" code. But you probably will regret one day...

Note: The bool() operator of std::ifstream only checks "fail" and "bad" bits, not "eof" bit.

[Duthomhas]

Proper error checking is pretty easy, and short. The only error check possibly missing in lastchance’s example was verifying that the number of elements obtained matched the give value. I suggest a single modification:

#include <iostream>
#include <fstream>
#include <valarray>
#include <string>
using namespace std;

valarray<double> readFile( const string &filename )
{
   ifstream in( filename );
   int npts;
   in >> npts;
   valarray<double> V( npts );
   for ( double &e : V ) in >> e;
   return in ? V : valarray<double>{};  // all or nothing, baby
}

int main()
{
   valarray<double> V = readFile( "f1.txt" );
   cout << "Points:  " << V.size() << '\n'
        << "Minimum: " << V.min()  << '\n'
        << "Maximum: " << V.max()  << '\n'
        << "Average: " << V.sum() / V.size()  << '\n';
}

Notice how we are NOT checking for EOF — that may be incorrect:

  • it is correct to check EOF if we know that the data is always
    the only content in the file AND we do something to find it

  • it is not correct if the data may be followed by any other data
    (of any kind, which wouldn't be read anyway)

We don’t know which to choose. lastchance’s code (correctly, IMHO) simply assumed the data had no error in format, which is something that is a fairly normal characteristic of scientific computing.

As it is, there is potential for another error: too little data in a file with additional data following. The current code cannot detect that. (Nor should it, because again, the “N a1 a2 ... aN” format is very standard.)

[edit] Note also that my suggested modification throws away data if something is wrong. This may not be desired.

[lastchance]

Depends on what you want from error checking.

You would have to check that ...
- the file opened
- the first item read was a positive integer
- exactly the right number of data points followed
- those data points were valid
etc. etc.

Life's too short.

Duthomas wrote:

simply assumed the data had no error in format, which is something that is a fairly normal characteristic of scientific computing

Yes, I'm afraid that is what I and most of my colleagues do. We prefer a code to crash if the input is wrong, rather than continue with faulty answers.

[Geckoo]

Hello. @Sreiner I don't know why you want to deal with a first entry given how many real numbers you have in the file. You have it according to the vector.size(). But why not? This program is yours :)
Including in your process some headers, I guess that you could easily compute and manage all your entries. Your program seems really interesting. Take a look at this code - a little bit different from the previous because I don't insert any first entry with a number of double(s) which the process has to read. I hope that it helps you - even a little bit ++

#include <vector>
#include <iostream>  // manages console
#include <fstream>   // reading file
#include <algorithm> // std::min_element
#include <numeric>   // std::accumulate

bool readFile(std::vector<double>& vd, const std::string& filename) 
{
    if (std::ifstream ifs{ filename }) 
    {   // read and store in the vector all entries
	for (double e; ifs >> e; vd.push_back(e));
	return true;
    }
    // something goes wrong
    return false;
}

int main() 
{
    std::vector<double> vd;

    if (readFile(vd, "file.txt")) 
    {   // display all entries with index (start with 1, not 0)
        for (auto it = vd.begin(); it != vd.end(); it++) 
	{
	    int index = std::distance(vd.begin(), it);
	    std::cout << index + 1 << "\t" << *it << std::endl;
	}

	std::cout << std::endl;
        // manages and computes entries
	std::cout << vd.size() << " entries have been found :" << std::endl;
	std::cout << "Minimal entry " << "\t" << *std::min_element(vd.begin(), vd.end()) << std::endl;
	std::cout << "Maximal entry " << "\t" << *std::max_element(vd.begin(), vd.end()) << std::endl;
	std::cout << "Entries sum " << "\t" << std::accumulate(vd.begin(), vd.end(), 0.0f) << std::endl;
	std::cout << "Average " << "\t" << (std::accumulate(vd.begin(), vd.end(), 0.0f) / vd.size()) << std::endl;
    }
    else
	std::cout << "Problem reading the file";
}

As an example, I use only the first 25 entries of your file f1.

1       0.0203134
2       0.0268741
3       -0.0394576
4       0.0480166
5       0.0108512
6       -0.0605254
7       0.000802261
8       0.0305134
9       0.0363904
10      -0.00426705
11      0.0203272
12      0.0462517
13      0.0340515
14      0.0200594
15      0.105429
16      0.030474
17      -0.0537028
18      -0.00757437
19      0.0498889
20      0.00434509
21      -0.176211
22      -0.168541
23      -0.034176
24      0.105787
25      0.0865092

25 elements has been found :
Minimal entry   -0.176211
Maximal entry   0.105787
Entries sum     0.13243
Average         0.00529719

[seeplus]

If you use a std::vector and need some stats from the data, then I suggest you calculate the required during the read phase - so that you only need to iterate over the data once. It doesn't matter here with this few number of data elements but does if you deal with multiple millions... Something like:

#include <vector>
#include <iostream>
#include <fstream>
#include <string>
#include <limits>

struct Result {
	enum class State {OK, OPEN, READ};
	State state {State::OPEN};
	size_t required {};
	double minval { std::numeric_limits<double>::max() };
	double maxval { std::numeric_limits<double>::min() };
	double sum {};
};

Result readFile(std::vector<double>& vd, const std::string& filename) {
	vd.clear();

	if (std::ifstream ifs { filename }) {
		Result res {};

		ifs >> res.required;
		vd.reserve(res.required);

		for (double elem; ifs >> elem; vd.push_back(elem)) {
			res.sum += elem;

			if (elem < res.minval)
				res.minval = elem;

			if (elem > res.maxval)
				res.maxval = elem;
		}

		res.state = (res.required == vd.size() ? Result::State::OK : Result::State::READ);
		return res;
	}

	return Result {};
}

int main() {
	std::vector<double> vd;

	if (auto res { readFile(vd, "f1.txt") }; res.state == Result::State::OK) {
		std::cout << vd.size() << " elements read\n";

		for (const auto& d : vd)
			std::cout << d << ' ';

		std::cout << "\n\nMinimal entry\t" << res.minval << '\n';
		std::cout << "Maximal entry\t" << res.maxval << '\n';
		std::cout << "Entries sum\t" << res.sum << '\n';
		std::cout << "Average\t\t" << res.sum / vd.size() << '\n';
	} else
		if (res.state == Result::State::OPEN)
			std::cout << "Problem opening the file\n";
		else
			std::cout << "Error reading data. Expected " << res.required << ". Read " << vd.size() << '\n';
}

[Geckoo]

@seeplus More sophisticated and clever. I like this one ++

[seeplus]

Also if data analysis doesn't require the actual data after it's been read, then just do the analysis as part of read and return what's needed. Something like:

#include <iostream>
#include <fstream>
#include <string>
#include <limits>

struct Result {
	enum class State {OK, OPEN, READ};
	State state {State::OPEN};
	size_t required {};
	size_t read {};
	double minval { std::numeric_limits<double>::max() };
	double maxval { std::numeric_limits<double>::min() };
	double sum {};
};

Result readFile(const std::string& filename) {
	if (std::ifstream ifs { filename }) {
		Result res {};

		ifs >> res.required;

		for (double elem; ifs >> elem; ++res.read) {
			res.sum += elem;

			if (elem < res.minval)
				res.minval = elem;

			if (elem > res.maxval)
				res.maxval = elem;
		}

		res.state = (res.required == res.read ? Result::State::OK : Result::State::READ);
		return res;
	}

	return Result {};
}

int main() {
	if (auto res { readFile("f1.txt") }; res.state == Result::State::OK) {
		std::cout << res.read << " elements read\n";
		std::cout << "\nMinimal entry\t" << res.minval << '\n';
		std::cout << "Maximal entry\t" << res.maxval << '\n';
		std::cout << "Entries sum\t" << res.sum << '\n';
		std::cout << "Average\t\t" << res.sum / res.read << '\n';
	} else
		if (res.state == Result::State::OPEN)
			std::cout << "Problem opening the file\n";
		else
			std::cout << "Error reading data. Expected " << res.required << ". Read " << res.read << '\n';
}

[Sreiner]

Hello again everyone,
I'm overwhelmed with all the replies I got. This is my first contact with the programming community and seeing you all optimizing each others algorithm is really moving. Your guys help was really great I managed to get it working.
Got to say though the thing that I gained is using error checking throughout my algorithm.
Thanks again everyone!