Problem with free() and virtual memory

[Alejandro Antonio]

Hello everybody, I am working with the malloc(), calloc() and free() functions and I am having a little problem. If I execute the code below the memory seems to stilll be reserved after the free(). However if I uncomment the free() line after the calloc (and comment the other one) everything seems to be working fine.

Does anyone know why is this happening or if I am checking the available memory in a wrong way?

Thanks!!

MEMORYSTATUSEX statex;
statex.dwLength = sizeof (statex);

const int N_ptr = 100000;
int* Ptr[N_ptr];

GlobalMemoryStatusEx (&statex);
cout << "Available virtual memory (MB): " << to_string(statex.ullAvailVirtual/DIV) << endl;

for (int i = 0; i < N_ptr; i++) {
	Ptr[i] = (int*) calloc(10, sizeof(int));
	//free(Ptr[i]);
}

GlobalMemoryStatusEx (&statex);
cout << "Available virtual memory (MB): " << to_string(statex.ullAvailVirtual/DIV) << endl;

for (int i = 0; i < N_ptr; i++) {
	free(Ptr[i]);
}

std::this_thread::sleep_for(std::chrono::seconds(1));

GlobalMemoryStatusEx (&statex);
cout << "Available virtual memory (MB): " << to_string(statex.ullAvailVirtual/DIV) << endl;

Available virtual memory (MB): 408
Available virtual memory (MB): 402
Available virtual memory (MB): 402

[kbw]

The C heap uses a suballocator, which allocations pages from the OS. These blocks may or may not be returned to the OS. If they're not, the OS will see them as allocated.

If your free() loop frees in reverse order to the allocation, and you remove those print statement, the heap stands a chance of returning those blocks.

[salem c]

> If I execute the code below the memory seems to stilll be reserved after the free()
When you first allocate memory, there is a relatively expensive call to the operating system for it to reserve a large-ish block of memory for your program. This is added to your program's memory pool.

So when you call free(), it's only returned back to the program's memory pool (on the basis you might be wanting it back again soon with another malloc). But as far as the OS is concerned, your program still has that memory in it's VM allocation.

Also, you're never going to allocate several MB by doing this.

Ptr[i] = calloc(10, sizeof(int));
free(Ptr[i]);

You're just going to get the same block of 10 integers over and over again.

If you're using C++ (as you seem to be with cout), you shouldn't be using malloc etc at all. You should be using new, new[], delete and delete[].

Another thing to bear in mind is the OS can be quite lazy about committing actual physical resources. Just saying you want the memory is a distinct step from actually using the memory as far as the OS is concerned.


GlobalMemoryStatusEx is also the wrong tool for the job if you're curious about just one process. The activities of every other process in the system are going to mess with the results, no matter how "quiet" you think your system currently is.

[Alejandro Antonio]

Hello and thanks for your help!

I have tried to use new, new[], delete and delete[] but it does not solve the problem. I am developing an application where thousands of objects need to be stored in memory (and later removed) but it crashes because the memory is not deallocated.

I would appreciate some advise if you have an idea about what can be happening.

Thank you guys!

[salem c]

> but it crashes because the memory is not deallocated.
Then there is something wrong with your code.

Trying to reverse engineer the underlying allocator isn't going to solve the problem.

Besides, in C++ you shouldn't really need to use the low-level allocators, and the attendant problems of not freeing memory properly (or at all).

For example, having int *p = new int[100]; should be replaced with std::vector<int> p(100);
But you don't have to worry about leaking memory; when it goes out of scope, the memory is freed for you.

[helios]

I would appreciate some advise if you have an idea about what can be happening.

If the program is crashing, the most likely cause is that you're corrupting the heap, either by double-freeing a pointer, by dereferencing an invalid pointer, or by writing to an array out of bounds.

[seeplus]

I am developing an application where thousands of objects need to be stored in memory (and later removed) but it crashes because the memory is not deallocated.

If you can't use a standard container, then create a class that implements RAII (allocates memory on constructor, frees on destructor). and don't allocate/free memory for those objects anywhere else.

https://en.cppreference.com/w/cpp/language/raii

[jonnin]

I am developing an application where thousands of objects need to be stored in memory (and later removed) but it crashes because the memory is not deallocated.

Ok, there are a few problems here.
1 ... thousands are small, and will fit in a single vector. Reserve more than you need, 10k, whatever, these are small values unless you are on an embedded potato machine. 10k of a 100 byte sized object is only a megabyte. There are simply far better ways than dynamic memory.
2) not delete will never crash your program. Running the machine out of memory will, and accessing invalid pointers will, and various other things, but not deleting will not. Leaking memory, akin to running the machine out of memory, will. Something else is wrong.
3) if you are doing this over and over again, stop. Keep the memory around and reuse it, allocate and free are slow. Clear out the old data and refill it if you must (would be nice if this were also mostly valid and you can just update it).
4) if you free or delete it, set it to nullptr. If it crashes again, it will TELL you that you tapped a null pointer (dereference) or if you free it again it will do no harm (safe to delete null all day long).

It does look like you are on some odd machine -- 400MB is not even a phone these days --- if you care to enlighten us a bit more it may help.

[deleted account xyzzy]

new, delete, new[], delete[]: manual memory management the C++ way. There are also smart pointers introduced in C++11 that automatically manage memory without dealing with the messy details of manual memory management.

std::unique_ptr<>, for example, will free the owned memory when the pointer object goes out of scope, or the memory can be released on demand.
https://en.cppreference.com/w/cpp/memory/unique_ptr

Smart pointers can also own dynamically allocated array of objects.
https://en.cppreference.com/w/cpp/memory/unique_ptr

I haven't used any of the smart pointers as much as I should, so the how-to-use techniques and potential pitfalls from misuse I don't have a lot of knowledge about.