std::allocator replacement

xtc77 (12)
Hi!

I need to write a slightly modified version of the built-in std::allocator implementation. The reason is that I need to track how much memory is used by a small test programm I'm writing that uses two possibile implementation, one with std::set<T> and another one with a special third-party library.
Can I specialize the default allocator and write only the tracking code before calling the original method (a standard override, in a few words)?
That is, something like this (Sorry for any errors, I wrote it just here to clear my intention):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 
template<typename T> class trackAllocator : public std::allocator<T>
{
 // ...
   std::vector<pointer> _loc;
   std::vector<size_t> _size;

public:
    inline pointer allocate(size_type cnt, typename std::allocator<void>::const_pointer = 0) { 
      
      pointer p = allocator<T>::allocate(cnt, const_pointer);
      if(p) {
         // save pointer/size pair for calculation
         _loc.push_back(p);
         _loc.push_back(cnt);
      }
      return p; 
    }

  //...
}


First of all...is this admitted? Can be done? Or I need to rewrite it completely following the rules of c++ allocator standard?

Thank you!
kbw (9488)
Yes you can. But you need to defined your own deallocate too. And you need to pass your allocator to the container.

Here's an example of a full implementation of an allocation. You may as well make your own (that doesn't derive from std::allocator).
http://www.codeproject.com/KB/cpp/allocator.aspx
xtc77 (12)
Hi...

thank you for your suggestion. I've read that article because I did a deep search in the web before posting. That library is too much complicated for my needs (and uses loki library that doesn't compile in my environment - VS2010...lots of errors). Anyway I did a try by simply using the first version posted (where I added a couple of += and -= here and there for my purposes). But VS.NET 2010 complains with this:

 
 
error C2440: 'default argument' : cannot convert from 'Allocator<T>' to 'Allocator<T>'	vector	437	1

The std::vector line is _Vector_val(_Alloc _Al = _Alloc())...I dont understand... :-/

The usage I tried is very simple typedef std::vector<int, Allocator<int> > IntVec;...

I cannot get rid of it...
So I tried a simple version by using inheritance (as I suggested):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
 
template<typename T> class track_allocator: public std::allocator<T>
{
public:
	  track_allocator() {
 		  current = 0;
		  peak = 0;
	  }

	  ~track_allocator() {
		  cout << "Current: " << current << endl;
		  cout << "Peak:    " << peak << endl;
	  }

      pointer allocate( size_type _Count, const void* _Hint)
      {
			pointer p = std::allocator<T>::allocate( size_type _Count, const void* _Hint);
			if(p) {
				current+=sizeof(T);
				if(current>peak) peak=current;
			}
			return p;
      }
      
	  void deallocate(pointer _Ptr, size_type _Count)
      {       
		    if(_Ptr) {
				current-=sizeof(T);
			}

			std::allocator<T>::deallocate(pointer _Ptr, size_type _Count);
      }

private:
	size_t current;
	size_t peak;
};


but it gives me errors on the same usage (just changed the name of allocator in my test code) with:

 
 
error C2664: 'std::_Vector_val<_Ty,_Alloc>::_Vector_val(_Alloc)' : cannot convert parameter 1 from 'const std::allocator<_Ty>' to 'track_allocator<T>'	vector	535	1


The std::vector line is:

1
2
 
vector(const _Myt& _Right)
		: _Mybase(_Right._Alval)


I'm totally stuck...probably my C++ knowledge is not good enough... :(
Anyway I'm thinking to switch to VS2008...perhaps it has better compatility with this codes...I don't know!

If someone can help me with a working (simple!) allocator replacement that I will use with std::set container instead of a std::vector.

Thank you!
kbw (9488)
Forget the downloaded version. What's wrong with your original one?
xtc77 (12)
Well honestly I don't know...

The original one is the one in the previous post, that is track_allocator...

I use this simple code to test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
 
int _tmain(int argc, _TCHAR* argv[])
{	
       typedef std::vector<int, track_allocator<int> > IntVec;

       {	   
	   IntVec v;
	   for(int i=0;i<1000;i++) v.push_back(i);
	  	
	   IntVec u = v;
        }

	system("pause");
        return 0;
}


The inner block is used just to see the actual result from destructors call.
The error I get is the last one in the previous post, that is C2664.
This code does not compile in Dev-C++ 4.9.9.2 giving me many errors about formal parameter and return types. DevC++ std::allocator implementation is somewhat different from the standardized one? :-/
Further...if I try this allocator with std::set container (the one I REALLY to use with) gives me other error in the xutility.h file.
Now I'm trying to install VS 2008...hope this implementation will work...but with these errors this code would not be really portable... I think. :-/

My god...is there any chance to track memory usage of a std::set instance in an easy way?

Thank you!
RedX (156)
Last edited on
xtc77 (12)
Hi RedX!

Thank you very much for your reply...

Yes I saw both of them...the second one (from cboard...) is the one I took as a equivalent replacement of my original implementation (that has the same errors). I tought I did some error I could not track down (sometimes happens that you make a little mistake you cant see for hours...) so I decided to copy varnames and so on on mine to see if I did some mistakes...well, not.
The errors are the same of my original implementation (a "completed" version like the one in the first post, that was only an example) so it wasn't my fault nor some sort of typos...

The first one (from velocityreview)...well I tried it and it gave me this error:

 
 
error C2664: 'counted_allocator<T>::counted_allocator(const counted_allocator<T> &) throw()' : cannot convert parameter 1 from 'counted_allocator<T>' to 'const counted_allocator<T> &'	vector	441	1	stl alloc


always in the library vector include file, this is the code section and the line highlighted:

1
2
3
4
5
6
7
8
9
10
11
12
13
 
_Vector_val(_Alloc _Al = _Alloc())
		: _Alval(_Al)
		{	// construct allocator from _Al
		typename _Alloc::template rebind<_Container_proxy>::other
			_Alproxy(_Alval);        // <-- this is the line 441
		this->_Myproxy = _Alproxy.allocate(1);
		_Cons_val(_Alproxy, this->_Myproxy, _Container_proxy());
		this->_Myproxy->_Mycont = this;

		_Myfirst = 0;
		_Mylast = 0;
		_Myend = 0;
		}


I don't know how to go on... I'm trying now VS.NET 2008 just to give it a try...

Thank you so much anyway!
RedX (156)
you are missing

1
2
 
track_allocator(const std::allocator<T> &){
}

as a constructor don't ask me why

1
2
 
track_alocator(const track_allocator<T> &){
}

does not do the job.
Last edited on
xtc77 (12)
Oh thank you!

I added it and now it compiles...but...doesn't work! I missed copy constructor...that was the line in vector include file.
I managed to do this:

1
2
3
4
 
          track_allocator(const track_allocator<T> & other){
		  current = other.current;
		  peak = other.peak;
	  }


saving each copy the amount of used memry so far...

It never calls neither allocate() nor deallocate() functions... doing a simple debug shows that it calls the destructor immediately after the constructor...why???
I've also added the allocate function with 2nd parameter as default = 0...looking down to vector soure it calls allocate(size) more than allocate(size, hint)..
I think I must provide a "fresh" implementation from scratch...not an easy task I think...

Does anyone a working allocator (even simple) for a std::set or std::vector containers?
Or can help write one down...?

Thank you again!
xtc77 (12)
Last try I did...

externalize size_t current and size_t peak vars, remove destructor and empty copy constructor (as RedX suggested).

The main file now changes in

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
 
        current = 0;
	peak = 0;
	
	typedef std::vector<int, track_allocator<int> > IntVec;
	
	
		IntVec v;
		for(int i=0;i<100;i++) v.push_back(i);	
		

	std::cout << "Current: " << current << std::endl;
	std::cout << "Peak:    " << peak << std::endl;

	system("pause");
	return 0;


but vars are always 0...seems that it never calls allocate o deallocate fuctions...I put a simple std::cout << "something" << std::endl; into those functions but nothing appears...where is the error? Do I miss something?

Really annoying... :-/
RedX (156)
http://www.codeproject.com/KB/cpp/allocator.aspx

This article has at the end a working allocator. I tried it. It compiles and the functions are called.

Here is a working allocator:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
 
template<typename T> class track_allocator : public std::allocator<T>
{

public:

	template<typename U>
		struct rebind {
			typedef track_allocator<U> other;
		};


	track_allocator():allocator<T>() {
		current = 0;
		peak = 0;
	}

	explicit track_allocator(const track_allocator &o):allocator<T>(o){
		this->current = o.current;
		this->peak = o.peak;
	}

	~track_allocator() {
		std::cout << "Current: " << current << std::endl;
		std::cout << "Peak:    " << peak << std::endl;
	}

	pointer allocate( size_type _Count, const_pointer _Hint = 0)
	{
		pointer p = std::allocator<T>::allocate( _Count, _Hint);
		if(p) {
			current += sizeof(T) * _Count;
			if(current > peak){
				peak = current;
			}
		}
		return p;

	}

	void deallocate(pointer _Ptr, size_type _Count)
	{       
		if(_Ptr) {
			current -= sizeof(T) * _Count;
		}

		std::allocator<T>::deallocate(_Ptr, _Count);

	}

private:
	size_t current;
	size_t peak;
};
Last edited on
xtc77 (12)
Which compiler are you using?

Visual C++ 2010 gives me this:

 
 
Error	1	error C2440: 'default argument' : cannot convert from 'track_allocator<T>' to 'track_allocator<T>'	\...\vector	437	1


the line in <vector> is

1
2
 
	_Vector_val(_Alloc _Al = _Alloc())
		: _Alval(_Al)


I'm installing Visual C++ 2008...hope it fix.
I've just tried id with Dev-C++ and doesn't compile (seems that it doesn't import names declared into allocator<T> like pointer, size_type and so on...)


xtc77 (12)
That's incredible...Visual C++ 2008 compiles flawlessly...

...now I'm asking myselft...what's wrong in Visual C++ 2010??

Something new? Some new C++ standards I'm unaware of (notably almost everything :D)?

Someone could help solve this strange behaviour?

Thank you!
RedX (156)
I did it in 2008 but i have 2010 also installed and will try it there.

Edit:
So i tried this in 2010 and it works

1
2
3
 
template<typename T> track_allocator(const track_allocator<T> &o){
		
}


But what i don't understand is that this does not seem to be a regular construtcor. For instance it does not support the :allocator(o) and o.peak gives me an error about visibility.

In the drdobbs link i gave above it is explained. This is the rebind constructor. It is used to transform one allocator into another.
Last edited on
xtc77 (12)
Yes..it works also in 2010 just doing your suggested modifications.
Oh yes...I've read the rebind constructor and its purposes...thank you.

The error of visibility you got is strange...I had to chage member variable to public to let it compile.
Why this?

I've tried successfully also with std::set container.
xtc77 (12)
Oooppsss...I noticed a strange behaviour during memory allocation/deallocation...

It seems that it make one more deallocation than allocation.
I added 2 new member vars in track_allocator<T> one is allocCount and the other deallocCount. Also I print out these 2 new vars in destructor. On each successfull allocation (that is, pointer not null from base member function allocator<T>::allocate() I increment allocCount and on each non-null pointer to deallocate I incremenet deallocCount.
In the copy-constructor I simply copy the values of the member vars from one instance to the other.
This is the test code:

1
2
3
4
5
6
 
typedef std::set<int, std::less<int>, track_allocator<int> >IntSet;
	
{		
	IntSet s;
	for(int i=0;i<1000;i++) s.insert(i);
}


The output i get is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
Requested: 1 of size 8
*********** Destructor ************
Current: 8
Peak:    8
# alloc: 1
# deall: 0
************ End Destr ************
*********** Destructor ************
Current: 0
Peak:    0
# alloc: 0
# deall: 0
************ End Destr ************
Requested: 1 of size 20
*********** Destructor ************
Current: 0
Peak:    0
# alloc: 0
# deall: 0
************ End Destr ************
*********** Destructor ************
Current: 0
Peak:    0
# alloc: 0
# deall: 0
************ End Destr ************
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Requested: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 20
Removing: 1 of size 8
*********** Destructor ************
Current: 4294967288
Peak:    220
# alloc: 11
# deall: 12
************ End Destr ************
*********** Destructor ************
Current: 0
Peak:    0
# alloc: 0
# deall: 0
************ End Destr ************
*********** Destructor ************
Current: 0
Peak:    220
# alloc: 11
# deall: 11
************ End Destr ************


That is...destructor called 7 times! I think it does some dirty jobs with copy-ctor and so on...
Well the problem disappear when I put the counting vars outside the class (non member) and I clear the constructor and the copy constructor, so this is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
 
template<typename U>
		struct rebind {
			typedef track_allocator<U> other;
		};
		

	track_allocator() : allocator<T>() { }

        // copy-ctor
	track_allocator(const track_allocator&) { }

	template<typename U> track_allocator(const track_allocator<U>&) { 

// rest of the class


Now the counting is good...same # of allocation and deallocation and bytes are ok.
I will use this version now...but I just want to understand what happens.

If someone can explain me I would be very happy!

Thank you to everyone helped me so far!!
RedX (156)
I also noted this when trying to create the code. I saw 2 deallocation for 1 allocation.

My best guess is that when it needs more space it makes more room, moves the data then calls it destructor.

Try reserving the needed size then push_back and see if that changes.
Topic archived. No new replies allowed.