problem when moving from 32 to 64 platform

Pages: 1234
helios (17574)
MPI [...] receiving data
How are you handling deserialization from MPI? Are you aware that, for example, a VecVecVecDbl_t will not have a contiguous memory layout? If you try to do something like
1
2
 
VecVecVecDbl_t received(3, VecVecDbl_t(3, VecDbl_t(3)));
receive(received.data(), 3 * 3 * 3 * sizeof(double));
you'll trash the data structure.
Cplusc (457)
@lastchance
I reinstall the windows and all softwares and now I am able to work on 64 bit platform. now I am facing with one problem in MPI. The below code will run with 2 MPI process and it's supposed to send a vector {0,2,3} to processor 1 and in the processor 1 I expect to receive {0,2,3} but instead I am receiving {0,2,0}. I don't understand why the value is changing?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
#include "includes.h"
#include "typedefs.h"
#include "mesh.h"
#include "blas.h"
#include "partitioning.h"
#include "utilities.h"
//#include "metis.h"



int main(int argc, char* argv[])
{
	int num_procs, myrank;
	MPI_Init(&argc, &argv);
	MPI_Status status;
	MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
	MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

	VecIdx_t Local2Global;
	VecVecIdx_t Local_Local2Global;

	Blas Blas;
	Utilities Utilities;

	VecInt_t MPI_reps;
	size_t size_mpi_reps = 0;
	size_t size_l2g = 0;

	size_t nNodes_global = 0;
	int tag = 1;
	if (myrank == 0) {

		Partitioning Partitioning("mesh.txt");
		Global_Coordinate = Partitioning.Global_Coordinate_;
		Connectivity = Partitioning.Master_Rank_Connectivity_;
		Coordinate = Partitioning.Master_Rank_Coordinate_;
		MPI_reps = Partitioning.MPI_reps_;
		//Local_Local2Global = Partitioning.Local_Local2Global_;
		//size_l2g = Local_Local2Global[0].size();
		//Local2Global = Partitioning.Local_Local2Global_[0];
		//nNodes_global = Partitioning.nNodes_global_;
		VecVecIdx_t Local_Local2Global;
		Local_Local2Global.resize(2);
		Local_Local2Global[0] = { 0,1,3 };
		Local_Local2Global[1] = { 0,2,3 };
		Local2Global = Partitioning.Local_Local2Global_[0];

		for (int i_proc = 0; i_proc < num_procs; i_proc++) {
			if (i_proc == 0) {

			}
			else {

				size_t size_local2global = Local_Local2Global[i_proc].size();
				MPI_Send(&size_local2global, 1, MPI_INT, i_proc, tag, MPI_COMM_WORLD);
				MPI_Send(Local_Local2Global[i_proc].data(), Local_Local2Global[i_proc].size(), MPI_INT, i_proc, tag, MPI_COMM_WORLD);

			}
		}


	}
	else {

		MPI_Recv(&size_l2g, 1, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);
		Local2Global.resize(size_l2g);
		MPI_Recv(Local2Global.data(), size_l2g, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);
		//MPI_Recv(&nNodes_global, 1, MPI_UNSIGNED, 0, 0, MPI_COMM_WORLD, &status);

		//int n;
		//MPI_Recv(&n, 1, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);                               
		//MPI_Recv(a.data(), n, MPI_DOUBLE, 0, tag, MPI_COMM_WORLD, &status);

	}

	printf("process %d prainting local2global\n", myrank);
	for (int i = 0; i < Local2Global.size(); i++)
	{
		printf("%d \t", Local2Global[i], myrank);
		printf("\n");
	}


	MPI_Finalize();
	return 0;
}
Last edited on
lastchance (6980)
You haven't given sufficient code for me to compile and run it.

Check (by PRINTING IT OUT) the value of:
- size_local2global that processor 0 is sending (should be 3)
- Local_Local2Global[i_proc].size() that processor 0 is sending (should be 3)
- size_l2g that processor 1 receives (should be 3)
Remember to put a clear indication of each item that you are printing out. You cannot guarantee the order of output in a multiprocessor configuration.
Cplusc (457)
@lastchance
This will execute in 64 platform. The size of sent and received are 3 but the same problem as I said is still there

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
#include <iostream>
#include <vector>
#include <mpi.h>



int main(int argc, char* argv[])
{
	int num_procs, myrank;
	MPI_Init(&argc, &argv);
	MPI_Status status;
	MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
	MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

	size_t size_mpi_reps = 0;
	size_t size_l2g = 0;
	std::vector<size_t> Local2Global;
	size_t nNodes_global = 0;
	int tag = 1;
	if (myrank == 0) {

		std::vector<std::vector<size_t> > Local_Local2Global;
		Local_Local2Global.resize(2);
		Local_Local2Global[0] = { 0,1,3 };
		Local_Local2Global[1] = { 0,2,3 };
		Local2Global = Local_Local2Global[0];
		size_l2g = Local2Global.size();

		for (int i_proc = 0; i_proc < num_procs; i_proc++) {
			if (i_proc == 0) {

			}
			else {

				size_t size_local2global = Local_Local2Global[i_proc].size();
				MPI_Send(&size_local2global, 1, MPI_INT, i_proc, tag, MPI_COMM_WORLD);
				MPI_Send(Local_Local2Global[i_proc].data(), Local_Local2Global[i_proc].size(), MPI_INT, i_proc, tag, MPI_COMM_WORLD);

			}
		}


	}
	else {

		MPI_Recv(&size_l2g, 1, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);
		Local2Global.resize(size_l2g);
		MPI_Recv(Local2Global.data(), size_l2g, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);

	}

	printf("process %d prainting size local2global\n", myrank);

		printf("%d \t", size_l2g, myrank);
		printf("\n");


	printf("process %d prainting local2global\n", myrank);
	for (int i = 0; i < Local2Global.size(); i++)
	{
		printf("%d \t", Local2Global[i], myrank);
		printf("\n");
	}

	MPI_Finalize();
	return 0;
}

Last edited on
lastchance (6980)
You are mixing size_t (which is unsigned) with int (which is signed).

MPI_INT won't know what to do with it.

Change all your size_t to int and it will work "fine". (Except for the fact that the format strings in lines 54 and 61 don't have enough fields to portray the data.)


I don't know who told you to use size_t, but it's not helping you here.
Cplusc (457)
Thanks for your answer. Exactly the problem is size_t. if you look at this threads since the beginning I was doubtful to the size_t. you mean if I use MPI_UNSIGNED instead of MPI_INT it will work? It is also my question why some people prefer using size_t over int. if I use MPI_UNSIGNED in 32 platform the code works fine but in 64 the same problem.
Last edited on
thmm (703)
It is also my question why some people prefer using size_t over int.

It depends on the context. If you need an array index, negative values makes no sense so a size_t seems a natural choice. Also a size_t can have abigger values - see std::string::npos.
Cplusc (457)
The code which I am developing, every vector and every iterator in for loop are defined as size_t. If I use MPI_UNSIGNED instead of MPI_INT it will work? I tried this in 32 it worked but in 64 for not? should I change the variable from size_t to int to solve this issue?
Cplusc (457)
@George P thanks for the answer
lastchance (6980)
Well, you are free to choose whoever you like to advise you on MPI, @cplusc, but my preference when using MPI would definitely be to stick throughout with int. The size of your array may always be positive, but that doesn't imply that its contents are.

Remember that your MPI system also has Fortran bindings, and Fortran definitely does allow negative array indices. Also, the limits of a size_t are not as well defined as int, and, as far as I am aware, there is no such MPI data type as MPI_SIZE_T.
Last edited on
Cplusc (457)
Thanks for your helpful answer.
seeplus (6591)
It is also my question why some people prefer using size_t over int


size_t is unsigned and will be a 32 bit value when compiled as 32 bit and 64 bit when compiled as 64 bit - ie can hold the largest unsigned integer permissible for the compile type.

When an unsigned value is used (ie array index etc), then it is common for a type of size_t to be used. Note that passing a type of size_t to a function that expects an int may not be as desired. Pass an argument of the required type.

So when moving from 32 bit to 64 bit the size of size_t changed from 32 bit to 64 bit. If some code uses int and some size_t then they would both be the same size for 32 bit but not for 64 bit.
Last edited on
deleted account xyzzy (5768)
Do note the above link shows why/when to use size_t in C. Much of the same ideas matter in C++.
https://en.cppreference.com/w/cpp/types/size_t

From this ^^ link the following might be relevant to your 32-bit to 64-bit issues:
std::size_t is commonly used for array indexing and loop counting. Programs that use other types, such as unsigned int, for array indexing may fail on, e.g. 64-bit systems when the index exceeds UINT_MAX or if it relies on 32-bit modular arithmetic.

IOWs, MPI might (MIGHT) be relying on 32-bit math no matter the bit-ness used to compile, C or C++.

A similar warning shows up for C's size_t entry at cppreference.

I can't offer anything else as help since I don't use the library.
lastchance (6980)
@cplusc,
Your MPI datatypes are here:
https://www.mpich.org/static/docs/v3.3/www3/Constants.html

If you are intending to use MPI communication then only use the C types that correspond to them.

That doesn't include size_t.



What you do in any other C++ code is up to you. Use size_t to your heart's content there.
Last edited on
seeplus (6591)
For 32-bit MPI_UNSIGNED_LONG would match size_t and for 64-bit MPI_UNSIGNED_LONG_LONG would match size_t.

lastchance (6980)
For 32-bit MPI_UNSIGNED_LONG would match size_t and for 64-bit MPI_UNSIGNED_LONG_LONG would match size_t.

And you will end up with code that only runs on one platform.
jlb (4973)
You may want to consider using the "fixed" integral types like int16_t.

But first make sure you specify the correct MPI library (32 or 64 bit) in your program, depending on the architecture of your machine. If there is only a 32 bit MPI library make sure to use "fixed" sizes that are available for a 32 bit system and don't use things like int, long, or size_t which can be different on different architectures.

lastchance (6980)
If you send/receive an int using an MPI_INT datatype it will work on any platform.

Just don't use size_t in an MPI context.
Cplusc (457)
Thank you all for your answers. Actually before posting this question I had tried the MPI_UNSIGNED_LONG for 32 and MPI_UNSIGNED_LONG_LONG for 64. but the problem was it was just working fine on one of the platform, either 32 or 64. The similar problem stands for using fixed-size integer, fixed-size integer types will mean that the sizes of one's variables won't change if compilers use different sizes for int, long and it won't necessarily guarantee that code will behave identically on machines with various integer sizes, even when the sizes are defined. I think the best option is to stick with c-type variable when using mpi communication.
Last edited on
Pages: 1234