FEM matrix assembly using MPI
hello,
I am trying to solve the Poisson equation using finite element method with MPI.
I already have the connectivity matrix and also I have the x and y coordinate of each node. I've broadcasted the connectivity and coordinate matrix to all processors. let's say that we have N nodes, so the global stiffness matrix would be N*N. Imagine we have just two processors and we send just the half of elements to each processor. Now my question is should I define a local matrix of N/2*N/2 for each processors or should I define a N*N matrix for all and each processor will fulfill its part in the procedure. In this case how can I gather the ultimate matrix in the master process?
If I define the local matrix of N/2*N/2 for each processor, some elements maybe share the same element in both processors and this would be a problem for assembly. I am attaching my code and I would be grateful if you could help me.
I am trying to solve the Poisson equation using finite element method with MPI.
I already have the connectivity matrix and also I have the x and y coordinate of each node. I've broadcasted the connectivity and coordinate matrix to all processors. let's say that we have N nodes, so the global stiffness matrix would be N*N. Imagine we have just two processors and we send just the half of elements to each processor. Now my question is should I define a local matrix of N/2*N/2 for each processors or should I define a N*N matrix for all and each processor will fulfill its part in the procedure. In this case how can I gather the ultimate matrix in the master process?
If I define the local matrix of N/2*N/2 for each processor, some elements maybe share the same element in both processors and this would be a problem for assembly. I am attaching my code and I would be grateful if you could help me.
Last edited on
Each processor need only have a buffer for the part of the matrix that it is solving - they don't
all need to be NxN.
If you use just two processors then use MPI_Sendrecv. If you use more, then MPI_Scatterv and MPI_Gatherv.
If you are solving a Poisson equation then you are presumably solving it iteratively. However, to do this on more than one processor you will also have to send HALO DATA - that is, the data for nodes just outside its solution domain (which will act as local boundary conditions).
all need to be NxN.
If you use just two processors then use MPI_Sendrecv. If you use more, then MPI_Scatterv and MPI_Gatherv.
If you are solving a Poisson equation then you are presumably solving it iteratively. However, to do this on more than one processor you will also have to send HALO DATA - that is, the data for nodes just outside its solution domain (which will act as local boundary conditions).
| resabzr wrote: |
|---|
| thanks for your help. could you please tell more bout HOLA DATA |
The term is "halo data".
If you are solving Poisson's equation then you can look at a lab from Chalmers, who are good at this HPC stuff:
http://www.math.chalmers.se/Math/Grundutb/CTH/tma881/1112/Assignments/MPI/poisson.pdf
See the diagram on page 3: the halo nodes are highlighted at the edges of each subdomain. Note that this is a fully domain-decomposed arrangement. Each processor treats its own domain. They only need to exchange halo data with other processors.
I'm not so sure that your Finite-Element stuff is going to end up as Poisson's equation - you need to show more detail of what equations you are actually solving. The sparsity (or otherwise) of your matrix will strongly affect what you do here.
Last edited on
Topic archived. No new replies allowed.