Advice on IPC

[VMsrVT2Tyc]

I've created a server program that scrounges data from various web sources and stores the data internally. I would like to create a client program that's able to interface with the server and access the data on the fly. A sort of IPC system.

I've looked at shmget but it seems a little inconvenient as you have to convert the data back into characters, then set memory aside, replicate the data and then send it.

The data class I wish to serve might look like this:

typedef struct
{
  string C;
  int j;
  double k;
} data_t;

class Data
{
  vector<data_t> D;

public:
  Data(void);
  ~Data(void);
  access_methods(void);
};

An instance of Data d, is continuously being updated and new data is being fed into it.

Ideally I'd like the client to capture a snapshot of d whenever an access is made. I've considered implementing a socket but I don't like the way everything is routed through the network stack unless this is still the best way to go in terms of being able to access entire chunks of memory held by the server on the fly.

What I have so far is the server has also made a copy of its data and arranged it into files on disk. In an almost ideal scenario I can have the client come in and access the files to read whatever it wants. Though I'd still prefer to read straight from the server's memory.

I'm looking for advice on the best way to do this.

[salem c]

I would first suggest you go with tried and tested approaches which are going to work before trying to second guess whether there is a performance issue or not.

For one thing, the simple and obvious solution may save you a lot of pointless work in optimising things.
For another, the simple and obvious solution will act as a baseline (functional and performance) for you to measure your successes (and failures) against.

The problem with using shmget etc is that you have to make sure you override the allocator for both vector and string (but only when you're manipulating data_t things). The whole thing (recursively) needs to be in shared memory, not just "Data d;"

Getting everything right so every 'C' is in shared memory and every 'non-C' isn't in shared memory will be enough of a challenge.

d.D[i].C = mystring;
myotherstring = d.D[i].C;

Further, you need to ensure that shmat() returns the SAME virtual address in both the client and the server. You have a lot of hidden internal pointers, and they'll only make sense if the virtual addresses are the same.

You would then have to arrange for some kind of mutex/semaphore (also in shared memory) to prevent the writer corrupting the data in the middle of reading.

Does vector<data_t> D; grow or shrink much in normal use?

> I've considered implementing a socket but I don't like the way everything is
> routed through the network stack
Might I draw your attention then to Unix Domain Sockets.
https://stackoverflow.com/questions/14973942/tcp-loopback-connection-vs-unix-domain-socket-performance
Whilst UDS use the socket API, they're more like bi-directional pipes.
You could even use read()/write() with your UDS descriptor, making it even easier to substitute for a pipe() if you wanted.

> I've created a server program that scrounges data from various web sources and stores the data internally.
I'm guessing that you're waiting around a lot (say using select() with a bunch of descriptors) for something to happen.

Do your descriptors map in any way to indices in your D vector? If they do, tracking which D[i] have changed will save some work.

Basically use the dead time between actual network messages to send some small fraction of D down your server->client connection.


Here's a "fun" idea ;)

if ( time_to_send_to_client ) {
    if ( fork() == 0 ) {
        // Note that because we're not calling exec(), we're in the same state as if
        // we were in a signal handler.  So man 7 signal for the restrictions on the
        // things you can and cannot do.
        // On the plus side, you get an instant snapshot of "Data d", because the OS
        // will automatically do the copy-on-write thing on any subsequent changes
        // the parent make to d.
        doSendToClient();
        _exit(0);
    }
}

[VMsrVT2Tyc]

Thanks for your reply.

Choosing which method to adopt relies on a few factors beyond performance considerations alone. I'd like something that's easy to use/maintain, has few overheads/dependencies. I prefer to build something from first principals. For example, a Boost library would be the least preferred option. I want to minimise the replication of code on both sides.

I agree the simplest and obvious solutions are the best ones indeed. Hence I prefer DIY.

In your first suggestion which is shmget(), do you mean to say that the server and client are each contained within each other? I understand shmget() is somewhat deprecated library but mmap that works in much the same way as shmget() is the more modern approach to use. In each of these cases I don't like that you have to set aside and replicate memory.

The vector<> D evolves over time, accumulating and discarding data which the server program handles. The rationale for splitting the overall system into a client and server is that over time the server will grow its database and I don't want to reset it everytime I alter something in the client. The client is there for alterations to its methods. The server is there to gather and hold the data.

I don't mind pipes but I'm still dubious to use any possibly unnecessary network framework if I don't have to. I would like to keep dependencies to a minimum. Ideally I prefer a more direct way to IPC on a local system.

[VMsrVT2Tyc]

I decided to use UNIX Domain Sockets to send messages across two programs. It's the most straight forward solution and I'm pleased with it. Initially what I had in mind was for one program to read another program's memory contents directly. But it seems there's no easy way to do this.

// UNIX Domain Sockets
// $ g++ twoway.cpp -o twoway -l pthread && ./twoway

#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <cstdlib>
#include <cstring>
#include <string>
#include <cstdio>
#include <thread>
#include <cstdint>
#include <csignal>
using namespace std;

#define SOCKET_PATH "/tmp/"

bool quit;

class Uds
{
  struct sockaddr_un addr;
  int32_t fd, rc, cl;
  std::string socket_path, buf;

public:
  Uds(string);
  ~Uds(void);
  bool writer(string);
  bool writer_connector(void);
  void reader(void);
};

Uds::Uds(string name)
{
  socket_path = SOCKET_PATH + name;

  if ((fd = socket(AF_UNIX, SOCK_STREAM, 0)) == -1)
    {
      perror("socket error");
      exit(-1);
    }

  memset(&addr, 0, sizeof(addr));
  addr.sun_family = AF_UNIX;
  strncpy(addr.sun_path, socket_path.c_str(), sizeof(addr.sun_path) - 1);
}

Uds::~Uds(void)
{

}

bool Uds::writer_connector(void)
{
  if (connect(fd, (struct sockaddr *) &addr, sizeof(addr)) == -1)
    return 0;

  return 1;
}

bool Uds::writer(string buf)
{
  if ((rc = write(fd, buf.c_str(), buf.size() * sizeof(char))) != buf.size())
    {
      if (rc > 0)
	fprintf(stderr,"partial write");

      else
	{
	  perror("write error");
	  return 0;
	}
    }

  return 1;
}

void Uds::reader(void)
{
  unlink(socket_path.c_str());

  if (bind(fd, (struct sockaddr *) &addr, sizeof(addr)) == -1)
    {
      perror("bind error");
      exit(-1);
    }

  if (listen(fd, 5) == -1)
    {
      perror("listen error");
      exit(-1);
    }

  while (!quit)
    {
      if ((cl = accept(fd, NULL, NULL)) == -1)
	{
	  perror("accept error");
	  continue;
	}
      
      while ((rc = read(cl, &buf[0], sizeof(buf))) > 0 && !quit)
	{
	  //buf[rc] = 0x00;
	  printf("read %u bytes: %*s\n", rc, rc, buf.c_str());
	}
      
      if (rc == -1)
	{
	  perror("read");
	  exit(-1);
	}
      
      else if (rc == 0)
	{
	  printf("EOF\n");
	  close(cl);
	}
    }

  close(cl);
  printf("Reader exit\n");
}

void writer_th(void)
{
  Uds w("name");
  // Send events
  srand(0);
  
  while (!w.writer_connector())
    sleep(1);
  
  while (!quit)
    {
      unsigned event = rand() % 100;

      if (w.writer(to_string(event)))
	printf("Wrote %u\n", event);

      sleep(5);
    }

  w.writer("0x00");
  printf("Writer exit\n");
}

void reader_th(void)
{
  Uds r("name");
  r.reader();
}

void signal_handler(int signum)
{
  printf("\nExit...\n");
  quit = 1;
}

int main()
{
  signal(SIGINT, signal_handler);
  thread writer(writer_th);
  thread reader(reader_th);
  writer.join();
  reader.join();
  return 0;
}