What is send over the socket when sending std::string?

[Bengbers]

With help from DizzyDon and kbw (see https://cplusplus.com/forum/beginner/285198/), I have been able to create a non-blocking socket and read from it. I use this code for writing to the socket. (The code for reading is copied from the previous thread):

int writeData(const std::string & input) {
  std::cout << __PRETTY_FUNCTION__ << std::endl;
  int bytes_sent = send(Master_sfd, input.c_str(), input.size() + 1, 0);
  return bytes_sent;
}

I use this write-function in an authentication procedure for connecting to a Basex server instance. The procedure is based on the information given in https://docs.basex.org/wiki/Server_Protocol . When using the fake authentication data that are given in this protocol, my procedure produces the same md5 hash so I am pretty confident that that part of my procedure functions well.

BasexSocket& BasexSocket::Authenticate (std::string user, std::string passwd) {
  std::string rts = {};
  int bytes_read, bytes_sent;
  vector <string> strings;
  std::string nonce;
  std::string code_wd;        // Either password (CRAM-MD5) or user:realm:password (Digest)
  std::string realm;
  std::string md5_codwd = {}; // md5_basex(code_wd)
  std::string auth;
  std::string def_auth;
  std::string sock_read_string = {};

  fd_set read_sfd_set;
  struct timeval timeout;
  FD_ZERO(&read_sfd_set);
  FD_SET(Master_sfd, &read_sfd_set);
  memset(&timeout, 0, sizeof(timeout));

// 1: Server sends the timestamp
  bytes_read = readSocket( rts);        // number of read bytes
  cout << "0  Realm en timestamp:        " << rts << endl;
  if (bytes_read == 0) {
    warnx("Reading timestamp failed"); Master_sfd = -1;
    return *this;
  }
// 2: Client sends the user name (must be \0-terminated)
  bytes_sent = writeData(user);
// 3: Compose the md5_auth hash, md5_auth code = md5(md5("<user>:BaseX:<password>") + "1234567890123")
  int string_cnt = StringSplit( strings , rts, ':');    // Split string on occurence of ':'
  if (strings.size() == 1) {                                        // CRAM-MD5
    nonce = rts;
    code_wd = passwd;
  } else {                                                          // Digest
    code_wd = user; realm = strings[0]; nonce = strings[1];
    code_wd.append(":").append(realm).append(":").append(passwd);
  }
  md5_codwd = md5_basex(code_wd);
  auth = md5_codwd.append(nonce);
  def_auth = md5_basex(auth);
  bytes_sent = writeData(def_auth);

  // select(Master_sfd + 1, &read_sfd_set, NULL, NULL, &timeout);

// 4: Handle server-response
  sock_read_string.clear();
  bytes_read = readSocket( sock_read_string);
  if (bytes_read > 0) {
    char success = sock_read_string.front();
    cout << "Auth-Test: " << success << endl;
    if (success != '\0') {
      Master_sfd = -1;
      warnx("Authentication failed");
      cout << endl;
    }
  } else {
    Master_sfd = -1;
    warnx("Reading authentication status byte failed");
  }
  return *this;
};

The problem is that the readSocket function call in line 46 returns \001 ... indicating a failure. I've used the credentials for connecting to a standalone basexclient and am certain that they are correct.

When I was working on the same procedure I developed earlier in R, I ran into a similar problem. When that procedure was performed during 'normal' use, the authentication failed. However, no errors were detected in the debugger.
The problem eventually turned out to be solved there by calling the socketSelect() function. I don't have an explanation for this behaviour. Maybe calling this function inserts a short wait?

My guess is that the select() I use in line 42 comes closest to the socketSelect as used in the R code. Adding select() however has no positive result.
I've used Wireshark to inspect the traffic between the client and the server, but I don't see anything out of the ordinary. The termination zeros are inserted in the correct place.

Assuming that the md5 hash is calculated correctly, I think the only cause that remains is that the writeData function is not working properly.
According to the specification, writeData must be able to be used for sending strings (zero-terminated) or raw data. What 'raw' data is is not further specified. In R, 'raw' data corresponds to hexadecimal values.

writeData returns the correct number of sent bytes but I can't look at the data that is being sent.
Is it possible to see those data?

[DizzyDon]

Post your readSocket function.

Why does returning 1 from your readSocket function indicate an error?
Presumably it means that it read 1 char.

[Bengbers]

No, int readSocket( std::string &sockResponseString) returns 1 indicating that it has read 1 byte but sockResponseString has value '\001'.
According to the protocol value \000 would indicate success. \01 indicates a failure.

  int readSocket( std::string &sockResponseString) {
    // use lambdas as local functions
    auto can_read = [](int s) -> bool {
      fd_set read_set;
      FD_ZERO(&read_set);
      FD_SET(s, &read_set);
      struct timeval timeout {};
      memset(&timeout, 0, sizeof(timeout));
      int rc = select(s + 1, &read_set, NULL, NULL, &timeout);
      return (rc == 1) && FD_ISSET(s, &read_set);
    };
    auto do_read = [&sockResponseString](int s) -> bool {
      // don't need non-blocking checks, code works with both blocking
      // and non-blocking sockets as select() says we're good to read
      char buf[BUFSIZ];
      // char buf[8];  // test if reading small chunks functions
      int nbytes = recv(s, buf, sizeof(buf), 0);
      if (nbytes <= 0)
        return false;
      sockResponseString += std::string(buf, static_cast<size_t>(nbytes));
      return true;
    };

    sockResponseString.clear();
    bool done{};
    while (!done) {
      // keep looping until first read
      if (can_read(Master_sfd) && do_read(Master_sfd)) {
        // then return once all the buffered input is read
        while (!done) {
          if (!can_read(Master_sfd))
            done = true;
          do_read(Master_sfd);
        }
      }
    }
    return static_cast<int>(sockResponseString.size());
  }

[DizzyDon]

Can you show me the output of the following two lines. Put them right after the bytes_read = readSocket(rts); line.

  cout << "bytes_read: " << bytes_read << "\n";
  cout << "[" << rts << "]\n";

[Bengbers]

This is the actual source code:

bytes_read = readSocket( rts);        // number of read bytes
cout << "bytes_read: " << bytes_read << "\n";
cout << "[" << rts << "]\n";
cout << "0  Realm en timestamp:        " << rts << endl;

The next 'cout'-line is line 49:

cout << "Auth-Test: " << success << endl;

In the Eclipse-console I see this output:

bytes_read: 21
[BaseX:37092680721629
0  Realm en timestamp:        BaseX:37092680721629
Auth-Test: ('Auth-Test: ', followed by a small square with 0001 in it)

Copy and paste only results in

bytes_read: 21
[BaseX:37092680721629

[DizzyDon]

Something weird about the output you posted is that it doesn't end with the final ']' character. If you could indulge me, put this line after the cout << "[" << rts << "]\n"; line.

  cout << "----Start\n";
  for (auto ch: rts)
    cout << int(ch) << '\n';
  cout << "----End\n";

EDIT: Try something simple like this:

int readSocket(std::string& sockResponse) {
  sockResponse.clear();
  char buf;
  while (recv(Master_sfd, &buf, 1, 0) > 0 && buf)
    sockResponse.append(1, buf);
  return sockResponse.size();
}

[DizzyDon]

I read some of that spec and I think you should use two different functions depending on whether you want to read a string or a single byte. The string-reading function needs to deal with both embedded 0's and the terminating 0. Embedded 0's are preceeded by 0xFF (and so is 0xFF) in order to distinguish them from the terminating 0. So maybe something like:

int readSocket(std::string& sockResponse) {
  sockResponse.clear();
  int ret = 0;
  bool seen_ff = false; // true if last char was 0xFF
  char buf[MAXBUF];

  while ((ret = recv(Master_sfd, buf, sizeof buf, 0)) > 0) {
    for (int i = 0; i < ret; ++i) {
      if (buf == 0xFF && !seen_ff)
        seen_ff = true;
      else if (buf == 0x00 && !seen_ff)
        return sockResponse.size();
      else {
        sockResponse.append(1, buf);
        seen_ff = false;
      }
    }
  }

  return sockResponse.size();
}

int readSocketByte() {
  char buf;
  if (recv(Master_sfd, &buf, 1, 0) <= 0)
    return -1;
  return (unsigned char)buf;
}

// So if you need to read a single-byte response (like the 0 or 1 error code)
int response = readSocketByte();
if (response == -1) {
    // error
}
...

[seeplus]

Now is the time to get to grips with free Wireshark:
https://www.wireshark.org/

This will enable you to see exactly what's happening over the network. Note that it has a learning curve.

[Bengbers]

Output with

cout << "bytes_read: " << bytes_read << "\n";
cout << "[" << rts << "]\n";
cout << "----Start\n";
  for (auto ch: rts)
    cout << int(ch) << '\n';
  cout << "----End\n";
cout << "0  Realm en timestamp:        " << rts << endl;

bytes_read: 19
[BaseX:700258050364
----Start
66
97
115
101
88
58
55
48
48
50
53
56
48
53
48
51
54
52
0
----End
0  Realm en timestamp:        BaseX:700258050364
Auth-Test: ('Auth-Test: ', followed by a small square with 0001 in it)

Strange?
1 Bytes-read used to be 20 instead of 19.
EDIT, I checked in my RBasex-client and length("Basex:<timestamp>") is 19
2 BaseX:700258050364 appears twice in the output. In both cases copy/paste discards eveything that follows BaseX:700258050364

When executing my app with your 'simple' readSocket as local C/C++ application, read_socket returns bytes_read: 0 and an empty sock_read_string.
When executing in the debugger, it returns similar output as above.

Your remarks on reading and handling the embedded are more or less correct.
In my R-client, I solved this by reading the entire buffer into memory and then removing all the embedded \FF's. After removing those embedded \FF's, a string can end with '...\0\0', thus indicating that all basex-commands were handled correct or with '...\0\1', meaning that the string contains an error-message and that there was an error in handling the basex-command.

Ben
PS. I already noticed that wireshark has a learning curve ;-)

[DizzyDon]

Did you bother to try the corrected code I put in my last post?

[Bengbers]

I replaced MAXBUF with BUFSIZ as MAXBUF was not recognized

Copying your code (Ctrl-C/Ctrl-V) into my code resulted in errors such as "expected primary-expression before '==' token" in line 9.
I'm trying to fix those errors.

After fixing this error, I saw new errors. I corrected(?) your code to

int readSocket(std::string& sockResponse) {
  sockResponse.clear();
  int ret = 0;
  bool seen_ff = false; // true if last char was 0xFF
  char buf[BUFSIZ];
    while ((ret = recv(Master_sfd, buf, sizeof(buf), 0)) > 0) {
    for (int i = 0; i <ret; ++i) {
      if (buf[i] == 0xFF &&!seen_ff)  seen_ff = true;
      else if (buf[i] == 0xFF &&!seen_ff) return sockResponse.size();
      else {
      	sockResponse.append(1, buf[i]);
      	seen_ff = false;
      }
    }
  }
  return sockResponse.size();
}

BTW, I learn a lot from reading your code!

The output is still

bytes_read: 21
[BaseX:19287579790199

EDIT
the output from

cout << "[" << rts << "]\n";
printf("[%s]\n", rts.c_str());

is

[BaseX:29817456722895
[BaseX:29817456722895]

Ben

[seeplus]

PS. I already noticed that wireshark has a learning curve ;-)

It'll be worth the time investment if you do much with networking.

I'm trying to fix those errors.

readSocket() should probably be:

int readSocket(std::string& sockResponse) {
    sockResponse.clear();

    bool seen_ff = false; // true if last char was 0xFF
    char buf[BUFSIZ] {};

    for (int ret = 0; ret = recv(Master_sfd, buf, sizeof buf, 0) > 0; )
        for (int i = 0; i < ret; ++i)
            if (buf[i] == 0xFF && !seen_ff)
                seen_ff = true;
             else {
                 if (buf[i] == 0x00 && !seen_ff)
                     return sockResponse.size();

                 sockResponse.append(1, buf[i]);
                 seen_ff = false;
             }

    return sockResponse.size();
}

Note that you're mixing signed and unsigned!

[kbw]

Dunno what BaseX is, beyond being a database of sorts.

In the language binding section of the documentation, there are some C++ samples. One of which has this code: https://github.com/bsella/BaseX_CppClient/blob/master/basexdbc.c#L94

It's the authentication method. I haven't read it in detail, but you might find it helpful.

[Bengbers]

BaseX is an open source XML-database with the best XQuery-engine I know of (that's why I am using it).

There are a few C or C++-examples but the are either too complicated for someone who is just starting to use C++ or they use plain C for authenticating. Since I want to learn C++, I am trying to convert all the C-code from the examples to C++.

I have done the same for R (which resulted in the RBaseX-client) and am confident that I will also manage to do the samen in C++.
I am retired so time won't be a problem ;-)

[George P]

Since I want to learn C++, I am trying to convert all the C-code from the examples to C++.

Learn the basics of C++ first without the idea of converting code or the overhead of working with high-level code.

Learn C++ might be helpful.
https://www.learncpp.com/