Saving huge arrays for my game

[sanoj00]

I have been coding a while on a 2D random terrain game. How would I go about saving the maps? I have an array for blocks, lighting, background and background lighting. The lighting is done in real-time, so exclude that.

But with two 32000x3200 arrays, I still need to store 204800000 separate numbers in a file. Oh god. How can I have this? I could write down the separate numbers, but...yeah.

If it matters, this is made in freeglut.

[Albatross]

Whew, that's a good chunk.

First off, let's run some numbers. Let's assume you're using 32-bit integers. That means that each integer is 4 bytes long. That comes out to 819.2 megabytes of data. As far as games go, that's not absolutely horrendous.

However, if you still want to be a bit more efficient on save sizes, then you can take the Minecraft approach and split your arrays down into smaller chunks (maybe 32 x 32), and save them only when the game needs to create one to generate unbroken terrain.

Alternatively (and I think this is a better albeit more complex solution), use chunks, but find a way to regenerate the chunks every time you need them, and only save them after they've been in some way changed from their default generated state. To clarify, when a player goes into a chunk, that chunk isn't saved until the chunk is changed in some way. Otherwise, it's simply regenerated over and over again, but ends up looking the same each time.

Good luck!

-Albatross

[SIK]

if these maps are to be loaded and saved quite frequently during game play then I would also suggest that only required chunks be saved.

if however these maps are to be loaded and saved near once during game play then i would suggest using the following crude methods for saving and loading map data to and from file:

#define   x_r    32000
#define   y_r    3200

void SaveMapToFile(int map[x_r][y_r], const std::string & strFile)
{
   FILE * fp = fopen(strFile.c_str(), "wb");
   if (!fp)    throw CAppException(eApp_FileCreateError, "Error Creating File [%s] due to [%s]", strFile.c_str(), strerror(errno));

   int nBytesToWrite = sizeof(int) * x_r * y_r;
   int nBytesWritten = fwrite(map, sizeof(char), nBytesToWrite, fp);
   fclose(fp);

   if (nBytesWritten != nBytesToWrite)    throw  CAppException(eApp_FileWriteError, "File Write Error [%s] - nBytesWritten [%d] != nBytesToWrite [%d] due to [%s]", strFile.c_str(), nBytesWritten, nBytesToWrite, strerror(errno));
}


void LoadMapFromFile(int map[x_r][y_r], const std::string & strFile)
{
   FILE * fp = fopen(strFile.c_str(), "rb");
   if (!fp)    throw  CAppException(eApp_FileOpenError, "Error Opening File [%s] due to [%s]", strFile.c_str(), strerror(errno));

   int nBytesToRead = sizeof(int) * x_r * y_r;
   int nBytesRead = fread(map, sizeof(char), nBytesToRead, fp);
   fclose(fp);

   if (nBytesRead != nBytesToRead)    throw  CAppException(eApp_FileReadError, "File Read Error [%s] - nBytesRead [%d] != nBytesToRead [%d] due to [%s]", strFile.c_str(), nBytesRead, nBytesToRead, strerror(errno));
}

saving a single map took 5 secs on a vm on my pc and sub second to load.