Creating Custom Binary Resource Files

Ever try using the standard resource file methodology included with Visual Basic? You likely found the utilities somewhat lacking (since the good ones are only included with VC++!) and difficult to use. Also, the result of using the standard resource methodology is that you end up with a HUGE FAT EXE file. Not very pleasant or professional, in my mind :)

Alternatives? Well you could store your bitmaps, waves, etc, as is in a separate directory... this doesn't sound so bad, but if you plan to distribute your game, aren't you a little worried that someone will change your files, redistribute the game, and take credit? I would be! In fact, I am! :)

But fear not! Your knight in shining armour is here! This tutorial will teach you how to create your own stand-alone resource files that need not be included in an EXE. Advantages to making your own resource files:

  • They are harder to tamper with
  • You can encrypt them
  • You can modify them at runtime with code (make your own editors!)
  • You can store any type of file you wish
  • You don't need to recompile your project just to change your data
  • They are fun! (To me anyway, what a nerd I am! Heh)


    Files and Binary Access Lessons

    Enough fluff, down to business. There are a few lessons that you need to learn regarding files and binary access before we can continue. First of all, you'll have to be constantly aware of the byte size of the various data types. Each data type is described by a set number of bytes, and as a result, they will take up a specified amount of space in your file:

  • BYTE - one byte
  • STRING - one byte per character
  • BOOLEAN - two bytes
  • INTEGER - two bytes
  • LONG - four bytes
  • SINGLE - four bytes
  • DOUBLES - eight bytes

    If you're using any other data types you're probably being a goof :) Ok, now the interesting thing is when you have an ARRAY of a specific type, when you store it in a file it takes up space according to the size of the data type and the number of elements in the array (SIZE*ELEMENTS). So an array of 4 INTEGERS would take up 8 bytes. Note that the values would be stored in the same order as they were in the array.

    User Defined Types (UDT's) act similarly. A UDT containing one BYTE and 3 LONGS, for example, would take up 13 bytes in a file. Note, the values will be stored in the order in which the UDT was defined. So if your UDT declaration looked like this:

    Private Type NEWTYPE
    bytTest as Byte
    intTest as Integer
    lngTest as Long
    End Type

    Dim udtNewType as NEWTYPE

    Then the BYTE ("bytTest") would be stored first, then the INTEGER ("intTest"), and finally the LONG ("lngTest"). You can also have ARRAYS of UDT's, which would result in orderly storage of UDT data (as just described) according to the order of elements in the array (see above).

    Confused yet? I thought so :)

    In order to access a binary file we need to use the "open" command with the "Binary" keyword:

    This would open a file in the application's directory called "TEST.BMP". If this file did not already exist, it would be created. The "Read Lock Write" portion of the statement indicates that we wish to read from this file and stop others from writing to it. You can alter these as you wish, even doubling up, "Read Write Lock Read Write" would allow you to read and write to a file while locking any other program from doing so.

    The "As #1" portion describes the number by which we would like to refer to this file. This number is used in "Get" and "Put" statements as well as in the "Close" command. If you would rather not hardcode the number you can use the FreeFile() function to give you a file handle (it'll be an INTEGER by the way) that's not currently in use:

    Dim intFileNum as Integer

    You can then use "intFileNum" later in code to refer to the file you've opened. Now you know how to open a file, next you need to learn how to close it. It is important that you remember to do so, otherwise errors may arise later in your code when you try to access a file that is still open and locked from a previous call. All you have to do is use the "Close" command followed by the file handle which you would like to close:

    Simple as that! Ok, now just a little bit more to learn before you can start building your own file format. The last thing you need to know is how to use the "Dir", "Put", "Get", "Seek", and "LOF" functions... "is THAT all?" you ask? :P

    Use the "Dir" function if you want to find out if a specific file or directory is currently in existence. For example, if your user wants to open a binary file for reading, you'd probably want to ensure that the file is actually there before proceeding!

    The "Dir" function will return a zero-length string ("") if the path specified is not valid. If the path is valid, a filename string will be returned.

    The "Put" statement will take a supplied piece of data and store it at the specified location within the given (open) file. Observe:

    This code would store the variable "udtNewType" in the file referenced by "intFileNum" at the first available location within the file (byte number "1"). Now to do the reverse of this, we use the "Get" statement:

    This code would extract data to fill the variable "udtNewType" from the file referenced by "intFileNum" starting at byte number "1". Now, you have to be careful, the "Get" and "Put" statements are none too smart, they'll do only exactly what you tell them. If you do not enter the correct byte at which to start, you will obtain unexpected results. Your variable will be filled with whatever data happens to be at that location in the file, there is no data-type checking here. If you "Get" an INTEGER, whatever 2 bytes happen to be at the location you specify, those are the two bytes you're a-gonna get! It doesn't matter if they "used to be" a STRING or part of a LONG or whatever, they'll now be treated as an integer.

    It is possible to leave the second parameter in a "Put" or "Get" statement blank. If you do this, the data will be stored/extracted from the location at which the LAST "Put" or "Get" statement left off. So if you do something like this:

    You'll end up with the first UDT stored at the beginning of the file, and the second one stored immediately after. This is kinda handy since it takes your mind off of what-goes-where in situations where you are reading/writing sequentially. Now, if you need to find out WHERE the current read/write location is, you can use the "Seek" function:

    This will return a LONG describing the current read/write location for the given file handle ("intFileNum"). If you were to perform a "Put" or "Get" without specifying a specific byte location, the value returned by "Seek" is the location at which the "Put"/"Get" would occur.

    Lastly, we have the "LOF" function. Simply pass a file handle to the "LOF" function and it will spit out a LONG describing the current size of the file in bytes:

    This can be useful in many situations, as I'm sure you can imagine. Ok, now you know all of the functions you need to set up your own file format. Take a deep breath, cuz here we go!


    Creating a Custom Resource File Format

    Storing data in binary format is not terribly difficult. All you'd have to do is read the data from a file and write it to another file. But, once you start compiling multiple files into a single binary, you run into referencing problems. Where does one file start and the other end? How do I retrieve only one of the files at a time? How do I ensure that my file has not been tampered with? .. all of these concerns can be addressed through judicious use of HEADER structures.

    The first structure I like to use is what I call my FILEHEADER (similar to the bitmap style file format):

    Private Type FILEHEADER
    intNumFiles As Integer
    lngFileSize As Long
    End Type

    This UDT contains data on the file in general. The overall size of the file is stored in the "lngFileSize" variable for use in validity checking. When you open a binary file, you should check the size of it (using the "LOF" function) against the size stored in the "lngFileSize" member, if they are conflicting, then you know that someone has tampered with your file! There are other validity checking methods, but I won't go into them here.

    The "intNumFiles" variable will describe the total number of original files that are now stored within this binary. This is very useful in the next step, the INFOHEADER:

    Private Type INFOHEADER
    lngFileSize As Long
    lngFileStart As Long
    strFileName As String * 16
    End Type

    You will need one INFOHEADER variable for each file you store in your binary. The INFOHEADER describes HOW the file is stored within the binary and what the file's reference name is.

    "lngFileSize" is the size of the stored file, "lngFileStart" is the starting byte location within the binary at which this file was "Put". "strFileName" is some sort of string handle that you can use to retrieve files from the binary.

    So now we have our header structures, all we need is some data. So, SAY you want to load three files into a single binary... go on... SAY IT! ... I'm waiting!

    Aw, you're no fun :P

    Ok, our example files will be called "SAMPLE1.BMP", "SAMPLE2.WAV", and "SAMPLE3.TXT". With this information we can create and define our header structures. First we'll need to create an INFOHEADER array containing 3 elements (for our 3 files) and open the files to determine their size (and store this in the "lngFileSize" member):

    Dim intSample1File As Integer
    Dim intSample2File As Integer
    Dim intSample3File As Integer
    Dim FileHead As FILEHEADER
    Dim InfoHead() As INFOHEADER

    Now that we've stored the file sizes we can continue and store their names:

    Things get a tad tricky here. In order to fill out the "lngFileStart" member of the INFOHEADER structure we have to first determine the amount of space that will be taken up by the INFOHEADER and the FILEHEADER:

    Dim lngFileStart as Long

    Our first file ("SAMPLE1.BMP") will be stored at the location given by this variable, "lngFileStart". To calculate "lngFileStart" we have to determine how many bytes our headers will take up, and how many of them there are. The FILEHEADER is made up of one INTEGER and one LONG, that's 6 bytes. The INFOHEADER is made up of two LONGS and one 16 character STRING, that's 24 bytes. So we add 6 bytes plus 24 times the number of INFOHEADERS we're using (in this case, 3). We then add 1 since we want to start at the byte immediately after the last byte of the INFOHEADER.

    This code stores the "lngFileStart" member for each of the INFOHEADER structures. It increments the "lngFileStart" variable after each INFOHEADER index by adding the size of the file. In this way, the first file will be stored immediately following the headers, and each subsequent file will be stored linearly thereafter.

    So, now our INFOHEADER is filled, we just need to fill out the FILEHEADER and then store the data. First, the FILEHEADER:

    This adds up the size of the headers and of the data we're going to store and places it into the "lngFileSize" member. Now we "Get" the data from our three files using appropriately sized byte arrays and close the files:

    Dim bytSample1Data() As Byte
    Dim bytSample2Data() As Byte
    Dim bytSample3Data() As Byte

    Almost done! Now we just open a new file for writing, and "Put" all of our data:

    Dim intBinaryFile as Integer

    That's all there is to it! You've now stored these three files in a new binary file called "BINARY.DAT". To extract the data, simply reverse the process using the data stored in the FILEHEADER and INFOHEADER structures stored at the start of the binary file. To see this put to use, check out my Binary Files Project source code.