Micro SD Card Duplicator 2

Yesterday I began speed testing reads and writes on micro SD cards. Much as I did for the Basic GPS Data Logger, I began by using Examples code in the Arduino editor and breaking it down into functions. I soon had this code. For brevity I have deleted comments, which doesn't help understanding I'm sorry. Look at the ReadWrite and Datalogger examples code under SD in the Arduino editor.



I'll run through setup() quickly to explain it. The goal is to write files of different sizes to an SD card and calculate the bits/second rate of the writing operation.

void setup()
{
  Serial.begin(115200);

  Serial.print("Initializing SD card ... ");
  pinMode(10, OUTPUT);

  if (!SD.begin(chipSelect)) {
    Serial.println("failed!");
    return;
  }
  Serial.println("success.");
  
  for (unsigned long i = 1; i <= writeSize; i*=2) {
    if (fileExists(fileName)) fileDelete(fileName);
    
    // Found that deleting and creating a file in subsequent operations
    // resulted in non-deletion and the file being reopened and appended.
    delay(100);
  
    unsigned long timeAlpha = millis();
    createFile(fileName);
    if (fileExists(fileName)) fileFill(fileName, i);
    unsigned long length = fileSize(fileName);
    unsigned long timeDelta = millis() - timeAlpha;  

    //fileRead(fileName);
    //Serial.println();
    Serial.print((float)length/timeDelta*1000);
    Serial.print(" bytes/sec. for ");
    Serial.print(length);
    Serial.print(" bytes read. ");
    Serial.print(i);
    Serial.println(" bytes written.");
  }
  
  Serial.println();
  Serial.println("End of program");
}

The for loop is where the fun begins. You can see the first file being written is 1 byte and doubles each time a file is written, up to writeSize as the maximum text size. The functions createFile(), fileFill() and so on are defined in my code. The behaviour is this:

1. Set the file size to 1 byte (the for loop)
2. If the file exists, delete it.
3. Start timing the operation.
4. Create a new file.
5. Fill the file
6. Read back the file length
7. Stop timing
8. Do calculations and display
9. End of for loop
10. Double the file size
11. Repeat

The first issue I encountered was that if I deleted the file and immediately recreated it, the delete/create operation failed to occur at all. What I got was data appended to the file as if it hadn't been cleared at all. (Actually, maybe I should empty the file instead of deleting it). But adding a delay gave the card time to execute the delete operation and it now works correctly - up to a point. The output looks like this:

Initializing SD card ... success.
4.22 bytes/sec. for 1 bytes read. 1 bytes written.
8.47 bytes/sec. for 2 bytes read. 2 bytes written.
24.39 bytes/sec. for 4 bytes read. 4 bytes written.
50.31 bytes/sec. for 8 bytes read. 8 bytes written.
100.63 bytes/sec. for 16 bytes read. 16 bytes written.
198.76 bytes/sec. for 32 bytes read. 32 bytes written.
392.64 bytes/sec. for 64 bytes read. 64 bytes written.
766.47 bytes/sec. for 128 bytes read. 128 bytes written.
1446.33 bytes/sec. for 256 bytes read. 256 bytes written.
2612.24 bytes/sec. for 512 bytes read. 512 bytes written.
4338.98 bytes/sec. for 1024 bytes read. 1024 bytes written.
5237.85 bytes/sec. for 2048 bytes read. 2048 bytes written.
7366.91 bytes/sec. for 4096 bytes read. 4096 bytes written.
9102.22 bytes/sec. for 8192 bytes read. 8192 bytes written.
10395.94 bytes/sec. for 16384 bytes read. 16384 bytes written.
0.00 bytes/sec. for 0 bytes read. 32768 bytes written.
0.00 bytes/sec. for 0 bytes read. 65536 bytes written.

End of program

Doubling, doubling, d'oh! Somewhere it fails. It looks suspiciously like an integer overflow. Your regular, garden variety integer data type can only store values up to 32,767. So maybe, just maybe, I'm overflowing an integer. You can see the second last line of the results above fails to write 32,768 bytes. I changed this line in setup(), from

    if (fileExists(fileName)) fileFill(fileName, i);

to

    if (fileExists(fileName)) fileFill(fileName, i - 1);

and sure enough, got one more correct result at 32,767 bytes.

Initializing SD card ... success.
0.00 bytes/sec. for 0 bytes read. 1 bytes written.
6.13 bytes/sec. for 1 bytes read. 2 bytes written.
19.11 bytes/sec. for 3 bytes read. 4 bytes written.
42.68 bytes/sec. for 7 bytes read. 8 bytes written.
93.75 bytes/sec. for 15 bytes read. 16 bytes written.
193.75 bytes/sec. for 31 bytes read. 32 bytes written.
386.50 bytes/sec. for 63 bytes read. 64 bytes written.
755.95 bytes/sec. for 127 bytes read. 128 bytes written.
947.96 bytes/sec. for 255 bytes read. 256 bytes written.
2607.14 bytes/sec. for 511 bytes read. 512 bytes written.
4353.19 bytes/sec. for 1023 bytes read. 1024 bytes written.
5235.29 bytes/sec. for 2047 bytes read. 2048 bytes written.
7234.98 bytes/sec. for 4095 bytes read. 4096 bytes written.
9111.23 bytes/sec. for 8191 bytes read. 8192 bytes written.
10415.13 bytes/sec. for 16383 bytes read. 16384 bytes written.
10821.33 bytes/sec. for 32767 bytes read. 32768 bytes written.
0.00 bytes/sec. for 0 bytes read. 65536 bytes written.

End of program

Damn. I see it now. The for loop in setup() is using and unsigned long data type, but this is being implicitly converted to int when it's passed to fileFill(). Rookie mistake. fileFill() has this correction made in two places:

boolean fileFill(char* file, unsigned long bytes)
{
  File dataFile = SD.open(file, FILE_WRITE);
  if (dataFile) {
    for (unsigned long i = 0; i < bytes; i++)
    {
      dataFile.print("A");
    }
    dataFile.close();
    return true;
  }  
  else {
    Serial.print("Error opening ");
    Serial.println(fileName);
    return false;
  } 
}

The counter variable in fileRead() likewise needed changing from int data type to unsigned long. A quick upload and run verifies we're ok. So I dropped the "- 1" I added earlier in the setup() line, and pumped up the maximum file size to 1,000,000 bytes. If I could be bothered, I could produce a pretty graph of the write speed using Processing, a progenitor of Arduino. But I can see the pattern, I'm levelling out at less than 12kB/s. And thus the point of this exercise.

First, I need to check that I've calculated my speed correctly. Second, I need to see what I can do to increase it! Oops. First mistake found. My timing was including writes and reads. By removing the read operations, the write speed is now about 21.5kB/s.

I'm going to leave the blog post here for now and go play. Final code is here.

[That is all]