Micro SD Card Duplicator 6

I’ve been stuck, not on what to do next but on time. Late nights, sick kids, friends over. I can’t complain really, but at the same time I haven’t made any progress on this project in over a week!

I figure there are two ways forward. I can trace through the code and see how the ‘usual’ file read and write operations work using the standard classes, or I can start from MULITPLE_BLOCK_WRITE in the Sd2Card class and work out how to expose and utilise that through the FAT file system. Other options?

Actually, I could switch to .NET Micro Framework and use my FEZ Panda II.

Tracing SD.write()

No time to write a detailed explanation. A quick summary of what I’ve done:

1. Committed SD_Speed_3 to Unfuddle
2. SD_Speed_3.ino calls SD.begin() to initialise the card
3. This calls card.init() with parameters
4. card is private member Sd2Card of class SDClass in SD.h
5. Sd2Card is not included by SD.h but SdFat.h and SdFatUtil.h are. Checking last blog post, SdFat.h includes Sd2Card.h.
6. First parameter in card.init() sets clock rate via Sd2Card::init() –> Sd2Card::setSckRate()
7. End of analysis of card start-up. Looking at SD.open().
8. SD_Speed_3.ino declares a File myFile; which is of the FIle class in SD.h – a wrapper class over the SdFile class
9. The SdFile class is delcared in SdFat.h and implemented in SdFile.cpp
10. Jumping to myFile.write().
11. SdFat.h defines three File::write() functions. One for a single byte, one for a pointer buffer of defined length and one for a c-string. My sketch is currently writing only a single byte.
12. write() for a single byte calls write() for a string of defined length 1.
13. Reading size_t SdFile::write(const void* buf, uint16_t nbyte). Sucky, what’s with the use of goto?
14. Now we’re down into blocks and clusters. I can see what’s going on, kinda. But I really need to have a definition of blocks and clusters in my head. Need FAT specification.
15. Read FAT32 File System Specification and various other source. This one was good.

So to sum up a whole load of stuff I’m writing a byte at a time. Each byte I write causes a block of 512 bytes to be read from the card, updated, and written back to the card. I’m incurring massive overhead by not writing more than a byte at a time.

I rejigged my sketch from my SD Card Duplicator 4 post and have got writes up to 47kB/s by writing 10 bytes at a time. That’s not a huge improvement, but it’s better. Now I’m reading a block from the SD card, making ten changes to it, then writing it back. It’s more efficient, but not by much. At 50 bytes per write, I’m hitting 55kB/s.

So I have learned a few things. Looking to use MULTIPLE_BLOCK_WRITE will likely improve performance, but I’m dealing with the wrong level of abstraction there. The fact is that the SD card is working in blocks and so, separately, is the FAT file system. Writing ASCII characters to the card this way incurs one or more read, update, write-back cycles if at least 512 bytes each.

FAT

I’ve spent a lot of the day reading about the FAT file system. It’s not too complex, but it’s also not much fun. It was invented in, what, the late 70’s? It’s almost as old as me! FAT works in clusters and blocks too, and it’s interesting to see how the SD card’s blocks and FAT blocks are not the same thing. FAT generally formats data in 512 byte blocks. SD cards too usually work in 512 byte blocks. But neither is constrained to this size, which is interesting.

I guess if the SD card was working with 1024 byte blocks and FAT was using 512 byte blocks (just for example) then every read from and write to an SD card would move two FAT blocks. Conversely, if FAT was using 2048 byte blocks and the SD card was using 1024 byte blocks, then reading a FAT block would require two read operations on the SD card.

It seems that FAT and SD are both usually working on 512 byte blocks. But I read something that talked about the FAT format “not necessarily observing the SD block boundaries”. So that might mean even at the same block size, multiple reads/writes are required to properly manipulate FAT blocks.

Writing from Scratch?

I’m tempted to start from scratch. Really. A class to handle SPI operations (like, say, the existing SPI class included with Arduino) then one to handle the SD card SPI protocol. From there up to the FAT file system. I’d like to be able to do this:

SDCard cardSource = New SDCard(PIN_1);
SDCard cardDestination1 = New SDCard(PIN_2);
SDCard cardDestination2 = New SDCard(PIN_3);
SDCard* cards[] = {cardDestination1, cardDestination2};
File file = cardSource->getFirstFile();
while(file) {
  file.copyTo(cards);
  file.getNext();
}

That way the source file’s blocks are only read once and are written to as many destination cards as required. Only one buffer would be required too. But oh the work! I can’t imagine the hours of slashing through code to interpret FAT boot sectors and file allocation tables. There’s enough of that kind of code already, but there are a few critical issues with what I’ve found so far.

First, nobody seems to have considered the need to access more than one SD card. Existing code makes a single object to read/write from what is assumed to be the only card attached to the system. This means it would require a significant re-write to allow the instantiation of multiple card objects.

Second, and this gets bonus points for being my pet peeve. Code is very ‘C’ and not very ‘C++’. I can’t help but feel that this scenario is perfect for OO techniques. Then again, if I had some more C experience I might think not.

I’m at a crossroads. I’ll either decide to sink over a hundred hours into this project, or I’ll scrap it. I have other time pressures and I think it’s quite likely I’ll get up to my armpits in it before conceding I can’t finish it. What to do?

[That is all]