Has anyone worked with embedded databases? A little advice?

[jnz]

I have a diagnostic project that will be plugged into many different devices over an RS-232 connection.

I can handle the work, the code, everything is fine, except I'm a little hung up on the idea of how to support multiple devices. A database of the relevant into seems obvious. Load that into a struct and treat it all the same. The issue I have is that:

A. I'm low on ROM, or at least I will be.
B. I'd like a back end format that's easy to work with. CSV or Excel is preferred since I can edit those with readily available tools. Unlike SQL that I think is way overkill for this.

Some objects I have to deal with are that each device has common and unique locations that direct me to things to look for or edit on the host device. These might be bits, bytes, words, 46 byte blocks, whatever. I also have formats that each device may use PER setting, and endian, etc etc.

_device.id = 0x21;                        //define the most common module
_device.serialnumber_addr = 0x31;
_device.serialnumber_loc = 48;       //location starts 6 bytes in on ID 0x31
_device.serialnumber_len = 16;   
_device.endian = END_LITTLE;
_device.hasLCD = 1;
_device.serialLocked = 0;
_device.voltage = 24;

if (this.id == 0x22) {
_device.endian = END_BIG;
_device.serialLocked = 1;
_device.voltage = 12;
}

....

This might be an example for ID 0x21, but I have probably 48 times what I've shown here, and at least 6 devices I'd like to support on the ROM.

I'm thinking about making an excel or cvs that I could write a little Python script to convert into a header file that populates the struct for each setting. Keeping in mind to  probably start with the most common modules that way if module ID 0x22 doesn't need a unique serial number address, it doesn't overwrite the "common" one defined first.

Has anyone seen or done anything like this? Tips? Maybe I'm missing something obvious in terms of C databases?

[pablo2048]

Hi,
what about antelope? It's Adam Dunkels database engine for low memory embedded system and sensors - part of Contiki operating system.

[CocaCola]

Is there a reason you can't keep it dead simple and use a flat database on an EEPROM or removal flash media, that can be updated easily?  Might not be the most efficient but it's one of the easiest to implement,l and offers lots of flexibility on the pc side as most database programs will import or export to flat, or you can just edit the flat...

[jnz]

I like the idea of antelope, but I don't really need a database on hand for runtime. I only need to determine which system ID I'm working with and load the 600 or so bytes of data for that. If I needed runtime add/drop from a database this would be a whole different story. I looked up a PDF for antelope and it quickly reminded me that I'm not very smart!

CocaCola, No, there isn't a reason I can't just keep it flat other than I need to update it a few times a year and technically someone else "could" update it with limited diagnostic tools and knowledge without me if it was in a friendly excel/csv format. But I'll need a way of representing all the data in csv, then getting that into a C/header. I'm running into a ROM usage issue for the most part.

Every set of devices can't have it's own full set of data, I just don't have enough space for that. But... if I took the two most common implementations and then just added "overwrite" data for the unique aspects of each, I might have enough room this year and going forward into 2018+.

It all has to be on chip as well, the product is entirely self contained, so no SD cards or USB update. The next version will be wireless, so far less of an issue, but the small (existing) version must be as cheap as possible.

[Signal]

jnz, it is still not clear for me, do you plan to handle CSV backend at runtime or at compilation stage?
For latter the preprocessor's "#define" macros with parameters (CSV) could help without any external (python) parsers:

device_descr.txt:
ID(0x21,0x31,48,16,END_LITTLE,1,0,24)
ID(0x22,0x33,48,16,END_LITTLE,1,0,24)
...

some_header_file.h:
...
#define ID(_id, _snadr, _snloc,.... ) \
...
#include "device_descr.txt"

Even structs with variable number of fields or with variable length arrays can be coded by preprocessor.
Metaprogramming by preprocessor rules ;)

[Mentor]

I only need to determine which system ID I'm working with and load the 600 or so bytes of data for that.

What about using an external eeprom, like the 24c04 or something newer? I think that's the most common way of storing configuration data.

You will have to create the functions to read and write the eeprom and define the way you will store the data. So that's probably just a matter of reading from a table.

[bobcat1]

Hi all

You can use a struct placed inside a union ,that way you can convert data base into streem of char and save it into an EEPROM or a file (if you have file system)
You can also build it into an XML structure for easy search.

Bobi