0
\$\begingroup\$

I want to save data associated to a timestamp (like a datalogger) along a month, in a very small memory (probably only 1024bytes). I was thinking using epoch time to save timestamp but it already take 4bytes. So I'm looking a way to save size of date or time data, idea?

EDIT: To be more accurate, the periodicity of the records will be change according the range of period (max a month): should be every 15min or each hour for example. Anyway, in this case it's a non-sense to have 'second' resolution. I though also to record a start date/time value and just a time offset for each record !

\$\endgroup\$
  • \$\begingroup\$ How often do you save a record with its time-stamp? \$\endgroup\$ – Bence Kaulics May 21 '16 at 10:47
  • \$\begingroup\$ How much resolution? 1 month = 2,678,400 seconds and this is a 22 bit number hence only 3 bytes are needed for 1 second resolution. \$\endgroup\$ – Andy aka May 21 '16 at 10:50
  • \$\begingroup\$ You are right both ! I just detail the process to give more informations \$\endgroup\$ – cHab May 21 '16 at 11:56
1
\$\begingroup\$

Start off by using a compact representation, a binary number of ticks for time, rather than something less efficient like an ASCII decimal string. Choose your tick to be as big as possible, to save bits.

Then, if you can guarrantee that your events occur with a specifiable maximum period between them (and if you can't, you could fix this by recording a dummy non-event if the period was about to be exceeded), then recording the deltas between events will be a simple next step. If your events are one per day, and you want to unambiguously record their occurence over one month, to the same resolution, each delta will need 5 fewer bits than the absolute time.

Each time you have to insert a dummy packing event, you introduce some inefficiency, so choose your minimum guarranteed period carefully.

Finally, if the typical occurence of your events is very non-random, you could consider coding your deltas with Huffman, or Huffman-like, coding, to further wring out that last bit of non uncertainty from them. This is where you could make your packing events less expensive, allocating fewer bits to them than to the real events.

For instance, a packing event could be a single one bit. A time delta would have a leading zero and always be n bits. That is not true Huffman coding, but does a similar thing of reducing the number of bits required, depending on your choice of minimum time period. The shorter the minimum time, the smaller you can make n, but the more packing events you require. You save (n-1) bits per packing event, but suffer one extra bit per real event. Only you know what the statistics of your event arrival times is, so only you would be able to come up with an optimum choice of minimum period.

EDIT: having seen your edit

If the recording times are regular (your MCU in charge of the timing), then no time needs to be stored at all, apart from the start time. If they are nominally regular, but an outside system is delivering the measurements sometimes a minute late, sometimes 3 minutes late, and that variation matters, then just record that variation from expected, not the whole delta time

Basically, anything you can predict, you don't have to store. Work hard to predict all you can. Then save the differences, to the minimum resolution, in a compact form.

\$\endgroup\$
  • \$\begingroup\$ Thanks. Sure, ASCII was already banned of the equation :) The idea to record only the delta (time offset) was already in my mind, but I didn't though to use a bit counter for record the deltas as I will now exactly the occurrence... I will give a look to Huffman coding method! \$\endgroup\$ – cHab May 21 '16 at 12:09
  • \$\begingroup\$ @cHab just spotted your edit. From the fact that you asked the question, I assumed that the recording events were irregular. \$\endgroup\$ – Neil_UK May 24 '16 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.