---

I have done a roast generating raw data without any filtering whatsoever, and have now done some experimenting in Excel. What I found was that I could get quite good cancellation of the jittering in the RoR signal simply by averaging successive values. I got quite reasonable results with a sample period of 4 seconds, averaging all the values of RoR in the sample and then moving on to the next sample period (i.e. this is not a running sample, which would be much less effective at cancelling the jitter due to rounding errors). I still got significant up and down movement of RoR, but I think that is not noise, but I think it is actual rising and falling of RoR in response to the heater going on and off (and possibly also bean temperature variation). It seems to have a certain periodicity to it. By increasing the sample size to 10 seconds the RoR curve is further smoothed out, but like the 4 second sample size there is still no distortion, because it is a simple average.

Thinking about this, I think this approach of averaging samples of RoR is a more direct and effective method of eliminating the jittering than the filtering technique, as it goes directly to the core of the problem - the rounding errors in the digitised samples of temperature. What we are trying to measure is the temperature at this moment, and the rate of change at this moment - this is not a function of past changes, and the most ideal measurement would be over a time period as short as possible. I don't see the filter model as being suitable in this case. When the total change in the period is relatively similar in size to the sampling error/rounding error, the calculated rate of change will fluctuate wildly around the actual momentary rate of change of temperature. What we want to see ideally is the actual rate of change at this moment, as independently of time before and after this moment as we can achieve.

We can take this argument further. Ever since Brad was asking about ADC sampling settings and Jim's answer I have been harbouring reservations about this (indeed before, for that matter). I agree that we need to sample temperature at an accuracy as high as possible in order to minimise sampling errors and get a meaningful value for RoR, but we also need to balance that with the number of samples, and here the inverse relation is true. The quantisation noise at 16 bits is 4 times the quantisation noise at 18 bits, but at 16 bits we can get 4 times as many samples per second. Which gives better results - more samples averaged before differentiating or less samples with lower noise? Right at this moment I cannot quite get my head around this question, whether the two are equivalent or whether one or the other is better, but my gut feeling is as follows: given sufficient number of samples in the sampling period the error due to quantisation will tend to zero, therefore [but is this exactly cancelled out by the increase in noise level???] slightly reducing the bit resolution and correspondingly increasing sample size ought to be an improvement.

In addition to the previous questionable gain we have two definite improvements from a lower resolution: (1) the signal noise is lower at 16 bits than at 18 bits, and (2) if we want to insist on a fixed number of samples per second, the sample size per second for two thermocouples at 16 bits is far more than 4 times the sample size at 18 bits.

What is more, if we give priority to T0 over other thermocouples and optimise the software for a total of two probes, then the sample size for T0 is almost doubled - that is, if we sample at 16 bits, then reserve one sample per second shared between all the thermocouples from T1 to T3 and use all the other 14 samples for T0 which is our highest priority, and then we average all 14 samples to give one reading of T0, and by comparison with the previous averaged T0 reading one reading of RoR, then we would certainly get a much cleaner reading of RoR with zero distortion and almost immediate response. If 14 samples is not enough we could average over 2 seconds and I am sure 28 samples will be more than adequate. A slight variation on this approach, significantly more complicated to program, might be to assume that all the 14 samples within the period approximate to a gaussian distribution on a straight line whose slope corresponds to the average rate of change DURING that one second period, with or without supplementing input from the geometric average of the samples in the preceding and subsequent one second periods. The latter would offer a sort of nearest neighbour filtration on the average rate of change of the samples within a one second period - draw a straight line between all the samples in the one second period, calculate its gradient, then average that with the difference between the geometric average of the immediately previous second and the geometric average of the immediately subsequent second.

Note that each individual sample can only be used once - if we calculate a running average of the most recent 10 samples every time we fetch a new sample from the ADC, then we are optimising the accuracy of the momentary temperature but this has a devastating effect on RoR, because the difference between successive averaged samples is determined exclusively by the first and last samples averaged, i.e. a sample size of only 2, so the RoR will fluctuate wildly.

I also think it is worth testing the validity of each individual sample, and throwing out any samples that are out of range before averaging. Any samples thrown out should be logged on the serial port, so that we get some feedback on any error states that may be arising, from grounded thermocouples for example. I have already mentioned the GreenBean logging project in the UK a few times - GreenBean reads 4 samples per second using a Max6675 chip for each probe, tests each sample for validity, then throws out invalid samples and averages the rest of the samples for one second (he is not using RoR however).

Finally, I have tried using the sample averaging feature in pBourbon, but without success - setting all filtration in aBourbon to 0, I set sample size in the configuration file for pBourbon to 10. The RoR is fluctuating just as wildly as with a sample size of 1, or at least close to that, unlike the results of my excel experiments. I think this is using a running average isn't it?

By the way, I personally don't see any need for filtering on anything at all other than RoR - I am not getting any noise problem at all, except with RoR. Firstly on ambient temperature: my experiments with breathing on it show that it is extraordinarily sensitive to change, but I see this as valid data not noise, and should be used in the correction of the raw thermocouple readings because the ambient temperature does actually change and the cold junction error changes with it. Insulating the TC4 from temperature changes is obviously worthwhile. Secondly on thermocouple temperature readings: again I have no problem of noise, and get extremely smooth temperature curves with zero filtering, both on my HotPot and on my HottoP. Obviously with some types of roaster the position will be different.

Here are some of my results. First the raw data from a roast on the HottoP (with one BT thermocouple), as displayed by the TC4 in Processing. The black trace is BT, green is RoR with zero filtration. (The red line shows the displayed T1 but the T1 contacts are open-circuit; I subsequently tried shorting the T1 contacts, and the proportional relation between T1 and T0 is thereby removed). To optimise accuracy I sent all the data to the serial port with three decimal places instead of one.



Next just for comparison a second roast under similar conditions using the recommended filter values (BT, ET, Rise, RoR, Amb 10, 10, 85, 80, 70 respectively).



Now to my excel experiments, the first of which are conducted on the raw data in the first roast above. The first graph shows a comparison of the raw RoR data (light green), raw RoR averaged over a 4 second sample period (red trace), and over a 10 second period (blue trace). This is on my analog HottoP with its rather crude built-in heater control, so I suspect the major oscillations in RoR correspond in large measure to the heater switching on and off; once I can get PWM control of the heater installed this oscillation should be reduced. If however the samples are averaged over a one second period any oscillations should correspond only to actual changes in RoR.



The next graph shows a comparison between filtered RoR using Jim's low pass filter (red trace), and averaged RoR as above (orange and blue traces).



The next two graphs give the results of another roast in which all filters in aBourbon are set to 0, but the sample size in pBourbon is set to 10. Both graphs are drawn from the data in excel, and show first the pBourbon averaging an a sample size of 10 seconds (green trace), then the samples averaged within excel as above (red and blue traces) - note that these traces are necessarily based on the output of the pBourbon averaging, averaged a second time in excel. It looks like the pBourbon averaging is not carried out for some reason, or if it is, it is not as effective as it could be.



Finally the filtered RoR, 4 second average and 10 second average comparison for the same data, exactly as above (red, orange, blue respectively).



This posting is already a marathon, so lets go the full 5 miles with one last point which I forgot to mention earlier: we can also make good use of the statistical variability of the BT temperatures between samples, because this should give an indication of temperature distribution of the beans. If we were to calculate the standard deviation of the individual samples returned by the ADC within each one second sampling period (or longer - one second might well be too short) we can display that live on the graph in Processing, and that will show up any problems with irregular bean temperature as they arise. For example if the green beans have too much moisture and have not yet fully dried out, or if the beans were picked with too uneven ripeness, or if we are heating the beans up much too fast, or if we are using a bean like Old Brown, all these things will show up in the standard deviation, and we can then modify the roast profile accordingly.

Bhante
(sorry about the long post!)