Pages

Sunday 17 November 2013

RTC without RTC ? How good is my crystal ?

Clock - I think that it's the most popular idea for a project with the MCU. It's not that hard from both software and hardware side to realize but not that trivial as well, not mentioning the fact that once realized it sits somewhere in the house, constantly reminding about itself and brings a kind of satisfaction :). There are a lot of options really to implement, a buzzer alarm, perhaps a thermometer, display and buttons servicing, perhaps maybe even serial port usage for NTP synchronization. But ...

The most challenging thing - like in every clock is the time reference source. There are a couple of options:

  • well calibrated frequency source acting as an interrupt source
  • Real Time Clock module with external crystal ( < 20 ppm )
  • internal clock


These days RTC chips are so cheap and widely available that they are really the only reasonable choice, but if we're only planning to play around a bit with the software without paying much attention to accuracy and just for the sake of curiosity let's check how good the crystal to generate the system clock on the Arduino board is.

System Clock


System clock is generated using an external 16 MHz crystal. This crystal is not ideal, although it's quite stable - it may be surprising that it doesn't resonate exactly with the referenced frequency. In fact this frequency is a little bit different. 

Stability


A crystal accuracy is measured with ppms (parts per million). A typical crystal used to generate a system clock is something around +/- 50 - 100 ppm. Let's assume 80 ppm. The day has:

day = 24 * 60 * 60 = 86400 s

the hour

hour = 60 * 60 = 3600 s

If we would want to use this crystal as a reference for the clock application our clock would loose/gain:

per day:

86400 * 80 ppm = 86400 * 0,0008 = 6,91 s

per hour:

3600 * 80 ppm = 3600 * 0,0008 = 0,288 s

On the worst case it would be around 9 s per day. That's not that bad for a toy project.

Resonance frequency


Although the stability is quite satisfying, information about the exact resonance frequency is needed. How different is the resonance frequency of a typical 16 MHz crystal from the referenced 16 MHz ?

Using some code I can roughly measure it without using any laboratory equipment like frequency meters or oscilloscope. In fact I don't have access to this hardware and for most of the simple projects which I present here, although it would be helpful it's not a must.

The simplest way to estimate the crystal frequency is to use the code already created in libpca. Let's generate a tone of let's say 1 kHz frequency with a libpca beeper API. Now, Instead of connecting the buzzer I'll connect the pin to my laptop's microphone input directly and try to record it. The code is trivial:

Recorded signal:

1 kHz square signal recorded from OC0A pin.

This rough measurement shows that the full period took 45 samples at 44,1 kHz sampling rate, this gives the period of ~1,02 ms. The crystal clock period is 16000 shorter. After dividing the measured period of 1 kHz beep by 16000 and converting it to frequency, the answer is:

f =~ 15,68 MHz.

This means that our clock application will loose seconds. Every second measured will in fact be a 1,02 seconds in reality. This gives 1.2 seconds behind after every minute ! Of course the measurements done cannot be taken very seriously since they are done very inaccurately and without a proper tools, but keeping them in mind, let's write a simple application and see how much our "clock application" will be wrong from real time.



The above code is self explaining more or less, the "epoch" variable is incremented in the interrupt service routine every second (since interrupt happens every 1/256 s and epoch is incremented only when the cnt variable wraps). The time is send via Serial Port as a string.

After flashing it and running. I did some comparison between the measured time and the "real" time with a stopwatch. After an hour, Arduino has been a second behind. The crystal instability (80 ppm) results in 0,3 seconds behind. Assuming 0,7 second lost per hour I can roughly estimate that my crystal is less than 16 MHz by something around 3 kHz. So, it's more like 15,9968 MHz.

Conclusion


When building a real clock application RTC chip is a must, the system clock won't provide enough accuracy to be considered as a frequency reference good enough. Not mentioning the obvious advantage of having the RTC - the time is counted even if the main CPU is not powered, since most of them have a backup lithium battery circuitry.

The methods used to estimate the Crystal's resonance frequency cannot be treated serously with any level of confidence and used for a real production purpose, since they present no practical level of accuracy and the results are only a general hint.

Friday 8 November 2013

Arduino PWM Audio DAC

This post is an addendum to the previous one. To make our lives (as a software developers) even more easier it is possible to recreate a real audio without even having to built the R-2R ladder. Atmega328 comes with a PWM capable timers and they're good enough to produce audio quality analog signal.

PWM fundamentals


I'm not going to go into details about PWM digital to analog conversion, on this blog I focus on my code and do not intend to cover the theoretical background very thoroughly. Please take some time and familiarize yourself with the PWM theory explained on those sites:

  1. Open Music - PWM DAC
  2. Open Music - Dual PWM
  3. Open Music - Distortion Analysis
  4. Open Music - PWM Tutorial

In general every PWM DAC can be described by a PWM frequency - fpwm (the frequency of the squarewave generated - which is constant) and the PWM bit resolution - bitres (which describes the granularity with which we can control the duty cycle).

All of three Arduino's Timers are capable of generating PWM signal. I'll focus of course on Timer1 since it's the only 16 bit timer. As mentioned above we have two properties to configure - the PWM frequency - fpwm and the bit resolution. The signal theory says that the fpwm should be at least two times higher than the highest frequency in our signal. How to calculate it and what is the highest possible we can achieve ? To answer that some details from the datasheet are needed and in fact an understanding of how Atmega's Timer really works.

Timer in PWM mode


Using either Fast (FPWM) or Phase & Frequency Correct PWM (PFPWM) the timer counts up from 0 to value in either OCR1A or ICR1 register (I'll use ICR1 to define the frequency, since obviously the value in OCR1A will control the comparator and in result the duty cycle). This means that ICR1 defines the TOP value for the timer (and thus implicitly defines how often the timer will overflow). The PWM frequency will be defined as:

Fast PWM:

fpwm = fclk / [ N * (1 + ICR1) ]

Phase and Frequency Correct:

fpwm = fclk / [ 2 * N * (1 + ICR1) ]

where N is prescaler.

I want the fpwm to be as high as possible so, obviously the prescaler will be configured to 1. It may not be visible at first glance but the fpwm has a major implications on the bit resolution as well. The value in OCR1A will control the duty cycle - it can't be any bigger than the value defined in ICR1 - which defines the frequency. if it will be higher than ICR1 than the timer simply will never reach it and the result will be a flat line - 100% duty cycle. It's more clear if you look at the picture

Principles of PWM with Atmega's Timers.

It's clear now that the demand for highest possible fpwm as well as highest possible bitrate is self contradicting. A golden merit must be established. In general it's worth to sacrifice some of the bit resolution just to increase the fpwm. I'll talk about real values in just a moment.

... Need more bandwidth


Indeed, just as previously the main bottleneck is the Serial Port. I did some tests pushing it to it's limits and it seems that the highest possible baud rate I can get is 230400. I don't want to change the firmware in Arduino's Atmega16U2 chip responsible for USB <-> serial communication, so I guess I'll have to live with this constrain and try to squeeze as much of the serial port as possible anyway.

230400 is not that bad. It's 23kB per second. What I can play with it ? There are a three options really (not mentioning the compressed formats - but that's a completely different story):

  • 16 kHz, 8 bit = 16 kB/s
  • 8 kHz, 16 bit = 16 kB/s
  • 22 kHz, 8 bit = 22 kB/s 18 kHz, 8bit = 18 kB/s
22 kHz sounds pretty good. This means that the highest frequency that can be recreated is around 11 kHz. As it will be shown later the bit resolution doesn't have that much of a spectacular influence on the quality, The bit resolution defines the so called "noise floor" which in simple words means that the lesser the bit resolution the noisier the signal will be and it will be impossible to recreate some very quiet - low amplitude sounds since they will drown in the digital noise itself. In this case, having the sampling rate as highest as possible is crucial.

22 kB/s doesn't leave much time margin though. It's almost feeding the DAC directly from the USART without much of a buffering - this may be a source of glitches and noise as well, not mentioning the glitches that will happen if the transmission is unsuccessful. The MCU will be pretty busy. In fact the USART is too slow to provide the data on time with this sampling rate. The highest I could get is 18 kHz unfortunately. 18 kHz is an absolute limit for the USART and even at that speed occasionally some glitches may happed

Data delivered to slow. A short moment of silence.

Anyway it's pretty good though, 9 kHz of audible bandwidth should sound good.

PWM frequency and bit resolution


Going back to the PWM frequency. I'll use the Phase & Frequency correct PWM mode. Our requirement is to play an audio with highest frequency up to 9 kHz. The fpwm must be higher than 18 kHz. We'll base our calculations on the bit resolution though, since we're not going to play anything above 8 bits (per PWM channel). Assuming that OCR1A must have an 8 bit resolution so ICR1 must be configured to 256. Taking those values into consideration:

fpwm = 16 MHz / [ 2 * 1 * (1 + 255) ] = 31,25 kHz

which is 3,4 higher than the highest frequency of the signal - that'll do.

16 bit playback


One of the options that I have is to play 16bits 8 kHz audio. Now, how to do this when just a paragraph before I configured the PWM resolution to 8 bits ? It's actually pretty easy. Atmega has two comparators per timer OC1A and OC1B. Each of them has an independent register OCR1A and OCR1B which value is constantly compared with TCNT1 value. The PWM duty cycle of the waveform on OC1A and OC1B pins is controlled by configuring OCR1A and OCR1B to appropriate values. In fact we can sum the signal from those two 8 bit outputs using resistors to form one combined output 16 bit output. The trick is that the resistor for the least signifficant byte has 256 smaller value than the one for the most signifficant byte - by natural way the levels of those two PWM outputs are shifted.

Playback


The playback loop is slightly different than the one for R-2R ladder. Now instead of pushing the data to the ports, it is "pushed" to the OCR1A timer registers:


you noticed the MODE macro. The software; during compilation, can be configured into any of the mentioned mode

  • #define MODE_8K_8B 0
  • #define MODE_8K_16B 1
  • #define MODE_16K_8B 2
  • #define MODE_18K_8B 3

By default MODE is configured to

#define MODE MODE_16K_8B

Dependently on the mode selected (16bit or 8 bit) the sample value is copied only to OCR1AL or to both OCR1AL and OCR1BL

The Circuit


PWM requires a couple of components forming a low pass filter in order to get rid of the PWM carrier frequency. The PWM frequency is 31,25 kHz, but we don't need anything really above 9 kHz, so the filter values should be:

f = 1/ [ 2 * 3,14 * R*C] = 1/[2 * 3.14 * 3k9 * 4n7] = 8,69 kHz 

The filter itself:

PWM filters for both PWM channels


PWM filters on the breadboard.

Software


As mentioned the software is more or less identical as for the R-2R ladder. It has only been enhanced to support conversion into couple of different formats instead of one (8kHz, 8 bit) and the baudrate has been increased to 230400 bps. As previously everything is done in binary manner to squeeze out everything that the USART can provide. The PWM configuration is being done by the libpca functions, there's no need to go into details about that, all the settings are standard, nothing extra out of ordinary. The timer is by defauilt configured into Phase & Frequency correct PWM mode, but it's possible to play with this setting in the code to compare the quality.

#if MODE == MODE_8K_16B
/* tpwm_fpwm_init(E_TIMER1, E_PWM_DOUBLE); */
tpwm_pwm_init(E_TIMER1, E_PWM_DOUBLE);
#else
/* tpwm_fpwm_init(E_TIMER1, E_PWM_SINGLE); */
tpwm_pwm_init(E_TIMER1, E_PWM_SINGLE);
#endif

How to use it


On the PC side, as previously one should use the player.sh script. The syntax has been changed slightly. The first argument is the mode, the rest are the audio files

player.sh <16b | 16k | 8k | 18k> <audiofile(s)>

The options are self explaining I think, In order to use the 16 kHz mode we should invoke the script like this:

./player.sh 16k myfile.mp3

One must remember to adjust the Arduino's serial port name in the wpwm.pl script accordingly if needed.

On the Arduino side, we must select the mode which we want to try by adjusting the MODE macro in main.c file (as mentioned previously). After compiling and flashing the firmware, one should be able to successfully use the player script in order to hear some music. 

I connect my Arduino directly to the MIC-IN of my laptop in order to record the sound - I don't use any amplifiers whatsoever.

The software as usual is available either as a snapshot from here (containing libpca snapshot and the project itself) or it can be clones from my github repositories:

git clone git@github.com:dagon666/avr_Libpca pca
git clone git@github.com:dagon666/avr_ArduinoProjects projects

cd projects/dac

Building & flashing:
make
make install

Let's hear some music finally


Just as expected the difference between 8 and 16 bits is almost unrecognizable. Only at the begining when the dynamic is quite low, the noise level for 16 bits is signifficantly lower. The most spectacular change comes along with the sampling rate - the higher the better.





Throughout the clip some glitches are noticable from time to time - they're the result of corrupted frames (CRC does not match) or simply the fact that the data was not delivered on time (18 kHz mode). In 18 kHz the USART is really operating on it's limit (as well as the CPU - which must service the RX interrupt), I mostly wanted to demonstrate that it's possible to go that high, but the audio is far more stable and less glitchy in 16 kHz mode.

Saturday 2 November 2013

Arduino R-2R ladder Audio DAC

There is a lot of projects out there which use R-2R ladder and an Arduino to recreate sounds from either SD card or short audio clips programmed directly to MCU's flash memory. Although using SD card is fairly reasonable and provides a lot of flexibility it's not very challenging (from the software point of view) and it requires some additional hardware. Believe it or not we already have all the needed hardware in the Arduino itself.

Assumptions


In this project I'll play mp3 or any other multimedia files from the PC using Arduino. Bellow are the details:
  • Play PCM 8kHz 8bit Audio with Arduino
  • Audio samples will be transfered via USART from the PC in binary format using SLIP
  • Audio files will be decoded on the PC side and only the RAW data will be send to Arduino

Timing


Arduino is a powerful machine, powerful enough that it's possible to play audio with even higher sampling rates and bit-resolutions than assumed above, the bottleneck in this case is the Serial Port. Assuming that I'll use the highest standard USART speed available which is 115200 bps (8 data bits one start bit and stop bit = 10 bits to send a byte) it's possible to send up to 11520 bytes per second. In order to play a second of 8 kHz 8 bit Audio I need to have at least 8 kB of data. So the USART is around 30% faster as a data producer than the audio samples consumption rate, which means that 8kHz standard sampling rate is the highest I can get. 

The timing is crucial thing in this project. The data must be buffered but what kind of buffer do I need to have ? At first I thought that the bigger the better, but that's not exactly true. Accordingly to assumptions in implementation of the SLIP protocol in libpca, one SLIP frame can carry up to 256 bytes of data, we must not forget that this transfer is not instantaneous it takes time and we're in the middle of the playback not really knowing (at the PC side) how much audio samples have already been consumed (the consequence is that it's hard to tell how much space is available in the buffer). If the buffer at Arduino side doesn't have enough data to hold the new upcoming chunk - the data will be lost and we'll hear an audio glitch - and that situation for sure will happen since we're sending faster than consuming. 

First approach that came to my mind is to simply wait after each data block in order to be sure that there will always be enough space in the buffer (since most of the data will be consumed). Let's think about it for a moment and evaluate some rough timing calculations.

  • playing 256 samples takes 1/8000 * 256 = 32 ms
  • sending 256 samples takes 256 + 2 (SLIP END characters) / 11520 = 22 ms 

The second calculation does not take into consideration any additional SLIP ESCAPE characters that may be included. But more or less I have a 10 ms I can wait after sending the chunk before sending another one, right ? WRONG. Those calculations do not consider that during those 22 ms when we were sending the data already something around 176 samples have been played (68% of the data). OK, so if I'll wait 32%*10ms = 3.2 ms then it should be fine, right ? WRONG again. By waiting, everything that is done is only hopelessly trying to make the data transfer speed equal to the audio consumption rate. It's impossible to synchronize ideally those two independent processes since there are so many factors that can have it's influence on the timing that all the efforts by definition have simply no point. If the sending rate will be too low, the audio will be chopped since there will be gaps in the playback, if the data will come too fast, there will be no space available and will have drop the data from time to time.

It's even worse. Let's look on the slip_recv function prototype:

uint8_t slip_recv(uint8_t *a_buff, uint8_t a_buflen);

It takes a pointer to a buffer for the incoming data, since this is a transmission buffer it can't be used to realize another transmission until the data is completely consumed. That means that after the reception we must COPY the data from the frame to the proper audio buffer. Which makes the timing COMPLETELY unpredictable with the required precision.

How to cope with this situation then ? First of all we cannot afford to copy the data from one buffer to another - it's simply a waste of time and by clever memory organization this problem can be easily eliminated. The playback must be done from the transmission buffer directly. But how to perform the transmission and audio playback using the same buffer in the same time ? It's actually pretty easy. Have a look:

I have four transmission buffers (let's call them buffer banks) each of them holding 64 samples. The audio will be played directly from the samples table. The beauty of the picked size is that

64 * 4 = 256

which means that the audio data can be addressed using a single byte, the following way:

sample = p[(g_tail >> 6) & 0x03].samples[g_tail & 0x3f];
g_tail++;

Since g_tail is an 8 bit variable 2 upper bits select the "bank" (0-3), and the rest, addresses the audio data (0 - 63). I address both the buffer bank and the audio data with a single variable. It more or less looks like using a single continuous buffer.

When receiving data I track which buffer bank is free (1-4) and I write to it. The playback happens from the previous buffers. If there is no free buffer available I send a "WAIT" command to the PC so it can wait a little while (a time shorter than the time needed to consume 3 buffers = 192 audio samples, basically the time must be longer than 8 ms (consume 64 bytes = 1 bank) and shorter than 24 ms). I chose 1,9 ms which lasts for around 15 audio samples. It's too short isn't it ? No it's not. Let's assume 2 ms of explicit waiting (due to the function inaccuracies) + time needed to receive the "WAIT" string through the Serial Line: 350 us which already makes it let's say 2.5 ms not mentioning any other processing times and of course the time needed to transfer 64 bytes block >= 8 ms, which in total gives at least 10,5 ms. Of course the time is much more longer (SLIP special characters have not been taken into consideration and as mentioned indeterminable processing time has been omitted as well).

The transmission/consumption process is depicted bellow:

Transmission of binary frames through the Serial Port and new data placement in the Arduino's buffers.


Playback


Let's talk about how to play the samples ? I use timer in CTC mode and play the samples by placing them on the port directly, but again there's a little catch here as well. The only "full" 8 bit port available on Arduino is PORTD, unfortunately we can't use it's two lower PINS 0,1 since they are shared with USART and I'm using USART as a data source. Because of that The bottom part of the byte (bits 0 - 5) is placed on PINS 2-7 of PORTD and the remaing two most significant bits (6 - 7) are placed on the adjacent PORTB. This may have an influence on the audio quality since placing the data on two ports is not an atomic operation - first we place one piece of data - which in effect generates some sort of voltage on the R-2R output, then we place the remaining piece of data on the other port - this will probably generate a high frequency glitch for every sample. Have a look on the schematics.


R-2R Ladder


A word about the Ladder itself. I chose a value of R = 5k. I had more in mind the 2R value = 10k which is the standard one. I had 4k7 resistors laying near hand so I decided to use them. It wasn't a good idea. The R-2R relation must be as good as possible. It's best to use the same resistors for both R and 2R and connect two of them in parallel to form a R value, so in my case R = 5k (two 10k connected in parallel) and 2R = 10k. I took some measurements using Arduinos ADC (the R-2R ladder output connected to the ADC) and below are the results. It's pretty visible that using 4,7k and 10k for R-2R ladder is a bad idea.

R-2R = 4k7-10k Ladder signal response (sawtooth).

R-2R = 4k7-10k Ladder signal response (sine wave).

Major differences when using two 10k in parallel as 5k.

"Steps" visible anyway - the curve magnified.

R-2R Ladder with 4k7 and 10k resistors. Work in progress.

R-2R Ladder with 10k restistors connected in parallel.

R-2R Ladder Schematics.

The source code.

Arduino Side


Arduino program is pretty simple. I already mentioned that samples are played in the timer interrupt. Besides that a standard data collection algorithm happens in the while loop in a very similar fashion to the one from the Arduino MIDI player.

The function collecting the samples, takes a pointer to the destination buffer. The buffer is selected from four available by using 2 most significant bits of the g_head counter. g_head & g_tail indexes realize a queue on the buffers. The g_tail is incremented by the timer interrupt whenever new sample is played. The g_head is incremented along with new data received. As long as g_tail != g_head I know that there is still data available in the buffers.

serial_collect_samples((void *)&p[(g_head >> 6) & 0x03]);

A word about the sample collect function



First thing is to check if number of available samples is higher than 192 (3 banks) if yes, then the "WAIT" command is send, to tell the PC side to refrain from sending new data for a while. The slip_recv call is blocking. It will block until new data has been collected, after CRC verification and making sure that the data is genuine the g_head index is incremented by the number of samples received in the frame.

PC Side


Perl script is responsible for feeding Arduino with data. This script accepts a 8 kHz 8 bit WAVE file as an input. It's pretty straight forward and similar to previous script for the Arduino MIDI player. First it initialize the Serial port, then it tries to open the WAVE file and read it's header. After that is successful I read 44 bytes of header and unpack them.

die "Unable to read WAV header\n" 
    unless($offset = read $g_fh, $header, 44);

my @header = unpack "a4la4a4ls2l2s2a4l", $header;

It get's more sense if you look on the WAVE file header:



So, the unpack call extracts RIFF file id, the WAVE file format and the rest of the header fields. The purpose is to detect if it's a WAVE file and if it has the only compatible sampling rate and bit rate. Once that is confirmed, the script goes to the "transfer loop". If there is no "WAIT" command received from Arduino it simply reads the 64 byte data chunk from the file and feeds the Arduino. If "WAIT" has been received, the script will wait for around 2 ms (1900 us). The loop continues until the whole file is processed. The frame contains two bytes of CRC, one byte indicating how many samples it conveys and the samples themselves.

Although it's completely fine to use this script directly I use it in a wrapper (I could've made everything in the single script, but I was too lazy :)). The "player.sh" shell script accepts any audio format as an input it uses sox to convert the provided audio file to an intermediate WAVE file which then will be provided to Perl in order to be played by Arduino. It's a lot more useful than converting the files manually every time.

How to use it ?


The whole project snapshot can be obtained here. One can fetch the newest version of libpca and the project itself from my public GitHub repository as well. The snapshot and the GitHub version have a slightly different Makefile, paths to the libpca are a little bit different, everything else is exactly the same.

GitHub


git clone git@github.com:dagon666/avr_Libpca pca
git clone git@github.com:dagon666/avr_ArduinoProjects projects

The project resides under dac directory. After navigating to it. One should change the branch to r2r_dac:

git checkout -b r2r_dac origin/r2r_dac

Snapshot


Download and unpack the snapshot from here

... the rest is common:

Build and flash the Arduino firmware:

cd projects/dac
make
make install

Launch the player.sh script with an audio file as a parameter:

$ ./player.sh audio.mp3

The Presentation


Finally we can play some audio. The video presents an Arduino playing mp3 of my metal project Tangible Void - you can check it out on youtube - here.