Pages

Saturday 26 October 2013

Arduino MIDI Music Box

It's time to do something useful with our small 8-bit computer. Music box is a fun and easy project. It's not demanding from the hardware point of view but it can be a little bit tricky from the software point of view itself - ideal project to learn a lot and not to spend the whole day soldering. What we will need: 

  • an Arduino (of course) 
  • Piezo buzzer

 ... and that's all. Now a couple of words of what we want to achieve. 

Requirements 


Let's point them out:

  • play monophonic melodies by generating tones (1 channel MIDI instrument)
  • the notes will be send by the PC through the serial port 
  • our musicbox should be visible to the PC as a MIDI port 
  • be able to use MIDI player of choice on PC to play MIDI files on our musicbox 


The requirements are specified. Unlike most of the simple projects around which play the same melody everytime, which most of the times is hardcoded in the program memory and in order to change it one must modify the tone frequencies and the timing between them and then re-flash the Arduino, our musicbox will simply act as a simple MIDI device and become a "MIDI sink" for incoming serial data. That means that more or less we can play any MIDI file with it that comes in our hands. The only limitation is the number of channels - the firmware which I'm going to discuss will play only one - the user will have to select what channel data should be send & played, but in general it's not impossible to enhance the implementation to support more, for demonstration purposes one is enough though. 

Technologies 


I need to discuss what kind of tools and technologies will be used. No matter how trivial a musicbox can look in the first place there will be quite a lot going on under the hood. Here are the things that will be used (in terms of software) on the Arduino side: 

  • beeper module from libpca - the timer will be used for tone generation 
  • SLIP module from libpca - the data will be send in binary format in SLIP frames from PC to the Arduino 

Everything has been already covered in my previous posts on this blog. If you're following the blog any of the above require no explanation. At the time being creating a more sophisticated project like this one is only a matter of putting the building blocks together. On the PC side I will use Perl to do the dirty work. No matter what you have heard or have been told, Perl is a great language far more superior than Python and I hope that again I'll prove it to every of you. Things that will be needed: 

  • Device::SerialPort - serial port communication 
  • MIDI::ALSA - for doing all the MIDI dirty work 
  • Digest::CRC - to calculate CRC checksum of the outgoing data PC side


PC side


Let's start with the PC side first. The Perl script will register new MIDI port and wait for the data written to it which then will be send through the Serial port to Arduino - pretty simple. The data I'm interested in from the MIDI stream incoming are so called "note_on/note_off" events. Unfortunately their not exactly in a format convenient and appropriate for this project. The note_on event defines the beginning of a particular not (a trigger to start generating tone of given key), we should stop generation when receiving note_off. This is not acceptable since it blocks possibility to generate any other tone in between and we would have to track when to really stop the generation on each incoming note_off event. 

Fortunately enough again Perl and CPAN saves the day. The MIDI::ALSA module defines a notion of "score" - a set of midi tracks in which every event has it's own length defined, so instead of two events note_on/note_off we have only one - note, which provides the following information:

{'note', start_time, duration, channel, note, velocity}

Exactly what we need. The bad news is that it doesn't work very well. In fact it seems that duration has some constantly increasing value along with the position in the MIDI file and in result isn't very useful at all, but at least instead of processing two events we have only one. From the data above I'm interested only in the event type -> 'note' the 'note' itself of course and the channel - I'll filter the data from one or couple of channels only to be send and played by Arduino since it would produce a complete chaos if we would want to try to play everything using only one instrument at once.

Since we will be sending the frequency to the Arduino, we need to convert the MIDI note number (0-127) into frequency domain. After doing a quick Google search for MIDI note frequencies I come up declaring the following array:

# midi notes
my @notes = (
8.1757989156,
8.6619572180,
9.1770239974,
10.3008611535,
10.3008611535,
10.9133822323,
11.5623257097,
12.2498573744,
12.9782717994,
13.7500000000,
14.5676175474,
15.4338531643,
16.3515978313,
17.3239144361,
18.3540479948,
19.4454364826,
20.6017223071,
21.8267644646,
23.1246514195,
24.4997147489,
25.9565435987,
27.5000000000,
29.1352350949,
30.8677063285,
32.7031956626,
34.6478288721,
36.7080959897,
38.8908729653,
41.2034446141,
43.6535289291,
46.2493028390,
48.9994294977,
51.9130871975,
55.0000000000,
58.2704701898,
61.7354126570,
65.4063913251,
69.2956577442,
73.4161919794,
77.7817459305,
82.4068892282,
87.3070578583,
92.4986056779,
97.9988589954,
103.8261743950,
110.0000000000,
116.5409403795,
123.4708253140,
130.8127826503,
138.5913154884,
146.8323839587,
155.5634918610,
164.8137784564,
174.6141157165,
184.9972113558,
195.9977179909,
207.6523487900,
220.0000000000,
233.0818807590,
246.9416506281,
261.6255653006,
277.1826309769,
293.6647679174,
311.1269837221,
329.6275569129,
349.2282314330,
369.9944227116,
391.9954359817,
415.3046975799,
440.0000000000,
466.1637615181,
493.8833012561,
523.2511306012,
554.3652619537,
587.3295358348,
622.2539674442,
659.2551138257,
698.4564628660,
739.9888454233,
783.9908719635,
830.6093951599,
880.0000000000,
932.3275230362,
987.7666025122,
1_046.5022612024,
1_108.7305239075,
1_174.6590716696,
1_244.5079348883,
1_318.5102276515,
1_396.9129257320,
1_479.9776908465,
1_567.9817439270,
1_661.2187903198,
1_760.0000000000,
1_864.6550460724,
1_975.5332050245,
2_093.0045224048,
2_217.4610478150,
2_349.3181433393,
2_489.0158697766,
2_637.0204553030,
2_793.8258514640,
2_959.9553816931,
3_135.9634878540,
3_322.4375806396,
3_520.0000000000,
3_729.3100921447,
3_951.0664100490,
4_186.0090448096,
4_434.9220956300,
4_698.6362866785,
4_978.0317395533,
5_274.0409106059,
5_587.6517029281,
5_919.9107633862,
5_919.9107633862,
6_644.8751612791,
7_040.0000000000,
7_458.6201842894,
7_902.1328200980,
8_372.0180896192,
8_869.8441912599,
9_397.2725733570,
9_956.0634791066,
10_548.0818212118,
11_175.3034058561,
11_839.8215267723,
12_543.8539514160,
); # notes

Great, one problem less. 

The MIDI sequencer


A MIDI sequencer must be registered on the PC side to provide an output for MIDI playback software. It's extremely easy with MIDI::ALSA module and Perl:

MIDI::ALSA::client('Arduino MIDI Beeper', 1, 1, 0);

and that's all. I register a new sequencer with one input port and one output port. We can already see it in the system:

$ aconnect -oil
client 0: 'System' [type=kernel]    0 'Timer           '    1 'Announce        'client 14: 'Midi Through' [type=kernel]    0 'Midi Through Port-0'client 128: 'Arduino MIDI Beeper' [type=user]    0 'Input port      '    1 'Output port     '

Ok, we are almost done, all that's left is to take the MIDI data and send it to Arduino. Let's analyze the main program loop, again there will be some Perl trickery but I'll try to de-crypt everything which is not obvious to everyone not very familiar with Perl.

The call to MIDI::ALSA::input() is blocking, it means that we will wait on the first line for the MIDI event before we'll send anything to the Arduino - that's a good thing, that way we do not have to worry about the timing, the notes will be played as soon as they arrive. Next I detect if the incoming event does not define the MIDI file end - I exit the loop if that's the case. 

Next thing is to convert the MIDI event to a "score event". Next two lines simply skip to another loop roll if the event does not contain any useful information (it's either empty or it's not a 'note' event). Just for the debugging purposes I dump the event to the console. Going further I filter the events (by skipping them) if in the command line channels to be player are specified, if not then data from every channel will be sent. Later on I simply convert the MIDI note to frequency and because of the problems with getting the reliable event 'duration' information from MIDI::ALSA I simply define a fixed duration for every note in milliseconds (50 in this example). Next the data is being placed in the SLIP frame and sent to Arduino to be played. Arduino should return the data back (again just for debugging purposes) - and that's all.

Arduino Side


Basically all the hard work has been already done and is hidden in libpca. Please refer to my previous posts for more details, since every aspect and function used for this mini project has been already thoroughly discussed. 

Arduino firmware will be very simple, all that needs to be done is to simply receive the SLIP frame, verify it and play the note inside. Let's have a look on the message format first.

Each note is defined as a pair of numbers - frequency and the duration. Each packet coming from the host contains the CRC16 checksum, a byte denoting how many notes have been sent and the notes themselves. I decided that in a single packet it will be possible to send up to four notes - that would require some timing information for each note itself so even though it's possible to sent up to 4 notes in one shot in the final version of this project I'm always sending only one.

The listing bellow present a complete source for the Arduino firmware.


In the while loop it wait's for a new SLIP frame, verify the data integrity and play the notes from the packet. After flashing the Arduino we can finally do some tests.

How to use it ?


The whole project snapshot can be obtained here. One can fetch the newest version of libpca and the project itself from my public GitHub repository as well. The snapshot and the GitHub version have a slightly different Makefile, paths to the libpca are a little bit different, everything else is exactly the same. 

GitHub


git clone git@github.com:dagon666/avr_Libpca pca
git clone git@github.com:dagon666/avr_ArduinoProjects projects

Snapshot


Download and unpack the snapshot from here


... the rest is common:

Build and flash the Arduino firmware:

cd projects/musicbox
make
make install

Launch the mid_converter.pl script (optionally provide channel numbers in the command line which will be played), get the sequencer details:

$ aconnect -oil

play your midi file using your player of choice (I'll use aplaymidi):

aplaymidi -p 128:0 <midi_file>

Musicbox in action 


Now that everything has been said it's time to see the thing in action. I'm using Timer 0 and OC0A compare output (which is located on PORTD6 - Pin6 on Arduino) to which the buzzer is connected.





Sunday 13 October 2013

Transmitter/Receiver synchronization during binary transfers with SLIP protocol

A word of warning, this post will be kind of long. I will try to cover the binary communication over serial port basics in 2 languages (C & perl), but I hope every one will bare with me until the end.

I'm almost ready to go into some of my "real" projects. In order to do that all that needs to be done is to talk a little about SLIP protocol. SLIP protocol is used to synchronize data between receiver and transmitter on a serial line (not only RS232, ethernet as well and many other mediums). SLIP is one of the most commonly used option, the other one is Modbus RTU, which describes some timing constraints and requirements during the data transfer. I haven't tried Modbus RTU yet, since SLIP is doing pretty good job for me. 

First of all why do we need to synchronize the data at all and what does it mean in details in the first place ? The reason for that is simple (maybe not so obvious in the first place). Consider the following situations:

We want to send 8 bytes of binary data from A to B over Serial line:

  • B is not yet ready to listen for data
  • A already send the first byte
  • B become ready to receive data
  • A is sending more data
  • B received only 7 bytes out of 8. 1 byte has been LOST

What comes to one's mind straight away when analyzing this example, let's force A to wait for some time so B for sure will become ready. Let's think about it:

A sends 8 bytes of data in two packs to B:
  • A waits a second (waiting for B to become ready to receive)
  • B becomes ready to listen
  • A sends first pack (4 bytes of data) to B
  • B receives 4 bytes and process it
  • A sends another pack of 4 bytes
  • B has not finished processing of previous package and lost some of the data from the second pack

As time decreases and transmission speeds are getting higher this leads to more and more problems. As long as we can get away with sending text data between peers without any kind of synchronization. It's very difficult to do that and to have a stable reliable binary data transfer without any. 

SLIP is a very simple, almost primitive solution, which solve that kind of problem. I'm not going to duplicate the description here, please refer to the rfc1055 for more details regarding it, briefly SLIP defines two special characters: an END byte and ESC byte. Receiver knows about the end of the data stream by listening for an END byte, when the data stream contains the END or ESC byte in it, then transmitter replaces it with the ESC byte and the ESC_END or ESC_ESC byte - those are replaced to the valid data by the receiving side. That's all - as I mentioned it's very simple. SLIP defines the exact values for END, ESC, ESC_ESC & ESC_END but those can have any values - it doesn't really matter:

/// 192 -> 0xc0
#define SLIP_END 0300

/// 219 -> 0xdb
#define SLIP_ESC 0333

/// 220 -> 0xdc
#define SLIP_ESC_END 0334

/// 221 -> 0xdd
#define SLIP_ESC_ESC 0335

For details of implementation (just as the protocol itself it is very simple), have a look into libpca (src/slip.c). One more thing must be mentioned, SLIP implementation from libpca can be used over many different links. One must only define the IO routines which will be used to retrieve data. Let's have a look on the fragment of slip.h header from the library:



By default libpca use the serial port IO routines, but this can be overridden if the user will provide it's own definition for symbols SLIP_CHAR_RECV/SLIP_CHAR_SEND.

Let's consider how we can use it, for that I'll define a simple software timer and a data model for our binary informations which will be exchanged on the serial line in order to modify the timer parameters. For the sake of simplicity we'll use LED on one of the port which will be toggled on every timer overflow. The "software timer" is defined as follows:

It's pretty simple we have the cnt variable itself and the maximum (max) value to which the timer can count, before being reseted to zero. Everything will get more obvious in the code using it. Let's define the data-model for the messages:

I use "packed" attribute here. It is needed (especially for the host machine) to prevent the compiler from aligning the structure to the machine architecture size. It doesn't have any influence on Arduino since it's an 8-bit machine, but since the header is shared between Arduino and the host machine when trying to compile the code on host machine without "packed" attribute applied to the structures probably size of struct timer_data would be 8 instead of 5 (alignment to the nearest 4-byte boundary, since most x86 are 32-bit machines). Of course the crc16 field is not mandatory in our communication, but just for demonstration purposes it's nice to know that if the data interchanged is really crucial it may be verified using CRC (I use crc16 but it's possible to use any other CRC32, CRC8, by default I decided to implement crc16 only in libpca). The host application will send new timer_data structure with new setting to reprogram the timer and in result the blinking speed will change. One thing must be mentioned here, Arduino will reset itself after establishing the serial connection, it's because of the board design itself - one should not worry about that. This behavior can be disabled by disabling the jumper on Arduino UNO rev 3. board (there are plenty of solutions available in the web for different boards - I don't want to go into details here).

This connection must be disabled in order to disable auto-reset on serial connect.



Arduino Side


Let's have a look on the Arduino program first, and do a little walk-through:


It looks pretty complicated at first glance but that's only the appearance. First of all the usual stuff, I configure Timer 0 to CTC mode and unmask it's interrupt (one must remember to disable the Timer's 0 interrupt implementation in libpca - #define TDELAY_IMPLEMENT_T0_INT 0, since we're defining our own custom handler). In the while loop, Arduino is waiting for a complete SLIP frame, any junk frames (which size is smaller than the struct timer_data) are discarded, the last step is to verify the data checksum of the frame and to send a SLIP frame containing struct response with NACK status in case the CRC doesn't match. slip_verify_crc16() takes the buffer, it's length and third argument which defines a position in the buffer of the CRC checksum - in our case the CRC is placed at the very beginning, so the position equals zero. If the CRC verification is successful then at line 104 we can be sure that the data is genuine and not corrupted in communication. Next step is to reprogram the timer and finally send the SLIP frame with struct response with ACK status. This frame has a checksum calculated as well, and we will be able to verify it on the other side. The code is more or less straight forward.

The conclusion is that the LED attached to the Arduino will blink with different speed, and the "timing" details are programmed remotely through the serial line and the binary messages exchanged using SLIP protocol.

Host side


We can now focus on discussing the details of the host application. To be honest writing an IO application to communicate through the serial port in C for the host machine is a little bit tedious and quite time consuming, that's why it is far more convenient and simply quicker to create such tool in a scripting language of choice (Perl, Python, Lua) since most of the dirty work has been already done under the hood. For the purpose of comparison and education I'll present a script in Perl which will talk to our Arduino as well as in C.

Let's start with Perl since it's far more simpler and I believe it will be easier to understand.

Host Slip Communication in Perl


The biggest advantage of Perl over any other scripting language is CPAN and the amount of modules available. I'll use some of the already available building blocks in order to make my life a little bit easier. First of all we need to somehow use the Serial Port, the second thing is the CRC checksum calculation. We will need the following modules (they are available in Arch Linux package manager - which is my distro of choice, the package names are listed as well - those can be named similarly in your distribution):

  • Device::SerialPort (perl-device-serialport)
  • Digest::CRC (perl-digest-crc)

We've got the tools, we can do the work finally. Let's have a look on the following code:


For those who are not familiar with Perl it can be hard to get used to it. I'll try to be very verbatim. Starting from the top, the first interesting line is the declaration of hash %g_dev which holds the details of the Arduino's Serial Port. Again, for those unfamiliar with Perl hashes are like associative arrays, next I define the SLIP special characters as constants. Next are two important subroutines which realize the communication. First of all, Perl does not define something like a structure and does not provide a direct access to the memory underlying the variables - just like in C, how can we talk then about binary data interchange in a format described previously by the C structures ? It happens in a little bit different way. Perl provides two functions pack/unpack to pack the variables into a string accordingly to the given template and to unpack the string to variables accordingly to the given template (please read the details in the perl documentation, perldoc -f pack, perldoc -f unpack, perldoc perlpacktut). In order to make it more reasonable let's have a look on the struct timer_data once again. We need to send a byte stream constructed the following way:


  • crc16 - unsigned short 16 bit variable - pack template: S
  • prescaler - unsigned char 8 bit variable - pack template C
  • ocr - unsigned char 8 bit variable - pack template C
  • max_st - unsigned char 8 bit variable - pack template C

That means that if we would write:

my $data = pack ("SCCC", $crc, $prescaler, $ocr, $max_st);

or shorter:

my $data = pack ("SC*", $crc, $prescaler, $ocr, $max_st);


the pack function will assemble our variables into a string which can be then directly send through the serial port. We need something more here, the SLIP_END characters, so the complete call could look like this:

my $data = pack ("CSC*", (SLIP_END, $crc, $prescaler, $ocr, $max_st, SLIP_END));

That's exactly what is happening in the slip_send subroutine. First the data passed to it is scanned for any SLIP special characters in order to escape them. Once that is done, the original data (unescaped) is passed to the crc16 function to calculate the checksum. A word about line 32. First a byte string is constructed containing two zeros in the beginning - which are a placeholder for the CRC, and the rest of the data. The placeholding zeros are replaced with actual CRC once it is calculated. The receiver will calculate the CRC the same way in order to verify the data. It will copy the received CRC from the frame to the temporary variable and zero it's bytes in the frame - now the frame is in the same state as on the transmitter side before the transmitter calculated the CRC. If that operation wouldn't have place of course the checksums would be different.

The slip_recv() subroutine is exactly implemented the same way as in libpca. One thing of explanation here, slip_recv() has been implemented in more or less universal fashion - it takes a references to a getchar routine (line 38) which can be implemented over any kind of link. Next the command line input is parsed into global variables (lines 68-74), the serial port is initialized (lines 76-84) and we wait for 4 seconds. Now why is that ? As mentioned previously, after establishing a serial connection RS232 control lines will be toggled which in result will reset an Arduino, so we need to wait some time in order for Arduino to become ready and operational. By default (as it is visible in the code), the Arduino's Timer0 and Software timer will be initialized so the overflow happens with a frequency of ~ 1 Hz. The LED will blink with that frequency until the SLIP frame will be received with new configuration values for the timer. In lines 100-101 the SLIP frame containing our binary data is send over the serial line next the script waits for the SLIP frame from Arduino containing the response. The struct response is very simple, it contains only the CRC16 (2 bytes) and the status. It is unpacked after reception (line 107):

# unpack the serialized data to variables
my ($crc, $status) = unpack("SC", $data);

# calculate the checksum of the response for verification
my $crc_calcd = crc16(pack("SC", (0,$status)));

... and repacked again with CRC set to zero - the CRC is recalculated again for verification purposes. The last line simply presents the results.

Host Slip Communication in C


As far as our simple communication program was relatively short in Perl. We will have to dedicate much more effort in order to achieve the same in C (I imagine that everyone straight away lost their good mood now :)), but it's possible and sometimes it can be a better solution than any scripting language (small systems without any kind of interpreter available). Let's try to create functionally identical program in C. As previously we need some tools before we start

  • Serial Port IO and init routines
  • CRC16 implementation
The bad news is that we need to create those ourselves. Let's start with the CRC16 algorithm.

CRC16 checksum calculation


The best and  most comprehensive source for any algorithm or data structure implementation is ... linux kernel. Yes, that's right, everything that you will probably ever need is already implemented there and tested, just have a look at kernel's lib/ directory and pick what you need. I'm sure it will be there. The crc16 implementation is there for sure, we need the crc16.c file, slightly modified since we don't need the kernel specific stuff, have a look bellow:


Next, the serial port IO and initialization.

Serial port initialization


Serial port is a tty device, we need to initialize the speed, parity, flow control and other stuff through a termios API. I found this code some time ago and simply copy and paste it to my projects, there's no philosophy behind it, the tty_attrib_conf() initializes the port speed and other IO specifics, the tty_block_conf() configure, whether the calls to read() should block until the requested amount of data is collected. We will use the port by simply opening it (as like a file) and write/read to/from it using standard file IO API.


The last thing is the SLIP implementation itself. Of course I'm not going to duplicate it for the host application. Since it is already implemented in the libpca I'll use that implementation. Some preparations are needed though. libpca SLIP module expects to see common.h header file and config.h header file. We'll create dummy versions of those. config.h will contain:

#define SLIP_IMPLEMENT_CRC16 0

to disable the CRC implementation - it relies on the functions from avr-libc which of course are not available in libc/glibc for the standard GNU/Linux system. The common.h will contain the overriden IO functions:

#define SLIP_CHAR_SEND(__x) slip_sendc(__x)
#define SLIP_CHAR_RECV(__x) slip_getc(__x)

unsigned int slip_sendc(unsigned char a_char);
unsigned char slip_getc(unsigned char *a_data);

unsigned int slip_sendc(unsigned char a_char) {
/* printf("Sending char: %02x\n", a_char); */
return write(g_fd, (void *)&a_char, 1);
}

unsigned char slip_getc(unsigned char *a_data) {

unsigned char x = read(g_fd, a_data, 1);
/* printf("Recv: %02x\n", *a_data); */
return x;
}


The slip_send and slip_recv functions in the host application will use those to perform IO. Of course we need a Makefile to solve the inclusion paths and to make the whole project compilable. Finally our C application to send the binary data will look the following way:



Let's shortly walk through the code. First I open and create the file descriptor for the serial port (lines: 14-17), next two lines configure the port attributes and set the port to blocking mode. Next 5 lines (22-27) simply set the data in the struct timer_data which will be sent through the serial port to the Arduino. In line 30 and 31 the crc of the data packet is calculated and placed in the packet itself, finally on lines 42,43 and that's it.

How to use it ?


That's really simple. In order to test the examples presented bellow one can obtain etheir a project snapshot from here - it contains a snapshot of libpca and the source for the Arduino firmware (arduino_slip) - simply navigate to this directory and type make to build it and make install to flash it to Arduino. On the host side one can use either the host_slip.pl Perl script or build the application in host_slip directory by typing make. The application can be used the same way as host_slip.pl script.

It's possible to obtain all the sources from my public repositories from github in order to do that, one should

  • clone the newest version of libpca 
git clone git@github.com:dagon666/avr_Libpca pca
  • clone my Arduino projects repository 
git clone git@github.com:dagon666/avr_ArduinoProjects projects
  • navigate to projects/binary_slip
  • build both arduino_slip (remember to disable the TDELAY_IMPLEMENT_T0_INT in libpca config.h file since we are defining our own custom interrupt handler) and host_slip application
How to use the application ? After flashing the Arduino, regardless if we'll gonna use the perl script or the C app, the syntax is the same:

$ ./host_slip.pl 
host_slip.pl <prescaler> <ocr> <max_st>

The first parameter is the prescaler value (0-5), the second one is the value which will be placed in the OCR, and the last is the maximum value for the software timer, by invoking:

$ ./host_slip.pl 5 255 64

we'll program the timer to blink with a frequency around ~ 0.5 Hz. Experiment with the values. 

I hope that this article shed some light on the aspects of binary data exchange between the Arduino and any other machine and I hope that it proved that it's not that very difficult. Next time I'll try to present a project using this knowledge to build some more useful stuff.





Tuesday 8 October 2013

Serial Port STDIO

In previous post I covered the basics of configuring and using the Atmega's USART as a standard RS232 port. This post will be more a little addendum to the previous one. In our program of course we can manually send strings using the serial_send/serial_poll_send routines, but we can use the serial port in more convenient and transparent way. If our application implements only some kind of CLI system or basically treats the serial IO as a Serial Console we can configure libc to use our serial IO routines for stdio functions like printf. What we have to do is to create an input output file streams and instruct avr-libc to use those. Let's have a look at the code bellow:

The code refers to the serial IO routines implemented by libpca. First of all _serial_putc/_serial_getc are just a wrappers, interface adapters sort of speak, to adapt our serial IO API to the library's one. In serial_install_stdio() we simply create two file streams, one to be used as STDOUT (and associate the necessary output routine) and one for input. Next we assign our custom streams to the standard library stdio streams utilized all over the stdio library. And that's the whole magic. Those are the internals which can be used as a reference, let's have a look how it can be done with libpca

As usual, let's clone the newest version of the library and create a project directory:

git clone git@github.com:dagon666/avr_Libpca pca
mkdir serial_stdio
cd serial_stdio

Our example program looks as follows:


After connecting to the Arduino's serial port (ttyACM0 in my case), we can see the output produced by printf(). That's great !

screen /dev/ttyACM0 9600

There is a small remark regarding the AVR libc printf routines. Since we're talking about embedded device, the default implementation is very limited (in order to save program space) and does not support floats. When trying to print a float value, we'll see a '?' instead of the number. We can enable the support for the floating point numbers though, by explicitly providing a linker flags (avr libc stdio.h):


-Wl,-u,vfprintf -lprintf_flt -lm

Those must be appended to the LDFLAGS variable in the Makefile. as usual the example code can be downloaded here.

Friday 4 October 2013

Atmega328's USART

It is essential to have a convenient functions allowing quick serial port setup and usage. In this post I'll try to cover the basics of using the serial port, data reception & transmission + using interrupts.

Serial driver routines are very similar to supervising a NIC. Of course they are much more simpler and the speeds are lower as well, but the general approach is very similar. In the simplest model we can realize serial transmission/reception directly - by simply sending the data byte by byte waiting for the hardware to finish each transmission and poll the hardware status register before queuing/dequeuing new data. Although this approach will work, it's not very elegant and it requires constant CPU attention which consumes a lot of processing power. Using interrupts provides a lot more flexibility and save some processing power since we don't need to constantly check the USART's registers in order to know whether we have any pending data or not.

As usual we need some building blocks. Regardless of how the transmission/reception will be realized, an USART initialization routine is a must. We'll setup the port in needed mode, configure the amount of data/start/stop bits, the parity and the baud rate. Of course we need a routines to transmit/receive a byte from the port. Those will be implemented in polling fashion in the first place, then we'll introduce interrupt driven transmission/reception.

The initialization

Atmega328p's USART is a very friendly peripheral to configure. For most of our applications a standard configuration of 8 data bits, 1 stop bit + no parity bits, will be sufficient and we're going to configure the port in that manner. First thing that has to be done is as usual with every Atmega's peripheral - enable to power to it, we can use the macro from avr libc power_usart0_enable(). Next let's configure the port baud rate. This is done by setting up values in three registers. UBRR0H/UBRR0L and UCSR0A - where we enable so called double speed. Without going into details for most of the standard speeds there is a nice table provided in the datasheet, and we're going to use it. Arduino is using standard crystal of 16MHz, in order to configure the port for 57600 bps We need to write 34 to UBRR0H/UBRR0L register pair and enable the double speed (to minimize the frequency error). Next thing is to enable asynchronous mode 8N1 by writing 0x06 to the UCSR0C register and finally enable receiver and transmitter. The whole procedure looks as follow:

Data reception

Accordingly to the datasheet, we must poll for the RXC0 bit to be set in the UCSR0A status register and read the data as soon as it has been set. The data must be read as quick as possible to empty the hardware FIFO and make some place for new data, otherwise we'll drop the incoming data. The simplest reception function will look the following way:

This function is blocking of course. This means that once we request for example a single byte from the serial port, the CPU will be looping inside until the byte has been received. This may be a problem since it prevents us from doing anything else in the meantime. 

Data Transmission

It is realized in a very similar fashion, we poll the status register for the value of UDRE bit. Low value indicates that the USART is busy performing transmission, high value indicates that USART is free and we can place new data into hardware FIFO UDR0 register. The data will be immediately transmitted. For code reference have a look at the serial_poll_send() function on the previous snippet. Again, the function is blocking. It will block until a complete data buffer provided as an argument will be send. This may be a problem when our application produces a lot of data and needs some time to generate it without being interrupted. In most of the application though, blocking data transmission does not pose any issue. The interrupt driven reception is more important.

Interrupt driven IO

If we're not particularly worried about every CPU microsecond, and our application is flexible enough that it can work without problems with blocking IO during USART communication, then more or less we're done and the presented set of routines is functional complete. To get the most though from our MCU and it's USART utilizing the interrupts is a must. By sacrificing some memory for so called reception/transmission ring buffers we can perform the USART IO operations "in the background" interfering the main processing flow almost insignificantly.

Operating in such manner, as mentioned previously, is very similar to what NIC device drivers perform, the principle stays the same. For both directions (or only one if there is no need for the other) we define a ring buffer which is a queue more or less. The USART will act as a producer and the CPU will act as the consumer (in RX direction). We assume that the CPU can really read the data faster than the USART can provide thus the situation when there is no space in the ring buffer and the data must be dropped, will occur very seldom. 

In TX direction it works the opposite way, our application is the producer and USART is the consumer. If there is no space in the buffer (which means that either we have a too small transmission buffer or we produce the data too fast) the transmission will fail and will have to be retried later on.

Ring buffer

This is the essential part of the implementation. Let's define it as:


The data structure is very simple and common. One may found a lot of references describing details behind it thus I'm not going to provide a very thorough description. In principle, a "producer" pushes the data at position pointed by the head and increments it. The "consumer" takes the data pointed by tail until tail == head. The write/read operations in the ring buffer are independent. Have a look at the diagrams bellow, which depict most of the states a ring buffer may be found:

Operations on Ring buffer.

Interrupt driven data reception

Interrupt driven reception is much more widely implemented and much more useful than interrupt driven transmission. In most of the applications there is only a need really to provide this kind of input data handling while the transmission is still realized the "simpler" polled way. Let's get into details. The model is quite simple. We have an RX ring buffer which will hold the incoming data and a bunch of routines which will simply get the data and return it out of the ring, freeing the space in it. So in a sense the incoming data is buffered and waiting before it is actually read by the application. To summarize it stepwise, it would be:

Reception:
  1. Configure the port and install USART RX interrupt.
  2. When data arrives an interrupt is called.
  3. In ISR if there is space available in the ring buffer, place the new data in the ring and increment the head index.

Application reading data:
  1. If head != tail we have data available in the ring buffer.
  2. Return data indexed by tail.
  3. Increment tail.
Let's have a look on the code bellow:




Interrupt driven data transmission

It's very similar to the reception. We will use the UDRE empty interrupt which is triggered when the data transmission FIFO is empty. This means that when the CPU and USART are idle this will be called ALWAYS. Now that's not exactly what we want. Unlike in case of data reception interrupt, we will unmask the UDRE interrupt only when there is data available in the TX buffer and mask it as soon as the last byte has been sent. Let's list the steps of the algorithm:

Application:
  1. Queue new data for transmission if there is space in the ring buffer
  2. Unmask UDRE interrupt


ISR:
  1. Transmit the data from the ring buffer
  2. Mask the UDRE interrupt if there are no new characters to be send


And the code:

The functions and code snippets presented are taken directly from the libpca library (with small modifications), I kept the function names in tact to be compatible with the library as well. The complete usage example will use the library, we'll create something simple, the serial echo program. As usual let's clone a fresh copy of the library and create a directory for our project:

git clone git@github.com:dagon666/avr_Libpca pca
mkdir serial_echo
cd serial_echo

Our main.c will look the following way:

libpca defines an enumeration of standard serial speeds. User can specify the speed manually (by providing the number), or using the enumeration. In the above example the reception is realized using interrupts and the transmission using polling. In order for that to work one mustn't forget about enabling the RX interrupt in the library configuration file (include/config.h):

#define SERIAL_IMPLEMENT_RX_INT 1

After compiling our program, we can test it with the terminal emulator of choice. I'll use screen, my Arduino's serial port is visible as /dev/ttyACM0, I connect to it by typing:

screen /dev/ttyACM0 57600

Now whatever you'll type should be echoed back by the Arduino. That's all, we've mastered the serial port.  The libpca project can be downloaded from here:

https://googledrive.com/host/0ByE_WFvvg-guTzE4TmFwMDhaZTA/serial_echo.tgz

We're ready to do some more complex stuff now. Now as long as transferring text information through the serial line is quite easy and error prone, binary data transfers may be a little bit trickier. I'll try to cover the binary data transmission quirks in my next post as this knowledge will be needed in order to introduce you to some of my projects.




Tuesday 1 October 2013

Generating Tones with Timers

Previously I presented a basic way of using timers in order to generate delays. At this time, we'll learn how to use timers and piezo buzzer to generate tones (beeps). The code for that won't be very different from what we were doing previously. Let's start with defining the requirements of our implementation. Our "beeper" function should allow us to generate a beep of given frequency and duration. Simple as that. Just to quickly summarize, when using timer for delay purposes we configured it in CTC mode and configured the prescaler and OCRXA register value so the timer "overflowed" with a 1 kHz frequency (1 ms period), the delay duration was specified by the number of "overflows". Previously, as long as counting the 1 ms cycles was good enough to determine the duration, now the duration depends on both parameters, the actual duration specified as an input parameter as well as the frequency itself. 

Let's list steps that our "beep" generation algorithm will have to take:
  1. Program the timer in CTC mode.
  2. Program the OCXA pin as output.
  3. Program the timer's compare output mode to toggle the OCXA on each compare match.
  4. Calculate the prescaler and the OCRXA values to generate the frequency of 2*f.
  5. Calculate the amount of cycles needed to "realize" the requested duration.
  6. Enable interrupts.
  7. Decrement the cycles counter (calculated in step 5) inside of the ISR.
  8. Stop generation and disable interrupts once cycles counter reaches zero.


That looks a lot more complicated than previously. Don't worry it only appears that way. The trickery is hidden in steps 4 and 5, the rest is more or less the same. In point 4, let's think a for moment; why does the frequency should be twice as high as requested ? The reason behind it is really trivial. The output will be generated on pin OCXA by the hardware, which in CTC mode and COM0A1/COM0A0 configured to 0x01 - will be toggled each time a timers TCNTX register value will be equal to the one programmed in OCRXA, after that the TCNTX register will be reloaded with zero. What that means is a single timer cycle generates only half of the square wave output. So, we need the frequency twice as high for the pin to be toggled with the one requested. 

First thing that we will have to think about is a universal function which will calculate the values of both the prescaler and the OCRXA register to program the timer to the requested frequency. The formula for the frequency for the Timer in CTC mode (as stated in the datasheet) is:

f = fclk / [ 2 * N * (1 + OCRXY) ]

where:

fclk - system clock (16 MHz for most of the Arduinos)
N - the prescaler value (1,8,64,256,1024 - for timers 0 and 1)
OCRXY - the value in the register OCR0A/OCR0B (register for timer 0 used as an example, by the way, through out this post and in most of my code I focus only on Output compare register and output compare A, but You can use register/output B good as well - it's a matter of preferences).

Using the formula above we will have to calculate the values of N and OCRXA in runtime. 

Let's have a look bellow on the code which does that for Timer0: 



Let's analyze this code. There are some improvements introduced inside to make it a little bit faster/smaller. In fact it is completely decoupled from hardware, since it's a routine performing calculations only, so we can successfully compile it on an x86 machine and do some debugging. First of all a table of "prescalers" is declared. At first glance it contains some strange values. It stores the number by which a 0x01 should be bitshifted in order to get a valid prescaler value. It has been designed this way to save memory. The prescaler values for timer0 are 1,8,64,256,1024. To store those values we would need an uint16_t data type, consuming 5 bytes more than now. Since the prescalers are a powers of two we can only store the number of bitshifts required to get the needed value. The Atmega is a very fast core, we have a lot more computing power than memory available, that's why in most of the applications we should try to minimize the memory usage over computing resources. The calculation of prescaler and the OCR value is done in the while loop, the formula has been transformed to the following one:

OCR = [ fclk / (2 * N * f)] - 1

In every loop roll we compute the OCR value using another available prescaler. The loop brakes as soon as the OCR value is bellow 256 - which means that we have found a usable set of prescaler and OCR values in order to program the timer to generate the requested frequency. The values are returned by reference. Let's test this function offline. Consider & compile the followig code:

gcc -o pocr_test pocr_test.c



Let's do some tests:

$ ./pocr_test 1000
Frequency: [1000 Hz], Prescaler: [  64], OCR: [124]

$ ./pocr_test 13000
Frequency: [13000 Hz], Prescaler: [  8], OCR: [ 75]

$ ./pocr_test 8000
Frequency: [8000 Hz], Prescaler: [   8], OCR: [124]

$ ./pocr_test 30
Frequency: [30 Hz], Prescaler: [1024], OCR: [255]

./pocr_test 31
Frequency: [31 Hz], Prescaler: [1024], OCR: [251]

$ ./pocr_test 20
Frequency: [20 Hz], Prescaler: [1024], OCR: [255]

Let's verify one of those manually:

- f = 16MHz / [ 2 * 64 * (1 + 124) ] = 1000
- f = 16MHz / [ 2 * 8 * (1 + 75) ] = 13157
- f = 16MHz / [ 2 * 8 * (1 + 124) ] = 8000
- f = 16MHz / [ 2 * 1024 * (1 + 255) ] = 30

Looks quite promising. Of course the function does not minimize the frequency error, it only tries to find first matching pair of values in order to satisfy to request. In the second example a values of (prescaler: 8, ocr: 76) would provide a better fit (12987 Hz, 13 Hz difference instead of 157), but for most of the applications (especially tone generation using buzzer) such error is really acceptable. For more precision we would have to gather the OCR results for all prescaler values and return the one which is less different from the requested frequency. To have a real control we would probably define those values ourselves and use some sort of lookup table. At the moment we do not have to worry about that.

Second thing that we need to do is to determine how many timer overflows we need to count in order to determine the duration. Let's think about it. For a given frequency 1 cycle lasts for :

T = 1/(2*f)

The frequency is multiplied by two since, the timer is running with frequency twice as high as needed. The duration will be specified in millisecods, so we need to determine how many cycles (x) do we need do realize it. This can be calculated using proportions:

duration * (1/1000) = x * (1/2*f)

so, x:

x = (2 * f * duration) / 1000

Simplifying:

x = (f * duration) / 500


and that's the answer. Now we have all the necessary tools to finish our implementation. Let's start:


Let's connect the buzzer to the Arduino board digital pin 7 (PORTD6) (no resistor is needed), and test our code.

Connecting piezo buzzer to Timer0 OC0A output.

This functionality is covered by the libpca as well. As previously it's very easy to use. The timer_freq_prescale() function equivalent is implemented there as well in a very similar fashion (it's called _timer_freq_prescale()). Again, it's done in way to provide flexibility and be as much universal as it's possible. Let's cover the implementation in the beeper library module. Most important functions are:

- void beeper_init(e_timer a_timer);
- void beeper_beep(e_timer a_timer, uint32_t freq, uint32_t duration);

The interface is self explanatory. Let's write a simple program which will do more or less the same as our previous example:

$ git clone git@github.com:dagon666/avr_Libpca pca
$ mkdir beep_libpca
$ cd beep_libpca
$ touch beep_libpca.c




We cannot forget to enable the Timer interrupt implementation in the library configuration file in order for the code to work (this will be TDELAY_IMPLEMENT_T0_INT variable which must be defined as 1). The implementation presented as well as the one in the library can be extended further on. We could use the PWM capability to provide the volume control to the beep generator. For the moment though I do not plan to cover this topic. I hope that this post has been useful. In my next posts I'm going to cover the basic usage of the UART port. After that I'll proceed to some more interesting stuff (I hope :)).