Max serial read speed ?

[FFMan]

I'm using a 128a1 running at 29mhz with an openlog sd card module.

this module is great at receiving fast incoming serial streams and logging them to a micro sd card. however its read functions are limited to issuing a 'read <filename>' which then streams the entire file at the baud rate until eof.

so writing at 38400 is great but reading a file back results in what appears to be dropped characters. i am reading records terminated with LF then doing some minor processing befofe reading the next record.

the incoming buffer is 200 chrs, but increase to 300 does not improve things, i think characters are being lost regardless of buffer size.

what the fastest one can realistically expect to read. the gps module runs at 115200 and doesn't drop chrs, but the stream is not so continuous.

this is the snippet of code doing the read

Code: Select all

	do until getqueuecount(Inqueue)=0
		Call GetQueue(Inqueue,SDIn(inPtnr),1)
		'debugprint &#40;chr&#40;SDIn&#40;inPtnr&#41;&#41; & 	"<" & cstr&#40;sdin&#40;inptnr&#41;&#41; & ">"&#41;
		if SDIn&#40;inPtnr&#41;=10 then
			debugprint &#40;"term LF"&#41;
			goto ExitPoint
		end if

		inPtnr=inPtnr+1
		snStart=timer
	loop

it then uses makestring to convert the byte array to a string. is this an expensive process ? is values particularly slow ?

any help much appreciated.

[FFMan]

on further checking i see that my 'minor' processing is actually writing to persistent variables.

this is a known slow operation isn't it. could this account for the apparent dropped characters ?

[dkinzer]

writing to persistent variables is a known slow operation isn't it. could this account for the apparent dropped characters ?

Indeed. The erase time for EEPROM is 6mS and the write time is 6mS, both typical values. This suggests that writing a single cell could take 12mS or more. With a full-speed 38.4K baud stream, a new character will be available about every 260uS. Even if the EEPROM write time were only 1mS you'd still drop characters.

[FFMan]

ok - i commented out the write to the persistent variables but i'm still dropping some characters.

in my test program that simply read a character and wrote it via debug.print i don't recall seeing any drops of characters (and pretty sure it was at the same speed but i would need to verify), so i am thinking that reading at this speed is possible so long as the intervening processing steps are light. is this likely to be true or is the arrival rate of characters just too fast.

i have enough ram free to read the file into a byte array and post process it as the records i am interested in are less than 1.5k characters. this might be the right approach, or make multiple passes discarding reads until the last known good point and then pick up the processing.

any comments much appreciated

[dkinzer]

is the arrival rate of characters just too fast.

There are two situations where characters might be lost. Firstly, if interrupts are disabled for more than two character times (typically 20 x the bit time) you can lose an arriving character at the USART. (The USART has two registers, one for holding the last fully received character and one for accumulating the bits of an in-process character.)

The second potential problem is the receive queue. If the queue is full when a "character received" interrupt is generated, the ISR simply discards the character.

You might try turning off the RTC (Option RTC Off) and see if the situation improves. If so, that probably suggests the first cause described above.

[FFMan]

i get this when i try and compile it

QualiAssist2013.bas:678: Error: to use "Timer" the RTC must be enabled

i trued increasing the incoming queue size from 200 to 300 but this did not appear to make any difference or push out the place in the file where the character loss occured.

i'll try some other approaches and see what i get.

Do cpu speed type calculatins suggest that capturing characters at 38400 is feasible ?

[dkinzer]

Error: to use "Timer" the RTC must be enabled

Well, yes. You must delete, comment out, or otherwise effectively remove code that requires the RTC if you turn off the RTC. There is a built-in Option.RTCEnabled (having type Boolean) that you can use in a conditional to effectively remove RTC-dependent code when it is disabled.

Code: Select all

#if Option.RTCEnabled
' RTC-dependent code here
#endif

Do cpu speed type calculatins suggest that capturing characters at 38400 is feasible ?

It is feasible. As mentioned earlier a new character will be received at the rate of 3840 per second, i.e. every 260uS. If you're using an xmega, it is running at 29.5MHz meaning that you have 7680 CPU cycles between character receptions. That is sufficient to do some processing but the amount must be necessarily limited.

You can instrument your code to measure how much processing time is required for a particular set of functionality, using the RTC for time measurement. The general idea is to get the RTC time, call the function of interest a number of times (say a hundred, a thousand or a million times and then get the RTC time again and report the difference divided by the iteration count. Note that no actual character reception is required for this kind of testing, just call the routine that would be called on character reception and measure its execution time.

[dkinzer]

[...] using the RTC for time measurement.

For an application that has the RTC disabled, you can also use a stopwatch or similar timing device. You'll just need to adjust the iteration count so that the total elapsed time is 10 seconds or more in order to get good resolution. The longer the elapsed time, the lower the error will be attributable to human observation and reaction time.

You can also run the test with and without the RTC being enabled to measure the effect of the RTC running. For this purpose, it is probably best to use the stopwatch method for both runs to avoid introducing more than one change that might affect the outcome.

[FFMan]

just for the record, by keeping inter-character processing very light the 128a1 can keep up with 38400 all day.

I had enough ram to read the file into an array and post process. Not the most elegant and size limited solution, but this was reading a very specific config file so the contents were clearly defined.