ZBasic Forum

I saw a thread on here recently regarding capturing GPS data output from a GPS module. It was agreed that string handling is not that efficient and the suggestion was that the data was captured in byte form and then processed.

Has anyone got any tips as to how best to do this. Given the output rate from the latest generation of gps unit, the more efficient the code the better.

I need to extract the positional data and compare it with a list of coordinates defining virtual lines across a race circuit. The intention is run it at 5hz which hopefully leaves plenty of processor time to service the other 2 tasks.

thanks

FFMan wrote:I saw a thread on here recently regarding capturing GPS data output from a GPS module. It was agreed that string handling is not that efficient and the suggestion was that the data was captured in byte form and then processed.

The intention is run it at 5hz which hopefully leaves plenty of processor time to service the other 2 tasks.

Well, using strings is not THAT inefficient. However, waiting for the GPS to spit out the data at 9600 baud is slow. 9600 baud is about 960 characters per second, or 192 characters in 200mS. Would it be OK if the data that you are currently processing is 200ms old? I.E. if the data is from the previous GPS update?

First, can you tell the GPS which updates to send, and avoid updates with non-essential info? Limiting the amount of data coming in at 9600 baud will be helpful.

Next, you will need to use a receive buffer that is large enough to hold all the data coming in from one full update. Then pull bytes out of the buffer checking for the '$' identifier then pull the next 5 characters to look for the correct sentence type. It is relatively fast to pull bytes looking for the '$'. Parsing the necessary sentence is not fast, but it is not terrible.

I have not timed my GPS parsing, and I have a 1Hz update. I can very easily keep up with the data stream and do other tasks. 5x more data would be pushing it. Using a native mode CPU may help some.

Does your GPS have the capability to send something other than NMEA sentences? Hopefully the data is not some poorly documented proprietary format!

One of the tasks can be the parser task. When it has successfully parsed the incoming data, and put the data in a buffer, then it can set a flag that the data is ready. Another task can copy the data out of the buffer, and reset the flag, and then process the data.

The parser can sleep for a while once it has emptied the receive buffer, rather than continually test for a non-empty buffer. This should save some CPU time doing task switches and pointless testing of the buffer. The sleep time can be somewhat less than the length of the NMEA sentence you want to read. At 9600 baud, it is about 1.04ms per character, and I bet you would need to read well over 20 characters, so a sleep of 20mS or longer might be reasonable. The goal would be to have the parser pull the last character of the sentence out of the receive buffer JUST after it got in the buffer. This would make the parser run at maximum efficiency. It would get the most done in its time slice and require the fewest task switches to get the job done.

The parser will know a whole lot about the incoming data, more than a general ValueI() or ValueS(). and it MIGHT be faster to apply that specific knowledge to write a custom version of the ValueX() routine. Remember the built-in routine will run in machine code, while the custom code will not (unless running on a native mode CPU).

With a little effort in making things efficient, I think you can sucessfully parse the NMEA sentences at 20Hz and still have time for meaningful processing of other tasks.

-Tony

thanks for the reply.

A few details to add to the mix now I have the module up and running.

I need it to run at its max which is 5hz. I've reduced the message output to just GLL positional info which is approx 51chrs per message, and its running at 19200. I may need to push it to 10hz once its all proven using the next module up.

the delay in decoding the message if acceptable so long as it is constant.

the target project will run on a 328n so plenty of speed.

I am wondering about the best language feature to use to capture the data. In the old days I used to program comms on Vax/vms and there was a wonderful feature whereby you could set a bit in a 255 bit flag which indicated what characters would terminate the read. Ideal if you were expecting cr/lf on the end as here.

the length of the received string doesn't vary as it pads with zeros, so should i use getqueue and specify the length i want ?


thanks

FFMan wrote:should i use getqueue and specify the length i want ?

That's probably not a good idea. The problem is that you want to operate on whole sentences. Just because the queue has at least N characters in it (where N is the expected size of the NMEA sentence) that doesn't necessarily mean that it contains a whole sentence nor that the lead character in the queue is the first character of the NMEA sentence. What you need is some code to synchronize on the dollar sign that begins a NMEA sentence but not process that sentence until you have the terminator (likely CR/LF or just CR).

If you have the RAM space available, it is simplest to extract characters from the queue as they are available, discarding all until a dollar sign is seen. Then, begin storing the received characters in a buffer (Byte array) until you see the terminator. At that point you know that you have a complete sentence in the buffer and you can go ahead and process it.

If you're short on RAM or don't want to use it as described above you can do essentially the same thing using the queue itself as a buffer. For this you would use PeekQueue() to see the next character in the queue. If it is not a dollar sign then remove it from the queue using GetQueue(). Once you know the dollar sign is the lead character in the queue you can then monitor the amount of data in the queue using GetQueueCount(). Once the queue contains the expected number of characters you can use SearchQueue() to confirm that the end-of-line is in the queue. At that point you know the queue contains a complete sentence with no other data preceding it and you can extract it from the queue to process it.

As usual, Don's reply is very complete. I was going to recommend using the receive buffer as the data buffer.

Also, I think it is not necessary to do any peeks at all You pul data off the buffer until you get the dollar-sign. Bot the head of the buffer has the start of an NMEA sentence.

It sounds as if you can limit the GPS output to just the one sentence you need, AND you know precisely how long that sentence will always be. Therefore, you can test the size of the receive buffer until it hits the requisite sentence length. Now you know it will have everything you need to parse efficiently.

Also, if you are only getting the one NMEA sentence you need, and no other stuff at all, then you know that when you have pulled a whole sentence out of the buffer, then anything else in the buffer must only be part of the next sentence. In that case you don't even need to look for the dollar-sign. Just wait until the buffer is the right size.

Again, you can use sleep() to prevent the task from activating for a pre-determined time.

You said that the time between processing sentences must always be the same. Most likely the time between complete sentences will be adequate to pace your processing, but if you need more accuracy, then you can use waitforinterval(). However, you can get out of synch with the GPS data if you wait too long.

In my previous message, I said that the parser might set a flag when the NMEA sentence was fully parsed. That will occur quite regularly, and probably will be adequate for pacing the processing to the data.

In the parse task, 51 characters at 19200 baud takes 26.6mS to complete. I don't know how long it takes to do the parsing, but a few mS would be my guess. You can sleep for at least (200-26.6) 173.4mS before restarting the parsing process. You might want to wait about 10 mS longer.

Probably the most accurate way to get exactly 200 mS pacing is to setinterval() for 173.4mS (or whatever, up to 200mS) right after the buffer gets the final character of a sentence, then when done with the parsing, use waitforinterval().

-Tony

-Tony

One tweak I've done: turn off GPS sentences except for the one(s) you need. To turn on/off, you need to use proprietary non-NMEA, as I recall. I've done so for SiRF chipset receivers. It can be hard to find the proprietary commands documents. And the same goes for changing the baud rate from the NMEA default of 4800 if you want to.

i've turned off all the stuff i don't need. there is a handy app called mimigps that allows you to change the sentences, baud rate, refresh rate etc with no drama

will do some work on it tonight.

Lots of GPS NMEA parsing code on the 'net. Some VB6 you can find would be easiest to adapt to ZB. Or roll your own; it's not complicated. I've done it, and the hardest part was assuring the parsing can cope with and recover from garbled messages - rare, but will happen.