Can't get remote dowloads to work via XBee, even at 19.2K

[Innovator_99]

At least one other forum member (sturgessb, Topic #1082) has reported success with remote downloads from the IDE via a point-to-point RF link using XBee RF modules--at least at 19,200 baud. I can't seem to get this to work, although DTR-less downloads works fine with a hardwired connection.

Here’s the configuration:

On the remote end of the link, I'm using a ZX24p loaded with the sample “hello world” program, per forum topic #956, that includes the ATNChar (&H04) detection task (and with the ZXCmdMode parameter set to false). This remote ZX24p is connected to an XBee via a 3-line interface (Tx, Rx, Gnd), with the appropriate transistor level shifters/inverters on the Tx and Rx lines.

On the local end, I'm using a PC running WinXPP connected to an XBee RF module via an FTDI-based USB-to-serial adapter (COM8), with hardware flow control disabled. I'm using Zbasic IDE v1.5 with “Use ATN Char” (4) enabled, and speed set to 19200, in the “Device Options” tab.

Both XBees are configured for 19200/8-N-1.

Here’s what happens:

“Talking” with the remote xBee/ZX-24p using a terminal emulator on the PC, through the RF link, yields the expected results:

--“Hello World” messages are received fine.

--Sending &H04 causes the remote ZX24 to return &H3E.

--Sending &H1B causes the ZX24 to return &H0D &H0A (CR/LF), followed by another &H3E.

--Sending &H21 &H0D returns &H21 &H0D &H0A, and puts the ZX24 back in execute mode.

And a couple of things also work fine when using the IDE:

--“Hello World” messages appear in the debug pane while ZX24 is in execute mode.

--The ZX24 can be remotely reset by selecting “Reset Device” from the Options menu, causing the proper reset message to be displayed in the debug pane.

But here’s what doesn’t work:

--Remote downloads fail, with a “Device failed to respond to ATN character (4) on COM8” message in the debug pane.

--“Auto-Select Device” on the Options menu fails, with a “No Response from Device on COM8” message box.

--The “Identify Device” button on the Device Options tab yields “Connected Device: Unknown” and “ATN Character: None”.

When any of these three functions (download, auto-select device, or identify device) is attempted from the IDE, the ZX24 gets stuck in some unknown state (until a hardware reset is performed). How the ZX behaves while in the stuck state depends on the value stored in the ZX24’s persistent memory at &H0013:

--If the value stored at &H0013 is &H00, then the “stuck” ZX24 responds to a “command mode” sequence (&H04 &H1B) by returning &HFB, and to a “execution mode” sequence (&H21 &H0D) by returning &HFF &HFD. It remains in the stuck state until a hardware reset.

--If the value stored at &H0013 is &H04, then the “stuck” ZX24 returns random bytes in response to the command mode and execution mode sequences. It remains in the stuck state.

Any tips on how I might get remote downloads to work would be most appreciated. Thanks in advance for any help.

[dkinzer]

I can't seem to get this to work, although DTR-less downloads works fine with a hardwired connection.

I believe that the crux of the issue is that the radio links like Bluetooth and XBee cannot change speed like hard-wired serial can. The download protocol for the ZX calls for switching to 115.2K baud for the downloading portion of the cycle. Since this does not work for the radio link, the download portion fails.

I'm using Zbasic IDE v1.5 with “Use ATN Char” (4) enabled, and speed set to 19200, in the “Device Options” tab.

It is important to note that the speed setting you refer to here applies only to the communication other than the download portion of the cycle. What is needed is some way to tell the IDE to not switch to 115.2K baud for downloading. Several experimental User Properties settings were added (see below) that Tom Becker has used to successfully download over Bluetooth. You may be able to use some combination to get your XBee downloading to work as well.

The new User Property options are shown below with their default values. The first gives the total amount of time, in mS, to await a response from the device after sending it the ATN character. The second gives the amount of time to wait, in mS, (while awating the response mentioned above) before sending a stimulus character (&H1B or Escape). The purpose of the stimulus is to evoke the prompt response so that commands may be sent. The third configuration item allows you to specify that downloading should be done at 19.2K baud instead of at the normal 115.2K baud.

download.wait.atn.char.resp=1000
download.stimulus.delay=500
download.speed.slow=0

You can set the values of these properties by selecting the "Open User Options File" entry on the Options menu. Once the file is open, you can add one or more of the lines above. Note that you can comment out lines in the User Properties file using a pound sign as the first character of the line.

If you add the download.speed.slow=1 line you may be able to download successfully over your XBee radio link. I would suggest trying this first without changing the first two properties above. Perhaps Tom can comment on what he ended up using with Bluetooth.

[Innovator_99]

Don, thank you very much; that makes perfect sense and is very helpful.

I assume that sturgessb (Topic 1082) used the same experimental User Properties settings to enable his successful remote downloads at 19.2K, but it wasn't clear from the thread. I'll try adjusting the settings as you suggest this evening and will report my results.

Thanks again to you and Tom.

[GTBecker]

Bluetooth downloads work here on two types of client adapters, but only
at 19200 - although, on occasion, the download needs to be restarted
immediately following a "Failure to respond to ATN character...".

I haven't researched this further since my note of a few weeks ago, but
will soon. My application seems able to run at 19200 for now but will
need the higher speed later, I suspect.

For now, my User Options file contains these entries:
download.identify=0
download.verify=1
download.clear=1
download.quiet=0
download.ext=zxb
download.use.atn.char=1
download.atn.char=11
download.wait.atn.char.resp=1000
download.stimulus.delay=1000
download.speed.slow=1

On the ZX-24n, I set the ATN capture character (at Persistent &H13) to
zero and test for the character in my code (at 19200), invoking
ZXCmdMode() after preparing my hardware for the download; if you use the
other method (Persistent &H13 = ATNChar) processor pins will likely be
floating for the download and verification duration.

The watchdog is also an issue. The bootloader code in ZX modules needs
to be rewritten if you need watchdog protection during downloading, or
you must close the watchdog before invoking ZXCmdMode() or allowing the
bootloader (Persistent &H13 = ATNChar) to do so.

You might also find that the ATN character you select is important.
Within reason the selection should be arbitrary, but I found that
ATNChar = 4 (Tab) or 8 (BS) seemed to be problematic even though I send
neither in normal application operation. Although I haven't needed it
much, I've included a recovery button on the PC applications that talk
to the project; it sends a reboot sequence (at 19200) to the ZX:
ATNChar, Esc, "!", CR.


Tom

[Innovator_99]

Thanks to both of you, remote downloads now work fine at 19.2k.

Tom, I appreciate your warning regarding the floating outputs when the ZX OS traps the ATN character, because my remote ZX24p is being used in a motor control application. And I will definitely incorporate your "recovery button" idea on the PC app (VBA for Excel) that will eventually talk to the remote ZX24p's.

I, too, would very much like the ability to run everything at 115.2k, and will eagerly follow any developments along those lines. But even at 19.2k, the capability for remote downloads is extremely convenient and useful.

Thanks again,
Phil

[stevech]

To prevent data overrun of the XBee (has a small buffer), you need to use CTS handshaking, or assure that the download protocol itself can preclude overrun.

The XBee 2.4Ghz runs 250Kbps air link rate which yields less than 100Kbps net, of overhead, in my experience. The 250Kbps rate is the IEEE defined modulation order. Be sure to have MAC layer ACKs enabled. There will be delays due to CCA (clear channel assessment) due to WiFi or other interference, and occasional timeout/retransmit (with MAC ACKs on). Such is life in wireless.

I run my XBees reliably at 115Kbps on the serial port but I use the API mode (data packets, not transparent serial). It works quite well and gets data to/from the XBee at high rates, though the net air link rate is far lower.

[Innovator_99]

Thanks. I had hoped that the xBee's 250K RF rate would be enough to support 115.2K on the UART without filling the buffer, but since that's not the case I'll plan on using hardware flow control for 115.2K operation.

[stevech]

Even then it's marginal. If you don't use API mode, then I suggest not going over about 56K on the serial port. Due to CTS latency.

[dkinzer]

If you don't use API mode, then I suggest not going over about 56K on the serial port.

I've been experimenting with the XBee devices a bit and found that outputting characters at full speed from a ZX at 57.6K baud seems to be OK, i.e. CTS never gets raised. The test was done with less than a foot between the XBee devices so the result may not be applicable to longer distances, with more RF noise, obstructions, etc. The same test at 115.2K causes CTS to toggle regularly.

I've made some experimental changes to native mode bootloader and the IDE and have been able to download to a ZX-24n over the XBee link at speeds ranging from 19.2K to 57.6K. It doesn't work at 115.2K, however.

Additional testing remains to be done but the general framework for downloading over a fixed-speed link using the ATN char method looks promising.

[stevech]

So many people read the wireless air link bit rate and make the mistake that this is the achievable bit rate at the user level. This isn't true even for 802.3 ethernet, due to ethernet packet header and checksums, plus IP. In 802.11 and 802.15.4, we have a half duplex medium whereas switched Ethernet (802.3) is usually full duplex. And in 802.11 and 802.15.4, we have CSMA/CA delays and waits for MAC layer ACKs. And each data frame is burdened with forward error correction bits. In 802.11, the air link bit rate is "rate adaptive", depending on the channel conditions - primarily the Signal To Noise ratio. And in 802.11, the rate used in both to-client and from-client must be the same - and this is profound.

In 802.15.4 for 2.4GHz, there is a single air link bit rate/modulation order; it's 250Kbps in a 2MHz channel.

So both .11 and .15.4 yield about 80-90% of the air link signaling rate before the network layer overhead, this is often IP for 802.3, 802.11.

For 802.15.4, most common is no formal network layer - and instead DIY, and less so use of ZigBee, 6LoWPAN, ISA 100, or emerging, 802.15.5. These add 15-40% more overhead. The bare 802.15.4 frames can hold 100 bytes each. And normally, each frame gets an ACK from the receiving unit.

You guys know, I assume, that XBee series 1 and series 2 or 2.5 are vastly different. Both use 802.15.4. Series 2.x are based on Ember's chips and I suspect that ZigBee MUST be used in series 2 by license agreement. Series 1 cannot use ZigBee but can use bare 802.15.4 or Digi's DigiMesh - arguably superior to ZigBee but proprietary. Both are meshing. Series 1 uses Freescale's chip and with T.I.'s acquisition of FreeScale's ZigBee provider company ChipCon, the situation arose.

I've used only series 1 because I don't want/need ZigBee.

Digi has a proprietary two or so bytes prefixed into the payload for their virtual wire and transparent serial firmware. There's an AT setup register to disable this so you can use the entire payload. And disable Coordinator mode, and disable Association Required.

This done, you have a peer to peer network, much like 802.11's Ad-hoc mode where each node has a priori knowledge of the address of the nodes it plans to talk to. For many applications, this is a vast simplification. You can easily build static-route meshes for nodes that don't move, without resorting to ZigBee.

[Innovator_99]

Don, will your experimental changes for selectable download speeds make it into a formal IDE release? In view of Stevech's comments, it would be very useful to be able to empirically determine---and then take advantage of---the fastest feasible download speed for a given link set-up and channel conditions.

[dkinzer]

[W]ill your experimental changes for selectable download speeds make it into a formal IDE release?

First, some background information. There are two download methods supported that differ primarily in the mechanism used to get the ZX device into command mode. Once it is in command mode, commands are sent to the device over the serial channel to retrieve the device's identity, initiate a download, etc. before finally making the device restart, executing the newly downloaded program.

The original mechanism to get the ZX into command mode is a timed series of pulses on the RESET line, effected by toggling the DTR of the serial channel. When the command mode is entered this way the Com1 speed of the ZX is automatically set to 115.2K baud and the IDE (or the zload utility) changes to that same speed to send commands, download the code, etc. This method cannot be used if the baud rate of the serial channel cannot be changed dynamically, if DTR is not supported or if DTR cannot effectively be toggled with the required timing.

The second method was added later to facilitate downloading - sending a special character over the serial channel that is recognized by the device as the ATN character. The recognition of the ATN character can be done by your application code (followed by a call to ZXCmdMode()) or it can be done by the Com1 receive code (matching the value of a Persistent Memory variable) which then calls ZXCmdMode() directly. In the original implementation of the ATN character method, when command mode is entered, the Com1 speed is set to 19.2K baud. After some housekeeping details, a command is sent to switch the Com1 speed to 115.2K baud and the PC COM port speed iss switched to match. Next the program is downloaded and optionally verified. Finally, a command is sent to switch back to 19.2K baud and a command is sent to cause the newly downloaded program to begin running.

The ATN character method of downloading works fine on communication channels that are capable of dynamically changing the baud rate but it doesn't work on fixed-speed links like BlueTooth, XBee, etc. A few months ago, I added support for a special property in the IDE that directs it to skip the speed changes described above when using the ATN character method. That change allows downloading using the ATN character method on a fixed-speed channel but only if the channel speed is set to 19.2K baud.

The experimental change that I made in recent days was to eliminate changing the Com1 speed to 19.2K baud when command mode is entered. Since the speed is not changed, this allows the ATN character method to work on a fixed-speed channel at whatever speed it happens to be running at at the time the ATN character is received. Essentially, this modification changes the semantics of the parameter to ZXCmdMode(). Formerly, the False value meant "switch to 19.2K baud" while the True value meant "switch to 115.2K baud". Now, the False value means "don't change speed" and that, together with the IDE property that says "don't change speed for downloading" makes the ATN character method work on fixed-speed channels at an arbitrary communication speed (assuming a suitable speed for the channel, of course).

For VM-based ZX devices, a new version of the VM is needed to support this semantic change - v3.0.4 or later (not yet in available). For native mode devices, the change had to be made in the bootloader, a part that is not field upgradeable. The native mode bootloader that supports the new semantics for ZXCmdMode() is v1.4 or later (not yet in production). If you're using a native mode device you'll either have to buy a new one that has the updated bootloader or send your devices back to be reprogrammed with the updated bootloader.

The only change made to the IDE was to add a checkbox for setting the "don't change speed" property of the IDE. A change was also made to the zload utility allowing you to specify "don't change speed" as well. I expect both of these updates to be in the next installer release (likely to occur by the end of April). The VM update and new native mode bootloader will likely be available by the end of April as well.

[twesthoff]

The experimental change that I made in recent days was to eliminate changing the Com1 speed to 19.2K baud when command mode is entered. Since the speed is not changed, this allows the ATN character method to work on a fixed-speed channel at whatever speed it happens to be running at at the time the ATN character is received. Essentially, this modification changes the semantics of the parameter to ZXCmdMode(). Formerly, the False value meant "switch to 19.2K baud" while the True value meant "switch to 115.2K baud". Now, the False value means "don't change speed" and that, together with the IDE property that says "don't change speed for downloading" makes the ATN character method work on fixed-speed channels at an arbitrary communication speed (assuming a suitable speed for the channel, of course).

Don,
How about instead of making the parameter True/False, make it specify the baud that is desired? Maybe 0 and 1 work as you described above, and other values select the speed. I can imagine many people wanting specific speeds for different communication methods.
Tom

[dkinzer]

How about instead of making the parameter True/False, make it specify the baud that is desired?

There is a way to specify one of a set of specific baud rates but I don't think it will be needed because the strategy of "don't change the speed" allows any suitable baud rate to be used. By this I mean that the user's program will have set the Com1 speed to some baud rate and the IDE will have been set to match that rate. Then, for downloading if the baud rate is kept the same on both ends it works as desired.

The key to this is that the downloader has to be operating at the Com1 speed in order to send an ATN character. Therefore, not changing the speed on either end allows communication to continue. I haven't yet thought of any useful scenario where "don't change the speed" will fail to work.

[stevech]

In 802.11, 802.15.4, et al, the channel data rate varies due to clear channel assessment (CSMA/CA) delays, interference, fading, etc. So if you stream data, you either need a good handshaking software protocol like Xmodem, or a downloader protocol that has inherent flow control. But more so, that flow has to deal with the (Xbee or other) buffering limits and not overrun.

Example: If the XBee drops CTS when 200 bytes are buffered, the sender must stop. If CTS is ignored/not used, then the streaming protocol needs to have the equivalent. In the downloader protocol, x number of bytes could be sent then the sender waits for the receiver to send ACK/go-ahead with more. If x is greater than the XBee's buffer size, which is small, there will be failures.

So to preclude this, either use CTS hardware handshaking or arrange for the stream's block size to match what the XBee's input buffer can accommodate.