Research

At the bottom of this post is something that we have been working on for a little bit now. Basically Documentation on two of the 4 tiers of our Wireless Sensor Network.

Since I have not updated since we decided to go from our 3 tier idea to a 4 tier idea, I will briefly explain the differences as well as the reasoning behind the switch, and why our 4 tier is better than our 3 tier.

First off, the description of the 3 tier model. The lowest tier was the Wireless Sensor Motes, the 2nd tier was the Stargates and the top tier was the clients. The clients would connect to all the stargates (at that point, the clients had to know the ips of the individual stargates to be able to connect) and get data from them. They could query individual motes and get continuous readings and change the frequency of updates for the motes or set the motes to only update if the sensor reading changed beyond a threshold value. Unfortunately, this system had the following issues: only getting data when a client was there, having to maintain a constant link, inability of the stargates to initiate connections to the clients (stargates didn’t have a method to figure out the client IPs), using ssh for communication was slightly problematic (that is to say, required key exchanges and/or setting up users on the clients).

To alleviate these issues (among others) we decided to implement a 4 tier system, inserting a BaseStation server inbetween the clients and the Stargates. We also pretty much scrapped the ssh tunnel idea (but it is still a useful concept, just not for this application) and went with a purely socket based communication system. We also added a DHCP server to make it easier for clients to join the network, as well as making it easier for setting up the network. The Basestation and stargates can now automatically find each other when they are turned on. They send broadcast packets looking for a basestation (if you are a stargate) and looking for stargates (if you are a basestation). The Stargates and Basestation now can find each other and communicate effectively. We also implemented three modes of communication: always on, event, and queue. Always on mode is basically a socket is opened between each stargate in always on mode and the basestation, sending all packets across without having to (re)create the socket. Event only sends communications back the basestation when the motes sense something. Queue allows for the clients to send a message to the basestation which in turn tells the specific mote(s) to send back their current status. These modes, especially event, make it possible to configure how the network is utilized. These changes allow for easy utilization of the system, just turn everything on, and start sensing. Another advantage of the basestation is that we can store all the data that we collect in one place and not have to worry about having a client there all the time.
Current known problems/possible issues with our 4 tier system.

• It is possible for third parties to inject data into the system (possible solutions, implement some sort of cryptosystem for communication (between stargates and basestation, client and basestation), as well as enabling wep (just a hindrance I know, but for such a low traffic network probably a good deal of hindrance))
• It is possible to emulate the “I’m a Basestation” packets, thereby effectively killing all communications from the real basestation (This could possibly be solved with something akin to GPG/PGP signing).

Current issues that we are working on.

• Implementing graphical viewing of data, we are thinking of modifying Oscilloscope from net.tinyos java package.
• Implement a “log file viewer” for the clients so that they can choose which data they want to display.

BaseStation Documentation
Stargate Documentation

These are clarifications, goals, notes, and thoughts from/about last meeting.

Current Defaults
1s periodic with 10/1024 threshold (Shane has specifics somewhere)
frequency of reading is 1 second

Threshold
Modes on all motes as well as specified motes.
-this calls for implementing a BCAST protocol in the sensor network
-or for sending the same command to every mote…(inefficient use of bandwidth, but
Query based
Partially implemented
Eventually do Implementation from the stargate conecting to the “base station”?

Periodic based
Partially implemented

Threshold based
Either Max (or min) or percent difference.
Mote implementation as well todo
As soon as it has passed threshold, ¿(continue to send data at the default Periodic rate (or prior rate that it was set to))?
Saves battery
Sort of inbetween Query and Periodic

Stargate init tunnel
queue all the information and pipe to file and have some way to import it
forget any information between the time tunnel is set up and data starts sending
Other problems is how will the java program know who is connected.
Solutions
1
SSH server in java…user defined in java…messy
or
SSH
2
Requires ssh server on the host (so perhaps not the best for iPaq, etc)
Have initial `ssh root@stargate01 “touch /etc/server/192.168.3.3″` or something similar to make the stargate aware of “server”
Then, ssh tunnel initialized from stargate to 192.168.3.3 on predefined ports
or edit the /etc/hosts file
3
use “broken” xlisten, just make it so that if it receives data, initialize the above (2).

Commands to switch modes?
Modes:
Query
Periodic
Threshold
Modes on all motes as well as specified motes.
-this calls for implementing a BCAST protocol in the sensor network
-or for sending the same command to every mote…(inefficient use of bandwidth, but partially already available)
Combinations of modes?

TODO
Focus:
Make current GUI solid in terms of no bugs (does daji want this)
look into netcat (is it like xlisten)
SSH tunnel stuff ((java), (stargate (Query)))
xlisten - fix (Query)
automation (java)
(Catching exceptions)
Later:
modify sf so that it is only open to localhost (for security reasons)
Randomization of ports (also for security reasons)
save all data to file(s) on the stargates and have import functionality

Notes:
Why tunnels rock
The lower level mesh routing can easily be changed to a different protocol without having to change any of the higher level tunneling/communication code.
Encryption.

Kyle’s Thoughts (Some already discussed with Shane)

#####################################
# Annoying Issues
#####################################

We have no pcmcia cards that will do Ad-Hoc mode under linux, have to use the mac for now.  Possible solution is to get a pcmcia to cf adapter and use the netgear cards.

The cards say they can do Ad-Hoc mode, but they will not associate.  I have been reading up on it and it appears that there is little demand for ad-hoc mode, and many different manufactures used different protocols for ad-hoc mode (on wireless b cards at least) and this has made interoperability between brands of wireless 802.11b cards painful at times.  Supposedly they all use the “same” method at 2M/s, but I have, as of yet, not been able to get the devices to communicate.

Fortunately, I have a mac laptop that is able to go into Ad-Hoc mode and communicate, the mac is also able to run the java program, yay!

#####################################
#####################################
# Java
#####################################
#####################################

#####################################
# Underlying structure
#####################################

Mote Number - Name associations (or is this more of a gui thing).

XML output/saving of Mote Number/Name associations as well as logged data, etc.

XML Format for data

*Design decisions.  Should we combine the multiple stargates mote “streams” into one output?  Should we denote group IDs and Mote IDs in the display.  Do we keep everything separate?

I think we should denote group IDs and Mote IDs somewhere in the display, and of course in the logs.

There also should probably be something somewhere that denotes which ip is connected to which stargate and which group id.

#####################################
# Gui goodness and settings
#####################################

Somewhere to input ips and ports.
Note: Eventually we want to not have the user put anything in by hand, but for all the stargates to be found without user interaction.  Perhaps an implementation of mobilemesh in java would be of use here (is this possible/practical?).  We could also use mDNS (aka as Rendevous, it is what powers Apples iTunes connectivity), there exists a java version as well as a version that I think I can install from the debian packages (no ipkg one though…that i found).  I do not know how these protocols handle ad-hoc networks…something to look into in more detail.

Related to *Design decisions.  Do we offer such things as “filters” for the display data?  I.e.  drop down that allows you to chose which group ID you want (I would rather not implement anything like searching if we could avoid it…).

How are we to display the mote data is relatively important.  Chart form?  Sorted form?  Spreadsheet like?  All of those showing all sensor data received.

Implementing a “realtime” or slightly delayed graph of the data.  Apparently there is an Oscilloscope application that can do this for us (if we give it the right data).

Conversions to “human readable” numbers.  For example, Temperature to degrees Celsius (Also displaying the raw data and converted data?).

Eventually options to get data from all the different types of sensors.

A list of all types of commands and what they do.

Giving the user some “helpful hints” or meanings to the options if they could be seen as cryptic.  Maybe mouse overs (with time delay of course)?

Updating the About page of the program.
Figuring out a License (The ssh implementations I have seen are under the GPL or MIT licenses).

#####################################
# stargate - pc communication
#####################################

Currently I am opening up a ssh tunnel via the commandline that then launches the java gui (faster to implement).  This could be implemented in java and run at the start of the program.  There is still the issue of getting the IPs of the running stargates with minimal user interaction.

We would like to use all java goodness to send commands to the motes.  This should be possible.  However, we can always fall back on Xcommand (over ssh) which has been modified to work correctly.

#####################################
#####################################
#Stargates
#####################################
#####################################

#####################################
# Connectivity
#####################################

###
Remember to add the ip’s of computers connecting to the stargates in the stargate’s /etc/hosts file.
###

Do we want to run “mmborder” on the stargates?

I have yet to actually test mobilemesh, since I really don’t have enough devices to give it a shot…
I know that mobilemesh is making routes (as I can see them with netstat) but they are all boring single hop of course…
I think that mobilemesh also has to be running on the pc…(because there will be no routes back to the PC on any of the stargates…)
TODO: test mobilemesh? - how with only two nodes?

Have not tried to compile mobilemesh on the mac, I am much more hoping that we can obtain a pcmcia to cf card adapter and run everything on linux.

I made a backup of the stargate before I attempted doing “radical” software changes, and am glad I did. I decided to try out mobilemesh, and since the stargate software is based on the software at [http://familiar.handhelds.org/releases/v0.8.2/feed/base/] I decided to download mobilemesh and all the dependencies that it listed for mobilemesh. Listed in the actual package file, which is a debian package file (do `file filename.ipk` if in doubt). Since I knew how to extract data from a debian package I just did a `ar x filename.ipk` and then I was left with a control.tar.gz, data.tar.gz, and debian-binary files. The interesting ones are the gz’ed ones. The data.tar.gz file holds the actual files that become installed. In this case, all that control.tar.gz contained was control, which has the dependency information that I was seeking. This is what the control file form Mobilemesh looked like.

Package: mobilemesh
Version: 1.2-r1
Description: MobileMesh mobile ad-hoc routing protocol
Section: console/network
Priority: optional
Maintainer: Bruno Randolf
Architecture: arm
OE: mobilemesh-1.2
Depends: libcrypto0.9.7 (>= 0.9.7e), libssl0.9.7 (>= 0.9.7e), libstdc++6 (>= 3.4.3), libc6 (>= 2.3.2+cvs20040726), libgcc1 (>= 3.4.3)
Source: svn://meshcube.org/svn/application;module=mobilemesh;proto=http
Thus I see the dependencies are libcrypto, libssl, etc.

I need to get all those files, so I did.  Then without really thinking all that hard about it, I decided to just install them all by copying them over via ssh.  I copied over some library that was being used (libc6……and caused a fun kernel panic and thus didn’t complete the file transfer.  This led me to have to follow the steps in the previous post to recover.  I think i will now just make a copy of my relevant config files and download the latest stargate release from [http://platformx.sourceforge.net/], since it now has ipkg (like dpkg, but slimmer) installed by default.

Backups are good.

I believe I convered earlier on how to get yourself a copy of the image on the stargate onto your linux hard drive. Now mounting it is a different matter. It is a little more complicated than I thought it should be, as it isn’t possible to mount the jffs2 image with loopback as I would normally do with an iso image or a hd image (mount -t fstype -o loop /path/to/image /mnt/point). This webpage [https://notfaq.wordpress.com/2006/07/20/mounting-a-jffs2-image/] proved rather useful.

The stargate appears to be 31488 Kbytes, and you can replace insmod with modprobe or modprobe -v if you want verbosity.

If you have a tftp server (or don’t mind setting one up) then [http://platformx.sourceforge.net/Documents/build/TFTPHowTo.txt] will be useful. However, if you don’t want to (and have the time to wait for a serial transfer) just type “set loadmethod serial” in the boot loader of the stargate. Then to load the image onto the stargate, just type load “partition name”. Then, if you are using minicom, just do ^A S and navigate to the correct image. It will take a great deal of time to do this via serial.

Update: Problem fixed.  I asked on the AmesFUG [http://amesfug.org/] mailing list and Tom Pohl resonded with the answer to add my ip into the hosts file (/etc/hosts).  The problem appeared to be that the stargate was looking for dns resolutions.
I am having some problems with the Stargate and the speed of the initial ssh login.

A Stargate is a tiny embedded arm machine running a extremely slimmed down version of debian (minus the packagemanager, etc) with a pcmcia slot, usb, ethernet, and cf card slot, and a specially designed 51 pin connecctor.
[http://www.xbow.com/Products/XScale.htm]

More background information

# cat /proc/version
Linux version 2.4.19-rmk7-pxa2-star (eepp@clyde) (gcc version 3.3.2) #1 Fri Aug 13 12:58:00 PDT 2004
STARGATE Version 7.2

I have the stargate set to automatically start in adhoc mode via /etc/pcmcia/wireless.opts
#AmbiCom Wireless Card….
*,*,*,*)
INFO=”Netgear”
ESSID=”sense”
Mode=”Ad-hoc”
Channel=”3″
Rate=”11M”
# KEY=”1234-1234-12 open”
;;

Is the first and only network.

I have commented out the pump -i eth0 lines in /etc/init.d/networking so that the wired devices does not come up on boot and waste time trying to bring up something that isn’t there.

Contents of /etc/network/interfaces
# /etc/network/interfaces — configuration file for ifup(8), ifdown(8)

# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback

# auto eth0 — don’t automatically bring this up
#iface eth0 inet dhcp

openSSH version
SSH Version OpenSSH_2.2.0p1, protocol versions 1.5/2.0.
Compiled with SSL (0×0090581f).

I have public key authentication set up (DSA) and that works.

I did the timing with the following command
time ssh -t root@stargate “echo”
(stargate is defined in my /etc/hosts file)

The actual problem

Whenever I boot up the Stargate I can connect via ssh over the wireless network as soon as I can see it. However, it takes between 20 and 25 seconds to actually connect and get a login prompt. However, when there is a wired ethernet device connected and I run
ifconfig eth0 up;pump -i eth0
I can then connect in under 4 seconds via either interface. Unplugging the network device again results in 20-25 second login times.

The code in cvs for xlisten does not seem to compile to anything close to that which has been distributed on the CD. I am confused. Even the Stock one does not seem to get parsed in the same way. The raw mode from the pre compiled one shows different data than the raw mode of the one compiled from cvs. I whasill have to investigate this more and find out why the cvs one doesn’t appear to work correctly. Doing any modifications for different AM Types seems a little harder now, since the original doesn’t work in the same way. It may be more convenient to use something else if xlisten turns out to be more difficult to modify/correct than originally thought. I will checkout the cvs of tinyos-2.x and see if there is a contrib there that works, I’m doubtful.

Update: The xlisten from beta doesn’t even compile.  There is no xlisten in tinyos-2.x.

This is a general overview, with some specifics.

Surge like code on the motes.
SurgeBase on the mote attached to the Stargate. The Stargate currently has version 7.2
–
Linux version 2.4.19-rmk7-pxa2-star (eepp@clyde) (gcc version 3.3.2) #1 Fri Aug 13 12:58:00 PDT 2004
STARGATE Version 7.2
–
Upgrading to 7.3 would be perhaps advantageous, it would be much simpler to add software, as the 7.3 build from
platformx.sf.net
has support for the ipkg system (the same system that would be used on the compaq’s).

The Stargate will listen with a modified version of xlisten that will put the data into a postgresql database. This database can be dumped, and transferred to a faster machine to do the processing of the data, or it can be viewed on the actual machine (with a little change to a config file) via a tcp port. What is stored in the database can be configured. Currently I am thinking about storing everything, the raw data packet, and the parsed data.

For sending commands to the network, the application Xcmd from xbow will be used. This was modified slightly in order to send all the packet types that we are using.

We will have to create a program to access the database (or some data file) and find the currentpath to the specific mote in order to send commands to that specific mote. This program will be essentially the glue that holds everything together. It will probably be a daemon of sorts, accessible on a port, and having a simple set of commands, which will be easily extendable.
It could be a web application as putting apache on the machine would not be that difficult. The advantage of the web application would be that no applications would have to be installed on the machine trying to access the data from the sensor network. Of course, this would make the stargate the machine doing all the calculations, which are mostly minimal, but with the Stargate running on batteries this could be an issue. This web frontend would probably be easiest for communication purposes, choosing from the options, then the machine would send the command via xcmd. Visualization and analysis of the data would probably be far more convenient on a different box of course. Output to csv may be more useful than a database file for many applications. This could be done through a web interface as well of course. Advantage of a web based application is that any client can access it provided it has a 802.11b card and a web browser. Disadvantage is of course the limitations of a web app.

Minimum requirements of the Application

Be able to display the current data in some manner. Likely graphs of some sort, or maybe just a giant table. With options on how many rows to show or something.
Be able to send a command to a specific Mote, likely selecting from drop down lists of all the currently connected elements.

Problems at this time:

USB

Usb mounting on boot seems to be an issue with the 2.4 kernel, the usb-storage driver is compiled as a driver, and thus will have to be loaded (via insmod or modprobe) before and usb-storage devices can be used…The problem comes as /dev/sda1 (the usb device) isn’t recognized fast enough.
This is written to a file when I have it try to mount in a bootup script.
mount: special device /dev/sda1 does not exist

There are also problems when there are usb devices plugged into the Stargate with `shutdown -r now` not working as it should. The machine has to be rebooted by hand, the command doesn’t terminate the usb devices properly unfortunately.

Possible solutions,: get a different kernel, add cron (to make it check to see if the device is mounted, and if it is not try to mount it), or get a cf card, which should mount on boot from what I have read.

Current Solution:
added
# Needed because usb isn’t found fast enough to mount
sleep 15
mount -t ext2 /dev/sda1 /mnt/cf1

to both /etc/init.d/cfcardapps and /sbin/setup.sh
Adding only one call to mount didn’t work for some odd reason, and having them both in one file didn’t work either, quite annoying.
This will of course sleep for 30 extra seconds on boot no matter what, not the most ideal of solutions.

/etc/init.d/rcS calls /sbin/setup.sh which calls `/etc/init.d/cfcardapps start` to start the postgresql apps (and whatever else we want to start on the card).

Lack of “glue” application

It has yet to be written, or even what type, c, web, java, socket, etc.

I have been working with xlisten (aka xserve) for the Stargate to interact with the motes.

The reason I have been looking at this is to make it so that we can have custom packets sent nicely to the postgresql database with minimal hassle. The best way for one to figure out what they need to do is of course stsart reading the xlisten.c source file (in the contrib/xbow/tools/src/xlisten folder of your tinyos-1.x install (if you came from CVS)) at the main part of the file. This is after you’ve mad sure that you can compile the code. I was able to compile it only for the arm (make xlisten-arm), but I did not try very hard to get a non arm version to compile. The process would involve getting the source for postgresql (same version for “best” results), but that doesn’t sound too bad…

This will lead you to amtypes and boards eventually, which will lead you to believe that you just have to create a copy of say surge.c and rename it and start modifying it to your specifications. You need to make sure that the AMTYPE_WHAT_YOU_WANT = 0x##; for your new packet type in xsensors.h in the typedef enum { … } XbowGeneralPacketType; (or similar)
Then you need to make sure that you have the whatever_initialize() called, as well as including the necessary source file(s).

If you look at amtypes/surge.c you will see a line like XPacketHandler surge_packet_handler; That makes it quite easy to add multiple other packet handlers.

This is just enough to get others started in the right direction.

I’ll add more later, and perhaps even some detailed steps on how to add a new Packet Handler (or AM Type) to the Stargate xlisten-arm/xserve-arm application.

This is a huge post, because I haven’t been keeping up with my activities. It will not be as detailed as previous posts, simply because it goes over a larger range of time.

Stargate

I have the stargate working as an ad-hoc network with a static ip. The relevant information to set up such a thing is available in two parts from two places.

Read the “tutorial” in the Doc folder provided by xbow and also read this, especially §3 about wireless configuration. As always, make a backup of any of the files you change, before you start changing them.

I have a usb drive plugged into the usb port on the stargate daughterboard, I have not yet set it to automount on boot, but I have put some software onto the device. Notably the postgresql and “xlisten” (named xserve-arm when the tgz is actually unarchived).

The xserve-arm program has many modes, run it with -? to see them all. Notable favorites include -r (for raw mode) and the default (no command line options) will allow you to see the original packet, the parsed data, and any processing from the “packet handler”. I have not sat down and figured out how the packet handler actually works…In other words, how to make my own packet handler to handle the specific data packets.

The motes that can be attached to the stargate that I have tried through the 51 pin connector are mica2 and micaz. The TOSBase app from the default tinyos-1.x/apps folder is the one that is attached to the stargate, and will forward all packets that it recieves to the stargate (assuming of course that it is of the
same group id).

The postgresql database does work (of course it has no useful data being placed in it) but I was able to capture a large number of packets and dump the database with no significant problems. To start the database, you’ll have to make sure that the stargate has the cf card (or usb drive) mounted so that it can have a place to store the database, and be able to execute the application. This can be done as said in the stargate doc. Or by executing `/etc/init.d/card* start` . I don’t have the stargate on right now to check that. That card* (don’t remember the full name) depends on the cf card or usb drive to be mounted in /mnt/cf1, that is easy enough to accomplish with the mount command (mount /dev/(insert device name, cf1 or sda1) /mnt/cf1 -t (filesystem type, you’ll probably want to have ext2 here for flash based drives that are formatted as ext2 (ext2 is better for flash drives because it does not have the journaling that ext3 has, and thus does not do as many writes as ext3 would. This is important because of the limited number of rewrites on flash media.)). The doc on the stargate CD explains how to do that (gives you the command to do it). It should be relatively simple to transfer the database data to another box (In fact there is an automated ftp transport daemon of sorts installable from the doc cd).
I have not worked on stargate to stargate communications, but if they are all set up as static it should be relatively simple to search through a range of ip addresses. Not efficient of course.

Backing up the Stargate

If you have a linux machine with a serial port, backing up the stargate is relativelty simple (if slightly time consuming). The data transfer rate will be quite slow (10K/s). Software that needs to be installed on a debian/ubuntu box are as follows: minicom, lrzsz. minicom allows communication, and should be set up as in the doc on the stargate board, or (http://www.pages.drexel.edu/~ttl28/tutorials/StargateTutorial.html). lrzsz provides xmodem, zmodem, and ymodem - we’ll be using xmodem.

To get an image of the current state of the machine, connect the serial cable to the stargate box, open up minicom. Make sure it is properly configured as the tutorial listed above. Also, you may want to set a download directory. Look in ctrl A-Z menu to find out how, hint “O”. Then turn on the stargate, it will come to a screen saying you have ten seconds to press a key or it’ll boot the default image. Press any key but the enter/return key. Then type ? or help for a list of all available commands. The ones that are most pertinent to backing up the machine are “partition”, “save partion”, and “save all”. Once you have the executed the save command that you want, type ctrl A-R and you’ll be given some choices. You’ll want xmodem, and you’ll want to give it a name to save as. Then you’ll wait, it’ll take some time, as it is a slow connection over serial. Loading the image back should be relatively straightforward from the help command in the bootloader. You may want to be careful on reprogramming, as some software updates require a JTAG board instead of being able to upload via serial. Look at http://platformx.sourceforge.net/ for more downloads and more details on how to do reprogramming.
Motes

Shane and I have played with different versions of Surge and Shane found a DSDV implementation with multiple variations for low power, high transmission rate, etc. We are finding that the DSDV implementation has noticeably shorter transmission ranges compared to the Surge implementations. We believe this may be related to the packet size, since the DSDV packets are larger than the Surge packets. The wireless link may only be able to transfer data reliably under a certain threshold. Of course, it could be related to power levels and power management, but we believe that we maxed both of those out for simple comparison purposes.

Unfortunately, the built in packet handlers in the version that I have on the stargateare unable to see the data correctly from some of the packets.  The route packets in surge are seen mostly correct.  I have not searched for newer versions of the software, nor successfully compiled the “xlisten” application in the contrib/xbow/tools/ directories.