Top 5 Reasons for Failures in ZigBee Deployments

When utilities are considering deploying ZigBee meters and HAN devices, the question often arises: “If I’m working with ZigBee certified devices, doesn’t that guarantee that they will work out of the box? Why do I need any more testing?” Testing mitigates the risks involved with problems beyond the scope of the specifications.

NTS is the first and leading test lab certified to test for the ZigBee Alliance. NTS has amassed an extensive knowledge base gained by its experience performing testing for hundreds of clients around the world. Ranging from RF distance and signal propagation to interoperability issues of incompatible revision levels, problems can surface for various reasons. Some are clearly outside the scope of ZigBee protocol testing, while others are due to different interpretations of the specification. Regardless of the reasons, these problems can end up causing serious headaches with field deployments, resulting in cost overruns and dissatisfied customers. NTS has compiled the five most common, major areas where testing was able to mitigate risk and save money, while preventing improper installations.

REASON 1: RF Signal Propagation
The single most frequent point of failure for low power transmissions is the lack of signal strength. Although the device’s radio is tested when first designed, often the final product can have significantly different RF characteristics due to placement of the chips on the circuit board with proximity to the power supply, material of the case, and antenna placement. The addition of a Power Amplifier (PA) or Low Noise Amplifier (LNA) - after testing - to improve the radio performance could also end up completely changing the device’s characteristics. Similarly, the choice of material for the device’s enclosure can also have an unforeseen detrimental effect. For example, adding a magnet to attach an in-home display (to attach the device to the customer’s refrigerator) can seriously affect the radio performance. In addition, different building materials negatively impact the propagation of the radio signal. In our evaluations, we’ve seen approximately 23% of devices that have had their RF signal propagation negatively impacted due to these issues.

REASON #2: End to End Registration
The cornerstone of any successful HAN deployment is the initial registration of the device onto the network. Although registration is designed to be transparent to the application and simple from a user perspective, it is still one of the areas most prone to errors. In our experience, end-to-end testing has reduced deployment issues by around 10% simply by solving registration issues before they occur. The assumption that multiple implementations to the same specification built by different manufacturers with different chip sets and revision levels will all come together and work well is a major cause for deployment failures. Each variable provides another exponential factor for failures – from the web portal’s security window where users enter the new device information, onto the utility backhaul that receives that data, to the meter which then opens the joining process and allow this new device access. With current deployments still in their infancy, we have seen ZigBee devices working perfectly only to be let down by a backhaul system that did not correctly handle their request to join. Other times, the backhaul did its job of opening up the registration, but the device itself forgot to bind to the smart meter on to the needed functionality. Regardless of the reasons, these problems can end up causing serious headaches, cost overruns, and dissatisfied customers.

REASON #3: Functionality
The main scope of ZigBee certification is to verify products conform to the specifications of the communication protocol, while actual functionality of these same devices is not looked at during product certification. Testing is necessary to go beyond just the communications validation performed by certification. What good is a thermostat that correctly answers the right request from a smart meter, but never adjusts its temperature settings correctly? Or a load control device that indicates its participation in an event without ever shedding load? Does a display simply acknowledge receiving a new price, or does it present the user with the right alarms (for example, flashing red) when peak pricing occurs? Can a user trust their in-home display consumption graphs? In over 90% of the devices we’ve tested, we’ve uncovered some functional anomaly. Although many are cosmetic or trivial in nature, we have also identified anomalies that have saved manufacturers literally millions of dollars in lost revenue due to utilities not disqualifying their products for release based on performance issues.

REASON #4: Interoperability
Outside the lab environment, real world conditions are varied and dynamic in nature. Different smart meters and devices support different optional features and attributes. Can the device handle these different configurations? For example, we have seen devices fail to indicate the correct time and therefore inaccurately display information because the associated meter did not support localtime as an optional attribute. We have seen devices not understand a price packet because it had an extra optional field they did not support. We have also seen devices crash when their binding attempts on the meter failed again because the meter did not support it. We have even seen devices fail to look for a network on specific channels which would render them completely useless in cases where the meter decided to form its own network (or move it) onto one of those channels. Another typically thorny issue is for devices such as IHDs to be able to differentiate consumption data coming from the meter (relevant to the household) from the data coming from, say, the washing machine. Interoperability is a key component in a complete HAN deployment, and almost 27% of the devices we’ve tested have had interoperability issues of one form or another.

REASON #5: Power/Loss of Network Recovery
Another tricky aspect of ZigBee is the network’s recovery of devices after the loss of communication or after a power outage (or simply a battery change). Although ZigBee provides the tools and best practices for these types of recoveries, we have found in roughly 33% of our tests that device manufacturers miss certain use cases or fail to fully evaluate the ramification of different scenarios. For example, some devices have saved their network information (e.g., PAN ID, channel, etc.) in their non-volatile memory but have neglected to retain the security keys. This rendered the device useless and required registration from scratch. Some other devices, after coming back on the network, stay on the old polling schedule for information. This may mean a device stays out of sync on the price, for example, for a whole day (if a price has been updated while the device was having its battery changed). A significant number of devices have some issues in this area and only when tested on a network can these nuances be discovered.

More on these issues and other specification interoperability scenarios can be obtained by contacting NTS Advanced Technology. This knowledge bank of historical information, testing, design and engineering details are available by contacting NTS direct at 310-641-7700 or emailing